postscreen - passive OS fingerprinting

[Tomoyuki Murakami]

Developers,

There are some sort of Passive OS Fingerprinting implements,
pof v3 (http://lcamtuf.coredump.cx/p0f3/) is one of that, and
do light-weight, very quick response in my environment.
I don't know the detail history of this app, there is
a simple API for utilize information of the other-end OSes.

Many experiences that possible bot machines running 'Doze OS
and if we can identify that, we could reduce cost of checking
DNSBLs, etc.

I know that Microsoft themselves running such OS in their
service environment, we should be careful about the result
of p0f's and we might fine tune fingerprints for a while.
And, also know that there are some implements as milter or
policy daemon utilizing p0f. I think that p0f's quickness
and stand points of connection checking, it has good reason
to cope with postscreen.

Attachement is a patch against postscreen implementing I/F to
p0f daemon, and do DROP/ENFORCE/IGNORE as well as other
checking methods. (diff to postfix-2.10-20120902)

If you would like test, you must get/compile/install p0f
first as instructed in
http://lcamtuf.coredump.cx/p0f3/README

I had setup on Linux machine, local unix socket, it's work
very fast like memcached.

Regards,
---
Tomo.

p.s.
If you install p0f, don't forget sending chocolate to the author ;-p

[Wietse Venema]

Post by Tomoyuki Murakami
Developers,
There are some sort of Passive OS Fingerprinting implements,
pof v3 (http://lcamtuf.coredump.cx/p0f3/) is one of that, and
do light-weight, very quick response in my environment.
I don't know the detail history of this app, there is
a simple API for utilize information of the other-end OSes.

First, I am pleased to read code that is well written. Thank you.

Post by Tomoyuki Murakami
Many experiences that possible bot machines running 'Doze OS
and if we can identify that, we could reduce cost of checking
DNSBLs, etc.
I know that Microsoft themselves running such OS in their
service environment, we should be careful about the result
of p0f's and we might fine tune fingerprints for a while.

Fundamental critique:

- There are many legitimate MTAs running on Microsoft systems,
including MTAs from vendors other than Microsoft. It would be a
mistake to veto connections because a site runs a Microsoft OS.
Postscreen is the wrong place to use methods that suffer from a
non-negligible false positive rate.

- Instead of fingerprinting the client TCP/IP stack, it would be
more appropriate to fingerprint the client SMTP stack. This is on
the TODO list with p0f which relies on passive observation. Sofar,
my own experiences with SMTP stack fingerprinting have been unrewarding
except for PREGREET detection (and that can't be done with passive
observation). There is a USENIX Security 2012 paper with examples
of active fingerprinting that send malformed SMTP responses to
distinguish between legitimate and spambot SMTP stacks (Gianluca
Stringhini et al. ***@bel: Leveraging Email Delivery for Spam Mitigation).

Implementation critique:

- Postscreen blocks until the p0f server replies. That violates
postscreen design which avoids blocking. When the p0f daemon becomes
unresponsive for whatever reason, that should not cripple postscreen.
There is no such problem with postscreen DNSBL lookups.

- The blocking queries are also a problem because you query the p0f
daemon on every connection, unconditionally, even for clients that
are permanently black- or whitelisted, or that exist in the temporary
whitelist cache.

Wietse

[Tomoyuki Murakami]

Post by Wietse Venema

Post by Tomoyuki Murakami
Developers,
There are some sort of Passive OS Fingerprinting implements,
pof v3 (http://lcamtuf.coredump.cx/p0f3/) is one of that, and
do light-weight, very quick response in my environment.
I don't know the detail history of this app, there is
a simple API for utilize information of the other-end OSes.

First, I am pleased to read code that is well written. Thank you.

you're welcome. thank you for reply.

Post by Wietse Venema
- There are many legitimate MTAs running on Microsoft systems,
including MTAs from vendors other than Microsoft. It would be a
mistake to veto connections because a site runs a Microsoft OS.
Postscreen is the wrong place to use methods that suffer from a
non-negligible false positive rate.

I generally agree with, but it's efficiency is$B!!(Bworth
consideration in my litte bit wild, spam-hating environment.
It might not be an everyone's solution.

Post by Wietse Venema
- Instead of fingerprinting the client TCP/IP stack, it would be
more appropriate to fingerprint the client SMTP stack. This is on
the TODO list with p0f which relies on passive observation. Sofar,
my own experiences with SMTP stack fingerprinting have been unrewarding
except for PREGREET detection (and that can't be done with passive
observation). There is a USENIX Security 2012 paper with examples
of active fingerprinting that send malformed SMTP responses to
distinguish between legitimate and spambot SMTP stacks (Gianluca

thank you for reference pointer. I will watch later.

Thank you,
---
Tomo.

[Wietse Venema]

Post by Tomoyuki Murakami

Post by Wietse Venema

Post by Tomoyuki Murakami
Developers,
There are some sort of Passive OS Fingerprinting implements,
pof v3 (http://lcamtuf.coredump.cx/p0f3/) is one of that, and
do light-weight, very quick response in my environment.
I don't know the detail history of this app, there is
a simple API for utilize information of the other-end OSes.

First, I am pleased to read code that is well written. Thank you.

you're welcome. thank you for reply.

Post by Wietse Venema
- There are many legitimate MTAs running on Microsoft systems,
including MTAs from vendors other than Microsoft. It would be a
mistake to veto connections because a site runs a Microsoft OS.
Postscreen is the wrong place to use methods that suffer from a
non-negligible false positive rate.

I generally agree with, but it's efficiency is?worth
consideration in my litte bit wild, spam-hating environment.
It might not be an everyone's solution.

I'm thinking about extending postscreen with an event-driven plug-in
interface to send out a query in parallel with the Pregreet and
DNSBL tests (this would make the response time less critical).

This interface would use a simplified version of the policy delegation
protocol.

Example postscreen query:
query_id=uniquestring
address_family=ipv4 (or ipv6)
protocol=tcp
local_address=1.2.3.4
local_port=25
remote_address=5.4.3.2
remote_port=6666

Example plug-in reply:
query_id=uniquestring
action=pass (or drop, enforce, ignore)
ttl=600 (time in seconds if "pass", otherwise ignored)

As with Pregreet and DNSBL, postscreen would not block waiting for
a reply. If the plug-in is down or too slow, postscreen just uses
a default action (I recommend "pass") and TTL (I recommend a few
minutes) to avoid hammering on a dead plug-in.

This would waste a few machine cycles, but it would provide a way
to make postscreen work with p0f or other things that we haven't
thought of yet.

Wietse

[Tomoyuki Murakami]

Post by Wietse Venema
I'm thinking about extending postscreen with an event-driven plug-in
interface to send out a query in parallel with the Pregreet and
DNSBL tests (this would make the response time less critical).
This interface would use a simplified version of the policy delegation
protocol.

Oh! It's a great news!. I'm already have policy daemon which can
deal with p0f. It may work with a little fix for that.

Many thanks!
--
Tomo.

[Wietse Venema]

Post by Tomoyuki Murakami

Post by Wietse Venema
I'm thinking about extending postscreen with an event-driven plug-in
interface to send out a query in parallel with the Pregreet and
DNSBL tests (this would make the response time less critical).
This interface would use a simplified version of the policy delegation
protocol.

Oh! It's a great news!. I'm already have policy daemon which can
deal with p0f. It may work with a little fix for that.

I expect to have time late this year when I can spend a couple days
on a solid implementation. Free software does not pay my bills.

Wietse

[Tomoyuki Murakami]

Post by Wietse Venema

Post by Tomoyuki Murakami

Post by Wietse Venema
I'm thinking about extending postscreen with an event-driven plug-in
interface to send out a query in parallel with the Pregreet and
DNSBL tests (this would make the response time less critical).
This interface would use a simplified version of the policy delegation
protocol.

Oh! It's a great news!. I'm already have policy daemon which can
deal with p0f. It may work with a little fix for that.

I expect to have time late this year when I can spend a couple days
on a solid implementation. Free software does not pay my bills.

yeah, yeah. I can wait and have time to spend observing traffic
tendency.

Best regards,
---
Tomo.