Need additional milter values

[Philip Prindeville]

Hi.

We need to implement various controls via mimedefang in our scenario, and need to use ${daemon_port} for some of those controls.

I'd like to add various milter variables like ${daemon_port}, ${if_addr}, ${if_name}, etc. but it's been a while since I stared at Postfix sources (8.3 was still out at the time).

Can someone walk me through the steps at a high-level, and I'll do the implementation and testing myself?

Thanks,

-Philip

[Wietse Venema]

Post by Philip Prindeville
Hi.
We need to implement various controls via mimedefang in our scenario,
and need to use ${daemon_port} for some of those controls.
I'd like to add various milter variables like ${daemon_port},
${if_addr}, ${if_name}, etc. but it's been a while since I stared
at Postfix sources (8.3 was still out at the time).
Can someone walk me through the steps at a high-level, and I'll
do the implementation and testing myself?

With each new attribute:

- An XCLIENT feature is needed if the attribute will be used for
access decisions. Otherwise, XCLIENT can no longer be used for
testing.

- A queue file attribute record is needed if the attribute will be
used for access decisions. Otherwise, unexpected things will happen
when someone does "postsuper -r" and the Milter is called by the
cleanup daemon. This requires changes to the cleanup daemon to
store the attribute, to rec_attr_map() to "bless" the queue file
attribute name, and to rec_type.h to define the pseudo record type
that rec_attr_map() needs.

- An XFORWARD feature is needed if the attribute will be used for
logging. Otherwise, logging will be inconsistent. This requires
changes in the Postfix SMTP server to receive the attribute, the
smtpd_proxy_filter client to forward the attribute, the queue manager
code to read the attribute from queue file, the queue manager to
delivery agent protocol to forward the attribute, and the Postfix
SMTP client to forward the attribute.

That's just off the top of my head.

Wietse

[Philip Prindeville]

[snip]
- An XCLIENT feature is needed if the attribute will be used for
access decisions. Otherwise, XCLIENT can no longer be used for
testing.
- A queue file attribute record is needed if the attribute will be
used for access decisions. Otherwise, unexpected things will happen
when someone does "postsuper -r" and the Milter is called by the
cleanup daemon. This requires changes to the cleanup daemon to
store the attribute, to rec_attr_map() to "bless" the queue file
attribute name, and to rec_type.h to define the pseudo record type
that rec_attr_map() needs.
- An XFORWARD feature is needed if the attribute will be used for
logging. Otherwise, logging will be inconsistent. This requires
changes in the Postfix SMTP server to receive the attribute, the
smtpd_proxy_filter client to forward the attribute, the queue manager
code to read the attribute from queue file, the queue manager to
delivery agent protocol to forward the attribute, and the Postfix
SMTP client to forward the attribute.
That's just off the top of my head.
Wietse

Thanks... Can you point me at any particular commits I could look at that added such a milter attribute?

-Philip

[Wietse Venema]

Post by Philip Prindeville

[snip]
- An XCLIENT feature is needed if the attribute will be used for
access decisions. Otherwise, XCLIENT can no longer be used for
testing.
- A queue file attribute record is needed if the attribute will be
used for access decisions. Otherwise, unexpected things will happen
when someone does "postsuper -r" and the Milter is called by the
cleanup daemon. This requires changes to the cleanup daemon to

For example, without the client name/address must be in the queue
file, mail requeued with "postsuper -r" would be processed as if
it is a local submission, and that would cause some Milters to make
the wrong security decisions such as signing other people's mail.

Post by Philip Prindeville

store the attribute, to rec_attr_map() to "bless" the queue file
attribute name, and to rec_type.h to define the pseudo record type
that rec_attr_map() needs.
- An XFORWARD feature is needed if the attribute will be used for
logging. Otherwise, logging will be inconsistent. This requires
changes in the Postfix SMTP server to receive the attribute, the
smtpd_proxy_filter client to forward the attribute, the queue manager
code to read the attribute from queue file, the queue manager to
delivery agent protocol to forward the attribute, and the Postfix
SMTP client to forward the attribute.
That's just off the top of my head.

Thanks... Can you point me at any particular commits I could look
at that added such a milter attribute?

Here is a good example of what it takes to add, document and test
a "daemon port":

$ find proto src -type f print | xargs grep -i client_port

This produces 151 lines of output with postfix-2.9-20111125 (of
which 6 lines are in files with test data).

Adding the server port would largely involve duplicating those
client_port lines, and maybe borrow some client port code that the
grep command did not reveal.

Wietse

[Philip Prindeville]

Post by Wietse Venema

Post by Philip Prindeville

[snip]
- An XCLIENT feature is needed if the attribute will be used for
access decisions. Otherwise, XCLIENT can no longer be used for
testing.
- A queue file attribute record is needed if the attribute will be
used for access decisions. Otherwise, unexpected things will happen
when someone does "postsuper -r" and the Milter is called by the
cleanup daemon. This requires changes to the cleanup daemon to

For example, without the client name/address must be in the queue
file, mail requeued with "postsuper -r" would be processed as if
it is a local submission, and that would cause some Milters to make
the wrong security decisions such as signing other people's mail.

Post by Philip Prindeville

store the attribute, to rec_attr_map() to "bless" the queue file
attribute name, and to rec_type.h to define the pseudo record type
that rec_attr_map() needs.
- An XFORWARD feature is needed if the attribute will be used for
logging. Otherwise, logging will be inconsistent. This requires
changes in the Postfix SMTP server to receive the attribute, the
smtpd_proxy_filter client to forward the attribute, the queue manager
code to read the attribute from queue file, the queue manager to
delivery agent protocol to forward the attribute, and the Postfix
SMTP client to forward the attribute.
That's just off the top of my head.

Thanks... Can you point me at any particular commits I could look
at that added such a milter attribute?

Here is a good example of what it takes to add, document and test
$ find proto src -type f print | xargs grep -i client_port
This produces 151 lines of output with postfix-2.9-20111125 (of
which 6 lines are in files with test data).
Adding the server port would largely involve duplicating those
client_port lines, and maybe borrow some client port code that the
grep command did not reveal.
Wietse

Well, if I'm going to do all that, I might as well add:

daemon_port
if_addr
if_name

Assuming that there might be further milter values added to the milter spec in the future, or that some of the existing milter values might need to be added to Postfix in the foreseeable future, would it make sense to add infrastructure code to simplify adding such variables in the future?

I can use getifaddrs() to populate both of these latter two.

-Philip

[Wietse Venema]

Post by Philip Prindeville

Post by Wietse Venema

Post by Philip Prindeville
Thanks... Can you point me at any particular commits I could look
at that added such a milter attribute?

Here is a good example of what it takes to add, document and test
$ find proto src -type f print | xargs grep -i client_port
This produces 151 lines of output with postfix-2.9-20111125 (of
which 6 lines are in files with test data).
Adding the server port would largely involve duplicating those
client_port lines, and maybe borrow some client port code that the
grep command did not reveal.
Wietse

daemon_port
if_addr
if_name
Assuming that there might be further milter values added to the
milter spec in the future, or that some of the existing milter
values might need to be added to Postfix in the foreseeable future,
would it make sense to add infrastructure code to simplify adding
such variables in the future?

Considering that these changes are scattered over some 40+ files,
and that there are already a half-dozen similar attributes, I would
expect that I have already exploited many opportunities for reuse,
but I am willing to be surprised.

Post by Philip Prindeville
I can use getifaddrs() to populate both of these latter two.

For {if_addr} you may have more luck with getsockname() which has
the same result on all supported platforms.

According to util/inet_addr_local.c, not all systems support
getifaddrs() in the same way.

And unless you turn off IP forwarding in the TCP/IP stack, the value
of {if_addr} and {if_name} says nothing about the path that packets
have taken. It just says something about the destination IP address.

Wietse

[Philip Prindeville]

Post by Wietse Venema

Post by Philip Prindeville

Post by Wietse Venema

Post by Philip Prindeville
Thanks... Can you point me at any particular commits I could look
at that added such a milter attribute?

Here is a good example of what it takes to add, document and test
$ find proto src -type f print | xargs grep -i client_port
This produces 151 lines of output with postfix-2.9-20111125 (of
which 6 lines are in files with test data).
Adding the server port would largely involve duplicating those
client_port lines, and maybe borrow some client port code that the
grep command did not reveal.
Wietse

daemon_port
if_addr
if_name
Assuming that there might be further milter values added to the
milter spec in the future, or that some of the existing milter
values might need to be added to Postfix in the foreseeable future,
would it make sense to add infrastructure code to simplify adding
such variables in the future?

Considering that these changes are scattered over some 40+ files,
and that there are already a half-dozen similar attributes, I would
expect that I have already exploited many opportunities for reuse,
but I am willing to be surprised.

Post by Philip Prindeville
I can use getifaddrs() to populate both of these latter two.

For {if_addr} you may have more luck with getsockname() which has
the same result on all supported platforms.

Right. Don't know why I wrote "these latter two"... meant just the last one. The fix is to do a getsockname() and then search the list for getifaddrs() for the corresponding match for {if_name}.

Post by Wietse Venema
According to util/inet_addr_local.c, not all systems support
getifaddrs() in the same way.
And unless you turn off IP forwarding in the TCP/IP stack, the value
of {if_addr} and {if_name} says nothing about the path that packets
have taken. It just says something about the destination IP address.
Wietse

Yes. The point is that one might want to apply Milter policies based on the names of the interfaces, not their actual addresses, such as:

wan0
lan0
tun0
vpn5

etc. So I'm not interested so much what interface the packets came in on, as what address (by name associated with the interface) they were destined for.

In a richly connected network with a multihomed host, packets might arrive on more than one interface during the course of a connection anyway, possibly even simultaneously.

-Philip

[Wietse Venema]

Philip Prindeville:
[ Charset ISO-8859-1 unsupported, converting... ]

Post by Philip Prindeville

Post by Wietse Venema
And unless you turn off IP forwarding in the TCP/IP stack, the value
of {if_addr} and {if_name} says nothing about the path that packets
have taken. It just says something about the destination IP address.

In a richly connected network with a multihomed host, packets might
arrive on more than one interface during the course of a connection
anyway, possibly even simultaneously.

Indeed. That's why {if_name} is totally meaningless in such
configurations, and {if_addr} mostly meaningless.

So, you can save yourself time and skip these features unless you
intend to run Postfix on a multi-homed firewall, which case I would
recommend running Postfix on single-homed hosts on both sides of a
"bare" firewall (the configuration of classical DEC SEAL firewall).

Wietse

[Philip Prindeville]

Post by Wietse Venema
[ Charset ISO-8859-1 unsupported, converting... ]

Post by Philip Prindeville

Post by Wietse Venema
And unless you turn off IP forwarding in the TCP/IP stack, the value
of {if_addr} and {if_name} says nothing about the path that packets
have taken. It just says something about the destination IP address.

In a richly connected network with a multihomed host, packets might
arrive on more than one interface during the course of a connection
anyway, possibly even simultaneously.

Indeed. That's why {if_name} is totally meaningless in such
configurations, and {if_addr} mostly meaningless.
So, you can save yourself time and skip these features unless you
intend to run Postfix on a multi-homed firewall, which case I would
recommend running Postfix on single-homed hosts on both sides of a
"bare" firewall (the configuration of classical DEC SEAL firewall).
Wietse

I really don't think it's that simple.

I know of one scenario for instance where the public interface speaks to outside mailers (and indeed is an MXer on that interface), but a loX interface (like 127.0.0.2) talks only to a another relay agent which is allowed to submit outbound messages for relaying.

Give people the capacity to do flexible and powerful things, and you're rarely disappointed.

-Philip

[Wietse Venema]

Philip Prindeville:
[ Charset ISO-8859-1 unsupported, converting... ]

Post by Philip Prindeville

Post by Wietse Venema
[ Charset ISO-8859-1 unsupported, converting... ]

Post by Philip Prindeville

Post by Wietse Venema
And unless you turn off IP forwarding in the TCP/IP stack, the value
of {if_addr} and {if_name} says nothing about the path that packets
have taken. It just says something about the destination IP address.

In a richly connected network with a multihomed host, packets might
arrive on more than one interface during the course of a connection
anyway, possibly even simultaneously.

Indeed. That's why {if_name} is totally meaningless in such
configurations, and {if_addr} mostly meaningless.
So, you can save yourself time and skip these features unless you
intend to run Postfix on a multi-homed firewall, which case I would
recommend running Postfix on single-homed hosts on both sides of a
"bare" firewall (the configuration of classical DEC SEAL firewall).
Wietse

I really don't think it's that simple.
I know of one scenario for instance where the public interface
speaks to outside mailers (and indeed is an MXer on that interface),
but a loX interface (like 127.0.0.2) talks only to a another relay
agent which is allowed to submit outbound messages for relaying.

If the client address is 127.0.0.1, you already know the interface
name and address, and if_addr, and there is no need add hundreds
of lines of code to Postfix to pass that information around.

If the client IP address is not 127.0.0.1, then the interface
information is useless in most practical configurations as discussed
above.

Wietse

[Murray S. Kucherawy]

-----Original Message-----
Sent: Saturday, November 26, 2011 12:14 PM
Subject: Re: Need additional milter values
Assuming that there might be further milter values added to the milter
spec in the future, or that some of the existing milter values might
need to be added to Postfix in the foreseeable future, would it make
sense to add infrastructure code to simplify adding such variables in
the future?

The milter specification doesn't list which values ("macros", in sendmail parlance) are passed down. It's entirely up to the MTA and the filter to negotiate that.

Recent versions of the milter protocol include a mechanism by which the filter can ask for specific ones, if the MTA supports that request. The MTA just needs to know what to return in each case.