Discussion:
Draft design: Building email archival support into Postfix
(too old to reply)
Wietse Venema
2013-03-29 00:10:33 UTC
Permalink
This week I was asked to help out colleagues who were trying to
make Postfix work with some email archive system. I drafted up a
few solutions, one of which was adopted.

During this exercise I realized that this would be less painful if
Postfix had built-in support to create "archive-quality" copies of
email messages. Below is a straw-man design.

Wietse

Draft design: Building email archival support into Postfix.

Over-all design
===============

- Create copies of email envelope addresses and content for archive
purposes, and send them as email messages to a configurable archive
recipient address. The archive recipient address may be specified
in main.cf (archive_recipient = ***@recipient-domain), or it
may be specified with an access map or header/body_checks action
(ARCHIVE ***@recipient-domain).

- When a queue file is created for a new email message, the file
may contain a record with an archive recipient address.

- When the queue manager opens a queue file that contains an archive
recipient address record, it first contacts the archive daemon
before attempting to deliver the message.

- When the archive daemon creates an archive message, it preserves
in the From: and To: headers the envelope sender and recipient
addresses of the original message, together with a copy of the Date:
and Subject: headers of the original message, making the email
archive easy to examine with standard email tools. The original
message contents (header and body) are preserved as a message/rfc822
attachment.

- The archive message queue file contains a flag saying that Postfix
must never bounce the message.

- After the archive daemon has queued an archive message, the archive
daemon clears the archive recipient record in the queue file and
reports successful completion to the queue manager. Then, the queue
manager processes the queue file for normal delivery.

New Postfix code and data
=========================

- A new queue file record (archive recipient) with the archive
recipient email address. This can be set with a new main.cf
archive_recipient parameter, and can be overruled with an ARCHIVE
action in an access map or header/body_checks action. Only the
"last" archive recipient address in a queue file will be used. This
similar to how Postfix handles a queue file with multiple FILTER
records.

- A new queue file record (soft bounce) saying that Postfix must
never bounce the message containing that record. This is a safety
mechanism for when delivery to the archive server fails.

- A new archive daemon that receives a queue manager request to
submit an archive message into the mail queue. This is similar to
how the queue manager asks the bounce daemon to submit a delivery
status notification message. After successful submission the archive
daemon clears all archive recipient records in the original queue
file and reports successful completion.

- A new archive client library module that talks to the archive
daemon. This will use event-driven communication, similar to the
abounce(3) and adefer(3) client library modules, and will use the
same communication infrastructure to handle timeouts and I/O errors.

- A new queue manager state. When a queue file contains an archive
recipient record, the queue manager first asks the archive daemon
to create an archive message. Once the archive daemon reports
successful completion, the queue manager delivers the queue file
as usual.

- A new receive_override_options option (no_archive) that suppresses
the generation of archive copies before or behind a post-queue
content filter.

- Gory details: need to decide whether an archive message should
be subject to any form of header/body_checks, Milter or external
content filter. It probably should not, to avoid archive loops.

Example 1: original SMTP transaction
====================================

In this transaction, Postfix receives an email message that will
later be archived (S=Postfix mailhub; C=remote mailhost):

S: 220 postfix.example.com ESMTP Postfix
C: EHLO mailhost.sender-domain
S: 250-postfix.example.com
...other ESMTP feature announcements...
C: MAIL FROM:<original-***@sender-domain>
S: 250 2.1.0 Ok
C: RCPT TO:<original-***@recipient-domain1>
S: 250 2.1.5 Ok
C: RCPT TO:<original-***@recipient-domain2>
S: 250 2.1.5 Ok
C: DATA:
S: 354 End data with <CR><LF>.<CR><LF>
C: Received: from hostname (hostname [ipaddr]) by mailhost.sender-domain....
Received: by hostname...
From: ...some sender address...
To: ...some recipient address...
Date: ...date...
Subject: ...subject...

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do
eiusmod tempor incididunt ut labore et dolore magna aliqua.
.
S: 250 2.0.0 Ok: queued as 3Zbm2D5lQZznjbn
C: QUIT
S: 221 2.0.0 Bye

Example 2: archive SMTP transaction
===================================

Here, Postfix delivers an archive copy of the message to the archive
server (S=archive server; C=Postfix hub), preserving the original
envelope sender and recipients in the archival copy's From: and To:
headers together with a copy of the Date: and Subject: headers, and
preserving the original message (headers and body) as a message/rfc822
attachment.

S: 220 mailhost.archive-domain ESMTP
C: EHLO splitter.example.com
S: 250-mailhost.archive-domain
...other ESMTP feature announcements...
C: MAIL FROM:<original-***@sender-domain>
S: 250 Ok
C: RCPT TO:<archive-***@archive-domain>
S: 250 Ok
C: DATA:
S: 354 End data with <CR><LF>.<CR><LF>
C: From: <original-***@sender-domain>
To: <original-***@recipient-domain1>,
<***@recipient-domain2>
Date: ...same date as original message...
Subject: ...same subject as original message...
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: Multipart/Mixed;
boundary="unique-boundary-string"

--unique-boundary-string
Content-Type: message/rfc822

Received: from mailhost.sender-domain (hostname [ipaddr]) by postfix.example.com...
Received: from hostname (hostname [ipaddr]) by mailhost.sender-domain....
Received: by hostname...
From: ...some sender address...
To: ...some recipient address...
Date: ...date...
Subject: ...subject...

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do
eiusmod tempor incididunt ut labore et dolore magna aliqua.

--unique-boundary-string--
.
S: 250 Ok: queued as 97C84AC2947
C: QUIT
S: 221 2.0.0 Bye
Viktor Dukhovni
2013-03-29 01:43:27 UTC
Permalink
Post by Wietse Venema
- A new queue file record (soft bounce) saying that Postfix must
never bounce the message containing that record. This is a safety
mechanism for when delivery to the archive server fails.
There should also be an "archive_sender" parameter that specifies
the envelope sender address of archive messages and defaults to
the empty string (null sender) (or perhaps "postmaster", if this
does not risk loops). This ensures that even if downstream systems
don't support DSN the archive message will never bounce or will
bounce to a suitable recipient, not the original sender.
Post by Wietse Venema
Example 2: archive SMTP transaction
===================================
Here, Postfix delivers an archive copy of the message to the archive
server (S=archive server; C=Postfix hub), preserving the original
headers together with a copy of the Date: and Subject: headers, and
preserving the original message (headers and body) as a message/rfc822
attachment.
S: 220 mailhost.archive-domain ESMTP
C: EHLO splitter.example.com
S: 250-mailhost.archive-domain
...other ESMTP feature announcements...
This should I think be "MAIL FROM:<$archive_sender>"
Post by Wietse Venema
...
S: 354 End data with <CR><LF>.<CR><LF>
Date: ...same date as original message...
Subject: ...same subject as original message...
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: Multipart/Mixed;
boundary="unique-boundary-string"
The message envelope could be quite large, since a message
can have thousands of recipients after virtual(5) expansion.
Putting all envelope recipients in the "To:" header risks
running into header length limits that cause header truncation
or parsing problems.

Unlike Postfix, some software imposes limits on the total size
of the header, not just individual header records.

Therefore, the envelope should be presented a MIME body part.

If the above is not compelling, and there is no message body part
containing the envelope (which is carried above in headers) then
we don't need a multipart encapsulation of the inner message, just
set the outer content-type to message/rfc822.
Post by Wietse Venema
--unique-boundary-string
Content-Type: message/rfc822
No need for a multipart/mixed with just a single part.
Post by Wietse Venema
...
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do
eiusmod tempor incididunt ut labore et dolore magna aliqua.
--unique-boundary-string--
Nor this final boundary string.
Post by Wietse Venema
.
S: 250 Ok: queued as 97C84AC2947
C: QUIT
S: 221 2.0.0 Bye
--
Viktor.
Axel Luttgens
2013-03-29 08:15:29 UTC
Permalink
Post by Viktor Dukhovni
[...]
Post by Wietse Venema
...
S: 354 End data with <CR><LF>.<CR><LF>
Date: ...same date as original message...
Subject: ...same subject as original message...
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: Multipart/Mixed;
boundary="unique-boundary-string"
The message envelope could be quite large, since a message
Can have thousands of recipients after virtual(5) expansion.
Putting all envelope recipients in the "To:" header risks
running into header length limits that cause header truncation
Or parsing problems.
Hello Viktor,

Do you think this should be submitted to expansion?
Wouldn't one then lose the original envelope sender and recipient(s)?

Axel
Viktor Dukhovni
2013-03-29 08:38:46 UTC
Permalink
Post by Axel Luttgens
Post by Viktor Dukhovni
[...]
Post by Wietse Venema
...
S: 354 End data with <CR><LF>.<CR><LF>
Date: ...same date as original message...
Subject: ...same subject as original message...
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: Multipart/Mixed;
boundary="unique-boundary-string"
The message envelope could be quite large, since a message
Can have thousands of recipients after virtual(5) expansion.
Putting all envelope recipients in the "To:" header risks
running into header length limits that cause header truncation
Or parsing problems.
Hello Viktor,
Do you think this should be submitted to expansion?
Yes.
Post by Axel Luttgens
Wouldn't one then lose the original envelope sender and recipient(s)?
Only if one uses virtual alias expansion. To archive the original
external recipients, expand downstream of the archive copy.

Typically, you want to archive who received the message, more so
than who it was sent to.

Actually, when archiving the message envelope as a message body
part, I used to record both the original and the final recipient
for each recipient.

Original-Sender: <address-if-different>
Sender: <address>

Original-Recipient: <address-if-different>
Recipient: <address>

Original-Recipient: <address-if-different>
Recipient: <address>

...

Various representations are possible, for example best-effort
consolidation of consecutive entries with the same original recipient:

Original-Sender: <address-if-different>
Sender: <address>

Original-Recipient: <address-if-different>
Recipient: <address>
Recipient: <address>
...

Original-Recipient: <address-if-different>
Recipient: <address>
Recipient: <address>
...

...
--
Viktor.
Patrik Rak
2013-03-29 09:50:07 UTC
Permalink
Post by Wietse Venema
- When the queue manager opens a queue file that contains an archive
recipient address record, it first contacts the archive daemon
before attempting to deliver the message.
....
- After the archive daemon has queued an archive message, the archive
daemon clears the archive recipient record in the queue file and
reports successful completion to the queue manager. Then, the queue
manager processes the queue file for normal delivery.
Is it really required that the message is archived before any further
processing continues? It seems to only introduce delays...

To me, it seems that the archive recipient from the queue manager's POV
is as any other recipient, except that it is automatically routed to it's
own archive transport which passes the message to the archive delivery
agent. Plus the extra treatment of the bounces. Other than that, it
doesn't seem to need to be treated much differently.

Of course, unless it is really mandatory that the message is passed to
archive before anywhere else. But if it is not necessary, I would say it
doesn't matter when exactly it is archived, as long as it is archived. And
it would be, because it would sit in the queue file until the last of the
recipients would be delivered, and would never bounce or otherwise go away
until the archive recipient is cleared.

And even if it would be required to be archived before anywhere else, I
think it would be quite easy to adjust qmgr so it ignores any recipients
other than the archive ones, and only after those were delivered, allows
reading in the other ones from the queue file. This behavior could be even
controlled on per message basis, if necessary.

So, to sum it up, I would rather treat the archive process as an delivery
agent, rather than a daemon similar to bounce or defer daemons. In this
form, archiving is just a special form of a bcc which automatically
transforms the message along the way. No risk of delays induced by waiting
for archival to complete either.
Post by Wietse Venema
Post by Wietse Venema
S: 354 End data with <CR><LF>.<CR><LF>
The message envelope could be quite large, since a message
can have thousands of recipients after virtual(5) expansion.
Putting all envelope recipients in the "To:" header risks
running into header length limits that cause header truncation
or parsing problems.
One needs to split it among multiple To: headers. It is the best
practice anyway.

Postfix itself can handle only limited amount of recipients in single
header, while it can deal with huge amounts when each is given its own
To/Cc/Bcc: header.
Post by Wietse Venema
Unlike Postfix, some software imposes limits on the total size
of the header, not just individual header records.
Therefore, the envelope should be presented a MIME body part.
If the above is not compelling, and there is no message body part
containing the envelope (which is carried above in headers) then
we don't need a multipart encapsulation of the inner message, just
set the outer content-type to message/rfc822.
I guess it would be best to make this part a configurable option of the
archiving daemon. After all, I guess that there might be couple more
transformation options which might people might want to adjust depending
on their needs.

Patrik
Wietse Venema
2013-03-29 12:14:19 UTC
Permalink
Post by Patrik Rak
Post by Wietse Venema
- When the queue manager opens a queue file that contains an archive
recipient address record, it first contacts the archive daemon
before attempting to deliver the message.
....
- After the archive daemon has queued an archive message, the archive
daemon clears the archive recipient record in the queue file and
reports successful completion to the queue manager. Then, the queue
manager processes the queue file for normal delivery.
Is it really required that the message is archived before any further
processing continues? It seems to only introduce delays...
In the draft, the archive copy is *created* before normal message
delivery to recipients. The archive copy can be delivered later.

The archive copy could be created *after* each delivery attempt.
Then, the queue file should not be deleted before the archive daemon
has cleared the archive-recipient record. This would allow deliveries
to continue while the archive daemon is unavailable.

Wietse
Patrik Rak
2013-03-29 13:24:12 UTC
Permalink
Post by Wietse Venema
Post by Patrik Rak
Post by Wietse Venema
- When the queue manager opens a queue file that contains an archive
recipient address record, it first contacts the archive daemon
before attempting to deliver the message.
....
- After the archive daemon has queued an archive message, the archive
daemon clears the archive recipient record in the queue file and
reports successful completion to the queue manager. Then, the queue
manager processes the queue file for normal delivery.
Is it really required that the message is archived before any further
processing continues? It seems to only introduce delays...
In the draft, the archive copy is *created* before normal message
delivery to recipients. The archive copy can be delivered later.
The archive copy could be created *after* each delivery attempt.
Then, the queue file should not be deleted before the archive daemon
has cleared the archive-recipient record. This would allow deliveries
to continue while the archive daemon is unavailable.
That was exactly my point - the archive copy might be delivered to
archive recipient after the original message was delivered to original
recipients anyway, so why bother with the whole asynchronous daemon
thing at all?

Imagine an archive delivery agent, which works basically like the local
delivery agent when it forwards a message copy. The only difference is
that it tweaks the message a bit by adding the expanded information for
archiving purposes as it queues it via the cleanup daemon back to postfix.

During this process, the archiving recipient passed to the agent becomes
a normal recipient of the new file, while the original envelope
recipients (both delivered and not-yet-delivered) are fetched directly
from the original queue file and stored in the body of the new queue
file in whatever format desired, together with any other envelope
information required. Of course, no special archive recipient record is
added to this new queue file.

Having such agent, all what needs to be changed in queue manager is that:

- When it sees the new Archive recipient record, it assigns that
recipient to this internal archive transport. Otherwise it treats it as
any other recipient. In particular, once this recipient is delivered
(which means the archive copy is created), it is reset by the queue
manager like any other recipient. No need for the archiving process to
trespass here at all.

- As long as there is an Archive recipient record active in the queue
file, the message may not be bounced or otherwise deleted. As you say,
this is to take care of the situation when the archive transport is not
working for whatever reasons.

All this can be built quite easily with the existing infrastructure. No
need to add another asynchronous client library talking to special
purpose daemon which would communicate by some other means. I think the
delivery route is the most natural one. It's like bcc-ing the message to
procmail script for archiving, except that it's all builtin and it can
collect the envelope information in its entirety.

Or am I missing something which makes the async daemon preferable?

Patrik
Wietse Venema
2013-03-29 14:28:43 UTC
Permalink
Post by Patrik Rak
Post by Wietse Venema
Post by Patrik Rak
Post by Wietse Venema
- When the queue manager opens a queue file that contains an archive
recipient address record, it first contacts the archive daemon
before attempting to deliver the message.
....
- After the archive daemon has queued an archive message, the archive
daemon clears the archive recipient record in the queue file and
reports successful completion to the queue manager. Then, the queue
manager processes the queue file for normal delivery.
Is it really required that the message is archived before any further
processing continues? It seems to only introduce delays...
In the draft, the archive copy is *created* before normal message
delivery to recipients. The archive copy can be delivered later.
The archive copy could be created *after* each delivery attempt.
Then, the queue file should not be deleted before the archive daemon
has cleared the archive-recipient record. This would allow deliveries
to continue while the archive daemon is unavailable.
That was exactly my point - the archive copy might be delivered to
archive recipient after the original message was delivered to original
recipients anyway, so why bother with the whole asynchronous daemon
thing at all?
The archive copy must be submitted asynchronously, just like email
is delivered asynchronously, and just like a bounce message is
submitted asynchronously.
Post by Patrik Rak
Imagine an archive delivery agent, which works basically like the local
delivery agent when it forwards a message copy. The only difference is
that it tweaks the message a bit by adding the expanded information for
archiving purposes as it queues it via the cleanup daemon back to postfix.
For the archive service to be a delivery agent, Postfix should
produce sensible behavior when people send mail directly to it with
transport_maps or whatever, including all the delivery agent features
that can be tweaked such as rate limits, concurrency scheduling,
and so on. If it doesn't produce sensible behavior in all forseeable
scenarios, then it would only save a few lines of code at the expense
of introducing more opportunities for mistake.

For example, if the bounce daemon were an agent that delivers mail
to the envelope sender, then that could result in chaos as people
configure Postfix to send mail there directly with transport_maps
or whatever (what does it mean to deliver mail directly to the
"trace" or "success" service, when that mail was submitted with
"sendmail -bv" or "sendmail -N success"?) Instead, Postfix has the
"error" and "retry" transports (and no transports that correspond
to the "success" or "trace" services).

Wietse
Patrik Rak
2013-03-29 14:47:28 UTC
Permalink
Post by Wietse Venema
The archive copy must be submitted asynchronously, just like email
is delivered asynchronously, and just like a bounce message is
submitted asynchronously.
Of course. It wasn't the asynchronicity I was questioning.
Post by Wietse Venema
For the archive service to be a delivery agent, Postfix should
produce sensible behavior when people send mail directly to it with
transport_maps or whatever, including all the delivery agent features
that can be tweaked such as rate limits, concurrency scheduling,
and so on.
Exactly. You would be able to use all these features, in the very same
way you can use them for the "error" transport.
Post by Wietse Venema
For example, if the bounce daemon were an agent that delivers mail
to the envelope sender, then that could result in chaos as people
configure Postfix to send mail there directly with transport_maps
or whatever (what does it mean to deliver mail directly to the
"trace" or "success" service, when that mail was submitted with
"sendmail -bv" or "sendmail -N success"?) Instead, Postfix has the
"error" and "retry" transports (and no transports that correspond
to the "success" or "trace" services).
Quite right. But it seems quite reasonable to send something to "archive
***@domain.tld" transport, in the very same way it makes sense to send
it to an "error msg" transport. I though of this as an advantage, not a
shortcoming.

Anyway, I saw you have been doing pretty cool changes recently, so I am
sure that whatever you'll choose will be great in this case as well. I
won't be around for few days now, so let me wish you Happy Easter now...

Patrik
Wietse Venema
2013-03-29 18:46:52 UTC
Permalink
Post by Patrik Rak
Post by Wietse Venema
For the archive service to be a delivery agent, Postfix should
produce sensible behavior when people send mail directly to it with
transport_maps or whatever, including all the delivery agent features
that can be tweaked such as rate limits, concurrency scheduling,
and so on.
Exactly. You would be able to use all these features, in the very same
way you can use them for the "error" transport.
[...] But it seems quite reasonable to send something to "archive
it to an "error msg" transport. I though of this as an advantage, not a
shortcoming.
If Postfix sends mail for ***@example.com directly to the archive
delivery agent using transport_maps or whatever, how would Postfix
deliver the output from the archive agent to ***@example.com without
running into a loop?

This problem exists because a) the archive service is a delivery
agent, and b) the input email address is the same as the output
email address.

Possible solutions:

- Use an input email address that differs from the output email address.

I think that this is just too bizarre to consider.

- Change Postfix routing policy, so that the same email address
produces different results in different contexts. Currently, the
Postfix address resolver implements two routing policies: one for
normal deliveries and one for address verification probes. The
policies may differ when probes must avoid some intermediate host.
The Postfix address resolver could implement a third routing
policy to avoid deliveries to the archive transport.

With this, even a simple archival scenario would be complicated.

- Don't use a delivery agent model, so that the archive daemon
receives the archive recipient address via a channel that is not
subject to Postfix routing policy.

This is the initial draft. In the simplest scenario it sends
achive copies to a specified email address without requiring
any configuration besides the email address itself.
Post by Patrik Rak
Anyway, I saw you have been doing pretty cool changes recently,
so I am sure that whatever you'll choose will be great in this
case as well. I won't be around for few days now, so let me wish
you Happy Easter now...
Thanks for the input. We aim to keep Postfix relevant.

Wietse

Viktor Dukhovni
2013-03-29 18:30:55 UTC
Permalink
Post by Patrik Rak
Is it really required that the message is archived before any further
processing continues? It seems to only introduce delays...
To me, it seems that the archive recipient from the queue manager's POV
is as any other recipient, except that it is automatically routed to it's
own archive transport which passes the message to the archive delivery
agent. Plus the extra treatment of the bounces. Other than that, it
doesn't seem to need to be treated much differently.
The main requirement for this to work reliably is for queue-file
REC_TYPE_DONE records to never be used for purposes other than
finalizing actual recipients (I believe this is true today). If
Postfix were to use REC_TYPE_DONE for other purposes, or to finalize
recipients with a different record type, the archive daemon would
have trouble determining the message envelope.

As for delivery-agent vs. internal service, the internal service
approach is likely more robust. There should only be one archive
recipient per message. If someone wants more, that can be handled
via virtual(5) downstream.
--
Viktor.
Wietse Venema
2013-03-29 11:07:53 UTC
Permalink
Viktor Dukhovni:
[archive transaction]
Post by Viktor Dukhovni
Post by Wietse Venema
...
S: 354 End data with <CR><LF>.<CR><LF>
Date: ...same date as original message...
Subject: ...same subject as original message...
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: Multipart/Mixed;
boundary="unique-boundary-string"
The message envelope could be quite large, since a message
No, this would show the original recipients, not the expanded ones.

However, the list could be long in case of "sendmail -t" submission.
This means the header may show only a sub-set with the complete list
im a MIME segment before the message/rfc822 with the original message.

Wietse
Wietse Venema
2013-03-29 13:04:57 UTC
Permalink
Post by Wietse Venema
[archive transaction]
Post by Viktor Dukhovni
Post by Wietse Venema
...
S: 354 End data with <CR><LF>.<CR><LF>
Date: ...same date as original message...
Subject: ...same subject as original message...
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: Multipart/Mixed;
boundary="unique-boundary-string"
The message envelope could be quite large, since a message
No, this would show the original recipients, not the expanded ones.
However, the list could be long in case of "sendmail -t" submission.
This means the header may show only a sub-set with the complete list
in a MIME segment before the message/rfc822 with the original message.
On the other hand, with this draft Postfix could easily list both
the current and original recipients (unlike a design based on an
SMTP-level "Y" splitter, which the current draft descends from).

If we do that, then we should almost certainly reuse existing syntax
instead of rolling our own. That will hopefully allow Postfix to
track developments as that format evolves to support richer
address/domain syntax.

Content-Type: multipart/related;
boundary=unique-boundary-string

--unique-boundary-string
Content-Description: Message envelope
Content-Type: text/plain

Reporting-MTA: dns; mailhost.example.com
X-Postfix-Queue-ID: 84863BC0E5
Sender: rfc822; ***@sender-domain
Original-Sender: rfc822; ***@sender-domain
Arrival-Date: ...mail date format...

Final-Recipient: rfc822; final-***@final-domain1
Original-Recipient: rfc822; original-***@original-domain1

Final-Recipient: rfc822; final-***@final-domain2
Original-Recipient: rfc822; original-***@original-domain2

-unique-boundary-string
Content-Description: Message content
Content-Type: message/rfc822
...original message headers...

...original message body...

--unique-boundary-string--

Wietse
Viktor Dukhovni
2013-03-29 18:04:23 UTC
Permalink
Post by Wietse Venema
On the other hand, with this draft Postfix could easily list both
the current and original recipients (unlike a design based on an
SMTP-level "Y" splitter, which the current draft descends from).
[ FWIW my original Y splitter parsed ORCPT from DSN "RCPT TO" commands ]
Post by Wietse Venema
If we do that, then we should almost certainly reuse existing syntax
instead of rolling our own. That will hopefully allow Postfix to
track developments as that format evolves to support richer
address/domain syntax.
Content-Type: multipart/related;
boundary=unique-boundary-string
Multipart/related is required to designate a top-level type for
the primary part, and may then use Content-ID linkage (e.g. <cid:....>
URIs):

Content-Type: multipart/related; type = message/rfc822;
boundary = unique-boundary-string

here it is not clear whether the envelope or the message is the
primary payload, and which is the "related", so I would just
go with multipart/mixed. That way MUAs will display two parts,
rather than just the "root" part.

https://tools.ietf.org/html/rfc2387
Post by Wietse Venema
--unique-boundary-string
Content-Description: Message envelope
Content-Type: text/plain
I think it is a fine idea (I would have used that in the Y-splitter
were it not for legacy compatibility issues) to use

https://tools.ietf.org/html/rfc3464

as the envelope encoding format, and to even tag the MIME type as
such:

Content-Type: message/delivery-status
Post by Wietse Venema
Reporting-MTA: dns; mailhost.example.com
X-Postfix-Queue-ID: 84863BC0E5
Arrival-Date: ...mail date format...
-unique-boundary-string
Content-Description: Message content
Content-Type: message/rfc822
...original message headers...
...original message body...
--unique-boundary-string--
Viktor Dukhovni
2013-03-29 18:07:08 UTC
Permalink
Post by Wietse Venema
[archive transaction]
Post by Viktor Dukhovni
Post by Wietse Venema
...
S: 354 End data with <CR><LF>.<CR><LF>
Date: ...same date as original message...
Subject: ...same subject as original message...
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: Multipart/Mixed;
boundary="unique-boundary-string"
The message envelope could be quite large, since a message
No, this would show the original recipients, not the expanded ones.
However, the list could be long in case of "sendmail -t" submission.
This means the header may show only a sub-set with the complete list
im a MIME segment before the message/rfc822 with the original message.
Sure, but I don't see much value in adding header payload when the
recipients are already in a body part. The original message header
is easily viewed in the message/rfc822 attachment.
--
Viktor.
Emmanuel Fusté
2013-03-29 13:38:25 UTC
Permalink
Post by Wietse Venema
This week I was asked to help out colleagues who were trying to
make Postfix work with some email archive system. I drafted up a
few solutions, one of which was adopted.
During this exercise I realized that this would be less painful if
Postfix had built-in support to create "archive-quality" copies of
email messages. Below is a straw-man design.
Wietse
Draft design: Building email archival support into Postfix.
........
Example 2: archive SMTP transaction
===================================
Here, Postfix delivers an archive copy of the message to the archive
server (S=archive server; C=Postfix hub), preserving the original
headers together with a copy of the Date: and Subject: headers, and
preserving the original message (headers and body) as a message/rfc822
attachment.
Hello,

If something like this is implemented (and it would be a useful thing ),
could you consider adding a configuration option to only preserve the
original message headers (not body) ?
We need something like your email archival support proposal in postfix,
but for legal reason we are not able to archive the body of the message
in our main use.

Emmanuel.
Loading...