Discussion:
RFC 5335, 5336, 5337 Support
(too old to reply)
Stefan Sels
2011-10-18 18:26:11 UTC
Permalink
Hi,

I just wanted to ask if there are intentions to support the new upcoming
i18n RFCs 5335, 5336 and 5337 listed here:

http://datatracker.ietf.org/wg/eai/

5336 defines an SMTP extension to allow the use of UTF8 within
local-parts, the hostname part (i guess) is still handled via IDN.

I would be willing to support the development of such...

greetings,
stefan
Wietse Venema
2011-10-18 18:54:57 UTC
Permalink
Post by Stefan Sels
Hi,
I just wanted to ask if there are intentions to support the new upcoming
http://datatracker.ietf.org/wg/eai/
5336 defines an SMTP extension to allow the use of UTF8 within
local-parts, the hostname part (i guess) is still handled via IDN.
I would be willing to support the development of such...
This is a major project that is going to affect many parts of
Postfix, because Postfix implements many email-related protocols
(unlike other source MTAs that pretend MIME and DSN don't exist).

I do not intend to take code donations; in my experience reviewing
and fixing other people's code results in CVEs like those for SASL
and TLS this year, and would take more time than writing the code
myself. My time is very limited, so this is going to take a while.

Wietse
Stefan Sels
2011-10-18 19:01:03 UTC
Permalink
Post by Wietse Venema
This is a major project that is going to affect many parts of
Postfix, because Postfix implements many email-related protocols
(unlike other source MTAs that pretend MIME and DSN don't exist).
I do not intend to take code donations; in my experience reviewing
and fixing other people's code results in CVEs like those for SASL
and TLS this year, and would take more time than writing the code
myself. My time is very limited, so this is going to take a while.
That sounds good anyway. I can't code in the quality that it might be
accepted into postfix. But I can offer to do testing and/or write a
test-suite that checks if the implementation is
working/accepting/rejecting standard conform messages.
Post by Wietse Venema
Wietse
greetings,
stefan
Viktor Dukhovni
2011-10-18 19:05:07 UTC
Permalink
Post by Stefan Sels
I just wanted to ask if there are intentions to support the new upcoming
http://datatracker.ietf.org/wg/eai/
5336 defines an SMTP extension to allow the use of UTF8 within
local-parts, the hostname part (i guess) is still handled via IDN.
My assessment is that EAI (as proposed in the RFCs you quote) is
a terrible design. I personally would prefer that nobody implement
these IMNSHO ill-conceived standards.
--
Viktor.
Stefan Sels
2011-10-18 19:08:49 UTC
Permalink
Post by Viktor Dukhovni
Post by Stefan Sels
I just wanted to ask if there are intentions to support the new upcoming
http://datatracker.ietf.org/wg/eai/
5336 defines an SMTP extension to allow the use of UTF8 within
local-parts, the hostname part (i guess) is still handled via IDN.
My assessment is that EAI (as proposed in the RFCs you quote) is
a terrible design. I personally would prefer that nobody implement
these IMNSHO ill-conceived standards.
Can you suggest a better standard to encode international characters
into local-parts?

And, if I may ask - as there are 12 days to moan about until it probably
gets finalized- did you complain about this to the working group?
What are the specific flaw you dislike?

greetings,
stefan
Viktor Dukhovni
2011-10-19 15:08:51 UTC
Permalink
Post by Stefan Sels
Post by Viktor Dukhovni
Post by Stefan Sels
5336 defines an SMTP extension to allow the use of UTF8 within
local-parts, the hostname part (i guess) is still handled via IDN.
My assessment is that EAI (as proposed in the RFCs you quote) is
a terrible design. I personally would prefer that nobody implement
these IMNSHO ill-conceived standards.
Can you suggest a better standard to encode international characters
into local-parts?
Doing nothing is I believe much better than the proposed standard.
As for a better standard, I would propose extending punycode syntax
to address localparts.
Post by Stefan Sels
And, if I may ask - as there are 12 days to moan about until it probably
gets finalized- did you complain about this to the working group?
The working group did not consult my opinion. I was unaware of
their existence and charter.
Post by Stefan Sels
What are the specific flaw you dislike?
The entire approach really, it is much too complex, but specifically:

- The standard proposes carrying two addresses in the message
header and envelope for every recipient. This way lies insanity.
Bugs, security issues, unpredictable message handling.

- The standard fundamentally violates the design of MIME by
allowing transfer-encoding of composite (mail/message-rfc822)
parts. Breaks content inspection, MIME normalizers, ...

These are not minor issues. My vote is that this standard is dead
on arrival, may it soon be forgotten.
--
Viktor.
Wietse Venema
2011-10-19 16:50:26 UTC
Permalink
Post by Viktor Dukhovni
- The standard proposes carrying two addresses in the message
header and envelope for every recipient. This way lies insanity.
Bugs, security issues, unpredictable message handling.
- The standard fundamentally violates the design of MIME by
allowing transfer-encoding of composite (mail/message-rfc822)
parts. Breaks content inspection, MIME normalizers, ...
These are not minor issues. My vote is that this standard is dead
on arrival, may it soon be forgotten.
I would not mind if it dies, for those reasons. That said, if it
gets adoption, then there is little choice but waste a year of
development and add it to Postfix.

Wietse
Patrick Ben Koetter
2011-10-19 18:33:59 UTC
Permalink
Post by Wietse Venema
Post by Viktor Dukhovni
- The standard proposes carrying two addresses in the message
header and envelope for every recipient. This way lies insanity.
Bugs, security issues, unpredictable message handling.
- The standard fundamentally violates the design of MIME by
allowing transfer-encoding of composite (mail/message-rfc822)
parts. Breaks content inspection, MIME normalizers, ...
These are not minor issues. My vote is that this standard is dead
on arrival, may it soon be forgotten.
I would not mind if it dies, for those reasons. That said, if it
gets adoption, then there is little choice but waste a year of
development and add it to Postfix.
Seriously. If both of you think the whole thing it crap, why don't you stand
up then and make yourself heared? Your voices have some wheight.

***@rick
--
All technical questions asked privately will be automatically answered on the
list and archived for public access unless privacy is explicitely required and
justified.

saslfinger (debugging SMTP AUTH):
<http://postfix.state-of-mind.de/patrick.koetter/saslfinger/>
Wietse Venema
2011-10-19 19:18:56 UTC
Permalink
Post by Patrick Ben Koetter
Post by Wietse Venema
Post by Viktor Dukhovni
- The standard proposes carrying two addresses in the message
header and envelope for every recipient. This way lies insanity.
Bugs, security issues, unpredictable message handling.
- The standard fundamentally violates the design of MIME by
allowing transfer-encoding of composite (mail/message-rfc822)
parts. Breaks content inspection, MIME normalizers, ...
These are not minor issues. My vote is that this standard is dead
on arrival, may it soon be forgotten.
I would not mind if it dies, for those reasons. That said, if it
gets adoption, then there is little choice but waste a year of
development and add it to Postfix.
Seriously. If both of you think the whole thing it crap, why don't you stand
up then and make yourself heared? Your voices have some wheight.
I have better things to do than to play Don Quixote.

To end this thread on a positive note, a great man wrote a book
chapter about the second system effect(*), and some people in IETF
would benefit from reading it.

Wietse

Fred Brooks, The Mythical Man-Month.
http://en.wikipedia.org/wiki/Second-system_effect
John Levine
2011-10-20 03:05:06 UTC
Permalink
Sheesh. If you're going to complain about EAI, would it be too
much work to at least find out what it says?
Post by Viktor Dukhovni
As for a better standard, I would propose extending punycode syntax
to address localparts.
There are real technical reasons that doesn't work, hashed out at
length in the EAI group.
Post by Viktor Dukhovni
- The standard proposes carrying two addresses in the message
header and envelope for every recipient.
No, it doesn't. Please read the current EAI drafts, not the old
experimental RFCs.
Post by Viktor Dukhovni
- The standard fundamentally violates the design of MIME by
allowing transfer-encoding of composite (mail/message-rfc822)
parts.
No, it doesn't. Please read the current EAI drafts, not the old
experimental RFCs.

R's,
John
Wietse Venema
2011-10-20 12:53:37 UTC
Permalink
Post by John Levine
Post by Viktor Dukhovni
- The standard fundamentally violates the design of MIME by
allowing transfer-encoding of composite (mail/message-rfc822)
parts.
No, it doesn't. Please read the current EAI drafts, not the old
experimental RFCs.
We are referring to this text:

This specification updates Section 6.4 of [RFC2045]. [RFC2045]
prohibits applying a content-transfer-encoding to any subtypes of
"message/". This specification relaxes that rule -- it allows newly
defined MIME types to permit content-transfer-encoding, and it allows
content-transfer-encoding for message/global (see Section 3.7).

in draft-ietf-eai-rfc5335bis-12.txt. While Victor is too specific
about message/rfc822, the draft explicitly allows transfer-encoding
of message/global, which is just as problematic as allowing
message/rfc822 non-identity encoding.

Wietse
John R. Levine
2011-10-20 15:20:47 UTC
Permalink
Post by Wietse Venema
in draft-ietf-eai-rfc5335bis-12.txt. While Victor is too specific
about message/rfc822, the draft explicitly allows transfer-encoding
of message/global, which is just as problematic as allowing
message/rfc822 non-identity encoding.
Yes, it does, although that's the best of a bad lot. People are going to
send EAI messages as attachments, some of the mail they're attaching it
to will be 5322. What else could they do? Tell people to hide them
inside base64 encoded zip files?

Really, they went through every option you can possibly think of to make
EAI mail somewhat more backwards compatible, and they all went down
ratholes, so they stripped it all out and made EAI a separate parallel
mailstream to 5321/5322

See my three-part blog entry at http://jl.ly/Email/i18n.html

Regards,
John Levine, ***@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly
Wietse Venema
2011-10-20 15:55:43 UTC
Permalink
Post by John R. Levine
Post by Wietse Venema
in draft-ietf-eai-rfc5335bis-12.txt. While Victor is too specific
about message/rfc822, the draft explicitly allows transfer-encoding
of message/global, which is just as problematic as allowing
message/rfc822 non-identity encoding.
Yes, it does, although that's the best of a bad lot. People are going to
send EAI messages as attachments, some of the mail they're attaching it
to will be 5322. What else could they do? Tell people to hide them
inside base64 encoded zip files?
Once you allow non-ascii headers, you break compatibility with
existing infrastructure, so you break other things to work around
it (with "you" not specifically aimed at John R. Levine, btw). For
security's sake I'd prefer if they had broken the 8bit header rule
instead of MIME.

Wietse
Viktor Dukhovni
2011-10-20 20:43:53 UTC
Permalink
Post by Wietse Venema
Post by John R. Levine
Post by Wietse Venema
in draft-ietf-eai-rfc5335bis-12.txt. While Victor is too specific
about message/rfc822, the draft explicitly allows transfer-encoding
of message/global, which is just as problematic as allowing
message/rfc822 non-identity encoding.
Yes, it does, although that's the best of a bad lot. People are going to
send EAI messages as attachments, some of the mail they're attaching it
to will be 5322. What else could they do? Tell people to hide them
inside base64 encoded zip files?
Once you allow non-ascii headers, you break compatibility with
existing infrastructure, so you break other things to work around
it (with "you" not specifically aimed at John R. Levine, btw). For
security's sake I'd prefer if they had broken the 8bit header rule
instead of MIME.
Indeed, I consider radically changing the design of MIME (composite
MIME entities are never subject to transfer encoding) a much more
substantial change than any change in the details of SMTP or
character restrictions on RFC822 headers.

There are a lot more MIME readers than MTAs, and fundamentally
changing MIME is going to cause breakage near and far from the MTA.

Having written a number of MIME parsers, a few of them security-oriented
normalizers, I can assure you that these took the MIME design
promise (of single-pass parsing of composites without nested
decoding) seriously, and messages that violate such promises are
and will be treated with prejudice.

I'll try to take a look at the final RFCs when I get the chance. If
the "<address1 <address2>>" syntax is gone, perhaps that objection
falls away...
--
Viktor.
Wietse Venema
2011-10-20 21:08:47 UTC
Permalink
Post by Viktor Dukhovni
I'll try to take a look at the final RFCs when I get the chance. If
the "<address1 <address2>>" syntax is gone, perhaps that objection
falls away...
This is gone, as far as I can tell.

The only interoperability issue that I could identify is with the
strict_mime_encoding_domain checks, which are off by default. I
have now excluded unknown message subtypes such as message/global*
from this check.

Otherwise, UTF8SMTP will inter-operate just fine with current Postfix
implementations. Postfix already converts unknown message subtypes
such as message/global* to 7bit when delivering to non-8bit
destinations, and such destinations should slowly go extinct.

But until EAI support is implemented in Postfix, there will be a
gaping hole because the non-identity content transfer encodings
cripple Postfix's MIME header_checks and body_checks.

Wietse
John Levine
2011-10-22 23:12:41 UTC
Permalink
For security's sake I'd prefer if they had broken the 8bit header
rule instead of MIME.
I think you're missing the point. You're suggesting that MTAs stuff 8
bit data down 7 bit datapaths, which I hope we agree is a non-starter.

All EAI MTAs will support 8BITMIME, for which there's no need to
recode message/global MIME parts. Many, perhaps most, 5321 MTAs also
support 8BITMIME, for which there's also no need to recode them. The
problem is when you have a dusty old MTA that isn't 8bit clean. You
can't send unencoded UTF-8, the channel won't support it. So what
else do you propose that they do?

I suppose they could have called message/global something like
application/hideous-chinese-mail, but that doesn't seem like it would
fix anything.

R's,
John
Wietse Venema
2011-10-22 23:26:03 UTC
Permalink
Post by John Levine
For security's sake I'd prefer if they had broken the 8bit header
rule instead of MIME.
I think you're missing the point. You're suggesting that MTAs stuff 8
bit data down 7 bit datapaths, which I hope we agree is a non-starter.
Isn't that exactly Exim and qmail (and perhaps others) have been
doing all along: accept 8BITMIME mail, and send it to 7bit MTAs
without return-to-sender or 8-to-7 downgrade?

I don't recall that and anyone has revoked their Internet driver's
licence, so I am somewhat puzzled by this comment about non-starters.

Wietse
Post by John Levine
All EAI MTAs will support 8BITMIME, for which there's no need to
recode message/global MIME parts. Many, perhaps most, 5321 MTAs also
support 8BITMIME, for which there's also no need to recode them. The
problem is when you have a dusty old MTA that isn't 8bit clean. You
can't send unencoded UTF-8, the channel won't support it. So what
else do you propose that they do?
I suppose they could have called message/global something like
application/hideous-chinese-mail, but that doesn't seem like it would
fix anything.
R's,
John
John Levine
2011-10-23 06:37:43 UTC
Permalink
Post by Wietse Venema
Isn't that exactly Exim and qmail (and perhaps others) have been
doing all along: accept 8BITMIME mail, and send it to 7bit MTAs
without return-to-sender or 8-to-7 downgrade?
If they do, it's a bug. Since a 7 bit MTA will typically lose the
high bit, it would be more honest just to throw the mail away, since
there's little chance it'll arrive in usable form.

In any event, EAI is what it is. If you don't think it's useful to
support non-ASCII mail addresses, you don't have to implement it.

R's,
John
Wietse Venema
2011-10-23 13:41:23 UTC
Permalink
Post by John Levine
Post by Wietse Venema
Isn't that exactly Exim and qmail (and perhaps others) have been
doing all along: accept 8BITMIME mail, and send it to 7bit MTAs
without return-to-sender or 8-to-7 downgrade?
If they do, it's a bug. Since a 7 bit MTA will typically lose the
high bit, it would be more honest just to throw the mail away, since
there's little chance it'll arrive in usable form.
See http://bugs.exim.org/show_bug.cgi?id=817; qmail has no bugs :-)

While corruption may have been a problem 15 years ago, nowadays
non-8BITMIME servers are more likely than not to be 8-bit clean.
There are some measurements in the last comments of exim bug 817.
Post by John Levine
In any event, EAI is what it is. If you don't think it's useful to
support non-ASCII mail addresses, you don't have to implement it.
I expect to do a full implementation including compatibility with
RFC532[12]. The spec was cleaned up significantly, so it is good
that I declined to start work on this in 2008/9.

Wietse

Loading...