Discussion:
Postfix 2.10.0, Mac OS X 10.8.3 and kqueue
(too old to reply)
Axel Luttgens
2013-03-21 22:24:08 UTC
Permalink
Since I needed to compile Postfix, I thought this could be the opportunity to check the state of kqueue on Mac OS X at the same time: the most recent thread I could find about those matters is the one that started with http://www.mailinglistarchive.com/html/postfix-***@postfix.org/2007-03/msg00051.html.

This is with Postfix 2.10.0 on Mac OS X 10.8.3.

The change introduced in the makedefs file is:

--- makedefs.original 2013-02-04 02:33:13.000000000 +0100
+++ makedefs 2013-03-21 13:47:26.000000000 +0100
@@ -476,7 +476,11 @@
*) CCARGS="$CCARGS -DRESOLVE_H_NEEDS_ARPA_NAMESER_COMPAT_H";;
esac
# kqueue and/or poll are broken up to and including MacOS X 10.5
- CCARGS="$CCARGS -DNO_KQUEUE"
+ #CCARGS="$CCARGS -DNO_KQUEUE"
+ case $RELEASE in
+ 12.*) ;;
+ *) CCARGS="$CCARGS -DNO_KQUEUE" ;;
+ esac
# # Darwin 8.11.1 has kqueue support, but let's play safe
# case $RELEASE in
# [1-8].*) CCARGS="$CCARGS -DNO_KQUEUE";;

Note that the change is a conservative one, since I haven't tried on Mac OS X 10.7 (Darwin 11) and earlier.

The change described in my other post ("Mac OS X and setrlimit(2)") is required as well, since one will now call open_limit(INT_MAX) instead of open_limit(FD_SETSIZE).

No compilation errors.
Not even a single additional warning. :-)
And the master process starts without a glitch.
Postfix 2.4 release candidate 8 attempts to use kqueue() and poll()
on MacOS 10.4 and later. This should work but it would be nice if
- the master starts smtpd processes as expected,
OK.
and smtpd can receive mail as expected.
- smtpd correctly times out when the client says nothing
for a long time (just telnet to the SMTP port and wait).
OK: "timeout after CONNECT from localhost[127.0.0.1]" after exactly 5 minutes.
- "postfix reload" starts a new queue manager immediately.
OK.
An smtp-source/sink test can help to further confirm proper operation
but should not be necessary.
OK: performed exactly the same test as the one described by Victor in http://www.mailinglistarchive.com/html/postfix-***@postfix.org/2007-03/msg00052.html.


So, looks quite promising. :-)
Are there other tests I could/should run in order to be fully reassured?

Axel
Wietse Venema
2013-03-21 23:11:39 UTC
Permalink
Post by Axel Luttgens
So, looks quite promising. :-)
Are there other tests I could/should run in order to be fully reassured?
Does it work with postscreen? (turn off postscreen cache, turn on
"after 220 greeting" tests, then do the same tests as with smtpd).

Does it work with FIFOs for qmgr and pickup? (edit master.cf then
"postfix reload").

Wietse
Axel Luttgens
2013-03-22 15:22:51 UTC
Permalink
Post by Wietse Venema
Post by Axel Luttgens
So, looks quite promising. :-)
Are there other tests I could/should run in order to be fully reassured?
Does it work with postscreen? (turn off postscreen cache, turn on
"after 220 greeting" tests, then do the same tests as with smtpd).
Did you mean something like:

#smtp inet n - n - - smtpd
smtp inet n - n - 1 postscreen
smtpd pass - - n - - smtpd
dnsblog unix - - n - 0 dnsblog

in master.cf and:

postscreen_cache_map =
postscreen_access_list =
postscreen_pipelining_enable = yes
postscreen_non_smtp_command_enable = yes
postscreen_bare_newline_enable = yes

in main.cf?

With above settings, I'm able to speak to postcreen up to the RCPT command inclusive, then get a "450 4.3.2 Service currently unavailable" message and have to QUIT.

As expected, a connection to postscreen left idle quits after 5 minutes.

Activating caching (i.e. removing the "postscreen_cache_map =" line from main.cf) then results in a "PASS OLD [127.0.0.1]:port" message and I'm handed off to a smtpd process, through which I may send a message.

So, as far as I can tell, everything appears to be fine with postscreen.
Post by Wietse Venema
Does it work with FIFOs for qmgr and pickup? (edit master.cf then
"postfix reload").
Having the following in master.cf

#pickup unix n - n 60 1 pickup
pickup fifo n - n 60 1 pickup
cleanup unix n - n - 0 cleanup
#qmgr unix n - n 300 1 qmgr
qmgr fifo n - n 300 1 qmgr

indeed creates FIFOs instead of sockets:

prw--w--w- 1 _postfix _postdrop 0 22 mar 15:27 pickup
prw--w--w- 1 _postfix _postdrop 0 22 mar 15:27 qmgr

and all tests from the "check-list" (see my initial message for this thread) succeed.

Again, seems to be quite conclusive.

Axel
Wietse Venema
2013-03-22 18:08:37 UTC
Permalink
Axel Luttgens:
[ Charset ISO-8859-1 unsupported, converting... ]
Post by Axel Luttgens
Post by Wietse Venema
Post by Axel Luttgens
So, looks quite promising. :-)
Are there other tests I could/should run in order to be fully reassured?
Does it work with postscreen? (turn off postscreen cache, turn on
"after 220 greeting" tests, then do the same tests as with smtpd).
[...]

Please refer to the POSTSCREEN_README file.
Post by Axel Luttgens
With above settings, I'm able to speak to postcreen up to the RCPT
command inclusive, then get a "450 4.3.2 Service currently
unavailable" message and have to QUIT.
As expected, a connection to postscreen left idle quits after 5 minutes.
Activating caching (i.e. removing the "postscreen_cache_map ="
line from main.cf) then results in a "PASS OLD [127.0.0.1]:port"
message and I'm handed off to a smtpd process, through which I may
send a message.
Excellent now you have to do it a couple thousand times in parallel
sessions.

[FIFOS]
Post by Axel Luttgens
Again, seems to be quite conclusive.
Great, to Apple fixed it.

Do you need code for testing the sysctl() stuff?

I was thinking of:

--------
if (limit > 0) {
#ifdef MACOSX
#define MAX_FILES_PER_PROC "kern.maxfilesperproc"
long current_limit;
size_t len = sizeof(current_limit);
if (sysctlbyname(MAX_FILES_PER_PROC, &current_limit, &len,
(void *) 0, (size_t) 0) < 0)
return (-1);
if (limit > current_limit)
limit = current_limit;
#else
if (limit > rl.rlim_max)
rl.rlim_cur = rl.rlim_max;
#endif
else
rl.rlim_cur = limit;
if (setrlimit(RLIMIT_NOFILE, &rl) < 0)
return (-1);
}
return (rl.rlim_cur);
--------

Or something close to that. I did not test the code.

Wietse
Wietse Venema
2013-03-22 19:46:58 UTC
Permalink
Post by Wietse Venema
Do you need code for testing the sysctl() stuff?
To make the job easier, below is a patch to unmodified open_limit.c

Wietse

*** ./open_limit.c- Fri Dec 11 13:55:29 1998
--- ./open_limit.c Fri Mar 22 15:44:13 2013
***************
*** 34,39 ****
--- 34,48 ----
#include <sys/resource.h>
#include <errno.h>

+ /*
+ * XXX MacOSX rlim_cur is only 256, rlim_max is infinite, so we use the
+ * kern.maxfilesperproc value instead.
+ */
+ #ifdef MACOSX
+ #include <sys/sysctl.h>
+ #define MAX_FILES_PER_PROC "kern.maxfilesperproc"
+ #endif
+
/* Application-specific. */

#include "iostuff.h"
***************
*** 63,68 ****
--- 72,87 ----
if (getrlimit(RLIMIT_NOFILE, &rl) < 0)
return (-1);
if (limit > 0) {
+ #ifdef MACOSX
+ long current_limit;
+ size_t len = sizeof(current_limit);
+
+ if (sysctlbyname(MAX_FILES_PER_PROC, &current_limit, &len,
+ (void *) 0, (size_t) 0) < 0)
+ return (-1);
+ if (limit > current_limit)
+ limit = current_limit;
+ #endif
if (limit > rl.rlim_max)
rl.rlim_cur = rl.rlim_max;
else
Axel Luttgens
2013-03-23 15:40:43 UTC
Permalink
Post by Wietse Venema
Post by Wietse Venema
Do you need code for testing the sysctl() stuff?
To make the job easier, below is a patch to unmodified open_limit.c
Wietse
[...]
Fine, thanks Wietse!

I like that solution, since it doesn't change the code's behavior for other OSes (a behavior that seems to have been adequate for more than a decade).

Still wondering whether Mac OS X is the only platform to set restrictions on the values passed to setrlimit().
Perhaps not, should the man page at:
http://www.unix.com/man-page/all/2/setrlimit/
be trusted (see description of error EPERM).

BTW, I didn't immediately notice the long type for current_limit, and thus got immediate failures: getsysctlbyname() returned values such as 0x7FD500002800 -note the 0x00002800 half- and this proved even worse than INT_MAX... ;-)

For our records, I reproduce hereafter the changes I have applied to open_limit.c.

Axel


--- src/util/open_limit.c.original 1998-12-11 19:55:29.000000000 +0100
+++ src/util/open_limit.c 2013-03-23 15:18:25.000000000 +0100
@@ -34,6 +34,16 @@
#include <sys/resource.h>
#include <errno.h>

+/*
+ * XXX MacOSX rlim_cur is only 256, rlim_max is infinite, so we use the
+ * kern.maxfilesperproc value instead, because setrlimit(2) on that OS
+ * doesn't allow arbitrary values for setting rlim_cur.
+ */
+#ifdef MACOSX
+#include <sys/sysctl.h>
+#define MAX_FILES_PER_PROC "kern.maxfilesperproc"
+#endif
+
/* Application-specific. */

#include "iostuff.h"
@@ -63,6 +73,16 @@
if (getrlimit(RLIMIT_NOFILE, &rl) < 0)
return (-1);
if (limit > 0) {
+#ifdef MACOSX
+ int current_limit;
+ size_t len = sizeof(current_limit);
+
+ if (sysctlbyname(MAX_FILES_PER_PROC, &current_limit, &len,
+ (void *) 0, (size_t) 0) < 0)
+ return (-1);
+ if (limit > current_limit)
+ limit = current_limit;
+#endif
if (limit > rl.rlim_max)
rl.rlim_cur = rl.rlim_max;
else
Wietse Venema
2013-03-24 01:33:59 UTC
Permalink
Axel Luttgens:
[ Charset ISO-8859-1 unsupported, converting... ]
Post by Axel Luttgens
Post by Wietse Venema
Post by Wietse Venema
Do you need code for testing the sysctl() stuff?
To make the job easier, below is a patch to unmodified open_limit.c
Wietse
[...]
Fine, thanks Wietse!
I like that solution, since it doesn't change the code's behavior
for other OSes (a behavior that seems to have been adequate for
more than a decade).
Still wondering whether Mac OS X is the only platform to set
restrictions on the values passed to setrlimit().
It certainly makes sense for setrlimit() to forbid rlim_cur values
larger than the hard upper limit. That's what the hard limit is for.

The bug is that Mac OS X rlim_max equals RLIM_INFINITY, instead of
the true hard limit kern.maxfilesperproc, as it is on FreeBSD 8.2
and 9.0. If they could fix that then setrlimit() would not have
this misleading behavior where it claims the limit is unbounded but
it is not.
Post by Axel Luttgens
BTW, I didn't immediately notice the long type for current_limit,
and thus got immediate failures: getsysctlbyname() returned values
such as 0x7FD500002800 -note the 0x00002800 half- and this proved
even worse than INT_MAX... ;-)
Yeah. It's "int" on 32-bit and 64-bit FreeBSD, too. So now we have
fixed open_limit.c.

Wietse

Axel Luttgens
2013-03-23 17:11:47 UTC
Permalink
Post by Wietse Venema
[...]
[...]
Please refer to the POSTSCREEN_README file.
Yes, of course; it was just the "turn off postscreen cache" part that made me hesitate a bit.
Post by Wietse Venema
With above settings, I'm able to speak to postcreen [...]
Excellent now you have to do it a couple thousand times in parallel
sessions.
I've tried variations around:

./smtp-source -s 50 -l 1024 -m 10000 -c -f luttgens@[127.0.0.1] -t luttgens@[127.0.0.1] 127.0.0.1:25

for a total of about 200K messages, without encountering any bugs, crashes or such amenities.

I would be tempted to believe that the kqueue variant is viable.


Axel
Wietse Venema
2013-03-22 00:21:57 UTC
Permalink
Post by Axel Luttgens
The change described in my other post ("Mac OS X and setrlimit(2)")
is required as well, since one will now call open_limit(INT_MAX)
instead of open_limit(FD_SETSIZE).
We don't use quick-and-dirty patches. Have you tried my (proper) fix?

Wietse
Axel Luttgens
2013-03-22 07:45:24 UTC
Permalink
Post by Wietse Venema
Post by Axel Luttgens
The change described in my other post ("Mac OS X and setrlimit(2)")
is required as well, since one will now call open_limit(INT_MAX)
instead of open_limit(FD_SETSIZE).
We don't use quick-and-dirty patches. Have you tried my (proper) fix?
No, not yet: I was of the intent to try it today.
Of course, I'll keep you informed.

Axel
Loading...