ronny.haryan.to

The Apache httpd that comes with Mac OS X Leopard runs in 64-bit mode, so everything else that’s loaded, including mod_wsgi and psycopg2 must be able to run in 64-bit mode as well. Graham Dumpleton explained this behaviour in a post to the django-users mailing list.

Compiling psycopg2 using MacPorts as well as from the tarball would result in something like this when loaded from a web application that runs on Apache:

ImproperlyConfigured: Error loading psycopg2 module: dlopen(/Library/Python/2.5/site-packages/psycopg2/_psycopg.so, 2): no suitable image found.  Did find: /Library/Python/2.5/site-packages/psycopg2/_psycopg.so: no matching architecture in universal wrapper

So all I needed to do was force psycopg2 extension to be built for 64-bit as well as 32-bit. After extracting the latest psycopg2 source, do this instead:

LDFLAGS="-arch ppc -arch i386 -arch x86_64" CFLAGS="-arch ppc -arch i386 -arch x86_64" python setup.py build

Then sudo python setup.py install. After that, check the .so file, it should look something like this:

$ file /Library/Python/2.5/site-packages/psycopg2/_psycopg.so 
/Library/Python/2.5/site-packages/psycopg2/_psycopg.so: Mach-O universal binary with 3 architectures
/Library/Python/2.5/site-packages/psycopg2/_psycopg.so (for architecture i386): Mach-O bundle i386
/Library/Python/2.5/site-packages/psycopg2/_psycopg.so (for architecture ppc7400):      Mach-O bundle ppc
/Library/Python/2.5/site-packages/psycopg2/_psycopg.so (for architecture x86_64):       Mach-O 64-bit bundle x86_64

Don’t forget to restart Apache, and that should be it.

Update: It happened to PIL too, it was giving this error: The _imaging C module is not installed, so I had to do the same thing as above: LDFLAGS=... CFLAGS=... python setup.py build followed by sudo python setup.py install.

At work I manage several servers that are mainly used to serve websites built using the Mambo content management system. Personally I don’t follow the development of Mambo, but I know that it had/has several security vulnerabilities. One of them was pretty easy to exploit. To make matter worse, Mambo has a distinct URL structure that makes it really easy to find Mambo sites using search engines. This combination attracts a lot of script kiddies from all over the world. I’ve seen several successful attempts of these idiots putting IRC DCC bots and spamming activities coming from Nigeria, Brazil, Indonesia (Jogja), Russia, Italy, etc.

Sometimes the problem is not from Mambo itself, but from poorly written third-party components that don’t give any shit about security (why don’t you write a big “hack me” sign on the front page while you’re chmod 777-ing all your files?). But as far as I know, Mambo doesn’t seem to have any built-in protection from poorly written third-party components. I’m by no means a Mambo expert, so I could be wrong. And please do correct me if I’m wrong.

I’ve had enough of these kiddie idiots running around thinking they’re cool hackers. The thing is I don’t have time to investigate why Mambo or its third-party components that were installed was broken. Luckily, there is a pattern in all of the incidents: they all exploit mosConfig_absolute_path remote script execution (thanks to PHP’s ability to include()/require() files over HTTP). In many cases I could disable allow_url_include and/or allow_url_fopen in php.ini, but in certain cases it’s needed, so I couldn’t disable it. So I wrote a quick fix using Apache’s mod_rewrite and put it in a .htaccess file:

RewriteEngine On
RewriteCond %{QUERY_STRING} mosConfig_absolute_path= [NC]
RewriteRule ^.*$ http://%{SERVER_NAME}/? [R]

This will redirect all exploit attempts to the front page, so adjust to your environment. You can change mosConfig_absolute_path= to just mosConfig if you want to be really anal. As I said, I’m not a Mambo expert, so I don’t know if something like this can or should be included in the default .htaccess that comes with Mambo. People more familiar with Mambo can figure that out. In the meantime, this small hack should deter the kiddies.

May God forgive all the spammers’ souls.

Meanwhile, I use this procmail recipe:

:0B
* ^Symbol: +[A-Z][A-Z][A-Z][A-Z]
* ^Current Price: +\$[0-9]\.
* Term Target( Price)?: +\$
/dev/null

This one recipe alone reduces my daily spam by about half! I get about 500 to 1,000 spam emails daily. This is the only reason I reactivated my procmail. My Bayesian filter, which has been working wonderfully for years now, caught all the penny stock spam just fine, it’s just that it’s really annoying and harder for me to go through these penny stock spam when checking my spam folder for false positives.

Here are some MTA (mail server) tricks that I use at work for testing an email application that we’re working on.

I wanted to do a rough benchmarking on how fast our email app can pump messages to the MTA. Of course I don’t want to actually send the emails, I don’t care where it’s sent to or what the content is, the MTA should just accept then drop them straight away.

I’m using Postfix. So I have the email app send 50,000 emails to x@blackhole.localdomain, where x is a number between 1 and 50,000. You can also do this easily with smtp-source program that comes with Postfix. In /etc/hosts I added 127.0.0.1 blackhole.localdomain, and then I set $transport_maps to hash:/etc/postfix/transport and in /etc/postfix/transport I added blackhole.localdomain blackhole: and ran postmap /etc/postfix/transport. This will make everything sent to blackhole.localdomain to be sent via the ‘blackhole’ transport, which we need to define in master.cf as follows:

blackhole unix  -       n       n       -       -       pipe
    user=nobody argv=/bin/true ${user}

The mail logs should tell us whether it’s working or not.

Another thing that sometimes I wanted to do is to capture the actual emails sent after it reaches the MTA but without the MTA actually sending the mail to the recipients. This is to get a better picture of how the emails will look like to the recipients. This is useful, for example, to verify that headers, newlines, charset, encoding, MIME, etc. are generated correctly. With Postfix, I used content_filter and did something like this in master.cf.

pickup    fifo  n       -       -       60      1       pickup
    -o content_filter=dbgcapture
20025     inet  n       -       -       -       -       smtpd
    -o content_filter=dbgcapture
20026     inet  n       -       -       -       -       smtpd
    -o debug_peer_list=127.0.0.1,192.168.1.0/24
    -o content_filter=dbgcapture
dbgcapture unix  -       n       n       -       -       pipe
  flags=FR user=ronny argv=/usr/bin/tee -a /tmp/capture.mbox

Mails sent via SMTP to $inet_interfaces:20025 and sent via /usr/sbin/sendmail will be captured in /tmp/capture.mbox. Similarly, port 20026, with additional SMTP debugging in the logs. Port 25 still works as usual. If you don’t want /usr/sbin/sendmail to go to dbgcapture simply comment out the -o content_filter line right after the pickup line. Note that using /usr/bin/tee here may not be the most efficient/correct. It works most of the time, but it’s asynchronous and it doesn’t guarantee all mails will be written in the correct order (or at all) in the mbox. If you can suggest a better way with similar level of ease, please let me know.

I was also looking at capturing mails with Sendmail. It’s possible using MIMEDefang milter with action_quarantine_whole_email() followed by action_discard(). It works but somehow it ran slower than a snail. I’m not that familiar with Sendmail. Meh. No big deal. I left the tests running, I went home, and check the test results the next morning.

In the next few months we will be doing a lot more of this kind of tests, so I’m planning to do similar thing with qmail and Exim to cover a lot more base.

Mutt has a little-known feature to detect duplicate messages. I’m quite sure it detects duplicates by their Message-ID header, but it could be from something else in addition to that. You can do a pattern match duplicate messages in a mailbox and do whatever with them, e.g. delete or move them somewhere else.

Let say you want to delete all duplicate messages in the current mailbox. Just do a tag-pattern (bound to T here), put in ~= as the pattern, then all duplicates will be tagged. After that, you can delete tagged messages (bound to ;d, or just d if you have $auto_tag=yes).

Duplicate messages are also indicated by = in the thread if you sort the messages by thread and you have $duplicates_thread=yes (it’s yes by default).