Greylisting with Exim and PostgreSQL

Greylisting is a technique to reduce spam by temporarily rejecting emails from unknown senders. The mail server is then expected to retry sending the email after a short delay. Since most spam is sent by machines that don't retry, this is an effective way to filter out a large portion of spam.

I've implemented greylisting for my mail server using Exim and PostgreSQL. Here's how it works:

How it works

When an email arrives, Exim checks if the combination of sender, recipient, and sending server has been seen before. If not, the email is temporarily rejected with a "try again later" response. If the combination has been seen before and enough time has passed, the email is accepted and the database is updated.

Implementation

The implementation uses a PostgreSQL database to store the greylisting information. A simple table stores the sender, recipient, sending server, and timestamp of the last attempt.

Configuration

To enable greylisting in Exim, you need to add an ACL rule that queries the database and decides whether to accept or reject the email. The rule should be placed in the DATA ACL.

Results

Since I've enabled greylisting on my mail server, the amount of spam has decreased significantly. Most spam never retries, so it's never delivered. Legitimate emails are still delivered, just with a slight delay on the first message from a new sender.

Tollef Fog Heen : Yahoo Breaking SMTP Standards

One of the reasons why I don't like Greylisting. In short, what greylisting is: when a server makes a connection to another server for mail delivery, a triple is formed from the sending host, destination address, and source address, and it is checked whether this combination is known. If not, the combination is noted and the current mail is rejected with a temporary rejection. The theory is that mail servers attempt redeliveries but spambots and virus distributors typically do not. So far, so good. Problems with this approach:

• not every mail server responds correctly to temporary rejections. Example: Yahoo. And that's far from the only server that reacts this way.
• even with temporary rejections, bounces often occur, which then cause mailing list hosts to unsubscribe you from lists.
• a spammer only needs to attempt to send the spam twice in quick succession and the spam gets through. This is minimal effort for spambots — either the user gets one or two spams — but they will get them.
• greylisting only works if you have control over all MX servers for your own domain, otherwise spam simply comes in through the other mail servers on which greylisting is not running.
• if all MXes use greylisting, delivery attempts of legitimate mail are slowed down, since these normally try the other MXes on temporary rejections and then also fail there. Depending on configuration, you then automatically end up in slower queues or longer waiting times on that server (because three delivery attempts have already failed at three MXes).
• Whitelisting (which is mentioned as a solution for some problems) is itself a problem: spam from servers on the whitelist is not detected. But precisely some of the large distribution servers have to be added to the whitelist because they have exactly the problems mentioned (Yahoo is not only a source for many mailing lists, but also for a lot of spam).
• Problems with greylisting are typically only noticed indirectly — since it is a largely transparent process and you can really only conclude that there are problems with greylisting from reactions by others.

All in all, greylisting only has an advantage temporarily: because it is rarely widespread, it is currently not taken into account by spambots. But taking it into account is trivial and would automatically happen with wider adoption. Thus greylisting is doomed to become ineffective if it spreads further.

Of course, many of the problems can be fixed. But ultimately, this is just as much an attempt to plug the holes in a sieve with paper as using rule-based spam filters against spam. Statistical spam filtering (Bayesian filter) is still the best available solution.

Here's the original article.

Gallery :: your photos on your website - Interesting software for photo albums on the web

Girls are Evil - Mathematical Proof

Here you can find the original article.

Omikron Basic 8.0 runs natively on Mac OS X

Yet another zombie that's still alive. When I read through the Features, tears of joy come to my eyes. Those were the days when you could program in Basic on home computers. The world was so much simpler then ... At The Macintosh News Network you can find the original article.

Photo Organizer - Feature-rich web photo album with a rather stylish default look

SCO vs. Linux: Mission impossible

First SCO stands up and says there are millions of stolen lines of code. And that they can name them. Then they demand sources. They get them. Search through them for ages and find nothing. Hello? Why do they even have to search if the locations are supposedly known? And why don't they notice that the JFS for Linux is based on the OS/2 JFS? That's even stated in the documentation - if they search the sources, why don't they read it at the same time? But probably that's exactly the problem: if you don't read text, you can search through it forever without ever finding anything.

At heise online news there's the original article.

Silverlab Partnerprogramm - Photo print service with financial participation of the photographer

Symantec Chief: Windows is not less secure than Linux

Sure, quite clearly. Windows is the easier target to hit, which is why it's not inherently less secure than Linux. And of course the security problems are due to attachment clickers - funny only that considerably more server attacks against Windows are possible, all of which have nothing to do with attachments. And all this despite the fact that with servers, Linux and Apache are definitely the train rolling through the whole city, while IIS - alongside IE and Outlook, the security hole par excellence - rather only runs in the seedier suburbs ... At heise online news there's the original article.

United States Patent: 6,727,830

What a load of rubbish: clicking an application button once is the standard function, holding it for at least one second is a secondary function, and double-clicking an application button is then a tertiary function.

And for such banality, the US Patent Office grants Microsoft a patent. And we're just introducing those oh-so-meaningful and innovative software patents in the EU. Thanks, Ms. Zypries, for letting us deal with such brilliant innovative solutions and such meaningful patents in the future.

Here's the original article.

EditThisPagePHP - Edit pages online in PHP - Alternative for situations where a real CMS is too large and a wiki or weblog is too rigid in structure

SCO vs. Linux: Investor Baystar exits

Final beginning of the preliminary end?

At heise online news there's the original article.

SF Author Bradbury: "Michael Moore is a terrible person" - Culture - SPIEGEL ONLINE

Bradbury certainly had a bit of a dark side. Instead of being pleased that someone like Moore was basing a film title on one of his book titles, he complained that he should have been asked. What more does he want? Should thermometer manufacturers with Fahrenheit scales also ask him for permission? Rarely a more foolish man than Bradbury...

Here's the original article.

Vellum: a weblogging system in Python - Nice little weblogging system in Python

Adventure Earth - To Hell

Just stumbled upon it while flipping through channels. Absolutely well done! And interesting - the giant Andean vulture with an 8-meter wingspan (extinct) was just as new to me as the fact that New World vultures (including the condor) are related to storks and not to birds of prey, like Old World vultures are.

Here's the original article.

drbs - Distributed Replicated Blob Server - Server system modeled after Google File System

GDL - GNU Data Language - Another one for the number crunchers

Maypole / Apache::MVC - Perl framework for creating web applications

mnot's Web log: Ubiquitious Fragment Identifiers - Interesting fragment link solution with CSS

paramiko: ssh2 protocol for python - SSH2 protocol implementation in Python

PYSH: A Python Shell - Shell that uses Python as a shell language

SoftPear - PC/Mac Interoperability

Wow - now they've got a recompiler for machine code in there too. That sounds increasingly interesting - a recompiler is the most important step for usable performance for such systems.

Here's the original article.

Sony's Clié Says Goodbye

Was obvious. After all, I bought one myself and I'm quite happy with it. Whenever I get something like this, the manufacturer goes bankrupt or discontinues the product line or does something else crazy. I'm starting to feel like the rain god in So Long, and Thanks for All the Fish. At heise online news you can find the original article.

Sun Insists that Red Hat Linux is Proprietary

Just to show that the IT world has more crazy people than just the SCO boss. The SUN boss's loss of touch with reality is also quite remarkable.

Here's the original article.

Telekom wants to collect customer data from internet users

They must be out of their minds. What a mess - since when can providers decide which of my email addresses are allowed to be published? The whole thing is absolutely ridiculous!

At heise online news there's the original article.

Acratech, Inc: Precision Machining & Photographic Equipment - Manufacturer and marketer of the Ultimate Ballhead

Curdled (1996)

Occasionally, real gems are shown on television

Here is the original article.

Jan Ullrich in the Tour of Germany

In the time trial, he clearly showed that he really is back to full strength. The fact that Michael Rich took 24 seconds from him in the end is not a big deal - you can afford to lose in a time trial against someone like Michael Rich, the man is simply an exceptional rider. So a really great race overall. But absolutely top-notch was the performance of the Gerolsteiner team: getting 4 riders into the top 8 positions is quite something. Not many people can match that.

However, the ARD should learn to use their microphones properly. That was total chaos at the end.

KODAK PROFESSIONAL READYLOAD Single Sheet Packs and Holder - 4x5 inch single sheet film cartridge system for easy sheet film use

Lycos - web hosting

Great terms they have there: Copyrighted or illegal material may not be stored and may not be offered for download. Under German law, everything someone writes is automatically protected by copyright, if I remember correctly. So I wouldn't be allowed to publish my own content that I write myself if I host it there, since this own content is protected by copyright.

Hier gibts den Originalartikel

Mark Lentczner's Journal

Ouch. A periodic table of Perl operators. Could it be that someone went a little too far with the definition of possible operators in Perl? Just a tiny bit? The original article is here.

Syndication formats cause of progressive dementia?

Who wants to know what a typical civilized discussion about syndication formats looks like should just follow the link. Surprisingly, Dave Winer behaves very orderly in that thread. The same cannot be said for Mark Pilgrim. Quite amusing - ultimately, what I suspected back then comes to pass: if it's good for nothing else, the Atom format at least makes for a great network psychology experiment. On the technical side: Atom is poorly designed. The louder Pilgrim and his Pilgrim-Fathers-of-Atom-Format shout, the more embarrassing the whole affair becomes. During the discussion about the Atom API, criticism was frequently expressed that PUT and DELETE as HTTP verbs were unusual and often not supported. The comment on that was that these people should just throw their toolkits away if they couldn't handle standard techniques. At the same time, these same people specially invented their own authentication scheme for HTTP just for Atom, which merely reinvents the technique of Digest Auth under a different name and with different syntax - arguing that many toolkits and server technologies don't support Digest Auth. Yes, that's right, the exact same argument was used by Pilgrim and company in two opposite directions. So much for consistency of argumentation.

Another example: there's constant harping on how consistent the Atom format is with respect to tags. Curiously though, while all links in the format are mapped via the Link tag (and specified with corresponding rel attributes), they define three different tags just for date specifications - even though a single Date tag with rel attribute for the type of date would be far more logical in this context.

Also amusing was the discussion about the type of API - many wanted an XMLRPC API, simply because RPC integrates well into programming. What prevailed was the document faction, who prefer an API with REST structure (because documents are natively managed there via GET/POST/PUT/DELETE). Fair enough - I can accept that. But embarrassing was the manner in which various REST proponents tried to argue why XMLRPC wouldn't work. Which is rather silly given the widespread use of XMLRPC for all kinds of purposes. And for someone familiar with RPC-style APIs, the whole discussion was more of a staircase joke than a serious technical discussion. How old is RPC as a programming technique in the Unix environment? 20 years? But of course that's all just imagination...

Well, what can you expect from people who take the fact that Googlegroups and Blogger all forcibly received Atom feeds as the basis for claiming that Atom is already more widespread than RSS today? Now can you understand why it's really no pleasure to deal with content syndication? Only psychopaths and cranks in that field, hardly a mentally normal person to be found. Can someone now explain to me why I programmed my own aggregator for the Python Desktop Server? Here's the original article.

Taxi (1998)

Very casual

Here you can find the original article.

The Contiki Operating System - System for computers with limited memory

Web Development Bookmarklets

Lots of nice bookmarklets (small JavaScript snippets that can be triggered via bookmarks) for web developers. The named anchor one is particularly handy - it makes anchors in text visible so you can link to them. Without this bookmarklet you're left digging through the source code (or possibly CSS hacks). Here's the original article.

79th Hainleite Round Tour: Wrolich wins - Ullrich fifth

Hmm. 13 seconds behind the leader is already a good sign. But let's wait and see how he does at the Deutschland Tour. The Hainleite circuit race may be a demanding race - but still, it's only a one-day race.

I found the original article at Radsport-News.com.

Gerolsteiner extends sponsor contract until 2008

Very nice. Of course, I'm a bit biased: the winner of the mountain classification in the Giro (ok, he still has to finish within the time limit today, but he should manage that) comes from Münster.

With all the sponsors that have dropped out and are dropping out, it's nice when a sponsor clearly stands by the sport.

At Radsport-News.com I found the original article.

Camera Memory Card for 12500 Euros

I find the price then - given the fact that a 1 GB microdrive can be obtained for a maximum of 150 euros, which for 12 GB would only be 1500 euros - a little bit overpriced

At heise online news you can find the original article.

Old Henrichenburg Ship Lift

Today we visited the Henrichenburg ship lift in Waltrop, near Datteln. It's only about an hour's drive from Münster - and it's absolutely worth it! For one thing, of course, there's the old ship lift from 1899. Then there are all the outdoor facilities - at the upper water level there are complete canal port facilities. And finally, there's also the museum in the boiler house, which houses a complete steam engine that still functions. In addition, there's plenty more to see all around: the motor boat harbors, the old lock system from the same time as the ship lift, and of course the new ship lift as well. All of it can be visited as part of the Westphalian Industrial Museum. However, you should bring more time than we had - you can easily spend an afternoon there seeing everything.

Here's the original article.

Steam Engine

Steam Engine

The steam engine stands in the boiler house of the Henrichenburg ship lift.

Papachristos

Papachristos

Tower of the Papachristos restaurant near the Henrichenburg ship lift.

Henrichenburg Boat Lift

Schiffshebewerk Henrichenburg

The ship lift in Henrichenburg - construction year 1899!

0190 operator warns Dialerschutz.de

Where we're on the subject of lawyers and their dubious dealings. It's not a violation of anything if a lawyer lends his name as Admin-C for something like malvorlagen.de. But of course it is a violation of the Legal Advice Act if one offers forums in which those affected by precisely this offer - as it exists on malvorlagen.de, for example - can exchange information. The lawyer certainly finds all of this quite normal. Does anyone still wonder why lawyers have such a terrible reputation?

At heise online news there's the original article.

Allergies

Little known, according to Huffnagle, is that the gut is also involved in immune defense against respiratory diseases. Pathogens arriving in the airways are transported by the cilia of the bronchial mucosa toward the pharynx and swallowed from there. This way they reach the gut, where the immune system recognizes them. Defense cells are formed that also reach the lungs via the blood, where they help fight the infection. Very interesting article. For someone with chronic allergies, finally a glimmer of hope - because previous allergy medications are either chemical sledgehammers or symptom suppressants. But one would have to get at the real causes of allergies; then one could finally get rid of these annoying problems.

At passe.par.tout I found the original article.

The Caliph of Cologne, Public Enemy No. 1!

I'd much rather deport hate preachers like Beckstein - but what country would want them... At das Netzbuch you can find the original article.

Jim Jarmusch Again

I already revealed (P2235) that I'm a Jim Jarmusch fan. But today I finally put the Ghost Dog DVD in the player. Simply genius. I had only seen it once on television before. Just fantastic. How those old mafiosi discuss rappers, Native Americans and their names - hilarious absurd dialogues. And then there's the ice cream vendor with the speech problem. How can anyone come up with such crazy characters? Brilliant.

Justice Minister Defends Software Patents

What stupid drivel from Zypries. Show me the open source project or small company that can afford patent proceedings - let alone the later enforcement if one of the large corporations grabs the subject. Conversely, large corporations will use the patents to block others. That won't create jobs. Except perhaps for patent lawyers ...

At heise online news there's the original article.

Survey: Germany Popular Among Top Managers

Of course. Where else can you achieve such high results with such cheap lies as here (wild rationalizations, idiotic high salaries and million-dollar severance packages for failed executives) ...

At tagesschau.de - Die Nachrichten der ARD you can find the original article.

=F6 über Debian ...

Well. What do I expect from a distribution? And why do I use Debian in particular - and have for years? Probably it's different expectations that's why I'm so satisfied with Debian.

A distribution must realize the base system for me - this must be stable (which is why I almost always use Debian Stable), but should be easily expandable (which is why I use backports from Unstable or Testing at selected points).

The distribution must make updating the base system simple - a base system consists of a bunch of components, all of which can have some vulnerabilities. I have no desire to deal with these potential holes - that's the job of the distribution. Debian makes this almost trivial through apt. I want to be able to see what an upgrade means - so I can decide whether to do it or not. Debian provides the tools for this (e.g., automatic display of changelogs and critical bugs before installing a package). The distribution must allow me at defined points and with simple means to break out of the normal distribution. Every binary distribution has the same problem: package maintainers decide how programs should be configured. This often works well - occasionally it goes extremely wrong. Therefore, a binary distribution must allow me to compile the packages myself if necessary. With Debian, the build structure for packages is very simple. Adapting packages, backporting packages from Unstable or Testing (to get newer versions than in Stable), and creating your own packages is easy. I'm not forced into the Stable corset - but I can still stay in Stable for the base system to take advantage of Debian's good security infrastructure. The fact that it's additionally trivial to distribute your own packages to many machines by setting up your own package repository and including it alongside the standard repositories is not just nice to have - it's essential with a sufficiently large number of machines. A distribution must have functioning package dependencies and actually use them. Consistently. I have no desire to start a program and then get strange messages just because some libraries or other tools are missing. Sure, other distributions also have dependencies - but sometimes they're optional or only used very shallowly. Debian is consistent and goes very deep - everything is built on dependencies. This means you can be relatively sure that dependencies are met when you install a package normally. If not, that's a clear bug and can be reported via bug reporting - and will be fixed. Dependencies are not nice to have, they're essential. Period. Of course, a distribution must also allow breaking out of the corset with dependencies. Debian has several nice utilities for this that let you resolve dependencies - e.g., pseudo-packages that simply say a particular package is installed. This package can certainly be installed manually. A distribution must know what config files are. That means it must under no circumstances trample on my config files. If a distribution overwrites my configs on update and I get comments like make backups of them first, the distribution is out. Sorry, but I have absolutely no tolerance for that. A distribution may only change a configuration under clearly defined circumstances. And no, I have little sympathy for Debian's debconf either - if a package upgrade shreds my configs, it rains bug reports. Config files belong to me, not the distribution. Period. A distribution should damn well not try to solve all the world's problems. And especially should not try to be smarter than the original programmers of a package. If a program has a structure of config files, then it should at least optionally be usable without problems with the distribution. And that also means the distribution doesn't trample on it just because it thinks it has a better tool for it. Besides, all configuration tools stink to high heaven.

What I'm not particularly keen on: always having the very latest packages. Sorry folks, but that's the stupid update-itis that spreads in the Windows world. Always having to have the latest. Such nonsense. Apache 1.3 does its job well, you don't even need the latest 1.3 - as long as security patches have been backported. And that's what Debian does. Security patches for Stable don't simply update silently to a new version with new, unknown problems. Instead, the patch is - if possible - backported to the old version and made available via that. Security updates should only under absolute exceptional circumstances require configuration changes from the admin or alter system components, which leads to potential problems. I want a smoothly running system before and after the update!

I'm also not particularly keen on nice graphical or text-based configuration or administration tools. Sorry, but the ideal tool for these purposes is called vim and the perfect data format is text files. And yes, I can't particularly stand debconf - fortunately you can simply work around it where it's annoying - and Debian keeps its hands off the existing standard configurations, even if a package normally uses debconf. If not, that's a bug.

But I do expect a certain transparency from a distribution in what it does. I don't like one-man shows that you can't see into - where someone autocratically decides what's good or right. Or perhaps a few. I want to be able to look into everything - because the process of distribution creation can also have bugs that are essential for me. Therefore, I'm also not keen on a company building a distribution. Sorry, but sooner or later come the nice profit-maximization strategies à la RedHat Enterprise or comparable Suse approaches. If a distribution changes the standard mailer, I want to see the discussion about why it was changed - with the pro and con arguments. I want to be able to understand why something develops the way it does. I want to be warned in advance. Of course, I'm not interested in this for every package - but for the essential ones that interest me, I want this information. Transparency is important - it starts with transparent bug tracking and ends with a transparent project structure. If I had no interest in transparency, I could just as well install Solaris. Or Windows. I have no problem with: a learning curve in using the system. System administration is a job. A job requires learning. Anyone not willing to learn should stay away from the job. Arguments like I first have to understand how the system works don't count. There are plenty of documentation and good books on Debian as a starting point. Read. Learn. That's just part of it. No colorful tools and no grandiose promises from manufacturers about the easiest-to-install Linux distribution help either - it's all bullshit. When push comes to shove, you have to master the system from the kernel to the dotfile. And you have to learn that anyway, no matter what the system is called. Learning a distribution and how it works is an investment for years. Therefore, I also don't want to see my investment go down the drain just because the system was suddenly rebuilt because it appeals to the manufacturer or because it's cooler or because it sells better or because another buzzword is fulfilled. Distributions need evolution, not revolution.

Debian is not the perfect Linux distribution - no such thing exists. But Debian is damn close.

At Die wunderbare Welt von Isotopp you can find the original article.

ish owners approve sale to Kabel Deutschland

How was that again? Privatization brings lower prices and better offerings through competition? Oh yeah. Absolutely clear.

Of course we must keep privatizing everything we can. There are still many new monopolies to create...

At heise online news there's the original article.