programmierung - 2.1.2006 - 18.2.2006

HOC: A Haskell to Objective-C Binding - even for Haskell there is an Objective-C Bridge that I didn't know about.

Management by Stupidity or by Corruption?

How is the ALGII software actually doing?

The subsidiary of Deutsche Telekom has, however, by now realized that the existing solution is "no longer repairable." There are simply too many fundamental errors in the software's architecture.

Ok, so far, so bad. And what is the Federal Employment Agency doing? Quite simple:

The series of mishaps with the unemployment software A2LL has, according to information from Netzeitung, prompted the Federal Employment Agency (BA) to commission T-Systems with the programming of a completely new software. "T-Systems is working on a fundamental solution," it was said in circles familiar with the situation. The creation of a new program is taking place "within the framework of the existing contract." The BA did not want to comment on request from Netzeitung.

Translated: someone has massively messed up a software, admits themselves that it is no good and gets the order for a new software within existing contracts (i.e. without tender!). This is how our tax money and unemployment insurance contributions are wasted. And the reason?

The crucial point for the BA is the compatibility with A2LL. "It must be possible to easily take over all eight million data records," it was said. For this reason, the authority has also decided to commission T-Systems again with the programming. It is important to have T-Systems "on board," even if the company is partly responsible for the A2LL fiasco. The BA has explicitly decided against the decentralized solution of the company Prosoz.

Please what? There was an alternative solution offered. But the loser from last time was used again to make the data records transferable? Who guarantees this? T-Systems has already proven its inability - why does anyone believe that they can correctly transfer the data if they cannot process it correctly?

Data transfers are really not tied to individuals or companies - instead of making the goat the gardener again, T-Systems should have been obliged to document and disclose all interfaces, data formats, and structures. And then make a tender based on this - and simply define compatibility with the old data base as a condition. This time, please with severe penalty clauses for non-fulfillment.

Either someone in the project management at the BA is completely incompetent, or completely corrupt. I can't think of any other explanation. When you consider the debacle with the online system, the whole thing becomes clear - with the money that was squandered there, one could have helped some unemployed people well over the winter.

pyOpenSSL - Python interface to the OpenSSL library - quite complete bindings. Looks much better than the previous libs I have looked at.

Statistical programming with R

I had already seen the first part (environment and basic structure) of "Statistical programming with R" earlier. Meanwhile, part 2 (functional programming and data analysis) and part 3 (object-oriented programming) are also online. Exciting for number crunchers.

Yahoo! Design Pattern Library - A collection of standard patterns in web GUI applications and how to solve them with the Yahoo JS library. Very interesting as a cookbook.

Yahoo! UI Library - the JS/Ajax lib used by Yahoo for its own applications. BSD license!

Language Design Is Not Just Solving Puzzles

Language Design Is Not Just Solving Puzzles is a rather interesting article by Guido van Rossum about the impossibility of an elegant syntax for multi-line lambdas in Python. Worth reading, and in large parts I agree with him. However, I then stumble over such a last paragraph:

And there's the rub: there's no way to make a Rube Goldberg language feature appear simple. Features of a programming language, whether syntactic or semantic, are all part of the language's user interface. And a user interface can handle only so much complexity or it becomes unusable. This is also the reason why Python will never have continuations, and even why I'm uninterested in optimizing tail recursion. But that's for another installment.

I am quite willing to accept that continuations are complex - but not because of the interface. For the interface for continuations, you only need the callcc call to bind the continuation and a simple function syntax to trigger the continuation. The main problem with continuations lies in the cooperation with generators and exceptions - what happens when a continuation is triggered within a generator? What happens when an exception is triggered within a continuation? These are the difficult aspects - which, by the way, also make Scheme implementers sweat, which is why exceptions are not particularly popular there (the same problem, just viewed from the other direction).

So okay, no continuations in Python - even though we already have poor-man's continuations with pickable generators (or with greenlets, or with cloneable coroutines, or one of the many other approaches to obtain subsets of continuation features).

But what on earth is complex about tail-call optimization (because it's not just about tail recursion)? It is so primitive that it can be implemented transparently for the programmer - if a tail call is present, do not note a return address on the stack, but reload the parameters in the stack frame and note a simple jump. If you want to be nice, you can introduce a pseudo-function "tailcall" that throws an exception if it is not to be executed in a tail call position. There may be further conditions under which tail calls cannot be optimized - but these can also be incorporated into a corresponding check.

It is precisely the function overhead that makes some algorithms only awkwardly implementable in scripting languages. And tail-call optimization would definitely help here. Especially in situations where you have a chain of small function calls. As far as I'm concerned, it can also be an optimization that is only activated at -O (or -O2 or something else).

Django Templates are not limited

shannon -jj behrens thinks that Django template language is limited - because it doesn't have functions with parameters to do html snippet reuse. Of course the official - and simplified - answer to this is, that Djangos template language is that simple by design, so that it can easily be learned by non-programmers (as often designers aren't necessarily programmers). This is a quite good reasoning, but I think it's a bit too simplified.

So here is the longer - more complete - answer to this accusition: the Django template language isn't limited at all. Yes, I know that the "include" and "block" tags aren't parameterizable and so aren't often that useful for more complex situations (at least if you don't want to end in namespace hell due to passing some template-globals in the context).

So what should you do if you notice that your templates would need more complex code? One way would be to precompute the data in the view function and pass it on via the context to the template - that way the template has the ready data and can directly present it.

But what to do if you can't precompute, because you are using generic views? You could wrap your generic view with your own code and call the original generic view in that function with the modified context. That way you have the same benefit as above - youre templates have the data readily available. If you have many view functions that all need the same context enrichment, you can write your wrapper as a decorator - and just decorate the generic views and use those decorated functions in your urlpatterns.

But what if even wrapping isn't the answer? Shouldn't there be some way to do more complex code without all that wrapping? Sure there is! The answer are custom template tags. This might sound like a bit of overkill, but believe me, writing some template tags isn't really that hard. There is documentation on using and extending the template system in python

An even easier way to write your own tags is to use the "simple_tag" or "inclusion_tag" helpers in django.template.Library. Those functions allow to build simple tags very easily - the inclusion tag will base it's output on some template snippet, so you can see it as a template function with paramerters. A lot of usage of custom templates is in the contrib/admin stuff.

The main problem with the newer stuff in the code is, there is documentation missing for it. Hopefully that will be solved over time. But please, if the next time someone tries to tell you that the Django Template Language is to primitive, don't believe him. The Django Template Language is easy to grasp for non-programmers - but it's very extensible for Python programmers. And you extend it in the language you like - in Python.

Hedgehog - Never underestimate the power of a small tactical Lisp interpreter.

GREYCSTORATION - denoising algorithm as an open source CLI tool.

Lightbox JS - Display photos with JS on the page. Nice effect.

lambda remains in Python

Let's just keep lambda - GvR gives up

Mandelbrot Set - Labix - Example source code that draws little apples with PyGame on the Nokia Tablet.

Eiffel For OS X - what you see is what you get. Only this terrible background graphic on the site ...

Just-In-Time Scheme

plt-scheme gets a JIT Compiler - which should provide a significant boost for DrScheme, the best Scheme in the world. So far, it has been a purely interpreted system with its own virtual machine - and that was already damn fast. But a JIT compiler can bring a lot, especially for larger string mountains or list gobblers and number tangles. It will be exciting to see how this compares to e.g. Gambit-C and Chicken.

Guido van Rossum and Web Frameworks

Guido van Rossum asks about web frameworks - nothing exciting in itself. He just hasn't done anything with them before and wants to inform himself. He makes some claims that aren't quite accurate (e.g., that Django's template language is similar to PHP), but given the likely brevity of his "looking into" it, it's forgivable.

It gets funny in the comments on his post. Mountains of frameworks, all of which aren't finished. Piles of comments like "take XYZ, it's great and in the next few months it will definitely be usable" - especially often TurboGears is suggested.

Sorry, what? If I'm looking for a web framework, I don't want one that will be usable in a few months. I want one that is usable now and for which there are clear statements about its fitness right now. We really don't need any more web frameworks that won't be finished.

I don't have anything against a variety of frameworks - it makes life exciting and interesting because you never know if you've bet on the right framework - but there are certainly more than enough unfinished frameworks that are pitched by their users as if they were the best thing since sliced bread.

By the way, I use Django for exactly these reasons: the stuff has been in use for quite some time and has proven that it is suitable for large sites and high loads. It was developed from real applications and is not the byproduct of some unimportant Web2.0 thing of which I have never heard outside the TurboGears clique. It was also not cobbled together by a kid alone who thinks he's the new Einstein and believes he's the only one who knows how frameworks should be. And it's not a project that has been dead in principle for over a year because the author has long since moved on to something else. And it's only called 0.9 at the moment because API changes and cleanup work are pending in the guts (which would be appropriate for any project that has been developed for two years in live operation) - not because it's only 90% finished.

Of course, after this Artima post, everyone will look at GvR and wait to see what he chooses. And of course, all the web framework authors will jump up and down and want to make themselves noticed. And of course, every word will be analyzed and rubbed in the other's face. And a whole series of projects will make short-term quick-fix changes because they hope GvR will choose their framework. All of which is a really insane waste of time. Sometimes these kids in the OSS projects really get on my nerves.

PythonForMaemo - Python for Maemo - Using Python on the Nokia 770 Tablet (the shipping confirmation arrived today, hopefully the device will arrive soon).

twill: a simple scripting language for Web browsing - a Python scriptable Web client. Interesting for automated page requests and for specialized robots. Possibly also for testing web applications.

pyvm home - another Python implementation. Its own bytecode interpreter and a Python compiler written in Python. Sounds almost like PyPy meets Parrot - though retaining the Python bytecode.

APRESS.COM: Practical Common Lisp - now also available as a free PDF download (go to the Free Download page and download it there).

Bill Clementson's Blog: Update on Termite (A Lisp for Concurrent/Parallel Programming) - Information about Termite, a Scheme based on Gambit-C with the concurrency features of Erlang. Sounds very interesting, check it out when the code is released.

Thinking Forth - now also available as a PDF download. For old Forthers like me, of course a must-download.

A list of open-source HTTP proxies written in python - many of them are still active, and potentially quite interesting for mobile use (especially those based on asyncore, due to the low resource usage)

WebCleaner - a filtering HTTP proxy - absolute high-tech, that part. Asyncore, so low resources, built-in JavaScript interpreter against obfuscation and a bunch of other features. Sounds very good on paper.

Hurring.com: Code: Python: PHP Serialize implemented in Python - Deserialize PHP data in Python. Could be interesting if I want to migrate further sites from PHP to Django and, for example, access Wordpress settings.

ScriptAculoUs - MochiKit - Trac - a port of the script.aculo.us effects to MochiKit. Finally, drag and drop and other effects with the significantly better technical MochiKit base.

To all Oberonistas

The Wirthians and Oberonistas. Run, no, sprint and get the PDF version of Project Oberon. Hey, I know it's not exactly state-of-the-art anymore and some aspects of the Oberon system were simply silly (especially its reasoning for non-overlapping windows in the windowing system), but nevertheless the book is absolutely worth reading. And the presented system still has a lot of charm, even if it has largely disappeared into obscurity.

Ancient Languages: Perl - already old, but simply great. A bitter account with Larry Wall and Perl.

Django Paste - Ian is starting to integrate Django with paste (and paste deploy). I for one will most definitely try to support that, so his list of related tickets is already down by one. Paste deploy might even be taken as the future default FCGI/SCGI solution - because it uses the same FLUP lib, it is as capable as my scripts, but due to the structure of Paste, installation should be much easier (and might even be standard in the future with Python hosters).

Apples Photocast Format

Apple now supports PhotoCasting - and Dave Winer comments on the format. In this case, I absolutely agree with him: the format is complete crap. On the one hand, they've invented something new instead of simply using Enclosures (which in RSS are exactly there for such purposes), on the other hand, the feed is also completely broken from beginning to end. What is this nonsense?

MoinMoin Release 1.5 - wow, the new MoinMoin looks really slick.

Microsoft retains FAT patents

Great, the US Patent Office confirmed Microsoft's patent on the FAT file system:

As part of its review (Re-Examination), the Patent Office initially declared the patents provisionally invalid (Non-Final-Ruling) due to "Prior Art". The decision now made, however, is final and Microsoft receives a so-called "Patent Re-Examination Certificate" from the USPTO for both patents. The Patent Office finally determined that the FAT file system had been a new development and therefore patentable, Microsoft further announced.

We can therefore wait and see when Microsoft will use the patent to take action against open source software that uses or supports FAT file systems.

ProGraph for OS X

Does anyone remember the Dataflow-Language ProGraph? The one where you just wired boxes together but didn't write normal code? The one where Spaghetti-Code still consisted of real twisted bands? There was once a version for the old, classic Mac OS.

Now there is a version from Andescotia Software for Mac OS X. At 60 dollars maybe not exactly a bargain for an obscure piece of programming language history, but somehow I still like it. Let's see if I have money for it.

If you want to see ProGraph in action, there is a quite usable tutorial online. And of course, there is a bit about the history and development on Wikipedia.

Addendum: of course I have the money. Was to be expected. A bit slow, the download, but it runs.

The open-source approaches are, by the way, quite emaciated so far - and on the websites to a larger extent consist of complaining, whining, and confusion.

Efficient Editing With vim - Jonathan McPherson - nice reference for more advanced key commands in vi. I don't even use all of them - I should get used to one or the other.

Free Alternative to Flash?

Gnash is a GNU project to implement a Flash client under GPL. Very interesting thing - especially interesting will be how the reaction from Adobe will look like. I wouldn't mind a bit of diversity in that area, even if I'm not usually such a Flash fan.

nadamac CamiScript - Script Repository - useful scripts for Camino, for use in CamiScript

Gambit Scheme - a new Beta

From the third best Scheme in the world - Gambit Scheme System - there is a new beta for version 4.0. Particularly interesting about Gambit-C is - besides the high performance of the code - the truly brilliant threading implementation. Normal Scheme continuations are used and then a dispatcher is placed on top. As a result, threads under Gambit-C hardly need any memory (on a 2G machine you can easily run over a million threads) and resources (and yes, the switching with a million threads is also quite decent). As a result, Gambit-C is the absolute number 1 for massive multithreading on single-processor systems - and web servers love many threads.

And before anyone asks: the best Scheme is of course PLT Scheme (Dr. Scheme) and the second best is Chicken Scheme - because Chicken Scheme still has the most impressive library of included code after PLT Scheme. Gambit-C could learn a lot here and include more libraries, because libraries are what really make the language usable. At the moment, it is still quite bleak for Gambit-C.

By the way, the license issue is finally resolved: you can now choose between LGPL and Apache license for Gambit-C, which should really make all license discussions obsolete.

AES (Rijndael) Encryption Test in JavaScript - AES Encryption in JavaScript

JavaScript Encryption Library - Blowfish Encryption in JavaScript

OpenPGP Message Encryption in JavaScript - AES and RSA and some support libraries in JavaScript - enables creating PGP-encrypted texts.

Rijndael in JavaScript - AES in JavaScript again, this time specifically designed for compatibility with older browsers as well.

twofish/javascript - TwoFish Encryption in JavaScript

code.enthought.com - Enthought Tool Suite - sounds like an interesting GUI library that builds on WxPython and enables even more comfortable application development. Particularly interesting is the use of the "Traits" concept for the automatic creation of interfaces.

/my/cssQuery/ - the IE7 author (not the one from Microsoft, but the compatibility hacker) has split out his CSS Query Engine as a standalone program. A very powerful way to find parts of documents using CSS selectors in JavaScript.

Aggregators and Referrer Spamming

Dave Winer I mean, with his newsRiver.root:

When the aggregator reads a feed it sets the referer for the request to http://www.newsriver.org/, a new site with a place-holder message for people who are unfamiliar with the aggregator. This address will appear in the referer logs for feeds that are read by users of this aggregator.

And again he spammed the referer, instead of adhering to the standard of putting the URL for the aggregator in the User-Agent header. No, not all hits from aggregators come from their pages, this is utter nonsense that he already perpetrated with the Radio Userland aggregator. And back then he showed himself to be forgetful when it was tried to explain it to him.

Why are programmers, especially in the field of web technologies and related topics, so incapable of looking at specifications? And simply adhering to what makes sense? The HTTP specification is not that complicated that one couldn't at least adhere to it rudimentarily ...

Codeville - and yet another version control system, this one is written in Python and specifically addresses the problem of merge algorithms.

monotone: distributed version control - by now Monotone is quite mature. Main advantage: it is distributed as a single executable. And the content-based addressing also sounds very interesting.

xmledit - A filetype plugin to help edit XML, HTML, and SGML documents : vim online - nice plugin for VIM (more precisely a macro package) to better edit XML files with tag completion and automatic closing and jumping between block start and end.

cucumber2: an object-relational mapping system for Python and PostgreSQL - another ORM for Python. Special feature here: PostgreSQL table inheritance is used to make the transitions between objects and classes easier. However, it is also not portable to other databases.

Lisp at Light Speed - a really interesting Lisp blog