programmierung - 25.4.2012 - 2.7.2012

Repo.js. I found the Source-Highlighter through this - a small jQuery plugin that easily embeds a GitHub repository into a website. Very practical if you want to integrate your projects into your own websites and don't want to hand over the entire website to GitHub.

isagalaev/highlight.js. Just stumbled upon it, a small JavaScript source highlighter that works with a language heuristic and thus does not require explicit specification of the language to be colored - it simply tries all languages and takes the one with the most recognized syntactic elements. There is also a jQuery VanGogh jQuery plugin for this.

ErlPort - Erlang port protocol for Python. A somewhat different approach to distributed communication is this module, which simply implements Erlang's built-in communication tools (ports and terms). With this, you can build systems that use Python or Erlang as the implementation language, depending on your needs.

Hurricane. Sounds interesting, it's a distributed messaging system that works with various languages and thus offers integration of different systems. Among other things, it includes Python with WSGI and Ruby with Rails, making a distributed system based on Rails and Django conceivable. Additionally, there is a process manager, with which the processes themselves only need to handle standard I/O and then can be directly managed by Hurricane. I could imagine using this for one or the other project at the company.

SQLite4: The Design Of SQLite4. That sounds very interesting. Especially the first paragraph in "2.0 Overview", where he briefly emphasizes that SQLite3 will continue to be supported and both versions will remain available in parallel. And of course, the various changes that SQLite4 will have compared to the other version, such as the significantly better encapsulation of the engine in its own object. This makes it quite possible to have multiple databases open at the same time without much juggling. And what particularly pleases me: all calculations in Decimal Math and no longer double or float. Sorry, but double (and especially not float) has anything to do in a database, except perhaps as a data type for rare special cases. Also some other nice things in it, for example covering indexes and of course the standard available foreign key constraints.

Jarvis. Someone is working on something like Light Table for Python (Light Table has also promised Python support, but so far there's only a preview for Clojure). Looks quite interesting and finally brings some fresh air into interactive environments for programming languages.

Remote Debugging - Real Software Documentation. Other systems should take a few slices from this. In my game project, I had a problem with the latest version under Linux. However, my development environment is OSX, where everything runs smoothly. So what to do? Well, a good opportunity to take a look at the remote debugger. And it's really trivial. Boot up a Linux system (I use Vagrant on my OSX machine), copy over the remote debugger stub, run it, configure it minimally, and get started. Directly from the OSX IDE, transfer the compiled code to the remote environment and start it, directly with breakpoints and variable watch and such things. Basically, all that I would also have locally with OSX. In principle, there is almost no difference, except that the program runs on Linux. Why is it so much more complicated in so many other environments? I guess I'm getting old, I just don't have the patience for cumbersome environments anymore.

..Nowhere.. has a series of free classes and plugins for RealBasic, including one for a syntax highlighting editor and list boxes with arbitrary cell content.

The Opa blog: Announcing Opa 1.0. Opa no longer generates native code for its own backend, but generates JavaScript for Node.js integration. Sounds quite interesting, as Node.js integration could eventually allow access to the libraries available there (though you would then naturally lose the guaranteed properties of Opa). And additionally, the chances are now higher that you will find a suitable host or that self-hosting could become more pleasant.

Make runfcgi fail when database connection is open before fork. This is something I've been chasing for ages, most recently in a few pretty important projects. Flup works by first initializing the WSGI application and then making the forks for the workers with this initialized WSGI application. Unfortunately, we have database accesses during application initialization - as a result, the base process already has an open database connection, and each fork copies these data. But the socket of the connection doesn't go with it - the new process just thinks it's connected, but it's not. Accesses from the new processes then fail with an exception. In the linked patch, you can also replace the raise on the exception with connection.connection = None. This simply discards the connection that is already defective and always builds a new connection in new processes. With this, we have at least been able to fix this in a production environment (with psycopg2) and are optimistic that it will also help in the environment with pyodbc.

LiveScript. Looks interesting, a descendant of CoffeeScript, but with a syntax that is strongly inspired by Haskell and a Prelude that also aligns with Haskell. With it, you can work in a functional language instead of JavaScript, which, however, continues to try to maintain a balance between functional and imperative elements - so not the pure approach as in Haskell.

SET TRANSACTION ISOLATION LEVEL Transact-SQL. There you can find more information about the isolation level in MSSQL, especially what the snapshot feature means. In principle, this makes MSSQL behave similarly to PostgreSQL.

#18251 multithreading deadlock in django.models.loading.get_apps – Django. And another thing that might affect us - race conditions between Django threads during the initialization of Django applications. There's already a patch for this that fixes it in the Django internals.

Enabling Snapshot Isolation - SQLAlchemy 0.7 Documentation. Could this help us? MSSQL seems to have a rather unfavorable isolation level as default. Hmm, we'll probably try it out.

Using SELECT FOR UPDATE in Django. That's what it's all about. Because the Django ORM cannot currently generate a SELECT FOR UPDATE, but sometimes you just need it.

Thomas Tempelmann | Arbed - The Advanced RB Editor. You stumble upon all sorts of things - I still know Thomas from Modula-2 times, he was like the Modula-2 compiler guru back then. And now he uses Real Basic. And he has built a quite interesting tool for working with RBP files, which I should take a look at because it could simplify a few things (especially versioning, the possibilities that RB offers out of the box are not very impressive). And yes, I really do use Basic. No joke.

ucsd-psystem-vm 0.11. I just had a flashback - UCSD-p System is alive! The link is a VM for UCSD Pascal that runs under Linux. And there is also the original UCSD-p System for download. There is even a version (IV) for DOS, which is probably the easiest to run.

TryAPL. Also online. Also, an APL in JavaScript. With a popup keyboard with the funny APL characters on it. Fun.

PEG.js – Parser Generator for JavaScript. I stumbled upon it while looking for some parser implementations, including PEG-based parser generators, as I have a PEG definition for Markdown. But I probably need to study some theory before I can use it meaningfully. Or I'll just handcraft the parser - the Markdown syntax isn't really that complicated after all. And for my game project, I need to extend the parser with various custom features anyway.

REALbasic | Open Source | Charcoal Design. Open Source for Real Basic. Yeah, I'm currently programming things for myself in Basic. And honestly: it's fun. I found this site while searching for JSON implementations for Real Basic.

Xcode 4.3 MacRuby compatible problem workaround - 東 . Blog. Annoying, but necessary - the AppStore migration of XCode has broken the template installation of MacRuby, so that even with the new MacRuby (0.11 with support for XCode 4.2 and 4.3 is out) the templates do not work right away and the integration with Interfacebuilder does not work. With the instructions here it works again.

Waterbear: Welcome. Falls into a similar category as Google Blockly, but it is more of a toolkit on which one can build visual languages. So maybe interesting for one or the other. Definitely interesting to play around with.

google-blockly - A visual programming language - Google Project Hosting. Generally, I consider visual programming languages to be dead-ends and rather useless, albeit entertaining toys, in most cases. I would also place Blockly in this category, especially since it is really just a visual representation of classic programming structures - such as IFs, variables, and the like. Actually disappointing, considering that Prograph already visualized data flow programming decades ago (and thus provided a much more interesting model for a visual programming environment).

RQ: Simple job queues for Python. Really simple - you only need a Redis server and the module and that's it more or less. Simple function calls are thrown into the queue, executed and results returned. No big overhead in the code.

git-annex. Interesting plugin for git with which you only manage the metadata of large files, but not their actual contents in git. The files themselves are managed as normal external files, but the directory structures and references are correctly versioned - and all this together with normally versioned files. This bypasses the problem that git has with large files. And thus something like a photo archive based on git becomes possible.

TypeQuery. Generic functions for Python. Only a simple variant with a restricted target range, but definitely worth considering for some things. Currently still single-dispatch on the first argument, but multiple dispatch is planned. At the moment, it is therefore pretty much identical in function to simplegeneric by PJE. And very similar to my multidispatch, in which I try to replicate the model of Clojure generic methods. I really should work on multidispatch again. Or just play more with clojure-py, then I can use "the real thing" right away.

PostgreSQL: Documentation: 8.4: hstore. From the series "Things your database can do that you might not know": Key-Value-Stores within a PostgreSQL data field. Or also poor-mans-object-notation. Or simply practical when you want to store loosely structured data but don't want to constantly adjust the schema - the schema is then the primary data for the order of the model, the dependencies, cardinalities, etc. - and the hstore field then stores the additional manifestations that, for example, result dynamically from business requirements. Sometimes the better alternative to a denormalized property table. And it's been there since the 8.x versions (though it is a loadable module).

Matasano Security - Matasano Web Security Assessments for Enterprises. Analysis of cryptography in JavaScript. Summary: Cryptography in JavaScript is usually a bad idea, as the JavaScript is loaded from an untrusted source or untrusted network (if you trusted it, cryptography would hardly be necessary) and therefore a chicken-and-egg problem exists. Regarding the previous 0bin project: cryptography was not implemented to secure the user, but to secure the 0bin operator - it is therefore relatively irrelevant to the operator whether users are secure or not, it is only about "plausible deniability" for the operator. The situation is different, however, when a JavaScript encryption is implemented instead of using SSL.

sametmax/0bin. Interesting approach to circumvent the problems of pastebin hosting. With pastebin-like sites, the problem is that users post all kinds of content and the server operator can quickly be held liable. 0bin tries to shift this problem so that the operator cannot know what is in the pastebins, as they are stored encrypted and the encryption is done by the client via JavaScript. This works, of course, only if the judges also accept that the server operator cannot know what is going on - and not, for example, argue that he should then just install different pastebin software where he can know and still hold him responsible. Certainly an interesting approach, especially the idea of putting the key for the encryption in the hash of the URL (i.e., the part after the # in the address) and thus having a functioning URL, but still not providing the server with the key (since the hash of a URL is only used by the client and not communicated to the server).

Embedding Python in Objective-C: Part 2. An interesting project that embeds Python in Objective-C projects and allows direct connections between Python and Objective-C code via automatically generated bridge modules. I might check this out at some point, I still have a game project that could benefit from this.

jodal/pykka. An actor library for Python, based on threads and optionally on GEvent. Looks quite good and could be quite practical for communicating processes if the actor model fits. Perhaps one could also use something like this as a basis to run my Linda Tuplespace in a distributed manner.

Jython 2.7 alpha1 released!. Wow, it took a long time, but now at least the three major alternative Python runtimes are all at 2.7 (IronPython, PyPy and now Jython). I hope that Jython 2.7 has gained significantly in performance, because otherwise it is far behind in comparison - my last tests were rather depressing (2-3 times slower than CPython).

cocoa-python - Port of Objective-C runtime to Python using ctypes. It's actually all in the link text. Interesting because the author uses it to write a port of Pyglet that does not require PyOBJC and would therefore be completely python-only source code.

R17 - flexible, scalable, relational data mining language. Looks quite interesting, basically something like a cross between AWK and SQL. The result isn't really pretty, but it seems practical - especially because you can easily use multiple processors, or even multiple machines (implicit parallelization), and thus also quite easily evaluate large amounts of data with ad-hoc queries. Because there is a simple format for passing data to further steps, it can also be easily adapted to new data sources without first running a lengthy export step there.

The Schemaverse. And while we're on the topic of strange projects: someone has programmed an MMO that runs entirely within PostgreSQL. So using pgSQL as the language. Something like a multi-user battleships game. It just runs in a database. And is controlled via SQL.

sipML5 - The worlds first open source HTML5 client. Oh - someone wrote a SIP client in JavaScript that runs entirely in the browser. It is based on WebRTC, a standard for real-time communication on the web.

Plumbum: Shell Combinators and More — Plumbum: Shell Combinators. Looks interesting and much more thought out than some alternatives I've looked at (and much more expanded than shutil+glob).

Clojure/core — Reducers - A Library and Model for Collection Processing. Why a new library? Because the standard functions are interesting and meaningful, but sometimes you need a stricter implementation that avoids intermediate temporary collections and provides direct out-of-the-box support for parallelization. Without much fuss and bother, just a different namespace.

A free book (that will also be available as dead tree or electronic wood from O'Reilly) about backbone.js. Backbone.js provides an MVC-inspired structure for JavaScript programming (where you can argue a lot about what exactly is the View and the Controller and whether it is "pure" MVC - or you just program and ignore the theoretical ballast).

PyPy Status Blog: STM update: back to threads?. The discussion and development continue to progress - and the discussion is returning to the old thread model, only expanded with a function to define atomic blocks of code. And all of this in such a way that the code would even run under normal CPython (though then of course without the advantages of STM, since there is no STM in CPython, there is still the GIL) but still makes sense to use multicore architectures. I like it more and more and I hope it will soon land in the PyPy Main Branch.

tuupola/jquery_lazyload. This might be interesting for displaying large amounts of images because it allows you to delay the loading of images until the user actually scrolls to them. The page is pre-built with placeholder images, and only the images are later sent over the line. However, I think Infinite Scroll is the better solution, as it allows not only images but also general content to be added - and it also works smoothly with Masonry so that these content blocks can also be visually well distributed.

dirq 1.1.2 documentation. Hey, I was recently looking for something like this - a queue based on the file system. The advantage: simplest persistence and good "debugging" by simply poking around in the file system. The disadvantage is that queues often need to perform some operations atomically - and this can become a bit tricky if you want to get it right. This is a port of an already existing Perl module, so the chances that most of the childhood diseases are out are quite high. The API, in any case, looks nicely simple. Definitely worth a closer look.

Hotcanvas: Realtime coding. I'm a fan of Processing-like environments, this one comes quite close, but runs as an interactive environment and looks quite nice. Ok, there are probably quite a few of these things by now, but still, I like this kind of stuff.

Bilderarchiv | rfc1437. My image archive was revised over the weekend and now comes with funny JavaScript gadgets for automatic tiling. Not really groundbreaking, but more interesting than the previous, rather boring format. jQuery Masonry is the main part of the work. Nice little JavaScript library. The most interesting part is probably the heuristic in my little plugin, with which I bring the images into portrait/landscape/square formats, so that the whole thing does not become too boring due to an excess of landscape formats. Unfortunately, there are still gaps in the image stream, where Masonry could actually become a bit smarter (and e.g. pull up small squares). Or I have to sit down again with the heuristic and make sure that the chances of gaps are reduced.

RubyMotion - Ruby for iOS. Since MacRuby already interests me, this sounds good too - directly programming for iOS in Ruby. However, there is no free version, which somewhat limits the trying out. Additionally, there is nothing in an IDE, it's pure command line. Which can be an advantage, but for debugging, for example, it's a real disadvantage.

Mojolicious - Perl real-time web framework. You had one problem, you took Perl. Now you have two problems and a bunch of sharp brackets to cut yourself on. Jokes aside: nice to see that Perl is not only still active, but also actively modern in the web environment. After all, Perl once set the standard in that area.

RavenDB - 2nd generation document database. Bookmarked because I want to check it out in the not-too-distant future. Sounds quite interesting from the features and might be useful for one or two projects that I am currently thinking about.

JSIL - .NET to JavaScript compiler. No idea if I already had this, but it's one of those crazy projects you can safely link twice. The actual .NET IL code is translated - so you can run programs in the browser that were not really intended for it. For example, part of XNA is supposed to run, so you can run games in the browser that are actually for the XBox.

the { buckblogs :here }: Maze Generation: Ellers Algorithm. Maze algorithms have fascinated me since my youth, but this one was new to me. It generates perfect mazes line by line, without trackback or recursion, with a runtime linearly dependent on the size of the maze. Pretty brilliant stuff.

jQuery Masonry. Alternative to BlocksIt. Worth checking out. Even pushes blocks more aggressively, rearranges the order if necessary.