Python Greenlets

For some time now I have been using Python as my language of choice, at least for tasks that don't require blazing speed. However one thing that I really wanted was the ability to do simple co-operative multitasking. However this didn't seem to be a feature of the language, as I had expected.

The reason I wanted to do this is mainly for games. A game often requires simulating a number of entities. The logic for these entities is most naturally described as normal procedural code. However, in many c-like languages it is not (normally) possible to write code like this, since the entities must return control of the CPU to other elements of the simulation.

Python 2.2 introduced generators, which allow us to write code which is along the lines of what I wanted:

def generate_stuff():
    for i in range(50):
       yield i
for ii in generate_stuff():
    print i

Here the function generate_stuff is a generator (due to the presence of the yield statement). This means that instead of returning the return value of the function, as a normal function would, this function returns a generator object. This object can then be iterated over (as in the for loop). To get each value control is passed to the generator function, which runs until a yield is encountered (or the function ends). Each value that is yielded is the result of the iteration.

This can be (ab)used to perform co-operative multitasking, such as I want. However, generators cannot be nested, so if you want a subroutine that your generator calls to yield control, you have to call it as a generator:

def sub(b):
    for i in range(5):
       yield i + b

def gen_stuff():
    for i in range(0, 50, 5):
       for j in sub(i):
          yield j

This is quite ugly, and is not the natural-feeling solution I had envisaged.

However, it turns out that Greenlets are the answer. Greenlets are a small extension package that basically implement coroutines for python. This is the kind of thing that the Stackless Python project also achieved, but that required significant changes to the interpreter, making this a much smaller impact change.

Using greenlets it is simple to implement the kind of cooperative multithreading I have been seeking. Better yet it doesn't require any counter-intuitive changes to code - any code can be run in a greenlet, you just need to sprinkle some calls around to Yield or whatever you call the function that returns control to the parent greenlet.

First Day at New Job

Yes it's true - not only have I mustered the enthusiasm to apply for work, but I have also received it. That's what I call beating the odds. I started work yesterday, which was a Friday. This was good, since it meant that my first week was manageably short.

The company I have started at is called Blue Tongue Entertainment. It is a video games developer that is owned by the global publisher THQ. My previous jobs have also been in games development, so I guess I am becoming quite specialised. Hopefully that won't hurt me later on in my career.

It has been almost 18 months since I last had an office job. The day seemed to drag on quite a lot, but then I didn't really have any given task, except to familiarise myself with the games they were working on and the codebase. Hopefully once I have a particular task, I will be able to get back into the swing of things.

Commuting is quite an issue. The offices are on St Kilda Road, near High Street. Taking the train would be troublesome, as I would have to go into Richmond and then take another train or something out. It is difficult to get parking there as well. I think the best plan might be to drive to the corner of Malvern Road and High Street and then take a tram in. The trip along Waverley Road to that point seems to be not too bad from a traffic point of view.

Interfacing Plone to Apache 1.3

Having a blog is great, but what's even better is if people outside of my own house are able to access it. To this end I had to interface zope to the apache 1.3 server that handles the msmith.id.au domain.

This is facilitated by the VirtualHostMonster tool in Zope. This 'gentle giant' handles the dirty work of rewriting all the urls in the site so that they point to locations relative to the root of the site. After this the actual interface in apache is simply to proxy access to the Zope server.

This proxying gave me some trouble. I got http 403 errors, and the error log showed the line:

'client denied by server configuration'

After a while I realised that it was the same problem I get every time I try to add a proxy directive to apache - the proxy destination must be specifically allowed by the httpd.conf file, using a line like this:

<IfModule mod_proxy.c>
    <Directory proxy:http://localhost:9673/>
        Order deny,allow
        Allow from all
    </Directory>
</IfModule>

It just goes to show that I'm too thick to learn from my mistakes - I'm doomed to repeat them endlessly.

Launch of New Blog

Welcome to the Moth-Eaten Boot. This web-based personal log, or 'blog', will serve as a receptacle for whatever deranged fancies spring out of my subconscious before my superego has a chance to quash them. Herein I shall give a glimpse into what makes me tick (a clue to get you started: it's not Hollywood musicals).

It's likely that regular updating will prove too onerous a task to be carried out with any regularity. However, as I am well aware my legions of adoring fans cry out in unison for my attention, I will endeavour to keep you reasonably well apprised of the sordid tabloid details of my affairs.

Grappling with Plone

After spending quite some time setting up the previous version of my web pages, I decided that it made more sense to use an off-the-shelf solution for my new version. This would allow me to save a lot of time, while also presumably achieving a more polished result.

The software I decided to try out is called Quills. It is a product designed for the Content Management System Plone. Plone, in turn, is built for the Web Application Server Zope.

This software includes some impressive features. Zope's mechanism of acquisition is quite powerful, allowing configuration to be made in an intuitive manner across as much or as little of the site as is appropriate. The out-of-the-box Plone setup is highly useful as is. All would have been wonderful if I had been happy with the set-up as it stood.

However, being one of the many unhappy perfectionists, I couldn't settle for that. I wanted the site to work the way I had imagined it. In particular, I wanted a more blog-like setup. Plone on its own makes it very easy to update content on the site, but I chose to use Quills anyway. This provides automatic archiving, commenting, trackback, syndication and other features. I had some difficulty with the last part: after a great deal of debugging I found out that for some reason the version of the python expat xml parser that this server had installed was taking a strict line on some of the xml namespaces that zope uses for page templates. I was forced to jimmy in a namespace declaration for i18n on the rss feed template. It seems to work now (fingers crossed).

This reflects the major issue I have with Zope et al. They are big and powerful, with active developer bases, which means that the code base is quite large and not exactly a walk in the park. Trying to fix a few issues I had was not easy. The code makes use of some of the more 'magic' features of Python, which is great, but needs to be documented in the code so saps like me can wade through it.

Never the less, it seems to have come up reasonably well. One other excellent feature of the setup, (which only benefits me, however), is Epoz. Epoz is a WYSIWIG editor for Plone content (I am using it to type this entry). It uses Javascript and a rich text control that is available on newer Gecko and IE based browsers (so pretty much everyone). It makes formatting text basically as easy as using Word.