Python Greenlets

For some time now I have been using Python as my language of choice, at least for tasks that don't require blazing speed. However one thing that I really wanted was the ability to do simple co-operative multitasking. However this didn't seem to be a feature of the language, as I had expected.

The reason I wanted to do this is mainly for games. A game often requires simulating a number of entities. The logic for these entities is most naturally described as normal procedural code. However, in many c-like languages it is not (normally) possible to write code like this, since the entities must return control of the CPU to other elements of the simulation.

Python 2.2 introduced generators, which allow us to write code which is along the lines of what I wanted:

def generate_stuff():
    for i in range(50):
       yield i
for ii in generate_stuff():
    print i

Here the function generate_stuff is a generator (due to the presence of the yield statement). This means that instead of returning the return value of the function, as a normal function would, this function returns a generator object. This object can then be iterated over (as in the for loop). To get each value control is passed to the generator function, which runs until a yield is encountered (or the function ends). Each value that is yielded is the result of the iteration.

This can be (ab)used to perform co-operative multitasking, such as I want. However, generators cannot be nested, so if you want a subroutine that your generator calls to yield control, you have to call it as a generator:

def sub(b):
    for i in range(5):
       yield i + b

def gen_stuff():
    for i in range(0, 50, 5):
       for j in sub(i):
          yield j

This is quite ugly, and is not the natural-feeling solution I had envisaged.

However, it turns out that Greenlets are the answer. Greenlets are a small extension package that basically implement coroutines for python. This is the kind of thing that the Stackless Python project also achieved, but that required significant changes to the interpreter, making this a much smaller impact change.

Using greenlets it is simple to implement the kind of cooperative multithreading I have been seeking. Better yet it doesn't require any counter-intuitive changes to code - any code can be run in a greenlet, you just need to sprinkle some calls around to Yield or whatever you call the function that returns control to the parent greenlet.