Tuesday, February 23, 2010

Pycon - The Second Half

Non-Blocking and Asynchronous I/O by Peter Portante
This was perhaps the best explanation that I have seen of this topic. If you have to deal with socket programming or network concurrency then checkout his presentation. The performance boost that you can achieve is remarkable. It made me start thinking what would happen if Python were to internally make all their I/O non-blocking, but more on that later. The slides are at http://goo.gl/1uRo.

Unladen Swallow: fewer coconuts, faster Python by Collin Winter
An overview of the current state of Unladen Swallow (http://goo.gl/8u9e), the proposed VM implmentation for Python slated for 3.3. Part of the mandate of the project is to not do anything new. That means that they draw only on published and tested concepts of VM and compiler architecture. The best line was when Collin explained that one of his team members came up with a "new" idea and he responded, "No. That makes the Baby Jesus cry profusely."

Understanding the Python GIL by David Beazley
This was the most discussed presentation of the conference. The GIL has long been a monkey on the back of Pthon, and few people understand how it actually works. Antoine Pitrou wrote a new version of the GIL based on timing instead of the current loop. At first it looked very promising, but Dave found a test case that killed it. If you want to know more about the inner workings of the GIL see http://goo.gl/QtvT.

Powering the real-time web with Python: PubSubHubbub on App Engine by Brett Slatkin
There has already been a lot of press about this since it is the underlying delivery mechanism for Google Buzz and it is being ported to many of Google's other offerings. pubsubhubbub provides a protocol for real time delivery of feeds and is dead simple to implement. If you want to publish or subscribe to a feed and you know a little XML you will be up and running in less than an hour. Watch the short video at http://goo.gl/llHP for a clear explanation.

Threading is not a model by Joe Gregorio
An interesting talk on how software design patterns that have really irritated programmers in the past eventually became integrated parts of languages. We should all be irritated that threading and concurrency are models that we have to think about. Go has made progress in this area with go-routines, but this should be the standard in all languages. http://goo.gl/0QYL

Open Spaces

Python and the GIL
This was my best time at the conference. It was lead by Dave Beazley and started off with 20 people, There were about 10 in the room at the finish. We looked at the C code for Antoine's implementation and the non-deterministic results that were generated based on how many CPUs were available. More CPUs could actually seriously degrade performance.

Dave had come up with a proof of concept for a simple scheduler based on priorities. The priorities are based upon whether a thread is CPU bound or blocking and how easily, or often, it releases the GIL. Cooperative threads are rewarded with an increase in priority, and rude threads are penalized. This looks to be the most promising idea for the GIL that I have seen in a long time.

There was a lot of talk of how to deal with large pools of threads. Peter Portante clarified the problem by explaining that we really only care about two threads: the one that has the GIL and the one that will execute next. The hard part is determining who is next and how they are notified by the current thread.

Overall, it was an informative and spirited discussion that was marred by only two people. One who prattled on about how he has worked with this type of thing for years, and blah, blah, blah. Guido von Rossum, the inventor of Python, cut him off by asking, "Is there a question in there?"

The other was this academic sounding man who had come to the conclusion that it was impossible to solve our problem and that years of research had shown this. Guido responded, "Fine. We are going to solve it anyway."

When Dave posts his code, this is definitely something I will work on.

Unladen Swallow
Steered by Collin Winter and the Unladen Swallow team. I could only stay about two hours since I had to make my flight, but I learned how to make the debug and optimized builds and how the VM fits into the Python roadmap. This was the most cooperative sprint that I have ever attended. You had a lot of really smart people with a lot of experience with this product but anyone could ask any question about anything. There was a programmer, named Jeff, who works with Collin who seemed to know everything. Not being able to stay longer was my one regret of the conference.

Overall the conference was quite good and I met a lot of interesting people, including a bunch from Portland, OR with whom I had dinner. Unlike some other conferences, everyone was open and willing to talk and share knowledge. Part of this is the stewardship of Steve Holden who leads the Python Software Foundation. He is optimistic and energetic, and is always asking how we can improve everything. He threw a party for the volunteers in his hotel room which became so rowdy that hotel security ejected him from his own room. Knowing Steve, who is from Yorkshire, he probably asked them to credit his account for the time he was barred from his room.

Next year they promise that there will be more extreme or in-depth subjects, so I will defintely sign up as soon as it is available.

No comments:

Post a Comment