tux21b.org - Web Playground of Christoph Hack

Up to now I was used to program in Python and the language is still one of my favorites. It’s just amazing what is possible with just a couple of lines and limited time. But I don’t like the platform, because it doesn’t scale. Threading is nearly useless, because of the GIL (Global Interpreter Lock) and so you need lots of processes, which might be really big, depending on the project. A single request may require less than 1 % of the application, but to provide that, the whole applications gets started again and again and requires a huge amount of CPU time and memory.

At work I learned a lot about Java EE development and I was positively surprised. New technologies like JSF/Facelets, EJB 3.0 (and/or Hibernate) and Seam/Spring, as well as the new Portlet specification, are making development a pleasure too. Depending on the case, it might be even easier than Python and with a good application server and container, the application is also able to scale a bit without any additional care about it. Unfortunately the Java platform tend to have huge overheads (e.g when you are planning to add a portal server) and Web 2.0 applications are surely not a strength of Java EE at the moment. Let’s see what the upcoming JSF 2.0 specification will archive, I am already looking forward to it.

I also did some researches about Scala, a combination of object-orientated and functional programming language. The language itself looks really pretty, has included nearly all benefits of its competitors and can access Java libraries. But I think the language is still a bit limited because of the JVM. I really like the Java platform and its probably the best available at all, but in my opinion it isn’t perfectly suited for functional programming, because it uses heavyweight system threads, doesn’t optimize tail recursion in some cases and wasn’t developed for concurrency.

So lets look at Erlang, a language and platform which really shines when talking about scalability. Features like fault tolerance, no locks, distributed and parallel computing, hot swapping and extremely easy and powerful use of the actor model makes it really look to be the best solution at all. For example imagine the management of the indexes for a bulletin board. The data here isn’t easy to fetch from a relational database, changes steadily and can’t be really cached. With Erlang it would be possible to hold all the important data in the distributed memory and even update it there. A Cron like task could be used to write the data back to the database for persistence or to the disk in case of a database problem.

I just think that Erlang is actually a bit clumsy at string handling, and I haven’t found a really good template engine for it. Two reason why I can’t imagine to use it for a nice looking front-end. One solution which I am beginning to like might be to use it only for the service layer and Python in the front. The Erlang service layer would be also become very useful for handling the AJAX load of modern web application and the Erlang community has already started to improve and simplify web development, so I think we can expect more soon.

But it seems that there is only scrappy information about such systems available and the most common way is still to buy lots of hardware, use database slaves for reading, relentless cache everything and often user session are binded to a specific server to “avoid” coherency. That might work for most of the cases, but in my opinion there are better solutions available and the increase of web 2.0 pages, social networks and user created content will continue to exceed the limitations of the current methods. If somebody has already thought about or even faced similar problems, it would be great to read something about it, so please share your experiences.