This post was first published on the imedo.de devblog. The blog went offline in 2012, so I migrated the posts to my personal blog.

Most people ask us why we use background_lite at all, as it doesn’t provide any background job functions itself. Why do you use ActiveRecord and not plain SQL? Abstraction? Exactly!

The goal of background_lite is to abstract the interface of background handling and let you configure the actual background layer. We’ve been using active_messaging for background jobs since we use background jobs (mid 2007). And we soon felt that the way we had to use its API was kind of obtrusive. So we created background and later background_lite as an abstraction layer for background jobs.

active_messaging was quite convenient to use but there were several things that we didn’t like about it:

  • The backend we use (ActiveMQ) is Java – This is not a Java-bashing reason. The problem with Java infrastructure (for us) is that monitoring and starting/stopping processes isn’t the same as all the other infrastructure we use
  • ActiveMessaging pollers aren’t easy to monitor/start/restart – They should be if you look at the code. But we never got them to start/restart as we wanted them to
  • ActiveMQ showed some strange behaviour: Polling the queue started to fail for no apparent reason from time to time and so did pushing stuff into the queue. Restarting ActiveMQ helped sometimes but not always. And it resulted in a corrupted “data” directory which means lost jobs. Recovering from this situation also required restarting our Mongrels (which is rather strange).

As github is successfully using Resque we tried it on a side project via background_lite and found that it works really well. The reasons to chose Resque (and not some other queuing solution) are pretty much the same as described by the github guys.

As the other project already used Resque via background_lite the required handler for Resque was already in place. So the only thing we needed to do to switch from active_messaging to Resque (besides the installation of Resque itself) was to change a simple configuration option!

We’re talking about changing a major piece of backend infrastructure. The transition went flawlessly (not a single lost job or application error) and since the switch one day ago Resque already has processed more than 2 million requests without a single hiccup.

Further information:

Related Posts