Updater is my background job queue processor. (GitHub Repository) It works a bit like delayed_job or rescue. While it is intended to work with a number of different ORM layers, at the moment only DataMapper is implemented. Native support for MongoDB and ActiveRecord are planed. Get it on GemCutter with gem install updater.

My original purpose in writing updater was to implement delayed_job with a DataMapper back end. The project has sense evolved based around what I need for AntaresTrader and my blog.

Design Goals

Updater is dynamically scalable over a broad range of loads. If there are no jobs, nothing hits the data store and only the minimum of workers are kept ready (one by default). It was important to me that the database not be polled. For my blogs, I anticipate 1-5 jobs per week and there is no need check for a job every 10 seconds. This magic is accomplished with the help of both process signals and socket communication if configured. When under a load the master process can add workers up to a configured maximum to work off jobs.

I also wanted flexible error handling and rescheduling. Chained Jobs, are my solution. each job has three sets of chained jobs, success, failure and ensure. The jobs in these queues are run after the first completes based on whether or not an error was raised. I envision writing a library of default jobs that will use this chaining mechanism to implement things like regular scheduling, logging, error handling and division of labor.

Updater configuration is setup to provide good defaults, but to also allow for lots of customization. by default it looks for a YAML config file in the root directory or in a /config directory. This file is run through an ERb filter so it can dynamically determine the proper settings if necessary. Both the client program and the job queue server use the same configuration file. Once the configuration file is created, client setup is as easy as Updater::Setup.client_setup.

Finally, Updater can use a time other then Time.now (though that is the default) This was done specifically with AntaresTrader in mind. I will need to run a Game Clock that I can pause for maintenance. (this feature is not fully implemented).

Release Goals

1.0

• Documentation!
• Screencast HOWTO
• Test Coverage

1.1

• Client Object API (eg send_later)
• MongoDB Support
• Possibly ActiveRecord Support
• Initial Chain Library (reschedule, log_error, schedule)

1.2

• HTML Monitor for server
• Time Class Configuration
• ThreadWorker / Windows Compatibility
• SimpleWorker

1.3

• Advanced Chain Library
• Developer Screencast

1.4

• Advanced load anticipation for server