Rails as a web application development framework is wonderful but running real-time processing with Rails (or any web application else for that matter) is not something really smart. For example, you need to send some email messages to a large number of recipients, you wouldn’t want to trigger the send and then wait for Rails to finish processing. Despite the nice Ajaxy things you can do, it’s still irritating to wait for 10 minutes Rails to finish its job.

What I used is probably one of most common application design patterns around — create a ‘job’ from the user interface and have a backend job scheduling mechanism to run it separately. In Java the most natural thing to do was to open up the magic open source hat and pull out Quartz(http://www.opensymphony.com/quartz/), the J2EE job scheduling framework. I will create a Quartz job and let Quartz take care of everything else.

But what can I use in Ruby on Rails? Searching through trusty ole Google found me a couple of possible solutions. One was rails_cron, managed by Kyle Maxwell which is a cron equivalent controlled by Rails. Another is BackgrounDRb, by Ezra Zygmuntowicz, a Rails plug-in that removes a processing task away from a normal HTTP request/response cycle. A third is to use Starfish by Lucas Carlson. Starfish, together with MapReduce (really, a ‘reduced’ functionality version of it) allows me to efficiently process large chunks of data using ActiveRecord but separately from Rails. So plenty of possibilities.

Of the few solutions, BackgrounDRb seems to be the most appropriate solution. Rails_cron is no longer maintained, Kyle decided to defer to BackgrounDRb, which is a more complete solution. I couldn’t get Starfish to work properly under Windows, and under Linux I faced problems with DRb and Rinda (Starfish is a wrapper around DRb and Rinda). In the end I didn’t go with BackgroundDRb either as it depends on DRb as well and I thought there isn’t such a need to make things so complex.

My alternative solution is to create a short Ruby script that retrieves all records through ActiveRecord and process them through a number of threads, then loop it over and over again. This can be created as a Windows Service using win32-service or as a *nix daemon using daemons.

This is the code snippet for the processing to be done:

require 'rubygems'
require 'active_record'
require 'app/models/message'

ActiveRecord::Base.establish_connection({
:adapter  => "mysql",
:host     => "localhost",
:username => "xxx",
:password => "xxx",
:database => "mydatabase"
})

# loop over and over again
loop do
messages = Message.find :all,
:conditions => ['sent = (?)', 0] # check if the message has been sent
:limit => 20 # retrieve and process 20 at a time

# array of threads
threads = []

# iterate through each message
for message in messages do
# start a new thread to process the message
threads <  e
# do some rescuing
puts "Failed: #{e.message}"
end
end
end

end

Creating a Windows service

For this I used Daniel Berger’s win32-service package. This is a great package that does literally everything for you to enable you to create a nice Windows service. Just install the gem like this:

gem install win32-service

I used examples/daemon_test.rb as the base template to make my Windows Service. This piece of code contains all that is needed to run the code snippet above as a service. In this file you will find a class called Daemon that has a number of methods necessary for running a Windows service. Put in the necessary requires and place your ActiveRecord initialization code at the beginning of the code. Then under service_main, while the status == RUNNING, put in the main processing code, minus the loop. Your code could possibly look something like this (don’t cut and paste this code, just use it as a reference):

require 'rubygems'
require 'logger'
require 'active_record'
require 'c:/myrailsapp/app/models/message' # remember to put in the absolute path here
require "win32/service"
include Win32

ActiveRecord::Base.establish_connection({
:adapter  => "mysql",
:host     => "localhost",
:username => "xxx",
:password => "xxx",
:database => "mydatabase"
})

# I start the service name with an 'A' so that it appears at the top
SERVICE_NAME = "MyProcess Service"
SERVICE_DISPLAYNAME = "MyProcess"

if ARGV[0] == "install"
svc = Service.new
svc.create_service{ |s|
s.service_name = SERVICE_NAME
s.display_name = SERVICE_DISPLAYNAME
s.binary_path_name = 'ruby ' + File.expand_path($0)
s.dependencies = []
}
svc.close
puts "installed"
elsif ARGV[0] == "start"
Service.start(SERVICE_NAME)
# do stuff before starting
puts "Ok, started"
elsif ARGV[0] == "stop"
Service.stop(SERVICE_NAME)
# do stuff before stopping
puts "Ok, stopped"
elsif ARGV[0] == "uninstall" || ARGV[0] == "delete"
begin
Service.stop(SERVICE_NAME)
rescue
end
Service.delete(SERVICE_NAME)
# do stuff before deleting
puts "deleted"
elsif ARGV[0] == "pause"
Service.pause(SERVICE_NAME)
# do stuff before pausing
puts "Ok, paused"
elsif ARGV[0] == "resume"
Service.resume(SERVICE_NAME)
# do stuff before resuming
puts "Ok, resumed"
else

if ENV["HOMEDRIVE"]!=nil
puts "No option provided.  You must provide an option.  Exiting..."
exit
end

## SERVICE BODY START
class Daemon
logger = Logger.new("c:/myprocess.log")

def service_stop
logger.info "Service stopped"
end

def service_pause
logger.info "Service paused"
end

def service_resume
logger.info "Service resumed"
end

def service_init
logger.info "Service initializing"
# some initialization code for your process
end

## worker function
def service_main
begin
while state == RUNNING || state == PAUSED
while state == RUNNING

# --- start processing code
messages = Message.find :all,
:conditions => ['sent = (?)', 0], # check if the message has been sent
:limit => 20 # retrieve and process 20 at a time

# array of threads
threads = []

# iterate through each message
for message in messages do
# start a new thread to process the message
threads <  e
# do some rescuing
puts "Failed: #{e.message}"
end
end
end

# --- end processing code

end
if state == PAUSED
# if you want do something when the process is paused
end
end
rescue StandardError, Interrupt => e
logger.error "Service error : #{e}"
end
end
end

d = Daemon.new
d.mainloop

end #if

Important to note that you need to put in the absolute path in the require as the service wouldn’t be starting at the Rails app.

Now you can install and start the Windows service (assuming the code is written in a file called ‘message_service.rb’:

c:/>ruby message_service.rb install
c:/>ruby messag_service.rb start

You can also control it from your Windows Services MMC console. What you have now is a Windows service that loops around until there is a message record in your database that is not sent (sent = 0). If there are, it will retrieve up to 20 messages at a go and process them with a thread each (parallelizing the processing to make it faster). Once it is processed, it will indicate the meesage has been sent (sent = 1) and loop again. Now you can happily create messages from your Rails app and stuff them into the database, while your message processor will process them separately.