saush

My first contribution to an O’Reilly book (Rails Cookbook)

Posted in general, Rails by sausheong on October 31, 2006

A few years ago, I was very much into writing technical articles for magazines. My most profilic years were from 2001 to 2002 and I published about 6 – 7 articles in various magazines, including Javaworld, Java Report and Dr. Dobb’s Journal. At that point in time one of my greatest ambition was to write a full-length technical book (on Java of course) for O’Reilly. I had racks of books by O’Reilly (close to 40 I think), mostly Java related and I had a burning desire to display my very own O’Reilly book among them.

I submitted a proposal of course, and other proposals to other publishers but sadly none went through.  After a while the burning desire simmered underneath other more critical issues like surviving the dot-com bust and the elipva Incident as well as other rising interests like my growing interest in Law.

So when I saw a mail in the Ruby on Rails mailing list calling out for contributions for an O’Reilly Rails book, I jumped at the opportunity. No doubt, my contribution is pretty small compared to the rest of the book (I contributed only 3 recipes) but at least I have made a small step towards it. Who knows, it may spark me again to start an O’Reilly book proposal again :)

Best of all O’Reilly is giving me 4 books (besides a token sum) as a gesture of appreciation. Time to stock up on the O’Reilly books again! Any suggestions on what to get?

My first contribution to an O’Reilly book (Rails Cookbook)

Posted in general, Rails by sausheong on October 31, 2006

A few years ago, I was very much into writing technical articles for magazines. My most profilic years were from 2001 to 2002 and I published about 6 – 7 articles in various magazines, including Javaworld, Java Report and Dr. Dobb’s Journal. At that point in time one of my greatest ambition was to write a full-length technical book (on Java of course) for O’Reilly. I had racks of books by O’Reilly (close to 40 I think), mostly Java related and I had a burning desire to display my very own O’Reilly book among them.

I submitted a proposal of course, and other proposals to other publishers but sadly none went through.  After a while the burning desire simmered underneath other more critical issues like surviving the dot-com bust and the elipva Incident as well as other rising interests like my growing interest in Law.

So when I saw a mail in the Ruby on Rails mailing list calling out for contributions for an O’Reilly Rails book, I jumped at the opportunity. No doubt, my contribution is pretty small compared to the rest of the book (I contributed only 3 recipes) but at least I have made a small step towards it. Who knows, it may spark me again to start an O’Reilly book proposal again :)

Best of all O’Reilly is giving me 4 books (besides a token sum) as a gesture of appreciation. Time to stock up on the O’Reilly books again! Any suggestions on what to get?

Processing Rails task in the background

Posted in Rails, Ruby by sausheong on October 22, 2006

Rails as a web application development framework is wonderful but running real-time processing with Rails (or any web application else for that matter) is not something really smart. For example, you need to send some email messages to a large number of recipients, you wouldn’t want to trigger the send and then wait for Rails to finish processing. Despite the nice Ajaxy things you can do, it’s still irritating to wait for 10 minutes Rails to finish its job.

What I used is probably one of most common application design patterns around — create a ‘job’ from the user interface and have a backend job scheduling mechanism to run it separately. In Java the most natural thing to do was to open up the magic open source hat and pull out Quartz(http://www.opensymphony.com/quartz/), the J2EE job scheduling framework. I will create a Quartz job and let Quartz take care of everything else.

But what can I use in Ruby on Rails? Searching through trusty ole Google found me a couple of possible solutions. One was rails_cron, managed by Kyle Maxwell which is a cron equivalent controlled by Rails. Another is BackgrounDRb, by Ezra Zygmuntowicz, a Rails plug-in that removes a processing task away from a normal HTTP request/response cycle. A third is to use Starfish by Lucas Carlson. Starfish, together with MapReduce (really, a ‘reduced’ functionality version of it) allows me to efficiently process large chunks of data using ActiveRecord but separately from Rails. So plenty of possibilities.

Of the few solutions, BackgrounDRb seems to be the most appropriate solution. Rails_cron is no longer maintained, Kyle decided to defer to BackgrounDRb, which is a more complete solution. I couldn’t get Starfish to work properly under Windows, and under Linux I faced problems with DRb and Rinda (Starfish is a wrapper around DRb and Rinda). In the end I didn’t go with BackgroundDRb either as it depends on DRb as well and I thought there isn’t such a need to make things so complex.

My alternative solution is to create a short Ruby script that retrieves all records through ActiveRecord and process them through a number of threads, then loop it over and over again. This can be created as a Windows Service using win32-service or as a *nix daemon using daemons.

This is the code snippet for the processing to be done:

require 'rubygems'
require 'active_record'
require 'app/models/message'

ActiveRecord::Base.establish_connection({
:adapter  => "mysql",
:host     => "localhost",
:username => "xxx",
:password => "xxx",
:database => "mydatabase"
})

# loop over and over again
loop do
messages = Message.find :all,
:conditions => ['sent = (?)', 0] # check if the message has been sent
:limit => 20 # retrieve and process 20 at a time

# array of threads
threads = []

# iterate through each message
for message in messages do
# start a new thread to process the message
threads <  e
# do some rescuing
puts "Failed: #{e.message}"
end
end
end

end

Creating a Windows service

For this I used Daniel Berger’s win32-service package. This is a great package that does literally everything for you to enable you to create a nice Windows service. Just install the gem like this:

gem install win32-service

I used examples/daemon_test.rb as the base template to make my Windows Service. This piece of code contains all that is needed to run the code snippet above as a service. In this file you will find a class called Daemon that has a number of methods necessary for running a Windows service. Put in the necessary requires and place your ActiveRecord initialization code at the beginning of the code. Then under service_main, while the status == RUNNING, put in the main processing code, minus the loop. Your code could possibly look something like this (don’t cut and paste this code, just use it as a reference):

require 'rubygems'
require 'logger'
require 'active_record'
require 'c:/myrailsapp/app/models/message' # remember to put in the absolute path here
require "win32/service"
include Win32

ActiveRecord::Base.establish_connection({
:adapter  => "mysql",
:host     => "localhost",
:username => "xxx",
:password => "xxx",
:database => "mydatabase"
})

# I start the service name with an 'A' so that it appears at the top
SERVICE_NAME = "MyProcess Service"
SERVICE_DISPLAYNAME = "MyProcess"

if ARGV[0] == "install"
svc = Service.new
svc.create_service{ |s|
s.service_name = SERVICE_NAME
s.display_name = SERVICE_DISPLAYNAME
s.binary_path_name = 'ruby ' + File.expand_path($0)
s.dependencies = []
}
svc.close
puts "installed"
elsif ARGV[0] == "start"
Service.start(SERVICE_NAME)
# do stuff before starting
puts "Ok, started"
elsif ARGV[0] == "stop"
Service.stop(SERVICE_NAME)
# do stuff before stopping
puts "Ok, stopped"
elsif ARGV[0] == "uninstall" || ARGV[0] == "delete"
begin
Service.stop(SERVICE_NAME)
rescue
end
Service.delete(SERVICE_NAME)
# do stuff before deleting
puts "deleted"
elsif ARGV[0] == "pause"
Service.pause(SERVICE_NAME)
# do stuff before pausing
puts "Ok, paused"
elsif ARGV[0] == "resume"
Service.resume(SERVICE_NAME)
# do stuff before resuming
puts "Ok, resumed"
else

if ENV["HOMEDRIVE"]!=nil
puts "No option provided.  You must provide an option.  Exiting..."
exit
end

## SERVICE BODY START
class Daemon
logger = Logger.new("c:/myprocess.log")

def service_stop
logger.info "Service stopped"
end

def service_pause
logger.info "Service paused"
end

def service_resume
logger.info "Service resumed"
end

def service_init
logger.info "Service initializing"
# some initialization code for your process
end

## worker function
def service_main
begin
while state == RUNNING || state == PAUSED
while state == RUNNING

# --- start processing code
messages = Message.find :all,
:conditions => ['sent = (?)', 0], # check if the message has been sent
:limit => 20 # retrieve and process 20 at a time

# array of threads
threads = []

# iterate through each message
for message in messages do
# start a new thread to process the message
threads <  e
# do some rescuing
puts "Failed: #{e.message}"
end
end
end

# --- end processing code

end
if state == PAUSED
# if you want do something when the process is paused
end
end
rescue StandardError, Interrupt => e
logger.error "Service error : #{e}"
end
end
end

d = Daemon.new
d.mainloop

end #if

Important to note that you need to put in the absolute path in the require as the service wouldn’t be starting at the Rails app.

Now you can install and start the Windows service (assuming the code is written in a file called ‘message_service.rb':

c:/>ruby message_service.rb install
c:/>ruby messag_service.rb start

You can also control it from your Windows Services MMC console. What you have now is a Windows service that loops around until there is a message record in your database that is not sent (sent = 0). If there are, it will retrieve up to 20 messages at a go and process them with a thread each (parallelizing the processing to make it faster). Once it is processed, it will indicate the meesage has been sent (sent = 1) and loop again. Now you can happily create messages from your Rails app and stuff them into the database, while your message processor will process them separately.

Conch shell

Posted in Origami by sausheong on October 9, 2006

This conch shell is from Toshikazu Kawasaki again, from the book Origami for the Connoisseur by Kunihiko Kasahara and Toshie Takahama. It is folded from a single piece of square paper, from an A4 sized copier paper (recycled from old printouts of Java articles). It took me about 30 minutes to fold this. It’s not too difficult though it looks a bit daunting at first. Thanks to Pierre for lending me the book! I need to get a copy of it for myself :(
This is the top view.

shell_top.png

This is the almost-side view.

shell_side.png

Voyer’s Cat

Posted in Origami by sausheong on October 6, 2006

This is the folding of a cat originally folded by J. Anibal Voyer Iniesta. I folded it in about 30 minutes with a single piece of square paper originating from an A4 sized sheet of paper. I couldn’t quite get the face properly though.
Voyer's Cat

Origami is quite relaxing actually — it helps to keep the hand and the mind busy on a single task, which comes to think about it, is quite a lot like meditation. After the folding you get a sense of achievement of creating something that looks like nothing of the original.

Follow

Get every new post delivered to your Inbox.

Join 449 other followers