saush

Write a Sinatra-based Twitter clone in 200 lines of Ruby code

Posted in DataMapper, general, Ruby, Sinatra, Twitter by sausheong on April 2, 2009

After trying out Sinatra as the interface to my search engine, I got hooked to it. I liked the no-frills approach to developing a web application. I liked it so much that I decided to write a fuller web app using Sinatra. Of course I had no idea what to write, so I decided to clone a random popular application. That was how I ended up writing a Twitter clone.

It was inevitable that halfway writing the Chirp-Chirp, my Sinatra-based Twitter clone, I found out that there are already quite a few Twitter clones out there, 262 to be specific, at least as of August 2008.

The design goals for Chirp-Chirp are very simple. I want to build a reasonably feature complete Twitter clone in the simplest way possible. The code should be minimal and easily understood. The user interface should be usable without additional extensive instructions. Also, the deployment should be straightforward.

To begin with, let’s look at a Twitter clone. Fundamentally it’s all about telling people what you’re doing with minimal text entry, in real-time. The people are friends who ‘follow’ you (naturally, called ‘followers’). The text that you type is  short, which is why such apps are also known as micro-blogs.

Let’s look at the various components of Chirp-Chirp:

  • User interface
  • Login and user management
  • Data modeling
  • Home page
  • Friends and followers
  • Public and direct messages

User interface
Simply put I just copied Twitter‘s user interface and slapped on a self-drawn logo.

Login and user management
I don’t really want to write a login module, nor manage users who log in to Chirp-Chirp but I certainly need one. So my solution is to ‘borrow’ someone else’s login module and kept minimal user management. The first thing I looked into is OpenID. Using OpenID is relatively simple but it still involved some code integration using an OpenID library.

Finally I settled on using RPX to handle the login, and doing minimal integration to single sign-on with Yahoo!, Google and Windows Live ID, 3 of the biggest players on the Internet scene. I figured almost everyone would have at least 1 of these 3 accounts. The advantage in using RPX is that there is really little integration needed, and if the user is already logged into either one of the 3 acounts, he doesn’t need to log in again. There is no user registration, choosing password, taking care of security and all that stuff. Just a simple click on the account the user wants to use, and he’s in.

User management is inevitable since I need to keep track on who write what messages. As the unique user ID, I use the email that is returned by the provider. OpenID (which all 3 of the providers support) returns me a unique identifier but that is comparatively difficult to type and remember. Email is a good compromise as it is something easily remembered and is unique.

Data modeling
I used DataMapper again for data modeling. The data model consists of 3 classes — User, Relationship and Chirp. Simply put — a User has Relationships with other Users, and a User has many Chirps (corny but, hey :)). A Chirp with a specific recipient User is a direct message.

Home page
All chirps sent out by the user and his friends are listed in his home page in a reverse timeline order (i.e. the latest chirp is listed at the top). Direct messages received are only shown in the direct message inbox and the direct messages sent are only shown in the sent box.

Friends and followers
The concept of friends and followers is a one way relationship. A friend is someone you follow, while a follower is someone following you. A user can be a friend or a follower or both. Following does not require the permission of the user. For example, you can follow anyone in the system without the explicit approval of that user.

Public messages (chirps) and direct messages
A chirp is a public message that is sent by a user that is viewable by the user and all his followers. Chirps belong to a user. Chirps containing the user ID (email) starting with ‘@’ will be hyperlinked to the user’s home page. 2 other features involving the message is direct messaging and following.

You can send direct message to specific users, who will receive them in their direct message inbox. To send a direct message start the chirp with ‘dm’, followed by the user ID. To follow a user, start the chirp with ‘follow’, followed by the user ID.

With this in mind let’s jump into the code. Here’s the stuff that I used:

Without looking the at view templates, the total number of lines of code is around 200 or so. I have only 2 non-view template files — chirp.rb is the web application while models.rb contains the 3 DataMapper data models. As we go along I’ll be extracting code fragments from these 2 files to explain in details.

Let’s talk about chirp.rb first. The first line after the requires tells Sinatra that I want to use sessions. This is followed by telling Sinatra to use the Rack::Flash library to use session-based flash. What you see next are a bunch of get and post blocks. Let’s look at the first one.

['/', '/home'].each do |path|
get path do
if session[:userid].nil? then
erb :login
else
redirect "/#{User.get(session[:userid]).email}"
end
end
end

You will notice that I wrap the get block with an array block, and the array contains 2 strings. These 2 strings are the routes that are associated with the subsequent get block. In the code above, we associate the ‘/’ and ‘/home’ routes to the get block. The get block here checks if the current session contains a user ID. If it doesn’t, it will redirect the user to the login view template. Otherwise it will get the user with this ID and redirect it to the route that contains the email. For example, if the user’s email is user@yahoo.com, then the route is ‘/user@yahoo.com’.

We’re going to look at the login module next. As mentioned, I used RPX (to be specific, I use RPX Basic, which is free!) to do single sign-on so to begin, go to RPXNow and register a web site. Once you do that, you will get an API key and that is really the only thing you need. For Windows Live ID, you’ll need to do a few more steps to register a Windows Live project but you can just follow the steps provided.

Now let’s look at the login template to understand more about the RPX single-sign on mechanism. You will notice that this is entirely HTML, and it contains 3 forms. The URLs for the single sign-on for Google and Yahoo! are OpenID based (which is not surprising as RPX is run by JanRain, one of the main supporters for OpenID). Each form has only 1 input, which is an image button. The user clicks on the account he wants to log in with, and following the given instructions, will be authenticated by the provider and redirected back to our web app. If the login is correct, our web app will receive some data on the user (the amount of data we get is different with different providers, and also configurable from RPX). I won’t go into the actual mechanism, the RPX documentation is pretty detailed.

Once RPX authenticates the user, it will redirect him to our web app along with a token. For Chirp-Chirp this goes back to ‘/login’. The ‘/login’ block then retrieves this token and uses it to retrieve the user information from RPX.

get '/login' do
openid_user = get_user(params[:token])
user = User.find(openid_user[:identifier])
user.update_attributes({:nickname => openid_user[:nickname], :email => openid_user[:email], :photo_url => "http://www.gravatar.com/avatar/#{Digest::MD5.hexdigest(openid_user[:email])}"}) if user.new_record?
session[:userid] = user.id # keep what is stored small
redirect "/#{user.email}"
end

For Chirp-Chirp, I store the unique identifier from RPX, the user’s nickname and his email. For his avatar picture, I use Gravatar which neatly also uses email address as a unique identifier although we need to use MD5 to hash it first. If this is the first time the user is logging in, I save the user information in the database. Finally I store the user’s serial ID (not the unique identifier) in the session and redirect the user to the user home page.

This is the code fragment that retrieves the user information from RPX.

def get_user(token)
u = URI.parse('https://rpxnow.com/api/v2/auth_info')
req = Net::HTTP::Post.new(u.path)
req.set_form_data({'token' => token, 'apiKey' => '<insert RPX API KEY HERE>', 'format' => 'json', 'extended' => 'true'})
http = Net::HTTP.new(u.host,u.port)
http.use_ssl = true if u.scheme == 'https'
json = JSON.parse(http.request(req).body)

if json['stat'] == 'ok'
identifier = json['profile']['identifier']
nickname = json['profile']['preferredUsername']
nickname = json['profile']['displayName'] if nickname.nil?
email = json['profile']['email']
{:identifier => identifier, :nickname => nickname, :email => email}
else
raise LoginFailedError, 'Cannot log in. Try another account!'
end
end

Let’s look at the home page next. I take the example of a user with the email user@yahoo.com, so the home page for him would be ‘/user@yahoo.com’. This is the code fragment with the get block for the home page:

get '/:email' do
@myself = User.get(session[:userid])
@user = @myself.email == params[:email] ? @myself : User.first(:email => params[:email])
@dm_count = dm_count
erb :home
end

The code is pretty self explanatory. The home page retrieves the currently logged in user (@myself) and if the requested user (@user) is not the currently logged in user, it will retrieve that use from the database as well. It also retrieves a count of the direct messages that this user has, and this is for display purposes.

def dm_count
Chirp.count(:recipient_id => session[:userid]) + Chirp.count(:user_id => session[:userid], :recipient_id.not => nil)
end

The number of direct messages is the number of direct messages sent and received.

Let’s look at the friending and following features.

get '/follow/:email' do
Relationship.create(:user => User.first(:email => params[:email]), :follower => User.get(session[:userid]))
redirect '/home'
end

This get block is called when you want to follow a particular user. Again, I just create a relationship between the person whom you want to follow and you. Deleting that relationship is also very simple.

delete '/follows/:user_id/:follows_id' do
Relationship.first(:follower_id => params[:user_id], :user_id => params[:follows_id]).destroy
redirect '/follows'
end

Note that this is a delete block and not get block any more. Finally to display the friends and followers, I have 2 separate routes, but I want to re-use the same code and view template:

['/follows', '/followers'].each do |path|
get path do
@myself = User.get(session[:userid])
@dm_count = dm_count
erb :follows
end
end

Next let’s look at the chirps and direct messages. To chirp, I use a post block:

post '/chirp' do
user = User.get(session[:userid])
Chirp.create(:text => params[:chirp], :user => user)
redirect "/#{user.email}"
end

However, to implement the various features like URL shortening and command level following and direct messaging, I placed the processing logic in the Chirp class itself.

before :save do
case
when starts_with?('dm ')
process_dm
when starts_with?('follow ')
process_follow
else
process
end
end

Before the chirp is save, I check if the text starts with ‘dm’ or ‘follow’ and I process the text accordingly. Let’s look at these various processing blocks of code.

def process_follow
Relationship.create(:user => User.first(:email => self.text.split[1]), :follower => self.user)
throw :halt # don't save
end

Implementing the ‘follow’ command in the chirp box is probably the easier. I just create the relationship and then throw halt to stop saving the chirp.

def process_dm
self.recipient = User.first(:email => self.text.split[1])
self.text = self.text.split[2..self.text.split.size].join(' ') # remove the first 2 words
process
end

Implementing direct message is also relatively simple. I just set the recipient of the message to be the person that is indicated in the chirp box and when saving the chirp, I remove the command. Finally I pass it on to the common processing block.

def process
# process url
urls = self.text.scan(URL_REGEXP)
urls.each { |url|
tiny_url = open("http://tinyurl.com/api-create.php?url=#{url[0]}") {|s| s.read}
self.text.sub!(url[0], "<a href='#{tiny_url}'>#{tiny_url}</a>")
}
# process @
ats = self.text.scan(AT_REGEXP)
ats.each { |at| self.text.sub!(at, "<a href='/#{at[2,at.length]}'>#{at}</a>") }
end

URL_REGEXP = Regexp.new('\b ((https?|telnet|gopher|file|wais|ftp) : [\w/#~:.?+=&%@!\-] +?) (?=[.:?\-] * (?: [^\w/#~:.?+=&%@!\-]| $ ))', Regexp::EXTENDED)
AT_REGEXP = Regexp.new('\s@[\w.@_-]+', Regexp::EXTENDED)

The common processing block scans the chirp text and does a few things:

  • I find the URLs in the code and replace it with a shortened URL from TinyURL.
  • I find all emails that has ‘@’ prefixed and I replace it with a link to that user email.

As for viewing direct messages, again I reuse the same view template, but for the sake of variety instead of having 2 routes, I used 1 route with a parameter:

get '/direct_messages/:dir' do
@myself = User.get(session[:userid])
case params[:dir]
when 'received' then @chirps = Chirp.all(:recipient_id => @myself.id)
when 'sent'     then @chirps = Chirp.all(:user_id => @myself.id, :recipient_id.not => nil)
end
@dm_count = dm_count
erb :direct_messages
end

In the case above, when I’m looking at the received direct messages (i.e. I’m looking at the inbox) I just want to find all chirps which recipient is myself. For the direct messages I sent, I find all chirps that belong to me, which recipient is not null.

That’s it! Of course there’s the view templates but that’s mainly HTML and some ERB snippets embedded in them. You can check out the entire repository at git://github.com/sausheong/chirp.git.

To run the app, get the code. Then go to the models.rb file and change the database URL accordingly. After that, go to irb, require the models file, and run this:

> DataMapper.auto_migrate!

This will create the necessary database. Then in the directory, run this

> ruby chirp.rb

Go to http://localhost:4567 and you’re up and running locally! Of course, you need to register your RPX account and then change login.erb accordingly.

To view this in a live site, go to http://chirp-chirp.heroku.com. This is deployed to Heroku, which is another amazing service, but that’s another story for another post.

About these ads

27 Responses

Subscribe to comments with RSS.

  1. Luca G. Soave said, on April 22, 2009 at 12:40 am

    Very nice pice of code here, I’d like to integrte that in a rails app of mine ( 2.3.2 rack + metal enabled ) Do you think is it possible call it from rails ? Do you have a suggestion on how to do it ?

    Thanks in advance
    Luca G. S oave

  2. Greg Moreno said, on April 22, 2009 at 1:03 pm

    @Luca, it wouldn’t be difficult porting this to Rails.

  3. sausheong said, on April 22, 2009 at 2:36 pm

    Hi Luca,

    It’s not difficult to take the code into Rails, in fact it should be pretty trivial. There is nothing special that Sinatra can do that Rails can’t. You can even change to ActiveRecord instead of DataMapper, their syntax is not radically different.

    • wuliwong said, on December 23, 2011 at 3:26 am

      In fact, the real point is that there is TONS that Rails does that Sinatra doesn’t. For some people that is a big plus for Sinatra and for others it’s a big minus.

  4. John Taylor said, on April 24, 2009 at 7:21 am

    Interesting blog post. What would you say was the most important marketing factor?

  5. subreto said, on September 9, 2009 at 11:04 pm

    What is Twitter and How Can I Use It?

  6. templerel said, on November 14, 2009 at 3:16 am

    Thank you for a very instructive article – more often I will go

  7. Joe said, on March 12, 2010 at 8:08 pm

    Seems like your session key has a huge security flaw. In your login route you set:

    session[:userid] = user.id

    But that’s just a serial primary key, which means it’s really easy to guess a valid session id (you wouldn’t know who you were impersonating, but you could easily impersonate *someone* else). It would be better to have separate LoginSession and User datamapper models, and do something like:

    session[:sessionid] = login_session.uuid

    Using a UUID makes it highly unlikely that one user can guess any other valid session identiier. It also allows easily expiring login sessions (delete the LoginSession records) without affecting users.

  8. Paul McClean said, on May 13, 2011 at 7:51 pm

    Sausheong, thank you for putting this together. You have a great knack for explaining things clearly and I find myself coming back to your site more and more as I’m getting deeper into Ruby.

  9. elbarto said, on November 16, 2012 at 11:51 pm

    application error ….

  10. science basedskills said, on November 23, 2012 at 11:56 pm

    Genuinely no matter if someone doesn’t understand then its up to other users that they will help, so here it occurs.

  11. parapharyngeal abscess said, on November 24, 2012 at 11:39 am

    I absolutely love your blog and find almost all
    of your post’s to be exactly I’m looking for.
    can you offer guest writers to write content for you personally?
    I wouldn’t mind publishing a post or elaborating on a few of the subjects you write in relation to here. Again, awesome weblog!

  12. Violet said, on December 7, 2012 at 5:50 pm

    I drop a leave a response when I appreciate a post on a website or if I have something to valuable to contribute to
    the discussion. Usually it’s caused by the passion displayed in the post I looked at. And on this article Write a Sinatra-based Twitter clone in 200 lines of Ruby code « saush. I was actually excited enough to drop a leave a responsea response :) I do have some questions for you if it’s
    okay. Could it be simply me or does it seem like some of the remarks
    appear like they are coming from brain dead visitors? :
    -P And, if you are posting on other social sites, I’d like to keep up with everything new you have to post. Would you list all of all your community pages like your twitter feed, Facebook page or linkedin profile?

  13. pictures of genital yeast infection in men said, on December 11, 2012 at 1:29 am

    For most recent information you have to visit world-wide-web and on the
    web I found this web site as a finest site for latest updates.

  14. how to treat hemorrhoids said, on December 11, 2012 at 7:16 pm

    I’m not sure where you’re getting your info, but great topic.
    I needs to spend some time learning much more
    or understanding more. Thanks for excellent info I was looking for this info for my mission.

    how to treat hemorrhoids and cure for hemroids and
    venapro reviews and also venapro pills

  15. stirling Solar said, on December 14, 2012 at 8:02 am

    After exploring a number of the articles on your website,
    I seriously appreciate your technique of writing a blog.
    I book marked it to my bookmark website list and will be checking back soon.

    Take a look at my website too and tell me what
    you think.

  16. resveratrol dosage said, on December 15, 2012 at 2:42 pm

    I constantly emailed this webpage post page to all my contacts, as if like to read it after that my contacts will too.

  17. http://thehollywoodcarshow.com said, on December 18, 2012 at 4:03 pm

    Thanks a lot for composing Write a Sinatra-based Twitter
    clone in 200 lines of Ruby code « saush, I just actually had
    been researching for anything related and was thankful to acquire the info via this content.

  18. Meghan said, on December 30, 2012 at 6:58 pm

    Searching in Google raised your website – I’m thankful it did, many thanks.

  19. hostgator business plan coupon code said, on February 10, 2013 at 7:30 pm

    I’m really enjoying the theme/design of your blog. Do you ever run into any browser compatibility issues? A few of my blog audience have complained about my blog not working correctly in Explorer but looks great in Safari. Do you have any tips to help fix this issue?

  20. Madeline said, on February 20, 2013 at 6:31 am

    I do that you will be elaborating more on this issue.
    I was hoping for a tad more information.

  21. Zachery said, on February 25, 2013 at 9:21 pm

    I do not know if it’s just me or if everybody else experiencing problems with your blog. It appears as if some of the written text on your posts are running off the screen. Can somebody else please comment and let me know if this is happening to them too? This may be a problem with my web browser because I’ve had
    this happen before. Thank you

  22. ceiling fans with lights and remote said, on March 19, 2013 at 12:54 pm

    More CFM is directly proportional to the cool air.
    Investing in this type of lighting unit is something
    that you should do for you can be sitting in your living room, dining room or bedroom and marveling
    at the lights and beautiful blades that they have plus you get to experience cool breezes at
    the same time. Manipulating the effects of lighting has
    since been known as one way of redecorating one’s office and house.

  23. kontaktannonser gratis said, on April 25, 2013 at 6:17 pm

    Det er også veldig sexy å ligge på ryggen, på en pute,
    og dra opp knærne til hver side. Registrer deg nå og arranger et knulletreff for i kveld.
    Ingenting er forpliktende her på knull kontakt og alt du trenger er en pc og internett.

  24. shorten website links said, on May 3, 2013 at 11:24 am

    Very quickly this site will be famous among all blog visitors,
    due to it’s good articles

  25. Weight Loss Motivation said, on May 11, 2013 at 12:44 am

    Wow, wonderful blog layout! How long have you been blogging for?
    you make blogging look easy. The overall look of your website is excellent, as
    well as the content!

  26. Hi there! I realize this is sort of off-topic but
    I needed to ask. Does managing a well-established blog like yours take a massive amount
    work? I am completely new to blogging however I do
    write in my diary on a daily basis. I’d like to start a blog so I can share my experience and feelings online. Please let me know if you have any kind of suggestions or tips for new aspiring blog owners. Thankyou!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 427 other followers

%d bloggers like this: