There comes a point where the application you’re building needs to run fast. People do not like waiting for screens to load. They need instant gratification. Not to worry, because background jobs are meant to take care of your computationally complex tasks so you can go on delighting your users.

Setup

Sidekiq’s initial setup is very simple. Just add it to your Gemfile:

  gem 'sidekiq'

Run bundle install and then rails server.

Sidekiq depends on Redis for storage and processing, so make sure you have that installed on your machine and run redis-server in another terminal tab.

You should also add this initializer to config/initializers:

url = URI.parse(ENV["REDISTOGO_URL"] || "redis://localhost:6379/")
REDIS = Redis.new(url: url)

This file determines which Redis server to connect to. ENV["REDISTOGO_URL"] is an environment variable that will be set on your production server in order to use the Redis To Go addon on Heroku.

Note that starting in version 3 of Sidekiq, ENV["REDISTOGO_URL"] is not recognized. You have to run heroku config:set REDIS_PROVIDER=REDISTOGO_URL in order for Sidekiq to properly work with Redis on Heroku.

If you do not have Redis installed, run brew install redis.

Finally, run bundle exec sidekiq.

This is definitely a tedious way to start your Rails app, so I recommend using Foreman to streamline the process. I’ll write another post on how to set it up.

E-mail notifications

The most common use-case for background jobs is sending emails. This post assumes that you already know how to set up a vanilla Rails mailer. If that’s not the case, read this short guide.

In a project I’m working on, I need to send a notification email to every user included in a conversation:

  class PostsController < ApplicationController
    #...

    def create
      @post = Post.new(post_params)
      
      if @post.save
        users_in_conversation.each do |user|
          NotificationMailer.send_notification(user).deliver
        end
      end
    end
  end

Pretty straightforward. Send a notification email to each user. The problem is that if you have a more than a handful of people to notify, the request can hang for seconds at a time. It gets worse the more people that are sent the email. This is because Rails will not redirect until all of the emails have been processed and sent. Sidekiq mitigates the issue by processing them on a Redis server in the background, so Rails can continue with the request.

All that needs to change in the controller is the call to send_notification:

   class PostsController < ApplicationController
    #...

    def create
      @post = Post.new(post_params)
      
      if @post.save
        users_in_conversation.each do |user|
          NotificationMailer.delay.send_notification(user)
        end
      end
    end
  end

By using Sidekiq’s delay method, you are putting the task on a queue to be processed in the background immediately, and Rails does not have to process the task. Now, the user does not have to wait for all the emails to send to continue using the app.

Sidekiq has some other helpful methods:

  NotificationMailer.delay_for(30.seconds).send_notification(user)

  # OR

  NotificationMailer.delay_until(1.day.from_now)

Other types of background processing

Sidekiq does not only take care of processing emails in the background. It can process anything you tell it to. This can be helpful when you have a service in your app that performs non-trivial amounts of communication with the database.

In a project I’m currently working on, I have service called RegistrationService:

  class RegistrationService

    def register_user(user)
      # create a bunch of ActiveRecord models
    end

  end

It gets called in the controller like so:

class UsersController < ApplicationController
  #...

  def create
    @user = User.new(user_params)

    if @user.save
      registration_service = RegistrationService.new 
      registration_service.register_user(@user)
    end
  end
end

This works fine, but since it creates a bunch of ActiveRecord objects, I thought it would be more efficient to move it into the background. This time, I’ll be using a Sidekiq worker:

  class RegistrationWorker
    include Sidekiq::Worker
    
    def perform(user_id)
      user = User.find(user_id) 
      registration_service = RegistrationService.new
      registration_service.register_user(@user)
    end
  end

Sidekiq workers will execute any code you put in the perform method. You can pass whatever arguments you wish. Note that I chose to pass in user_id and finding the user after the worker has started. I did this as opposed to passing in a full object because perform serializes all arguments into JSON to send to Redis. By saving state to Redis, you risk differences between what is saved in your database, and what Redis sends from the queue back to your perform. The wiki talks about this in greater detail.

In the controller:

  #...

  def create
    @user = User.new(user_params)

    if @user.save
      RegistrationWorker.perform_async(@user.id)
    end
  end

Now I have my registration service run in the background whenever a user is created.

Conclusion

Working with background jobs and Sidekiq has shown me that just because Rails is MVC, does not mean you can’t use plain Ruby to make the architecture of your app make sense.

Update

Job queues are now built into Rails 4.2. The framework, called ActiveJob, will serve as an adapter for many of the popular job queues out there, including Sidekiq. This means that you can write the same code to interact with Sidekiq as you would with another system like Resque. You can change the queuing backend without re-writing any code.

Another awesome library built into Rails 4.2 is GlobalId. GlobalId takes out the headache of having to pass ids into your jobs to use ActiveRecord objects within the job. GlobalId lets you pass an entire ActiveRecord object by using a global identifier in the form of a URI, and it takes care of the JSON serializaion for you.