There comes a point where the application you’re building needs to run fast. People do not like waiting for screens to load. They need instant gratification. Not to worry, because background jobs are meant to take care of your computationally complex tasks so you can go on delighting your users.
Sidekiq’s initial setup is very simple. Just add it to your
bundle install and then
Sidekiq depends on Redis for storage and processing, so make sure you have that installed on your machine and run
redis-server in another terminal tab.
You should also add this initializer to
This file determines which Redis server to connect to.
ENV["REDISTOGO_URL"] is an environment variable that will be set on your production server in order to use the Redis To Go addon on Heroku.
Note that starting in version 3 of Sidekiq,
ENV["REDISTOGO_URL"] is not recognized. You have to run
heroku config:set REDIS_PROVIDER=REDISTOGO_URL in order for Sidekiq to properly work with Redis on Heroku.
If you do not have Redis installed, run
brew install redis.
bundle exec sidekiq.
This is definitely a tedious way to start your Rails app, so I recommend using Foreman to streamline the process. I’ll write another post on how to set it up.
The most common use-case for background jobs is sending emails. This post assumes that you already know how to set up a vanilla Rails mailer. If that’s not the case, read this short guide.
In a project I’m working on, I need to send a notification email to every user included in a conversation:
Pretty straightforward. Send a notification email to each user. The problem is that if you have a more than a handful of people to notify, the request can hang for seconds at a time. It gets worse the more people that are sent the email. This is because Rails will not redirect until all of the emails have been processed and sent. Sidekiq mitigates the issue by processing them on a Redis server in the background, so Rails can continue with the request.
All that needs to change in the controller is the call to
By using Sidekiq’s
delay method, you are putting the task on a queue to be processed in the background immediately, and Rails does not have to process the task. Now, the user does not have to wait for all the emails to send to continue using the app.
Sidekiq has some other helpful methods:
Other types of background processing
Sidekiq does not only take care of processing emails in the background. It can process anything you tell it to. This can be helpful when you have a service in your app that performs non-trivial amounts of communication with the database.
In a project I’m currently working on, I have service called
It gets called in the controller like so:
This works fine, but since it creates a bunch of ActiveRecord objects, I thought it would be more efficient to move it into the background. This time, I’ll be using a Sidekiq worker:
Sidekiq workers will execute any code you put in the
perform method. You can pass whatever arguments you wish. Note that I chose to pass in
user_id and finding the
user after the worker has started. I did this as opposed to passing in a full object because
perform serializes all arguments into JSON to send to Redis. By saving state to Redis, you risk differences between what is saved in your database, and what Redis sends from the queue back to your
perform. The wiki talks about this in greater detail.
In the controller:
Now I have my registration service run in the background whenever a user is created.
Working with background jobs and Sidekiq has shown me that just because Rails is MVC, does not mean you can’t use plain Ruby to make the architecture of your app make sense.
Job queues are now built into Rails 4.2. The framework, called ActiveJob, will serve as an adapter for many of the popular job queues out there, including Sidekiq. This means that you can write the same code to interact with Sidekiq as you would with another system like Resque. You can change the queuing backend without re-writing any code.
Another awesome library built into Rails 4.2 is GlobalId. GlobalId takes out the headache of having to pass ids into your jobs to use ActiveRecord objects within the job. GlobalId lets you pass an entire ActiveRecord object by using a global identifier in the form of a URI, and it takes care of the JSON serializaion for you.