We help our clients to roll-out MVP as earliest as possible using a unique Rapid Prototyping approach of our Canvas to Cloud methodology. For that purpose, we use tools and technologies which are easy to learn and integrate. So delayed_job with ActiveJob has been our first choice for any MVP being built at BoTree for all background processing.

Once clients see traction in the MVP, they want us to build the enterprise solution and scale it. By nature, delayed_job doesn’t scale well for large number of jobs because of following limitations:

  1. Stores jobs in a database table, it slows down your application if there is a huge backlog of pending/failed jobs.
  2. Spin off processes instead of thread for each job.

To overcome this limitation in terms of scalability, we have a few choices like Sidekiq, Resque, Sucker Punch etc. for background processing and Sidekiq has always been our choice for following reasons:

  • Stores jobs in Redis (in-memory database) which is inherently faster over other databases.
  • Spin off very lightweight workers for each job to conserve resources.
  • Provides Web UI to monitor the job processing.

Here are a few simple steps to replace delayed_job with Sidekiq:

Step 1: Update the Gemfile


gem 'delayed_job_active_record'


gem 'delayed_job_mongoid'


gem 'sidekiq'

and bundle the application again.

Step 2: Change ActiveJob queue adapter in config/application.rb


config.active_job.queue_adapter = :delayed_job


config.active_job.queue_adapter = :sidekiq

Step 3: Create config/sidekiq.yml and configure queues

  - [default, 1]

You can configure as many queues with priority as required here.

Step 4: Start Sidekiq

We can start delayed_job with following command

bin/delayed_job start

and Sidekiq with following command from the application root


Step 5. Cleanup

delayed_job stores jobs in delayed_jobs table which now no longer required. So write a migration to drop this table

class DropDelayedJobTable < ActiveRecord::Migration
  def up
    drop_table :delayed_jobs

  def down
    raise ActiveRecord::IrreversibleMigration

and run the migration to free up some space.

Additional Changes


If you are using devise and send notification emails to users in background, then you need to make following changes for Sidekiq

Step 1: Modify config/initializers/devise_async.rb


Devise::Async.backend = :delayed_job


Devise::Async.backend = :sidekiq

Step 2: Add mailer queue to config/sidekiq.yml

  - [mailer, 2]
  - [default, 1]

Steps 3: Make sure you override following method in User model
def send_devise_notification(notification, *args)
  devise_mailer.send(notification, self, *args).deliver_later


If you deploy application using capistrano then you need to do the following steps to replace delayed_job with Sidekiq

Step 1: Modify Gemfile


gem 'capistrano3-delayed-job', require: false


gem 'capistrano-sidekiq', require: false

and bundle the application.

Step 2: Modify Capfile


require 'capistrano/delayed-job'


require 'capistrano/sidekiq'

Step 3: Modify config/deploy.rb

There is a known bug that prevents sidekiq from starting when pty is true on Capistrano 3.

Set capistrano environment variable pty for that reason

set :pty, false

and then deploy the application again.

Step 4: Manage remote Sidekiq from local machine

cap production sidekiq:start            # Start sidekiq
cap production sidekiq:stop             # Stop sidekiq
cap production sidekiq:restart          # Restart sidekiq

Sidekiq Web

Let us enable this useful visual interface to monitor the jobs. Add following configuration in routes.rb of your application.