This article shows how we implemented background ActiveRecord model indexing using Sidekiq. By default, the SearchKick gem allows for 4 ways of automatically indexing a Rails model when it is saved to the backing database:

1. Inline (the default, synchronously, anytime a record is inserted, updated, or deleted)
2. Asynchronous (using ActiveJob)
3. Asynchronous (using a Redis Queue)
4. Manual (i.e. disabled automatic syncing)

Since we are using Sidekiq for our background jobs, we clearly needed a custom way to automatically index our models. Enter the Indexable concern:

#
# Adds support for calling a sidekiq job to index the model 
# that includes this module.
#
module Indexable
  extend ActiveSupport::Concern

  included do
    def index_elasticsearch
      ElasticSearchWorker.perform_async(self.class.to_s, id)
      self
    end
  end
end

Indexing the model when it updates then becomes relatively generic and straightforward:

class Vehicle < ActiveRecord::Base
  include Indexable

  searchkick callbacks: false
  after_commit :index_elasticsearch

  def search_data
    {
      brand: self.brand,
      licence_plate: self.licence_plate,
      (...)
    }
  end

  (...)
end

Notice that:

1. We include the concern in the model
2. We instruct SearchKick to deactivate automatic indexing (by setting callbacks = false)
3. We tell Rails to run the Indexable concern’s index_elasticsearch method whenever there is a database commit on this model
4. And finally we just define the Hash we want to index regarding this model (in our example just the brand and licence plate)

There’s only one missing piece of the puzzle, which is the Sidekiq worker to index any models we mark as needing reindexing:

class ElasticSearchWorker
  include Sidekiq::Worker
  sidekiq_options queue: :elastic_search, retry: 10

def perform(clazz, id)
    Rails.logger.debug("Indexing #{clazz} id #{id}")
    clazz.constantize.find(id).reindex
  end
end

In this file we basically just define which queue name we want for these jobs and call SearchKick’s reindex method on the model we are indexing.

As long as you have sidekiq up and running, its workers should pickup these jobs and index your data.

Happy indexing!

Background Elastic Search indexing using Sidekiq was originally published in Drover Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.