Easy Benchmarking with Ruby on Rails MiniTest

12 Jan 2016

Do you want a native benchmarking tool for Rails 4? Good luck.

You can use the fantastic tool, wrk - “a HTTP benchmarking tool”. But, I wanted to run the benchmarks in my test suite.

Rails 4 removed the benchmarking test class ActionDispatch::PerformanceTest.

Needing a benchmarking tool, I looked around for a Ruby gem. I found several gems; the rails-perftest gem is the closest one I found to what I wanted, but I ran into bugs with Rails 4.2.x.

I’m unsure if those bugs are specific to Rails 4.2 or not. And, I don’t care.

Then I found the largely undocumented Benchmark class. More Benchmark documentation is here.

The Benchmark class basically compares start and end time and finds the delta:

beginning_time = Time.now
(1..100).each { |i| i }
end_time = Time.now
puts "Time elapsed #{(end_time - beginning_time)*1000} ms"

Knowing this, lets add a new method to our test_helper.rb .

# test/test_helper.rb
class ActiveSupport::TestCase
  ...
  def benchmark_with_limit(timeout = 150, &block)
    return nil unless block_given?
    actual = Benchmark.ms { @response = yield }
    assert(actual < timeout, "Response is too slow: Limit was #{timeout}, actual was #{actual}")
    @response
  end
  ...
end

Setting and returning @response will allow us to set the original block’s response as a testable variable in our test class.

Using it in the controller tests:

class UsersControllerTest < ActionController::TestCase

  test '#create persists new user' do
    payload = { name: 'Bob',
                email: 'bob@aol.com'
                format: :json }

    # using default benchmark time
    benchmark_with_limit { post :create, payload }

    assert_response 201

    actual = JSON.parse @response.body

    assert_kind_of(Hash, actual, 'Unexpected response type')
    assert_equal(payload[:name], actual['name'], 'Unexpected name')
    ...
  end

  test '#destroy deletes user' do
    user = Fabricate(:user)

    # with do block
    benchmark_with_limit do
      delete :destroy, {id: user.id, format: :json}
    end

    assert_response 200
  end
end

So far so good. Lets add the same to our model tests.

class OrderTest < ActiveSupport::TestCase

  test '#new should persist new order' do

      order_details = { ... }

      # Setting a custom benchmark limit of 250 ms
      order = benchmark_with_limit(250) do
        Order.new(order_details)
      end

      # we set the output of the block as variable order so we can continue testing against it.

      assert order.valid?, 'Order is not valid'
      ...
  end
end

And that’s it! We’ve added some light-weight benchmarking to ensure out app stays performant.