The perils of writing request specs using concurrent-ruby under the JVM

This article describes a 2017 JRuby, concurrent-ruby, and RSpec setup. Current APIs, integrations, and behavior may differ, so the workaround below should be read as historical rather than current guidance.

When I write an API, though I'm not a hard core TDD practitioner, I do like writing specs - especially request specs that test the whole stack.

Adding them into an API is fast and yields quite good results compared to an app with a UI where you have to use chrome-cli or phantomjs just to get near that level but at the cost of painfully slow execution time (yes, even if you optimise them to hell and back and get a runtime of five minutes - they're still slow in my book).

Anyhow, for some context - I've been writing Ruby APIs for quite some time but using the classic CRuby VM/interpreter - this time we switched to JRuby (thanks to Max ) since we needed to render extremely quick JSON responses (i.e. 202 status codes) - so everything from ActiveRecord CRUD operations to processing business logic is done using concurrent-ruby with Futures and ThreadPools (which on JRuby use Java's native implementation - awesome stuff since one can play with STM and all that good stuff - if required).

Next stop: the core of the issue: running request specs with all this parallelism in the background breaks most of our specs because RSpec is not aware of anything which runs in parallel and the inherent non-deterministic nature of running code in parallel.

The initial fix for this was to add a helper that basically forces a shutdown of the ThreadPool which will block until it's done - this "ensured" that all the parallel tasks (like creating a record) would finish before we got to the expect part in the specs. All nice and rosy except it did not work as expected.

The failing synchronization approach

This was the initial implementation, by the way:

def wait_for_thread_pool!(sleep_time = nil)
  thread_pool_executor = executor.instance
  pool = thread_pool_executor.executor

  # shutdown the pool and wait as long as it takes
  pool.shutdown
  pool.wait_for_termination

  # we want a fresh thread pool for the next test
  thread_pool_executor.send(:initialize)
end

This is quite self-explanatory - it issues a ThreadPool shutdown which waits for all the threads to finish their work - the problem? In 10% of the cases using RSpec random spec execution it failed.

Running tests sequentially

For the time being - the next best fix was to monkey patch the library (well, to be more precise our adapter for it) in order to run everything sequentially - especially keeping in mind that concurrent-ruby is an externally tested library and the issue should be an odd interaction between RSpec, DatabaseCleaner and concurrent-ruby.

In any case until I can start digging into all those dependencies and see what the actual issue was this fixed the issue:

# Run everything sequentially for now
def disable_parallelism!(context: nil)
  case context
  when :context_a
    future.instance_eval do
      def execute(executor: nil, &block)
        block.call
      end
    end
  when :context_b
    future.class_eval do
      def add_observer(observer)
        @observer = observer
      end

      def initialize(opts = {}, &block)
        @block_to_call = block
      end

      def execute
        proc_response = @block_to_call.call
        @observer.update(time.now, proc_response)
      end
    end
  end
end

I usually avoid monkey patching stuff like this - but then again this is also where monkey patching can give you some breathing time and keep you sane whilst the long process of debugging why:

pool.shutdown
pool.wait_for_termination

not doing its job properly can continue as an async (pun intended) task.

NB: please read the Updates section below on how this was fixed - to completely avoid the monkey-patching we can use:

def disable_parallelism!
  CustomExecutor.instance.executor = Concurrent::ImmediateExecutor.new
end

Regarding specs/coverage - the next step is to improve code-coverage in the unit-tests realm to ensure all the small parts are working correctly.

Conclusion

JRuby + concurrent-ruby is a boon - there are some drawbacks as pointed out above and many more which I'll detail in future posts but it really pays off when you really need quick responses. En plus, concurrent-ruby literally levels up JRuby to a place where it can compete with Elixir, well Erlang's concurrency model (yes it has Actors but they're in the edge branch for now). Its abstractions are top-notch and they remove a lot of pain from dealing with Threads with some minor caveats.

Updates

After the r/ruby discussion some updates are required:

A small caveat here: it works correctly except when using Futures with the option dup_on_deref set to true.

This feature was implemented in concurrent-ruby by jrochkind here i.e. Improve behaviour when posting to a shutdown thread pool

Tagged under: