The perils of writing request specs using concurrent-ruby under the JVM

When I write an API, though I'm not a hard core TDD practitioner, I do like writing specs - especially requests specs that test the whole stack.

Adding them into an API is fast and yields quite good results compared to an app with an UI where you have to use chrome-cli or phantomjs just to get near of that level but at the cost of painfully slow execution time (yes, even if you optimise them to hell and back and get a runtime of five minutes - they're still slow in my book).

Anyhow, for some context - I've been writing Ruby APIs for quite some time but using the classic CRuby VM/interpreter - this time we switched to JRuby (thanks to Max ) since we needed to render extremely quick JSON responses (i.e. 202 status codes) - so everything from ActiveRecord CRUD operations to processing business logic is done using concurrent-ruby with Futures and ThreadPools (which in JRuby they use Java's native implementation - awesome stuff since one can play with STM and all that good stuff - if required).

Next stop: the core of the issue: running request specs with all this parallelism in the background breaks most of our specs because RSpec is not aware of anything which runs in parallel and the inherent non-deterministic nature of running code in parallel.

The initial fix for this was to add a helper that basically forces a shutdown of the ThreadPool which will block until it's done - this "ensured" that all the parallel tasks (like creating a record) would finish before we got to the expect part in the specs. All nice and rosy except it did not work as expected.

This was the initial implementation btw. :

def wait_for_thread_pool!(sleep_time = nil)
  thread_pool_executor = executor.instance
  pool = thread_pool_executor.executor

  # shutdown the pool and wait as long as it takes
  pool.shutdown
  pool.wait_for_termination

  # we want a fresh thread pool for the next test
  thread_pool_executor.send(:initialize)
end

This is quite self-explanatory - it issues a ThreadPool shutdown which waits for all the threads to finish their work - the problem? In 10% of the cases using RSpec random spec execution it failed.

For the time being - the next best fix was to monkey patch the library (well, to be more precise our adapter for it) in order to run everything sequentially - especially keeping in mind that concurrent-ruby is an externally tested library and the issue should be an odd interaction between RSpec, DatabaseCleaner and concurrent-ruby.

In any case until I can start digging into all those dependencies and see what the actual issue was this fixed the issue:

# run everything sequantial for now
def disable_parallelism!(context: nil)
  case context
  when :context_a
    future.instance_eval do
      def execute(executor: nil, &block)
        block.call
      end
    end
  when :context_b
    future.class_eval do
      def add_observer(observer)
        @observer = observer
      end

      def initialize(opts = {}, &block)
        @block_to_call = block
      end

      def execute
        proc_response = @block_to_call.call
        @observer.update(time.now, proc_response)
      end
    end
  end
end

I usually avoid monkey patching stuff like this - but then again this is also where monkey patching can give you some breathing time and keep you sane whilst the long process of debugging why:

pool.shutdown
pool.wait_for_termination

it's not doing its job properly can continue as an async (pun intended) task.

NB: please read the Updates section below on how this was fixed - to completely avoid the monkey-patching we can use:

def disable_parallelism!
  CustomExecutor.instance.executor = Concurrent::ImmediateExecutor.new
end

Regarding specs/coverage - the next step is to improve code-coverage in the unit-tests realm as to be sure all the small parts are working correctly.

Conclusion

JRuby + concurrent-ruby is a boon - there are some drawbacks as pointed above and many more which I'll detail in future posts but it really pays off when you really need quick responses. En plus, concurrent-ruby literally levels up JRuby to a place where it can compete with Elixir, well Erlang's concurrency model (yes it has Actors but they're in the edge branch for now). Its abstractions are top notch and they remove a lot of pain from dealing with Threads with some minor caveats.

Updates

After the r/ruby discussion some updates are required:

A fellow redditor - i_know_sherman suggested concurrent-ruby's ImmediateExecutor which is a special executor that basically runs everything sequentially:

An executor service which runs all operations on the current thread, blocking as necessary. Operations are performed in the order they are received and no two operations can be performed simultaneously.

A small caveat here: it works correctly except when using Futures with the option dup_on_deref set to true.

Another suggestion from moomaka - this time for keeping parallelism on:

Problem may be that sometimes the tasks in the queue needs to queue other tasks on the same executor and those tasks are being rejected. May be able to fix this by setting the :fallback_policy of the executor to :caller_runs which will then run the rejected tasks immediately in the caller thread allowing everything to complete.

This feature was implemented in concurrent-ruby by jrochkind here i.e. Improve behaviour when posting to a shutdown thread pool

Tagged under: