Ruby 2.2.0 Preview 1 quick Rails benchmarks

Great news everyone! Ruby 2.2 preview 1 has been released ! I'm really curios about the Incremental GC and Symbol GC so let's run some quick Rails benchmarks on a normal Rails API app.

First off let's install the preview via RVM:

rvm install ruby-2.2.0-preview1

After fiddling around about five minutes trying to find a part of the application that doesn't fail under the Preview I stopped at the simple /profiles endpoint that just renders a JSON of all profiles, quite simple indeed. Using the trusty wrk I fired up a quick bench:

wrk -t10 -c10 -d20s http://localhost:8080/profiles

The results are as follows:

Ruby 2.1.2p95

Running 20s test @ http://localhost:8080/profiles
  10 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   255.02ms   25.10ms 372.80ms   67.61%
    Req/Sec     3.21      0.70     5.00     71.13%
  771 requests in 20.01s, 4.40MB read
Requests/sec:     38.53
Transfer/sec:    225.31KB
------------------------------
50%,252 ms
90%,285 ms
99%,328 ms
99.999%,372 ms

Ruby 2.2.0preview1

Running 20s test @ http://localhost:8080/profiles
  10 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   253.27ms   30.64ms 344.75ms   64.21%
    Req/Sec     3.34      0.70     5.00     89.63%
  786 requests in 20.02s, 4.49MB read
Requests/sec:     39.26
Transfer/sec:    229.60KB
------------------------------
50%,251 ms
90%,291 ms
99%,329 ms
99.999%,344 ms

I'm not really sure that I should interpret them yet it seems that under the Preview we have a slight improvement but within margins of error. At this point I don't think is the best benchmark for the Preview as we don't use views thus Rails won't bloat up the memory with Strings.

On the memory usage side we have 65M vs 75M (Preview vs. 2.1) so in this scenario we clearly have a winner.

note: this was measured using OSX's Activity Monitor after wrk finished the benchmark and it's the average of the unicorn workers sizes.

Issues

Bundler and all the gems installed without issue but in some cases I got silent failures. The benchmarks were run on an actual working/production Rails 4.0.x app with around 25 gems. Nonetheless all of the gems installed and I could boot up Rails with unicorn and benchmark the simpler endpoints which is great.

Conclusion

TBD - this is work in progress I will update it with more information

Tagged under: