The half-life of a programmer

Half-life (t1/2) is the amount of time required for the amount of something to fall to half its initial value. The term is very commonly used in nuclear physics to describe how quickly unstable atoms undergo radioactive decay, but it is also used more generally for discussing any type of exponential decay.

I had this almost random idea of when a programmer hits his or hers career half-life.

Tangents

First a few points that need explaining: almost random is in the sense that next year this time I'll be hitting thirty (Faith No More track redirect is already playing in my head).

The next point is how one should understand the word "career" in the context of the field I'm working in. The word itself makes me think of a reductionist fifties like view of how one works and/or deals with work.

This begets the question "What does 'career' mean for a programmer?" - tricky question and interesting tangent: methinks it refers to that period of time when one starts to work professionally (getting paid) to the point in time when one stops doing that.

Let's jump on another tangent: "Why is the end of a programming career when one stops writing code?" - clearly one could easily jump on a management position and still call oneself a programmer - that for me is not programming any more, maybe I'm a purist and yes I'm not fond of labels like "team leader", "CTO", "lead developer", "senior developer" and all that paraphernalia that in the end is just utter non-sense.

Relation to age

If one looks around the "average" age or more likely the anecdotal age (yes this is not even proper empiric data) of developers in shiny start-ups is around twenty seven, I reckon. And it makes sense - they're relatively fresh out of university programmers with a decent experience on that hi(y)ppie new programming language or paradigm.

I don't want to turn this into a rant, methinks I was also in that situation and it's a sane approach as long as one doesn't go into extremes (like using MongoDB, OrientDB, InfluxDB, etc. for a problem's domain where a relational DB is eye melting clear).

In any case, years pass, one gets experience and more importantly, experience at a higher level i.e. when one can much more easily understand the inner-workings of complex systems. The problem that stems out of this is that at some point work gets repetitive (if you let it) and one can easily get entrenched in a tool-chain.

The half-life of a programmer

I'm starting to feel this is the crux of the problem: a programmer hits half-life when he realizes that most of the work he does is repetitive and he or she gets entrenched in a tool-chain. Clearly, if one cannot overcome these slight issues then we have a problem that might lead to a non-programming career or an illusory one: "team leader", "CTO", "lead developer", "senior developer", "consultant" etc.

The issue with those terms is that some programmers start to take it way too seriously whilst completely ignoring what I like to call referential humility.

Most of my heroes: the likes of Jim Weirich redirect, Sandi Metz redirect and more seem to be in for life (the career) and I find that very reassuring.

Referential humility

What does that mean in the end? - just a play of words from referential integrity:

Referential integrity is a property of data which, when satisfied, requires every value of one attribute (column) of a relation (table) to exist as a value of another attribute in a different (or the same) relation (table).

In other words: there's always something "new" or something we still don't have a grasp of - hence the end of the road in a programmer's career is virtually non-existent.

Conclusion

Once you hit this half-life - choose wisely, work in the end is just another distraction from death, it's in our best interest to make that distraction authentic.

Vaguely related to the distraction from death tangent:

Ruby 2.2.0 Preview 1 quick Rails benchmarks

Great news everyone! Ruby 2.2 preview 1 has been released redirect! I'm really curios about the Incremental GC redirect and Symbol GC redirect so let's run some quick Rails benchmarks on a normal Rails API app.

First off let's install the preview via RVM:

rvm install ruby-2.2.0-preview1

After fiddling around about five minutes trying to find a part of the application that doesn't fail under the Preview I stopped at the simple /profiles endpoint that just renders a JSON of all profiles, quite simple indeed. Using the trusty wrk redirect I fired up a quick bench:

wrk -t10 -c10 -d20s http://localhost:8080/profiles

The results are as follows:

Ruby 2.1.2p95

Running 20s test @ http://localhost:8080/profiles
  10 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   255.02ms   25.10ms 372.80ms   67.61%
    Req/Sec     3.21      0.70     5.00     71.13%
  771 requests in 20.01s, 4.40MB read
Requests/sec:     38.53
Transfer/sec:    225.31KB
------------------------------
50%,252 ms
90%,285 ms
99%,328 ms
99.999%,372 ms

Ruby 2.2.0preview1

Running 20s test @ http://localhost:8080/profiles
  10 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   253.27ms   30.64ms 344.75ms   64.21%
    Req/Sec     3.34      0.70     5.00     89.63%
  786 requests in 20.02s, 4.49MB read
Requests/sec:     39.26
Transfer/sec:    229.60KB
------------------------------
50%,251 ms
90%,291 ms
99%,329 ms
99.999%,344 ms

I'm not really sure that I should interpret them yet it seems that under the Preview we have a slight improvement but within margins of error. At this point I don't think is the best benchmark for the Preview as we don't use views thus Rails won't bloat up the memory with Strings.

On the memory usage side we have 65M vs 75M (Preview vs. 2.1) so in this scenario we clearly have a winner.

note: this was measured using OSX's Activity Monitor after wrk finished the benchmark and it's the average of the unicorn workers sizes.

Issues

Bundler and all the gems installed without issue but in some cases I got silent failures. The benchmarks were run on an actual working/production Rails 4.0.x app with around 25 gems. Nonetheless all of the gems installed and I could boot up Rails with unicorn and benchmark the simpler endpoints which is great.

Conclusion

TBD - this is work in progress I will update it with more information

Improve Rails performance by adding a few gems

Working with Rails for some time you start nitpicking on how to improve it. This is a first in the series of articles regarding on how to improve (even marginally) Rails's performance.

I'll focus on a bunch of gems that speed up, in some cases considerably, small parts of Rails, like the html escaping, the String#blank? and JSON utils.

Benchmarking methodology

Methodology is a strong word for just running a couple of times in the console wrk but I'm not searching for the holy grail here, just to get a raw idea.

I switched from the old apache ab to wrk redirect:

wrk is a modern HTTP benchmarking tool capable of generating significant
load when run on a single multi-core CPU.

wrk -t10 -c10 -d10s http://localhost:3000

This runs a benchmark for 10 seconds, using 10 threads, and keeping 50 HTTP connections open i.e. this should suffice. Just remember to benchmark on your actual app to see the real improvements.

The escape_utils gem

Just faster all html escaping via the lovely escape_utils redirect gem. In order to use it in Rails one needs to add an initializer that patches things up:

begin
  require 'escape_utils/html/rack' # to patch Rack::Utils
  require 'escape_utils/html/erb' # to patch ERB::Util
  require 'escape_utils/html/cgi' # to patch CGI
  require 'escape_utils/html/haml' # to patch Haml::Helpers
rescue LoadError
  Rails.logger.info 'Escape_utils is not in the gemfile'
end

The logic to test it:

def escape_utils
  @escape_me = <<-HTML
    <body class="application articles_show">
      <!-- Responsive navigation
      ==================================================== -->
      <div class="container">
        <nav id="nav">
      <ul>
        <li><a href="/"><i class="ss-standard ss-home"></i>home</a></li>
        <li><a href="/home/about"><i class="ss-standard ss-info"></i>about</a></li>
        <li><a href="/contact"><i class="ss-standard ss-ellipsischat"></i>contact</a></li>
        <li><a href="/home/projects"><i class="ss-standard ss-fork"></i>projects</a></li>
        <li><a href="/tags"><i class="ss-standard ss-tag"></i>tags</a></li>
        <li><a href="/articles?query=code"><i class="ss-standard ss-search"></i>search</a></li>
      </ul>
    </nav>
    <a href="#" class="ss-standard ss-list" id="nav-toggle" aria-hidden="true"></a>
  HTML

  render inline: "Hello  world <%= @escape_me %>"
end

With standard Rails:

Running 10s test @ http://localhost:3000/sidechannels/bench
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    35.40ms    3.55ms  64.70ms   91.98%
    Req/Sec   142.19     11.68   164.00     83.12%
  2837 requests in 10.00s, 4.92MB read
Requests/sec:    283.61
Transfer/sec:    503.34KB

With the escape_utils gem:

Running 10s test @ http://localhost:3000/sidechannels/bench
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    34.06ms    3.89ms  63.92ms   89.10%
    Req/Sec   148.65     13.36   180.00     75.94%
  2960 requests in 10.00s, 5.46MB read
Requests/sec:    295.98
Transfer/sec:    558.72KB

The fast_blank gem

Living under the impression that the blank? method is too slow? say no more and just try the fast_blank redirect gem!

Just add gem 'fast_blank' to your Gemfile and this should speed up quite nicely the String#blank? method as described in this article redirect. For testing I just added this code:

fast_blank is a simple extension which provides a fast implementation of active support's string#blank? function

  def fast_blank_test
    n = 1000

    strings = [
      "",
      "\r\n\r\n  ",
      "this is a test",
      "   this is a longer test",
      "   this is a longer test
      this is a longer test
      this is a longer test
      this is a longer test
      this is a longer test"
    ]

    Benchmark.bmbm  do |x|
      strings.each do |s|
        x.report("Fast Blank #{s.length}    :") do
          n.times { s.blank? }
        end
      end
    end

    render nothing: true
  end

With standard Rails:

Running 10s test @ http://localhost:3000/sidechannels/bench
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.40s   207.72ms   1.58s    92.68%
    Req/Sec     3.10      2.11     6.00     53.66%
  69 requests in 10.01s, 33.08KB read
Requests/sec:      6.90
Transfer/sec:      3.31KB

With the fast_blank gem:

Running 10s test @ http://localhost:3000/sidechannels/bench
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.33s   179.56ms   1.41s    93.33%
    Req/Sec     3.07      0.80     4.00     40.00%
  72 requests in 10.00s, 34.52KB read
Requests/sec:      7.20
Transfer/sec:      3.45KB

The oj gem

# oj gem
gem 'oj'
gem 'oj_mimic_json' # we need this for Rails 4.1.x

The test logic is simple, just serialize all articles into JSON:

class SidechannelsController < ApplicationController
  def oj
    render json: Article.all
  end
end

With standard Rails serializers:

Running 10s test @ http://localhost:3000/sidechannels/bench
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   108.37ms    5.12ms 134.90ms   83.33%
    Req/Sec    45.76      3.60    55.00     57.69%
  922 requests in 10.00s, 57.41MB read
Requests/sec:     92.17
Transfer/sec:      5.74MB

With oj gem:

Running 10s test @ http://localhost:3000/sidechannels/bench
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    78.06ms    4.43ms  92.83ms   81.31%
    Req/Sec    63.64      5.33    71.00     64.49%
  1277 requests in 10.00s, 79.83MB read
Requests/sec:    127.65
Transfer/sec:      7.98MB

Using jemalloc

OK, this is not really a gem, if you want to dig into it then do check out my gist redirect. On initial testing it won't yield much performance gains, at least for my use case.

note: it will be included by default in Ruby at some point.

update: do try the jemalloc redirect gem by kzk:

gem install jemalloc

je -v rails s

Dig into your Rails app

Fear not and use MiniProfiler redirect with the awesome FlameGraphs redirect by Sam Saffron.

Conclusion

Depending on what your app is doing you might want to add to your Gemfile some of these gems, I usually add them all just for good measure (you might want to check your RAM usage and have a full test suite before doing this though).

The oj gem is just great for a Rails based JSON API where you can drop the views and just serialize using representers or your pattern of choice.