Sunday, June 10, 2012

Engineering and Integrity

Feynman on integrity.  http://www.lhup.edu/~DSIMANEK/cargocul.htm

This made me think of conversations I have had with programmers over the last year.  I have the feeling, and it may be confirmation bias on my part since I do not have raw data, that I am hearing more Cargo Cult arguments.


I recently gave a talk in Washington, D.C. on Go.  During the talk, I cited a statistic from “Inside the Erlang VM: http://www.erlang.se/euc/08/euc_smp.pdf.  “If a program scale[s] well with the SMP VM over many cores depends very much on the characteristics of the program, some programs scale linearly up to 8 and even 16 cores while other programs barely scale at all even on 2 cores.”



I said, “Even with a six processor commodity PC Erlang performance can degrade.”  This was in the context of the discussion of the penalty languages pay for dealing with locks and context switches.  The purpose was to point out that even a language as highly optimized for concurrency as Erlang is subject to the bookkeeping effect of locks and context switches produced by our current kernel designs.

I received an email from one of the participants who had posted on his local Erlang Meetup board the message, “did anybody hear that erlang speed drops after a 6th core is involved...”


A member of the group, whom I conclude is an Erlang adherent, replied, “Six sounds pretty low, but I didn't have any good numbers, so I did a bit of digging and found a recent (2011) academic paper on Characterizing the Scalability of Erlang VM on Many-core Processors
It's got way more detail than I was willing to wade through, but the conclusion seems to be that Erlang works well up to at least 60 cores.”


I waded through the details of the thesis, and the author concludes in Section 5.2 “"The scalability of the Erlang VM can be improved by reducing lock contention and the overhead associated with it. The most critical locks are those for memory allocators. They are based upon Pthread mutex locks." ..."Reducing lock contention is not sustainable if the number of cores keeps increasing. "


The paper had “way more detail” than the respondent “was willing to wade through”, yet he was able to draw a conclusion that Erlang “works well” in a general case.  


The fan-boy mentality has always been present in the consumer population and is an integral part of how marketing campaigns are designed.  It is the iPhone versus Android argument and is in the same category as whether the Red Sox or the the Yankees are the better team.  It is bar talk based solely on personal preference and opinion.


When applied to a an engineering problem it produces a result that allows us to fool ourselves and feel good about our current assumptions, but tells us nothing about a real solution.  In fact, the diffusion of this type of opinion as fact, and the ready acceptance by programmers, is harmful to both our profession and the people who ultimately use our software.


So, in a totally unscientific manner, let me know if it is my own confirmation bias at work, or you have experienced the same phenomenon.


#golang, #erlang, #programming

2 comments:

  1. I actually have a confirmation bias that points the other way.

    Nowadays, with so much wild claims (NoSQL people saying it will cure cancer, parallelism guys also curing cancer with user-level threading, etc), people have to get their hands dirty and confirm the claims because there's so much people trying to sell you snake oil.

    ReplyDelete
  2. I know that I am guilty of cargo-cult thinking - there is so much information to absorb, and so little time to absorb it in, that I feel unable to do adequate due diligence on the assertions that I come across. For example, I know that I should spend a lot more time characterizing and testing the systems that I write, but the pressure is always on to knock out features and get the next system completed and out the door, so we end up making design decisions based on gut instinct and half-misheard rumors of what works and what does not, using whatever the technology du jour happens to be, and holding the whole creaking mass together with duct-tape and wishful thinking. When every day passes in a blind panic, there is little time for reflection, analytical thinking, or critical thought.

    ReplyDelete