Thursday, September 20, 2012

Burning Down My Laptop with Go


[This is an extended version of a previous post I made directly to Google+.]

I wrote a program in Go that clusters unstructured data and involves a lot of multi-dimensional distance calculations.  You can see the ongoing process at https://github.com/bobhancock/goxmeans.git.  It contains a pipeline that prepares jobs, fans out to worker goroutines (one for each CPU) and stores the results.  All communication is done with channels.

I’ll write more about goxmeans in the future.

I started some performance testing with relatively small datasets of 250,000 points and 2-6 centroids and it worked as expected.  When I increased the number of centroids to 9, the program would run for a while, but then my laptop would shut down.  My first thought was that there was some sort of bug with memory or the underlying mechanism of pointers.  

I viewed syslog and it showed that the Intel ACPI controller registered that the temperature had hit 100 degrees celsius.  psensor showed that it had hit a max of 94 degrees--still high compared to the average of 55 but not deadly.  In order to save my hardware, it did not issue any warnings, it immediately shut down the computer.

I've never had this type of problem with C++ or Python.  Go was so efficiently feeding jobs to the CPUs that the temperature rose almost 100% and stayed there.  I reset the fan to manual and ran it at max and everything worked fine, however, to be safe I reset the default to auto.  I'll perform the rest of the performance tests on a larger machine.

If I had tried to write this in C++ with threads it would have taken much, much longer and experience makes me doubt that the performance would have been as good.  Tests show that the CPUs are being used at close to 100% with occasional small dips for what, from profiling, appears to be garbage collection.  The shape of CPU usage on the Ubuntu system monitor is an almost vertical rise to 100% and then a flat line for all CPUs until there is a vertical drop back to normal usage at program completion.

So, Go was so efficient that a Thinkpad X201 almost burned down.  I wish I had problems like this with other languages.


#golang #goxmeans

8 comments:

  1. Not so much "cooking with gas" as "cooking with go" :-)

    ReplyDelete
  2. I have seen quite a few laptops that have CPUs in them where the cooling system isn't actually able to deal with the consistent peak TDP. Presumably this is a software problem where the max fan speed is not being used and that this sort of thermal problem would show up in other programs that also achieved all core utilisation.

    Its not so much Go its your highly parallel program that achieves really high utilisation unlike most normal consumer programs.

    ReplyDelete
    Replies
    1. A lot really is Go. I've tried to accomplish similar situations with C++ with posix threads and couldn't come close. The use of goroutines that are handled by the runtime, as opposed to the system scheduler, is a major factor in what allows this type of throughput.

      Delete
  3. Check out my answer to this question on Stackoverflow:

    http://stackoverflow.com/questions/11641098/interpreting-a-benchmark-in-c-clojure-python-ruby-scala-and-others/11642757#11642757

    What I tried to say there was that concurrency is so easy to achieve in Go that it should be fair to compare it with single threaded C.

    Nobody seemed to care, but Go had the best performance of all benchmarked languages.

    ReplyDelete
    Replies
    1. Your post is interesting and informative. I can't wait to see what happens when they improve the Go runtime scheduler. Dimitri Yukov has presented his ideas in the Golang group, and Rob mentioned at Google IO in June that this was how they are proposing to move forward.

      Delete
  4. Open your laptop, take off the keyboard, and blow the dust *out* of your fan vents. Lenovo publishes great hardware manuals for free, and the Thinkpads are meant to be disassembled like that for maintenance.

    No, really. I actually had a similar problem, and discovered that my laptop was consistently almost 12C hotter than usual simply because I hadn't dusted it out. It went from a routine 60C down to 48C after I did that.

    ReplyDelete
    Replies
    1. I took your suggestion and indeed the Lenovo laptops make it very easy to remove the top and keyboard. I used compressed air on all the areas around the vents and from the outside. I did not see any physical dust raised.

      I reassembled, re-ran the tests and got the same results.

      Delete