The price of open-source software - a joint response

This viewpoint is published jointly on software.ac.uk, hpcnotes.com (personal blog), danielskatzblog.wordpress.com (personal blog) under a CC-BY licence. It was written by Neil Chue Hong (Software Sustainability Institute), Simon Hettrick (Software Sustainability Institute), Andrew Jones (@hpcnotes & NAG), and Daniel S. Katz (University of Chicago & Argonne National Laboratory)

In their recent paper, Krylov et al. [1] state that the goal of the research community is to advance “what is good for scientific discovery.” We wholeheartedly agree. We also welcome the debate on the role of open source in research, begun by Gezelter [2], in which Krylov was participating. However, we have several concerns with Krylov’s arguments and reasoning on the best way to advance scientific discovery with respect to research software.

Gezelter raises the question of whether it should be standard practice for software developed by publicly funded researchers to be released under an open-source licence. Krylov responds that research software should be developed by professional software developers and sold to researchers.

We advocate that software developed with public funds should be released as open-source by default (supporting Gezelter’s position). However, we also support Krylov’s call for the involvement of professional software developers where appropriate, and support Krylov’s argument that researchers should be encouraged to use existing software where possible. We acknowledge many of Krylov’s arguments of the benefits of professionally written and supported software.

Our first major concern with Krylov’s paper is its focus on arguing against an open-source mandate on software developed by publicly funded researchers. To the knowledge of the authors, no such mandate exists. It appears that Krylov is pre-emptively arguing against the establishment of such a mandate, or even against it becoming “standard practice” in academia. There is a significant difference between a recommendation of releasing as open-source by default (which we firmly support) and a mandate that all research software must be open source (which we don’t support, because it hinders the flexibility that scientific discovery needs).

Our second major concern is Krylov’s assumption that the research community could rely entirely on software purchased from professional software developers. We agree with this approach whenever it is feasible. However, by concentrating on large-scale quantum chemistry software, Krylov overlooks the diversity of software used in research. A significant amount of research software is at a smaller scale: from few line scripts to short programs. Although it is of fundamental importance to research, this small-scale software is typically used by only a handful of researchers. There are many benefits in employing professionals to develop research software but, since so much research software is not commercially viable, the vast majority of it will continue to be developed by researchers for their own use. We do advocate researchers engaging with professional software developers as far as appropriate when developing their own software.

Our desire is to maximise the benefit of software by making it open—allowing researchers other than the developers to read, understand, modify, and use it in their own research—by default. This does not preclude commercial licensing where it both is feasible and is the best way of maximising the software benefit. We believe this is also the central message of Gezelter.

In addition to these two fundamental issues with Krylov, we would like to respond to some of the individual points raised.

Secrets of the Supercomputers

These are revelations from inside the strange world of supercomputing centers. Nobody is pretending these are real stories. They couldn’t possibly be. Could they?

On one of my many long haul airplane plane journeys this year, I caught myself thinking about the strange things that go on inside supercomputer centers - and other parts of the HPC world. I thought it might be fun to poke at and mock such activities while trying to make some serious points.

Since the flight was a long one, I started writing ... and so "Secrets of the Supercomputers" was born.

You can find Episode 1 at HPC Wire today, touching on the topic of HPC procurement.

No offense to anyone intended. Gentle mocking maybe. Serious lessons definitely.

Take a look here for some serious comments on HPC procurement at the NAG blog.

Silence ...

Really, October 2013? That long since I wrote a blog? Not even anything for SC13? Oops. Still, busy is good. Be nice to get some more blog posts again though. Maybe a preview of ISC14 in the next few days ...

Essential guide to HPC on twitter

Please read the updated version of this post at:
https://www.hpcnotes.com/p/hpc-on-twitter.html

(Original kept here for reference)

Who are the best HPC people on twitter?

A good question posed by Suhaib Khan (@suhaibkhan) - which he made tougher by saying "pick your top 5". A short debate followed on twitter but I thought the content was useful enough to record in a blog post for community reference. I also strongly urge anyone to provide further input to this topic and I'll update this post.

Some rules (mine not Suhaib's):

1. What are the minimum set of accounts you can follow and still expect to catch most of the HPC news, gossip, opinion pieces, analysis and key technical content?
2. How to avoid too much marketing?
3. How to access comment and debate beyond the news headlines?
4. Which HPC people are not only active but also interactive on twitter?

I cheated a little on the "top 5" by suggesting 4 themes:

Supercomputing - the reality behind the vision

My opinion piece "Supercomputing - the reality behind the vision" was published today in Scientific Computing World, where I:

• liken a supercomputer to a "pile of silicon, copper, optical fibre, pipework, and other heavy hardware [...] an imposing monument that politicians can cut ribbons in front of";
• describe system architecture as "the art of balancing the desires of capacity, performance and resilience against the frustrations of power, cooling, dollars, space, and so on";
• introduce software as magic and infrastructure and a virtual knowledge engine;
• and note that "delivering science insight or engineering results from [supercomputing] requires users";
• and propose that we need a roadmap for people just as much as for the hardware technology.

Read the full article here: http://www.scientific-computing.com/news/news_story.php?news_id=2270.

All software needs to be parallel

I often use this slide to show why all software has to be aware of parallel processing now.

In short, if your software does not exploit parallel processing techniques, then your code is limited to less than 2% of the potential performance of the processor. And this is just for a single processor - it is even more critical if the code has to run on a cluster or a supercomputer.

An early blog about SC13 Denver - just for fun ...

As SC13 registration opens this week, it occurs to me both how far away SC13 is (a whole summer and several months after that) but also how close SC13 is (only a summer and a month or two). It got me thinking how far ahead people plan for SC. I have heard of people who book hotels for the next SC as soon as they home from the previous SC (to secure the best deal/hotel/etc.). I have also heard stories of those who still have not booked flights only days before SC.

So, just for fun - how far ahead do you plan your travel for SC? Are you the kind of HPC person who books SC13 as soon as SC12 has ended? Or do you leave SC13 travel booking until a week or two before SC13? Of course, it may not be up to you - many attendees need to get travel authority etc. and this is often hard to get a long time in advance.

Please complete the survey here - http://www.surveymonkey.com/s/3MRSYYH

Once I have enough reponses, I will write another blog revealing the results.

Enjoy!

[PS - this survey is not on behalf of, or affiliated with, either the SC13 organisers or anyone else - it's just a curiosity and to share in a blog later.]

China supercomputer to be world's fastest (again) - Tianhe-2

It seems that China's Tianhe-2 supercomputer will confirmed as the world's fastest supercomputer at next Top500 list to be revealed at the ISC'13 conference next week.

I was going to write about the Chinese Tianhe-2 supercomputer and how it matters to the USA and Europe - then I found these old blog posts of mine:

• Why does the China supercomputer matter to western governments?
• Comparing HPC across China, USA and Europe

Supercomputing goes to Leipzig - a preview of ISC13

I have written my preview of ISC13 over at the NAG Blog ... a new location, Tianhe-2, MIC vs. GPU, industry, exascale, big data and ecoystems. Not quite HPC keyword bingo but close :-)

See you there!

Name that supercomputer (Quiz)

Instead of a sensible HPC blog post, how about some fun? Can you name these supercomputers?

I'm looking for actual machine names (e.g. 'Sequoia') and the host site (e.g. LLNL). Bonus points for the funding agency (e.g. DOE NNSA) and the machine type (e.g. IBM BlueGene/Q).

Submit your guesses or knowledgeable answers either through the comments field below, or to me on twitter (@hpcnotes).

For the photos, if you are stuck, you might need to use clues from my twitter stream as to where I have been recently.

Answers will be revealed once there have been enough guesses to amuse me. Have fun!

1. Which supercomputer are we looking underneath?
   


2. Acceptance of this leading system became a HPC news topic recently


3. NAG provides the Computational Science & Engineering Support Service for this one


4. One letter is all that’s needed to describe this supercomputer


5. Racing cattle powered by Greek letters


6. Spock was one of these


7. Which supercomputer does this photo show the inner rows of?
   


8. Memory with a deerstalker & pipe


9. Put an end to Ming (or did he)?


10. This plant/leaf is normally silver when used as the national symbol of this one’s host country

Predictions for 2013 in HPC

As we stumble into the first weeks of 2013, it is the season for predictions about what the coming year will bring. In my case, following my recent review of HPC in 2012, I get to make some predictions for the world of HPC in 2013.


Buzzwords

First up, this year’s buzzword for HPC marketing and technology talks. Last year was very much the year of “Big Data” as a buzzword. As that starts to become old hat (and real work) a new buzzword will be required. Cynical? My prediction is that this year will see Big Data still present in HPC discussions and real usage but it will diminish in use as a buzzword. 2013 will probably spawn two buzzwords.

The first buzzword will be “energy-efficient computing”. We saw the use of this a little last year but I think it will become the dominant buzzword this year. Most technical talks will include some reference to energy-efficient computing (or the energy cost of the solution or etc.). All marketing departments will swing into action to brand their HPC products and services as energy efficient computing – much as they did with Big Data and before that, Cloud Computing, and so on. Yes, I’m being a tad cynical about the whole thing. I’m not suggesting that energy efficiency is not important – in fact it is essential to meet our ambitions in HPC. I’m merely noting its impending over-use as a theme. And of course, energy efficient computing is not the same as Green Computing – after all that buzzword is several years old now.

Energy efficiency will be driven by the need to find lower power solutions for exascale-era supercomputers (not just exascale systems but the small department petascale systems that will be expected at that time – not to mention consumer scale devices). It is worth noting that optimizing for power and energy may not be the same thing. The technology will also drive the debate – especially the anticipated contest between GPUs and Xeon Phi. And politically, energy efficient computing sounds better for attracting investment rather than “HPC technology research”.