As a research scientist writing software it is not necessarily obvious where to go to for help concerning good software development practices. So, where to start learning such practices? Here is a list of reading resources that I have found helpful.
The Pragmatic Programmer
Andrew Hunt and David Thomas, Addison-Wesley, 2000
This was the book that really got me interested in software development as a craft and not just a means to an end. It confirmed what I had noticed over several years through my own experience in developing software and extended my horizons about what is and should be possible if one wishes to create quality software. Since software is so intensively used in science today, it is thus obvious that quality software is a prerequisite for quality scientific output. This book is a great place to start your journey from hacking scripts together to creating software that you can be proud of.
Robert C. Martin, Prentice Hall, 2008
Uncle Bob filled this book with the distilled knowledge of his many years of experience in the software development/engineering industry. He has an amazing passion for the topic and a depth and breadth of knowledge which is enviable. In a sense “The Pragmatic Programmer” and “Clean Code” cover similar topics, however from slightly different angles. If you’ve already read “The Pragmatic Programmer” allow the ideas to sink in—and start practising them—before you read this book. “Clean Code” can be read along with the (entertaining) screencasts about the same topics on http://cleancoders.com.
Remember that these two books are directed mainly at software developers in general and so the examples don’t always seem to apply to those issues arising within scientific software development. Nevertheless, most of the issues are the same and the advice should be heeded if you wish to be productive in software and thus in science.
Working Effectively With Legacy Code
Michael Feathers, Prentice Hall, 2004
Lots of scientific software falls into the category of Legacy Code. How many Fortran programs have been handed down from professor to postdoc and/or to doctorate student? Does this sound familiar to you? Did you find it hard to modify the program so that you could get your research done? This is but one of the reasons why one should write clean code, however, we are seldom so lucky. This book contains strategies for dealing with old programs and how you can safely update and improve them so that you can finally implement that model which will help you obtain your PhD.
Code Complete (2nd Edition)
Steve McConnell, Microsoft Press, 2004
This is an updated edition of the classic work on software construction. Where “The Pragmatic Programmer” and “Clean Code” both discuss software construction and the mechanics and behaviours involved in that construction (with a focus on agile techniques), this book takes the topic much deeper. It provides not only advice and insights from the author’s personal experience (as the other books do) but also the results of research and case studies into software development. If you wish more detail and more depth concerning software development, and possibly research-supported reasons for writing clean, maintainable code, then this is a must-read. Because of the level of depth and detail it is quite a tome, so be prepared for the time investment required to read it.
Perl Best Practices
Damian Conway, O’Reilly, 2005
This is the standard work concerning best programming practices in the Perl programming language. Of course this book is focussed on Perl, nevertheless, it also contains many nuggets of good advice which are applicable in any programming language.
Martin Fowler, Addison-Wesley, 1999
Software is almost never written correctly the first time. There might be code smells, the design might not be quite right, the software might not actually deliver what the users actually want. This isn’t really an issue since software is so malleable and as such the software, its architecture and its design can be improved over time. This is especially important if users’ requirements also change over time. The process of moulding software into that which does the required job is called refactoring, and this is the book on the topic. If you have inherited software and need to extend it as part of your research (without breaking existing functionality!) then this book will provide you with advice as to how to do this. This is a book aimed at professional software developers in industry and may not seem so relevant to a physicist or chemist writing a simulation code. However, after having read the first 5 chapters you should be able to see how the advice applies to your situation. The remaining chapters focus on individual refactorings and can be used as a reference as soon as you notice particular code smells in your own software.
Drew Neil, Pragmatic Bookshelf, 2012
If you use any other editor other than
vim this book probably won’t help
you much. However, if you use
vim as your editor this is the book to
buy. Even if you’ve been using
vim for several years (I’ve been using
vim intensively since about 2000) and you think there isn’t much more to
learn, this book will still teach you something.
Computing in Science and Engineering
IEEE and AIP
If you wish to keep up to date with current topics and research in the interaction space between computing and science/engineering, this is definitely a good journal to read. Konrad Hinsen’s column about scientific programming is always a good place to look for topics related to scientific software development.
Destroy all software
Gary Bernhardt presents in his series of screencasts a varied mixture of topics within and surrounding software development. He has obviously thought very deeply about the topics he presents, since he often focuses on why something is the way it is or why it should be done in a particular way. The screencasts are very focussed and contain a large amount of information compressed into the 10 to 15 minute long videos. Your head will likely hurt after watching a screencast since there is so much information packed into each one. The videos are mostly presented in the Ruby programming language and often discuss web-based applications using the Model-View-Controller design pattern. While this may not sound like much help to you in getting your simulation code working, the presentations about testing and design principles will help you partition your code to make it much more easily extensible so that adding that new solver or computational model will be much simpler. Even if that isn’t really what you’re looking for, how he combines the Unix shell, Git and Vim into a complete and powerful development environment will show you how to save time in the regular tasks so that you can spend more time thinking about interesting scientific problems.
Robert Martin (“Uncle Bob” and author of “Clean Code”) has a wealth of experience in the software development industry. This experience, and lots of advice, has been distilled into a series of screencasts (usually about an hour in length) concerning various software development topics. The screencasts mesh well with his books and are a dynamic way of disseminating effectively the same knowledge. Be prepared for his zany and entertaining style of presentation. Unfortunately, even though the presentation style is certainly entertaining, it can also be somewhat distracting and is therefore a bit harder to extract the advice he is trying to get across. Don’t be put off by this though! If you are interested in improving your software development abilities these videos are certainly worthwhile.
The screencasts can be downloaded or viewed online for a fee per screencast; take a look at his introductory screencast for a taste of his style and then you can decide if this is something for you.
Not only has Drew Neil written a great book on the subject, he also creates
free screencasts concerning
vim tips and usage. My recommendation: start
from the beginning and work your way through to the most recent episodes
(there are currently over 50 episodes). They are usually about 5–10
minutes long and contain a very focussed, clear and polished discussion
about a given
vim feature (of which there are many).