Could visualization help make better software?
December 18, 2008 ⋅ 4 Comments »Writing software is incredibly hard. Every programmer knows this. The software we write is complex, unreliable, and difficult to maintain. And this isn’t a new thing — the term “the software crisis” was coined in 1968.
The thing about software is that it’s remarkably easy to write a program that mostly works. And it’s difficult to tell the difference between a quick hack and a stable, reliable, and robust system, because the software development process produces almost no visible artifacts.
When you look at a building, it’s easy to get a quick sense of how well-built it is. Which of the two buildings below would you rather be in during a heavy storm?
Besides the program itself, the only visible artifact of the software development process is the source code. And that is only viewed by the programmers, through the tiny lens of the text editor. What if we could make the entire process more visible?
I’m thinking of visualizations along the lines of the comparison of system calls in Linux/Apache and Windows/IIS that I posted a while back. But this is just one idea. What other ways could we visualize Code Smells? (Maybe we could actually smell them!)
Aside from visualizing various aspects of the source code itself, could we show how well tested a piece of software is? We could show how many tests were run recently, what their results were, how good the code coverage is, the number of crashes encountered in the field, etc. With projectors and LCD displays being so cheap these days, there’s no reason a development team couldn’t have a few displays dedicated to these kinds of visualizations.
What do you think? Could this help improve the quality of software? What if companies openly published these kinds of things?
Chris - December 18, 2008:
I think visualizations help.
You'll never get everyone to agree - but I'd really find something along these lines useful.
Some of the problems come from what makes code attractive. You gave some samples - but some of those areas are heavily debated across the industry. Something like Test Coverage is hard to visualize in a complex way. I can currently use a tool like NCover and see a visualization of my code in the test coverage sphere, and I find that useful . . . but it has to be considered carefully. I can have nearly 100% coverage - but still not have true stability, and worse, the code could be extremely fragile or just not meet requirements at all.
I know that you're not suggesting using a single metric to form a visualization. But, I think that greatest challenge would be to find the right combination so that the visualization really means something. If we can't trust the visualization it becomes more of a fun thing to have, but not actually useful for really fixing software.
But, every day I use source code analysis tools, test coverage tools, profilers and unit testing. There are also tools that will discover the level of testability that your source code offers. If these sorts of things could be combined together to display a picture, it would be pretty darn cool.
Patrick - December 18, 2008:
@Chris,
You're right, the key thing is nailing down the right metrics.
It's true that code coverage (for example) doesn't necessarily imply any particular level of correctness or stability. But, it does give an indication of how much effort is put into testing.
There are lots of metrics and qualifications in the real world that don't guarantee anything, but in the absence of any other information, they can still be helpful to ensure quality. For example, if I go to a restaurant which has a Michelin 2-star rating, I might still get food poisoning; but relative to some of the restaurants near my house the chances are probably lower.
Yandu - December 19, 2008:
One thing that would be interesting is to see how the visualization changes depending on the scenario that the program is in. This I think is key since what is good enough for a single user and may look great visually, won't be all that great for 100 users. For instance if you look at a rope bridge across a river, it looks pretty good. Expand that rope bridge to be used for four lanes of traffic and suddenly it does not look as good or safe.
Erigami - December 19, 2008:
Well built is a rabbit hole. "Well built" from whose perspective?
This is going to sound lame, but stars are pretty good. I dread making my Miniposts plugin available in the wordpress plugin listing because my users will suddenly have a mechanism to tell the public what they think of my software. The bugs that I've been allowing to linger, because they're outside my use cases will suddenly be important.
Forget coverage, forget visualization, forget everything complex. A fair system that allows users to honestly comment on a piece of software is probably more than enough to tell if users think a piece of software works well. At the end of the day, it's the users perspective that matters most.