Busy software developers tend to think of refactoring as a luxury activity, and separate from writing new code or fixing defects. At first glance, the latter two activities clearly contribute to why we write software — to get new, working functionality from hardware — while refactoring, by definition, doesn’t change the software’s behavior at all.
But if we recognize that every code change or addition may affect both the dynamic behavior (what the software does), and the static behavior (how easy it is for the developer to modify it as desired), then all three code-change activities come together. Every code change, whether it’s refactoring, new code for new feature, or changes to fix a defect — and no matter how small a change — is a change to the entire codebase with the purpose of increasing its value.
This mindset is what motivates code quality engineering practices such as in-person design and code review, static analysis, and automated whole-system regression testing (including load testing and robustness testing). As well as, of course, refactoring!
And while time and resources for these activities can be planned as part of any software development methodology, it seems easier in Agile. Include all these activities regularly in sprint tasks per user story, evaluate them all in planning based on their size (cost) and their benefit (value), and get them done regularly. Apply this whole-code awareness, and watch defects decrease and velocity (and enjoyment) increase!
Thanks to Will McKinley for the post Code Refactoring – Dealing with Legacy Code that made me ask myself what are the differences, and similarities, between refactoring and other types of code changes.
When somebody asks me what I do and I say, “I’m in software”, or “I’m in computers”, they usually give me a “Oh, that’s nice,” but it’s clear they aren’t any wiser than before they asked. Why is that such a conversation-stopper? If you’re a lawyer or a doctor, people think they know, and jump right in.
Maybe that’s because lawyers deal with people and justice, or at least arguing, which sounds familiar. And doctors — well, everyone’s been to the doctor. The drama of dealing with people is what gets lawyer and doctor shows on TV, and I’ll be the first to admit that my picture of what they do comes straight from there. Really that means I have no clue.
After a bit of thought, I realize that software is more like art, if that helps.
(Right — saying you’re an artist is like saying you’re in software, only poorer.)
But really, it can help.
The solution to explaining what “I’m in software” means is to realize that software, and software development, is really three very different things. To the software developer, it is code — lists of instructions to make a generic machine behave in a specific way. To people in general, it is invisible: as mobile phone users, airplane travelers, or just people turning on the faucet and getting water, the world is full of things made of plastic and metal that behave or respond more actively than a cup or a pair of scissors. And finally, software development, the activity that goes on in a hi-tech software office, includes a good deal of accounting, scheduling, and event planning.
This triple personality parallels art. When someone tells you they’re an artist, do you think of paints, chemicals, brushes, and canvas? Or paintings on the wall? Or the business of running an art gallery?
I’m pretty sure most people hear “artist” and think of paintings. But the “what the artist does all day”, whether it’s looking, thinking, or carefully mixing paints, is invisible in the painting. We see the painting and know nothing about the work life of the artist.
So being in software, like being an artist, means doing something unseen to materials and creating interactive things that everybody then takes for granted.
Probably the only way forward for more people to understand what’s “in software” is to take them into the studio — the hi-tech office — and let them try their own hand at coding a mobile phone app. Or on the other hand, attending a project meeting.
Say, why aren’t those kinds of experiences available at the local computer or science museum — not playing with the things, but making them come alive?
Error-proofing is preventing errors rather than just warning about them.
With compiler warnings, that means setting warnings-as-errors in your compiler so you cannot complete your build. Yes, you also need tests that can fail and a source control system that refuses promotion of versions that fail test. And code review to ensure that code changes to address compiler warnings only improve the code. But the key is warnings-as-errors.
While error-proofing is simpler with fast-food trays, the idea is the same. Read “Simple, Brilliant, Error Proofing at the Amazing In-N-Out Burger” (Mark Graban | 04/11/2011 Quality Digest) and look for where else you can error-proof your process.
Recently I went to a workshop on Exploratory Testing given by John Stevenson. Just as exploratory testing finds the unexpected, so I learned something unexpected. Every time John used the word “testing”, he meant it as (re)defined in Michael Bolton’s Testing vs. Checking. It’s a powerful redefinition, and simpler to say than what I’ve always meant by “adversarial testing”. Here’s how the redefinition works:
- Notice that many tests don’t find defects
- Decide we need a new definition for the word “test”
- But the current definition is useful, so move it to the word “check”
- Now the word “test” is open for redefinition
- Define “test” as in Testing vs. Checking
Quick summary of new and useful definition:
Let “test” mean an exercise of the software under test that helps a thinking tester answer the question, “Is there a problem here?”
Recently I became the unwitting user of a new piece of consumer software, when our office upgraded our mobile phones. Suddenly, with every phone call, appointment reminder, and even when charging the phone, I’m acutely aware of what I want from this piece of embedded software that I did not even buy. As a software user I want only two things: more or better features, and problems fixed. I don’t care what version the software is — I care what’s changed. It would make sense, then, that software developers should focus on changes, not versions, and that tools should support them.
The most common tool on a software development project is always the “bug tracker” or defect tracking system. It’s really a change request tracking system, with two kinds of requests for changes: enhancements (performance improvement or new feature requests) and defects (fix requests). Tracking requests for changes. That’s good.
Now look at the other major tool in every software development group — it’s usually called a “version control system”. But see above — users don’t care about versions. And developers have to report to the defect tracking system what they’ve done, and that system is which is full of requests for changes, not requests for versions. So why would we want a version control system?
That’s why the light went on for me when I read Joel Spolsky’s Hg Init: a Mercurial tutorial. Not because Mercurial (or Git) makes branching cheap and merges easy. (It does.) But because, as Spolsky writes:
Subversion likes to think about revisions. A revision is what the entire file system looked like at some particular point in time. In Mercurial, you think about changesets. A changeset is a concise list of the changes between one revision and the next revision. (When Spolsky writes “revisions”, read “versions”.)
Now I understand why there’s so much trouble connecting our defect (change) tracking and version control workflows. Testing too — what we really want to know is, “Which change broke the build?” And bringing in code review, more problems, because people want to review whether each change accomplishes the goal of adding or fixing something. Software development is all about changes: specifying them, designing them, coding them, testing them, and delivering them. Not versions. Changes. Sure, versions have a place in software delivery — they are labels for the software after a certain number of changes. (Mercurial supports them, as “tags”.) So if you want all the tools to work together and make your software development life easier, switch to a software repository that lets you control changes.
What should we call them? Maybe “software changeset control systems”. The two big ones these days are Mercurial, and Git. For Mercurial, see Spolsky’s tutorial. For Git, listen to Linus Torvalds on git.
Software is not construction
Software is not construction, no matter how popular that statement may be. A program is not a building. Parts are not chosen first and then fitted into place. Design continues almost to the last minutes of coding. Further, almost nobody spends time creating new code, at least not for very long. Sure there are new products and new codebases. But even then, after a short period, most additional coding is extending or changing an existing codebase. So if coding is not construction, and mostly involves changing what’s already created, what is it?
Well, coding is writing. It’s using text to communicate a message. That may not sound like much, but understanding this helps a lot. Instead of struggling to make up new ways of thinking and learning, as if programming were a completely new human activity, we can look at how good writers write. People have been writing for many hundreds of years, and have become pretty good at it.
Learning to write
How do people get into writing? Do we start, as most do in programming, full grown, writing essays? No, not at all. We start as children, and learn first to talk, then to read, and finally to write. Yes, I know that school seems to teach writing, at least forming the letters, before reading. But it’s an illusion: listening to a bedtime story is essentially reading – paying attention to someone else’s writing, and understanding it – and comes before writing. Most people don’t try writing that communicates a message – expository prose – until secondary school or later.
Good writers have their audience in mind at all times. They edit their writing through several drafts, as well as getting others to review it, and give written comments. And they talk about their writing. Maybe not always directly, but when a political columnist goes to a dinner party, you can bet he’ll talk politics. When a novelist chats in the park, she’s talking about the same human struggles that will later appear in her books. Even for the solitary writer typing in a darkened room, the life that feeds the writing is one of talking with, listening to, and reading about people.
Software is Writing
Now back to software: software is not like writing, it is writing. In our industrial setting, it is too late or too radical to suggest re-teaching programming, by teaching first how to talk, then to read, then to write programs. But we can highlight where daily programming work already involves each of these three activities, and where it doesn’t, but should.
Talking includes formal design reviews and code reviews, but also whiteboard conversations and hallway arguments. How should I write this here? Why is that data structure that way? Why does the compiler give that warning? Don’t just think alone – talk it out with someone nearby. It is the same with tests. Why does that scenario fail intermittently? How should I write it differently? And even for defect entries -what does the submitter really mean? Phone him up. Why does this change fix the problem? Find the developer and ask.
Reading is for design documents, training materials and perhaps professional books. But mainly, it includes reading lots of existing code. That’s reading it until you’re really sure what it does, so you know how to change it, fix it, extend it. There is also code review — where the author asks you to read their code, not just to say that it looks fine, but to identify what’s wrong, or unclear.
Writing is the coding, of course. But who is the audience? There are two audiences. One, the compiler, which is your personal translator, from a non-native language (nobody’s mother spoke to them in C), to an almost unknown language – the machine language that the hardware actually understands. The compiler is a rigid, simple-minded, sometimes unpredictable audience, just a ghost of the person who wrote it. Here you have to be very precise, careful, and limit yourself to what the compiler understands and to what the hardware will accept. The other audience, is so very different – the next developer. This may be yourself a few weeks later, or someone else, but there will always be another developer, coming back months or years later, to fix or extend the code. She wants comments, explanations, meaningful names for things, and an easily readable structure. Because now you’re long gone, she can’t talk to you: she must read, so you must write.
Well, with all this talk about programming being writing, let’s come back to earth. Software developers are technical people, and expect to use the best tools to do their jobs. There’s the whiteboard in the hallway — we need more of those, for talking. Defect tracking systems, where we can make sure we write clear resolution notes. And for automatically sharing and discussing code, web-hosted code tools for static analysis and code review are the way to go.
Originally published in company internal newsletter. Edited and released here with permission.
After spending more time than I should have troubleshooting a coding (actually static code analysis) problem, I shared the story with a co-worker, who reminded me that Kernighan and Pike had been there before:
Another effective technique is to explain your code to someone else. This will often cause you to explain the bug to yourself. Sometimes it takes no more than a few sentences, followed by an embarrassed “Never mind, I see what’s wrong. Sorry to bother you.” This works remarkably well; you can even use non-programmers as listeners. One university computer center kept a teddy bear near the help desk. Students with mysterious bugs were required to explain them to the bear before they could speak to a human counselor.
Seems that there are rules that we learn, then love to break, and are always sorry afterwards. Here’s how.
My code doesn’t work
- I’ll just try this change to see if that fixes everything. And then this change, and this other change.
- I’ve been stuck for 10 minutes, but since nobody knows this code like I do, I won’t ask anyone for help.
- I know I don’t understand what every line of code does, but it’s OK — I’ll figure out the problem anyway.
- A complete build takes only a few minutes, so there’s no need to try a 3-line sample instead.
- I’ve read the documentation and I’m sure I’ve understood it, so I won’t ask anyone for a second opinion.
- I already asked someone 3 times for help and got it, so I can’t bother him or her a fourth time.
- It doesn’t compile (cleanly, or at all) just now, but soon it will again if I just keep at it
(The only way I got home today was that somehow I did realize it was, um, OK to call the tool support line.)
What are your favorite “justifications” for why it’s better to stay stuck than to get help, or at least take a break and share your problem with someone else?
While I’m waiting, I’ll go talk to my teddy bear.