Talk About Quality

Someone comes to me with a problem that sounds like,

“I wanted to do X but it doesn’t work.”

When I hear that request, often they know a lot more about the technical details than I do. I need them to teach me in a way that lets me put things in order. My method starts from success … and progresses to failure. Strange? But it works.

The questioner is focused on the failure scenario. I crave success. There must be some around — just have to find it. I ask what similar scenario succeeds. What version of the file did work? What build compiled successfully? When did this test last pass? As we discuss the successful scenario, I learn why it worked. What requirements were met by that configuration. I repeat those requirements out loud for confirmation.

With a success in hand, well understood, we have a checklist to apply to the failed configuration. We can go over the failure and check whether all the things that made something similar work are present. Of course, one will be missing. We have found the problem. I may not know how to fix it. But usually the questioner does.

It’s Thom Holwerda — keeping things simple for us:

Simple Code Quality Metric

Here’s his site — with a search for “code quality” (warning: if you can’t ignore “Evony” web game ad graphics, stay away).

A medical radiation machine operator types the letter ‘x’, realizes it’s an error, backspaces, types ‘e’, and continues. Consequences of the error and related defects lead to the patient’s untimely death.

Ten years later, a Japanese stock broker mistakenly switches the share price and amount on the sell order of a new stock. He tries to cancel the order, but fails. His employers lose $225 million, and are involved in lawsuits for years.

A blogger clicks the “back arrow” by mistake, gets a warning message, concludes he meant to continue writing, and clicks “OK” to continue. He loses his work, and has to rewrite it.  Here, the consequences are annoying, if trivial by comparison.

What do all these cases have in common? Canceling a request. If it’s not hard enough to program computers to do what we want them to do, who would have thought that telling them not to do it would be hard too?

The cancellation scenario — or “use case” as it’s called in software design — is the silent partner of every positive request supported by a piece of software. It has to cover giving the user clear options, executing the cancellation, and rolling back any partial results. Things get more complicated if authorization is required, or if the transaction has already gone through (both of those requirements figured into the Japanese broker error story). Cancellation and rollback are also part of automatic requests that may occur if one software module (the “server”) cannot complete a request by another (the “client”) and has to make sure to put everything back the way it was, and send the proper response code.

So the next time you’re designing a piece of software, no matter how simple, think what it’s supposed to do, but also what it will do if the user or client calls out, “Cancel that!”

I know what I want: highlight-and-comment capability in a text editor. And since plain text really shouldn’t be marked up, I’d want it to save the comments in a separate text file, with line numbers of what was commented on, and the comments’ text. I’m not the first one looking for this capability: “climbanymountain” asked about it on Joel on Software back in 2006: “win32 text editor for code review?“

I started thinking about writing a plugin for some popular text editor, such as Notepad++ or Notepad2 (the latter is my favorite for simple editing). But before the “build or buy” decision, I thought I’d ask around the office.

A lot of ideas, but here was the simplest. Light and off-the-shelf. Use a diff tool. 

The following procedure works fine, and “supports” any version control system. In my example, I’m using open-source WinMerge as my diff tool, IBM Rational ClearCase as my version control system, and Microsoft Windows Explorer as my tree browser. Say the source file to review is called “MyFile.c”, and the reviewer’s name is Firstname Lastname.

The parts in square brackets are optional: omit them if you don’t want to clutter up your version control system with code review records, but just want to use them to prepare for sitting with code author.

1. Browse to the source code file I want to review; [check it out and] open it.
2. Immediately Save As MyFile_flastname_review.c in the same directory. Close the source code file.
3. Select the source code file and the review file (they are next to each other in alphabetical order), right-click and open WinMerge on them.
4. Go to the pane of the review file and type review comments anywhere I want. Save regularly with CTRL-S. (Optionally, for better syntax highlighting, can precede comments with comment character such as “//”. But don’t have to.)
5. Generate a diff (patch) file, call it the same name as the review file, but add .diff on the end. (In WinMerge, that’s on the menu with Tools > Generate a patch.)
6. [Check in all 3 files (adding 2 new ones to source control) -- check-in activity per defect tracking system, and comment with one-line summary of the review.]

From this point, code review comments are easily viewable side-by-side with the code by viewing in WinMerge or similar viewer.

Also, the diff file clearly identifies the line number commented on, and the comment text. I figure I could write an AWK script to turn that into CSV, for tabular display and tracking in Microsoft Excel, if I wanted to.

Google “use case format” and you’ll get lots of templates. Which one to use? Here’s my really simple template that people around me have started using:

• Who I am (role)
• What I’m doing (current task)
• What I want (current goal)
• What I type (into the system being defined)
• What I expect to get (out of the system being defined)

True, it doesn’t address pre-conditions, post-conditions, extensions, and more. But it serves well for high-level discussions, and gives a clear understanding of what use cases are for. After using this template for a while, go to Google and get one of those fancier ones if you want.

This month, CrossTalk, The Journal of Defense Software Engineering, has once again provided a great service, at no charge. (Well, no subscription fee–it’s your tax dollars at work.) It’s their 20th Anniversary Issue, and they’ve brought together some of the real greats to talk about the last 20 years of software engineering, and its future. Even though these thought leaders, among them Watts Humphrey, Alistair Cockburn, and Gerald Weinberg, have all taken different directions over the years, it’s perhaps no surprise that there’s a common thread.

How to Keep Software Engineering Running in Place

• Don’t deploy what works
• Don’t build teams
• Don’t learn on the job
• Ignore the data

If you don’t believe me, take it from the experts: http://www.stsc.hill.af.mil/crosstalk/2008/08/index.html

This week, I moved from a general process improvement group to a software development department, to specifically focus on code quality. The most mature tool and process I left behind was a defect tracking system. Of course, in our development department, we use that defect tracking system. In case anyone asks me what it’s for, here’s my answer.

Prevent Errors, Fix Defects, Describe Failures

There are many definitions of those terms, but since when we fix problematic code we close “defects” in the “defect tracking system”, I started with that word in the middle. Leads to these definitions:

Error: The mistake someone made. (The root cause is the earliest error in the chain.)

Defect: The wrong code (missing, extra, or incorrect).

Failure: The unacceptable behavior of the code when running.

Detect each as early as possible

Each can be detected with appropriate tools. Detect earlier to minimize cost and annoyance.

Meanwhile, do it right

How?

With tools, learning, and discussion for writing good code.

With the enormous popularity of Google, who could resist an article in the New York Times (front page, four pages long!) about how Google Keeps Tweaking Its Search Engine? Maybe we’d learn some upcoming changes (hardly) or inner secrets (definitely not). But in among the intentionally vague references to formulas and advanced mathematics, the following caught my eye:

Mr. Singhal often doesn’t rush to fix everything he hears about, because each change can affect the rankings of many sites. “You can’t just react on the first complaint,” he says. “You let things simmer.”

So he monitors complaints on his white board, prioritizing them if they keep coming back.

Singhal is responsible for nothing less than Google’s search algorithm—surely a mixture of mathematics and magic.
The article also tells us that Google has a computerized problem-reporting system where

Any of Google’s 10,000 employees can use its “Buganizer” system to report a search problem, and about 100 times a day they do — listing Mr. Singhal as the person responsible to squash them.

So why is he prioritizing his work using… a whiteboard?

Change scenes now to Yahoo’s pragprog discussion list, where experienced developers discuss their craft. (You have to sign up to see the messages.) A few days ago someone asked for recommendations on the best tool for UML. The first answer from one of the members was:

“I’ve always found a pen and a large sheet of paper rather effective myself “

Software development is still largely a human activity. Planning is human. Design is human. Only some of coding gets down to the machine level.

Humans like and need simple, flexible tools that don’t hold much more than people can think about at once. With planning—in its essential activity of prioritizing—if you get more than 10-15 items, the bottom of the list never gets done. No point keeping them on the list at all. For that, even better than paper, the whiteboard comes with a great accessory: the eraser. Likewise with design: the important part is the discussion, the thinking out loud. Whiteboards go great in hallways for such work.

Yes, sooner or later there’s code, and compilers, and syntax, regression test suites and repeatable builds. For those machine activities, use computerized tools and scripted automation.

But for people, stick with the whiteboard.

When I talk about tools for coding, some people ask,

“How can we be free when the environment is computer-controlled?”

It’s all about the right metaphor. One of mine is woodworking.

“By definition, a jig guides your tool and a fixture guides your work.”

Thus begins Jim Stack in The Best Jigs and Fixtures for Your Workshop. And he claims the 30 best in his book. But here’s a similar example from another source, to give you the idea.

There’s no improving on the authors’ explanation:

“If you’ve ever built a picture frame you’ve experienced the frustration of trying to get all of the parts to fit perfectly. While trying to “dry-fit” and clamp the frame’s pieces together the parts often fit perfectly. But add glue, and a little clamp pressure, and the parts begin to slide apart. It can be a frustrating race against the clock to even things out before the glue begins to set.

(By the way, by Stack’s definition, this example is really a fixture, not a jig.)

If you’ve ever tried framing a picture, you may know what they’re talking about.

Equally well if you’ve ever raced to get your program to compile and run before you go home.

Yes, that picture frame looks pretty tightly constrained. But it’s not limiting my creativity as a carpenter.  It’s just holding the pieces in alignment so they don’t all fall apart. I don’t find myself thinking creatively when I have glue all over my fingers and wood stuck to the carpet.

Your tools include the IDE and the compiler. Your work is, clearly, the code.

What kinds of add-ons (“jigs”) guide your tools? What kind of fixtures can help guide your work?

Here’s the rest of that site: http://www.woodzone.com/tips.htm. Look through it and then make your own list for coding.

You don’t even have to watch your fingers.

I’ve often thought about the advantage that athletes and musicians have over software developers: time to practice—lots of it. Most time is spent in exercises and rehearsals, to produce the best peformance.

In software, everyone spends most of the time producing, with a bit of “time off” for learning new technologies. No wonder perfection seems so far off.

A friend pointed out that Dave Thomas has proposed CodeKata as a way of doing that practice. I read it, and while the ideas look good, I felt there was something missing. The “katas” (formal patterns in karate, and here, in design and coding) seemed somehow schoolwork-like and disconnected from real work. I could only imagine myself “practicing” on assignments where I needed the result, even if for something trivial like importing a bunch of e-mails into SharePoint (more about that another time).

I don’t have an answer today, but the question is much bigger than just finding time to practice in software development. So instead, have a look at these posts and presentations where people are discussing the issue, and see what you think.

Level 5 means never having to say you’re sorry (Jeff Atwood)

Big Macs vs. The Naked Chef (Joel Spolsky)

No Best Practices (James Bach)

Herding Racehorses and Racing Sheep (.ppt) (The Pragmatic Programmer)

Competence is a Habit (.ppt) (David Leach)