I’ve been reading David Anderson’s Agile Management for Software Engineering book. The subtitle “Applying the Theory of Constraints for Business Results” piqued my interest as I’ve been following ToC/Lean in the manufacturing space for some time. We also took the principles of ToC and Lean and applied them to our software process a couple of years ago and I have a very different viewpoint on the approach.
While I am enjoying David’s book, I am currently hung up on Chapter 3 titled “TOC in Software Production”. The first step in ToC is identifying the constraint and David identifies the constraint as the “individual developer”. I fundamentally disagree with David’s position at its core:
1. The individual steps in software development are too inter-related to identify individuals as the constraint.
2. Identifying individuals as the constraint never works in production environments. It ultimately leads to a number of adverse consequences, not the least of which tends to lead to a bad environment (a.k.a. “I’m going to code me a minivan” from Dilbert)
David goes on to follow this argument to its logical conclusion - asking if the 8-hour work day is the real constraint:
“Goldratt argues that the 8-hour work day is not a useful constraint because it is not a bottleneck | 1994, chs. 30& 31 |. Rather, he would call this an issuficiently buffered resource, that is, demand may outstrip supply. However, … the bottleneck would be a capacity constrained resource (CCR) in front of which a stockpile of inventory is apt to accumulate and beyond which resources would starve for input.” - Anderson, David, "Agile Management for Software Engineering", page 34.
David uses the last line to override Goldratt’s point that an 8-hour workday cannot be a constraining factor. The reasoning he uses to override is also not uncommon in other environments, but Goldratt's objection still holds in those situations. I believe that Goldratt’s initial position not only holds here, but the decision to hold the 8-hour work day as a constraint is a fatal error.
I propose that the proper constraint is a given project’s development group. While the difference seems subtle, the implications are far reaching. By focusing on development as a process, the ToC job becomes creating an environment focused on development which raises a different set of questions and provides different approaches.
I’ve decided to document the process I mentioned earlier – the process shows one take on identifying development as the constraining process and adapting the surrounding processes accordingly. The process focuses on providing constant feedback to development and sets up a framework for adapting any software process to this model.
The Core Process Model
The Core Process Model borrows from Theory of Constraints to identify the key players. I’ll hold off on discussing this topic as it’s rather involved, there are plenty of other sources that discuss Theory of Constraints quite well. These key players are the focus of the process and the fast, repetitive cycling of work through these key components make the entire process
work.
First, development is the constraining process. In other words, regardless of how many specs are written, how many designs are written, and how many QA people you have, you can only create product as quickly as development can produce. Development is also a complex process – for starters, good programmers are hard to find and productivity does not scale with the number of programmers hired. Software development is also challenging – there are no single development paradigms that guarantee success, therefore pragmatism reigns supreme. I’d venture to say that any single development paradigm taken to its extreme practically guarantees failure (O-O, meta-data, functional programming, etc.).
Second, Integration is not found in other development lifecycles and this is a tremendous oversight. Integration involves building, packaging and deploying the product to the host environments. While integration steps are usually implemented by development, it has a number of characteristics that make it distinct from development:
* Integration is often required to adapt to outside factors (3rd party tools, changing destination/host environments,
* Integration and deployment issues can trigger a release by themselves.
* The integration process must be tested and the knowledge required for testing integration is usually very different than the knowledge needed for product testing.
* Integration done properly plays a key role in the overall process.
Finally, QA plays a number of key roles that are crucial to the process. First, QA is responsible for declaring the product ready for release. If you want to know the status of the project, ask QA. If QA cannot answer to the state of the project, then the process is broken. Second, QA consumes the inventory of unchecked development. Third, QA provides feedback to the development process.
Introducing Buffers…
This brings us conveniently to the concept of buffers. When you look at the development and integration process, you are creating an inventory of unchecked code – in ToC terms this is referred to as a buffer. In production environments, it’s a common fact that inventory is wasteful, expensive and degrades over time. In software development this is even more critical – the longer you wait to fix a bug, the more it costs. And the more you build on top of a bug, the more difficult it becomes to fix. It’s true that other buffers exist in the project, but this buffer is the most critical.
To illustrate the importance, I could bring up a number of scenarios but the one most illustrative to me is a conversation I had at a former employer. This company had implemented ISO900x for Software, Six Sigma for Software and another one that I don’t remember – in other words, they documented every step and had their numbers very well detailed. They said “development isn’t that critical because we only spend ~20% of our entire cycle in development”. When asked about testing they said “that’s where we spend all of our time, we never know when we can release a product because we’re stuck in QA without any way to predict when we’ll surface the product.”
This illustrates a key point – QA is part of development. Without QA you aren’t really developing a shippable product, you are creating a inventory of unknown quality. And not only that, by virtue of how development works, you are progressively building on top of questionable code that becomes increasingly more brittle as this charade continues. Delaying the interaction with QA means your code becomes more costly to change and more difficult to predict when you’ll be done.
In other words, development without QA is coding with your eyes closed – development with QA is coding with your eyes open.
Putting it All Together
Now that we have identified the key players at the heart of the Core Process Model, let’s take a look at how this works. Using the Theory of Constraints model…
(1) Identify the Constraint. As mentioned earlier, we have identified the constraint as Development. Not developers, but the entire development team for a given project. This constraint may move over time and you must be aware of it, but the natural tendency is for the constraint to move back to development.
(2) Exploit. Once the constraint is identified, the process is improved or otherwise supported to achieve its utmost capacity. In terms of software development it’s important to remember that starting and stopping the constraint or requiring the constraint to multi-task leads to huge productivity problems. In terms of software, you create an environment where development has an adequate environment, the proper tools and a productive team. If you're wondering where to start, I view Spolsky's 12 Steps to Better Code as the starting point.
(3) Subordinate. When the working process is working at maximum capacity, the speeds of other subordinate processes are paced to the speed or capacity of the constraint. QA is measured by how small the output buffer coming from development and integration is. And the specification/design buffers leading into development are large enough to keep the development/integration/qa process from starting and stopping. It’s also important to keep in mind that the specification/design buffer is high enough quality to base development on without having to constantly request clarification. And most importantly, create specs and designs at the last responsible moment to avoid waste and bad specifications.
(4) Elevate. If the output of the overall system is not satisfactory, choices can now be made regarding the constraint. Either improve the process, hire programmers and improve output over time or invest in new tools to make the team more effective. This is the process of elevating the constraint and eliminating the bottleneck.
(5) Repeat. When development is no longer the constraint, you can identify the other constraints – QA, Specifications, Designs or Integration can become the constraining factors. Address these constraints using the same process above to improve the flow of work. From what I’ve seen, the other players are much more easily elevated and adjusted, thereby returning the constraint to development after a short time.
The reason I place QA and Integration in the core loop is that given these two components, you can measure the true throughput of the system – measured by how fast you can get working product out the door. To quote Jim McCarthy (Dynamics of Software Development),
“Don’t try this without a QA group. It won’t work without a QA group. You’ve got to have someone whose job it is to articulate status. If you ask a developer the status of something he’s working on, his answer might be correct, but if it is, that’s just a coincidence.”
The Core Process model places QA in charge of keeping this buffer size minimized. This requires frequent pushes to QA – preferably daily as development progresses. QA is triaging the newly completed features and providing immediate feedback to the development team in terms of what is working and, more importantly, what isn’t.
Setting the Workflow
Once the pieces are in place, setting up the workflow is the final process-related step. While these items can be tracked in a Bug/Issue Tracking software package, tools like VS-Team System shines in its ability to handle the gating of dependent tasks. Gating is the toughest job to handle with a conventional bug tracking package and it can take a lot more thought and effort to manage.
A base list of issue status codes follows:
Requirement Complete
* Design Ready
In Design
Design Complete
* Development Ready
In Development
Development Complete
* QA Ready
In QA
*Pre-Validated
Final QA-Ready
Release Validated
The status codes above are used to determine the active state of the bug.
Developers triage the “Development Ready” list, pick the item and mark it “In Development”.
At any point, there is complete visibility of what each person is working on.
The same goes for QA in triaging the “QA Ready” status, this status enables QA to focus on the key buffer of the core process.
Note that the starred items are gating points – a gating point being the place where a decision is made when to release the work item into the rest of the process flow.
This list is not complete as items may be marked at the beginning to gather the Requirement Complete items. The needs in this area depend on how requirements are spelled out for the particular project.
Summary
The Core Process model focuses on the workflow, quality assurance and providing constant feedback during the entire process. The combination of these traits provides a very public and healthy view of progress. Progress can be measured against goals very easily and pertinent questions can be brought forth much more quickly than traditional models. In addition, because the process is responding to issue tickets and constantly being QA’d, the task list can be modified on-the-fly to adjust for new items and needs to adjust to new shipment dates.
The Core Process model does not call out release timeframes – sticking to hard-wired release timeframes is inviting gross sub-optimization. The common reason for implementing a standard, say 6-week, release cycle is to float the product up and stabilize it. The Core Process model keeps the product floating to QA throughout the lifecycle and the pre-release QA cycles are, in turn, shorter because of this focus. Prior to release, the entire set of "Pre-Validated" items are triaged by QA to a "Release Validated" state for one final check.
To recap, some of the strengths of the Core Process model are:
*Constant feedback on development code quality.
*Integration that evolves hand-in-hand with product development.
*Increased visibility of status and efficiency throughout the process.
*Ability to react to new requirements far more quickly – scope creep becomes *more manageable in terms of scheduling and describing impact more easily.
Note: Measuring efficiency can be a dangerous task. Comparing individuals based on data derived from issues completed can easily turn into a game – leading to sub-optimization at best and decreased moral at worst. This is nothing new to this process, but the increased information can provide the inexperienced manager with a temptation to do the wrong thing.
Some of the requirements of the Core Process model are:
* An increased load on Project/Team management to collaborate on gating schedules.
* Competent, technically skilled management that can identify and properly break apart the issues into discrete working issues.
The Core Process model has other implications not discussed here. For instance, Agile practices have a very natural home within this process. This is especially apparent as team size grows. In turn, this entire process takes the traditional testing ‘V’ and drives it into the process. There are a number of things I've glossed over such as the flow of items status, progression of pre-1.0 products, requirements for issue tracking software, etc.. These are all valid topics for an SDLC but I'll save those for another day...