I spent a couple
years as an engineer on large petrochemical construction
projects. A lot of the mental work in a large construction
project is determining the order of construction for the various
subsystems and physical structures of the plant. In construction
that order is more or less governed by physical realities and it’s a
well understood problem domain. In the world of software
development determining the order of coding construction is more
nebulous because we’re not bound by physical laws. All the same though, the success of our projects is influenced by the order in which we code the pieces of the project.
Framework Fool’s Gold
I’ll freely admit it; I’m a recovering “framework-aholic” that’s had some bouts with “pattern-itis” as well. That’s
what makes me qualified to say that one of the worst project
anti-patterns in software development is building an elaborate
framework first. It’s a seductive path. On
one hand there’s the whispered belief that if we can just get the core
framework right, building new services/screens/controllers/applications
will be easy, right? On the other hand,
it’s just plain fun to exert your creativity to the maximum to build
the ultimate metadata-driven, pluggable framework so new features can
be added without any new code.
There are a couple little problems with building frameworks upfront. The main problem is that it very rarely works. You’re essentially guessing what the framework needs to do, and even the best architects can easily guess wrong. Even
if you’re the best architect in the world, the folks feeding you the
requirements are always changing their minds or coming to a different
understanding of their needs. Building an all-encompassing framework outside the scope of a specific business project almost always results in wasted effort. The worst case scenario is that you’re framework won’t deliver any value and results in a lot of rework. The
best case scenario is that your framework does work, but you don’t reap
much early feedback or have any opportunities to deploy the system
early.
Too much abstraction with an overarching framework early on can make an application viscous. Uncle Bob describes design viscosity as a malady that makes a design harder to preserve as changes are made. If
it’s easier to make a change by going around a framework or drilling
exception cases into a framework with “if/then” statements, you’ve
created a viscous framework that’s might be causing you more harm than
good. If you start to hear the phrases
“can’t we just go around the framework?” or “this would be easier
without the framework” or even worse “we need to get [Some Guy] to
change the framework first” you’ll know that you’re going down a
painful path with your framework.
The worst is when the
framework turns into the mutant plant from the Little Shop of Horrors
as it takes on a perverted life of its own. “Feed me Architect, Feed me!”
To give you an
anecdote that’ll get me in trouble when they read this, a part of my
team this past spring was embarking on an overly elaborate,
future-proofed architecture for our product rewrite. They
were having fits debugging and testing the application in large part
due to some unnecessary technical infrastructure (and severe
performance problems down the road). About halfway through the project they decided to simplify their architecture. The result was what they call the “Great Refactoring of Aught Five.” Funny name aside, there’s no such thing as a big refactoring, only rewrites. They
would have clearly been better off starting simple without the
abstractions and infrastructure (plus there’s still some dead code
detritus lurking in the codebase we’ve got to kill off some day).
I proscribe (now) to
the view that you should harvest a framework from an application only
after it has been proven to be useful or necessary. When you do build a framework, don’t try to create something grander than what you need right now. I’d
also say that it’s much, much better to “dogfood” a new framework as
part of a project rather than trying to write a “Hammer without a Nail”
framework that doesn’t really fit the needs of any project.
Slice the Work Vertically, Not Horizontally
The metaphor of software development as construction is common, but horribly wrong. Approaching
software development as constructing a structure on top of a
foundation, then adding the finishing trim can be a slow, painful path
to project failure.
Time and time again I’ve learned or observed that projects go much smoother when you build vertically instead of horizontally. What
I mean by this is that you build a new system by creating a feature at
a time with the entire stack of UI, business logic, and database work
to make the feature fully functional. As you build new features you religiously eliminate duplication to avoid creating a Stovepipe anti-pattern and harvest reusable code.
One of the more disappointing project failures I’ve observed was the very first .Net pilot project in a VB6 shop. I
wasn’t involved with the project, but I had a stake in the project as
it was going to prove the viability of .Net for application development
and also be a pilot for doing iterative development.* Unfortunately
they blundered by crafting an iteration plan that called for them to
write the entire data access layer first, the business layer second,
then finally put together the user interface. It failed miserably. When the project was cancelled midway they didn’t have the slightest bit of deployable code because they had focused solely on the backend database. One
of the other things they found out was that the data access they coded
wasn’t exactly what the business and UI layers needed. In other words, they had wasted effort by speculatively creating code that turned out to be unnecessary.
One of the best, positive lessons I’ve learned working on Agile projects is the importance of getting feedback early. You
can’t get much usable feedback from the end user when all you have to
show is database diagrams or UML class diagrams (a word of advice,
don’t ever show a customer a UML diagram unless it’s some kind of
experiment in hypnosis). Besides, you know damn well that the thing most likely to change is the user interface. You’ll also understand any piece of the system better when you’ve built the entire chain of calls. You simply cannot know any piece of code is done until its clients are complete.
I did a project about
3 years ago where I created an elaborate, overly generalized workflow
framework first before creating any working user interface. The framework worked fine and the system still humming along, but the lack of a UI became a problem. The
other members of the team didn’t truly understand the point of the
framework because they didn’t have a set of screens to see the concrete
usage of the framework. The users and the project manager had no idea what our progress was because they couldn’t see anything working. We didn’t get any usability feedback from our user base until dangerously late in the project. Yeah, you can claim earned hours for the code you create, but what value does that really represent? The
Agile methods have gotten it exactly right by tracking progress by user
stories completed, and defining that “done, done, done” for each user
story requires customer approval. And besides, the customer probably only counts a working user interface as progress anyway.
One of the desirable
qualities of an iterative process is being able to deploy a partially
complete system if the project has to come to an unexpected stop due to
changing priorities or funding problems. The only way to accomplish that goal is to build an application feature by feature rather than layer by layer.
Client First
Earlier this year
there was a raging argument on the web on the merits of writing web
services “Code First” (generate WSDL from code) versus “Contract First”
(generate code from WSDL)** that I mostly ignored. I
think there’s a time and place for both approaches, but the one
constant is to write (or at least simulate) the client first whenever
possible. The requirements and the signature of any service will be largely determined by the needs of its clients. “Client First” doesn’t just pertain to web services, it useful for any class that’s a service provider to other classes. If
you’re doing Test Driven Development (or not I suppose), you might use
a mock object or a stub to be a stand in for a service while
you concentrate on creating the service client. Once they’re completed, the unit tests for the service client will very accurately define the requirements for the service. For the obvious example, don’t write data access or persistence code long before the business logic that needs to be persisted. I
know some SOA enthusiasts who have a “build it and they will come”
mentality to creating web services left and right, but what’s the use
of building something that nobody wants?
Configuration Last
When it is time to
build some sort of configurable framework, my strongly worded advice is
to concentrate on the classes that do work first, and worry about the
configuration strategy second. I read that years ago in Craig Larman’s excellent book on design patterns and I clearly see the wisdom in that advice. StructureMap is a working dependency injection tool, but it started its life as the world’s greatest ORM tool. I got bogged down obsessing with the configuration infrastructure and never wrote a single line of actual O/R mapping code. Only later on did I take the abandoned configuration code and turn it into a dependency injection tool to use on a project. I still wonder what would have happened if I’d been able to release a solid O/R mapping tool for .Net 3 years ago.
Besides, I think
StructureMap and the other open source IoC/DI tools make it
ridiculously easy to create configurable plugin strategies compared to
what it took 3 or 4 years ago. Just push all
configuration information and dependencies in through constructors
or setter properties and let the IoC/DI tool put things together for
you later.
* This is a terrible example of working iteratively. The idea of working iteratively and incrementally is to build fully working pieces of the application, one at a time. What they did was just BDUF + Waterfall with some trappings of iterative development.
** I
had a colleague in my old architecture team who thought all of us
should be able to write WSDL by hand as a qualification for being on
the team. I blew him off while the others nodded fervently. There’s
a long list of holes in my resume I wish I had more familiarity with,
but writing WSDL by hand isn’t on that list at all.