When you write a unit test, what is it that your asserting? Most of the time you're doing state-based testing, i.e. your unit tests either validates a return value from a method call or a change in a property of the class being tested. Here is a typical example of state-based testing like you'll find in almost every introductory TDD article ever published.
[TestFixture]
public class StateTester
{
[Test]
public void TwoPlusThreeIsFive()
{
RunningSum sum = new RunningSum();
int actual = sum.AddIntegers(2, 3);
Assert.AreEqual(5, actual);
}
[Test]
public void AddSeveralNumbersAndGetTheRunningSum()
{
RunningSum sum = new RunningSum();
sum.AddInteger(2);
sum.AddInteger(3);
sum.AddInteger(5);
Assert.AreEqual(10, sum.Total);
}
}
Easy money, all you have to do is check an expected outcome for a known set of inputs. Unfortunately, the real world of enterprise software development quickly intrudes into your newfound TDD bliss. Toss in a bunch of dependencies external to your application and you quickly find that:
- The expected outcome is difficult or time-consuming to verify in an automated test
- The known inputs are difficult or time-consuming to setup in an automated test
- Tests that depend on external services can be brittle or just plain slow
- Measuring the outcome requires checking the state of all a class's dependencies, often making the unit tests too coarse and dependent upon the actual implementation details
- Too many classes have to be involved in the unit test
Teams abandon TDD as a failure when the tests are too difficult to write or just plain laborious. Nobody is going to willingly use a technique that is more bang than buck.
One way to alleviate the problem is to use interaction-based testing. Just like it sounds, interaction-based testing is verifying the expected interaction between a class and its dependencies. Typically you would use either a mock or a stub object as a stand-in for the real dependency. The unit test will verify that the expected interaction via method calls took place.
To illustrate the difference between testing state or interaction, here's a somewhat contrived user story (for best results, user stories should really be much finer grained than this):
Email Alert for Invoice Validation Failures User Story
- When an invoice is being processed, if certain nebulous validation failure conditions are met, send an email to the accounts payable individual assigned to the invoice. The email should list all of the invoice errors in a certain html format.
- If the invoice is really, really bad, send a summary email to the proper expediter. The email message will be html formatted in a particular manner.
- If the invoice is successfully validated, publish the invoice (via MSMQ)
Here's a possible C# class for this story designed with absolutely no regard for testability.
public class InvoiceProcessor
{
public void ProcessInvoices(Invoice[] invoices)
{
InvoiceErrorReport report = new InvoiceErrorReport();
foreach (Invoice invoice in invoices)
{
// Execute the validation business logic
InvoiceError[] invoiceErrors = this.validateQuantities(invoice);
report.StoreErrors(invoice, invoiceErrors);
}
// determine if the critical error threshold has been exceeded
if (this.hasTooManyCriticalErrors())
{
MailMessage message = this.createErrorSummaryMessage(report);
// Call to the local (or default) SMTP service to send the email
SmtpMail.Send(message);
}
// determine if the warning threshold for the error count has been reached
else if (this.hasTooManyErrors())
{
MailMessage message = this.createDetailsMessage(report);
SmtpMail.Send(message);
}
else
{
// If the invoices are okay, send the invoices on to the next process via MSMQ
MessageQueue queue = new MessageQueue(this.getMessageQueuePath());
queue.Send(invoices);
}
}
public MailMessage createErrorSummaryMessage(InvoiceErrorReport report)
{
MailMessage email = new MailMessage();
// Go get the user email addresses from the user information storage
// (database or Active Directory)
UserInformationStore userStore = new UserInformationStore();
email.To = userStore.GetEmailAddressesForInvoices(report);
email.Body = this.createErrorSummaryMessageBody(report);
return email;
}
// #region Other Methods
}
We could use a state-based strategy to test the email. We could run the test, then run around and ask the expected recipients to check their inbox (don't laugh too hard, I've been a party to this). Since we do want an automated test, we could query the Exchange server to see if the email was sent out correctly, but I'd rather gouge out my eyeballs with a dull spoon or code in Fortran 77 than attempt this (not to mention the look of apoplexy on the sysadmin's face when I ask for administrative rights to the Exchange server). You could also check an audit trail, but that's not the real functionality being tested.
The easiest approach in the case of the email tests is to use interaction testing to verify that the email was sent to the SMTP service. Now look at the same class, but somewhat more testable because it is designed for interaction testing between the class and the SMTP service.
public class InvoiceProcessor
{
// "Dependency Inversion Principle" -- All dependencies are now to an abstracted interface
private readonly IEmailGateway _emailGateway;
private readonly IMessagingGateway _messagingGateway;
private readonly IUserInformationStore _userStore;
// Use "Constructor Injection" to push dependencies into InvoiceProcessor
public InvoiceProcessor(
IEmailGateway emailGateway,
IMessagingGateway messagingGateway,
IUserInformationStore userStore)
{
_emailGateway = emailGateway;
_messagingGateway = messagingGateway;
_userStore = userStore;
}
public void ProcessInvoices(Invoice[] invoices)
{
InvoiceErrorReport report = new InvoiceErrorReport();
foreach (Invoice invoice in invoices)
{
// Execute the validation business logic
InvoiceError[] invoiceErrors = this.validateQuantities(invoice);
report.StoreErrors(invoice, invoiceErrors);
}
// determine if the critical error threshold has been exceeded
if (this.hasTooManyCriticalErrors())
{
MailMessage message = this.createErrorSummaryMessage(report);
_emailGateway.SendMail(message);
}
// determine if the warning threshold for the error count has been reached
else if (this.hasTooManyErrors())
{
MailMessage message = this.createDetailsMessage(report);
_emailGateway.SendMail(message);
}
else
{
// no longer responsible for knowing where the MSMQ path is. Or even if this
// is an MSMQ
_messagingGateway.SendInvoices(invoices);
}
}
public MailMessage createErrorSummaryMessage(InvoiceErrorReport report)
{
MailMessage email = new MailMessage();
// Go get the user email addresses from the user information storage
//(database or Active Directory)
email.To = _userStore.GetEmailAddressesForInvoices(report);
email.Body = this.createErrorSummaryMessageBody(report);
return email;
}
// Other methods ...
}
public interface IMessagingGateway
{
public void SendInvoices(Invoice[] invoices);
}
public interface IEmailGateway
{
public void SendMail(MailMessage message);
}
What's different here? First, I used the Dependency Inversion Principle to encapsulate the interaction with the SMTP email service and the MSMQ queue behind .Net interfaces (IEmailGateway and IMessagingGateway). The second change was to use Dependency Injection to push in the IEmailGateway and IMessagingGateway dependencies into the constructor function. Here's what a unit test might look like to test the email and MSMQ interaction using NMock to create mock objects as placeholders for the real dependencies.
[Test]
public void SendTheSummaryEmailIfTheCriticalErrorThresholdIsMet()
{
// 1.) Create the NMock objects
DynamicMock messagingMock = new DynamicMock(typeof(IMessagingGateway));
DynamicMock emailMock = new DynamicMock(typeof(IEmailGateway));
DynamicMock userStoreMock = new DynamicMock(typeof(IUserInformationStore));
// Setup the test data
Invoice[] invoices =
this.setUpArrayOfInvoicesThatWillSpawnACriticalNumberOfInvoiceErrors();
// 2.) Setup the expectations. Gets the email
// addresses, sends an email address, but does not put the invoice array onto
// the outgoing queue
emailMock.ExpectAndReturn(
"GetEmailAddressesForInvoices",
"somebody@somewhere.com",
new IsTypeOf(typeof(InvoiceErrorReport)));
messagingMock.ExpectNoCall("SendInvoices", typeof(Invoice[]));
emailMock.Expect("SendMail", new IsTypeOf(typeof(MailMessage)));
// 3.) Create the InvoiceProcessor object with the Mock instances
InvoiceProcessor processor = new InvoiceProcessor(
(IEmailGateway) emailMock.MockInstance,
(IMessagingGateway) messagingMock.MockInstance,
(IUserInformationStore) userStoreMock.MockInstance);
// 4.) Execute
processor.ProcessInvoices(invoices);
// 5.) Verify the correct calls were made
messagingMock.Verify();
emailMock.Verify();
userStoreMock.Verify();
}
All I've really tested in this unit test is that the InvoiceProcessor sent the proper signals to the IEmailGateway and IMessagingGateway interfaces. The test is way too coarse, but I had to leave something for later posts.
State or Interaction Testing, Which Do I Want to Use?
Not to be flippant, but use whichever mode of testing is easiest to apply on a unit test by unit test basis. Both styles of testing need to be in your intellectual toolbox. State-based testing is generally easier because of the overhead associated with setting up the mock objects, but interaction-based testing is essential for dealing with external dependencies like databases or middleware and user interface testing. I'll blog much more in the near future about how to maximize testability by design.
What Next?
The InvoiceProcessor structure above is better, but it can still be improved. The next step is to carefully consider how responsibilities can be assigned to classes to make testing easier. Design concepts like Inversion of Control and Dependency Injection (DI is a subset of IoC) can help tremendously to improve testability by reducing coupling. One of the best design philosophies is to quarantine external dependencies into narrow Gateway classes to minimize direct coupling to external code. It's also helpful to understand the best practices for using mocks or stubs. In my next set of blog posts, I'm going to further refine the invoice processing user story.
TDD Design Starter Kit Roadmap
- Assigning Responsibilities
- Inversion of Control
- Dependency Injection (with and without a Container)
- Gateway Pattern
- Mocks and Stubs - Best Practices
More Information
Read more about this from Nat Pryce and Martin Fowler. I actually disagree a bit with Fowler's discussion of interaction testing, so no more calling me a "Fowlbot" for a little while.
UPDATE 6/11/2005: Martin Fowler dinged me a little bit about this one. His main point about the difference in testing styles is that state-based testers use mocks or stubs for exceptional cases and interaction-based testers will try to use mocks in almost every case, including domain model classes. Once you put it that way, I'd lean more towards the state-based testing approach. Mocks get you out of tight spots, but overuse is a bad, bad thing.
UPDATE 7/20/2005: One of the points the interaction testing advocates suggest is the desire to keep classes segregated. State-based testing may lead to class's being less encapsulated as more state needs to be exposed as public getters. Enforcing an interaction based testing style might force the class design to a more desirable "Tell, don't ask" interaction between classes.