CodeBetter.Com
CodeBetter.Com
RSS 2.0 via Feedburner
           Do you Twitter? Follow us @CodeBetter

Jeremy D. Miller -- The Shade Tree Developer

Under the hood and working with .Net, TDD, Software Design, and Agile Stuff

Spiking a new syntax for integration tests. Whaddaya think?

Hello, my name is Jeremy, and I sometimes write unit tests specifications that are uglier than sin.  Specifically, when I started to look back at the interaction style tests I've been writing over the past couple years I'm finding a bunch of unit tests that are indecipherable to me, the author of those very tests.  I came to the conclusion that I needed to start writing my interaction tests in a different fashion to promote readability -- and hence better maintainability. 

I was really concerned about readability more than anything else, but a quicker way to write integration tests would be nice to.  To that end, I've been following the Eleutian guys' posts on their AutoMockingContainer strategy.  Honestly, I've been dubious about the value of the AutoMockingContainer idea.  I get the mechanical savings, but worried that it was too much magic that would obfuscate the unit tests somewhat.  I've got the StructureMap code open for its upcoming 2.5 release, so I decided to stick in some facility for "AutoMocking." 

The auto mocking stuff was easy enough to do, but I wanted to go farther in terms of readability.  I've found that one of the factors related to integration tests being confusing to newbies and hard to read for anyone is simply the order in which things happen.  In the traditional RhinoMocks record/replay model I call a bunch of expectations on the mock objects, then perform the actual action on the class under test, then call VerifyAll().  It's Yoda syntax.  Instead, what if we could express the tests in a more natural order of Given blah, when I execute blah, this stuff should happen.

Here's what I came up with in my spike.  This is working code, but I don't want to talk about the mechanics of the underlying code.  I'm mostly interested in what y'all think about the syntax and flow of the test.  I'm purposely leaving out comments or trying to explain the grammar upfront because I want to see how "soluble" this syntax is if you're looking at it cold.

This is a rewrite of an existing unit test in StoryTeller.  Upon looking at the code, I think there was a simpler way to write the code, but that's a refactoring for another day.  The new and old code effectively express the exact same unit test.  The auto mocking does do a lot to cut down on monotonous setup code.  I definitely think this code is going to be smoother in C# 3.0, but I want to release a 2.0 version before I do that.

The New Code

        [Test]
        public void Execute_the_command()
        {
            string theFragmentName = "Fragment1";
            Fragment theNewFragment = new Fragment(theFragmentName);
            
            GivenThatThe<Suite>().Is(new Suite());

            Given = delegate {
                That.Calling(Service<IInputCommand>().PropertyValue).Return(theFragmentName);              
            };

            WhenExecuting = delegate  {
                AddNamedFragmentCommand.Execute();
            };

            TheInteractionShouldBe = delegate {
                Call<ISystemUnderTest>().Save(theNewFragment);
                Call<ILeafNode>().RefreshNodes();
                Call<IFragmentPresenter>().StartEditing(theNewFragment, Member<ISystemUnderTest>());
                Call<IApplicationController>().OpenScreen(Member<IFragmentPresenter>());
            };

            Then = delegate {
                Assert.IsNotNull(Member<Suite>().FindFragment(theFragmentName));           
            };
        }

The Old Code for the Same Test

        [SetUp]
        public void SetUp()
        {
            _mocks = new MockRepository();
            _node = _mocks.CreateMock<ILeafNode>();
            _suite = new Suite();
            _systemUnderTest = _mocks.CreateMock<ISystemUnderTest>();
            _inputDialog = _mocks.CreateMock<IInputCommand>();

            _applicationController = _mocks.CreateMock<IApplicationController>();

            _command =
                new AddNamedFragmentCommand(_node, _suite, _systemUnderTest, _inputDialog, _applicationController);
        }

        #endregion

        private MockRepository _mocks;
        private ILeafNode _node;
        private Suite _suite;
        private ISystemUnderTest _systemUnderTest;
        private IInputCommand _inputDialog;
        private AddNamedFragmentCommand _command;
        private IApplicationController _applicationController;

        [Test]
        public void Execute()
        {
            string theFragmentName = "Fragment1";
            And fragmentConstraint = new And(
                new TypeOf(typeof (Fragment)),
                new PropertyIs("Name", theFragmentName)
                );

            IFragmentPresenter fragmentPresenter = _mocks.CreateMock<IFragmentPresenter>();
            ObjectFactory.InjectStub(typeof (IFragmentPresenter), fragmentPresenter);


            Expect.Call(_inputDialog.PropertyValue).Return(theFragmentName);


            // Save the fragment
            _systemUnderTest.Save(null);
            LastCall.Constraints(fragmentConstraint);

            _node.RefreshNodes();

            // Open the screen
            fragmentPresenter.StartEditing(null, _systemUnderTest);
            LastCall.Constraints(fragmentConstraint, new Equal(_systemUnderTest));
            _applicationController.OpenScreen(fragmentPresenter);

            _mocks.ReplayAll();
            _command.Execute();
            _mocks.VerifyAll();

            Assert.IsNotNull(_suite.FindFragment(theFragmentName));
        }

 



Comments

ISerializable - Roy Osherove's Blog said:

The semantics of how you write a unit test, the basic syntax, or Domain Specific language of how we write

# January 17, 2008 6:23 AM

Igor Brejc said:

I agree with jason. Tests using mocks will always be as complicated as the system that is being tested. As someone who's seen your code snippet for the first time, I still had to make an effort to understand it.

My approach to increasing readability is by using code comments, both as headers of test methods describing what is being tested and how (if it is really complicated) and also inline comments for certain things which may not be obvious.

I have the same problem with forgetting what some old test code was all about ("old" in my case means few months ) and code comments do help. But I support your effort if you think it helps.

# January 17, 2008 7:01 AM

Kyle Baley said:

It's hard to say after a first glance. The Given...WhenExecuting...TheInteractionShouldBe...Then is easy enough to follow but the stuff inside each of these blocks isn't right off the bat. But I do get the impression it would be easy to get used to.

The thing is that it looks even more magical than the AutoMockingContainer. Having said that, I like the magic of AutoMockingContainer so I could get used to what you're doing pretty easily. It's nice that I don't even need to retrieve a reference to the mocked objects directly. Others may argue it's hard to figure out what it's calling all these methods on in the TheInterfactionShouldBe section.

Side note: I find using the Record and Play methods in Rhino.Mocks goes a long way to adding readability:

using ( _mocks.Record )

{

   // Set expectations

}

using ( _mocks.Play( ) )

{

   // Replay code

}

Still requires knowledge of mocking but it's dead simple to see how the Replay/Verify code is separated.

# January 17, 2008 9:14 AM

Christopher Bennage said:

I think C# is getting in the way: all of the <> () delegate ...

# January 17, 2008 9:53 AM

Jeremy D. Miller said:

@Christopher,

Yeah.  Chad Myers played with it a bit using C# 3.0 lambdas and it still had a lot of code noise.  This is looking like it's going to end up being yet another "C# just cannot be Ruby" idea.

# January 17, 2008 9:59 AM

Troy DeMonbreun said:

"Yoda syntax" - Genious wit!

# January 17, 2008 4:55 PM

Mike Suarez said:

I think the new approach is very stylish but not that readable. I do something like your old code but with more refactors for readability. Here’s how I would refactor it.

1. Rename _command to subject. It’s odd that there’s a _systemUnderTest there already n’ it’s mocked. The first line in my BehaviourTest is the subject (or sut declaration).

AddNamedFragmentCommand subject;

2. Extract method givenTheFragment(string name). I use assume instead of given but at the end it the same, setup a stub for the test.

3. Extract method expectToSave(Fragment fragment).

4. Extract method expectToRefreshTreeNodes().

5. Extract method expectToStartEditing(Fragment fragment).

6. Extract method expectToOpenScreen().

7. Extract method assertThatExistFragment(string name).

Then all those methods go into regions and all mocks into another region called main cast and the test would look like:

public class AddNamedFragmentCommandTest : BehaviourTest {

private AddNamedFragmentCommand subject;

#region main cast

#region fixture setup

#region given

#region expectations

#region assertions

[TestMethod]

public void AddNamedFragmentCommand_adds_new_Fragment(){

            string name = "Fragment1";

Fragment fragment = new Fragment(name);

givenTheFragment(name);

expectToSave(fragment);

expectToRefreshTreeNodes();

expectToOpenScreen();

subject.execute;

assertThatExistFragment(name).

}

}

*: I hide ReplayAll in the last expectCall. Shame on me, but takes away the noise from the test. The VerifyAll() call is hidden in the [TestCleanup].

# January 17, 2008 7:47 PM

Javier Lozano said:

I totally agree with jason that the new syntax is a bit harder to read but easy to understand since it "reads" as if you were expressing it verbally.  However, like Chris said, C# does "get in the way" since, like you said, "it's not Ruby".

For my tests, I'm currently using the AutoMockingContainer that's part of Rhino.Testing and it's working pretty well for me in conjunction with the Record and Playback syntax.  For example, here's  snippet:

using (Record)

{

   PatientEntity patient = Mock<PatientEntity>();

   PatientEntityDTO dto = Mock<PatientEntityDTO>();

   info.PatientEntity = dto;

   IEntityTranslator<PatientEntityDTO, PatientEntity> translator =

       Get<IEntityTranslator<PatientEntityDTO, PatientEntity>>();

   Expect.Call(translator.ConvertTo(dto)).Return(patient);

   IFormXmlProvider<PatientEntity> xmlProvider = Get<IFormXmlProvider<PatientEntity>>();

   Expect.Call(xmlProvider.GetFormXml(patient, null)).Return(eventXml);

   LastCall.IgnoreArguments();

}

My Get<T>() method is part of my AutoMockingFixture class that handles all the "dirty" syntax needed to create mocks or get items from the container.

Granted my tests are probably not as complex as yours.

# January 18, 2008 10:36 AM

Jeremy D. Miller said:

@Javier,

I have an automocking container that basically does the same thing as building the mocks and class under test as the Get<T>.  I just wanted to flirt with going farther with a new mini language that's built around the automocking container.  I think I'm going to try it just a little bit more, but only with C#3.

"Granted my tests are probably not as complex as yours."

That might be a design problem in *my* code.

# January 18, 2008 10:50 AM

Mike Suarez said:

I was showing this post to a coworker and I did a bit more of refactor. Here’s the result:

[TestMethod]

public void AddNamedFragmentCommand_adds_new_Fragment(){

 givenTheFragment(name);

 expectToSave(fragment);

 expectToRefreshTreeNodes();

 expectToShowInScreen(fragment);

 subject.execute();

 assertThatExistFragment(name).

}

# January 18, 2008 11:45 AM

Colin Jack said:

You seem to be using integration and interaction interchangeably in this article, is that intentional?

I don't find the new one particularly readable. Its probably better but still not that understandable and Mike Suarez's solution seems to me to be more readable.

# January 18, 2008 12:20 PM

Jeremy D. Miller said:

@Colin,

That was me being sloppy.  It should always read "interaction."  I thought I caught that.

Mike's is clearly more readable, but a bit more work.  I was trying to make the mechanics easier too.

# January 18, 2008 12:41 PM

Mike Suarez said:

@Jeremy

I don’t actually refactor into those given/expect/assert methods. I write then in Pseudo Code style, not as comments (like in Code Complete) but as real functions and Resharper generates then. The extra effort is moving the pack into a region but it pays off in readability and Test as living documentation. What you see in a class after collapsing all is something like:

[TestClass]

public class AddNamedFragmentCommandTest : BehaviourTest {

private AddNamedFragmentCommand subject;

#region main cast

#region supporting roles

#region given

#region expectations

#region assertions

[TestMethod]

public void AddNamedFragmentCommand_adds_new_Fragment(){}

}

*: fragment and name are declared in the supporting roles section.

You can start expanding accordingly your depth interest. If that’s going too far I’m not so sure. A friend recommended throwing the Symetry patterm from Becks new book. Plus I’d like to add the “treat your tests as your production code” + Custom Assertions from Meszaros’s . Anyways I’m a n00b and I tend to go too far. So it’s a good sign that you like my approach as more readable but it looks like is not worst the effort. Thanks

# January 18, 2008 1:23 PM

Jeremy D. Miller said:

@Mike,

Gotcha.  I think I like that.  I've done that a bit with Extract Method refactorings, but I think you're on to something.

# January 18, 2008 6:54 PM

Colin Jack said:

Yeah I like that too, testing by intention at the high level seems to me like a good way to go.

# January 19, 2008 2:06 PM

Leave a Comment

(required)  
(optional)
(required)  

Enter the numbers above:
Add

About Jeremy D. Miller

Jeremy began his IT career writing "Shadow IT" applications to automate his engineering documentation, then wandered into software development because it looked like more fun. Jeremy previously worked as a systems architect building mission critical supply chain software for a Fortune 100 company and learned agile development practices as a .Net consultant at ThoughtWorks, one of the pioneers of agile development. Jeremy is the author of the open source StructureMap (http://structuremap.sourceforge.net) tool for Dependency Injection with .Net and the forthcoming StoryTeller (http://storyteller.tigris.org) tool for supercharged FIT testing in .Net. Jeremy's thoughts on just about everything software related can be found on his weblog "The Shade Tree Developer" at http://codebetter.com/blogs/jeremy.miller, part of the popular CodeBetter site. Jeremy is a Microsoft MVP for C#. Check out Devlicio.us!

This Blog

Syndication

News

All opinions expressed here constitute my (Jeremy D. Miller's) personal opinion, and do not necessarily represent the opinion of any other organization or person, including (but not limited to) my fellow employees, my employer, its clients or their agents.

About Me

"Best Of" Compendium

StructureMap (Dependency Injection for .Net)

StoryTeller (Supercharged Fit)

Build your own Cab

TestDriven

MVP