CodeBetter.Com
CodeBetter.Com
RSS 2.0 via Feedburner
           Do you Twitter? Follow us @CodeBetter

Aaron Jensen

May 2008 - Posts

  • Rhino.Mocks Performance Issue Fixed

    Just wanted to quickly note that I tracked down the performance issue in Rhino.Mocks and patched it. I also updated the original post with the new numbers. Enjoy!

  • Mock Framework Benchmarks

    UPDATE: I tracked down the issue and committed a patch to Rhino.Mocks. Rhino.Mocks is now much more competetive performance wise, our CI build time nearly halved, and about 4 minutes out of 7 of our test time has disappeared. New numbers below.

    I've complained before that Mocking is Slow but I never really dove further into it. Today I decided to actually compare Rhino.Mocks to other mock frameworks on a pure performance basis to see if it was a global problem. I timed 2000 unit tests across 100 classes with 20 tests each. The results were a bit surprising:

    Framework TD.NET Time nunit-console Time
    Rhino.Mocks old trunk 57.36s 28.82s
    Rhino.Mocks new trunk 22.94s 7.59s
    Moq trunk 18.30s 5.91s
    TypeMock 4.2.3 Reflective Mocks 15.36s 9.35s
    TypeMock 4.2.3 Natural Mocks 16.92s 9.56s

    That's right, according to these tests, Rhino.Mocks is at least 3 times slower than the other frameworks when under heavy load in TD.NET and five times slower in the console according to these tests. It's also interesting to note that TypeMock is faster than Moq in TD.NET, but slower in the console runner.

    While running the Rhino.Mocks tests it is very clear that there is a degrading performance issue. All the other frameworks executed tests with a near constant speed per test, but Rhino.Mocks slowed down noticeably about half way through.

    Please feel free to try it yourself, grab the project here. You should be able to just run the 4 strategy .bat files (run-rhino, run-moq, run-tmock-reflective, run-tmock-natural). Let me know if you find anything interesting.

  • Introducing Machine.Specifications (or MSpec for short)

    As some of you who follow me on twitter know, I've been working on Yet Another Context/Specification Framework as an experiment. Yeah, I know we already have NSpec and NBehave, and they're great and all, but MSpec takes things on from a slightly different angle, and it's just an experiment (for now). Here's a sample Description:

    [Description]
    public class Transferring_between_from_account_and_to_account
    {
      static Account fromAccount;
      static Account toAccount;
    
      Context before_each =()=>
      {
        fromAccount = new Account {Balance = 1m};
        toAccount = new Account {Balance = 1m};
      };
      
      When the_transfer_is_made =()=>
      {
        fromAccount.Transfer(1m, toAccount);
      };
       
      It should_debit_the_from_account_by_the_amount_transferred =()=>
      {
        fromAccount.Balance.ShouldEqual(0m);
      };
    
      It should_credit_the_to_account_by_the_amount_transferred =()=>
      {
        toAccount.Balance.ShouldEqual(2m);
      };
    }

    And a TestDriven.NET run:

    ------ Test started: Assembly: Machine.Specifications.Example.dll ------
    
    Transferring between from account and to account
      When the transfer is made
        * It should debit the from account by the amount transferred
        * It should credit the to account by the amount transferred
    
    
    2 passed, 0 failed, 0 skipped, took 0.79 seconds.

    Err, What?

    Different eh? The idea was heavily inspired by Scott Bellware's SpecUnit.Net framework he showed at the ALT.NET conference. It also took heavy cues from RSpec and my insanity. I realize that the the code doesn't look much like C# code and I'm OK with that. Many have and will ask why I don't just use Boo or RSpec w/ IronRuby eventually or even one of the existing Context/Spec/BDD frameworks. Those are good questions, but my main motivations are tooling and syntax. I enjoy the tooling I get in C# and I personally like the syntax in this library considering the limitations imposed by C#.

    How's it work?

    The simplest way to describe it is to compare it to a normal *Unit style testing framework:

    • Description = TestContext
    • Context before_each = SetUp
    • Context before_all = SetUpFixture
    • Context after_each = TearDown
    • Context after_all = TearDownFixture
    • When = Also SetUp, but happens after Context before_each
    • It = Test

    Rather than methods and attributes, MSpec uses named delegates and anonymous functions. The only reason for this is readability. You'll also notice that the fields used in the context are static. This is necessary so that the anonymous functions in the field initializers can access them. Probably the first thing you noticed is the =()=> construct. I won't mention the names that this was given on twitter, but I think it's an acceptable thing to have to deal with in exchange for the cleanliness of the rest of the syntax.

    Ok, you're crazy, but how do I try it?

    First, this is a very rough cut. Everything is subject to change as we experiment with the language. That said, here's how you play with it:

    1. Grab the drop here.
    2. Extract it somewhere. Put it somewhere semi-permanent because the TestDriven.NET runner will need a static location for the MSpec TDNet Runner.
    3. If you want TestDriven.NET support, run InstallTDNetRunner.bat
    4. Check out the example in Machine.Specifications.Example. Note that you can run with TD.NET.
    5. Create a project of your own. Just add Machine.Specifications.dll and get started.
    6. Send me feedback! Leave comments, email me, tweet me, whatever.

    Also, this is part of Machine, so feel free to take a look at the code and/or submit patches. There's also a Gallio adapter in there, but I didn't include it in the release as it's not quite polished enough yet. If you're interested in it, talk to me. Special thanks to Scott Bellware, Jeff Brown and Jamie Cansdale for their help and support.

More Posts