LogicBoost blog

July 16th 2009 | Jimmy Bosse

StrictMock vs. DynamicMock: What are You Testing Here Anyway?

Here at Thycotic, we are TDD enthusiasts and pair developers. Test Driven Development and Pair Programming go together like chocolate and peanut butter, especially when you tackle a brand new piece of functionality and practice some green-field Ping-Pong programming.

For those unfamiliar with the concept of Ping-Pong programming, it’s a fun way to develop a new piece of business logic when you are not sure what the best method of implementation will be.

To start, one of you writes a failing test. The other makes the test pass, and then writes the next failing test (refactoring as needed). You make that test pass and write a new failing test. The goal of the session is to write the smallest amount of code possible, then try to write a test that effectively tests the desired functionality while providing a challenge to your pair. The session is easy at first, but as the tests move toward satisfying the more complicated requirements of the business, the exercise becomes more challenging.

Once your code becomes suitably complex, you’ll start to have multiple classes with dependencies—and with these dependencies come mock tests.

Then you have to decide: StrictMock or DynamicMock?

The problem with Strict Mocks is that you are required to set up your entire mock dependency far beyond the scope of the subject/system under test (SUT).

Take the following example.

[Test]
public virtual void ShouldSubscribeToViewEventWhenConstructed()
{
    MockRepository mocks = new MockRepository();
    IView mockIView = mock.StrictMock<IView>();
    mockView.SomeEvent += delegate{};
    LastCall.Constraints(Is.NotNull());
    mocks.ReplayAll();
    IPresenter presenter = new Presenter(mockView);
    mocks.VerifyAll();
}

To implement a presenter:

public class Presenter : IPresenter
{
    private IView _view;

    public Presenter(IView view)
    {
        _view = view;
        BindToEventsOn(_view);
    }

    private void BindToEventsOn(_view)
    {
        _view.SomeEvent += SomeEventHandler;
    }

    private void SomeEventHandler()
    {
        //Do something…
    }
}

The test is now green because you’ve subscribed to the event. But now that you’ve bound to the view in your constructor, you must always expect a call in your test whenever you mock IView with a StrictMock. This will make your tests verbose and it will be difficult to determine the actual SUT in each test.

Another issue is this: When you use a StrictMock you are essentially telling your pair what to program. Let’s say you and your pair sit down to create a calculator business object that must be able to add numbers together.

Your pair writes the following test:

[Test]
public virtual void ShouldReturnFourWhenGivenThreeAndOne()
{
    int firstNumber = 3;
    int secondNumber = 1;
    int expectedSum = 4;
    MockRepository mocks = new MockRepository();
    IAdder mockIAdder = mock.StrictMock<IAdder>();
    IAdderFactory mockIAdderFactory = mock.StrictMock<IAdderFactory>();
    Expect.Call(mockIAdderFactory.Create()).Return(mockIAdder);
    Expect.Call(mockIAdder.Add(firstNumber, secondNumber)).Return(4);
    mocks.ReplayAll();
    ICalculator calculator = new Calculator(mockIAdderFactory);
    int result = calculator.Add(firstNumber, secondNumber);
    mocks.VerifyAll();
    Assert.AreEqual(expectedSum, result);
}

Well, there’s nothing to think about here is there? If this is the way you want to write tests, invest some time and write a tool that will parse your test and create your business object for you. The test tells you “use the factory, create an adder, call the add method on the adder and return the result. Where’s the fun in that? What are you actually testing? Does it really matter how the calculator does what it needs to do?

Actually, I think the above test could be a good one if it was created three days into the calculator object—when you were refactoring the different pieces of the calculator into distinct service objects.

But right now, all you need is a calculator that can add two numbers together:

[Test]
public virtual void ShouldReturnFourWhenGivenThreeAndOne()
{
    ICaclulator calculator = new Calculator();
    Assert.AreEqual(4, calculator.Add(3, 1));
}

This should be your first failing test. I don’t care how you add it, but by golly you’d better give me back 4 when I give you 3 and 1.

I love DynamicMock because I am a believer that a test should test a very specific piece of code. Recently, however, I came across this poignant counterpoint that shattered my DynamicMock utopia. I had written tests and an object that looked something like this:

[Test]
public virtual void ShouldReturnTrueIfBigIsTrueAndBadIsFalse()
{
    MockRepository mocks = new MockRepository();
    IDataWrapper mockIDataWrapper = mock.DynamicMock<IDataWrapper>();
    SetupResult.For(mockIDataWrapper.IsBig).Return(true);
    SetupResult.For(mockIDataWrapper.IsBad).Return(false);
    mocks.ReplayAll();
    IDad dad = new Dad();
    Bool result = dad.IsABigBad(mockIDataWrapper);
    mocks.VerifyAll();
    Assert.IsTrue(result);
}

[Test]
public virtual void ShouldReturnTrueIfBigIsFalseAndBadIsTrue()
{
    MockRepository mocks = new MockRepository();
    IDataWrapper mockIDataWrapper = mock.DynamicMock<IDataWrapper>();
    SetupResult.For(mockIDataWrapper.IsBig).Return(false);
    SetupResult.For(mockIDataWrapper.IsBad).Return(true);
    mocks.ReplayAll();
    IDad dad = new Dad();
    Bool result = dad.IsABigBad(mockIDataWrapper);
    mocks.VerifyAll();
    Assert.IsTrue(result);
}

[Test]
public virtual void ShouldReturnFalseIfBigIsFalseAndBadIsFalse()
{
    MockRepository mocks = new MockRepository();
    IDataWrapper mockIDataWrapper = mock.DynamicMock<IDataWrapper>();
    SetupResult.For(mockIDataWrapper.IsBig).Return(false);
    SetupResult.For(mockIDataWrapper.IsBad).Return(false);
    mocks.ReplayAll();
    IDad dad = new Dad();
    Bool result = dad.IsABigBad(mockIDataWrapper);
    mocks.VerifyAll();
    Assert.IsFalse(result);
}

public interface IDataWrapper
{
    public IsBig { get; }
    public IsBad { get; }
}

public class Dad : IDad
{
    public virtual bool IsABigBad(IDataWrapper dataWrapper)
    {
        return dataWrapper.IsBig || dataWrapper.IsBad;
    }
}

My test was green and I was happy, but my pair informed me that a bug could be introduced accidentally and no one might ever notice:

public interface IDataWrapper
{
    public IsBig { get; }
    public IsBad { get; }
    public IsFat { get; }
}

public class Dad : IDad
{
    public virtual bool IsABigBad(IDataWrapper dataWrapper)
    {
        return dataWrapper.IsBig || dataWrapper.IsBad || dataWrapper.IsFat;
    }
}

Because a DynamicMock will not throw an exception for unexpectedly calling IsFat and will return false as the default for the bool base type, my tests will all remain green. But in production my code might not work as expected.

There is seldom a single solution that works in every situation. I have learned to find the proper place for both Dynamic and Strict Mocks in my TDD toolbox and hope that this encourages you to think more deeply about your own toolbox.

Jimmy Bosse is a Senior .NET Consultant and Team Lead at Thycotic Software, an agile software consulting and product development company based in Washington DC. Secret Server is our flagship password management software product.