Mock Objects

TDD. Killing the messenger?

2008-09-25T20:30:00.002+01:00

Are we really doomed to another generation of procedural code masquerading as objects?

Roy Osherove has come out against "mocking" in favour of "isolating". One reading of his point is that mocking is just too hard for the bulk of the programmers he sees. Perhaps that's true (the others developers who discovered the technique are all very good) but I hope not (either that or I should increase my rates). And I'm pleased to see Oren, for one, has responded.

Roy addresses important issues which we need to think about, but we shouldn't pitch the whole industry at the level of its least skilled members; I've spent too much of my life cleaning up code that should never have been written in the first place. In the meantime, if his staff don't understand basic Object-Orientation, then that's what he should be teaching first, with a sprinkling of acceptance testing on top for regression.

Reading the post and its comments again, I see repeated "Doctor, it hurts when I do that" moments. Roy's response is to back off interaction testing, but that misses the experience and insight that led us into this situation in the first place. My call is that, like Object-Orientation, the technique is widely misunderstood, which is why the same objections keep coming back. This isn't a comfortable position.

To take the most obvious example, as Roy says himself, record/replay just doesn't cut it. That's why we've never supported it. It's easy to start with, but then my experience is that it encourages unmaintainable tests. It focusses the language of a test on the accident of the current implementation, rather than on the significant relationships between objects. It also encourages asserting every interaction, which leads to just the sort of problem that everyone complains about, brittle overspecified tests. That's why we don't do it, we only assert interactions that can affect the world around the object under test and stub the rest.

The other disaster (yes, I think record/replay has been a disaster) was our early attempt to mock the entire Java libraries. Apart from being unrealistic, it sent entirely the wrong message, emphasising isolating from databases over guiding the design. Later, Joe Walnes coined the notion of "Only Mock types you own", which really helped some people get the idea.

As someone posted somewhere in this exchange, TDD will punish you if you don't understand design. I'd argue that that's what it's for, and that it's essential to listen to what the tests are saying rather than turning down the volume.

Reordering chapters

2008-09-23T08:44:00.003+01:00

We've cut down the introduction and moved some of the material into a new section, Examining the Process. Check it out and let us know what you think. More content to follow soon.

Chapter 9. Passing the First Test

2008-09-04T08:03:00.005+01:00

We've posted another episode from our long worked example.

At the start of every test run, we need our test script to start up the Openfire server, create accounts for the Sniper and Auction, and then run the tests. Each test will start instances of the application and fake auction and test their communication through the server.

We've also made some changes in response to comments on the discussion list

Long Worked Example

2008-08-28T10:44:00.002+01:00

We've posted the first couple of chapters of the long worked example from our book, Growing Object Oriented Software Guided by Tests. It's work in progress: some of the diagrams have not been finished quite yet. As ever, feedback is welcome via the discussion group.

New Table of Contents Online

2008-08-21T23:03:00.003+01:00

We've posted the new Table of Contents. From now on, we'll update this as we put chapters online and shuffle content about.

Chapter 5. An Introduction to the Tools

2008-08-21T13:59:00.008+01:00

We have put a new chapter of the book online: Chapter 5. An Introduction to the Tools.

What happened to chapter 4? We've moved some chapters around after reading people's comments in the discussion group and will publish chapter 4 as soon as we've finished it and polished it up. We'll put the new table of contents up ASAP.

You can give comments and feedback via the discussion group or as blog comments.

Update: explained why we are only giving a cursory introduction to the tools. Improve explanation of why we chose JUnit and include where you can download it from.

Chapter 3. Kick-Starting the Test-Driven Cycle

2008-08-06T11:52:00.006+01:00

We have put a new chapter of the book online: Chapter 3. Kick-Starting the Test-Driven Cycle.

You can give comments and feedback via the discussion group or as blog comments.

Update: added a sidebar about kick-starting "brownfield" development.

Chapter 2. Test-Driven Development with Objects

2008-07-25T12:50:00.004+01:00

We are now publishing chapters of our book, Growing Object-Oriented Software Guided by Tests in a separate section of the site. We will publish news about new content in the blog but will not include the entire contents of each chapter as blog posts.

We have updated Chapter 2, Test-Driven Development with Objects in response to feedback from blog comments and the discussion group and improved some of the diagrams. More comments are always welcome.

Book Discussion Group

2008-07-16T13:13:00.003+01:00

We have created a Yahoo Group to host feedback and discussion about the book we're working on. If you have any feedback we'd love to hear from you.

Chapter 1. What's the point of Test-Driven Development?

2008-07-14T21:34:00.013+01:00

We have updated Chapter 1, What's the Point of Test-Driven Development? in response to feedback from blog comments and the discussion group and improved some of the diagrams. More comments are always welcome.

Growing Object-Oriented Software, Guided by Tests

2008-07-13T23:49:00.014+01:00

We have been working for some time on a book about our approach to developing software test-first. We've been working at it long enough that we though it was time to start putting up some material to get some feedback which, after all, is a technique we go on and on about. We'll start with chapters in HTML, then add some PDFs on the side as the material grows.

Here's our opening.

A Deceptively Simple Idea

Test-Driven Development (TDD) is a deceptively simple idea: write the tests for your code before writing the code itself. We say “deceptively simple” because this reversal fundamentally changes the role that testing plays in the development process and challenges the industry's assumptions about what testing is for. Testing is no longer just about keeping defects from the users, instead it's about helping the team to understand the features that the users need and to deliver those features reliably and predictably. When followed to its conclusions, TDD radically changes the way we develop software and, in our experience, dramatically improves the quality of the systems we build, in particular their reliability and their flexibility in response to new requirements.

Test-Driven Development is widely used in “agile” software development methods. It is a core practice of Extreme Programming (XP) [Beck99], is recommended by Crystal Clear [Cockburn04] and often used in Scrum™ [SB01] projects. If you're going to work on an agile project, you will probably use TDD. We've used TDD on every agile project we've been involved in and have snuck the practises in to non-agile projects. We've even found it useful in pure R&D projects that explore possibilities rather than meet customer needs and so have no need of the customer-facing project management practices of agile methods.

More...

The Table of Contents links to the published chapters.

Feedback and Discussion

If you want to give us feedback on what we've written so far and see what other people think about it, please join the discussion group.

Public training courses?

2008-06-05T15:43:00.004+01:00

We've had a couple of inquiries about whether we will be running our training course publicly. We hadn't planned to, but perhaps we should. We propose to run a two-day course priced at £1100 if booked less than 4 weeks ahead of the teach date, with a discount to £800 if booked earlier (plus UK VAT). The most obvious location is London (UK).

If you think you might be interested (no commitment implied), please drop us an email at public.course@mockobjects.com with this information:

Name: 
London is acceptable:
Preferred other location:
Preferred month for teach date:
Possible number of attendees:
Coding experience (years): 
TDD experience (years):
Contact email:
Contact phone:

We'll see if we can work something out

In case you're feeling nervous, whatever details we receive will be used only by us just to coordinate a course. We won't hassle you or pass them on to anyone else.

Talk at London Geek Night.

2008-05-30T10:47:00.007+01:00

Nat and I gave a talk at the most recent London Geek Night. It was an introduction to our approach to Test-Driven Development, incidentally making the point that some of the state- vs. interaction-based TDD wars are a bit specious; it's a matter of picking the right tool for the job. We had a good crowd and some useful questions. Afterwards, there was a Code Dojo in which Nat took part while I got stuck at the back talking to people. Thanks again to our hosts for an interesting evening.

We've uploaded the slides with notes. The format's a little odd because we used the toolchain we use for our courses, but the material's all there.

There are couple of other relevant links. Nils Haugen has a nice talk he did on Improving Test Maintainability, which is about making sure the tests make sense. Our interest is more in the stage after that, which is about listening to what the tests are telling you about the design of your code.

Learn from the source!

2008-05-23T17:37:00.006+01:00

Nat, Romilly Cocking, and I are now open for business for training. We have a course on Test-Driven Development that reflects nearly a decade of experience writing tests first (that's 30 years in parallel). We also have significant experience in training, Romilly has a couple of decades on his own, and Nat and I have taught commercially and at two of the top CS departments in the country.

The course teaches the practice of Test-Driven Development, based on a solid understanding of Object-Oriented design, and with a strong emphasis on making tests expressive and maintainable. We also make a point of exploring how unit tests can be used to drive the design of the code, not just protect it.

The content is modular, so we can adjust it to match the needs of the participants. The full-length course is three busy days, and we've taught versions that are a half-day and whole-day long.

We teach the course using jUnit4, Hamcrest, and (of course) jMock2. Talk to us if you want something different.

For more information, contact us at: training@mockobjects.com

Here's what some attendees liked about the course:

“The practical exposure to a better way of testing”

“Practical lab work, real-world concrete examples, Expert lecturers!”

“Chance to make mistakes and learn. Exposure to new technology.”

“A very good whole picture view of TDD and mocks”

“Interaction and explanation of course materials particularly through the exercises.”

Another round in the testability debate

2008-03-08T09:52:00.003Z

This time a posting from Mark Seemann has raised a slew of comments.

One of them is a note from Colin Jack about the annoyance of producing interface/implementation pairs all the time. My first response is that that sounds to me a bit like a problem with style. Maybe it's just wordplay, but usually I don't extract interfaces from classes, I implement interfaces that I've already discovered in some previous test.

My second response is to wonder how much this is a tools issue. I don't believe there's anything in the .Net world that yet matches the responsiveness of the usual Java IDE's. It makes a difference as to what's plausible. I remember the huge shift in perception when first VisualAge for Java and then JetBrains' Idea came out. In retrospect, I always spent more on time on rework than many people I worked with ¹ but it sure took a lot more time in emacs (and I was pretty good at it), even if I was working in a better language.

1) which is not to say who was right, I'm just wired that way...

New Tutorial at QCon

2008-02-12T10:03:00.000Z

On Tuesday 11th March, at QCon London, Romilly Cocking and I will be running a new tutorial we're developing with Nat Pryce on Test-Driven Development with Mock Objects following on from a successful first run at the last XpDay. Sign up soon!

Just when you thought it was safe to go back in the water...

2008-01-20T11:57:00.000Z

Rising to the bait again (to keep the fishy metaphor going), there's yet another discussion of how the ability to hack the runtime changes the world. At one level that's true, but not in this case.

Dependency Injection is not a recent invention that a cabal of TDDers forced on the world, and it's absolutely not something that we came up with just so we could crowbar in our Mock Objects.

The relevant design guideline, named the Law of Demeter ("Strong Suggestion" isn't very catchy) was first described by Karl Lieberherr in 1987! The Mock Object pattern came from people applying their substantial O-O experience to TDD in Java, trying to figure out how best to avoid exposing implementation details in unit tests. One of the critical lessons from Demeter is that objects should have explicit dependencies. It helps to keep them focussed on their responsibilities and, as a result, easier to maintain—and a good way to make dependencies explicit is to pass them in.

Unfortunately, since then the world seems to have filled up with DI frameworks which cloud what should be a coding style discussion. This is not about having to configure every last corner of your application in XML, this is about how objects, or small clusters of them, get to talk to each other.

Roy asks, "[...] do you need DI all over the place, or just in specific places where you know dependencies could be a problem?" Well no—if you have enough foresight to know where those places are. I'm struggling at the moment to test against a framework where everything is helpfully packaged up nice and tight so I can't create an instance of one of its core objects. It's actually well written, but the authors just weren't good enough at prophecy to figure out my particular need. That's why I don't rely just on my intuition, I use the needs of my unit tests to help me figure out where the seams should be. To counter the FUD argument, I have absolutely no problem with saying that I don't want tools that do magic because I need guidance with my code.

As Roy (very politely) concludes, there isn't a high enough proportion of really good code in the world (some of it mine) that we should be in hurry to cut back on techniques such as DI. Just because something has been around for a while doesn't mean it's been superseded, especially in as conventional an environment as .Net.

Avoid mega unit tests

2008-01-13T09:59:00.001Z

We've just had a posting to the jMock user list that included the following:

I was involved in a project recently where JMock was used quite heavily. Looking back, here's what I found:

The unit tests where at times unreadable (no idea what they were doing).

Some tests classes would reach 500 lines in addition to inheriting an abstract class which also would have up to 500 lines.

Refactoring would lead to massive changes in test code.

I've seen this failure mode on another project I've been helping with, so I think there might be a common pattern.

I don't think any unit test code should get that large, except for unusual circumstances. Unit tests are supposed to focus on at most a few classes and shouldn't need a large amount of set up. What I saw on the other project was enormous amounts of preparation and positioning to get the objects into a state where the expected feature could be exercised. Of course it's hard to understand the point of a test when there's just so much code. If you see this pattern, then I'd suggest that the code (or at least the test code) needs breaking up a bit. On the other hand, integration and acceptance tests that happen to be written using a unit test framework might well be larger.

One thing I need to explore is whether the naive use of interaction-testing is particularly susceptible to this failing, or whether it happens all the time and we're the only ones who get complained to. I am, however, convinced that the emphasis on mainly using Mocks to substitute external systems (some of which I perpetrated myself in the early days) is a deeply bad idea which pushes teams towards the sort of problem described here.

Mock objects for Java ME

2007-12-22T14:13:00.000Z

Carl Meijer has asked us to announce his new project...

Yesterday I uploaded a personal project, Hammock, to sourceforge (http://sourceforge.net/projects/hammockmocks). This a mock object framework for Java ME. Java ME, unfortunately, doesn't support reflection making jMock, EasyMock, etc. unsuitable for use on a mobile device or emulator. Obviously this makes creating mocks more difficult than with the Java SE mock object frameworks but the distribution includes a utility, HammockMaker, for creating the source for a mock object given a class file (of a non-final class or of an interface). The framework includes mocks of many of the standard Java ME classes for communications such as HTTP, SMS, OBEX, etc so that one can test without actually hitting a network. I've used it on three projects at my current employer.

Don't tie down your code, use interfaces

2007-12-04T11:33:00.000Z

Recently, I was responding to a question on the TDD list and found myself writing this:

Although there's only one use of the interface, you could think of it as a way of keeping a clean distinction between the naming of different domains (user and persistence). It makes managing package dependencies easier too.

which is what I've been meaning to say for ages. The reason we're keen on discovering interfaces in our code is that we think they're a good way of expressing concepts in a domain.

If I'm writing a ring tones store (the video store folded last year) and I need to look up, say, the customers that were active last month, I'll ask a SalesLedger object. So, what type is SalesLedger? If it's a class then my marketing module will have a dependency on whatever persistence framework I use. If my SalesLedger class is clean and delegates to some further persistence type then I've only deferred the problem. I'm importing, directly or indirectly, the persistence packages into my domain code. If I do the same with a few other frameworks (printing, display, network, etc.) then I'll have tied down my domain module with third party dependencies. The module dependency chain looks like this:

There are two problems with this, one obvious and one subtle. The obvious point is that I've raised the risk of inflexible code. One day I'll have to rip out a messy chain of dependencies to get to some code I want to change. Now, because I'm another World's Greatest Software Developer, I'll be careful not to embed the dependency too deeply and keep my packages clean and layered, but that's intellectual overhead I could do without. When SalesLedger is an interface, I have a clear distinctions between the domains of marketing and persistence, and I cannot let the abstractions leak through. If I do this for all my external dependencies, I can package up my domain code and use it in all sorts of ways I haven't thought of yet. The top-level Application then becomes a matchmaker, introducing all the modules to each other and helping them get started.

The more subtle issue is that SalesLedger, the interface I use in the body of my code, is in a different domain from the HibernateLedger, a class that implements it (and maybe some other interfaces). The code that uses a SalesLedger is concerned with marketing and payments, the code that uses a HibernateLedger is concerned with setting up connections. There may only be one implementation of an interface, but the two types are addressing different things.

Is this breaking YAGNI? I don't think so (although there are others that do). I'm not adding features I haven't identified a need for yet. Part of the deal with YAGNI is that I make the code as expressive of my current intentions as I can, and I think I'm expressing the needs of the domain code needs more clearly by limiting its dependency to an interface that defines just the services it needs from other parts of the system.

n.b. Gulliver image taken from news.bbc.co.uk

MockObjects in Bangalore

2007-11-21T14:46:00.001Z

I'll be presenting a tutorial/workshop/Dojo on TDD with Mock Objects at Agile India in Bangalore this Sunday. I hope the locals will find the material interesting (and that they give me an easy ride while I struggle with the jet lag :).

"Testing towards A Specification: A Systematic Approach"

2007-10-19T10:19:00.000+01:00

Peter Dimov has published his Master's thesis. To quote him:

It is based on Test Driven Development with Mock Objects, hence the focus is on interaction-based testing. I'm introducing the mocking framework CalitkoMocks, which is specially designed to provide a notation for writing tests that can be mapped one-to-one to sequence diagrams. Sequence diagrams are classically used to specify object-oriented software systems, however, it is obvious that just a bunch of diagrams representing individual cases cannot really be regarded as a specification. To address this point, I'm also presenting a few test refactorings which help to extract test scenarios (i.e. generalized test cases), which are more suitable to use as a specification.

He's implemented a mock framework for C++ that's part of Calitko

I especially liked his closing remark

After I drew the diagram corresponding to my test, I decided to try to write a test corresponding one-to-one to the sequence diagram. The sequence diagram showed the calls to my stub objects but my test didn’t (because the implementation invoked them implicitly). I ended up writing smarter versions of my stubs which eventually evolved into mocks. Though I had already read about mocks at that time, I didn’t quite get it what kind of beasts these were. It was only after I reinvented the mock that I truly understood the concept.

Perhaps this is a step on the way to addressing Jason Gorman's concerns.

XpDay London 19/20 November

2007-10-16T13:29:00.000+01:00

We have a couple of sessions at XpDay London. Steve will be giving our Synaesthesia talk on Monday. On Tuesday, Nat, with Romily Cocking, Andy Pols, and Steve (part-time) will be giving an introductory tutorial on TDD with Mocks. Book soon!

Mock Throwable

2007-10-13T11:13:00.000+01:00

Yesterday I visited a team at a Cyrus Innovation site. They came up with this example. If you can't read it, the writing on the ball says "Throwable". Cute.

Blogger in German?

2007-10-02T00:14:00.000+01:00

On a side note, this blog is "powered by Blogger" and I keep getting bits of the UI text in German. Does anyone have any idea what's happening?