Defect Driven Testing… and You’re the Driver

November 21, 2010 § 3 Comments

Recently at $work I’ve been doing, quite frankly, a poor job of maintaining one of my areas of responsibility. I am the developer of a script that deals with the migration of about 12 million rows of data from one data format to another. This work has become sufficiently complicated that it can’t really be done in SQL, and has a set of Perl classes to do the migration. It’s a very critical piece of work right now, because it defines how long we have down time, and we only get to run this script once – if it’s wrong, then I may have corrupt a decades worth of historic data (effectively an audit trail). This is Not A Good Thing.

As you can see, I’m aware of these problems, and have made progress in increasing the speed of script, along with the quality of it. What I’m not so happy to tell you, is that I’ve also managed to commit this script in various forms of stability – from syntax errors, start up runtime errors, to errors that happen at the end of the script – something that you still end up waiting a good hour until you find out. I’m a perfectionist, and no doubt this cycle of “hack-break-hack-break-hack-hey it works!-hack-break-hack-fix” is getting extremely tedious, and doesn’t feel at all representative of my actual ability. Finally, that little moment of enlightenment has come and it’s time to sort this out. My solution? >buzzword<developer driven defect driven development>/buzzword< (someone please make me a manager).

Defect Driven Development

Before we get on to talking about my revolutionary testing approach which will probably cure world hunger, we need to discuss the concept of Defect Driven Development (DDD, for now). DDD is a testing methodology, where by you ensure you have a test for a bug. Dead simple. Here’s the basic methodology: you work on a great new feature and release it. User U finds a bug in your feature and reports it. You, the developer, write a test and then fix the bug. Now the bug will never occur again (theory here, work with me…)

Now here’s the twist:

You Are the User

While developing, consider yourself the user as well. So you hack away, and something breaks. At this point, you don’t fix the break, you write a test to reproduce it. And I mean everywhere. Even down to the smallest details – no matter where the bug is. Syntax error? Reproduce it in a test1. A function crashes with some input? Reproduce it in a test. The world is beginning to implode? Reprod… ok, it won’t really help here.

I doubt that I’ve really made much of an impact of this post but that’s my point. This is a very simple way to get tests into your codebase, and it has one some excellent consequences when you actually get to the end of your feature. Every new feature or enhancement really needs tests to go with it, but tests are hard, and testing is boring. I’ve written unit tests before that test basic exercise, but finding the edge cases that do break the system is difficult. I don’t think my tests often help me develop my feature, they are more useful later as I start to integrate tests for bug fixes. But with this new approach, you automatically do get some tests of where things have broken in the past – and I believe if it’s broke before, it’ll probably break again, unless you’re taking steps to prevent that.

1 While yes, the code wouldn’t actually run, in Perl we can still have a test to make sure something compiles. use_ok in the Test::More distribution can do this, for example.


Introducing Magpie – flexible test doubles & mocking for Perl

November 19, 2010 § 3 Comments


Magpie is a new distribution I have just released which brings the power of test doubles to Perl. There are already a few solutions to test doubles in Perl, but Magpie takes a different approach. Inspired heavily by Mockito for Java, Magpie gives you test doubles that are based are spying and verification, not expectations. So, before we really dive into it, how does it look?

use Test::Magpie qw( mock when );
use Test::More;
my $mocked_list = mock;


is($mocked_list->get(0) => 'first');
ok(exception { $mocked_list->get(1) });
is($mocked_list->get => undef);

So, what’s going on here? First of all we create a mock object. This object does every role, is a subclass of every class, and can run any method (returning undef by default). We then stub this object to handle some method calls using the when construct. We specify that when we request item 0 from our mocked list we should return the string ‘first’ and when request item 1, we throw an exception string ‘Kaboom!’. Simple! And as you can see, the tests following all verify this behaviour – this example is straight out of t/mockito_examples.t

What does Magpie have to offer?

What you just saw was the basics of Magpie – there are a lot more cool features available, that come in to be very useful!


As well as stubbing methods, you can also verify that methods were called. The following example from the synopsis illustrates how this may be useful:

use Test::Magpie qw( mock verify when );

my $baker = mock;
my $bakery = Bakery->new( bakers => [ $baker ] );
my $bread = $bakery->buy_loaf( amount => 2, type => 'white' );
verify($baker, times => 2)->bake_loaf('white');

As you can see, we are able to verify a method was called, and also add some extra details – for now the amount of times a method was called, and which arguments it was called with.

Argument matchers

Argument matchers allow you to be more general in your specification for stubs and verification. Rather than saying “when this method is called with exactly these arguments” we can say the more general “when this method is called with arguments that match these predicates.” In practice, it might look like this:

when($child)->eat(type(SugaryGoodness))->then_return('Ooo, yum!')

In this example Broccoli and SugaryGoodness are type constraints. There are already a few argument matchers that ship with Magpie, and it’s trivial to define your own with the custom_matcher generator.

Extra extra! Read more about it!

There’s more to Test::Magpie than what I’ve mentioned in this post, but if you’re interested, I recommend the official documentation. The basic and Mockito example tests serve as a great demonstration of how Magpie can be practically used.

Go go gadget CPAN – installing Magpie

Magpie is already available for use now, and is on cpan:

cpan Test::Magpie

From your shell, or however you wish to install CPAN modules

I really hope you enjoy this module, I’m already finding it powerful enough to use at work. If you have any criticisms, bugs, feature requests, or ponys to give me – drop me a comment here, an issue on RT, or poke me on IRC (I’m ocharles).

Happy testing!

Edit: 0.04 had a release problem and might not have installed cleanly. 0.05 should fix this. Sorry!

SQL – It’s data, not a string

April 3, 2010 § 2 Comments

Recently I’ve been becoming more and more frustrated with how we handle database interactions at work. We’ve just written a new version of our data objects and while they are considerably better than what we had before, they are still both limited and have too much boiler plate for my liking. As such, I spent some of my free time researching a better solution. So far, I’ve settled on Fey and have been extremely happy with the results.

One of the key differences between our old system and Fey is how SQL is represented. Previously, SQL was just written as a string, using a few helpers functions and the odd measure of string interpolation. To be able to compose queries, there were methods that returned just the set of columns, just the tables and joins – it worked, to a degree. Fey takes a different approach – instead of working with SQL strings, you work with SQL as data. And this abstraction turns out to be extremely powerful.

Here’s a real world example of why this matters so much. Our schema has “entity tables” and “name tables.” Many entities can have the same name, and good database design tells us we shouldn’t repeat ourselves, so we have “name tables” too. That is, each entity has one (or more) names. But not every entity has this, some do have the name just as a string, only the so called “core entities” have name tables.

But these name tables are are a technical detail, not something to be exposed by the API. That, is if I do Artist->get_by_id, I shouldn’t see this relationship at all. It turns out with roles and a bit of method modification, we can introduce this transparently. Here’s how I’ve started using Fey query objects:

package Entity;
has 'table' => ( is => 'ro', required => 1, init_arg => undef );
has 'select' => ( lazy_build => 1, is => 'ro' );
method _build_select { return Fey::SQL->new_select->select($self->table)->from($self->table);

Entity is an abstract base class – it can’t be instantiated because it requires the table accessor. So, our core entity just extends it:

package CoreEntity;
extends 'Entity';
has '+table' => ( default => sub { $schema->table('core_entity') } );

Just like this, we will bring in the name foreign key, but we actually want to bring in the actual name – the relationship should be transparent. We could modify core entity to do this, but that’s not particularly elegant. A better approach is to have a Name role:

package Name;
use Moose::Role;
around _build_select => sub {
my $orig = shift;
my ($self) = @_;
$self->$orig->from($self->table, $schema->table('name'));

Now, our core entity role only needs to consume the name role, and any selects done (using select) will automatically bring in that relationship. Even better, you can now throw this select object around to other classes and let them do even more with it. This is proving to be an extremely powerful way to write code that just dwim, with a minimum amount of boilerplate.

The example above is simplified, the actual implementation of these classes does a bit more by doing a little bit of database metadata poking – determining foreign keys and working out the join tables for you. But it’s built on the same principles of above. I think this reinforces one of my guiding points of programming – never use strings unless you are passing them over a boundary between your program. No XML strings, no HTML strings, no SQL strings, no strings to represent data (MooseX::Types instead of the standard Moose type constraints)… the more places you can apply this, the more you can transform your code with code. Maybe I’ve been reading too much Lisp.

Test driving Catalyst/DBIx::Class development: Introduction

March 21, 2010 § 1 Comment

Right now, I’m really big on test driven development. However, I always found it difficult to really apply TDD to my work – my work is not isolated enough to be easily approachable, not to mention I normally make use of a ton of modules that weren’t always designed with the test driven developer in mind. To push myself to learn more about overcoming these challenges, I decided to start work on a hobby project and be as religiously test driven as possible. And as if that wasn’t useful enough, I thought I’d share it all with you in blog form!

So, an introduction about the project and motivation first. My text editor of choice is Emacs – it’s hugely extendable with vast amounts of user contributed scripts and plenty of new things to learn about. However, these scripts and this knowledge is very loosely distributed. The best resource at the moment is the Emacs Wiki project, but I have a general distaste for wikis as a main source of information. Contrast this with Vim (a similarly well-extended text editor) and the homepage itself has a repository of scripts and tips.

I’ve decided to start a site my self – M-x customize. M-x customize will hold a repository of scripts and tips, and present them in a structured manor with searching, reviews and ratings. Hopefully in the long run I’ll actually integrate a package manager such as ELPA (I’ve contacted the author, but yet to get a reply).

A quick word on technology… because if you’re like me… the cooler the tech the more likely I am to read the blog posts! We’re going to be using Test::Mock – a mocking framework I’m working on in parallel with this project, Catalyst & Moose (of course), and templating with HTML::Zoom.

Next article we’ll get started with finishing the first user story for viewing scripts. Hold tight and watch this space!

Rocket powered rollerskates templating with HTML::Zoom & Catalyst

March 20, 2010 § 1 Comment

Matt Trout has produced yet more Perl magic with a new templating system – HTML::Zoom. I’d accuse him of committing a cardinal sin, but I guess we can let him off seeing as it’s badass.

A templating engine isn’t much use unless we’ve got something to drive it – and what’s a more natural choice than Catalyst? As always, Catalyst makes gluing things together trivial – so I’ve adopted some prior art from t0m and released Catalyst::View::HTML::Zoom. Now, this is version 0.001 so it wouldn’t be complete without a big fat blinking disclaimer: we might have done this wrong. I chatted about it briefly in #catalyst before starting work, but I still could have got it wrong! So the API could change and everything might blow up in your face… but that’s the fun, right?

Anyway, here’s a brief look at how we’ve got it structured at the moment (shamelessly plagiarized from my own CPAN synopsis):

package MyApp::View::HTML;
use Moose;
extends 'Catalyst::View::HTML::Zoom';

package MyApp::Controller::Wobble;
use Moose; BEGIN { extends 'Catalyst::Controller' }
sub dance : Local {
    my ($self, $c) = @_;
    $c->stash( shaking => 'hips' );

package MyApp::View::HTML::Controller;
use Moose;
sub dance {
    my ($self, $stash) = @_;

<p>Shake those <span id="shake" />!</p>

/wobble/dance => "<p>Shake those <span id="shake">hips!</span></p>";

The key class here is MyApp::View::HTML::Wobble which does the HTML::Zoom substitutions. $_ is locally aliased to the HTML::Zoom object for convenience and the first parameter is the stash object.

Happy hacking – let’s see where this ends up!

Marketing & CPAN – follow up and an example of changes

March 16, 2010 § 7 Comments

If you read my last post, you will have heard me complaining about’s interface about providing documentation. I argued that the quality of documentation was mixed, though the big projects (where it’s important to get as much wow factor as possible) had some great writing. However, the presentation is thoroughly lacking.

I don’t like complaining without trying to actually help, so I spent a bit of my free time today by making something that tries to incorporate the changes I mentioned.

A Catalyst documentation landing page

It’s probably lacking a little in the punch visually, but just look at that structure. As a new user I’ve got plenty of documentation about getting started – as a developer I’ve got easy access to the important APIs, and if I’m getting really geeky, there’s material for me to geek out with.

Now I should emphasize. None of this content is new – every single one of this pages mirrors something in the Catalyst::Manual. But I think that added structuring and a few explanatory paragraphs has made a world of difference. This change carries over nicely to the documentation pages themselves:

Viewing some Catalyst documentation

A readable structure, and I can jump straight into the content. If you think these arguments still hold for Catalyst, clearly you seen some of the pages…

I should stress that this is nothing more than an idea in my head at the moment. The above aren’t lockups, they are brain dumps. But I’m really keen to try and make this reality – I just don’t know enough about the Perl community to know who to talk to in order to get it done. So, if you want to talk about this – leave some comments and we’ll work something out 🙂 Even advice on who to talk too is much appreciated!

Where Am I?

You are currently browsing the Perl category at Cycles.