Thursday
Jan022014

The Future Of Computing

The future is already here! - it's just not very evenly distributed.

I have some ideas about what computing will be like in the future but it is composed mostly of pieces we already have - or have the promise of. At the centre of my vision is the evolution of the Post-PC device

What is Post PC anyway?

Many people attribute this term to Steve Jobs, who certainly brought it to the mainstream in 2007, using it to describe iOS devices and how they would come to eclipse "traditional" PCs in sales and use. This is already coming to pass. But it was actually David Clark who coined the phrase, back in 1999. That article is really worth a read. You should go and read it now. Go on. I'll wait. (Actually I'll just carry on writing - but the appearance will be the same).

So while the Jobsian vision (initially, at least) refers to the reset in expectation, interaction and ease of use that iOS devices ushered in, Clarks original words encompass more - including Cloud Services, cashless payment systems, and most interestingly (to me) finer grained distribution of responsibilities.

It's that last one where I think the most opportunities are yet to play out.

For two or three decades we have obsessed over convergence. Traditional PC systems converged to a single device - the laptop. Post-PC devices have taken that to the next level - a single slab, fronted by a piece of glass that is both the display and primary input. These tiny devices also pack in cameras, extra sensors and even fingerprint scanners and replace what used to be dozens of separate devices. But they have also been born into a world where wireless communication technologies are ubiquitous and come in many forms. Many of their capabilities are distributed in "the cloud", or consist of sending things between devices or connecting wirelessly with additional "smart" peripherals such as cameras, fitness trackers, printers and other devices. They are intensely personal yet highly social. Autonomous yet democratised. Functions such as Airplay and its counterparts reinforce the idea that these devices are not isolated computing silos. They are participants in a computing ecosystem that is distributed at many different levels. And all so seamlessly that entire demographics that were previously written off as "computer illiterate" are regularly using these devices. They are barely even considered "computers" anymore. The term has come to be associated with that clunky, finicky, bulky thing you used to struggle to get to do anything you want.

This new generation of devices, finally, "just works".

The NeXT Steps

So where does it go from here? Have we reached the end of the evolution of the personal computing device?

Not by a long shot! We're just getting warmed up!

We have just crossed the threshold from general-purpose computers being primarily for the focused used of businesses and enthusiasts to being something that everyone uses and carries with them everywhere. That in itself has been opening up possibilities that had been hitherto unseen or simply not feasible.

The degree to which these devices and their interconnections have embedded themselves into our lives already is quite breath-taking when you take a step back. While, admittedly, I'm a bit of an early adopter, none of the following is particularly extreme:

On a typical, weekday, morning I am awoken by music served as an alarm from my phone. I get up and go to begin my bathroom routine. Part of that routine involves stepping onto a set of scales that take my weight and fat mass and automatically send the figures, via wi-fi, to a cloud service that is immediately accessible to my phone, collated together with a number of other metrics that are tracked over time.

Once finished and dressed I leave the house and go to my car, which automatically unlocks itself due to the proximity of the key fob in my pocket. I get in and push a button and the car starts. As I start driving the media system in the car has automatically connected, via bluetooth, to my phone, which is also still in my pocket, and continues playing the podcast that I had previously been listening to. I drive to the station and park the car.

As I get out I put my bluetooth headphones on and, at the push of another button, they too have connected to my phone (still in my pocket) and the podcast resumes once again. I get on the train and get my laptop out to do some development work. It connects via a personal wi-fi network to my phone for an internet connection (which, when I pick up LTE, is faster than my home broadband was only a few years ago) - all the time it is still sending audio to my headphones. Later I get off the train and walk to my office. As I walk my steps are being counted by a device on my belt that intermittently sends this information on to my phone via Bluetooth LE, where it is sent to the cloud service that is collating my health related measurements - including heart rate and blood pressure. Along my journey something interesting and unexpected happens. I take out my phone and take a photo, then continue on. As I get near the office a reminder pops up that I had set to go off in that proximity. Eventually I get to my desk where I put my phone in a dock to charge because battery technology is still struggling to keep up with all these demands!

We're only just getting started, so it's not all as seamless as it could be yet, but the story I've just recounted is real and usually all "just works" without a hitch. I think, as time goes on, these sort of experiences will become more reliable and encompass more things.

But that's the present - wasn't I going to be talking about the future? Well I apologise for burying the lede but it's important to remember how much of the future is already here (albeit not evenly distributed). And my vision is really an extension of the things already discussed. That may sound a little uninspiring - but remember that phenomenon of incremental advances suddenly creating whole new opportunities?

Evenly distributed

One of the criticisms often levelled at the current crop of Post-PC devices is that they are great for consumption, but less so for content creation - or "real work". Many contend that you still need a "real" PC for that. I don't think it's quite so black and white - but there do remain many tasks that are cumbersome to undertake with a tablet or smartphone. It won't always be that way, though. Although tablets with keyboards and mice, and hybrid operating systems, exist now - that's not the way of the future.

I believe that in the not too distant future touch-screens, keyboards, and other input devices will all be merely components of a distributed "system" that consists of both cloud services and local sharing of storage and processing. This system will scale seamlessly to the task at hand. Whether you need more computational power, a different input metaphor, or a different way to output you should be able to add what you need without missing a beat. Right now if your needs outgrow a tablet you have to switch to a whole different device (a laptop, say) - which may or may not sync over data you were working on - in this future you would just add the keyboard if you need it (more easily than now), add some extra processing units (you can do this now in certain limited ways), extra storage (again cloud services already play a role here - as does card based storage in some tablets) or even an extra display (technologies like AirPlay are showing the promise of this).

Each of these components would be what we call "smart". That is they are computers in their own right with enough processing power and sensors to be aware of their environment and how they connect and interact. Take a display, for example. The display itself would contain accelerometers and gyroscopes so it is aware of it's orientation in the real world and whether it is being moved - just like your tablet or smartphone does now. It would also know when another display is nearby, and if so how near and in what direction. Of course the display would be a touch-screen. Imagine you have an object on one display. You could start up a new display, place it next to the first one, touch the object and "flick" it over to the second display. All without any need to configure anything.

Now this system, distributed as it is, would need a centralised "brain". It must scale down to a single device that can be used in isolation. It would make sense for this to be what we currently think of as a smartphone. We would need to carry them with us everywhere and use them for communication, so it would be equipped with audio input and output and cameras - just as our current smartphones are. In fact they needn't be much different to the smartphones we have now. They would be more powerful - but needn't be much more powerful as they can scale up the processing power as needed with additional devices and/ or cloud services. And with all data synced to cloud services an alternate device could be picked up and made into your primary hub for the day as necessary.

Everyday revisited

Most of the pieces are already there. There are some challenges - mostly business-oriented rather than technical - but the trend is already in this direction. Yet it all seems very incremental. To see how transformative it would be consider a re-run of my story earlier, reworked to showcase these future technologies (and a few others to spice it up a bit).

It's a typical, weekday, morning. I am awoken by music serving as an alarm on my primary computing device (which will have a really cool name). I get up and go to begin my bathroom routine. Part of that routine involves having various health metrics samples and sent to a cloud service. Another part is that my bathroom mirror presents me with some curated information pertinent to the day ahead - the current weather, traffic conditions and any early appointments I have set. Perhaps also the days news headlines.

Once finished and dressed I leave the house and go to my car, which automatically unlocks itself due to the proximity of the computing device in my pocket. I get in and push a button and the car starts. As I start driving the media system in the car has automatically connected to my computing device, which is still in my pocket, and continues playing the podcast that I had previously been listening to. I drive to the station and park the car. My computing device knows that I have just parked in a car park and automatically communicates with the car park server and pays for my day's stay.

Just before I get out I ask the device to switch it's audio over to the earpieces embedded in my ears and the podcast resumes once again. I get on the train and get my tablet out to do some development work - which is, of course, already online. I might also fish out a keyboard - which automatically connects as it comes into proximity with the tablet. Later I get off the train and walk to my office. As I walk my steps are being counted by the peripheral device on my wrist where it is collated along with my other health measurements and sent to the cloud. Along my journey something interesting and unexpected happens. I bring out my device to take a photo. But I really want a good quality picture, so I quickly fish out a lens with a full size sensor from my bag, which wirelessly connects to my device and instantly beefs up the optics to professional standards. I take a great picture then continue on. As I get near the office a reminder pops up on my wrist that I had set to go off in that vicinity. Eventually I get to my desk where I put my device on the wireless charging pad as it connects to my keyboard and large displays and I continue the work I started on the train.

The task at hand

One consequence of this more distributed way of working is that the single-(main-)tasking metaphor that the iPhone doggedly champions is allowed to survive while still allowing multiple applications to run and be interactive. The metaphor becomes "one app per device". Each device is typically running one interactive application at a time - for some devices it is the same app at any time (a keyboard, for example). For a more general purpose device, such as a tablet, it may run one app, while a different app runs on the "phone" beside it. But the devices can see each other and documents and other data may be shared between them - probably using real-world metaphors like the "flick" mentioned earlier.

Conversely at any one time two or more devices may appear to be running a portion of the same app - but in truth they will be running their own instances - with tight integration between them.

My vision of the future is one of heterogenous, smart devices - some specialised, some generalised - participating in the fabric of a system that surrounds us - and which tends to recede into our surroundings. The seeds are there - and they're growing. I think the next decade is going to be an exciting and transformative time in technology - perhaps even more so than the last!

Postscript...

I had wanted to publish this post by New Year's Eve (2013) but didn't get time to finish up by then. I'm pushing it now, largely un-edited, to try and keep it relatively seasonal (but I may come back and edit more aggressively yet - it's much too rambling for my liking).

As I was finishing I saw blog post by Dave Addey - which he actually posted back in September - covering very similar material. I haven't had a chance to think how to work it in organically to this post (yet) but didn't want to miss the opportunity to link to it - so I'll do that explicitly here. Go read it now. Go on, I'll wait.

Friday
Dec132013

Optional streaming

Catch has a number of macros that allow values of arbitrary types to be streamed into an ostringstream. The canonical example is the INFO macro:

INFO( "There were " << bottles.size() << " green bottles, hanging on the wall" );

This macro builds up a string that will be passed to the next assertion to be included as an annotation. Note that, unlike with a naked ostringstream there is no leading <<. This makes it clean and uncluttered when you just want to log a single value (such as a string), for example:

INFO( "Weirdness" );
The obvious way to do this is for the macro to provide the leading << prior to its argument. Conceptually something like this:
#define INFO( log ) { \
	std::ostringstream oss; \
	oss << log; \
	useTheString( oss.str() ); 
}

This all works quite nicely. But there are a few other macros that use this idiom, too: WARN, SUCCEED and FAIL.

The last two are of interest because the logging behaviour is more of a secondary concern. The primary behaviour is to appear like a passing or failing assertion, respectively, without the need to actually assert on anything. SUCCEED can be useful if you otherwise have no assertions in a test and you don't want to see warnings about it. FAIL is useful if the situation that leads to the failure is not captured in an expression for some reason. It can also be useful to force a test to fail, perhaps as a placeholder. These are useful macros to have available, but they are not often needed in practice. So when they are it's nice to be able to annotate their useage inline - hence the streamed argument.

This is all well and good. But I've found there are still enough cases where I don't want to annotate that having to pass an empty string or make something up is a little annoying. I also use a similar idiom in other projects where it would be nice to be able to make the stream completely optional.

This is not as easy as it sounds, though. The first, and most obvious, issue is that this requires support for variadic macros. Catch has made use of variadic macros, where available, for some time now. In theory they are available to any C++11 compiler. In practice most, if not all, compilers that support any reasonable chunk of C++11 support variadic macros - and most supported them as an extension even before that. That's certainly true of Visual C++, GCC and Clang.

The technically more interesting problem, though, is dealing with that initial <<. Remember the first << is being supplied inside the macro. It will still be there even if the caller does not supply an argument to the macro. If we wrote FAIL the same way we presented INFO earlier (but with variadic macros) it might look something like this:

#define FAIL( ... ) { \
	std::ostringstream oss; \
	oss << __VA_ARGS__; \
	notifyFail( oss.str() ); \
}
... which, with no argument provided, would expand to...
{ 
	std::ostringstream oss; 
	oss << ; 
	notifyFail( oss.str() ); 
}

Do you see the problem? With nothing following the << this will not compile.

So do we need a different operator? What properties would we need? It seems we'd need an operator that comes in two forms: a binary operator that allows us to capture an argument, and a unary operator that allows us to omit the argument. Furthermore the binary form must not require its argument to be enclosed in any sort of brackets. Finally it must have higher precedence than << so we can switch over to normal stream insertion at that point.

That's a long list. Does such an operator exist? Fortunately there's not just one but two such operators to choose from! + and -. The only slight hitch is that the unary form is right-to-left associative, whereas the binary form is left-to-right. So how can we work these in?

Let's pick one of the operators. I've gone with +, but I don't think there is any advantage either way. Because unary + is right-to-left associative it needs to prefix something. So we can't use it at the start of our streaming expression. We can, however, use it at the end. Then we'll need an object to apply it to. The object doesn't actually need to do anything else. I've gone with this implementation of StreamEndStop in Catch:

struct StreamEndStop {
    std::string operator+() {
        return std::string();
    }
};
With this definition the expression, +StreamEndStop() now yields an empty string - which is idempotent with a stringstream. Which means we can write:
{
	std::ostringstream oss; 
	oss << +StreamEndStop();
	notifyFail( oss.str() ); 
}
And oss.str() evaluates to an empty string. Perfect. But what about when we do stream something? Well that would expand to:
{
	std::ostringstream oss; 
	oss << something +StreamEndStop();
	notifyFail( oss.str() ); 
}
... where something could be a string or variable or literal of any type. So we need some way for the expression:
something +StreamEndStop()
to yield the value of something. That's where the binary form of operator+ comes in:
template<typename T>
T const& operator + ( T const& value, StreamEndStop& ) {
	return value;
}
Now, whether we supply nothing, a single value or multiple values joined by <<s we'll end up with a stringstream containing what we expect. The relevant bit of code in Catch actually looks like this:
Catch::ExpressionResultBuilder( messageType ) \
	<< __VA_ARGS__ \
	+::Catch::StreamEndStop()
which yields an ExpressionResultBuilder that gets passed on elsewhere. This is all protected by CATCH_CONFIG_VARIADIC_MACROS. Otherwise it falls back to:
Catch::ExpressionResultBuilder( messageType ) << log
So a lot of work to save a few explicit empty strings, but sometimes it's the little things.
Saturday
Sep072013

The Minimum Viable Wiki

Documentation is usually considered a necessary evil. The truth is probably closer to it being necessary and evil. For most of us it's one of those things we feel we should have more of (or any at all) but never have the time. And how much documentation do we need anyway?

Most project documentation is worse than useless. Why? For the same reason that most comments in code are useless-to-dangerous - only more so. Documentation, when written, is rarely kept up-to-date. It is rarely complete. It was often never "true" to start with.

In short the docs are just not reliable

Oh, there are exceptions, of course. API docs generated from source code, for example. They will be consistent, right? Perhaps. Assuming they are kept maintained within the source. It is still possible for them to lie.

If we are Agile we don't do documentation, do we? After all the manifesto states:

Working software over comprehensive documentation

Of course it doesn't say we don't value documentation - just that we value working software more. And even then it talks about comprehensive documentation.

There is the idea of "Just Enough Documentation". I hear people talk about this and see it referred to, along with some personal interpretation. But I have yet to see a definitive write-up of this idea (please let me know if you know of one). Perhaps this is meant to be deliberately ironic? In any case, my understanding is that this relates to the sort of documents that are often seen as deliverables of a project - for example a requirements doc.

I'm not going to talk so much about those, although the principle I'll describe may often be applied. I want to concentrate on the internal, developer-oriented, documentation that describes things such as how a project is organised, what bits do what, what to expect in certain situations etc. This is often captured in a wiki that is specific to a team.

This sort of documentation is often seen as unnecessary in a Agile team. This is where face-to-face communication between team members is sufficient, isn't it?

That view seems to be borne out on projects that do have a wiki. The wiki is rarely complete or up-to-date. It quickly falls out of use altogether, or is approached with trepidation as it is as likely to mislead as inform. If it is kept up-to-date it is likely due to a Mandate From Above, which leads to more time invested in its maintenance than is worth it.

So is there a way this can be made to work? Can it ever be worth it? Why would we need it at all?

Well that first depends on your project and team. Maybe the conversations alone really are sufficient. My experience is that this reaches a limit pretty quickly. We need something. How many times have you, or someone on your team, wasted a day looking for something or going down a wrong path, only for someone else to then say, "oh, that's over here", or, "that's done differently for x reason". Yes the conversation was had - but time was already wasted

So the problem is knowing when and who to ask - especially for things you would have thought were "obvious"

In terms of the Four stages of competence this is about moving from Unconscious Incompetence (you don't know what you don't know) to Conscious Incompetence (you at least know what things you don't know). This is much more valuable than it sounds at first! If you don't know that adding a setting to a config file dumps the information you need to a log you won't spend six hours writing a whole chunk of ad-hoc logging code to do the same thing, for example. Instead you'll remember, "there was something about being able to enable logging in a config file". Now you know to ask the question and finding the answer should be quick and accurate.

Having a full specification, perhaps along with examples, of every attribute that could go in the config file might sound better. And if that config file was meant for end-user customisation you'd probably need that. But if it's some internal thing then any attempt at such documentation is likely to be incomplete and/ or out of date - if written at all

But a line that says, "logging is configurable", somewhere that people will read leaves out those noisy, unstable, details and simply informs you of what is possible. You now know what you don't know (i.e. how to enable logging), whereas before you didn't even know it was possible. Details can be asked for.

So my recommendation for maintaining team information on a wiki is to stick to the briefest possible comment that moves the reader from Unconscious to Conscious Incompetence. Unless you really need it avoid the level of detail that will leave it unmaintained and incomplete.

Thursday
Jul112013

Injecting Singletons in Objective-C Unit Tests

I've promised to write this up a few times now. As I've just given another talk that covers it I thought it was time to make good on that promise.

The topic is the use of singletons in UIKit (and AppKit) and how that makes code using them hard to test. These APIs are riddled with singletons and you can't really avoid them. In case you need convincing that singletons are problematic take this contrived function:

NSString* makeWidget() {
    NSString* colour = 
        [[NSUserDefaults standardUserDefaults] stringForKey: @"defaultColour"];
    return [colour stringByAppendingString: @"Widget"];
}

NSUserDefaults is a singleton - the sole instance of which is returned when you call standardUserDefaults.

Monster1

A perturbing problem

Now consider how we might test this code. Obviously in an example this trivial there are various ways we could change the code to make the problem go away. Consider this a scaled down example of a problem that may be deeper in the code - perhaps a legacy code-base (or even some third party library!).

A naive test might set the "defaultColour" key in NSUserDefaults prior to calling makeWidget(). The problem with that is that the environment is left in a changed state after the test. Subsequent tests may now pick up a different value if they use NSUserDefaults. Worse: NSUserDefaults is backed by persistent storage that can potentially leave your whole user account in a changed state!

So, at the very least, we should restore the prior value at the end of the test. This leads to further problems: If the test fails, or an exception is otherwise thrown, the clean-up would not be called. So we'd need to wrap it in a @try-@finally too. Then, can we be sure we know what value to restore it to. It's probably nil - but if it's not the environment is still in a different state. So we should capture the prior value first and hold it in a variable.

Now what if you need to set more than one value. Or you change the keys used. We're starting to do a lot of bookkeeping just to compensate for the fact that a singleton is being used. Not only is it ugly but it's increasingly error prone.

Better if we can avoid this in the first place. If we have the option - prefer to pass dependencies in - rather than have your code reach out to these Dependency Singularities. In our example either pass in the default colour, or failing that, pass in NSUserDefaults.

NSString* makeWidget( NSUserDefaults* defaults ) {
    NSString* colour = [defaults stringForKey: @"defaultColour"];
    return [colour stringByAppendingString: @"Widget"];
}

At first this doesn't seem to buy us much. We still need an instance of NSUserDefaults. Even if we alloc-init it we'll get a copy of the global one. That's better but we'd still be dependent on the environment and have to take steps to compensate. And in other cases we may not even have that option

Monster2

If you can't make it - fake it!

We might not be able to create completely fresh instances of NSUserDefaults - but we can create instances of a stand-in class. Due to Objective-C's dynamic nature we don't even need to subclass - and we only have to implement the methods that are actually called - in this case stringForKey:. We could do that with a Mock Object. Or we can build our own Fake. Let's assume you've written a Fake called FakeUserDefaults, which contains an NSMutableDictionary, a means to populate it (perhaps via an initialiser) and an implementation of stringForKey: that looks the key up in the dictionary. Now we can test like this:

TEST_CASE() {
    id defaults =
        [[FakeUserDefaults alloc] initWithValue: @"Red" 
                                         forKey: @"defaultColour"];
    REQUIRE_THAT( makeWidget( defaults ), StartsWith( @"Red" ) );    
}

Great. That seems to tick all the boxes. We have complete control of the default value and we haven't perturbed our environment. No clean-up is required at the end of the test (not even memory, if we're using ARC)

Assuming you have the freedom to change the code under test, here, of course. If makeWidget() was buried deep in some legacy code, for example, it may not be feasible to make such a change (yet). Even if we can make the change it can be useful to be able to put the test in first to watch your back while you change it. If we need to leave the call to [NSUserDefaults standardUserDefaults] baked into the code under test for whatever reason what else can we do?

Monster3

To catch a singleton we must think like a singleton

What we'd like is that, when standardUserDefaults is called on NSUserDefaults deep in the bowels of the code under test, it returns an instance of our fake class instead - but only while we're testing. Again, due to Objective-C's dynamic nature we can achieve this. But it starts to get messier. It involves gritty low-level functions from objc/runtime.h. Can we package that away somewhere?

Of course we can! Enter TBCSingletonInjector. I've uploaded the code to GitHub, but there's actually not much to it. It exposes one public (class) method:

+(void) injectSingleton: (id) injectedSingleton
              intoClass: (Class) originalClass
            forSelector: (SEL)originalSelector
              withBlock: (void (^)(void) ) code;

The usage is best explained by example:

TEST_CASE() {
    id defaults =
        [[FakeUserDefaults alloc] initWithValue:@"Red" forKey:@"defaultColour"];

    [TBCSingletonInjector injectSingleton: defaults
                                intoClass: [NSUserDefaults class]
                              forSelector: @selector(standardUserDefaults)
                                withBlock: ^ {
            REQUIRE_THAT( makeWidget(), StartsWith( @"Red" ) );
        } ];
}

Magic! How does it work? It uses a technique known as "method swizzling" (Ruby or Pythonists know it as "monkey patching"). In short we replace a singleton accessor method (such as standardUserDefaults) with one we control (actually another, not otherwise exposed, class method of TBCSingletonInjector). More specifically we swap the two implementations. This is so we can swap them back again when we're done. Then we call the code block - all within a @try-@finally - so no matter what happens we always restore everything to its previous state.

What does the method we swap in do? It returns a global variable.

Wait, what? I thought globals and singletons were basically the same thing? Aren't we out of the frying pan into the fire?

In the war against singletons we must fight them with singletons! Well it's not all bad. This global is only in our test code and we have full control over it. It gets set to our "injected" singleton instance (and set back to nil at the end). It's not perfect - we can only use this implementation to handle one singleton at a time. I've not yet needed to handle more than one but I daresay the implementation could be extended to handle it.

Keep it clean

Since we've hand rolled our own fake class here (FakeUserDefaults) we can tidy things up further if we encapsulate the use of the singleton injector within it. Just adding a method like this should do the trick:

-(void) use:(void (^)(void) ) code
{
    [TBCSingletonInjector injectSingleton: self
                                intoClass: [NSUserDefaults class]
                              forSelector: @selector(standardUserDefaults)
                                withBlock: code ];
}

Now the test code becomes:

    FakeUserDefaults* defs = 
        [[FakeUserDefaults alloc] initWithValue: @"Red" 
                                         forKey: @"defaultColour"];
    [defs use:^{
            REQUIRE_THAT( makeWidget(), StartsWith( @"Red" ) );
         }];

Or, if you prefer, even:

    [[[FakeUserDefaults alloc] initWithValue: @"Red" 
                                      forKey: @"defaultColour"]
    	use:^{
            REQUIRE_THAT( makeWidget(), StartsWith( @"Red" ) );
         }];

Not too bad, really. But, still, prefer to avoid the singletons in the first place if you have the option.

Monster4

Mocking a monster

Rather than hand rolling a Fake you might prefer to use a Mock object too. I've found OCMock does the job well enough. I'm sure other mocking frameworks would do so at least as well. I prefer to use mocks when I want to test the behaviour, though. In this context that might equate to testing that some code under test sets a value in a singleton (e.g. sets a key in NSUserDefaults). The Singleton Injector works just as well for that, of course.

So there we have it. When you really have to deal with the beast you now have some tools to do so. If you do it please consider only doing so until you are able to replace the singularity with something better behaved instead.

Friday
Jun282013

Catch 1.0

Catch logo

Since Catch first went public, two and a half years ago, at time of this writing, I've made a point of describing it as a "developer preview". Think of it as you might a Google beta and you won't go far wrong. I did this because I knew that there was a lot that needed doing - and in particular that some of the public interfaces would be subject to change. While I have tried to mitigate exposure to this as much as possible (as we'll see) I had wanted to reach a point that I could say things have stabilised and I'm happy to call it a true 1.0 release.

That time has come.

As of today the version of Catch available on the Master branch on GitHub is 1.0 and I would encourage you to update to it if you're already using an older version. But before you do so please read the rest of this post as there are a few breaking changes that may impact you

What's new?

Output

One of the biggest changes is in the console reporter's output. This has been rethought from the ground up (perhaps more accurately: it has now been thought through at all). For example a failure now looks like this:
ClassTests.cpp:28: FAILED:
  REQUIRE( s == "world" )
with expansion:
  "hello" == "world"

That indentation is applied after a wrap too, so long lines of output are much more clearly separated from the surrounding context. This use of indentation has been used throughout.

But there's a lot more to the new look. You'll just have to try it for yourself

Naming and tags

One of the features of Catch since day one has been the ability to name test cases (and sections) using free-form strings. Except that I then went and imposed a convention on those names so they should be hierarchical. You didn't have to follow the convention but if you did you got the ability to group related tests together in a similar manner to related files in a folder in a file system. When combined with wild-cards this gave a lot of power.

The trouble was test names needed to be short and simple otherwise they got very long. So I felt the need to have a second argument where you could supply a longer form description. Of course this was rarely used (even by me!) and so you'd see a lot of this:

TEST_CASE( "project/widgets/foo/bar/size", "" ) { /*..*/ }

The name doesn't really tell you what the test does and the description (which should have) is unused but must be supplied anyway so is just an ugly empty string.

This was not what I signed up for!

Now there is a better way.

It has all the advantages of the old system, but none of the disadvantages - and all without breaking backwards compatibility - so you won't have to go back and rewrite all your existing test cases. Phew!

Test cases can now be tagged. Tags are placed in the second argument (that description argument that nobody was using) and are each enclosed in square brackets. Anything outside of square brackets are still considered the description - although that use is now deprecated. Tags fulfil the same role (and more) as the old hierarchical names, so the name field is now freed up to be more descriptive. The previous example might now look like:

TEST_CASE( "the size changes when the bar grows", "[widgets][foo][bar]" ) 
{ /*..*/ }

But now you can run all tests with the [widgets] tag. Or with the [foo] tag. Or with the [bar] tag. Or all tests tagged [foo] and [bar] but not [widgets]. Tags are much more powerful.

Variadic macros

But if you don't need tags the second argument is now optional (assuming your compiler supports variadic macros - or, more specifically, Catch knows that it supports them). So TEST_CASEs can be written with one argument - or even none at all (an anonymous test case is given a generated name internally - useful if you're just exploring an idea).

Most, if not all, macros where it makes sense now take advantage of variadic macro support.

If you know that your compiler supports variadic macros, yet Catch is not letting you, please let me know and we'll see if we can add the support in.

On your best behaviour

In my first post on Catch, under "A SECTION on specifications", I talked a little about how, while Catch is not a BDD framework, it supports writing in a BDD style. Of note I said,
There is more planned in this area. For example I'm considering offering a GIVEN() macro for defining instances of test data, which can then be logged.
Well I've taken this further and you can now write tests using the following form:
SCENARIO( "name for scenario", "[optional tags]" ) {
    GIVEN( "some initial state" ) {
        // set up initial state

        WHEN( "an operation is performed" ) {
            // perform operation

            THEN( "we arrive at some expected state" ) {
                // assert expected state
            }
        }
    }
}

You can have as many peer WHEN and THEN and even GIVEN sections as you like. You can even nest them with AND_WHEN and AND_THEN. In fact all of these macros are (currently) just aliases for SECTION. SCENARIO is an alias for TEST_CASE.

Although I mentioned BDD you do not need to assert on behaviour here. I typically use the THEN block to assert purely on state. Nonetheless I often find the GIVEN-WHEN-THEN structure useful in organising my tests. They also read well in the output. Here's an example straight from the self test suite:

-------------------------------------------------------------------------------
Scenario: Vector resizing affects size and capacity
     Given: an empty vector
      When: we reserve more space
      Then: the capacity is increased but the size remains the same
-------------------------------------------------------------------------------
That alignment of the colons of Given, When and Then is very deliberate - and is treated specially in the reporter. If the description strings get very long they will wrap after the colons.

Meet Clara

Catch has always had rich command line support. The first implementation was very ad-hoc but as it evolved it become more like an embedded library in itself. For this release I have taken this to its logical conclusion and spun the - completely rewritten - command line parser out into its own library. At time of writing this is still part of the Catch code-base, and depends on a couple of other parts of Catch. The intention is to break those dependencies and extract the code into its own repository on GitHub. But what of the zero-dependency ethos of Catch? Don't worry - the new library will follow the same principle of being header-only and embeddable. So a copy will continue to be included in the Catch code-base and Catch will continue to be distributed as a single header file.

A new library needs a new name. Since it's a Command Line ARgument Assigner I felt Clara was a good name.

As a result of this change some of the specific options have changed (details in the "breaking changes" section). This is to accommodate a closer adherence to POSIX guidelines for command line options. All short-form option names are now single characters and those that take no arguments can be combined after a single -. e.g. to combine -s, -a and -b you can now use -sab.

Options with arguments always have arguments (and can only have one). This leads to a couple of interesting consequences: first the separator character between option and argument can be a space or either : or =. Secondly the non-option arguments (test specs) can appear before or after options.

So the following are all equivalent:
./CatchSelfTest "test name" -b -x 1
./CatchSelfTest "test name" -b -x:1
./CatchSelfTest -b -x 1 "test name"
./CatchSelfTest -x=1 "test name" -b

What's up, Doc?

The documentation for Catch, such as it was, had been provided in the wiki for the GitHub repos. There were a couple of drawbacks to this - most significantly it meant I couldn't have different documentation for different branches, or earlier versions. I also find it much easier to edit documents offline.

So I've now moved (and updated) all the existing documentation into markdown files in the repository itself. These are in the /docs folder, but the README.md file in the root links into them, acting as a convenient launch point.

Breaking changes

This section is only really of interest if you are an active user of an earlier version of Catch.

Under new command

As well as the improvements described there have had to be some changes to the command line options to accommodate them. The list of available options continues to be available by running with the -?, -h or --help switches. They are more fully described in the documentation, now in the repository (rather than the wiki). The in-depth descriptions have been removed from the code.

But here's a quick summary of the changes to be aware of

  1. Test case specs (names or wild carded patterns) and tags are now only specified as application arguments (previously they were introduced using the -t or -g options). In fact -t now means something different!
  2. Listing tests, tags or reporters now all have their own options. Previously you used -l for all of them, with an optional argument to disambiguate. -l no longer takes an argument and just means "list tests". Tags are listed with -t (which formerly meant "run with this/ these test case(s)". Listing reporters is less commonly used so has no short-form. They can be listed with --list-reporters
  3. -nt ("no throw") has become -e (because short form options are single character only)
  4. -a ? has been split into -a and -x ? (because options may have zero or on arguments - but not both)

Writing your own main()

Catch can provide its own main() function but if you write your own there were a few points you could hook into Catch, with different degrees of control over how it is configured.

This continues to be the case but the interface has completely changed. The new interface is more flexible, safer and better encapsulates things like the clean-up of statically allocated state (important if you do leak-detection).

The new interface is documented in the own-main.md file in the docs folder. It is based around a Session class - which must have exactly one instantiation in your code. However, within the instantiation you can invoke Catch test runs as many times as you like (the Session class encapsulates the config and is responsible for the clean-up of statics - in the future those statics may migrate to the session class itself).

Reporters

Catch has a modular reporting system and comes with three reporters bundled (console, xml and JUnit). You can also supply your own reporter by (previously) implementing the IReporter interface. This was one area that was often being slightly tweaked - and would frequently break implementations of the interface. More often than not any changes need not be used by client code - but they would have to update their interfaces anyway!

To make the reporter interface more robust to change I've created a whole new interface, (IStreamingReporter). Most of the methods of this new interface take structs instead of a list of arguments. Those structs can now change with little to no impact on client code (obviously depending on the changes). They are also richer and provide more information than before so I think we're set for a while now

To ease the transition for anyone who has already implemented IReporter I've provided the INTERNAL_CATCH_REGISTER_LEGACY_REPORTER macro (which wraps your reporter in the LegacyReporterAdapter adapter class).

At time of writing documentation for the new reporter interface is coming

It's not just me

Although I have used the personal pronoun, I, a lot in this post (and I continue to be the benevolent dictator on this project) Catch has greatly benefited from the on-going contributions of others - whether that be through pull-requests, bug reports, feature requests and other suggestions, actively maintained forks or just plain evangelising. All of this has been much appreciated and I hope to grow that even more now we have a stable base. Thanks!

Where to go from here

Catch is hosted on GitHub. The preferred url to follow is catch-lib.net, which redirects there - but may become a landing page in the future (an embryonic version of which is already at builds.catch-lib.net).

There's also a forum on Google Groups.