VoIP on the iPhone

It's now 3 weeks since we released the world's first VoIP application on the iPhone, there at day 1 of the App Store, and what a wild ride it has been. The application (free to download on iTunes) has brought in as many paying subscribers in 3 weeks as our previous Nokia smart-phone application has in the past two years! Anyone who has tried to develop software for the iPhone/iTouch during the Beta SDK period will empathise with just how hard this has been, and just how intransigent and unhelpful Apple have been to the developer community during the course of the Beta.

Back last late July I saw that my CTO had one of those new-fangled iPhoney-thingies, so I schmoosed on over to have a look. It was fresh back from the US (the iPhone wouldn't launch in the UK till Nov) and espoused the typical Apple design philosophy. I noticed that the Jobs' mouse manifesto of 'one button is all you will ever need' had manifested itself yet again with the conspicuous 'home key' design, nevertheless the hardware designers had still managed to create a stylish and modernist device even when hamstrung by the evangelical zeal of their CEO.

I managed to convince our gullible CTO that if I could take the sexy new toy away for the weekend, then I would produce a native 'Hello World' app to prove that there was at least a shadow of a possibility that we could launch a real native VoIP application on the device. Remember that at this time we were being told that that there would never be native software development allowed on the device and that web-apps were the future; 'embrace the future, citizens!' - ironic how Apple has gone from being the underdog in the seminal 1984 Superbowl advert to the Evil Empire it railed against.

Of course, little did our CTO realise that all I wanted to do was get my grubby little mitts on the virgin device and would only return it once it had been digitally abused in every possible way you could imagine. Needless to say, with the help of NerveGas and the iPhone dev team's hacked toolchain I jailbroke the device, cross-compiled a simple app and even minicom'd into the baseband in order to SIM unlock it - ah, those were the days. Now it's all UUIDs, provisioning profiles and official SDK releases, ho hum. So I managed to convince my CTO that we should launch a product on the iPhone, and that I was the right person to lead the development. IWIN button!

We did a proof-of-concept at DemoCon 2007, then another for VON. Then the official SDK was released and we went all corduroys and sensible shoes. Then the steady leaks of the 3G iPhone began to build up, and at WWDC all was revealed, and the world and his dog were marvelling at the 'revolutionary' iPhone 3G. But really, how revolutionary is it?

I was reminded of the Mayor of New York's profound prediction on being shown the first telephone by Alexander Graham Bell - 'What a marvellous device!' he exclaimed, 'I can foresee a time when every city will have one'. For me the really revolutionary thing about the iPhone 3G is not 'the internet in your pocket' (although this is persuasive) nor the 'trouser-based computing' argument of computing power at your fingertips whenever you need it. Nor is the convergence of photography, telephony, GPS and internet the most revolutionary aspect - anyone remember the Nokia N95?

No, the revolutionary aspect of the iPhone v2 (rather than 3G, as I'm referring here to the firmware release rather than the hardware) is the iTunes App Store. The App Store? Bear with me on this. In the past the process of buying software has been a fragmented and somewhat user-unfriendly distribution model. If I want to buy professional software I must buy it in a shop, or buy it on-line from an 'e-tailer' and wait for it to be shipped to me. If I'm lucky then I can download a trial version whilst I wait for the real software to be shipped to me, which may take several days to a week or more depending on package stocks and where it's being shipped from - a classic example being from Amazon (not even the marketplace) where although an item was marked as 'shortly available' I waited 3 months before being eventually told that my order had been cancelled for me as they could not source stock, and I had joy-of-joy been refunded. Thanks. Perhaps Amazon would like to refund me the interest, or even 3 months of my life. No, I didn't think so.

One of the really nice things about Open Source software is not so much that it is free (as in beer) but that being free I can download it, install it and run it NOW, not wait for my payment to be authorised and an activation key to be sent to me. Yes sir, it's in the post. Post?!?!?! Yes, you know who you are Orange. The success of direct2drive and other paid-for downloading services is a testament to the 'instant gratification' generation that our corporate masters and their media slaves has programmed.

People ask why I add cold water to my coffee - my answer: to cool it to a comfortably drinkable temperature, after all there's no point in having a cup of instant coffee and then having to wait for it to cool down to become drinkable now is there? That's hardly 'instant'.

iTunes is the instant coffee of the music world - you programme it with your details, and your preferred payment method, and then whenever you hear something that you like, you can buy it. Instantly! Just click!

Of course, having to synchronise your iPod to you desktop/laptop ruined the whole instant gratification thing, so now if you have an iPod Touch or an iPhone you can just buy tunes straight from your pocket, and hear them straight away. Marvellous. See - buy - enjoy, all within the course of a few heartbeats. Instant gratification - just add water. This has, of course, been a huge hit, and Apple have, and will continue to, milk it for all it's worth.

The (evil) genius[1] is in applying the same model to software. Buying software is now as simple as buying a hit single, once click and it's yours. One click and your money is theirs - 'they' being Apple and the evil alliance of music publishers and distributors. The master stroke here is that with the App Store Apple cuts our the publishers and the distributors, and takes a straight 30% cut of the whole pie, which is a win-win for both the software developers and Apple over the old model.

Let's say your friend has found this really neat app, which has a social element built into it e.g. Spore - you see it on their iPhone and go 'Wow! that's really neat - I'd like to play with you on my iPhone!' - you just go into the app store, click and buy and a short download later it's on your phone and you're happily evolving away with your friend. How powerful a marketing model is that? No 'oh, I'll have to remember to download that next time I'm back at my PC', and certainly no 'that's cool, I might even pop in and buy a copy next time I pass a Virgin Megastore'.

Apple now owns your pocket - the money in it anyway - and that's the real meaning of 'trouser-based computing'.

[1] How Apple Got Everything Right By Doing Everything Wrong - Wired, April 2008

Demon ex machina

Why are speed cameras like software metrics?

Here I should state that I'm referring to a huge subset of software metrics that rely, to some extent or another on LOC (Lines of Code). Speed cameras are (or so we are told in the UK) a 'Road Safety initiative' that is designed to reduce the number of fatalities and serious injuries in road traffic accidents. A software metric is measurement of a software system's properties that is used (or so we are told) to increase quality and decrease cost and risk from software development projects.

So what do 'Scam-eras' and software metrics have in common? Easy: both use measures that are easy to determine, but that extensive research tells us are *not* highly correlated to the programmes stated objectives.

Context: At Google's Open Source Code Jam last month I had the pleasure to present; the theme of the event was 'Productivity'.

There were many interesting and thoughtful presentations on technical matters e.g. Continuous Integration and Domain Specific Languages cropped up several times. Predictably there was a lot of talk about the 'what' and 'how', but little of the higher knowledge levels of 'when' and even 'why'. This is not unexpected in a profession that always hits the top 5 in evaluations of levels of workplace stress.

I thought it would be interesting to look at what exactly we mean when we use the term 'productivity' in the context of knowledge workers. Productivity is well defined for other industries e.g. manufacturing - 'units of output per unit of input', or 'widgets per labour hour'. The problem comes when we try and apply the same reasoning to knowledge workers, what exactly constitutes 'output'?

Much research has been done around the LOC measure and how it relates to system complexity, cost and reliability. Unsurprisingly, this research tends to be littered with caveats and fudge factors, if you want to see for yourself take a look at COCOMO, or SMERFS. My own investigation into the research on this topic whilst I was at Symbian (working to improve their code reliability metrics system) is that the only thing that the research *does* prove is the positive correlation between LOC added/changed and the defect injection rate, which is hardly an earth-shaking revelation.

The chief complaint when using LOC as a measure of output is that the measure is one of volume of output, and is not representative either of a) effort or b) of value. Let me explain: Let's say Gary Guru has a complex problem to solve, and spends hours thinking of the solution before writing a small, elegant module with 1000 LOC. Now compare this with Archie Average, who only superficially analyses the problem and spends all his time coding a suite of inter-linked modules that solve the problem in 10,000 LOC.

Both have accumulated the same T&M cost (assuming the same labour rates). If we were to use LOC as a measure then Archie would seem to be an order of magnitude more productive than Gary. However, as we know LOC is positively correlated with defect injection rate, we know that we will spend at least ten times as much fixing Archie's code, and the TCO will be much higher after release as it will take considerably longer to diagnose and fix problems in Archie's rambling code-base than Gary's.

It gets worse as soon as the knowledge leaks out that developers know that they are being evaluated on the measure, as now they will attempt to game the measure. If a goal of the organisation is to encourage reuse (and what organisation wouldn't want to encourage what has become the 'holy grail' of software development?) then the implementation of LOC as a productivity measure positively discourages this, as every time I reuse someone else's code I appear less productive!

My own experience over my two decades of software development experience mirrors that of Douglas Hoffman who wrote the excellent paper: 'The Darker Side of Metrics' (Hoffman, 2000) which can be found here:
http://www.softwarequalitymethods.com/Papers/DarkMets%20Paper.pdf

He eloquently states that whenever you introduce software metrics to an organisation, there is a hidden 'dark side' effect based on how human beings will game with the metric to their own perceived advantage. Hoffman notes that if we predict that testing will find 100 bugs in the next testing period, this will become a self-fulfilling prophecy. If there are less than 100 bugs found then it will appear that the testing team are not working hard enough, and if much more than 100 bugs are found then it will appear that the development team are producing shoddy work, and as the testing and development teams work closely together in most organisations this would be bad for the interpersonal relationships of the teams. Consciously or sub-consciously the 'bugs found' metric will converge on the prediction, bugs will be split or rolled-up arbitrarily, reclassified as 'working as designed' or 'not to be fixed in this release' or any one of a number of other strategies to allow the prediction to be correct and achieve the 'win-win' situation.

Hoffman notes that the research seems to show that the only thing that the 'defects found per unit time' metric actually positively correlates to is 'testing effort'...

If you want to see a wonderful discourse on LOC as a measure and basis for productivity metrics in software development then you have only to look here:
http://forums.construx.com/forums/p/186/498.aspx#498

This was a piece that I found in my research for improvement of the 'codechurn' system at Symbian, and was highly topical as I had just come from GE, an early adopter and keen proponent of Six Sigma. The more experienced and erudite of the contributors voiced an opinion that mirrored my experience at GE, that whilst Six Sigma can and does do wonders for manufacturing companies, it's employment in software development organisations is problematical and anything but straight-forward.

With this in mind, let's return to the theme: 'measuring the output of knowledge workers'. We could posit that some sort of 'function point' or 'Use Case' based scheme of measurement would surely be better, the difficulty with this approach is that we need to normalise for complexity and granularity, as one 'function' is not necessarily as complex (and thus easy to implement) as another.

I remember asking my tutor during my post-graduate year doing comp-sci during a tutorial 'how do you measure system (rather than algorithmic) complexity?' and his answer: 'in inches'. Seeing my blank expression he explained that you took a ruler and measured the length of shelf space that the printed documentation took up.

It turns out that we cannot even measure (the much simpler) Algorithmic Complexity accurately, for anything above a trivial limit to system size, as the following paper 'Large Limits to Software Estimation' (J.P. Lewis, 2001) shows:
http://www.idiom.com/~zilla/Work/kcsest.pdf

Lewis uses an analogue to Godel's incompleteness theorem (Chaitin's incompleteness theorem) for Algorithmic Complexity to prove that it is impossible to objectively measure system complexity. This, at one stroke, makes a mockery of all those management methodology and software methodology zealots whose rationale for why their dogma does not produce verifiable, repeatable results is that 'the process was not followed diligently enough' or 'the measures were not made accurately enough' - the snake oil that they are pushing is simply *incapable* of delivering what it promises.

So the next time someone tries to foist their plan-driven dogma on you, you have some ammunition to counter them with! Hopefully we can move the industry away from the 'target-driven' mentality one project at a time.

Software storytelling

Many cultures had, at their pre-industrial stages, a rich oral tradition of storytelling as a means of communicating knowledge, before literacy became widespread. Bards would gather people around the fire and tell rousing tales, inspiring and edifying a new generation of citizens with their practised craft of passing along knowledge, morals and cultural values using the medium of entertainment. It seems that now, certain sub-cultures have developed 'secondary orality' as Walter Ong [1] calls it, the creation of an oral transmission that exists alongside and is enabled by modern media, radio, television and the Internet. I propose that professional software developers can use the ancient art of storytelling to enhance their daily working practices.

I remember reading in Alistair Cockburn's excellent book 'Agile Software Development: Software through People' that Xerox discovered that their copier repair engineers didn't actually get most of their working knowledge from training or reading technical manuals, instead their operational knowledge was mostly gleaned from sitting around drinking coffee with senior copier repair engineers who would tell them tales of how they diagnosed or repaired difficult or obscure faults with particular models, whilst waiting for the next call-out.

When reading the original Gang of Four book on Patterns [2] I realised that a chord had been struck with my early teaching experience (I taught Computing at a local college of technology after finishing post-grad studies), namely that people learn better by example. I have not however noticed a particular culture of software professionals 'telling stories' or 'describing patterns' in casual conversation around the water cooler in many of the places that I've worked, as usually people are so pressured that they talk only about the current task at hand. Discussion of 'patterns' tends to be limited to a sub-set of concrete formalisations rather than a contributory or exploratory activity.

In the UK, particularly in London where I work there *is* however a culture of going out for a drink down the local pub with your work colleagues, and there we do start to talk about previous experiences and start to share knowledge by communal storytelling - but surely this kind of information exchange should be encouraged at work rather than being left to the ad-hoc and time-constrained nature of an occasional after-work outing?

As an example of how this might work, whilst I worked for 3 we had a weekly 'architecture group meeting' (there were >40 architects at 3 during the pre-launch build phase), organised during the lunch break, where we would bring our sandwiches and listen to a presentation by a different group member each week and ask questions, it was a good forum for knowledge exchange and one in which we could explore architecture patterns and discuss better ways of working.

In further posts I'd like to elaborate on how the craft of 'storytelling' fits in with the tool-kit of the software professional, and how we can leverage this skill to help us both become better individually and to increase the organisational capability of the team you work with.

[1] Oral_tradition: Walter_Ong
[2] Gang of Four: Design_Patterns