First a little about me. I truly believe that AI can help organisations produce better things faster when used well and since I don’t intend this to be a linkedin article I’m going to give you my view on what “Used Well” looks like.

It’s just too early for any opinions, especially my own, to be anything more than a well-educated guess. There are no right or wrong answers to the problems of AI adoption and anyone who tells you there are is almost certainly trying to sell something. This article is a collection of my own experiences, conversations I’ve had with others and things that I have read that feel like they’re improving our knowledge of solid AI practices.

Almost every organisation I talk with seems to tip-toe around the subject of AI adoption. I say tip-toe because in most cases no-one is able to articulate what they want from AI other than a fear of missing out or pressure from investors or CEOs.

If that’s touched a nerve then don’t fret. It’s early days and we’re all making this up as we go along. What doesn’t help is the constant claim, counter-claim and irresponsible hype from the big tech competitors as they vie with each other to grab AI market share by the throat.

Most companies are aware that some of their developers use AI tools a lot, some use them a bit whilst others are pushing back. No-one seems able to measure effective usage or the benefits that usage brings only the sure knowledge that these wonder tools come with ever-increasing costs.

So what does AI adoption even mean ?

#

AI tooling doesn’t just make development faster, it can also change the process by which better products can be made. Let’s take a quick look at each.

Speed of Development

#

This is so difficult to pin down. Vendors such as Github Co-Pilot often claim to have benchmarked 55% increases in development speed but these are usually under ’lab’ conditions. Working with AI tools on a new system that isn’t yet in production is very, very different to working on an existing one in production with real end-users.

At the other end of the scale a study on Open Source development demonstrated that AI slowed developers by about 19% when, interestingly, the developers subjectively felt they were 20% faster.

Generally independent studies seem to benchmark increases in productivity about 20-40% though it’s worth noting that AI tooling and models have evolved significantly since any of these studies were made.

As for my own, more current, experiences just pure code crunching with AI tools I’d summarise them as:

• Building a working prototype from scratch - weeks become days as long as I have domain knowledge or good access to someone that has.
• The bulk of my coding experience is Python and Ruby. If I’m writing backend code in either of these languages the 30-40% uplift in output feels about right.
• Much greater gains can be had on less familiar parts of the tech stack. I’m not that excited by front-end tools like React and find this kind of tasks to be a chore. Not only is AI coding way faster than me at writing the code I’m sure it does a much better job.
• I have significant DevOps experience, especially with Terraform, so I know what good looks like but I don’t use Terraform every day. Claude has built me several Terraform setups for things like static websites and Lambda-based applications in minutes that would typically take me a day by the time I make my mistakes and get it all to work.
• Similarly AI tools frequently save me small amounts of time here and there by asking them to commit working files, deploy something, merge a branch etc. All those chores that typically suck-up my time and sap my energy.
• Last and by no means least - debugging. All developers hit that wall where we just can’t get the f*cking thing to work. For me it’s usually something so trivial I just can’t see it any more or something poorly explained in the documentation. The number of times I’ve turned to Claude in total frustration to ask why something isn’t working only to get a simple and obvious breakdown of “why” - something I’d completely overlooked and/or quite obvious with hindsight. This alone is worth the token spend.

Based on this experience I believe that AI coding tools are money well spent. I was always proud of my code and felt it was clean, well structured and readable but in all honesty these days I’d rather focus on specifying the problem than crafting lines of code. I don’t miss coding at all.

What does ‘Used Well’ actually look like ?

#

So I’ve given plenty of anecdotal evidence of why I like coding tools but my feeling is that the real benefits for any organisation are when they both change & formalise the way we all work. Formalisation makes the process measurable and removes the constant question of who is using the tools, how they’re using them & why.

In my experience there are two opportunities to change the way an organisation works, not just within the engineering team but also the entire lifecycle.

Changing the way engineers work.

#

At developer level, the worst thing you can do is hand out AI tools and then measure success by how much each developer is spending on tokens. Spend tells you who’s using the tools, not who’s using them well. The two are easily confused and rarely the same thing. I’ve even heard developers talk about how they were gaming the system by creating large tasks for agents with little or no intention of using their output.

The challenge of agents is how to work with them effectively which to me means finding that sweet spot that balances cost and output. Getting this right feels more like real engineering than crafting code. AI isn’t magic if you only have a vague understanding of what you want to build a coding agent will amplify the old adage of garbage-in garbage-out to a terrifying degree whilst charging increasingly large amounts of money for the privilege.

Many moons ago I was a civil engineer mostly building underground railway systems in London. We didn’t pour concrete or erect steel; we planned, specified and designed for others to do the craftsmanship. That experience made it natural for me to work this way with agents, thinking of them as tradesmen that did the real heavy lifting before I approved their output.

Writing specifications is an art that doesn’t come naturally for most but I can honestly say that I have yet to write a specification where, at some point, I had to stop and think because the plan that was floating around in my head was incomplete.

At the moment I write Intent Driven Specifications which are slightly more structured than starting with a blank sheet and generally get me a much better result.

I write specifications, get the agent to generate a written plan from that specification - this is usually an iterative process. If the plan is missing something I’ll improve the spec and get the agent to update the plan. When the plan is ready the agent builds the code, I check the output and adjust if necessary.

This is real engineering that makes efficient use of token spend, keeps me close enough to the code and prevents drift. I know there are people out there who believe that all I need do is create a specification and let the agent go do its stuff but that feels like a completely unnecessary risk - risk that huge amounts of tokens will be spent on the wrong thing and risk that something nasty may be going on in all that unseen code. Before pilots take off they cross check their systems and controls. Of course they could skip that and save time but at some point that could go badly wrong so why take the risk ?

Best of all, spec driven development in whatever flavour is a formalised process that can be measured and the artefacts that developers produce - specifications and plans - are a much better metric than token spend.

Changing the Software Development Lifecycle

#

As I mentioned above specifying is hard and that’s no different for clients, customers and stakeholders. Pretty much everything we buy has been manufactured so we’ve lost the art of thinking about what we want and conveying that to others.

All too often typical Software Development Cycles result, eventually, in something that the stakeholders may have asked for but now they see it maybe they should have asked for something a little different. In some enterprise cultures this process can be so prolonged that requirements change, the stakeholders tell you it wasn’t that important or they simply forget why they wanted it in the first place. Either way it inevitably results in difficult conversations, varying degrees of compromise and something going live that could have been better.

Agents can create working prototypes fast giving whoever was asking for the system or feature a chance to feedback early on whether that was what they wanted. Twice now I’ve written a specification, had an agent build it, demoed the result, and watched the stakeholders realise that wasn’t what they meant—or that I’d misunderstood them in the first place. In the old world, that just left us all feeling deflated but now I’m familiar with my spec, I was now familiar enough with version 2 of whatever it was the stakeholders wanted so I could get the next iteration ready to demonstrate literally in the time it took me to drink a double espresso.

AI doesn’t just make us faster at building the thing—it makes us faster at finding out which thing to build, which is where most projects go wrong in the first place. Hard to benchmark but I’ll wager good money that outcomes are significantly better.

The Real Measure of AI Adoption

#

Perhaps that is the most useful way to think about AI adoption. It isn’t a race to produce more code, consume more tokens or claim the largest productivity increase. It is an opportunity to shorten the distance between an idea and the evidence that tells us whether the idea is any good.

Used well, AI makes misunderstandings cheaper. It allows engineers and stakeholders to test their assumptions while changing direction is still easy, rather than discovering months later that they have efficiently built the wrong thing.

So the question I would ask of any AI adoption programme is not, “How much faster are our developers coding?” It is, “How much faster are we learning what is worth building?”

That is harder to put on a dashboard than token spend. It is also far more likely to produce better software.

If you want the shorter, more practical version of this argument, we’ve distilled it into two recipes: measure business outcomes, not AI activity and why AI tools alone won’t guarantee ROI.