AI gave me an answer. I still had to decide whether to trust it.

AI made some parts of a recent piece of analysis much faster than they would normally have been.

Charts that used to take an afternoon in Excel took minutes. Calculations that would previously have required me to build and check a spreadsheet were completed in a fraction of the time. Comparing different cuts of the data was quicker, and producing an initial draft was faster too.

But the overall process was not simply faster. It was different.

The answer arrived before the understanding

When I build analysis myself, I follow the reasoning as it develops. I know why I chose a particular dataset, which alternatives I considered and how the conclusion took shape. A lot of the checking happens naturally while doing the work because I have been involved in each decision along the way.

With AI, parts of that process happened much more quickly, but I sometimes had to reconstruct the reasoning afterwards. A single response could introduce several calculations, assumptions and possible explanations at once. Over a long conversation, it became surprisingly easy to lose track of where a particular conclusion had come from or why the analysis had moved in a certain direction.

Some of the time saved at the production stage therefore came back later in the review.

The challenge was no longer simply producing the analysis. It was understanding exactly what had been produced, checking what sat behind it and deciding what I was prepared to stand behind.

Checking the numbers, and the words

I still checked every number back to source. That meant going to the original release rather than relying on a figure quoted from it. I checked the series title, what it included and excluded, and the period it actually covered.

AI was very useful once I knew I had the right data. It could compare series, test different cuts and help me work through what the numbers were showing. But that is different from establishing that the underlying data is appropriate for the question. I still had to do that myself.

The writing required the same level of attention.

I found claims that were stated more firmly than the evidence supported. Caveats sometimes disappeared between drafts. Some sentences sounded convincing but went slightly further than the numbers justified.

A drafting assistant, not a junior analyst

This changed the way I started using AI.

At first, my instinct was to treat it like a junior analyst. I would give it a task, let it produce something and then review what came back. For work that requires judgement, I found that this was not always the most effective approach.

Professional analysis rarely depends only on the document or dataset in front of you. It can depend on earlier work, previous conversations, wider context and knowledge of why something was done in a particular way.

That meant I still did most of the substantive thinking.

Where AI worked best was as a drafting assistant and a second pair of eyes. It could produce a first attempt at a paragraph much faster than I could. It could restructure something I had written, compare different ways of explaining an argument and point out inconsistencies.

But I am less convinced that it saved the same amount of time across repeated rounds of drafting.

There is value in having to write a sentence yourself. It forces you to decide exactly what you mean, how strongly the evidence supports what you are saying and what caveats need to sit alongside it.

When I handed over too much of that process, I sometimes spent the time saved on the first draft putting the nuance back into later versions.

The most useful lesson was not that AI could replace part of the analytical process. It was that I needed to be clearer about which parts of the process I wanted it to help with.

The question, not just the answer

One example made this particularly clear.

The analysis began with the obvious case. It was a perfectly reasonable place to start. When I ran the same measure across other categories, however, the finding changed. The case we had started with turned out to be the weakest example rather than a representative one.

AI had not made a mistake. It had answered the question we were working on, and I would have run the wider check whether AI was involved or not.

The important point was that it remained my responsibility to decide whether we were asking the right question.

This is where previous experience of checking analytical work became important.

Good checking is not only about confirming whether a number is correct. It is also about knowing where to probe more deeply.

After reviewing many pieces of analysis, you start to recognise familiar weaknesses. A comparison may not fully support the conclusion. An assumption may not have been tested. The framing may leave out an important alternative. Something may simply not fit with what you have seen elsewhere.

You still check the work thoroughly. Experience does not tell you which facts you can safely ignore. What it gives you is a better sense of which questions are worth asking and where an analytical weakness might sit.

Two reviews are not two points of view

In my experience, this judgement forms part of a wider professional checking process. Good analysis is exposed to challenge from different perspectives. Assumptions are questioned and evidence is checked back to source.

AI can reproduce parts of that process.

A second AI reviewer can identify unsupported claims, catch mistakes and challenge an argument. This can genuinely improve the work.

But two AI reviews are not necessarily the same as two independent points of view. If both start with the same information and the same framing, some assumptions may still go unchallenged. Somebody also has to decide what the second reviewer is being asked to check.

Human challenge is not perfect either. Poor handovers, outdated systems and lost knowledge can allow the same mistakes to appear again. Better tools, including AI, can help make existing information easier to retrieve and compare.

This became more obvious as my working conversations with AI became longer. Figures, assumptions, possible explanations and changes in direction accumulated across many exchanges. The information remained available, but reconstructing the reasoning became harder.

There is also a difference between retaining information and building judgement.

A record can tell you that something went wrong before. Experience helps you recognise the circumstances in which a similar problem might appear again.

That judgement develops over time through exposure to different pieces of work, having assumptions challenged and seeing where analysis succeeds and where it fails.

Does it come back to you?

The same principle applies to trust.

For me, the test for AI-assisted work is not how much of the final product a person wrote themselves. It is whether somebody remains responsible for the result.

That matters because responsibility affects the way people check. When your name is attached to the decision that the work is good enough to use, you have a reason to understand what sits behind the answer rather than simply accepting what has been produced.

The people who check least may appear to gain the most

There is also an uncomfortable side to the time saving.

Someone without enough experience to know which assumptions, sources or conclusions need challenging can produce something very quickly and feel that AI has saved them a significant amount of time. They may also be less likely to recognise what they have missed.

Someone who spends hours going back to source, checking definitions, reconstructing the reasoning and challenging the framing may report a smaller productivity gain. That does not necessarily mean AI was less useful to them. Part of the difference may simply be the amount of work carried out after the answer was produced.

This does not reduce the value of the tool. It does mean that the size of the time saving tells us very little about the reliability of the final work.

What I would take from it

I would not go back to working without AI. It made calculation, comparison, chart production and first-pass drafting considerably faster.

However, my experience was not simply that the same work took less time. The nature of the process changed. Some stages became much faster, while the effort required elsewhere increased. Verification changed as well and, in some cases, became more demanding because I had to reconstruct reasoning that I would normally have developed myself while carrying out the analysis.

Used as a junior analyst, AI sometimes created additional work, because I still had to supply the context it did not have. Used as a drafting assistant, a way to test my thinking and a second pair of eyes, it was genuinely useful.

That is the main lesson I would take from the experience.

AI can give me an answer much faster.

The piece this refers to is Investing £20,000 in 2015: where would you be now?, an eleven-year comparison of three investment routes. It carries a corrections appendix listing what changed during the analysis and why.

← All analysis