Published on
Early in my work with AI, I was researching security tools for software developers.
I was trying to find a very specific type of scanner. The developers were working in Go, and I wanted a tool that could integrate into their pipeline and help them identify security issues.
AI suggested one that sounded perfect.
I asked follow-up questions. It described how the tool worked. It showed me the type of output it could create. It explained how developers could use it.
I was already thinking about the lesson I was going to build.
Then I went to find the tool.
It was not there.
I searched again. Nothing.
Eventually, I went back to the AI and asked where the product was. The answer was essentially: that tool does not exist, but that is what the perfect tool for your request would look like.
I had spent hours talking about a fictional product.
It was funny after the fact. It was also a useful lesson.
AI can produce answers that sound complete, confident and convincing without those answers being true. Using AI well requires more than knowing how to get a response. Employees need enough knowledge and judgment to recognize what kind of answer they received, when to question it and when the consequences require verification against a trusted source.
AI fluency can be mistaken for accuracy
Large language models are very convincing because they are fluent.
The response sounds complete. It sounds professional. It continues naturally from whatever you said before.
People mistake that fluency for reality.
In my case, I kept responding positively because the answer was exactly what I wanted. The system continued to give me more of it.
From the system’s perspective, it was fulfilling the interaction. It was producing an answer aligned with my request and my feedback.
It was not verifying that the tool existed.
This is why I describe AI as an intelligence tool rather than a truth tool.
It can help us infer, explore, compare and create. It can bring together large amounts of information and surface signals that would be difficult to find manually.
That does not make every answer factual.
Expertise still matters when using AI
A lot of people talk about AI as though it reduces the need for expertise.
I think the opposite can be true.
When I know a subject well, I can use AI to extend what I already know. I can recognize when an answer is weak, challenge the assumptions and direct the system toward something more useful.
If I am weak on the topic, AI can multiply that weakness just as quickly.
If I do not know what right looks like, I can let AI do wrong all day.
The output may still look polished. That is what makes the problem difficult.
Employees need enough knowledge of their work to recognize when something does not make sense.
They also need to understand when a result must be checked against a trusted source rather than accepted as a convenient answer.
Use AI to get to the first useful step
None of this means AI is unreliable to the point of being useless.
I have used it to pull together information across customer calls, emails and other records while planning a conference. I wanted to identify people who might be a good fit and understand how we could approach them.
Doing all of that manually would have taken weeks. Realistically, I probably would not have done the full analysis at all.
AI made the first step dramatically easier.
Were all of the recommendations perfect? No.
Some were not aligned. Some required additional context.
But I did not need the system to make the final decision for me. I needed it to bring forward useful signals so that I could apply my judgment.
That is where the relationship works well.
The AI reduces the friction involved in reaching a useful starting point. The person decides what is real, relevant and actionable.
Not every AI output needs the same level of scrutiny
A brainstorming idea and a financial conclusion do not carry the same consequences.
Neither do an internal rough draft and a commitment made to a customer.
Employees should learn to adjust the level of verification based on what the output will be used for.
If I am asking AI to give me five directions for a campaign, I may be comfortable treating the answers as possibilities.
If I am asking it to produce figures that will drive a financial decision, I need reliable source information and accountable human review.
The same applies to decisions involving people.
AI may help an organization examine aggregated workforce information and identify a signal that a department is struggling. That can be useful. A leader can investigate the signal and decide whether coaching or another intervention is appropriate.
That is very different from allowing AI to decide how an individual employee is performing.
The system does not have the complete context of that person, their team or the conditions surrounding their work.
Fluent output can be helpful. It can also make us overconfident.
The skill is not simply learning how to get an answer.
It is learning what kind of answer you received.
Recent posts by Michael Burch
AI Fluency Is Not Truth
Early in my work with AI, I was researching security tools for software developers. ...
AI Does Not Stand for Automation Intelligence
AI does not stand for automation intelligence. I say that partly as a joke, but it ...
Access Is Not Capability: What Corporate AI Training Must Actually Accomplish
Artificial intelligence has not made my job easier. I know that is not the answer ...
