Today, we need to be as deliberate about developing expertise as we are about automating execution. AI can support that development, but leaders need to design work so employees still practice making decisions, identifying mistakes, and understanding consequences.
The experience gap created by AI
When we give an assignment to AI that was once performed by a junior associate or intern, we can see the time saved. What’s harder to see is the experience someone might have gained by doing that work. Those routine assignments are often where people learn the process, come across something unexpected, and ask a more experienced colleague for help or research an answer from a different source. If AI takes over that work, where does the next employee get that practice?
AI is not going away, and I’m very comfortable saying that the capabilities will continue to get better. So, how do we avoid rolling out a bunch of zombies who only rely on AI for answers with no real-world experience to validate whether the AI is correct? I believe there are approaches to using the technology responsibly to get trustworthy outcomes. Training for the tools is part of that, but so is giving people opportunities to learn the work itself.
In today’s industry, we have a host of folks who are experts at what they do. There is much nuance in the typical day to day office workload, and the folks who do the work today know the ins and outs of their jobs. I won’t go as far as to say that they are experts in the top-down function of their role, but they know what is required of them, and they perform their duties. Think about how a payroll clerk learns the job.
A payroll clerk has a process they were taught, and assuming they do their job correctly, they know that process very well and could be considered an expert in that role to the point where challenges arise, they know how to handle them. That same person, however, may not have the broader knowledge of Generally Accepted Accounting Principles (GAAP) expected of a Controller or CFO, even though they are performing an accounting function. That doesn’t make them any less of an expert in the payroll work they do.
Assuming they are very good at their job, and want to ascend the corporate ladder, they will move to a newer role eventually and take on more responsibility and become an expert in another role, until they hit a ceiling of capability or the top of the ladder. Each role along the way, they are gaining expertise and if they stay in the same company, can help the folks who are backfilling their role when they run into challenges. Certain aspects of the function will be replaced by AI Agents in the coming years, but for the time being, and the foreseeable future, there will still be a need for human oversight. Especially in a function so critical of ensuring employees get paid. This process can have complications and should not allow agents to make decisions on their own.
Humans learn by making mistakes and learning from those mistakes. The ramifications of those mistakes can vary from something innocuous that’s barely noticed, to something that causes a person to lose their job. Some mistakes have consequences that can’t be undone, but even those experiences can teach the employee something they carry into their next role. Let’s look at why experience through learning makes such a big difference.
Why experience and judgment matter
So, in a world where we know the technology isn’t perfect, this month I want to look down the road at what the future of skill development and gaining expertise could look like in this new AI era. I recall being an intern in my first foray in the corporate world working on the IT help desk. The job was relatively straight-forward: go install this RAM in a computer, go run this script to update the startup process, and really just about anything an IT intern would be doing back before managed services existed. Today, I could look up how to upgrade the RAM on said computer with YouTube, and I could use ChatGPT to get a startup script. What I still needed was experience putting that advice into practice. I couldn’t just barge into someone’s office, make them stop everything they were doing, commandeer their computer, complete the process, and hand it back to them. AI could suggest how to approach that person: “Hey, could I do a quick upgrade when you head out for lunch, or could you be sure to close everything tonight before you leave?” But I still had to learn how to work with people, so I didn’t interrupt their work on multiple levels. How to speak with the person, communicate what I needed to get done, and understand the business impact of them being without their computer.
Today, I could get a strategy from ChatGPT, but as soon as my script for walking into that office went awry, I’d be sunk unless I asked the tool in real time how to deal with the situation. Experience matters, and this is just a simple example. There are countless others I could come up with reflecting on my years of experience. At the same time, if I met resistance from that user, I could go back to my boss and tell him about my experience to get advice. “Oh, yeah, Tom can be like that when he’s in a bad mood, just try to catch him tomorrow morning early, in passing to see if there might be a better time.” Unless ChatGPT knows about Tom and his brusque way of working with people (it doesn’t), without that guidance from my boss, I could continue to hit that resistance. Experience matters.
Looking back, that assignment had two outcomes. The computer got its upgrade, and I learned something about getting work done with other people. My boss’s advice gave me something to try, but I still had to approach Tom, see how he responded, and adjust. If the upgrade had been completed in a vacuum where I didn’t have to deal with Tom at all, the business would have gotten the same result, but I would have missed that opportunity to learn.
Tools alone don’t build expertise
Back in October we discussed how AI tools are making the jobs of junior individuals easier by having more matter-of-fact information at our fingertips. Gone are the days when you’d go to your favorite search engine to ask a question and having to parse page after page of information that was somehow related to your query but requiring a collection of thoughts based on the results you found. Now, you ask a question, and you get an answer. Seems simple enough, but how carefully do we verify the results? Too often, I’ve seen the response come back where the engine has returned an answer in AI mode, even citing its references, yet when you check the references, the response is incorrect. If not using the “trust but verify” approach, and action is taken based on the bad information, what then? Of course, it depends on the situation, but best case is that you’re just spouting some wrong information in a group of friends, and worst case is that the information is used in a life and death situation. Currently one of my favorite memes:
Granted, the tools are becoming better, so the chances of these types of events are decreasing, but is it worth the risk to not double-check? It is very tempting for a junior associate to just take what they’ve received as gospel and move on to the next task. So my question this month is: how do we avoid those situations and ensure that the “trust but verify” method is being followed? I think it starts with the tools we are allowing users to use and how their information is validated.
So now, what do we do when we have a conflict in what the AI answers? I’m guessing that many people don’t use Claude for something, then turn around and ask ChatGPT for the same thing to be able to compare the answers. More than likely, there would be some overlap in the response, but also some differentiation. Which one is correct? Maybe both, but each is giving part of an answer? Or perhaps you get a response with conflicting views or answers if looking at something that has had research done on it.
It’s easy to tell someone to “trust but verify,” but what are they verifying against if they haven’t learned the work themselves? Training someone to use an AI tool doesn’t necessarily teach them how to recognize a bad payroll calculation or notice that an important exception has been missed. They need to understand the work well enough to question the answer, and they need opportunities to develop that understanding. So, how do we make using AI part of gaining experience?
Redesigning work as an apprenticeship
So, what does this mean for the future of this type of work? There needs to be a controlled way in which the tools are used and the information received can be verified. The hard thing is that it is too easy to just take the answer as correct and move on. It is quick, and it helps us tick a box on the to do list for the day. Organizations need to minimize the opportunity for just taking the default and moving on. Instead, the person with less experience should be asked to explain whether they agree with the answer and why. This would create a learning loop for the employee, helping them learn how to handle anomalies that arise.
Going back to the payroll example, we don’t need to let someone make an incorrect payment to give them an opportunity to learn. We could use a past payroll situation involving holiday pay and overtime, give the employee an AI-generated calculation, and ask them to check it against the applicable rules. Before seeing the approved result, they should explain whether they agree with the calculation and why. An experienced colleague could then review their reasoning, point out anything they missed, and explain why it matters. If the employee gets it wrong, there is room to work through the mistake without affecting anyone’s paycheck. The next time a similar situation comes up, have them try it again with less guidance. That gives us a way to use AI while still giving people practice making decisions and learning from the results.
Developing expertise becomes a leadership responsibility
How do we know if any of this is working? Getting more work done is useful, but that alone doesn’t tell us whether someone is gaining expertise. In our normal work reviews, we should be asking employees to explain their recommendations, identify assumptions they questioned, and discuss situations where they needed help. Going back to payroll, can the employee now work through an exception that previously required guidance? Can they apply what they learned to a different situation? Those are things a manager can look for over time, with responsibility for providing feedback and making sure the employee gets opportunities to keep practicing.
Start with one assignment your junior employees perform today. What does it accomplish for the business, and what does someone learn by doing it? Then decide which parts AI can handle, which decisions the employee should still work through, and who will review the work with them. We don’t need to redesign every job at once, but we do need to be deliberate about where that experience will come from while still taking advantage of the tools our organizations are investing in.
If we expect people to exercise judgment, where are we giving them the opportunity to develop it?
Sources and Further Reading
- Brynjolfsson, E., Li, D., & Raymond, L. (2025). Generative AI at Work. The Quarterly Journal of Economics.
- Bainbridge, L. (1983). Ironies of Automation. Automatica.
- Lee, H.-P., et al. (2025). The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers. Proceedings of CHI 2025.
- Collins, A., Brown, J. S., & Holum, A. (1991). Cognitive Apprenticeship: Making Thinking Visible. American Educator.
- Bastani, H., et al. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences.
