AI Can Give You an Answer. But Can You Trust It?
Governance, responsibility, and the human judgement behind good business decisions
AI is changing the way accountants, advisors, and business owners work. We can now analyse financial information, research tax issues, prepare forecasts, and produce reports in a fraction of the time previously required.
And that’s exciting.
But today, we need to focus on a different question: Just because AI can give us an answer, does that mean we should trust it?
And perhaps more importantly: If that answer is wrong, who’s responsible? The accountant? The CFO? The business owner? The software provider?
AI might change how the work gets done, but it doesn’t remove our responsibility for the decisions we make or the advice we provide.
Two Opportunities: Efficiency and Value Creation
There are two quite different ways we should be thinking about AI in business.
The first is using AI to do what we already do, but better. Processing transactions faster. Automating repetitive tasks. Preparing reports and researching information in a fraction of the time. That’s about efficiency, productivity, and reducing costs.
The second opportunity is potentially much more exciting. It’s using AI to do things that previously weren’t practical, affordable, or even possible.
For an accountant, that might mean moving beyond preparing financial statements to helping a small business owner understand their cash flow, explore different growth scenarios, or make better business decisions.
One is about doing the work better; the other is about delivering more value from the work. Both are tremendous opportunities, but they raise a vital question: As AI becomes more involved in both the work we do and the advice we provide, how do we know we can trust the results?
A Confident Answer Isn’t Necessarily a Correct Answer
One of the most interesting things about AI is how convincing its answers can be. It can produce a beautifully structured report, complete with financial analysis, recommendations, and references.
But that doesn’t necessarily mean the information is correct. AI can misinterpret information, make assumptions without telling us, rely on outdated sources, or reference legislation incorrectly. And sometimes the calculations look perfectly reasonable, even though the underlying assumptions are wrong.
The danger isn’t always an obviously wrong answer. It’s a convincing answer that nobody questions.
Consider an accountant using AI to research a tax issue. The AI produces an answer referencing an actual ATO ruling. The ruling exists. The explanation sounds reasonable. But what if that ruling doesn’t actually apply to the client’s circumstances? Unless the accountant understands the underlying issue, how will they know?
The more professional an answer looks, the easier it can be to forget to check it.
When AI Gets the Financial Model Wrong
Financial modelling is a great example of AI creating new value. A business owner or founder can now use AI to build a financial model that might previously have taken considerable time and money to develop. That’s a fantastic opportunity, but there’s a difference between being able to build a financial model and knowing whether the model is actually right.
Example 1: The Model Didn’t Follow Its Own Assumptions
A startup founder had built their financial model entirely using Claude AI. The model looked impressive, featuring financial projections, calculations, and a detailed set of assumptions.
But upon review, within 20 minutes it became clear that the financial projections weren’t actually being calculated using the assumptions documented in the model. The model presented one set of assumptions but produced projections completely inconsistent with them. For a startup using that model to make business decisions or raise investment, that is a serious problem.
Example 2: Fixing One Error, Creating Another
In another startup advisory engagement reviewing a Claude-built model, several mistakes were identified and sent back to Claude for correction.
It made the corrections, but in doing so, it also changed one of the key assumptions underpinning the financial model, without being asked! The founder didn’t notice the change. It was only caught because an Excel version was downloaded to compare underlying assumptions side-by-side.
The Lesson
Neither example means AI shouldn’t be used for financial modelling. In fact, it makes modelling faster and more accessible. But it highlights a crucial truth: A professional-looking financial model isn’t necessarily a reliable financial model. If you don’t understand the model and its assumptions, how will you know when AI gets it wrong?
Three Questions Every Business Should Ask
When people hear “AI governance,” they imagine complicated policies and committees. For most small businesses and accounting firms, good governance starts with three simple questions:
- Where did the information come from? Is it accurate, complete, and current? And should confidential client or business information have been uploaded into the AI system in the first place?
- Who has checked the answer? Has someone with appropriate knowledge reviewed the output? Have we checked the assumptions, calculations, and conclusions, or are we accepting the answer because it looks convincing?
- Who owns the decision? Who has the authority to act on the recommendation? Who is accountable if something goes wrong, and can we explain how the decision was reached?
Governance doesn’t need to be complicated, but responsibility must be clear.
Not All AI Decisions Carry the Same Risk
Not every use of AI carries the same risk. Think of it as a traffic light system:
- GREEN – Lower Risk: Brainstorming, drafting emails, and preparing general content. AI can do much of the heavy lifting with routine human review.
- AMBER – Moderate Risk: Financial analysis, cash flow forecasts, and management reports. These require careful checking of underlying information, calculations, and assumptions.
- RED – Higher Risk: Tax and legal advice, lending decisions, payroll compliance, and decisions affecting employees. These require appropriately qualified people, independent verification, and clear approval processes.
The principle is straightforward: The greater the consequences of getting something wrong, the greater the level of human oversight required.
Trust Isn’t a Technology Feature
For accountants and advisors, trust has traditionally been built through qualifications, experience, judgement, and relationships. AI doesn’t replace those things; it changes how we apply them.
The accounting profession faces a choice: We can use AI to do what we’ve always done faster and cheaper, or we can use that saved time to deliver higher value. Instead of just preparing financial statements, we can help business owners understand why cash flow is deteriorating or whether they can afford to grow.
However, AI can make someone without deep expertise appear very knowledgeable. Will AI improve the quality of advice, or simply make it easier for more people to produce advice that looks professional? The answer depends entirely on how we use it.
Clients don’t pay us simply to produce numbers; they rely on us to help them understand what those numbers mean.
Use AI to Improve Your Judgement. Not Replace It.
Before introducing another AI tool, every business should be able to answer five core questions:
- What are we allowing AI to do?
- What information are we comfortable sharing?
- What outputs must be checked?
- Who approves important decisions?
- What happens when AI gets something wrong?
The biggest risk may not be that AI produces the wrong answer, it may be that we stop asking the right questions.
AI may change how the work gets done, but trust still comes from the people willing to stand behind the work.
Wayne Wanders is an experienced Business Advisor and Outsourced CFO who can help to scale and grow your business profitably.
Contact Wayne on wayne@arealcfo.com.au or 0412 227 052.
Click on the below buttons to access free Resources developed by Wayne Wanders, A Real CFO to help your business scale and grow profitably
And Wayne is always posting about new grants, funding options and other resources on LinkedIn that can help your business scale and grow profitably. Click on the below links and connect with Wayne or follow A Real CFO on LinkedIn.
Want a confidential discussion on your business situation, help with your grant application or to learn more about my Outsourced CFO Services, simply email me at wayne@aRealCFO.com.au or call me on 0412 227 052
