Rolling out an AI tool now takes days. Proving that it has changed anything that matters can take years, and many organisations never get there.
In Australia, AI adoption is no longer the barrier. AI is already part of everyday work, yet fewer than one in four Australians have received any AI training [1]. The technology has arrived faster than the capability to use it well.
For mid-sized organisations, that gap carries a real cost. They are large enough for scattered experiments to waste meaningful budget, but rarely large enough to fund a dedicated team to turn those experiments into results.
The Adoption Trap: Why Do AI Pilots Stall?
Most AI initiatives do not fail because the technology is weak. They stall because nobody has decided what success looks like. Preliminary research from MIT found that 95 per cent of organisations studied could not point to any measurable financial impact from their generative AI pilots [2].
Boston Consulting Group reached a similar conclusion: the hardest obstacles are rarely technical. They sit with people, processes and the way decisions are made [3]. The software is the visible part of an AI project, but the change in how people work is where value is won or lost.
This is where mid-sized organisations hold a quiet advantage. In many cases, they can move from pilot to everyday use much faster than large enterprises [2]. Agility is an asset, provided it is aimed at the right problem.
The Value Gap in Practice: What Does It Look Like in Care and Government?
An NDIS or aged care provider. Picture a provider that gives support workers an AI tool to help draft shift notes and progress reports. Within weeks, staff report that they are saving time. Yet nobody recorded how long the notes took before, and every note still has to be checked and re-entered. The time saved is absorbed by rework, and a new privacy risk has quietly appeared. The answer is not a better tool. It is a baseline, a clear policy on client data, and a review process redesigned around the new way of working.
The public sector. A whole-of-government trial of a generative AI assistant shows the same pattern in the real world. Only a third of staff used the tool daily, and early expectations went partly unmet [4]. Treasury’s own review found that individual time savings did not translate into noticeable gains for whole teams [5]. Use was higher where people received training tailored to their work [4].
Not-for-profits and financial services face similar challenges, from privacy concerns [6] to governance that lags behind adoption [7]. What is often missing is a clear link between the tool, the workflow and the outcome it is meant to improve.
Measuring What Matters: How Can You Tell If AI Is Delivering a Return?
Coming from a background in projects and finance, I apply the same test to AI as to any investment: define the return before you spend.
Set a baseline. Record how long a process takes, what it costs and how often errors occur before AI is introduced. Without a baseline, “it feels faster” becomes the only evidence.
Know your break-even point. Treasury’s evaluators estimated that a mid-level staff member would need to redirect only around 13 minutes a week to higher-value work to cover the cost of a licence [5]. The bar is modest, but having a number turns opinion into something you can test.
Measure at team level. If one person saves an hour but turnaround times, service quality or client outcomes stay the same, the value has not yet reached the organisation.
From Experiment to Impact: What Do Successful Organisations Do Differently?
The organisations that capture value tend to do less, but do it deliberately [3]. In practice, that means:
- Choosing one or two workflows that matter, such as client intake at a care provider or customer correspondence at a utility.
- Training staff in that specific context, not in AI in general.
- Setting clear rules and ownership, with a usage policy and an accountable lead.
- Reviewing results after 90 days against the baseline, then scaling what works and stopping what does not.
The Real Question for Leaders
Adopting AI is now the easy part. The organisations that pull ahead will treat it as a business decision with a baseline, an owner and a clear measure of success. That discipline does not slow innovation down. It is what allows a promising pilot to become a lasting part of how the organisation works.
The question is no longer “Are we using AI?” It is “What has it changed, and how do we know?”
References
- Gillespie, N., Lockey, S., Ward, T., Macdade, A. & Hassed, G. (2025). Trust, attitudes and use of artificial intelligence: A global study 2025. The University of Melbourne and KPMG. https://doi.org/10.26188/28822919; https://mbs.edu/faculty-and-research/trust-and-ai/australian-story
- MIT NANDA (2025). The GenAI Divide: State of AI in Business 2025 (preliminary report). Coverage: https://virtualizationreview.com/articles/2025/08/19/mit-report-finds-most-ai-business-investments-fail-reveals-genai-divide.aspx; https://rcpmag.com/articles/2025/08/25/only-1-in-20-ai-investments-deliver-roi.aspx
- Boston Consulting Group (2024). Where’s the Value in AI? https://www.bcg.com/press/24october2024-ai-adoption-in-2024-74-of-companies-struggle-to-achieve-and-scale-value; full report: https://media-publications.bcg.com/BCG-Wheres-the-Value-in-AI.pdf
- Digital Transformation Agency (2024). Evaluation of the whole-of-government trial of generative AI. https://www.digital.gov.au/initiatives/copilot-trial/microsoft-365-copilot-evaluation-report-full
- Australian Centre for Evaluation, Department of the Treasury (2025). Evaluation of a trial of generative artificial intelligence in the Department of the Treasury. https://evaluation.treasury.gov.au/publications/evaluation-generative-artificial-intelligence
- Infoxchange (2025). Digital Technology in the Not-for-Profit Sector. https://apo.org.au/node/332934; https://www.infoxchange.org/au/digital-technology-not-for-profit-sector
- Australian Securities and Investments Commission (2024). Report 798: Beware the gap: Governance arrangements in the face of AI innovation. https://download.asic.gov.au/media/mtllqjo0/rep-798-published-29-october-2024.pdf
At Centelon Solutions, we help mid-sized organisations move from AI pilots to measurable results, by setting baselines, redesigning workflows and bringing their people along with the technology. See how this works in Aged Care & NDIS, the Public Sector, Energy & Utilities and Non-Profit, or take our free AI readiness assessment to find out where your organisation stands today.
