From trusting AI to taking ownership of AI
An autonomous system approves a payment to a supplier. The payment turns out to be fraudulent. Who is responsible for this? It’s a question most organizations can’t answer.
When there’s no owner, there’s little accountability. And that’s where things often go wrong.
Step 1: Start where there is the most room to learn
Step 2: Make explainability a requirement
Step 3:Investment Ownership
Step 1: Start where there is the most room to learn
Step 2: Make explainability a requirement
Step 3:Investment Ownership
AI implementations rarely fail because of the technology itself. They fail because of issues surrounding it that haven’t been addressed: ownership, explainability, and the discipline to start small. When you do address these issues, you not only get more out of autonomous AI, but you also build an organization that can take responsibility for it.
AI is increasingly taking the wheel in business processes. But are organizations ready for this? 317 IT and business leaders share their perspectives on autonomous decision-making, control, and trust.
The temptation to deploy AI straight away for high-stakes decisions is understandable. But an early mistake in a critical process costs more than just time. It also costs trust. And once that trust is gone, you can’t simply rebuild it.
So start with processes where a mistake is manageable and the impact remains internal: routine approvals, internal planning issues, recurring payments within set guidelines. In that environment, you’ll discover how the system makes decisions, where the limits lie and when human intervention is needed. That experience forms the basis for everything that follows.
Autonomous decisions are only justifiable if you can reconstruct them. Who did the system decide, based on what data and with what result? It must always be possible to trace this. Not only internally, but also for customers, auditors or regulators.
You should therefore ensure that explainability is not merely a preference but a strict requirement during implementation. Document every decision. Make it traceable. This enables you to make adjustments if the system goes wrong, to account for your actions when asked, and to improve the system based on what you learn. With legislation such as the EU AI Act on the horizon, this traceability is also becoming increasingly mandatory. You would be well advised to prepare for this in good time.
Explainability only works if there is someone to provide the explanation. Ownership is something you organise in advance so that there is always someone who knows what is going on, who is keeping an eye on things and who intervenes where necessary.
This requires explicit decisions. Who determines the parameters within which the system is permitted to operate, who endorses the objectives, and who is the first to respond if a decision goes wrong? A useful starting point is to give the system the same decision-making authority as your staff. If a member of staff is not permitted to authorise a payment above a certain amount without additional approval, then the same limit applies to the system that takes over that task. In this way, responsibility remains clear and manageable.
Back to that fraudulent payment. Anyone who has followed these three steps knows who is responsible, can demonstrate how the decision was reached, and knows how to rectify the situation. Ownership not only gives you control over AI, but also control over the decisions made in your organisation’s name.
Take our Autonomous ERP Readiness Quick Scan and find out in 5 minutes how your organisation scores in terms of Technology, Organisation, People and Governance. You’ll receive personalised results straight away, highlighting specific areas for attention.