Around the world, attempts to automate welfare systems for this purpose have ended in disaster. In the Netherlands, an algorithm falsely flagged childcare benefits claims from 26,000 parents

predominantly immigrant families

as fraudulent between 2013 and 2019. In Australia, an illegally implemented AI system known as Robodebt demanded repayments from tens of thousands of welfare recipients for debts they didn't owe.

"Government bodies are experimenting with uses of AI in areas where there's already widespread systemic inequality," Joanna Redden, co-director of the Starling Centre at Western University and the UK-based Data Justice Lab, told Ricochet Media. "This raises serious concerns about the negative implications of the uses of new and emerging tech in areas like benefits administration."

Despite Ottawa's growing appetite for AI, oversight hasn't kept up.

Federal ministries are required to complete risk evaluations of AI tools at the design stage and before deployment. These evaluations, called Algorithmic Impact Assessments (AIA), comprise dozens of questions to pinpoint potential risks, biases, and transparency issues with AI systems. The Machine Trust Platform also integrates AIAs into its evaluation of AI tools.

When Sieber and PhD candidate Ana Brandusescu reviewed AIAs in a 2025 study, they discovered that almost every department refused to release them. To date, only 30 such assessments have been published.