Despite its efficiencies and innovative capacities, the increasing use of Artificial Intelligence (AI) in high-risk applications is raising important legal and ethical questions, with serious repercussions for those affected by its misuse. Legal battles and class actions are beginning to arise around the unsafe and inappropriate use of AI, prompting the need for a closer examination of its implications and risks. The consequences of AI misuse can be severe, from job losses, asset seizure, and wrongful foster care placements of children, threats to individuals’ life chances have left many to seek redress. The Netherlands recently experienced this first-hand when 1.4 million people were wrongfully flagged as fraud by an AI-powered risk profile system. Michigan, too, faced a $20 million settlement after 40,000 state residents were wrongfully accused of fraud by an automated system. To prevent devastating harms and loss of opportunity, it is critical to ensure that AI systems are governed with an ethical approach and equipped with the necessary safeguards to protect individuals from misuse.