Artificial intelligence (AI) and automation are rapidly transforming the insurance sector, with $8 billion invested in insurtech start-ups between 2018 and 2019. However, the use of algorithms in insurance has come under fire for biased outcomes, resulting in policy makers introducing regulation targeting the algorithms used in insurance. Colorado's Senate Bill 21-169 and the European Commission's EU AI Act seek to prohibit insurers from unfair discrimination based on protected characteristics and ensure that AI systems meet certain obligations. The National Association of Insurance Commissioners has also emphasized the importance of accountability, compliance, and transparency in the use of AI in insurance throughout its entire lifecycle.

The New York Department of Financial Services published a circular letter in January 2019 to insurers authorized to write life insurance in the state. The letter warns insurers not to use external data sources, algorithms, or predictive models in underwriting or rating unless it has been determined that the system does not collect or use prohibited criteria. The burden and liability lie with the insurer, and the NYDFS reserves the right to audit and examine an insurer’s underwriting criteria, programs, algorithms, and models and can take disciplinary action if necessary. The letter also highlights the obligation to comply with existing anti-discrimination and civil rights laws and regulations. Insurers should provide transparency to consumers regarding the reason or reasons for any adverse underwriting decisions made using external data sources or predictive models. Failure to comply may result in an NYDFS audit and breach of existing anti-discrimination laws.

The global AI market is set to reach $500 billion by 2023, and the insurance industry is using AI in purchasing, underwriting and claims activities. However, there are risks of perpetuating existing biases such as charging higher car insurance premiums to minority groups. As a result, AI risk management frameworks are needed in the insurance industry, and forthcoming regulations such as the EU AI Act and Colorado State legislation aim to ensure high-quality and unbiased data, transparency, and appropriate accountability. Taking early steps in managing AI risks allows enterprises to embrace AI with more confidence.

Colorado has enacted legislation that restricts insurers’ use of ‘external consumer data’ and prohibits data, algorithms, or predictive models from unfairly discriminating. Insurers are required to outline the type of external customer data and information sources used by their algorithms and predictive models, establish a risk management framework, provide an assessment of the results of the risk management framework, and provide an attestation that the risk management framework has been implemented. Unfair discrimination occurs when external customer data and information sources or algorithms or predictive models correlate with protected characteristics and result in a disproportionately negative outcome for these groups. The law will come into effect on 1st January 2023 at the earliest. Holistic AI can help firms establish a risk management framework for continuous monitoring of data, algorithms, and predictive models and provide expert evidence for non-discriminatory practices.