»ÆÉ«¶ÌÊÓƵ

Andrew McStay

Moving fast and breaking people? Empathic AI companions and UK citizen perspectives

This talk examines the ethical challenges and societal implications of empathic AI companions, drawing on UK public attitudes and civil lawsuits against Character.ai. The lawsuit highlights critical design flaws, inadequate safeguards, and ethical dilemmas, especially the blurred boundaries between reality and fiction. Survey findings reveal demographic divides in familiarity and usage, but also shared concerns about privacy, emotional dependency, and the appropriateness of AI companions for children and older adults. While respondents recognise benefits such as reducing loneliness and aiding education, anthropomorphic design elements evoke mixed reactions, raising ethical questions about simulated emotion and inappropriate user deception. The talk advocates for age-appropriate design and stronger regulatory frameworks, emphasising the need for balanced policies to protect vulnerable populations while fostering creativity and responsible innovation. Actionable recommendations aim to guide policymakers, standards development, industry leaders, and scholars in addressing the ethical complexities of this emerging digital technology.

Short bio

Andrew McStay is Professor of Technology & Society at Bangor University and the author of Automating Empathy: Decoding Technologies that Gauge Intimate Life, published open access in 2024 with Oxford University Press. His work explores the ethical implications of AI systems claimed empathise and understand emotion. Director of the Emotional AI Lab, his current projects include Responsible AI (RAI) funded work to diversify regional input into IEEE-based ethical technical standards for emulated empathy and human-AI partnering (IEEE P7014.1). Other recent work includes a project for the Office of the Privacy Commissioner of Canada on child-focused emotional AI systems. He is also a technology advisory panel member for the UK’s Information Commissioners’ Office.