As the cost of living continues to rise, the price of basic necessities is a major concern for many people in Ontario. That becomes even more concerning when the price of these essentials could vary by certain retailers depending on who you are.
This is the result of surveillance pricing, where a retailer employs automated algorithms and artificial intelligence technologies to analyze personal data and set individualized prices in real time. Unlike dynamic pricing, which adjusts prices based on overall market demand - such as higher airfare during holiday weekends - surveillance pricing tailors prices to each consumer’s personal circumstances and behaviour, often without their knowledge. As a result, two individuals may be shown very different prices for the same product, with no transparency or way to compare.
In Ontario, surveillance pricing poses a serious risk of violating the Human Rights Code (the Code). The Code is a provincial law that protects individuals from discrimination based on listed characteristics (or grounds) including race, sex, disability, age, or family status (e.g., being in a parent/guardian-child relationship). If surveillance pricing relies, even partially, on data that includes or is connected to someone’s protected grounds to set higher prices, it could violate their right to be free from discrimination under the Code. Notably, the intention behind a business’s decision to use surveillance pricing does not matter. Rather, it is the discriminatory impact that matters. In the case of surveillance pricing, the impact may well be that we are all offered different prices for the same products based on our race, age, disability or other grounds.
Consider some potential scenarios: A woman using a ride-sharing app late at night is charged a higher rate than men. A person with a mobility disability may pay more for food delivery. An Asian person is charged more for decorative items at an online store just before Lunar New Year.
The Code was established to protect Ontarians from these types of situations. Currently, the Ontario Human Rights Commission (the Commission) has increased its work on how emerging practices and technologies, such as surveillance pricing and other artificial intelligence technologies, intersect with human rights.
In partnership with other leading human rights, privacy, and law reform organizations, the Commission has released a Human Rights AI Impact Assessment with practical guidance to identify and mitigate potential discrimination from AI systems, and Principles for the Responsible Use of Artificial Intelligence. These guides help organisations develop and use of AI in ways that maintain public trust by respecting privacy and human rights. Any business or organization in Ontario contemplating the use of personal data or AI should use them to ensure compliance with the Code.
The Commission’s mandate is to promote and protect human rights across Ontario. That includes proactively addressing new risks posed by emerging practices, such as surveillance pricing, to prevent discrimination. As the potential benefits of tools like artificial intelligence are being explored, our fundamental human rights must be continuously monitored and protected.