A food delivery driver has reached an undisclosed financial settlement with Uber Eats for race discrimination after ethnically-biased facial recognition technology led to his suspension.
Pa Edrissa Manjang, who has worked as an Uber Eats driver in Oxfordshire since 2019, filed an employment tribunal claim for indirect race discrimination, harassment and victimisation against the food delivery company in October 2021.
Uber Eats attempted to have the claim struck out, but the judge refused after a hearing in May 2022 and largely confirmed his ruling on reconsideration in September 2023. The case was still in its preliminary stages, which the EHRC said shows the complexity of a claim dealing with AI technology. The full hearing had been scheduled for 17 days later this year.
The Equality and Human Rights Commission (EHRC) and the App Drivers and Couriers Union (ADCU) provided funding for Manjang’s case after he continuously experienced difficulties with Uber Eats’ verification checks, which use facial recognition software, powered by artificial intelligence.
Manjang was removed from Uber Eats’ drivers platform, powered by Microsoft, following a failed recognition check and subsequent automated process in 2021. Uber Eats told Manjang they had found “continued mismatches” in the photos of his face he had taken to access the platform.
Facial recognition
Couriers allege facial ID software used by Uber is ‘indirectly racist’
PwC facial recognition tool criticised for home working privacy invasion
The EHRC and ADCU were concerned by the use of AI and automated processes in this case, particularly how it could be used to permanently suspend a driver’s access to the app, depriving them of an income.
It became clear that Uber Eats had not made Mr Manjang aware of the processes that led to his suspension from the app, nor provided an effective route to challenge this decision. Uber now says its identification technology includes “robust human review”.
Facial recognition technology has been criticised for discriminating against ethnic minorities.
Dr Gideon Christian, assistant professor at the University of Calgary who researches race, gender and privacy impacts of AI facial recognition technology, said last year: “There is this false notion that technology, unlike humans, is not biased. That’s not accurate. Technology has been shown to have the capacity to replicate human bias.
“In some facial recognition technology, there is over 99% accuracy in recognising white male faces. But, unfortunately, when it comes to recognising faces of colour, especially the faces of Black women, the technology seems to manifest its highest error rate, which is about 35%.”
EHRC chairwoman Baroness Kishwer Falkner today said: “I am pleased that Pa Edrissa Manjang was able to settle his case with Uber Eats, but he should never have had to bring a legal claim in order to understand the opaque processes that affected his work.
If AI is deployed in a way that lacks transparency and fairness, it will have a highly corrosive effect on trust. It is vital that any automated decision-making processes affecting people’s livelihoods are not opaque. Businesses using these systems must absolutely prioritise openness and accountability” – Paul Jennings, Bates Wells
“AI is complex and presents unique challenges for employers, lawyers and regulators. It is important to understand that as AI usage increases, the technology can lead to discrimination and human rights abuses.
“We are particularly concerned that Mr Manjang was not made aware that his account was in the process of deactivation, nor provided any clear and effective route to challenge the technology. More needs to be done to ensure employers are transparent and open with their workforces about when and how they use AI.
“Every employer should take note of this, even major companies like Uber. When such companies rely on automation to help manage their staff they need to guard against unlawful discrimination.”
Manjang continues to work for Uber Eats. He was suspended from the app in 2021, but his access was later restored and he started working for the company again. The process that led to him being restored was never fully explained to him by Uber Eats.
Manjang said: “My case shines a spotlight on the potential problems with the use of AI, particularly for low-paid workers in the gig economy who want to understand how decisions are being taken which affect their livelihoods. This marks the end of a long and difficult case.”
Zamir Dreni, vice chair of ADCU London said: “As the pace of development of AI and machine learning tools in the workplace accelerates, trade unions are working hard to ensure that workers’ rights are protected. The ADCU particularly appreciates the support and attention given to this important case by the EHRC as Britain’s equality and human rights regulator.”
Hannah Wright, senior associate at Bates Wells who represented Mr Manjang said: “This has been a very important case. It is among the first to consider AI and automated decision-making in the context of work and the potential for unfairness and discrimination.
“Sophisticated AI systems are increasingly becoming a part of how people are managed at work. This carries substantial risks, particularly where decision-making processes are opaque and particularly in terms of equality. The current protections are inadequate and the process for challenging decisions involving AI is fraught with difficulty.”
Paul Jennings, partner at Bates Wells, added: “AI has very rapidly become part of workforce management. If it is deployed in a way that lacks transparency and fairness, it will have a highly corrosive effect on trust. It is vital that any automated decision-making processes affecting people’s livelihoods are not opaque. Businesses using these systems must absolutely prioritise openness and accountability.”
An Uber spokesperson said: “Our real-time ID check is designed to help keep everyone who uses our app safe and includes robust human review to make sure that we’re not making decisions about someone’s livelihood in a vacuum, without oversight.”
The government has this week published guidance on responsible AI in recruitment to help employers reduce the likelihood of introducing systems that perpetuate biases and discrimination.
Sign up to our weekly round-up of HR news and guidance
Receive the Personnel Today Direct e-newsletter every Wednesday
