First, understanding an "algorithm"
According to the Oxford Dictionary, the definitions of ALGORITHM and LAW are very similar.
“A process or set of rules to be followed in calculations or other problem-solving operations, especially by a computer.”
“The system of rules . . . regulating the actions of its members and which . . . may [be] enforce[d] by the imposition of penalties.”
The only difference, really, is that in a democracy, citizens have a right to set the laws that govern them. Not so with algorithms – yet.
Letting math break the law
Democracies the world over have enacted laws to protect human rights. When people in a corporation make decisions that breach those laws, there is a price to pay.
But when a person writes an algorithm that breaks the law and a corporation then profits from that algorithm, the damage gets mostly shrugged off.
Ergo, breaking the law using math can be reasonably profitable.
As Cathy O’Neil so clearly highlights in her book, “Weapons of Math Destruction“, algorithms now determine many aspects of our lives. Pointing out that these math equations come with hidden and not so hidden biases, she exposes the myriad ways that math can undermine people’s individuality and hard work based on biased assumptions baked into the code.
- run company hiring practices,
- decide who gets a loan,
- determine how student papers are graded,
- decide who gets accepted to the most prestigious schools,
- decide what sentence to give to someone convicted of a crime,
- determine who should be cropped out of photos (spoiler alert: brown-skinned people),
- and the list goes on.
Yes, there are lots of great algorithms that are beneficial, too, but not one algorithm, good or bad, is required to function transparently in order for the public to see how their lives are being affected and whether the math is abiding by the laws of the land.
Bad math hurts real people
Why does transparency matter?
Bad math extends into the real world with real consequences.
Yet when it is discovered that an algorithm has broken the law, as in the case of Amazon’s “secret AI recruiting tool that showed bias against women,” regulators seldom offer up more than a mild shrug.
In Amazon’s case, its algorithm-driven recruiting system “was not rating candidates for software developer jobs and other technical posts in a gender-neutral way.”
When it came to female engineers, the Amazon’s recruiting engine learned to “penalize resumes including the word ‘women’s’ until the company discovered the problem.” As a result, too many women engineers had their resumes shoved to the bottom of a very large pile. Hiring managers simply never saw them.
Bias was baked into the math. Bias gets baked into lots of things – including hand sanitizing dispensers, it turns out.
Can a hand sanitizing dispenser be racist?
Yes it can. And soap dispensers, too.
Just this week, a friend’s husband, a surgeon in private practice, came home and said, “I think my hand sanitizers are racist.”
The new hand sanitizing dispensers that the surgeon recently purchased work fine for light-skinned people. But for his darker-skinned staff and patients, the machine will not dispense any sanitizer. (The doctor offered us this video as an example, using a dark glove instead of asking his staff to yet again put their hands under a dispenser that won’t work for them.)
This issue has been going on for years. In a 2017 video, purportedly from a soap dispenser in a men’s washroom at Facebook, we are shown that the machine simply does not provide soap for darker-skinned people. To get some soap, the man has to place a paper towel under the machine’s sensor.
Twitter crops out black faces
In September 2020, a number of Twitter users noticed that the company’s algorithms were racist, as the math routines cropped out black faces, favoring white faces in their users’ feeds. One user, Colin Madland, looked for this racial bias and quickly found it. Why was he specifically looking? Because earlier he’d noticed that a black colleague had been algorithmically erased from a Zoom conference so he began to check other services.
Of course, Twitter apologized and claimed they had tested these math rules.
Racist and sexist algorithms are not new
We apologize in advance for this next video – many of the Microsoft robot tweets are extremely offensive.
Do you recall Microsoft’s robot twitterer, “TayTweets” in 2016?
After going through a “learning” exercise, TayTweets spewed out unbelievably racist and sexist tweets, some of which were captured in this video.
Bad math has been impacting our lives for years, with the burden falling mostly to people of colour, religious and ethnic minorities, and, of course, to women.
Companies don't set out to break laws (usually)
Algorithms that breach human rights legislation are algorithms that have a glitch. Having worked shoulder to shoulder with many programmers, I haven’t met one who would purposefully write a racist or sexist set of math rules to determine mortgage qualifications or evaluate resumes. At the same time, I haven’t worked at any company that mandated their code be tested for this sort of programmed bias.
In fact, there seems to be some magical thinking around this issue: if math or a robot is making a decision, it must be de facto neutral because, after all, it is a machine, not human. Time and again, decision-makers keep forgetting that it is humans who code the machines. There is no magic.
Regulations setting out the criteria to which an algorithm must be tested to prove that it is not harming people or society would have helped Twitter properly test and fix the racist algorithm before offering it to their users.
Amazon may have hired some incredibly gifted female engineers had they first taken the time to ensure the data sets they were using to score candidates did not already contain implicit bias in favour of male engineers.
These regulations might have helped Microsoft executives have a second thought about using Twitter users to “train” its twitter bot. Had the company been expected to first prove (test) that the product they were delivering to the public passed certain hurdles that reflected human rights, more robust verifications would have taken place. Instead, as The New York Times reported, by “learning” from Twitter users, Microsoft’s twitter bot quickly became a “racist jerk.”
When our democracies finally demand that companies produce products and services that respect the laws that our citizens have put in place, those companies will build things like dispensers that won’t act like a “racist jerk” and withhold soap or hand sanitizer based on the color of a person’s skin.