Announced a company Twitter said Friday that it will change the way the image cropping function works on its website after concerns that the machine learning-based algorithm the company uses was racially biased in how images are cropped, particularly by preferring white faces over black faces.
And in Published Delving into the issue, Parag Agrawal – the company’s chief technology officer – and head of design (Dantley Davis) explained how the company tested the model with racial or gender bias before implementing it. But Twitter did not publish how it conducted these tests at the time so that external analysis could be done, in what the company calls “supervision.”
To fix this, Twitter is currently running “additional analysis to add more rigor to our tests, is committed to sharing our findings, and is exploring ways to open up our analysis so that others can help us with accountability,” company officials wrote.
The company is also working on more tangible changes to its site to ensure that the images are displayed the way users want them to. Specifically, Twitter promises to focus less on using machine learning-based technologies to crop images, and instead will develop tools that show users what images will look like in their posts when writing a tweet. The company also says it has already started trying new image cropping and previews options to give users more control.
From now on, Twitter promised to enact a “what you see is what you get” policy, so that the picture a user attaches when writing a tweet is what he and other users will see when they view it on Twitter. There are some advanced cases that Twitter will have to solve, such as very long or very wide images, but the goal is to make sure that users know from the start how their photos will be displayed on the site.
The company switched to its existing machine learning-based cropping system in 2018, which uses a neural network to shear previews of images based on ‘prominence’. And unlike the company’s previous system, which was designed to focus on faces, the current model is designed to focus on what the algorithm identifies as “the most interesting part of the image” by predicting where the average person will appear for the first time when viewing the full picture.
There is no information yet about when these changes will be released to Twitter’s photo cropping system.