Although the sources of bias in AI systems are varied, one of the main causes is stereotyping.
To help fix this problem in herOriginally, researchers at the Princeton Visual AI Lab have developed an open source tool that automatically detects potential corruptions in visual datasets. The tool allows dataset creators and users to correct under-representation or stereotyped image issues before image collections are used to train computer vision models. In related work, members of the Visual AI Lab published a comparison of existing bias-prevention methods in computer vision models themselves and proposed a new, more effective approach to bias reduction.
The first tool, called REVISE, usesstatistical methods to examine a data set for potential biases or underrepresentation issues along three dimensions: object, gender, and geography.
REVISE evaluates the contents of the dataset,using existing image annotations and measurements such as the number of objects, the combined presence of objects and people, and the country of origin of the images. Among these measurements, the tool identifies patterns that differ from the median distribution.
In one dataset, REVISE identified a potentialgender bias in images depicting people (red rectangles) and the organ of a musical instrument (blue rectangles). An analysis of the distribution of the estimated three-dimensional distances between a person and an organ showed that men were usually depicted as actually playing the instrument, while women were often simply in the same space as the instrument. Credit: Princeton Visual AI Lab.
For example, in one of the tested setsREVISE data showed that images involving people and flowers differed between men and women: men were more likely to appear with flowers at ceremonies or meetings, and women were more likely to appear in staged settings or paintings.
Once the tool detects this kind ofdiscrepancies, “the question arises whether this is a completely harmless fact or whether there is something more important going on, and this is very difficult to automate,” explains Olga Rusakovskaya, assistant professor of computer science and principal investigator of the Visual AI Lab.
“The practice of collecting datasets in computerscience until recently has not been studied so thoroughly, ”concludes co-author Angelina Wang, a graduate student in computer science. She explains that the images are mostly “from the Internet, and people don't always understand that their images are being used [in datasets]. We need to collect images from more diverse groups of people and do it respectfully. "
Read also:
It is possible to create a thermonuclear reactor on Earth. What will be the consequences?
The Doomsday glacier turned out to be more dangerous than scientists thought. We tell the main thing
On day 3 of illness, most COVID-19 patients lose their sense of smell and often suffer from a runny nose