Machine learning is quickly becoming one of the most popular technologies that companies are investing in. Experts are growing increasingly worried that these models have a dangerous propensity for making mistakes when it comes to applications such as image recognition software used to diagnose illnesses, or surveillance software used to recognize human faces. However, advancements in machine learning may soon help reduce bias in these systems.

Data Diversity Key to Overcoming Bias in Neural Networks

A team of researchers from MIT and Harvard have found that training machine learning models on diverse sets of data can help them reduce bias, MIT News reports. Data sets that contain limited data are much more likely to discriminate when they make decisions. For example, facial recognition systems trained on data sets containing images of mostly white men are much more likely to give incorrect results when given images featuring women and people of color. 

Relying on a method that used controlled data sets, the researchers sought to learn how training data impacts whether an artificial neural network (a machine learning model that uses brain-like nodes to process data) can figure out how to recognize new objects. 

The researchers created data sets that contained an equal number of images of various objects in different positions (for example, photos of a car from multiple angles). They made some of these data sets more diverse by displaying the images from different points of view. Machine learning models the researchers trained on the more diverse data sets were better at generalizing new viewpoints. The result supports the idea that data diversity is necessary for overcoming bias. However, the researchers also found that the better a model gets at recognizing new objects, the worse it gets at recognizing objects it has already seen.

“A neural network can overcome dataset bias, which is encouraging,” Xavier Boix, a research scientist and senior author of the paper, told MIT News. “But the main takeaway here is that we need to take into account data diversity. We need to stop thinking that if you just collect a ton of raw data, that is going to get you somewhere. We need to be very careful about how we design data sets in the first place.”

The team also found that training a model separately for individual tasks, rather than training a model for each task at the same time, helped models become less biased. This largely has to do with neuron specialization. During separate training, neural networks produce two different kinds of neurons, which Boix finds fascinating. One neuron becomes good at recognizing object categories, and the other learns how to recognize viewpoints. Conversely, if these neurons are trained simultaneously, they can become diluted and confused. 

Machine learning has come a long way, but there is still much to learn in order to develop the field. While the technology is promising, organizations should take steps to ensure they are doing their best to prevent bias in the systems they use or create. 

What Uses Do You Predict Machine Learning Will Have in Your Company?

By providing AI with the ability to learn from its experiences without needing explicit programming, machine learning plays a critical role in developing the technology. Covering machine learning models, algorithms, and platforms, Machine Learning: Predictive Analysis for Business Decisions, is a five-course program from IEEE.

Connect with an IEEE Content Specialist today to learn more about this program and how to get access to it for your organization.

Interested in the program for yourself? Visit the IEEE Learning Network.

Resources

Zewe, Adam. (21 February 2022). Can machine-learning models overcome biased datasets? MIT News.

In recent years, artificial intelligence (AI) applications have surged in popularity. Examples include text editors, facial recognition systems, and digital assistants. Simply put, AI enables machines to perform tasks that require intelligence. As a branch of computer science, AI contains several subsets, with machine learning and deep learning among the most common.

What is Machine Learning?

Machine learning algorithms parse data into smaller pieces, then recombine that data to learn and solve problems. Through this process, they make informed decisions. Engineers train machine learning systems with structured data, where patterns are clearly defined. Although models improve with limited supervision, they still need human guidance when they get stuck.

In practice, machine learning powers image and speech recognition, email spam filters, and predictions in weather and stock markets.

What is Deep Learning?

Deep learning builds on machine learning. Unlike traditional models, deep learning uses artificial neural networks to process unstructured data such as images. These networks run advanced algorithms that detect complex relationships in data, imitating the human brain.

For example, online retailers and streaming services use deep learning to recommend products and shows. It also drives facial recognition software and autonomous vehicle systems.

Deep learning represents a breakthrough. Unlike traditional approaches, it can solve problems in a single step and improve without human intervention. Many experts even call it the “backbone of true AI“.

GPT-3, CLIP, and DALL-E

Deep learning is propelling radical advancements in search engine technology and natural language processing (NLP) models—artificial intelligence that can automatically manipulate speech and text.

GPT-3

Last year, OpenAI, an AI research and deployment company, unveiled a breakthrough NLP model that applies deep learning to mimic human language. Known as Generative Pre-trained Transformer 3, or GPT-3, the system uses autocomplete technology, a feature in Google Search that can speed up searches by predicting what you are going to type next. (It does this by matching what you are typing to commonly searched for words and phrases).

Combining autocomplete technology with massive amounts of data gleaned from the internet, GPT-3 can generate text on its own. As an example, here is an article written by a GPT-3 application without human assistance.

CLIP

Similarly, OpenAI recently built a pair of new deep learning models dubbed “DALL-E” and “CLIP,” which merge image detection with language. As such, they can help language models such as GPT-3 better understand what they are trying to communicate. 

CLIP (Contrastive Language-Image Re-Training) is trained to predict which image caption out of 32,768 random images is the right caption for a specific image. It learns image content based on descriptions instead of one-word labels (like “dog” or “house”.) It then learns to connect a wide array of objects with their names in addition to words that describe them. This allows CLIP to identify objects within images outside the training set, meaning it’s less likely to be confused by subtle similarities between objects.

DALL-E

Unlike CLIP, DALL-E doesn’t recognize images—it illustrates them. For example, if you give DALL-E a natural-language caption, it will draw a variety of images that matches it. In one example, DALL-E was asked to create armchairs that looked like avocados, and it successfully produced a number of different results, all which were accurate. See a picture of the results here. 

“The thing that surprised me the most is that the model can take two unrelated concepts and put them together in a way that results in something kind of functional,” Aditya Ramesh, one of DALL·E’s designers, told MIT Technology Review. 

Making Deep Learning Cheaper, Faster, and More Efficient

While deep learning can deliver impressive results, it has some limitations. For example, it has vast data and mathematical processing needs that consume enormous amounts of energy. Deep learning also relies on more sophisticated hardware than traditional machine learning systems, in addition to requiring extensive data and lengthy training times. 

However, engineers are working on innovative solutions to this problem. For example, Lightmatter, a company that makes next-generation computing platforms for artificial intelligence systems, has developed a neural-network accelerator chip that can make calculations more efficiently by using photons instead of electrons. While the technology isn’t currently as precise as today’s chips, it represents a step forward in the quest to make deep learning cheaper, faster, and more efficient. 

As machine learning and deep learning evolve, they drive advancements in autonomous vehicles and the Internet of Things. Looking ahead, these technologies will continue reshaping industries.

Understand Machine Learning

Machine learning is a vital aspect of artificial intelligence (AI). Because machine learning allows AI systems to learn from experiences without needing explicit programming, it’s key for the future of AI technology.

Check out these new courses on machine learning, available on the IEEE Learning Network today.

Machine Learning in the Age of Enterprise Big Data
Machine Learning in a Data-Driven Business Environment
Machine Learning Algorithms, Models, and Systems Integration

Resources

Schneider, David. (8 January 2021). Deep Learning at the Speed of Light. IEEE Spectrum.

Douglas Heaven, Will. (5 January 2021). This avocado armchair could be the future of AI.
MIT Technology Review.

(17 December 2020). The Difference Between Deep Learning and Machine Learning. Tech Funnel. 

(20 May 2019). Deep learning & Machine learning: what’s the difference? Parsers.

Grossfeld, Brett. (23 January 2020). Deep learning vs machine learning: a simple way to understand the difference. ZenDesk.