Machine learning is quickly becoming one of the most popular technologies that companies are investing in. Experts are growing increasingly worried that these models have a dangerous propensity for making mistakes when it comes to applications such as image recognition software used to diagnose illnesses, or surveillance software used to recognize human faces. However, advancements in machine learning may soon help reduce bias in these systems.

Data Diversity Key to Overcoming Bias in Neural Networks

A team of researchers from MIT and Harvard have found that training machine learning models on diverse sets of data can help them reduce bias, MIT News reports. Data sets that contain limited data are much more likely to discriminate when they make decisions. For example, facial recognition systems trained on data sets containing images of mostly white men are much more likely to give incorrect results when given images featuring women and people of color. 

Relying on a method that used controlled data sets, the researchers sought to learn how training data impacts whether an artificial neural network (a machine learning model that uses brain-like nodes to process data) can figure out how to recognize new objects. 

The researchers created data sets that contained an equal number of images of various objects in different positions (for example, photos of a car from multiple angles). They made some of these data sets more diverse by displaying the images from different points of view. Machine learning models the researchers trained on the more diverse data sets were better at generalizing new viewpoints. The result supports the idea that data diversity is necessary for overcoming bias. However, the researchers also found that the better a model gets at recognizing new objects, the worse it gets at recognizing objects it has already seen.

“A neural network can overcome dataset bias, which is encouraging,” Xavier Boix, a research scientist and senior author of the paper, told MIT News. “But the main takeaway here is that we need to take into account data diversity. We need to stop thinking that if you just collect a ton of raw data, that is going to get you somewhere. We need to be very careful about how we design data sets in the first place.”

The team also found that training a model separately for individual tasks, rather than training a model for each task at the same time, helped models become less biased. This largely has to do with neuron specialization. During separate training, neural networks produce two different kinds of neurons, which Boix finds fascinating. One neuron becomes good at recognizing object categories, and the other learns how to recognize viewpoints. Conversely, if these neurons are trained simultaneously, they can become diluted and confused. 

Machine learning has come a long way, but there is still much to learn in order to develop the field. While the technology is promising, organizations should take steps to ensure they are doing their best to prevent bias in the systems they use or create. 

What Uses Do You Predict Machine Learning Will Have in Your Company?

By providing AI with the ability to learn from its experiences without needing explicit programming, machine learning plays a critical role in developing the technology. Covering machine learning models, algorithms, and platforms, Machine Learning: Predictive Analysis for Business Decisions, is a five-course program from IEEE.

Connect with an IEEE Content Specialist today to learn more about this program and how to get access to it for your organization.

Interested in the program for yourself? Visit the IEEE Learning Network.

Resources

Zewe, Adam. (21 February 2022). Can machine-learning models overcome biased datasets? MIT News.

In recent years, artificial intelligence (AI) applications have exploded in popularity. A few examples include text editors, facial recognition systems, digital assistants, and much more. Simply put, AI is the ability for machines to perform tasks that require a certain level of intelligence. As an overarching branch of computer science, AI contains a number of subsets, two of the most common are machine learning and deep learning. 

What is Machine Learning?

Machine learning algorithms are trained to parse data into bits, then recombine that data to learn and solve problems in order to make knowledgeable decisions. Machine learning systems are trained with structured data in which patterns are clearly defined. While machine learning models can get better at solving problems with limited supervision, they can still require some human guidance, especially if they get stuck on a problem. 

Machine learning is commonly used in image and speech recognition, email spam detectors, and to predict shifts in weather and stock markets.

What is Deep Learning?

Deep learning is considered by many experts to be an evolved subset of machine learning. Whereas traditional machine learning systems rely on structured data, deep learning continually analyzes data using an advanced technology known as “artificial neural networks,” which can process unstructured data such as images. These networks are operated by a series of algorithms that can perceive complex relationships in data sets through a process that imitates the human brain. 

Deep learning models are used by online retail companies and streaming services to suggest products and TV shows you might be interested in based on previous selections. It’s also used in facial recognition software and in autonomous vehicle systems. 

Deep learning is a revolutionary technology that some “consider [to be] the backbone of true artificial intelligence.” Unlike traditional machine learning applications, deep learning models can analyze and solve a problem in a single instance, much like the human brain, and can get better at problem solving without human intervention. 

GPT-3, CLIP, and DALL-E

Deep learning is propelling radical advancements in search engine technology and natural language processing (NLP) models—artificial intelligence that can automatically manipulate speech and text.

Last year, OpenAI, an AI research and deployment company, unveiled a breakthrough NLP model that applies deep learning to mimic human language. Known as Generative Pre-trained Transformer 3, or GPT-3, the system uses autocomplete technology, a feature in Google Search that can speed up searches by predicting what you are going to type next. (It does this by matching what you are typing to commonly searched for words and phrases).

Combining autocomplete technology with massive amounts of data gleaned from the internet, GPT-3 can generate text on its own. As an example, here is an article written by a GPT-3 application without human assistance.

Similarly, OpenAI recently built a pair of new deep learning models dubbed “DALL-E” and “CLIP,” which merge image detection with language. As such, they can help language models such as GPT-3 better understand what they are trying to communicate. 

CLIP (Contrastive Language-Image Re-Training) is trained to predict which image caption out of 32,768 random images is the right caption for a specific image. It learns image content based on descriptions instead of one-word labels (like “dog” or “house”.) It then learns to connect a wide array of objects with their names in addition to words that describe them. This allows CLIP to identify objects within images outside the training set, meaning it’s less likely to be confused by subtle similarities between objects.

Unlike CLIP, DALL-E doesn’t recognize images—it illustrates them. For example, if you give DALL-E a natural-language caption, it will draw a variety of images that matches it. In one example, DALL-E was asked to create armchairs that looked like avocados, and it successfully produced a number of different results, all which were accurate. See a picture of the results here. 

“The thing that surprised me the most is that the model can take two unrelated concepts and put them together in a way that results in something kind of functional,” Aditya Ramesh, one of DALL·E’s designers, told MIT Technology Review. 

Making Deep Learning Cheaper, Faster, and More Efficient

While deep learning can deliver impressive results, it has some limitations. For example, it has vast data and mathematical processing needs that consume enormous amounts of energy. Deep learning also relies on more sophisticated hardware than traditional machine learning systems, in addition to requiring extensive data and lengthy training times. 

However, engineers are working on innovative solutions to this problem. For example, Lightmatter, a company that makes next-generation computing platforms for artificial intelligence systems, has developed a neural-network accelerator chip that can make calculations more efficiently by using photons instead of electrons. While the technology isn’t currently as precise as today’s chips, it represents a step forward in the quest to make deep learning cheaper, faster, and more efficient. 

As machine learning and deep learning models evolve, they are spurring revolutionary advancements in other emerging technologies, including autonomous vehicles and the internet of things. 

Understand Machine Learning

Machine learning is a vital aspect of artificial intelligence (AI). Because machine learning allows AI systems to learn from experiences without needing explicit programming, it’s key for the future of AI technology.

Check out these new courses on machine learning, available on the IEEE Learning Network today.

Machine Learning in the Age of Enterprise Big Data
Machine Learning in a Data-Driven Business Environment
Machine Learning Algorithms, Models, and Systems Integration

Resources

Schneider, David. (8 January 2021). Deep Learning at the Speed of Light. IEEE Spectrum.

Douglas Heaven, Will. (5 January 2021). This avocado armchair could be the future of AI.
MIT Technology Review.

(17 December 2020). The Difference Between Deep Learning and Machine Learning. Tech Funnel. 

(20 May 2019). Deep learning & Machine learning: what’s the difference? Parsers.

Grossfeld, Brett. (23 January 2020). Deep learning vs machine learning: a simple way to understand the difference. ZenDesk.