Machine learning models often rely on the simple features of a dataset to make decisions. Known as “shortcuts,” these types of decisions can lead to serious errors. For example, these shortcuts can cause models to make inaccurate medical diagnoses. However, a recent study from MIT poses a possible solution. By removing the simple characteristics of a dataset, the researchers forced the model to examine the more complex features of a dataset.
“It is still difficult to tell why deep networks make the decisions that they do, and in particular, which parts of the data these networks choose to focus upon when making a decision,” Joshua Robinson, a PhD student in the Computer Science and Artificial Intelligence Laboratory (CSAIL) and lead author of the paper, told MIT News. “If we can understand how shortcuts work in further detail, we can go even farther to answer some of the fundamental but very practical questions that are really important to people who are trying to deploy these networks.”
How To Avoid Shortcuts in Machine Learning
As MIT News reports, the new research centers on a type of self-supervised machine learning known as contrastive learning. Self-supervised models are trained on raw data that don’t have any label descriptions. In contrastive learning models, an encoder algorithm is trained to distinguish between pairs of similar inputs and pairs of dissimilar inputs, which encodes complex data, such as images, in a way the model can decipher. While this makes decision making more effective, the researchers found that these models also tended to fall victim to making shortcuts by fixating on the simplest features of an image to determine which pairs of inputs are similar and which are not. To solve this, the researchers made it more difficult for the model to differentiate similar and dissimilar pairs, which altered the features the encoder used to make a decision.
“If you make the task of discriminating between similar and dissimilar items harder and harder, then your system is forced to learn more meaningful information in the data, because without learning that it cannot solve the task,” Stefanie Jegelka, one of the researchers, told MIT News.
However, this caused the encoder to get worse at focusing on some features over others, particularly the simpler ones. To solve this, the researchers required the encoder to discriminate between the pairs using the simpler feature, as well as after the researchers removed the data it already learned. Having the encoder solve the problem both ways simultaneously forced it to make better decisions.
Known as “implicit feature modification,” this groundbreaking method does not rely on any input from humans. While it has the potential to help machine learning models avoid shortcuts, the researchers told MIT that it still needs to be refined on other types of self-supervised learning.
Machine learning is still in its infancy. However, innovations such as implicit feature modification have the potential to give artificial intelligence (AI) the ability to learn on its own. Not only will this make AI smarter and more efficient, it can also lead to revolutionary technological and scientific discoveries, as machine learning has the ability to solve complex problems— such as determining protein’s 3D shape— that humans cannot.
Understand Machine Learning
By providing AI with the ability to learn from its experiences without needing explicit programming, machine learning plays a critical role in developing the technology. Covering machine learning models, algorithms, and platforms, Machine Learning: Predictive Analysis for Business Decisions, is a five-course program from IEEE.
Connect with an IEEE Content Specialist today to learn more about this program and how to get access to it for your organization.
Interested in the program for yourself? Visit the IEEE Learning Network.
Resources
Zewe, Adam. (2 November 2021). Avoiding shortcut solutions in artificial intelligence. MIT News.
Callaway, Ewen. (30 Nov 2020). ‘It will change everything’: DeepMind’s AI makes gigantic leap in solving protein structures. Nature.
No comments yet.