Image segmentation, key to both human and computer vision, is the process of parsing data into distinct groups. While this process comes naturally to humans, it has been a frustrating challenge for the machine learning models vital for enabling computer vision.
Currently, segmentation depends on large, compute-heavy neural networks, which make running deep learning models challenging without a connection to a cloud server. However, a new neural network architecture developed by artificial intelligence (AI) researchers at DarwinAI and the University of Waterloo could make image segmentation for machine learning a less arduous task. The deep learning model, dubbed “AttendSeg,” is a “tiny attention condenser neural network” small enough to allow mobile devices to perform image segmentation on the edge of the cloud. Details about the technology were published by Cornell University in a recent paper accepted at this year’s Conference on Computer Vision and Pattern Recognition (CVPR).
Machine Learning on the Edge
Image segmentation is the most complex classification task. Traditional segmentation networks depend on hardware and a cloud connection that can run deep learning models. However, these pose limitations affecting access (for example, if there is no internet connection), as well as privacy and security (for instance, when images are sent to the cloud). Other limitations include speed, memory, and power.
AttendSeg seeks to eliminate these challenges. According to the paper, the technology is “a low-precision, highly compact deep semantic segmentation network tailored for TinyML applications.” Short for “tiny machine learning,” TinyML refers to machine learning models that can run on low-power and low-memory devices without having to depend on connection to the cloud.
AttendSeg performs highly accurate semantic segmentation while consuming slightly over one megabyte of memory, which fits most edge devices. Semantic image segmentation refers to the process of connecting each pixel in an image to a class label (as opposed to instance image segmentation, the process of identifying and describing each distinct object of interest in an image).
Alexander Wong, co-founder at DarwinAI and co-author of the paper, told TechTalks that the idea for AttendSeg was driven by the researchers’ “desire to advance the field of TinyML.”
“There are numerous industrial applications for highly efficient edge-ready segmentation approaches, and that’s the kind of feedback along with market needs that I see that drives such research,” he said.
To reduce model size without hindering performance, attendSeg uses “light-weight attention condensers”–mechanisms that work in a series to enhance the efficiency of neural networks by focusing only on information that is absolutely necessary.
“[Attention condensers] allow for very compact deep neural network architectures that can still achieve high performance, making them very well suited for edge/TinyML applications,” Wong told Venture Beat.
However, TinyML has a major problem. It depends on high-performing neural networks, but it needs to be affordable enough to meet the budget constraints of device manufacturers. To overcome this, the researchers leveraged “generative synthesis,” a machine learning method that generates neural network architectures that meet specific goals and limitations set by engineers. Rather than spending time and money determining the best solution, the researchers simply gave the machine learning model a problem space (all the different components that go into finding a solution to a problem) and allowed the model to determine the best solution on its own.
AttendSeg can be leveraged for a range of applications, including autonomous vehicles, manufacturing, medical, satellite remote sensing, and mobile applications such as augmented reality. While the technology is still young, it presents potentially revolutionary implications for machine learning.
Understanding Machine Learning
By providing AI with the ability to learn from its experiences without needing explicit programming, machine learning is important to developing the technology. Covering machine learning models, algorithms, and platforms, Machine Learning: Predictive Analysis for Business Decisions, is a five-course program from IEEE.
Connect with an IEEE Content Specialist today to learn more about this program and how to get access to it for your organization.
Interested in the program for yourself? Visit the IEEE Learning Network.
Dickson, Ben. (14 May 2021). New deep learning model brings image segmentation to edge devices. Venture Beat.
Dickson, Ben. (7 May 2021). New deep learning model brings image segmentation to edge devices. TechTalks.