From Cornell University 22/12/23
Cornell University researchers have released a new, open-source platform called Cascade that can run artificial intelligence models in a way that slashes expenses and energy costs while dramatically improving performance.
Cascade is designed for settings like smart traffic intersections, medical diagnostics, equipment servicing using augmented reality, digital agriculture, smart power grids and automatic product inspection during manufacturing – situations where AI models must react within a fraction of a second.
With the rise of AI, many companies are eager to leverage new capabilities but worried about the associated computing costs and the risks of sharing private data with AI companies or sending sensitive information into the cloud.
Also, today’s AI models are slow, limiting their use in settings where data must be transferred back and forth or the model is controlling an automated system.
A team led by Ken Birman, professor of computer science, combined several innovations to address these concerns.
Birman partnered with Weijia Song, a senior research associate, to develop an edge computing system they named Cascade.
Edge computing is an approach that places the computation and data storage closer to the sources of data, protecting sensitive information.
Song’s “zero copy” edge computing design minimizes data movement.
The AI models don’t have to wait to fetch data when reacting to an event, which enables faster responses, the researchers said.
“Cascade enables users to put machine learning and data fusion really close to the edge of the internet, so artificially intelligent actions can occur instantly,” Birman said.
“This contrasts with standard cloud computing approaches, where the frequent movement of data from machine to machine forces those same AIs to wait, resulting in long delays perceptible to the user.”
Cascade is giving impressive results, with most programs running two to 10 times faster than cloud-based applications, and some computer vision tasks speeding up by factors of 20 or more.
Larger AI models see the most benefit.
Moreover, the approach is easy to use: “Cascade often requires no changes at all to the AI software,” Birman said.
With the new open-source release, Birman’s group hopes other researchers will explore possible uses for Cascade, making AI applications more widely accessible.
