For certain applications, it can make sense to move the AI model closer to where the data is being generated versus to generate and ship the data to some other location to get it to the model. This is a really fundamental idea of edge ML. A good example of this might be having audio or visual sensors in a manufacturing production line that are doing quality control or listening or watching the machines for predictive maintenance purposes and rather than sending all of the bulk data back to a central model in the cloud or a data center, the sensor data is processed locally.
It’s not possible however to just move the model from a data center to the edge as the factory floor will not have the same amount and type of computing power and storage. The model quite often has to be both compressed so that it can run on edge devices without sacrificing predictive capability too much for the task at hand.
A second major issue is that it is difficult to ensure that a model will run on a variety of different and rapidly evolving edge devices with different processors, and significant compute and storage constraints. There are now systems in place that will provide automated compiler frameworks to optimize the model to run on a variety of different hardware processor targets, taking that onerous adaptation requirement away from the central model and application development team, and letting them focus on the overall design of the system.
With true Edge ML support, it is now possible to reduce overall development cycle time, simplify the development pipeline, and spend less time managing your development infrastructure across both the center and the edge.