The Device Chronicle interviewed Jacobo Fanjul, an Edge researcher working in the IoT & Digital Platforms Group at Ikerlan. Jacobo explains the paradox of developing and deploying ever more complex machine learning models to constrained devices on the edge.
Ikerlan is one of Europe’s leading research and technology transfer agencies focussed on helping industrial companies and public sector organisations leverage innovations in IoT, Cloud and AI to advance their operations. The agency focuses on 3 core multidisciplinary areas of competence: Electronics, Information and communication technologies; energy and power electronics, and advanced manufacturing.
The organisation is research-led and a cooperative member of the Mondragon corporation and is also connected to the Basque Research & Technology Alliance. By combining high quality research activities with technology transfer and highly trained internal personnel, the agency has become an important partner not just for many of Spain’s leading companies but also for European companies.
In collaboration with partner’s development teams, Jacobo’s applied research and translational work focuses on edge computing and the development of microservices architecture to address use cases involving sometimes constrained edge devices but not exclusively so. Jacobo adds “We do work with Nvidia Jetson hardware in the EU-funded ELASTIC Project in areas such as high-performance computing, smart mobility and predictive maintenance. Jacobo’s group has developed their own streamlined AI and Edge Inference microservices architecture called the Konnekt Box and the Konnekt Cloud ecosystem.
Nvidia Jetson experience
One project where Nvidia Jetson is used is with the municipal tram network of the City of Florence. Ikerlan’s role in this project is to monitor communications and security. The team is developing a non functional requirement tool which retrieves all information about network performance, security rules and audits. The orchestrator has been developed by another project partner – the Barcelona Supercomputing Center. This orchestrator takes the non functional metrics and adapts the computational loads of all the Jetson devices so they can be adapted to the real conditions in the field.
Working with more constrained devices
While Jacobo and his team are thrilled to be able to utilise the advanced capabilities of the Nvidia processors in the Florence project, they must also put their expertise to work with much less powerful hardware. Jacobo says “We also work with clients who want us to perform tiny machine learning models and inference tasks with lower cost devices. Such units have a relatively constrained CPU architecture and we have to figure out how to perform computational loads with CPUs that have limited performance.”
Inference use cases in constrained edge devices
Jacobo provides an example of an inference use case. There is a sensor in a mechanism and its role is to monitor and detect an anomaly or a failure in a piece of equipment. A whole sequence of samples is analysed for a set time period of perhaps 5 seconds of samples to see if any anomaly is present in this signal. The model does not need to be complex, a subset of samples is transmitted to a cloud for training and then the trained model is downloaded to the edge device. This way savings are made on the communication and the inference tasks can be performed at the edge.
Constrained devices connectivity
And often it isn’t just constrained devices that are a challenge on the edge, but also constrained connectivity. This, Jacobo points out, is a challenge in many use cases. “Many of Ikerlan’s private sector clients are still using 3G and GPRS transceivers for network connectivity to their edge devices. Jacobo laments “You do not have the bandwidth at your disposal to perform almost real time or almost real time training and to download heavy inference models from the cloud. The latency would be too high to do this because of the outdated communication technology.” For this reason, the computational work must be done at the edge. Jacobo explains how it works in one of their projects. For development, the maintenance operators retrieve the data for development from devices communicating via the CANbus, which has a throughput limitation. Not all the central information can be transmitted through the bus. Human operators retrieve the data and then the models are trained and downloaded by the Ikerlan team. For future projects, Ikerlan wants to replace human intervention and perform the data pick up with wireless transmission. Jacobo describes what they want to do “We will take one device in the deployment with a wireless interface, then our sensors will send the data to that device. It will act as a gateway to the other devices so the data can be used as a training data set for the training stages or transmitted to the cloud so the models can be trained and downloaded back to the edge.”
3G and GPRS and constrained devices
LTE and Narrowband IoT CAT-M transceivers is a prospective technology. But the reality is that many manufacturers still have hundreds of thousands of 3G GPRS transceivers. Jacobo describes it as a matter of cost where they have already invested in this infrastructure and as a result are “married” to 3G communication for at least 10 years in most cases. The conundrum comes when a novel computer application for a machine learning or AI task is released to production and the client needs this application to work with devices already deployed in the field. It must do it within a very constrained communication channel.
This scenario, Jacobo observes, is the polar opposite to the smartphone market where what you can do with the hardware and the bandwidth available is far greater than what many present mobile applications demand. In IoT and edge use cases, the reverse is often the norm where the hardware and the bandwidth is far less advanced than the software application available to run on them.
IoT comes with cost considerations for constrained devices
Ikerlan works with private sector clients including train manufacturers and elevator companies. Here often the realities of economics means it can be challenging to quickly embrace all that IoT has to offer. If you take the example of an elevator fleet with hundreds of thousands of units in the field. Each of these elevator units may have 7 to 12 devices with IoT connectivity. The product manager and designers have to think about upgrading these components to LTE and CAT-M and new advanced embedded hardware where one component could cost €400. They are naturally cautious in the way they approach these investments despite the promise. The elevator manufacturer may have a door opening mechanism with a certain established commodity cost, and then you say the door needs connectivity because the sensor needs to provide data to another device that is going to predict anomalies. This can be alot for the purchaser to comprehend when they are thinking about total cost management of the elevator unit. “Cost goes up from 5% to 15% dedicated to connected devices and this looks large for the whole build of the elevators.”
Modular edge approach in constrained devices
Jacobo’s team has developed its own edge microservices architecture. This platform has been designed with core components such as a deployment orchestrator and an export service for MQTT reporting. The remainder of the platform is customised for specific client needs. “Ikerlan provides a framework so that all the components can communicate with each other. It is a very lightweight platform based on modularity. The starting point is relatively simple use cases involving two to three tasks. With this approach, new components can be added on demand remotely from the cloud. This is useful, where for example, you are working with a CAN monitoring application to retrieve variables from a CANbus, then a new device is added in the deployment and uses modbus or OPC rather than CANbus. You deploy a new component. If the new component is then no longer needed, it can be easily removed and you get back to the simpler deployment. In another scenario, you want to change cloud platforms and in this case all that needs updating is the export component. In a final scenario, you may want to switch from MQTT to a lightweight M2M protocol and this could be done very easily. Ikerlan offers its Konnekt platform to clients but also works on client projects involving AWS IoT Core and Azure IoT Hub. Jacobo points out “Clients want an edge platform and they want it to work with AWS IoT Core for example”.
OTA software updates and constrained edge devices
Ikerlan has also been thinking through the best approach to OTA software updates to embedded devices. They have moved from a reverse SSH approach (which would require an open port) to an MQTT subscription to tell the devices out in the field to poll the server to download an update. With Mender.io there is no need for an open port to perform the update.
Elastic Project for IoT, AI and Computer Vision
Ikerlan is at the leading edge of the mass transit interface in the Elastic Project. They are serving use cases for predictive maintenance and obstacle detection. They look at the interactions between public transportation such as a tram and private transportation, pedestrians and cyclists. Machine learning models, computer vision and edge devices are used to detect obstacles at a crossroads, to synchronise with traffic lights, to monitor the profile of the track’s condition including anomaly checking.
Functional and non functional paradigms
For Jacobo, there are two paradigms in this project – one functional and the other non functional. One is to manage the computational load distribution or the orchestration. This involves monitoring functional requirements such as task distribution, CPU usage, and to meet the deadlines for all the real time tasks that need to be carried out.
The second non functional paradigm is the management of a reporting tool for monitoring:
- Communications – is the network performing as expected?
- Security monitoring – if several nodes have not been updated, or are not satisfying a set of security rules and requirements, then you would not want to distribute working loads to those potentially vulnerable nodes
- Energy consumption – balancing energy consumption between nodes, if one node consuming a lot of power, you would redistribute the load to other nodes
- Real time requirements – to assess fulfilment of deadlines, and to monitor to ensure computer vision tasks are being performed in time.
These paradigms serve the improved outcomes in predictive maintenance. Companies save money and save lives through a reduction of parts failure, human labor and human error. “You can detect the anomaly and save costs, large breaks and ultimately prevent accidents.”
Interest higher than investment
The interest in edge and IoT is high. But the level of investment is not yet as high as the interest level. For Jacobo, this often comes back to the historical investments that companies have already made in highly constrained edge devices. “Companies want to save a lot of cost with maintenance, use advanced inference models and still keep using their lower specified devices. These devices have a lower limit to what they can deliver when it comes to AI.” Ultimately companies need to see proven ROI so they can allocate larger proportions of the budgets to IoT devices and the advancement and innovation in the field.