From Embedded Sensors to Advanced Intelligence: Driving Industry 4.0 Innovation with TinyML

Our previous article about Industry 4.0 and the Arduino Pro ecosystem highlighted the importance of embedded sensors for the fourth industrial revolution. Specifically, it outlined how embedded sensors play a key role in transforming machines and automation devices to Cyber Physical Production Systems (CPPS). The latter bridges the gap between physical production processes and advanced IT-based industrial applications such as digital twins. Leveraging CPPS systems, manufacturers and industrial automation solution providers can capture data from the shopfloor and use it to drive optimizations in areas like production schedules, process control, and quality management. Most of these optimizations are data-driven i.e., they are based on the analysis of large volumes of shopfloor data, which are often combined with datasets from enterprise systems (e.g., Enterprise Resource Planning (ERP) systems), business information systems and historian databases. Overall, manufacturing intelligence in the Industry 4.0 era stems from the power of data analytics, which is the reason why data are often presented as the oil of the 21st century.  

Deployment Options for IoT Analytics: From Cloud Analytics to TinyML

While embedded sensors and other Internet of Things (IoT) devices provide the means for collecting data from machinery and other shopfloor devices, production intelligence is based on IoT analytics. IoT analytics are used to derive insights about production operations and manufacturing performance. These insights are used to either improve the worker’s decisions in human-in-the-loop processes or to drive optimal automation and actuation functions. 

For most industrial use cases, IoT analytics functions entail statistical data processing and employ Machine Learning (ML) techniques. ML is the primary option when it comes to mining industrial data and extracting knowledge about production processes. In recent years, Deep Learning (DL) techniques i.e., ML models based on deep neural networks, are widely used. This is because they outperform traditional ML when trained with large amounts of data. 

Nowadays, manufacturing enterprises are provided with different options for deploying IoT analytics. For instance, there is a cloud analytics option, which benefits from the vast amounts of data that can be collected and managed in the cloud. Cloud-based IoT analytics leverage large numbers of data points for training DL algorithms, which yields accurate models. Accuracy is very important for many industrial use cases like Remaining Useful Life (RUL) calculation in asset management and predictive maintenance. On the other hand, it is also possible to run analytics close to the field i.e., at the edge of the network. Edge analytics functions are deployed within an embedded device or even at an edge computing cluster within the factory’s local area network. The deployment of IoT analytics at the edge is more appropriate for real-time use cases, where low latency is a primary concern. For instance, in the case of quality management and predictive maintenance, edge analytics functions can provide way faster detection of defective products and of machine malfunctions. Furthermore, edge analytics deployments are much more power-efficient than cloud analytics. Also, they provide increased security and data protection, as data need not travel outside the factory’s local area network.

During the last couple of years, TinyML enables industrial organizations to fully leverage the edge analytics paradigm. TinyML is a novel machine learning paradigm, which entails the execution of ML models (including DL models) within CPU and memory-constrained devices such as microcontrollers.  TinyML pushes the benefits of edge computing to their limits: It is fast, real-time, more power-efficient, and more privacy-friendly than any other form of edge analytics. The execution of DL enables instant identification of events like machine failures and obviates the need for continuous reliable connectivity that cannot always be taken for granted. Most importantly, TinyML enables industrial organizations to take full advantage of their data, as they can process data at the source of their generation (i.e. the embedded sensor or device) rather than requiring complex data collection processes. In this way, TinyML provides distinct benefits for most of the Industry 4.0 use cases that were presented in our earlier article.

Figure 1: TinyML pushes the benefits of edge computing to their limits: It is fast, real-time, more power-efficient, and more privacy friendly than any other form of edge analytics. Image credit: Carbon Robotics.

Building TinyML Applications

The development of TinyML applications requires expertise in both embedded systems programming and data science. Specifically, a TinyML application involves the development of an ML pipeline and its deployment in an embedded device. 

The development of such ML pipelines can be based on standard machine learning tools such as Jupyter Notebooks using popular ML libraries like PyTorch and Keras. Following the development of a pipeline, developers can use specialized TinyML and embedded ML libraries like TensorFlow Lite for microcontrollers to shrink the size of the ML model and to make it appropriate for deployment in a memory-constrained device. Overall, the end-to-end process of developing and deploying a TinyML application involves the following basic steps:

1. Getting or Producing a Dataset: This is the dataset that will be used for training the TinyML model. In this direction, data from sensors or production logs can be used. 

2. Train an ML or DL Model: In this step, a ML model is trained using standard tools and libraries like a Jupyter Notebook and Python packages like TensorFlow and NumPy.  The work entails some Exploratory Data Analysis (EDA) steps towards understanding the data, identifying a proper ML model, and preparing the data for training the model.

3. Evaluate the Model’s Performance: In this direction, the trained model must be used for predictions. Likewise, various error metrics (e.g., mean absolute error values) must be calculated. Depending on the achieved performance, the TinyML engineer may have to improve the model and avoid the pitfalls of overfitting on the data.  Different models might have to be tested prior to finding the best one.

4. Make the Model Appropriate to Run on an Embedded Device: As already outlined, TensorFlow Lite provides the means for converting the trained model to a form suitable for deployment on an embedded device. To this end, TensorFlow Lite provides a “converter” library that turns a model into a space-efficient format for use on memory limited devices. It applies optimizations that reduce the model size and make it run faster on small devices. Furthermore, an “interpreter” library runs the converted model using the most efficient operations for a given device. As part of this step, a C/C++ file (sketch) is produced to enable deployment on the device.

5. On-device Inference and Binary Development: This step involves the C/C++ and embedded systems development part, which produces a binary application for on-device inference. The development can be tested on a development machine, yet it has ultimately to be ported on the microcontroller.  

6. Deploying the Binary to a Microcontroller: As part of this step, the binary is deployed on a board or microcontroller. Following the deployment, the microcontroller becomes able to analyse data and derive real-time insights.

Figure 2: A schematic for building a Google Assistant using tinyML. Image credit: Arduino

Leveraging AutoML for Faster Development with Arduino Pro

Nowadays, it is possible to leverage Automatic Machine Learning (AutoML) to develop TinyML on various boards, including Arduino boards. This is thanks to emerging platforms such as Edge Impulse, Qeexo and SensiML, among others, which provide AutoML tools and developers’ resources for embedded Machine Learning development.  Arduino is collaborating with several such platforms as part of their strategy to make complex technologies open and simple to use by anyone, which unleashes the full innovation potential of Arduino boards.

Within these platform ecosystems, it is possible to collect real-world sensor data, train ML models on this data using a cloud-based infrastructure, and ultimately deploy the model back to an Arduino device. It is also possible to integrate ML models with Arduino sketches based on simple function call. By automating AutoML pipelines it is straightforward to (re)develop and (re)deploy models as needed to improve the performance of the model and to meet a variety of complex requirements.

In more practical terms, the collaboration between Arduino and ML platforms enables thousands of developers to build applications that embed intelligence in smart devices such as applications that recognize spoken keywords, gestures, or even people and animals. Even with limited knowledge or time, building simple applications that control IoT devices via natural language or gestures is relatively straightforward for developers who are familiar with Arduino boards and can engage with these emerging platforms services. With further effort and dedication, it is possible to develop more sophisticated applications like edge computing systems for industrial control in settings such as manufacturing lines and precision agriculture.

With the goal of enabling business transformation, Arduino has recently introduced its new Arduino Pro ecosystem of industrial-grade products and services. These products and services support the full development, production and operation lifecycle from Hardware and Firmware to Low Code, Clouds, and Mobile Apps. Hence, the Pro ecosystem empowers thousands of developers not only to jump into basic Industry 4.0 development, but also to employ advanced analytics at the edge and far-edge of the shopfloor. 

Big opportunity at every scale

The Arduino ecosystem provides ample opportunities and excellent support for developing and deploying TinyML applications, including applications for both education purposes and for enterprise scale deployments. Specifically, Arduino offers boards that facilitate TinyML development and deployment, along with a rich set of tools and documentation. As a prominent example, the Arduino Nano 33 BLE Sense board is one of the most popular boards for TinyML and edge analytics development.  The board comes with a well-known form factor and various embedded sensors. The latter include a 9-axis inertial sensor that makes the board ideal for wearable devices, as well as humidity and temperature sensors that enable the acquisition of accurate measurements about the environmental conditions. As another example, Arduino’s Portenta H7 board includes two asymmetric cores, which enables simultaneously runs of high level code such as protocol stacks, machine learning or even interpreted languages (e.g., MicroPython or JavaScript) along with low-level real time tasks. Furthermore, the Arduino IDE (Integrated Development Environment) provides the means for customizing the embedded versions of machine learning pipelines and deploying them in Arduino boards. The IDE makes it easy to write code and upload it to any board. 

In a Nutshell

Most Industry 4.0 applications employ some form of IoT analytics over data collected from embedded sensors. Advanced IoT analytics leverage machine learning and artificial intelligence to extract unique insights about production processes and their optimization. Contrary to popular belief, ML and AI models need not always run over powerful cloud computing infrastructures and related High Performance Computing (HPC) services. Nowadays, it is possible to execute deep neural networks over tiny memory-limited devices like microcontrollers, which opens unprecedented opportunities for pervasive intelligence within industrial sites. The Arduino ecosystem offers developers the systems and resources they need to ride the wave of Industry 4.0 and TinyML developments. The Arduino boards and the Arduino IDE lower the barriers for thousands of embedded systems developers to engage with TinyML and IoT analytics for industrial intelligence.