Imagine being a passenger in a self-driving car as the vehicle starts veering off the road. It’s not a faulty sensor causing the dangerous situation — it’s a cyberattack. Hackers can access the deep learning (DL) neural networks at the heart of the vehicle’s computer system, compromising the safety of its passengers, as well as other drivers and pedestrians.&nbsp;Stopping such cyberattacks requires understanding them first, but this can be challenging. Finding a computing system’s exact deep neural network has many roadblocks. They are often proprietary and, therefore, inaccessible to investigators without considerable legal intervention. Another common problem is that they are updated frequently, making it difficult for investigating researchers to access the most current network iteration. But a new tool from Georgia Tech could unlock the mysterious malware on myriad neural networks in everything from self-driving cars to the IMDB entertainment database. AI Psychiatry (AiP) is a postmortem cybersecurity forensic tool that uses artificial intelligence to recover the exact models a compromised machine runs on and discover where the fatal error occurred.“We trust self-driving cars with our lives and ChatGPT with our careers, but when those systems fail, how are we going to investigate them?” said <a href="https://saltaformaggio.ece.gatech.edu/">Brendan Saltaformaggio</a>, an associate professor with joint appointments in the School of <a href="https://scp.cc.gatech.edu/">&nbsp;Cybersecurity and Privacy</a> and the School of <a href="https://www.ece.gatech.edu/">Electrical and Computer Engineering</a> (ECE).AiP can recover the original DL model on both the local network’s memory and the graphics processing unit that trains the network. It can accomplish this without any specific knowledge of the model’s framework, platform, or version. Instead, it recreates the model using what&nbsp;<a>Saltaformaggio</a> refers to as “clues,” or common components in all neural networks. These include weights, biases, shapes, and layers from the model’s memory image — a frozen set of the bits and bytes operating when the model is running normally. The memory image is crucial because it enables AiP to compare it with the model post-attack.“These models often refine their information as they go, based on their current environment, so an attack might happen as a result of an attacker poisoning the information a particular model is learning,” said David Oygenblik, an ECE Ph.D. student. “We determined that a memory image would capture all those changes that occur during a runtime.”\Once the model is recovered, AiP can run it on another device, letting investigators test it thoroughly to determine where the flaws lie. AiP has been tested with different versions of both popular machine learning frameworks (TensorFlow and PyTorch) and datasets (CIFAR-10, LISA, and IMDB). It successfully recovered and rehosted 30 models with 100% accuracy.&nbsp;“Before our research, you couldn't go to the cyber ‘crime scene’ and find clues because there was no technique available to do that,” Saltaformaggio said. “That's what we are pioneering in the cyber forensics lab right now — techniques to get that evidence out of a crime scene.”Tools like AiP will allow cyber investigators to see the whole picture immediately. Solving cybercrimes can help prevent future ones, from safeguarding a user’s data to keeping a car on the road.&nbsp;

New tool AI Psychiatry recovers compromised deep-learning models so researchers can understand what went wrong.

The Sherlock Holmes of AI

Even if you’re not very familiar with deep learning, you’ve probably heard about it and how it can, among other things, help automate the driving experience, increase manufacturing efficiency and change the consumer shopping experience. Deep learning is the latest evolution of artificial intelligence (AI) and machine learning. Both machine learning and deep learning are subsets of AI. While traditional machine learning algorithms need to be specifically programmed by people with domain-level expertise, deep learning algorithms utilize neural networks that can be trained by feeding them data. The same network can be used to solve very different problems by training it with different data, removing the requirement of having specific expertise in the problem that needs to be solved. Because of this, deep learning is considered a foundational technology that has the potential to make an impact in many industries.For those new to deep learning, think of it as a technology that can classify things. Many real-world problems can be broken down into a classification problem, for example:<ul><li>Autonomous and semi-autonomous vehicles need to be able to classify roads, other vehicles, people and road signs as shown in Figure 1.</li><li>Smart factories need to be able to classify a product as defective</li><li>Automated retail outlets need to classify if a consumer is removing a product from a store for purchase or just looking at it</li></ul><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1722330537960-Screenshot+2024-07-30+110736.png" style="width: 100%px;" class="fr-fic fr-dib">Figure 1. Classification Example of an Automotive Use Case. Every Object in the Scene Is Determined to Be One of the Five Different Classes.There are two main parts to deep learning: training and inference, as shown in Figure 2. During training, a model of a neural network that has been created to solve a classification problem is fed huge amounts of labeled data, from which it learns how to classify things. In the example scenario shown in Figure 1 of an autonomous vehicle, the model would be fed images of pedestrians, road signs, cars and roads with each of these objects correctly labeled. Real-time performance or power is not an issue during training, so it usually occurs on a desktop or cloud platform.During inference, the trained network that has been deployed into its end application makes a classification decision, such as determining whether or not a part on an assembly line is defective.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1722330524914-Screenshot+2024-07-30+110753.png" style="width: 100%px;" class="fr-fic fr-dib">Figure 2. Typical Development Flow of a Deep Learning Network.For a combination of reasons – including reliability, low latency, privacy, power and cost – it is beneficial for inference to run close to where sensors gather data; that is, at the “edge” of the network. Advanced sensors that can handle some amount of AI are capable of running a simple inference in the sensor itself. For a more complex inference, a separate edge processor is needed.TI Sitara™ processors are great devices for edge processing. With a scalable portfolio integrating Arm® Cortex®-A cores with flexible peripherals, Sitara processors provide industrial-grade solutions with an extensive mix of processing, integration and power efficiency. With TI Deep Learning (TIDL) software, Sitara AM57x processors leverage hardware acceleration to improve the performance of machine learning and deep learning inference at the edge.TIDL software is a set of open-source Linux software packages included on the processor software development kit (SDK) and tools that enable the hardware-accelerated offloading of deep learning inference in an AM57x device. It’s possible to accelerate inference across embedded vision engine (EVE) subsystems, C66x digital signal processor cores or a combination of both, using a set of application programming interfaces built with Open Computing Language (OpenCL).For those who already have an existing neural network framework, TIDL software bridges the gap between the popular Caffe and TensorFlow frameworks with accelerated hardware on the AM57x through an import tool, as shown in Figure 3. To maximize efficiency on the hardware, or for those who don’t have an existing neural network framework, Caffe Jacinto, a TI-developed framework forked from NVIDIA/Caffe and designed for the embedded space, can perform the training part of deep learning.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1722330564360-Screenshot+2024-07-30+110804.png" style="width: 100%px;" class="fr-fic fr-dib">Figure 3. Texas Instruments Deep Learning (TIDL) Software Development Flow.Note that while TI has developed a framework that can be used to train a network model for an embedded application, the training itself still takes place on a desktop or cloud platform. It is the inference that runs on the TI embedded processor.With free software as part of TI’s Processor SDK, training videos and a deep learning TI reference design, whether you’re new to deep learning or a seasoned expert, TI provides the building blocks to get you started quickly and easily.Additional Resources:• <a href="http://www.ti.com/tool/sitara-machine-learning" target="_blank" rel="noopener noreferrer">Visit the TI Sitara Machine Learning page.</a>• <a href="https://e2e.ti.com/blogs_/b/behind_the_wheel/archive/2018/02/08/ai-in-automotive-practical-deep-learning?" target="_blank" rel="noopener noreferrer">Read more about machine learning</a>

Even if you’re not very familiar with deep learning, you’ve probably heard about it and how it can, among
other things, help automate the driving experience, increase manufacturing efficiency and change the consumer
shopping experience.

Bringing the Next Evolution of Machine Learning to the Edge

Neural networks have made a seismic impact on how engineers design controllers for robots, catalyzing more adaptive and efficient machines. Still, these brain-like machine-learning systems are a double-edged sword: Their complexity makes them powerful, but it also makes it difficult to guarantee that a robot powered by a neural network will safely accomplish its task. The traditional way to verify safety and stability is through techniques called Lyapunov functions. If you can find a Lyapunov function whose value consistently decreases, then you can know that unsafe or unstable situations associated with higher values will never happen. For robots controlled by neural networks, though, prior approaches for verifying Lyapunov conditions didn’t scale well to complex machines.Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and elsewhere have now developed new techniques that rigorously certify Lyapunov calculations in more elaborate systems. Their algorithm efficiently searches for and verifies a Lyapunov function, providing a stability guarantee for the system. This approach could potentially enable safer deployment of robots and autonomous vehicles, including aircraft and spacecraft.To outperform previous algorithms, the researchers found a frugal shortcut to the training and verification process. They generated cheaper counterexamples — for example, adversarial data from sensors that could’ve thrown off the controller — and then optimized the robotic system to account for them. Understanding these edge cases helped machines learn how to handle challenging circumstances, which enabled them to operate safely in a wider range of conditions than previously possible. Then, they developed a novel verification formulation that enables the use of a scalable neural network verifier, α,β-CROWN, to provide rigorous worst-case scenario guarantees beyond the counterexamples.“We’ve seen some impressive empirical performances in AI-controlled machines like humanoids and robotic dogs, but these AI controllers lack the formal guarantees that are crucial for safety-critical systems,” says Lujie Yang, MIT electrical engineering and computer science (EECS) PhD student and CSAIL affiliate who is a co-lead author of a new paper on the project alongside Toyota Research Institute researcher Hongkai Dai SM ’12, PhD ’16. “Our work bridges the gap between that level of performance from neural network controllers and the safety guarantees needed to deploy more complex neural network controllers in the real world,” notes Yang.For a digital demonstration, the team simulated how a quadrotor drone with lidar sensors would stabilize in a two-dimensional environment. Their algorithm successfully guided the drone to a stable hover position, using only the limited environmental information provided by the lidar sensors. In two other experiments, their approach enabled the stable operation of two simulated robotic systems over a wider range of conditions: an inverted pendulum and a path-tracking vehicle. These experiments, though modest, are relatively more complex than what the neural network verification community could have done before, especially because they included sensor models. “Unlike common machine learning problems, the rigorous use of neural networks as Lyapunov functions requires solving hard global optimization problems, and thus scalability is the key bottleneck,” says Sicun Gao, associate professor of computer science and engineering at the University of California at San Diego, who wasn’t involved in this work. “The current work makes an important contribution by developing algorithmic approaches that are much better tailored to the particular use of neural networks as Lyapunov functions in control problems. It achieves impressive improvement in scalability and the quality of solutions over existing approaches. The work opens up exciting directions for further development of optimization algorithms for neural Lyapunov methods and the rigorous use of deep learning in control and robotics in general.”Yang and her colleagues’ stability approach has potential wide-ranging applications where guaranteeing safety is crucial. It could help ensure a smoother ride for autonomous vehicles, like aircraft and spacecraft. Likewise, a drone delivering items or mapping out different terrains could benefit from such safety guarantees.The techniques developed here are very general and aren’t just specific to robotics; the same techniques could potentially assist with other applications, such as biomedicine and industrial processing, in the future. While the technique is an upgrade from prior works in terms of scalability, the researchers are exploring how it can perform better in systems with higher dimensions. They’d also like to account for data beyond lidar readings, like images and point clouds.As a future research direction, the team would like to provide the same stability guarantees for systems that are in uncertain environments and subject to disturbances. For instance, if a drone faces a strong gust of wind, Yang and her colleagues want to ensure it’ll still fly steadily and complete the desired task.&nbsp; Also, they intend to apply their method to optimization problems, where the goal would be to minimize the time and distance a robot needs to complete a task while remaining steady. They plan to extend their technique to humanoids and other real-world machines, where a robot needs to stay stable while making contact with its surroundings. Russ Tedrake, the Toyota Professor of EECS, Aeronautics and Astronautics, and Mechanical Engineering at MIT, vice president of robotics research at TRI, and CSAIL member, is a senior author of this research. The paper also credits University of California at Los Angeles PhD student Zhouxing Shi and associate professor Cho-Jui Hsieh, as well as University of Illinois Urbana-Champaign assistant professor Huan Zhang. Their work was supported, in part, by Amazon, the National Science Foundation, the Office of Naval Research, and the AI2050 program at Schmidt Sciences. The researchers’ paper will be presented at the 2024 International Conference on Machine Learning.

MIT CSAIL researchers helped design a new technique that can guarantee the stability of robots controlled by neural networks. This development could eventually lead to safer autonomous vehicles and industrial robots. Image: Alex Shipps/MIT CSAIL

Neural network controllers provide complex robots with stability guarantees, paving the way for the safer deployment of autonomous vehicles and industrial machines.

Creating and verifying stable AI-controlled systems in a rigorous and flexible way

The world of artificial intelligence (AI) has recently seen significant advancements in generative models, a type of machine-learning algorithms that “learn” patterns from set of data in order to generate new, similar sets of data. Generative models are often used for things like drawing images and natural language generation – a famous example are the models used to develop chatGPT.Generative models have had remarkable success in various applications, from image and video generation to composing music and to language modeling. The problem is that we are lacking in theory, when it comes to the capabilities and limitations of generative models; understandably, this gap can seriously affect how we develop and use them down the line.One of the main challenges has been the ability to effectively pick samples from complicated data patterns, especially given the limitations of traditional methods when dealing with the kind of high-dimensional and complex data commonly encountered in modern AI applications.Now, a team of scientists led by Florent Krzakala and Lenka Zdeborová at EPFL has investigated the efficiency of modern neural network-based generative models. The study, now published in&nbsp;PNAS, compares these contemporary methods against traditional sampling techniques, focusing on a specific class of probability distributions related to spin glasses and statistical inference problems.The researchers analyzed generative models that use neural networks in unique ways to learn data distributions and generate new data instances that mimic the original data.The team looked at flow-based generative models, which learn from a relatively simple distribution of data and “flow” to a more complex one; diffusion-based models, which remove noise from data; and generative autoregressive neural networks, which generate sequential data by predicting each new piece based on the previously generated ones.The researchers employed a theoretical framework to analyze the performance of the models in sampling from known probability distributions. This involved mapping the sampling process of these neural network methods to a Bayes optimal denoising problem – essentially, they compared how each model generates data by likening it to a problem of removing noise from information.The scientists drew inspiration from the complex world of spin glasses, materials with intriguing magnetic behavior, to analyze modern data generation techniques. This allowed them to explore how neural network-based generative models navigate the intricate landscapes of data.The approach allowed them to study the nuanced capabilities and limitations of the generative models against more traditional algorithms like Monte Carlo Markov Chains (algorithms used to generate samples from complex probability distributions) and Langevin Dynamics (a technique for sampling from complex distributions by simulating the motion of particles under thermal fluctuations).The study revealed that modern diffusion-based methods may face challenges in sampling due to a first-order phase transition in the algorithm’s denoising path. What this means is that they can run into problems because of sudden change in how they remove noise from the data they're working with. Despite identifying regions where traditional methods outperform, the research also highlighted scenarios where neural network-based models exhibit superior efficiency.This nuanced understanding offers a balanced perspective on the strengths and limitations of both traditional and contemporary sampling methods. The research is a guide to more robust and efficient generative models in AI; by providing a clearer theoretical foundation, it can help develop next-generation neural networks capable of handling complex data generation tasks with unprecedented efficiency and accuracy.<hr>ReferencesDavide Ghio, Yatin Dandi, Florent Krzakala, Lenka Zdeborovà. Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspective. PNAS 24 June 2024. DOI:&nbsp;<a href="https://doi.org/10.1073/pnas.2311810121" target="_blank" rel="noopener">10.1073/pnas.2311810121</a>

Researchers at EPFL have made a breakthrough in understanding how neural network-based generative models perform against traditional data sampling techniques in complex systems, unveiling both challenges and opportunities for AI's future in data generation.

Navigating the labyrinth: How AI tackles complex data sampling

Our brains are marvels of efficiency, capable of doing complex tasks while consuming minimal power. But replicating that efficiency with traditional computers has proven difficult. This is where neuromorphic computing comes in. Inspired by the brain's architecture, neuromorphic chips are poised to revolutionize AI, especially at the edge where low-power processing is crucial.This episode of the <a href="https://brainchip.com/" target="_blank" rel="noopener noreferrer">BrainChip</a> podcast features a conversation between BrainChip CEO Sean Hehir and Dr. Jason K. Eshraghian, a leading expert in neuromorphic computing. They explore the advantages of this new technology over traditional AI methods and consider its potential to transform the future of Edge AI.<h2 dir="ltr">Neuromorphic Approach to AI</h2>When asked about the core principles of neuromorphic computing, Dr. Jason K. Eshraghian highlighted some key differences from traditional AI architectures:<ul><li dir="ltr">Brain-Inspired design: Neuromorphic computing takes inspiration from the human brain," explained Dr. Eshraghian. "Unlike CPUs and GPUs that separate processing and memory, neuromorphic chips integrate both functions, mimicking the brain's structure for improved efficiency.”</li><li dir="ltr">Event-driven computation:&nbsp;The human brain doesn't waste energy on constantly processing information. Instead, it relies on a system of "spikes" to transmit signals when relevant activity occurs. Neuromorphic chips emulate this approach, only processing data when needed, further reducing power consumption.</li><li dir="ltr">Sparse representations: The human brain is not uniformly active. Similarly, neuromorphic chips can exploit the inherent sparsity in AI models, focusing on the relevant parts of the data and ignoring the rest. This allows for more efficient use of memory and bandwidth compared to dense representations used in conventional neural networks.</li></ul><h2 dir="ltr">Neuromorphic Chips and Edge AI</h2>The interview shifted gears when the topic of edge AI came up. Dr. Eshraghian expressed his enthusiasm for how neuromorphic computing aligns perfectly with this growing field:"The advantages of neuromorphic chips become particularly compelling when applied to edge AI applications,"&nbsp;Dr. Eshraghian explained. "By ditching the traditional separation of computation and memory, neuromorphic chips achieve significant efficiency gains. This, when combined with their ability only to process data when necessary, makes them ideal for powering intelligent systems directly on devices, closer to the source of the data."He elaborated on the benefits of edge AI:<ul><li dir="ltr">Enabling AI at the edge: The energy efficiency and low latency of neuromorphic architectures make them well-suited for powering AI models directly on edge devices, rather than relying on cloud-based processing. This allows for real-time inference and decision-making without the need to transmit data to remote servers.</li><li dir="ltr">Widespread deployment:&nbsp;While traditional neural networks running on GPUs can struggle to handle massive numbers of concurrent requests for language models, neuromorphic chips like BrainChip's Akida are designed to process many parallel requests efficiently. This makes them a better fit for large-scale edge deployments.</li><li dir="ltr">Sparse event-driven models: The sparse, event-driven nature of neuromorphic computing aligns well with language models like BrainChip's TENN. TENN also uses sparse representations and only processes data when necessary. This synergy allows for highly efficient edge AI systems that can handle real-world workloads.</li></ul><h2 dir="ltr">BrainChip's Akida and TENN: Hardware Solution Aligned with Neuromorphic Principles</h2>The podcast interview discussed BrainChip's Akida, a neuromorphic chip designed with the principles discussed earlier in mind. As Dr. Eshraghian emphasized, the event-driven nature of Akida aligns perfectly with his vision for efficient neuromorphic computing.Akida's architecture departs from traditional approaches by relying on an event-based processing system. This means the chip only activates specific neurons when necessary, mimicking the brain's spiking mechanism. This significantly reduces power consumption compared to constantly processing data streams.But BrainChip doesn't stop there. The introduction of TENN adds another layer of efficiency. <a href="https://brainchip.com/tenns-a-new-approach-to-streaming-and-sequential-data/" target="_blank">TENN</a> is a state-space model, a type of AI architecture that offers several advantages:<ul><li dir="ltr">Lightweight and efficient: TENN models require less memory and computational resources compared to traditional deep learning models. This is particularly beneficial for resource-constrained edge devices.</li><li dir="ltr">Fast processing:&nbsp;TENN models can be processed quickly on neuromorphic chips like Akida, enabling real-time decision-making at the edge.</li><li dir="ltr">Compatibility with Akida:&nbsp;The design of TENN makes it well-suited for implementation on Akida's architecture, further enhancing the efficiency of the entire system.</li></ul>By combining Akida's neuromorphic processing power with the lightweight nature of TENN, BrainChip offers a compelling solution for running powerful AI algorithms on edge devices.&nbsp;The potential of this approach goes beyond just edge AI. As Dr. Jason K. Eshraghian mentions in the podcast, TENN models can also be used in conjunction with large-scale cloud systems. This creates an interesting combination where lightweight models handle tasks at the edge while complex computations can be offloaded to the cloud when necessary.<h2 dir="ltr">Conclusion: A Look Ahead</h2>The conversation with Dr. Eshraghian sheds light on the exciting potential of neuromorphic computing. It's not about creating faster, more powerful AI; it's about achieving efficiency and unlocking new possibilities at the edge of networks. While neuromorphic computing is still evolving, it holds the promise to transform various fields, from industrial automation to personal devices.

In this podcast, Sean Hehir, CEO of BrainChip, chats with Dr. Jason K. Eshraghian about neuromorphic computing benefits over traditional AI and its potential to revolutionize the future of computing. Listen in to learn how this emerging technology is shaping the world of AI.


BrainChip Podcast: Neuromorphic Computing Shaping the Future of AI With Dr. Jason K. Eshraghian

In this episode, we discuss research coming out of the Karlsruhe Institute of Technology to detect the emotions of tennis players better than humans can!

Podcast: Catching Feelings with 69 percent Accuracy: AI Reads Emotions in Sports

This article was discussed in our <a href="https://www.wevolver.com/profile/the.next.byte" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/5bf51127-82c4-4165-8792-9944842f1dbc?dark=false" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe> The full article will continue below.<hr>Researchers from the Karlsruhe Institute of Technology (KIT) and the University of Duisburg Essen have managed to use computer-aided neural networks to precisely identify affective states based on the body language of tennis players during the game. For the first time, a model based on artificial intelligence (AI) was trained with data from real competition scenes. The study, published in the journal Knowledge-Based Systems, shows that AI can evaluate body language and emotions with similar accuracy to a human, but it also points to ethical aspects. ( <a href="https://www.sciencedirect.com/science/article/pii/S0950705124004908?via%3Dihub" target="_blank">DOI: 10.1016/j.knosys.2024.111856</a> )For the study “Detection of emotional states from the expressive behavior of tennis players using convolutional neural networks,” researchers from sports science, software development and computer science at KIT and the University of Duisburg Essen have developed a special AI model. Using computer-based pattern recognition programs, they analyzed video recordings of tennis players during real competitions.&nbsp;<h2>68.9 percent hit rate</h2>“Our model detects affective states with an accuracy of up to 68.9 percent. This is comparable to, and in some cases even superior to, the assessment by human observers as well as to previous automated methods,” explains study leader Professor Darko Jekauc from the Institute for Sport and Sports Science (IfSS) at KIT.&nbsp;An important unique feature of the study is that the project team did not train the AI system using simulated or posed situations, but rather using real scenes. To do this, the researchers recorded video sequences of 15 tennis players in a fixed setting. The focus was on body language when a point was lost or gained. The videos showed players with so-called cues, i.e. with cues such as a lowered head, arms thrown in the air in celebration, a hanging racket or differences in walking speed, which could be used to identify the players' affective state.&nbsp;Fed with this data, the AI learned to assign the body language signals to different affective reactions and to recognize whether a point was won (positive body language) or lost (negative body language). "Training in natural contexts represents a significant advance in recognizing real emotional states and enabling predictions in real-world scenarios," explains Jekauc.<h2>Humans and machines recognize negative emotions better than positive ones</h2>The research not only shows that AI algorithms could outperform human observers in emotion recognition in the future. The evaluation revealed another interesting aspect: both AI and humans are better at recognizing negative emotions. “This may be because they are easier to identify due to their clearer expressions,” explains Jekauc. “Psychological theories suggest that humans are evolutionarily more attuned to perceiving negative emotional expressions. For example, because it is crucial for the social context to defuse conflicts in a timely manner.”<h2>Before use: Clarification of ethical aspects necessary</h2>The study sees numerous areas of application for realistic emotion recognition in sport, from optimizing training methods, team dynamics and improving performance to preventing burnout. In addition, other areas could benefit from reliable early detection of emotional states, such as healthcare, educational institutions, customer service and automotive safety.&nbsp;“Although this technology offers significant benefits in the future, the potential risks associated with it must be taken into account, especially with regard to privacy and data misuse,” emphasizes Jekauc. “Our study was strictly based on existing ethical guidelines and data protection regulations. Even with a view to future applications of such technology in practice, it is essential to clarify ethical and legal questions in advance.”&nbsp;<hr><h3 id="isPasted">Original publication</h3>Darko Jekauc, Diana Burkart, Julian Fritsch, Marc Hesenius, Ole Meyer, Saquib Sarfraz, Rainer Schuhehagen: Recognizing affective states from the expressive behavior of tennis players using convolutional neural networks. Knowledge-Based Systems, Vol. 295, 2024. <a href="https://doi.org/10.1016/j.knosys.2024.111856" target="_blank">DOI: 10.1016/j.knosys.2024.111856</a>

KIT researchers have developed an AI model that can use body language to recognize emotions in tennis players. (Photo: PantherMedia / Wavebreakmedia ltd)

Researchers at KIT have developed a model for emotion analysis that can determine affective states with similar accuracy to a human

AI recognizes emotions in real sports situations

<h1 dir="ltr" id="isPasted">Introduction</h1>In an era where rapid technological advancements continuously reshape how industries operate, edge artificial intelligence (AI) has emerged as a critical driver of innovation and efficiency. Edge AI brings computation and data storage closer to the locations where it's needed, accelerating response times while reducing bandwidth use. This technology is particularly transformative in industrial and commercial applications, where real-time data processing can significantly enhance operational efficiency and decision-making.Enter NVIDIA Train Adapt Optimize (TAO) toolkit—an open-source suite designed to simplify and accelerate the creation of high-quality, customized, and enterprise-ready AI models. This toolkit democratizes access to state-of-the-art AI capabilities and streamlines the development process, enabling rapid deployment across a range of devices from cloud infrastructures to edge units.&nbsp;When integrated with Edge Impulse, a leading development platform for edge AI, NVIDIA TAO becomes part of a powerful ecosystem that enhances the capabilities of developers and engineers to bring AI models to production faster. NVIDIA TAO and Edge Impulse represent a formidable combination, poised to accelerate the adoption of edge AI technologies across various sectors by offering tools that reduce complexity and enhance productivity.This article explores how the NVIDIA TAO toolkit, along with Edge Impulse, facilitates the development and deployment of AI models on edge devices, illustrating the tangible benefits and real-world applications of this integration. By harnessing these advanced tools, developers can speed up the AI development lifecycle and achieve greater scalability in their AI projects.<iframe width="540" height="600" src="https://c8056756.sibforms.com/serve/MUIFAMyA6BGksOUPEokHuBnEmZRHhYgJ4RXb4BV0gvzN98ymNIvQdTcBFQnRVfviT6yZSFW_R2g1hENBu7p7Afnyv9Kt0unAAbfFU5mMQhBwI-8hQnKf2r1mmRKhUgDLmtk8bpVSZ7kZGpleCPqtYRv8_kkvODL7F4neGLNGgdpFDm9pPzoYpwbQkc0BU5GVOFoq8CSmD0DDvDjG" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe> <h2 dir="ltr">The NVIDIA TAO Toolkit</h2>NVIDIA TAO Toolkit is a great tool for developers aiming to harness the power of AI without the complexity typically associated with such endeavors. As an open-source solution, it democratizes advanced technology, allowing a broad range of developers—from startups to large enterprises—to create customized, high-performance AI models that are both scalable and adaptable.<h3 dir="ltr">Overview of NVIDIA TAO</h3>At its core, the NVIDIA TAO Toolkit facilitates AI model development by leveraging NVIDIA's extensive expertise in AI and deep learning. Designed to streamline the process of model training and optimization, NVIDIA TAO empowers developers to build sophisticated AI models without deep specialization in AI technologies. Its open-source nature ensures that it is continually refined by contributions from the global development community, enhancing its capabilities and accessibility.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzE1MjMyMTA3NTEyLWltYWdlMi5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 100%px;" class="fr-fic fr-dib" alt="nvidia-tao-workflow">Fig. 1: Working with Nvidia TAO toolkit. Source:&nbsp;<a href="https://developer.nvidia.com/tao-toolkit-usecases-whitepaper/1-introduction" target="_blank" rel="noopener noreferrer">Nvidia</a><h3 dir="ltr">The Power of Transfer Learning</h3>One of the most significant features of the NVIDIA TAO Toolkit is its utilization of transfer learning. This technique revolutionizes how models are built by allowing developers to start with an existing model—pre-trained by NVIDIA or third-party sources on vast and diverse data sets—and customize it to their specific needs with relatively little data. This method drastically reduces the time and resources required for model training and enhances the model’s accuracy and performance from the outset.Transfer learning mimics human learning dynamics, where knowledge gained in one context is repurposed for another. For AI development, this means a developer can take a model that recognizes general objects and retrain it to recognize specialized equipment or scenarios specific to their industry. This flexibility opens up a plethora of applications, from enhancing manufacturing processes with visual inspection tools to improving safety protocols through real-time anomaly detection.<h2 dir="ltr"><iframe width="640" height="360" src="https://www.youtube.com/embed/kErfKDZ50To?&amp;ab_channel=NVIDIADeveloper&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true">&nbsp;</iframe>Key Features and Benefits of NVIDIA TAO</h2>The NVIDIA TAO Toolkit offers a suite of capabilities that streamline the development process for AI models. Here, we explore how its features cater to efficiency, accuracy, customization, and deployment flexibility.<h3 dir="ltr">Efficient Model Training</h3>The NVIDIA TAO Toolkit simplifies the model training process significantly. Developers can leverage TAO’s AutoML capability, which automates much of the model tuning. This not only speeds up the model development cycle but also makes it more accessible to non-specialists. The AutoML feature efficiently navigates through the complexities of model tuning, allowing developers to focus more on application development rather than the nuances of model configuration.<h3 dir="ltr">High Accuracy and Customization</h3>NVIDIA is renowned for its high-performance computing and deep learning technologies, and TAO is no exception. It enables the creation of highly accurate AI models tailored to specific needs using state-of-the-art vision transformers and NVIDIA's pre-trained models. These capabilities allow for the development of bespoke AI solutions that are finely tuned to the unique requirements of different use cases, from healthcare diagnostics to automated surveillance systems.<h3 dir="ltr">Optimization for Inference and Deployment Flexibility</h3>Once a model is trained, ensuring it performs efficiently on the intended hardware is crucial. NVIDIA TAO facilitates this by enabling significant optimizations for inference, enhancing model accuracy and performance across GPUs, CPUs, and cloud servers. Additionally, Edge Impulse extends this adaptability to various deployment platforms, including microcontrollers and various other edge devices, ensuring that models integrate smoothly with existing infrastructures.<h3 dir="ltr">AI-Assisted Data Annotations</h3>Preparing data for training AI models is often labor-intensive. NVIDIA TAO eases this burden with AI-assisted data annotation capabilities. This feature accelerates the preparation of training data, making it faster and less costly.<h2 dir="ltr">Integrating NVIDIA TAO with Edge Impulse</h2>The collaboration between NVIDIA TAO and Edge Impulse presents a synergistic approach to edge AI development, significantly reducing time-to-market while increasing the scalability and efficiency of AI solutions. This integration facilitates a streamlined workflow from model creation to deployment, addressing key challenges in edge AI development.<h3 dir="ltr">Enhancing Edge AI Development</h3>By incorporating NVIDIA TAO into the Edge Impulse development platform, developers gain access to a robust set of tools designed to expedite the creation of efficient models. This integration leverages the strengths of both platforms—TAO’s powerful model training and Edge Impulse’s sophisticated deployment capabilities. Together, they simplify the development cycle, enabling developers to rapidly prototype, iterate, and deploy AI models that are both robust and scalable.<h3 dir="ltr">Comprehensive Edge AI Solution</h3>The combined power of NVIDIA TAO and Edge Impulse provides a comprehensive solution for edge AI. From collecting data on edge devices to training and validating models and finally optimizing libraries to run efficiently on target hardware, this integration covers all bases. It supports a wide range of devices, from low-power microcontrollers to more capable GPUs, ensuring that developers can deploy their AI solutions across diverse modern IoT and edge computing environments.Developers can tailor their solutions to specific hardware and performance needs, optimizing resource usage and operational efficiency. Additionally, the combined ecosystem supports a variety of silicon and sensors, making it an all-in-one solution for edge AI applications.<h2 dir="ltr">Practical Applications and Success Stories</h2>The integration of NVIDIA TAO with Edge Impulse offers a powerful platform for practical applications in various sectors. Here, we explore some specific use cases that demonstrate the effectiveness of this technology.PCB Classification:&nbsp;In electronics manufacturing, ensuring that all components are correctly placed and functional on a PCB is crucial. A minor error can lead to a malfunctioning device, leading to increased costs and reduced consumer trust. The PCBClassification model, developed using NVIDIA TAO, addresses this challenge effectively.<a href="https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/pcb_classification" target="_blank">[1]</a>This model utilizes an advanced AI architecture known as global context vision transformer (GCViT), initially trained on a comprehensive image dataset (NVImageNet). The model's training involves transfer learning, where a model developed for one task is partially retrained to perform another. In this case, the base model, proficient in general image recognition, is fine-tuned to specifically identify missing or defective components in PCB images.The process is straightforward yet powerful: images of PCBs are fed into the model, which has been trained to recognize various components like resistors, capacitors, and inductors. The model assesses each component's presence or absence and checks for defects. This automated inspection helps quickly spot errors during the manufacturing process, reducing the likelihood of defective products reaching the market.The accuracy of this model is notable, achieving near-perfect precision in tests. This high level of performance is crucial for industries where the cost of errors can be exceptionally high, both financially and in terms of brand reputation.Reducing Baggage Delays at Airports:&nbsp;The case of Trifork using NVIDIA TAO for baggage tracking at airports highlights the transformative potential of these technologies in enhancing operational efficiency.<a href="https://resources.nvidia.com/en-us-metropolis-software-success-stories/trifork" target="_blank">[2]</a> By deploying vision AI solutions, Trifork has been able to track millions of pieces of luggage, significantly reducing baggage delays and lost baggage incidents. This application not only improves the passenger experience but also optimizes the workflow and resource allocation at airports.Industrial Defect Detection:&nbsp;Another compelling application is in the field of industrial defect detection.<a href="https://developer.nvidia.com/blog/transforming-industrial-defect-detection-with-nvidia-tao-and-vision-ai-models/" target="_blank">[3]</a> Using NVIDIA TAO, developers have created models that can identify defects in various industrial products with high precision. This application is particularly beneficial for quality control in manufacturing, where detecting and addressing defects early in the production process can lead to substantial cost savings and maintain high-quality standards.Retail Recognition:&nbsp;In the retail sector, the application of NVIDIA TAO for retail recognition tasks illustrates how edge AI can transform inventory management. The models could help with identifying a vast array of retail Stock Keeping Units (SKUs), facilitating better stock management, customer service, and logistical efficiency. By automating the recognition process, retailers can ensure more accurate inventory tracking and faster response times, leading to enhanced operational efficiency and customer satisfaction.These practical applications and success stories demonstrate the broad utility and effectiveness of NVIDIA TAO when integrated with tools like Edge Impulse. The technology not only drives innovation and efficiency in existing processes but also opens up new possibilities for smart manufacturing, retail management, and beyond.Further reading:&nbsp;<a href="https://www.wevolver.com/article/a-practical-guide-to-understanding-sensor-fusion-with-edge-impulse-building-a-smart-running-jacket" target="_blank" id="isPasted">A Practical Guide to Understanding Sensor Fusion with Edge Impulse: Building a Smart Running Jacket</a><h2 dir="ltr">Getting Started and Resources</h2>To begin utilizing NVIDIA TAO and Edge Impulse to develop and deploy AI models on edge devices, it's essential to engage with a variety of resources and steps designed to streamline this process.Firstly, a thorough exploration of the available documentation is crucial. Both <a href="https://docs.edgeimpulse.com/docs/edge-impulse-studio/learning-blocks/nvidia-tao" target="_blank">Edge Impulse</a> and <a href="https://developer.nvidia.com/tao-toolkit" target="_blank">NVIDIA</a> provide comprehensive guides and tutorials on their respective websites. These documents are invaluable for understanding how to effectively integrate NVIDIA TAO models into Edge Impulse projects.Additionally, Edge Impulse and Nvidia provide YouTube videos that detail the application and integration of NVIDIA TAO. These resources complement the written documentation and provide a medium through which users can understand how to work with these technologies.<iframe width="640" height="360" src="https://www.youtube.com/embed/bwquofcF94g?&amp;ab_channel=EdgeImpulse&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe>Support doesn’t end with documentation and tutorials. NVIDIA and Edge Impulse have active forums like - <a href="https://forum.edgeimpulse.com/" target="_blank">Edge Impulse Forums</a> and <a href="https://forums.developer.nvidia.com/c/accelerated-computing/intelligent-video-analytics/tao-toolkit/17" target="_blank">NVIDIA Developer Forums</a>, where users can engage with a community of like-minded individuals. These forums are a great place to seek advice, ask questions, and share experiences related to edge AI development.By leveraging these resources, developers can effectively navigate the initial learning curve and start applying NVIDIA TAO and Edge Impulse to real-world projects, gradually building up to more complex applications as familiarity with the tools increases.Further reading:&nbsp;<a href="https://www.wevolver.com/article/building-high-performance-edge-ai-solutions-with-nvidia-jetson-and-edge-impulse" target="_blank" rel="noopener noreferrer">Building High-Performance Edge AI Solutions with Nvidia Jetson and Edge Impulse</a><h2 dir="ltr">Conclusion</h2>The integration of NVIDIA TAO and Edge Impulse brings in a new era of edge AI development, offering developers powerful tools to enhance the efficiency and effectiveness of their AI models. This combination brings together NVIDIA's advanced AI modeling capabilities with Edge Impulse's robust customization and deployment framework, enabling rapid development, optimization, and deployment of AI applications across a variety of platforms and devices.For developers and companies venturing into edge AI, the synergy between NVIDIA TAO and Edge Impulse provides a competitive edge. By simplifying complex processes and reducing the time-to-market, this integration democratizes advanced AI technologies and empowers a broader range of users to innovate and solve real-world problems.<iframe width="540" height="600" src="https://c8056756.sibforms.com/serve/MUIFAMyA6BGksOUPEokHuBnEmZRHhYgJ4RXb4BV0gvzN98ymNIvQdTcBFQnRVfviT6yZSFW_R2g1hENBu7p7Afnyv9Kt0unAAbfFU5mMQhBwI-8hQnKf2r1mmRKhUgDLmtk8bpVSZ7kZGpleCPqtYRv8_kkvODL7F4neGLNGgdpFDm9pPzoYpwbQkc0BU5GVOFoq8CSmD0DDvDjG" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe> <h2 dir="ltr">References</h2>[1] PCBClassification Model Card, Nvidia, 17 Oct 2023, [Online], Available from: <a href="https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/pcb_classification" target="_blank">https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/pcb_classification</a>[2] Trifork - Reducing baggage delay at airports with vision AI, Nvidia, [Online], Available from: <a href="https://resources.nvidia.com/en-us-metropolis-software-success-stories/trifork" target="_blank">https://resources.nvidia.com/en-us-metropolis-software-success-stories/trifork</a>[3] Nirmal Kumar Juluru, Zaid Pervaiz Bhat and Samuel Ochoa, ‘Transforming Industrial Defect Detection with NVIDIA TAO and Vision AI Models’, Nvidia, [Online], Available from: <a href="https://developer.nvidia.com/blog/transforming-industrial-defect-detection-with-nvidia-tao-and-vision-ai-models/" target="_blank">https://developer.nvidia.com/blog/transforming-industrial-defect-detection-with-nvidia-tao-and-vision-ai-models/</a>

NVIDIA TAO and Edge Impulse integrate to streamline the development and deployment of edge AI models, empowering developers to quickly build and optimize AI applications for diverse hardware environments.

Accelerating Edge AI Model Development Using Nvidia TAO and Edge Impulse

<h1 dir="ltr" id="isPasted">Introduction</h1>The modern workplace is rapidly changing and AI is emerging as a transformative force. With the growing use of AI-powered tools, companies are rethinking how to approach hiring, employee onboarding, and enhancing the efficiency, development, and well-being of employees.In this article, we delve into the role of AI in the workplace, highlighting its impact on hiring processes, improving employee retention, and workplace conditions. We present case studies of companies who have found success with AI tools. We also explore future trends and ethical considerations, providing an overview of how AI is positioned to help companies hire more effectively and create better places to work.<h1 dir="ltr">AI in the Hiring Process</h1>Hiring is time-consuming and costly for companies, and often frustratingly involves processes that overlook the most qualified candidates. Some problems with candidate screening, matching, and interviewing are:<ul><li dir="ltr">Over-reliance on keywords and algorithms, which may screen out very qualified candidates whose resumes do not precisely match job descriptions.</li><li dir="ltr">Bias in recruitment, which can hinder diversity and inclusion efforts and lead to a homogenized workforce.</li><li dir="ltr">High-volume applications and initial screenings, a task that relies heavily on the judgment of recruiters and is subject to many human factors.[1]</li></ul>Companies can address these challenges through the implementation of AI technologies in the recruitment process. The following are a few examples of the potential for AI to help companies make hiring more efficient and inclusive.<h2 dir="ltr">Streamlining candidate screening</h2>AI-powered applicant tracking systems can streamline sourcing and screening automating resume analysis and identifying qualified candidates efficiently and accurately. These are some case studies where AI is already improving efficiency in candidate screening.<ul><li dir="ltr">IBM Watson recruitment at Hilton&nbsp;– By integrating IBM Watson for candidate screening and matching, the Hilton hospitality chain saw a 50% reduction in time-to-fill, improved candidate quality, and enhanced hiring manager satisfaction.</li><li dir="ltr">Pymetrics gamified assessments at Deloitte&nbsp;– Deloitte, the US auditing and tax services company incorporated Pymetrics’ AI platform for gamified assessments into their candidate screening process. As a result, the company saw a 60% reduction in time spent on initial screenings, a 30% increase in diverse hires, and enhanced candidate experience.</li><li dir="ltr">HireVue video interviews at McDonald’s&nbsp;– The fast food chain McDonald’s implemented HireVue’s AI-enhanced video screening to streamline their high-volume hiring process. Use of the platform resulted in a 40% reduction in time-time hire, improved candidate engagement, and enhanced scalability.[2]</li></ul><h2 dir="ltr">Enhancing candidate matching</h2>AI algorithms can be used to match job skills and requirements without the over-reliance on specific job description keywords that is often the case with traditional algorithms. AI algorithms can use natural language processing to analyze candidate data and match actual qualifications with job requirements, improving the accuracy of candidate matching.[3]Additionally, by focusing on skills, qualifications, and experience rather than personal characteristics, AI can help reduce unconscious bias in recruitment, leading to a more diverse and inclusive workforce.[4]<h2 dir="ltr">Interview and assessment automation</h2>Companies can harness the power of AI using deep learning and natural language processing to improve the interview process for both candidates and recruiters. Image credit: mikemacmarketing | Source: wikimedia.orgAI-driven recruitment strategies can help make the interview process a much better experience for candidates and recruiters alike. AI tools can automate interview scheduling, conduct initial screenings, ask follow-up questions, and use natural language processing to analyze responses, making the interview process more effective and efficient.[5] In the case studies discussed above, the introduction of AI tools improved the recruitment experience for hiring managers and candidates, and improved candidate engagement.By incorporating AI tools into hiring, organizations can streamline candidate screening, improve matching accuracy, reduce bias, enhance efficiency, and ultimately create a more inclusive and effective hiring environment.<h1 dir="ltr">AI's Role in Employee Onboarding and Retention</h1>While it is important to find and hire the best candidates for the job, it is equally important to make sure that new employees are quickly brought up to speed and productive. Perhaps even more critical is to keep turnover rates low and retain employees by making sure that they are satisfied with the job. Companies can use AI-powered tools to build a more effective and engaged workforce.<h2 dir="ltr">Automated onboarding processes</h2>AI-powered recruitment software can analyze the experience, knowledge, and learning style of a new hire and generate tailored onboarding programs. This type of onboarding takes the employee’s personality and background into account, to help a new employee quickly become engaged and part of company culture.This type of streamlined onboarding means that a new employee becomes productive faster and may be more satisfied with the job overall. Research shows that a successful onboarding experience leads to greater employee retention — employees are twice as likely to be satisfied at work and contribute to a positive company culture.[6]<h2 dir="ltr">Predictive analytics for employee turnover</h2>AI is a powerful tool because it can collect and process large volumes of data. This ability to sift through data and identify patterns and correlations can be leveraged to pinpoint the things that contribute to employee turnover. Companies can use historical turnover data to develop models that predict potential turnover risks.With the power of AI, companies can better understand the specific reasons for turnover, make improvements, and test the outcome. For example, if historical turnover data points to dissatisfaction with limited opportunities for growth, the company can offer career development programs, use AI to monitor the result, and make improvements to the programs.[7]AI tools can have a significant impact on employee onboarding and retention, leading to more engaged, productive, and loyal employees.<h1 dir="ltr">Improving Workplace Conditions with AI</h1>Working conditions are another factor in employee satisfaction and retention. Companies can leverage AI-powered tools to optimize the workspace, support career development, and maintain employee health.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1710149861252-1NAGBQZbUUYzSHV_ckw0Z_sG2he6dCo8.jpg" style="width: 100%px;" class="fr-fic fr-dib">Employers can leverage AI to optimize the working environment while improving employee well-being and retention. Image credit: Alice Mai Ah | Source: wikimedia.org<h2 dir="ltr" id="isPasted">AI for workspace optimization</h2>Using AI, companies can create smart work environments that adapt to employee patterns, preferences, and usage. Automatic environmental adjustments like lighting, temperature, and desk configurations can lead to increased comfort, productivity, and energy cost savings for the company.AI systems can also be used to optimize resources like meeting rooms, workstations, and common areas, making sure facilities are used effectively and that employee needs are supported.<h2 dir="ltr">AI-enhanced training and development</h2>As discussed above, AI can be used during onboarding to get new employees working quickly. In addition to new hire training, AI tools can be used for ongoing growth, tailoring development programs to each employee’s learning styles, career goals, and performance. Using AI analytics, organizations can track the effectiveness of development programs and continuously improve them, leading to better career growth opportunities and satisfied employees.<h2 dir="ltr">Ensuring health and well-being</h2>Predictive AI analytics can play a crucial role in improving employee mental health by analyzing patterns in employee data and feedback to catch signs of stress or burnout. Companies can use this data to take proactive measures that support the well-being of employees.AI-powered ergonomic assessments and activity reminders can also help improve the workplace. Wearable devices and AI applications can provide data on workplace ergonomics and suggest improvements. Wearables can also remind employees to take breaks and stretch to maintain both mental and physical health.<h1 dir="ltr">Case Studies: AI-Driven Success in Hiring and Workplace Enhancement</h1>Companies today are already leveraging powerful AI tools to enhance hiring and the workplace. These are just a few case studies that show how companies are seeing success with these tools.<ul><li dir="ltr">ClearCompany&nbsp;– ClearCompany, a recruiting software platform, has automated employment verification, document submission, and analysis of learning pase using AI-driven onboarding software. This has resulted in a more efficient onboarding process and increased focus on strategic work that drives company growth.[6]</li><li dir="ltr">Genpact&nbsp;– This company has used AI to automate their entire onboarding experience, providing personalized experiences for new hires, which has helped speed up the onboarding process and improve retention.[8]</li><li dir="ltr">Tier 4 Group&nbsp;– This talent acquisition firm worked with TechScreen to streamline their tech hiring processes, resulting in a more effective, ethical, and considerate candidate screening process.[9]</li><li dir="ltr">Iceland Foods – During the pandemic, Iceland Foods partnered with Sapia.ai to adapt their recruitment processes to a changing world. AI automation helped the company find the right candidates in the midst of global upheaval, demonstrating the resilience of AI-driven recruitment.[10]</li></ul>These examples illustrate the diverse applications of AI in hiring and workplace enhancement, from improving candidate experience and streamlining the recruitment process, to fostering internal development and supporting ethical hiring practices.<h1 dir="ltr">Looking ahead: the next generation of AI in the workplace</h1>The integration of AI into the workplace is poised to have a huge impact on how we work. Here are some ways that we predict AI will transform the workplace in specific industries.<ul><li dir="ltr">Healthcare&nbsp;– By integrating data like medical imaging, lab results, and checkups, AI advancements will likely lead to more efficient healthcare systems, as well as more effective care, with better outcomes for patients.</li><li dir="ltr">Finance&nbsp;– The role of AI in fraud detection, personalized financial advising, and record keeping is likely to grow, but a strong emphasis on human oversight and ethical decision-making will remain a key part of the finance industry.</li><li dir="ltr">Education&nbsp;– In education, AI will be able to analyze and provide insights into student learning, and will help institutions create more engaging and personalized experiences for both students and teachers.</li><li dir="ltr">Retail&nbsp;– Predictive analytics will enable retailers to understand customer behavior, manage inventory, and improve customer service. AI has the potential to boost customer satisfaction and operational efficiency.</li><li dir="ltr">Manufacturing&nbsp;– AI will significantly impact how work is done in the manufacturing industry as well. Collaborative robots (cobots), predictive maintenance, and smart factories are AI-powered tools that will likely improve the safety and efficiency of manufacturing.</li></ul><h1 dir="ltr">Ethical considerations: balancing technology and humanity</h1>AI is a promising tool for improving workplaces and the efficiency, health, and well-being of workers. And because AI is for the benefit of humanity, it is important to examine ethical considerations and maintain a balance between technology and human values.When using AI tools in hiring and HR functions, It is critical for companies to remain vigilant against bias and discrimination. Laws, standards, and guidelines that require AI tools to be audited for bias are already in place. It is also important for companies to monitor AI tools for bias during recruiting, hiring, and promotion.[11]In an increasingly automated workplace, it is also essential for companies to balance AI and human interaction. While AI magnifies our human capabilities and streamlines the work that we do, it is important for companies to foster human relationships and empathy, and to take an ethical stance to put humans first when it comes to AI strategies.<h2 dir="ltr">Conclusion</h2>The integration of AI into the hiring process, employee onboarding, and workplace improvement is a significant shift towards a more data-driven, efficient, and employee-centric approach to company management. As underscored by the case studies above, there are already tangible benefits to AI-driven strategies, from streamlined hiring processes to personalized onboarding experiences, to optimized workplace conditions that prioritize employee health and well-being.As AI continues to evolve and is increasingly integrated into our workplaces, it is important to address ethical considerations and maintain a balance between technological advancements and human values. The future of AI in the workplace holds great promise for improving how we hire and work, and has the power to transform industries, improve the lives of workers, and shape how we think about work.<h2>References</h2>[1] “Navigating the Maze: The Fracture’s in Today’s IT Recruiting Process &amp; the AI Solution”. TechScreen. November 27, 2023. Accessed from <a href="https://techscreen.com/navigating-the-maze-the-fractures-in-todays-it-recruiting-process-the-ai-solution/" target="_blank">https://techscreen.com/navigating-the-maze-the-fractures-in-todays-it-recruiting-process-the-ai-solution/</a>&nbsp;[2] Jessica Dennis. “AI Recruiting: Uses, Advantages, &amp; Disadvantages”. TechnologyAdvice. 22 May 2023. Accessed from <a href="https://technologyadvice.com/blog/human-resources/ai-recruiting/" target="_blank">https://technologyadvice.com/blog/human-resources/ai-recruiting/</a>&nbsp;[3] Ranganath Ponnaboyina, et al. “Smart Recruitment System Using Deep Learning with Natural Language Processing”. Intelligent Systems and Sustainable Computing, Vol. 289. 29 May 2022. Accessed from <a href="https://link.springer.com/chapter/10.1007/978-981-19-0011-2_58" target="_blank">https://link.springer.com/chapter/10.1007/978-981-19-0011-2_58</a>&nbsp;[4] “Pros and Cons of Using AI in Recruiting”. Recruiter.com. Accessed from <a href="https://www.recruiter.com/recruiting/pros-and-cons-of-using-ai-in-recruiting/" target="_blank">https://www.recruiter.com/recruiting/pros-and-cons-of-using-ai-in-recruiting/</a>&nbsp;[5] Paul Arolye. “Navigating Recruitment Challenges: 5 Ways AI Can Revolutionize HR”. eLearning Industry. 27 January 2024. Accessed from <a href="https://elearningindustry.com/navigating-recruitment-challenges-ways-ai-can-revolutionize-hr" target="_blank">https://elearningindustry.com/navigating-recruitment-challenges-ways-ai-can-revolutionize-hr</a>&nbsp;[6] ClearCompany. “3 Benefits of Automating Employee Onboarding with AI-Driven Software”. 5 September 2023. Accessed from <a href="https://blog.clearcompany.com/benefits-of-automating-employee-onboarding-with-ai" target="_blank">https://blog.clearcompany.com/benefits-of-automating-employee-onboarding-with-ai</a>&nbsp;[7] Cabot Jaffee. “Leveraging AI to predict and reduce employee turnover.” Talent Management. 14 August 2023. Accessed from <a href="https://www.talentmgt.com/articles/2023/08/14/leveraging-ai-to-predict-and-reduce-employee-turnover/" target="_blank">https://www.talentmgt.com/articles/2023/08/14/leveraging-ai-to-predict-and-reduce-employee-turnover/</a>&nbsp;[8] Maria Korolov. “AI takes aim at employee turnover”. CIO. 7 April 2022. Accessed from <a href="https://www.cio.com/article/307980/ai-takes-aim-at-employee-turnover.html" target="_blank">https://www.cio.com/article/307980/ai-takes-aim-at-employee-turnover.html</a>&nbsp;[9] TechScreen. “Case Studies in Tech Hiring: AI Success Stories and Lessons Learned from Top Companies”. 28 December, 2023. Accessed from <a href="https://techscreen.com/case-studies-in-tech-hiring-ai-success-stories-and-lessons-learned-from-top-companies/" target="_blank">https://techscreen.com/case-studies-in-tech-hiring-ai-success-stories-and-lessons-learned-from-top-companies/</a>&nbsp;[10] Sapia.ai. “How Iceland Foods smashed its volume hiring goals”. Accessed from <a href="https://sapia.ai/resources/case-study/iceland/" target="_blank">https://sapia.ai/resources/case-study/iceland/</a>&nbsp;[11] Roy Mauer. “HR Must Be Vigilant About the Ethical Use of AI Technology”. SHRM. 26 September 2022. Accessed from <a href="https://www.shrm.org/topics-tools/news/technology/hr-must-vigilant-ethical-use-ai-technology" target="_blank">https://www.shrm.org/topics-tools/news/technology/hr-must-vigilant-ethical-use-ai-technology</a>&nbsp;

 AI is having an impact on hiring processes, employee retention, and workplace conditions. 

How AI will help companies hire more people and make the workplace better

We’ve all heard of Large Language Models, or LLMs – massive scale deep learning models trained on huge amounts of text that form the basis for chatbots like OpenAI’s ChatGPT. Next-generation Multimodal Models (MMs) can learn from inputs beyond text, including video, images, and sound.Creating MM models at a smaller scale poses significant challenges, including the problem of being robust to non-random missing information. This is information that a model doesn’t have, often due to some biased availability in resources. It is thus critical to ensure the model does not learn the patterns of biased missingness in making its predictions.<h2>MultiModN turns this around</h2>In response to this problem, researchers from the&nbsp;<a href="https://www.epfl.ch/labs/ml4ed/" target="_blank">Machine Learning for Education</a> (ML4ED) and&nbsp;<a href="https://www.epfl.ch/labs/mlo/" target="_blank">Machine Learning and Optimization</a> (MLO) Laboratories in<a href="https://www.epfl.ch/schools/ic/" target="_blank">&nbsp;EPFL’s School of Computer and Communication Sciences</a> have developed and tested the exact opposite to a Large Language Model.Spearheaded by Professor Mary-Anne Hartley, head of the Laboratory for<a href="https://www.epfl.ch/labs/mlo/igh-intelligent-global-health/" target="_blank">&nbsp;intelligent Global Health Technologies</a> hosted jointly in the MLO and the Yale School of Medicine and Professor Tanja Käser, head of ML4ED,&nbsp;MultiModN&nbsp;is a unique&nbsp;<a href="https://arxiv.org/abs/2309.14118" target="_blank" rel="noopener">Modular Multimodal Model</a>, presented recently at the&nbsp;<a href="https://neurips.cc/Conferences/2023" target="_blank" rel="noopener">NeurIPS2023</a> conference.Like existing Multimodal Models&nbsp;MultiModN can learn from text, images, video, and sound. Unlike existing MMs, it is made up of any number of smaller, self-contained, and input-specific modules that can be selected depending on the information available, and then strung together in a sequence of any number, combination, or type of input. Itcan then output any number, or combination, of predictions.“We evaluated&nbsp;MultiModN across ten real-world tasks including medical diagnosis support, academic performance prediction, and weather forecasting. Through these experiments, we believe that&nbsp;MultiModN is the first inherently interpretable, MNAR-resistant approach to multimodal modelling,” explained Vinitra Swamy, a PhD student with ML4ED and MLO and joint first author on the project.<h3>A first use case: medical decision-making</h3>The first use case for&nbsp;MultiModN will be as a clinical decision support system for medical personnel in low-resource settings. In healthcare, clinical data is often missing, perhaps due to resource constraints (a patient can’t afford the test) or resource abundance (the test is redundant due to a superior one that was performed).&nbsp;MultiModN is able to learn from this real-world data without adopting its biases, as well as adapting predictions to any combination or number of inputs.“Missingness is a hallmark of data in low-resource settings and when models learn these patterns of missingness, they may encode bias into their predictions. The need for flexibility in the face of unpredictably available resources is what inspired&nbsp;MultiModN,” explained Hartley, who is also a medical doctor.<h3>From the lab to real life</h3>Publication, however, is just the first step towards implementation. Hartley has been working with colleagues at Lausanne University Hospital<a href="https://www.chuv.ch/fr/chuv-home/" target="_blank" rel="noopener">&nbsp;(CHUV</a>) and Inselspital, University Hospital Bern&nbsp;<a href="https://www.insel.ch/fr/" target="_blank" rel="noopener">uBern</a> to conduct clinical studies focused on pneumonia and tuberculosis diagnosis in low resource settings and they are recruiting thousands of patients in South Africa, Tanzania, Namibia and Benin.The research teams undertook a large training initiative, teaching more than 100 doctors to systematically collect multimodal data including images and ultrasound video, so that&nbsp;MultiModN can be trained to be sensitive to real data coming from low resource regions.“We are collecting exactly the kind of complex multimodal data that&nbsp;MultiModN is designed to handle,” said Dr Noémie Boillat-Blanco, an infectious diseases doctor at CHUV. “We are excited to see a model that appreciates the complexity of missing resources in our settings and of systematic missingness of routine clinical assessments,” added Dr Kristina Keitel at Inselspital, University Hospital Bern.<h3>Machine learning for the public good</h3>The development and training of&nbsp;MultiModN is a continuation of EPFL efforts to adapt machine learning tools to reality and for the public good. It comes not long after the launch of<a href="https://actu.epfl.ch/news/epfl-s-new-large-language-model-for-medical-knowle/" target="_blank">&nbsp;Meditron</a>, the world’s best performing open source LLM also designed to help guide clinical decision-making.Both tools align with the mission of the new&nbsp;<a href="https://actu.epfl.ch/news/opening-of-the-new-epfl-ai-center-a-hub-for-ai-i-2/" target="_blank">EPFL AI Center</a> that focuses on how responsible and effective AI can advance technological innovation for the benefit of all sectors of society.Dr. Mary-Anne Hartley is a keynote speaker at the upcoming Applied Machine Learning Days (AMLD) at the SwissTech Convention Center March 23-26, 2024. Vinitra Swamy will be presenting MultiModN at AMLD’s Applied eXplainable AI track. Discover other keynotes<a href="https://2024.appliedmldays.org/media-9-speakers" target="_blank" rel="noopener">here</a>&nbsp;and register<a href="https://2024.appliedmldays.org/media-12-registration" target="_blank" rel="noopener">here</a>.

Machine Learning Neural Network © iStock

Researchers at EPFL have developed a new, uniquely modular machine learning model for flexible decision-making. It is able to input any mode of text, video, image, sound, and time-series and then output any number, or combination, of predictions. 

Anything-in-anything-out: a new modular AI model

Behrooz Tahmasebi — an MIT PhD student in the Department of Electrical Engineering and Computer Science (EECS) and an affiliate of the Computer Science and Artificial Intelligence Laboratory (CSAIL) — was taking a mathematics course on differential equations in late 2021 when a glimmer of inspiration struck. In that class, he learned for the first time about Weyl’s law, which had been formulated 110 years earlier by the German mathematician Hermann Weyl. Tahmasebi realized it might have some relevance to the computer science problem he was then wrestling with, even though the connection appeared — on the surface — to be thin, at best. Weyl’s law, he says, provides a formula that measures the complexity of the spectral information, or data, contained within the fundamental frequencies of a drum head or guitar string.Tahmasebi was, at the same time, thinking about measuring the complexity of the input data to a neural network, wondering whether that complexity could be reduced by taking into account some of the symmetries inherent to the dataset. Such a reduction, in turn, could facilitate — as well as speed up — machine learning processes.Weyl’s law, conceived about a century before the boom in machine learning, had traditionally been applied to very different physical situations — such as those concerning the vibrations of a string or the spectrum of electromagnetic (black-body) radiation given off by a heated object. Nevertheless, Tahmasebi believed that a customized version of that law might help with the machine learning problem he was pursuing. And if the approach panned out, the payoff could be considerable.He spoke with his advisor, Stefanie Jegelka — an associate professor in EECS and affiliate of CSAIL and the MIT Institute for Data, Systems, and Society — who believed the idea was definitely worth looking into. As Tahmasebi saw it, Weyl’s law had to do with gauging the complexity of data, and so did this project. But Weyl’s law, in its original form, said nothing about symmetry.He and Jegelka have now succeeded in modifying Weyl’s law so that symmetry can be factored into the assessment of a dataset’s complexity. “To the best of my knowledge,” Tahmasebi says, “this is the first time Weyl’s law has been used to determine how machine learning can be enhanced by symmetry.”The&nbsp;<a href="https://arxiv.org/abs/2303.14269" target="_blank">paper</a> he and Jegelka wrote earned a “Spotlight” designation when it was presented at the December 2023 conference on Neural Information Processing Systems — widely regarded as the world’s top conference on machine learning.This work, comments Soledad Villar, an applied mathematician at Johns Hopkins University, “shows that models that satisfy the symmetries of the problem are not only correct but also can produce predictions with smaller errors, using a small amount of training points. [This] is especially important in scientific domains, like computational chemistry, where training data can be scarce.”In their paper, Tahmasebi and Jegelka explored the ways in which symmetries, or so-called “invariances,” could benefit machine learning. Suppose, for example, the goal of a particular computer run is to pick out every image that contains the numeral 3. That task can be a lot easier, and go a lot quicker, if the algorithm can identify the 3 regardless of where it is placed in the box — whether it’s exactly in the center or off to the side — and whether it is pointed right-side up, upside down, or oriented at a random angle. An algorithm equipped with the latter capability can take advantage of the symmetries of translation and rotations, meaning that a 3, or any other object, is not changed in itself by altering its position or by rotating it around an arbitrary axis. It is said to be invariant to those shifts. The same logic can be applied to algorithms charged with identifying dogs or cats. A dog is a dog is a dog, one might say, irrespective of how it is embedded within an image.&nbsp;The point of the entire exercise, the authors explain, is to exploit a dataset’s intrinsic symmetries in order to reduce the complexity of machine learning tasks. That, in turn, can lead to a reduction in the amount of data needed for learning. Concretely, the new work answers the question: How many fewer data are needed to train a machine learning model if the data contain symmetries?There are two ways of achieving a gain, or benefit, by capitalizing on the symmetries present. The first has to do with the size of the sample to be looked at. Let’s imagine that you are charged, for instance, with analyzing an image that has mirror symmetry — the right side being an exact replica, or mirror image, of the left. In that case, you don’t have to look at every pixel; you can get all the information you need from half of the image — a factor of two improvement. If, on the other hand, the image can be partitioned into 10 identical parts, you can get a factor of 10 improvement. This kind of boosting effect is linear.To take another example, imagine you are sifting through a dataset, trying to find sequences of blocks that have seven different colors — black, blue, green, purple, red, white, and yellow. Your job becomes much easier if you don’t care about the order in which the blocks are arranged. If the order mattered, there would be 5,040 different combinations to look for. But if all you care about are sequences of blocks in which all seven colors appear, then you have reduced the number of things — or sequences — you are searching for from 5,040 to just one.Tahmasebi and Jegelka discovered that it is possible to achieve a different kind of gain — one that is exponential — that can be reaped for symmetries that operate over many dimensions. This advantage is related to the notion that the complexity of a learning task grows exponentially with the dimensionality of the data space. Making use of a multidimensional symmetry can therefore yield a disproportionately large return. “This is a new contribution that is basically telling us that symmetries of higher dimension are more important because they can give us an exponential gain,” Tahmasebi says.&nbsp;The NeurIPS 2023 paper that he wrote with Jegelka contains two theorems that were proved mathematically. “The first theorem shows that an improvement in sample complexity is achievable with the general algorithm we provide,” Tahmasebi says. The second theorem complements the first, he added, “showing that this is the best possible gain you can get; nothing else is achievable.”He and Jegelka have provided a formula that predicts the gain one can obtain from a particular symmetry in a given application. A virtue of this formula is its generality, Tahmasebi notes. “It works for any symmetry and any input space.” It works not only for symmetries that are known today, but it could also be applied in the future to symmetries that are yet to be discovered. The latter prospect is not too farfetched to consider, given that the search for new symmetries has long been a major thrust in physics. That suggests that, as more symmetries are found, the methodology introduced by Tahmasebi and Jegelka should only get better over time.According to Haggai Maron, a computer scientist at Technion (the Israel Institute of Technology) and NVIDIA who was not involved in the work, the approach presented in the paper “diverges substantially from related previous works, adopting a geometric perspective and employing tools from differential geometry. This theoretical contribution lends mathematical support to the emerging subfield of ‘Geometric Deep Learning,’ which has applications in graph learning, 3D data, and more. The paper helps establish a theoretical basis to guide further developments in this rapidly expanding research area.”

New MIT research provides a theoretical proof for a phenomenon observed in practice: that encoding symmetries in the machine learning model helps the model learn with fewer data. Image: Alex Shipps/MIT CSAIL

Exploiting the symmetry within datasets, MIT researchers show, can decrease the amount of data needed for training neural networks.

How symmetry can come to the aid of machine learning

In this episode, we talk about MIT researchers making a smart tool (like a robot scientist) that uses AI to understand and explain how other AI brains (neural networks) work.

Podcast: AI Model Tests AI Models To Tell Us How They Work

<section id="isPasted">Imagine a task that used to take 11 minutes now taking less time than the blink of an eye. Couple that speed increase with 97% accuracy, and these are the results researchers at Texas A&amp;M University achieved when combining machine learning, neural networks and novel compression tactics in a new project advancing reservoir production forecasts.Why is the research important? Because advancing forecasts improves the availability and accuracy of information oil and gas companies use to make sound financial and operating decisions."Production forecasting can be done several ways, but everything depends on it," said researcher Mohammad Elkady. "Any decision – if you want to take out a loan, do an economic study, or make a development phase decision – depends on the forecast because it tells you how much oil, gas or water you're going to produce."&nbsp;Elkady and fellow researcher Veena Kumar created a flexible method they say could apply to understanding and evaluating any complex reservoir. Both are graduate students in the Harold Vance Department of Petroleum Engineering and work under the guidance of their advisor, Dr. Siddharth Misra, the Ted H. Smith, Jr. '75 and Max R. Vordenbaum '73 DVG Associate Professor.</section><blockquote>Any decision – if you want to take out a loan, do an economic study, or make a development phase decision – depends on the forecast because it tells you how much oil, gas or water you're going to produce.<cite>Mohammad Elkady</cite></blockquote><section>Misra and Elkady introduced the work in the paper, "<a href="https://doi.org/10.2118/216935-MS" rel="noopener" target="_blank">Ultrafast Multiphase Production Forecasting for Large Gas Condensate Shale Reservoirs</a>," during the Abu Dhabi International Progressive Energy Congress in Oct 2023.&nbsp;Misra also presented the work in the paper,&nbsp;"Rapid Production Forecasting for Hydraulically Fractured Wells in Large, Heterogeneous Shale Reservoir," at the Society of Petroleum Engineers’ Workshop in New Orleans, Louisiana in Nov. 2023.<h2>Production forecasting history</h2>To make a profit, oil and gas companies weigh equipment, human resources, and other costs against the profitable hydrocarbons they can produce from subsurface reservoirs. Such decisions would be easy if petroleum engineers could peek inside the earth and see what's there. Since the subsurface is opaque, it's tricky to accurately predict oil recovery from reservoirs as conditions change over time.In the past, engineers used two main methods for forecasting. The first and oldest plots a well's oil recovery rates and declines over time, and that decline curve analysis can help estimate the recoveries in nearby or similar wells. The second is newer and uses computational resources to create a visual map of a reservoir by compressing the massive amounts of data gathered from surface and subsurface sensors and simulating the information into simplified renderings.<h2>Refining data to enhance forecasting</h2>This research is not the first to use machine learning or even neural networks to get a more accurate forecast, but it is the first to combine those processes with geomodel data compression. Elkady said data compression isn't new, as it's been used successfully in areas like image and video analysis. High-resolution videos must be compressed when shared around the world on restrictive bandwidths, yet it must be done selectively to keep the subjects true to the original imaging. Though the application in petroleum is novel, data compression makes sense for similar reasons."Geological data usually has a lot of data, and some may not be as important as others," said Elkady. "That's the beauty of data compression; it's not just compression, it extracts the significant features out of these data."</section><blockquote>That's the beauty of data compression; it's not just compression, it extracts the significant features out of these data.<cite>Mohammad Elkady</cite></blockquote><section>Elkady and Kumar began by creating 4,000 virtual homogenous or uniformly structured reservoirs containing 88,000 grid cells. Each cell had three randomly sampled geological properties: permeability, porosity, water saturation, fracture space, etc. The total data generated for each reservoir equaled over a quarter of a million points. The researchers ran these original reservoirs through a simulator for the most accurate production forecasts possible.The researchers then compressed the original data using two different methods simultaneously to crunch the numbers down with a 50,000 to 1 ratio. After training in Python, they taught the neural network to do production forecast workflows of the compressed reservoir model. Results showed the workflow was much faster than the simulator – under a second for every 700 seconds the simulator took – and had an error rate of 3% when compared to simulated forecasts.Next, the researchers used simulated data guided by Aramco that mimicked actual reservoir information to create about 3,000 virtual reservoirs that were heterogeneous or complex in structure and content yet still had the same number of grid cells.“We used geostatistical tools to generate the geology to be as realistic as we could," said Elkady. "The Aramco team supervised the generation of these datasets to make sure that, from their experience, this looks like a real reservoir."When that set of comparisons between workflow and simulator forecasts was completed, the researchers added more parameters, starting with six or seven. Eventually, they included human-related parameters, such as when operators choose how much gas they wish to produce despite what the reservoir is capable of, a complex concept for neural networks to factor in.</section><blockquote>Testing the model's effectiveness and precision on a variety of real-world shale gas reservoirs will undoubtedly demonstrate its potential and adaptability in diverse situations.<cite>Veena Kumar</cite></blockquote><section>"Initially, the amount of accuracy to error was very bad when we added human variations to the data set," Kumar said. "I am proud of getting the error percentage down, but we're looking into the reason for the initial higher percentages."The model-building stage – including reservoir parameter definitions, geomodel compressions, and prediction parameter factoring – required about one hour, depending on the data complexity. Once the models were trained and tuned, the time ratio for generating forecasts from the models remained less than one second for the researcher’s workflow to every 700 seconds, or about 11 minutes, when using a commercial software simulator.<h2>Future direction</h2>Currently, the work is expanding the number of parameters, refining the techniques and input factors, and testing methods to reduce the training times needed to learn the software. After that, the researchers want to test it out on actual reservoir data, ideally a reservoir where some production has already been completed.They are optimistic that the work can be applied effectively in many different formations."The goal of refining the multivariate forecast model for long-term projections, potentially over a decade, is one of the approaches we're excited to try out," said Kumar. "Testing the model's effectiveness and precision on a variety of real-world shale gas reservoirs will undoubtedly demonstrate its potential and adaptability in diverse situations."</section>

Imagine a task that used to take 11 minutes now taking less time than the blink of an eye. Couple that speed increase with 97% accuracy, and these are the results researchers achieved when combining ML, neural networks and novel compression in a new project advancing reservoir production forecasts.

Ultrafast Reservoir Forecasting Boasts Speed and Accuracy

This article was discussed in our <a href="https://www.wevolver.com/profile/the.next.byte" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/8a9e30e0-92a2-49c6-afc7-1ca5a7d907f4?dark=false" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe> The full article will continue below.<hr>Explaining the behavior of trained neural networks remains a compelling puzzle, especially as these models grow in size and sophistication. Like other scientific challenges throughout history, reverse-engineering how artificial intelligence systems work requires a substantial amount of experimentation: making hypotheses, intervening on behavior, and even dissecting large networks to examine individual neurons. To date, most successful experiments have involved large amounts of human oversight. Explaining every computation inside models the size of GPT-4 and larger will almost certainly require more automation — perhaps even using AI models themselves.&nbsp;Facilitating this timely endeavor, researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed a novel approach that uses AI models to conduct experiments on other systems and explain their behavior. Their method uses agents built from pretrained language models to produce intuitive explanations of computations inside trained networks.Central to this strategy is the “automated interpretability agent” (AIA), designed to mimic a scientist’s experimental processes. Interpretability agents plan and perform tests on other computational systems, which can range in scale from individual neurons to entire models, in order to produce explanations of these systems in a variety of forms: language descriptions of what a system does and where it fails, and code that reproduces the system’s behavior. Unlike existing interpretability procedures that passively classify or summarize examples, the AIA actively participates in hypothesis formation, experimental testing, and iterative learning, thereby refining its understanding of other systems in real time.&nbsp;Complementing the AIA method is the new “function interpretation and description” (<a href="https://multimodal-interpretability.csail.mit.edu/FIND-benchmark/" target="_blank">FIND</a>) benchmark, a test bed of functions resembling computations inside trained networks, and accompanying descriptions of their behavior. One key challenge in evaluating the quality of descriptions of real-world network components is that descriptions are only as good as their explanatory power: Researchers don’t have access to ground-truth&nbsp;labels of units or descriptions of learned computations. FIND addresses this long-standing issue in the field by providing a reliable standard for evaluating interpretability procedures: explanations of functions (e.g., produced by an AIA) can be evaluated against function descriptions in the benchmark. &nbsp;For example, FIND contains synthetic neurons designed to mimic the behavior of real neurons inside language models, some of which are selective for individual concepts such as “ground transportation.” AIAs are given black-box access to synthetic neurons and design inputs (such as “tree,” “happiness,” and “car”) to test a neuron’s response. After noticing that a synthetic neuron produces higher response values for “car” than other inputs, an AIA might design more fine-grained tests to distinguish the neuron’s selectivity for cars from other forms of transportation, such as planes and boats. When the AIA produces a description such as “this neuron is selective for road transportation, and not air or sea travel,” this description is evaluated against the ground-truth description of the synthetic neuron (“selective for ground transportation”) in FIND. The benchmark can then be used to compare the capabilities of AIAs to other methods in the literature.&nbsp;Sarah Schwettmann PhD '21, co-lead author of a <a href="https://arxiv.org/pdf/2309.03886.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">paper on the new work</a> and a research scientist at CSAIL, emphasizes the advantages of this approach. “The AIAs’ capacity for autonomous hypothesis generation and testing may be able to surface behaviors that would otherwise be difficult for scientists to detect. It’s remarkable that language models, when equipped with tools for probing other systems, are capable of this type of experimental design,” says Schwettmann. “Clean, simple benchmarks with ground-truth answers have been a major driver of more general capabilities in language models, and we hope that FIND can play a similar role in interpretability research.”<h2>Automating interpretability&nbsp;</h2>Large language models are still holding their status as the in-demand celebrities of the tech world. The recent advancements in LLMs have highlighted their ability to perform complex reasoning tasks across diverse domains. The team at CSAIL recognized that given these capabilities, language models may be able to serve as backbones of generalized agents for automated interpretability. “Interpretability has historically been a very multifaceted field,” says Schwettmann. “There is no one-size-fits-all approach; most procedures are very specific to individual questions we might have about a system, and to individual modalities like vision or language. Existing approaches to labeling individual neurons inside vision models have required training specialized models on human data, where these models perform only this single task. Interpretability agents built from language models could provide a general interface for explaining other systems — synthesizing results across experiments, integrating over different modalities, even discovering new experimental techniques at a very fundamental level.”&nbsp;As we enter a regime where the models doing the explaining are black boxes themselves, external evaluations of interpretability methods are becoming increasingly vital. The team’s new benchmark addresses this need with a suite of functions with known structure, that are modeled after behaviors observed in the wild. The functions inside FIND span a diversity of domains, from mathematical reasoning to symbolic operations on strings to synthetic neurons built from word-level tasks. The dataset of interactive functions is procedurally constructed; real-world complexity is introduced to simple functions by adding noise, composing functions, and simulating biases. This allows for comparison of interpretability methods in a setting that translates to real-world performance. &nbsp; &nbsp; &nbsp;In addition to the dataset of functions, the researchers introduced an innovative evaluation protocol to assess the effectiveness of AIAs and existing automated interpretability methods. This protocol involves two approaches. For tasks that require replicating the function in code, the evaluation directly compares the AI-generated estimations and the original, ground-truth functions. The evaluation becomes more intricate for tasks involving natural language descriptions of functions. In these cases, accurately gauging the quality of these descriptions requires an automated understanding of their semantic content. To tackle this challenge, the researchers developed a specialized “third-party” language model. This model is specifically trained to evaluate the accuracy and coherence of the natural language descriptions provided by the AI systems, and compares it to the ground-truth function behavior.&nbsp;FIND enables evaluation revealing that we are still far from fully automating interpretability; although AIAs outperform existing interpretability approaches, they still fail to accurately describe almost half of the functions in the benchmark. Tamar Rott Shaham, co-lead author of the study and a postdoc in CSAIL, notes that “while this generation of AIAs is effective in describing high-level functionality, they still often overlook finer-grained details, particularly in function subdomains with noise or irregular behavior. This likely stems from insufficient sampling in these areas. One issue is that the AIAs’ effectiveness may be hampered by their initial exploratory data. To counter this, we tried guiding the AIAs’ exploration by initializing their search with specific, relevant inputs, which significantly enhanced interpretation accuracy.” This approach combines new AIA methods with previous techniques using pre-computed examples for initiating the interpretation process.The researchers are also developing a toolkit to augment the AIAs’ ability to conduct more precise experiments on neural networks, both in black-box and white-box settings. This toolkit aims to equip AIAs with better tools for selecting inputs and refining hypothesis-testing capabilities for more nuanced and accurate neural network analysis. The team is also tackling practical challenges in AI interpretability, focusing on determining the right questions to ask when analyzing models in real-world scenarios. Their goal is to develop automated interpretability procedures that could eventually help people audit systems — e.g., for autonomous driving or face recognition — to diagnose potential failure modes, hidden biases, or surprising behaviors before deployment.&nbsp;<h2>Watching the watchers</h2>The team envisions one day developing nearly autonomous AIAs that can audit other systems, with human scientists providing oversight and guidance. Advanced AIAs could develop new kinds of experiments and questions, potentially beyond human scientists’ initial considerations. The focus is on expanding AI interpretability to include more complex behaviors, such as entire neural circuits or subnetworks, and predicting inputs that might lead to undesired behaviors. This development represents a significant step forward in AI research, aiming to make AI systems more understandable and reliable.“A good benchmark is a power tool for tackling difficult challenges,” says Martin Wattenberg, computer science professor at Harvard University who was not involved in the study. “It's wonderful to see this sophisticated benchmark for interpretability, one of the most important challenges in machine learning today. I'm particularly impressed with the automated interpretability agent the authors created. It's a kind of interpretability jiu-jitsu, turning AI back on itself in order to help human understanding.”Schwettmann, Rott Shaham, and their colleagues presented their work at NeurIPS 2023 in December. &nbsp;Additional MIT coauthors, all affiliates of the CSAIL and the Department of Electrical Engineering and Computer Science (EECS), include graduate student Joanna Materzynska, undergraduate student Neil Chowdhury, Shuang Li PhD ’23, Assistant Professor Jacob Andreas, and Professor Antonio Torralba. Northeastern University Assistant Professor David Bau is an additional coauthor.The work was supported, in part, by the MIT-IBM Watson AI Lab, Open Philanthropy, an Amazon Research Award, Hyundai NGV, the U.S. Army Research Laboratory, the U.S. National Science Foundation, the Zuckerman STEM Leadership Program, and a Viterbi Fellowship.

FIND is a new benchmark suite for evaluating automated interpretability methods in neural networks, featuring functions that mimic real-world network components and their complexities. It also presents a novel interactive method using automated interpretability agents, which employ pretrained language models to generate descriptions of function behavior, demonstrating the agent's ability to infer function structure while highlighting the need for further refinement in capturing local details. Image: Alex Shipps/MIT CSAIL

MIT researchers introduce a method that uses artificial intelligence to automate the explanation of complex neural networks.

AI agents help explain other AI systems

<h2>Introduction to Temporal Event-Based Neural Networks (TENNs)</h2>In the realm of Artificial Intelligence (AI), the evolution of neural network technologies has reached a pivotal juncture with the advent of Temporal Event-Based Neural Networks (TENNs). Developed by BrainChip, TENNs represent a groundbreaking approach to processing sequential spatiotemporal data, addressing the limitations of traditional neural network models like Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs).CNNs, while proficient in processing spatial information in images, falter in effectively utilizing temporal data. RNNs, on the other hand, capture sequential dependencies but face challenges like vanishing gradients and difficulty in parallelization. TENNs emerge as a solution, efficiently combining spatial and temporal convolutions, thus excelling in processing sequential data like video and time series data.TENNs leverage the strengths of both CNNs and RNNs, integrating spatial and temporal characteristics of data. This integration enables TENNs to learn spatial and temporal correlations effectively, distinguishing them from traditional state-space models. Their flexibility allows for operation in either temporal convolution mode or recurrent mode, adapting to specific application requirements.BrainChip's 2nd Generation Akida platform supports TENNs, offering state-of-the-art performance across various domains of sequential data. This includes applications in speech recognition, patient monitoring in medical equipment, and streaming video object detection and tracking. With their superior performance, low computational requirements, and versatility, TENNs are poised to revolutionize intelligent Edge solutions, providing highly accurate models for sequential data analysis with a fraction of the computational resources required by other network architectures.For a deeper understanding of TENNs and their applications, BrainChip's white paper, "Temporal Event-based Neural Networks: A New Approach to Temporal Processing," offers comprehensive insights into this innovative technology.<a href="https://wevlv.co/brainchip-tenns-whitepaper" target="_blank" rel="noopener noreferrer">Read the whitepaper here. </a><h3 dir="ltr">More about BrainChip: Making AI ubiquitous</h3>The integration of sophisticated IoT infrastructure with Artificial Intelligence technologies, particularly those modeled after human neural processes, results in a significant enhancement of operational efficiency. This amalgamation facilitates proactive problem-solving, augments interactions between humans and machines, and fortifies security measures.A crucial aspect of proliferating AI ubiquitously lies in simplifying its deployment and scalability. Achieving this necessitates a reduction in costs, autonomy from cloud reliance, and the localization of intelligence in proximity to sensory inputs.Brainchip's comprehensive and decentralized strategy alleviates the burden on cloud resources, thereby catalyzing a surge in capable and intelligent sensing devices at the Edge. This approach is instrumental in accelerating the global advancement of artificial intelligence, marking a significant milestone in the AIoT domain.Brainchip’s holistic, distributed approach frees up the cloud and creates an explosion of capable, intelligent sensing devices on the Edge that accelerate global artificial intelligence.

Learn more about a groundbreaking solution called Temporal Event-based Neural Networks (TENNs) developed by BrainChip, which efficiently combines spatial and temporal convolutions to process sequential data like never before.

neural network

Profiles