Detecting Unwanted & Hazardous Objects with Vision AI

This article was discussed in our Next Byte podcast. 

The full article will continue below.

Foreign Object Detection In Food Production:

Manually spotting unwanted material on manufacturing lines can sometimes feel like looking for a needle in a haystack. But, the fast and accurate detection of foreign objects is a crucial step to prevent damage to products, machinery shutdowns, and food contamination.

Once the unwanted material is identified, an action takes place, such as stopping the production line, removing the materials, and sometimes even auditing recently processed products to verify there is no uncaught contamination.

Some variation of this process is required in many industries such as fishing, meat processing, and manufacturing.

How Vision AI Can Help (Creating a Solution with Vision AI)

There are many computer vision methods that can be utilized to detect hazardous or unwanted objects.

Anomaly Detection

Using anomaly detection architectures, such as Autoencoders or Long Short Term Memory (LSTM) networks can learn to find outliers. This is a good option if you don’t know what types of objects may show up.

Classification

Classification methods can learn to detect when specific objects are present in a video, but won't count how many or estimate where the object is located in the frame. This method typically requires a labeled dataset with examples of frames with and without the objects.

Object Detection & Instance Segmentation

Using object detection or instance segmentation methods not only detect when one or more objects has entered the frame, but also provide the object count and locations by returning coordinates. This requires a more intensive labeling procedure using rectangles or polygons to trace where objects are in the training dataset. 

These are a few examples for consideration. Choosing the right method for each use case depends on factors such as, detection accuracy needed, variations of unwanted objects, types of data available, and what action needs to be taken after detection.

How This Example Was Created:

Our example of tomatoes in the manufacturing process uses object detection to determine when the rock is detected. A trigger occurs that causes the alert text to be displayed. This trigger can also be used to send an alert, automatically stop the machinery, or provide location coordinates of the object for a pick and place robotic arm to remove it.

The method used in this example involved training an object detection model on variations of the objects we wanted to detect. In this case, rocks that can appear while processing tomatoes.

Recommended reading: Data Annotation for Computer Vision

Steps included:

1. Collect or create data (images containing the types of objects we want to detect)

2. Label objects with polygons for instance segmentation to result in more precise object detection (Using the Plainsight smart polygon selection tool)

3. Train a computer vision model for instance segmentation and object detection

4. Create a trigger that activates when an object is detected in the video frame. This can be used to take an action automatically, such as stopping a conveyor belt, sending alerts, or signaling a robotic arm with the location of where to grab an object.

Creating accurate models and datasets can be complicated and time consuming. Learn how plainsight.ai can help build better vision AI applications faster.

Full foreign object detection clip from plainsight.ai: