How singulation and sorting of parcels can benefit from AI-powered robots

Manual singulation — a thing of the past?

The parcel flows experience wild fluctuations throughout the year, typically peaking around Black Friday and the Christmas period. Global parcel volumes have generally been growing at a rate of up to 25% per year.

The COVID-19 outbreak has also tested on-time order fulfillment. The accelerated reliance on e-commerce has increased demand for capacity. This, in turn, leads to more operational costs related not only to the need to hire more employees but also to the recruitment process itself. Periods that would normally be considered the best and most profitable from a sales point of view thus become the worst.

In addition to throughput requirements and spikes in demand, worker health and satisfaction are challenges for manual singulation and sorting. Employees can face boredom from repetitive tasks, as well as injury from handling irregular heavy items. This can lead to workers’ compensation expenses and decreased productivity. 

Vision and robotics to the rescue

The above challenges can be overcome by automating the singulation and sorting process, which can make costs more predictable and provide scalability for peaks in demand.

The most advanced automation today combines 3D machine vision, AI algorithms, and compatibility with major robot brands. It is also possible to measure the quality and success of a particular pick-and-place system. Let’s turn to concrete examples. How can a company benefit from implementing an automated system if it needs to singulate and sort large, unstructured loads of parcels?

For instance, Photoneo integrates 3D machine vision developed in house with algorithms that enable robots to pick more than 2,250 parcels per hour. The vision system provides accurate 3D data and enables precise localization that leads to a gripping accuracy of +/-3 mm.

The company said its system is based on a pre-trained neural network that can recognize parcels out of the box, without any training, for a picking success rate of 95%.

The remaining 5% is the result of the parcels’ mechanical properties and the material. For example, if an object has a wrinkled surface or is made of fabric, it may fall off the gripper and need to be picked again. Such objects are always successfully picked on the second attempt, according to Photoneo. It claimed it can achieve a cycle time of fewer than 1.5 seconds, and it is compatible with a range of robot brands.

Single-scan versus multi-pick mode

The performance speed is dependent on the chosen scanning mode. A single-scan mode makes a scan, processes the data, localizes an object, and sends a command to the robot to pick it. This process is repeated for every object. The processing delay is generally not more than 0.5 seconds.

Another option is a multi-pick mode, in which case the scanner/camera makes a scan, the system recognizes all pickable objects, and the robot picks them one after another without any interruption. The number of scans can be adjusted to the particular application. Because there is no processing delay in the multi-pick mode, the performance is faster, and the cycle time is limited only by the speed of the robot.

Recommended reading: End-to-end solution for 3D scanning and assembly of aircraft wings

Vision and intelligence

The quality of 3D data determines the success of an automated singulation and sorting solution. One could have the most intelligent system, but without good 3D data to work with and lean on, its output would be useless. A good 3D camera needs to provide high resolution and accuracy, large scanning volume, and depth of field, as well as a high scanning speed.

Other important factors are the ability to suppress ambient light and “plug-and-play” performance. If the deployed 3D camera offers all these features, the system gets enough data for AI to process them and successfully localize each object.

The most modern approach to AI-powered segmentation and localization of parcels is to use convolutional neural networks, which have made great progress in the past several years. These neural networks can recognize parcels, envelopes, and even bags of any shape, texture, and material, as well as their dimensions, position, and orientation.

The best solutions are based on algorithms that were trained on huge databases of objects and can therefore easily and quickly generalize and recognize new types of objects which they have never seen before. Wrinkles, deformations, and other irregularities should pose no obstacle to fast recognition.

After successful detection and localization, the robot gets a command to pick a particular object and then places it to a predefined location, such as onto a conveyor belt. 

Major challenges for singulation

Developers of robotic singulation systems face numerous challenges. A major problem for 3D vision is posed by surfaces that are shiny or reflecting, contain various patterns and pictures, or are black. Varieties in texture also cause difficulties. Parcels are usually piled up in an unstructured way, overlapping one another, which makes it hard to localize them.

One of the biggest challenges resides in the nature of bags – their shape is deformed, full of folds and wrinkles, which makes it extremely difficult for a robot gripper to pick them. These are the reasons why it is so essential to combine high-quality 3D vision with advanced and sophisticated AI algorithms – only this powerful combo can reliably tackle all the above challenges. 

Extending the range of applications

The utilization of an AI-powered automation solution does not end at simple singulation and sorting of parcels. If a 3D camera can scan moving scenes in high quality and at high speed, it is possible to measure packages on the fly and sort them on the basis of their size or other criteria.

For instance, Photoneo MotionCam-3D, which is able to capture objects moving up to 40 m/s, can reach a measurement precision of 1 cm and provides a depth map resolution of ~2 Mpx and 15 Million 3D points/sec.

Systems combining AI and 3D machine vision can also be used for unfolding or unwrapping wrinkled envelopes and parcels — and virtually make geometric transformations — to improve the readability of OCR for further processing. What some of these systems also enable is the sorting of parcels on the basis of barcodes.

The possible applications and capabilities of these systems develop and extend with the advances in AI and machine vision, but also with market demands that dictate the direction of this development.

Robotic singulation can increase safety, productivity, and reliability, as well as significantly cuts costs. Automation has become a crucial tool to optimize processes in logistics. Warehouses and distribution centers have struggled with retaining workers during experienced enormous increases in the flow of parcels around the holidays and now from e-commerce during the COVID-19 pandemic. The deployment of vision-guided, intelligent robots for singulation, and sorting of huge, unstructured flows of parcels is the way to respond to these challenges.

This article was originally published in The Robot Report.