Robotic picking of bags - not so utopic after all

Bin picking – i.e. robotic detecting and picking of randomly arranged objects from within a bin on the basis of CAD models – is a common type of application we deal with even in challenging industrial environments. The Bin Picking Studio is an all-in-one picking solution for robotic integrators, who do not need to know anything about computer vision to set it up. Its robust features made it possible to enter basically all industrial sectors – the automotive, manufacturing, logistics, and others.

However, we gradually started to receive requests from our customers for a solution that would enable them to pick objects of irregular shapes and in random positions within the bin. We set to work on algorithms that would enable us to pick these kinds of objects and after some time of intensive endeavor, our AI-driven solutions first saw the light of day. Thanks to sophisticated algorithms, they enabled us to localize and pick objects of different sizes, various textures, and arbitrary shapes including deformations. What’s more, the objects do not need to be stacked in ordered patterns but can be placed absolutely randomly and the robot is still able to recognize and pick them. The AI-powered solutions enabled by a combination of precise 3D vision and advanced AI algorithms showed the world the immense opportunities for automation of material handling, manufacturing, logistics, and many more.

And it was logistics that showed us there was one last gap left to cover all kinds of objects and applications – and this resided in the fact that goods are not always transported in solid boxes but some of them come in bags. These pose a great challenge for automated sorting as they do not keep their shape – they are flexible, wrinkled, easily deformable, and thus very difficult to localize. We decided to utilize our experience, knowledge, and skills to push the performance of our AI-powered solutions even further and “teach” them detect and pick bags.

The recognition of boundaries between bags that are chaotically placed in a container can often be difficult even for humans. The task gets even more complicated for transparent bags. Difficult – but possible. We developed a network that is able to recognize bags, sometimes better than we are. They may be overlapping and blocking each other from being picked but our system can handle this with ease and won’t allow the robot to grab fully or partly covered bags.

Another common difficulty arises from the nature of the material bags are made of – they are full of folds and wrinkles which often causes them to fall off the gripper just after being successfully detected and lifted. We recommend using a vacuum gripper with feedback to prevent such failures.