Hand-eye bin picking

Bin picking, i.e. robotic picking of objects with random poses from a bin, is one of the most common robotic tasks yet also one that poses some of the greatest technological challenges. A robot needs to be equipped with exceptional vision and robotic intelligence to be able to localize each part in a bin, navigate to it without colliding with its environment or other parts, then pick it, and safely place it to another location in an aligned position.

There are two ways to mount a 3D vision system in the robotic cell, each having its advantages and each being more suitable for certain applications:

A. Extrinsic calibration

B. Hand-eye calibration

Extrinsic calibration

Extrinsic calibration is used in setups where the scanner is mounted in a fixed, stationary position in the robotic cell, usually above the bin. The scanner must not be moved in relation to the robot after the system has been calibrated but it does not need to be stationary in relation to the robotic cell. 

Photoneo offers five models of the PhoXi 3D Scanner (XS – XL), each providing a different scanning volume ranging from 16 centimeters up to 4 meters. As a rule of thumb, the more room is required for the bin picking application – including the space for robot movement, the size of the bin and parts, etc. – the larger model of the PhoXi 3D Scanner is required.

Extrinsic calibration is done with any suitable ball attached to the endpoint of the robotic arm or to the gripper. The ball needs to be made of a material that is appropriate for scanning, that means smooth and not too reflective.

Extrinsic calibration: The 3D scanner is mounted in a stationary position. 

 

Hand-eye picking

Hand-eye calibration allows you to mount the 3D vision system directly onto the robotic arm. For a successful calibration, the scanner must be mounted behind the very last joint (e.g. on the gripper). Any changes made to the scanner’s position after the calibration renders the calibration matrix invalid and the whole calibration procedure must be carried out again.

Hand-eye calibration is done with a marker pattern – a flat sheet of paper (or another material) with a special pattern recognized by Photoneo PhoXi 3D scanners.

Hand-eye calibration: The 3D scanner is mounted on the robotic arm. 

 

Advantages of hand-eye picking

Hand-eye picking offers a number of advantages over the traditional setup. If you have a large bin and a scanner mounted above it with a fixed viewpoint, you will not be able to scan the bin with a small scanner. If you mount a small scanner directly onto the robotic arm, you can come closer to the contents of the bin with the scanner and choose which part of the bin to scan.

Another advantage of hand-eye bin picking comes in case you have two bins. One scanner mounted firmly in a fixed position will not be able to make scans of both bins – this would require two scanners at least. In contrast to this, a scanner mounted on the robotic arm will move from one bin to another and make scans of their contents from appropriate distance and angle. It is thus able to cover a larger area than extrinsic calibration.

Hand-eye bin picking may also eliminate the need to darken the room where the robotic cell is placed. The ambient light coming from a window might pose serious challenges to the deployed 3D vision system and often requires special effort to darken the whole room. A scanner attached to the robot can make scans of a bin from one side first and then from another, minimizing the need to make any special adjustments to the environment.

It can also happen that the 3D vision system itself casts shadows on the bin and inhibits a high-quality acquisition of the scene. This problem is usually solved by making a compromise and finding the most optimal position for the scanner in relation to the bin or by manually rearranging the parts within the bin so that the vision system captures them all in the end. Hand-eye bin picking eliminates this problem as it enables the scanner to “look” at the scene from any angle and from any position.

Similarly, the hand-eye approach is an optimal solution for picking parts that are placed close to the walls of the bin. The walls – and even the parts themselves – may cast shadows and inhibit the recognition of the parts so mounting the vision system onto the robotic arm will allow you to approach and pick the parts from an appropriate angle. 

While hand-eye bin picking provides numerous advantages over extrinsic calibration, it is also fair to say that a scanner attached to the robotic arm may restrict the robot’s movements. Bearing this limitation in mind, it is advisable to choose a smaller scanner. In the case of Photoneo PhoXi 3D Scanner family, the appropriate models are XS, S, M, and L, covering a scanning range from 16 cm up to 2 m.

Another important fact is that the robot needs to stay still during scan acquisition, which may prolong the cycle time. Therefore, the hand-eye approach is not suitable for time-critical applications.

If you want to automate your application but are not sure what type of bin picking solution would be best suited for your project, contact Photoneo and let’s discuss it. We will be happy to assist you.