While picking up and sorting objects is a task most people – and some animals – would think nothing of, this has proved a complex task for roboticists.

While there are already many robots handling objects in tightly controlled environments – such as factories – these robots will tend to perform just one repetitive task. Equipping a robot to grasp, recognise and sort is more of a challenge, particularly in uncontrolled, unfamiliar environments, such as in rubble following an earthquake.

In a step towards developing more versatile robots capable of assisting humans in a range of environments, a team of engineers at MIT has developed a robotic sorting system which grasps objects before recognising and sorting them.

“This can be applied to warehouse sorting, but also may be used to pick things from your kitchen cabinet or clear debris after an accident,” said Professor Alberto Rodriguez, an MIT mechanical engineer. “There are many situations where picking technologies could have an impact.”

First, the system determines the best way to pick up unknown objects from a pile using a gripper, slider and suction cup, and an “object agnostic” grasping algorithm. The researchers achieved this by showing the robot hundreds of images of piles of objects, the objects marked by the most appropriate grasping mechanism. The robot uses a deep neural network – commonly used for efficient image recognition – to determine the best grasping mechanism based on this information.

“We developed a system where, just by looking at a tote filled with objects, the robot knew how to predict which ones were graspable or suctionable, and which configuration of these picking behaviours was likely to be successful,” said Rodriguez.

“Once it was in the gripper, the object was much easier to recognise without all the clutter.”

The system identifies the object using a set of cameras and an image-recognition algorithm, which compares the object to a library of images and identify it. Once it has determined what sort of object it is holding, it can sort it appropriately.

In the 2017 Amazon Robotics Challenge – which invites teams to develop robotic systems which could assist in warehouse operations – the robotic system performed well, picking up objects with success rates of 54 per cent (grasping) and 75 per cent (suction), and recognising new objects with 100 per cent accuracy.

Rodriguez was granted an Amazon Research Award following his success and will work with the company to improve this technology, particularly its speed and responsiveness.

“Picking in unstructured environments is not reliable unless you add some level of reactiveness,” he said. “When humans pick, we do small adjustments as we are picking. Figuring out how to do this more responsive picking, I think, is one of the key technologies we’re interested in.”

While picking up and sorting objects is a task most people – and some animals – would think nothing of, this has proved a complex task for roboticists.

While there are already many robots handling objects in tightly controlled environments – such as factories – these robots will tend to perform just one repetitive task. Equipping a robot to grasp, recognise and sort is more of a challenge, particularly in uncontrolled, unfamiliar environments, such as in rubble following an earthquake.

In a step towards developing more versatile robots capable of assisting humans in a range of environments, a team of engineers at MIT has developed a robotic sorting system which grasps objects before recognising and sorting them.

“This can be applied to warehouse sorting, but also may be used to pick things from your kitchen cabinet or clear debris after an accident,” said Professor Alberto Rodriguez, an MIT mechanical engineer. “There are many situations where picking technologies could have an impact.”

First, the system determines the best way to pick up unknown objects from a pile using a gripper, slider and suction cup, and an “object agnostic” grasping algorithm. The researchers achieved this by showing the robot hundreds of images of piles of objects, the objects marked by the most appropriate grasping mechanism. The robot uses a deep neural network – commonly used for efficient image recognition – to determine the best grasping mechanism based on this information.

“We developed a system where, just by looking at a tote filled with objects, the robot knew how to predict which ones were graspable or suctionable, and which configuration of these picking behaviours was likely to be successful,” said Rodriguez.

“Once it was in the gripper, the object was much easier to recognise without all the clutter.”

The system identifies the object using a set of cameras and an image-recognition algorithm, which compares the object to a library of images and identify it. Once it has determined what sort of object it is holding, it can sort it appropriately.

In the 2017 Amazon Robotics Challenge – which invites teams to develop robotic systems which could assist in warehouse operations – the robotic system performed well, picking up objects with success rates of 54 per cent (grasping) and 75 per cent (suction), and recognising new objects with 100 per cent accuracy.

Rodriguez was granted an Amazon Research Award following his success and will work with the company to improve this technology, particularly its speed and responsiveness.

“Picking in unstructured environments is not reliable unless you add some level of reactiveness,” he said. “When humans pick, we do small adjustments as we are picking. Figuring out how to do this more responsive picking, I think, is one of the key technologies we’re interested in.”