Researchers at the Max Planck Institute for Intelligent Systems in Germany have created a fingertip sensor that allows a robot, including a robotic prosthesis, to very sensitively gauge how much force is applied to it. The system is based on a camera that is mounted inside the rubbery robot finger. The camera observes the internal walls of a hollow area within the finger, which is lit using a ring of LEDs, and a neural network interprets the camera feed to accurately calculate the location, magnitude, and direction of the force applied to the finger. The system aims to offer a level of sensitivity to robotic fingers that is similar to that of our own skin, and the technology could have various medical applications, including haptic feedback for robotic prostheses and as a safety mechanism for future assistive robotic technologies.
The robot butlers (or doctors) we were all promised have not come to pass, yet, but researchers are still working steadily on increasing the capabilities of robotic systems. Eventually, assistive robotic technologies may become more mainstream. With our aging population, robots that can assist with daily living and medical care, such as nursing, will be very useful, and robotic prostheses are poised to provide unprecedented utility for amputees and those with other mobility issues.
However, if robots are to interact closely with humans it is important that they don’t cause injury or damage. A large part of this involves applying an appropriate amount of force to perform a specific task. This latest technology aims to allow a robot to precisely gauge the force applied by or to a rubbery robotic finger.
The finger consists of an elastomer coating mounted on a ‘skeleton’ frame. In a hollow space within, the researchers mounted a fisheye camera lens and illuminated the space with a ring of colorful LEDs. When the finger is deformed by touching something, the colorful pattern inside changes. Cleverly, the researchers used a neural network that learned how to interpret the changing patterns and infer the force that is being applied with great sensitivity.
“We achieved this excellent sensing performance through the innovative mechanical design of the shell, the tailored imaging system inside, automatic data collection, and cutting-edge deep learning,” said Georg Martius, a researcher involved in the study. “Our unique hybrid structure of a soft shell enclosing a stiff skeleton ensures high sensitivity and robustness. Our camera can detect even the slightest deformations of the surface from one single image,” added Huanbo Sun, another researcher involved in the project.
See a video about the technology below.
Study in Nature Machine Intelligence: A soft thumb-sized vision-based sensor with accurate all-round force perception
