Robot Simulation Closer to in-Hand Object Manipulation

A team of scientists in the Human Brain Project is using the EBRAINS research infrastructure to learn more about how the brain coordinates complex hand movements. The Human Brain Project says that whilst these tasks may seem simple, in-hand object manipulation is actually very complex and engages a large-scale brain network encompassing sensory, association, and motor regions.

The Human Brain Project (HBP) is the largest brain science project in Europe and stands among the biggest research projects ever funded by the European Union. At the interface of neuroscience and information technology, the HBP investigates the brain and its diseases with the help of highly advanced methods from computing, neuroinformatics and artificial intelligence, and drives innovation in fields like brain-inspired computing and neurorobotics.

A team of scientists in the Human Brain Project is implementing this brain network as a biologically constrained recurrent convolutional neural network (RCNN) on EBRAINS. This type of network is often used to model sequence data. The network’s ability to learn sequences of states – for example, how a sequence of limb positions evolves when a certain force is applied to a joint – is especially relevant to robotics.

The RCNN was tasked with generating a simple digit configuration – to join fingers with the thumb – which it successfully performed after 2,000 iterations of self-generated experience. “The RCNN requires hours of simulated experience to train”, says Mario Senden, one of the lead scientists. “The EBRAINS high-performance computing infrastructure allows us to speed up the process immensely and train the network in just a few minutes”.

The design of the RCNN was informed by the EBRAINS Multilevel Human Brain Atlas, which allowed the researchers to identify the connection profile of brain regions involved in complex hand movements. The RCNN was then trained in-silico on supercomputers at the Swiss National Supercomputing Centre, which is part of the EBRAINS high-performance computing infrastructure.

The team of scientists is now trying to train the network to perform in-hand object manipulation. “Our aim is to better understand the brain through biologically-inspired functional models and to provide brain-based alternatives to modern engineering solutions”, says Rainer Goebel, one of the head researchers.

Work is also being done to implement the Shadowhand robot – which performed the digit configurations in a simulator – on the EBRAINS Neurorobotics Platform. “The unique combination of interoperable services on EBRAINS will allow us to break new ground in cognitive computational neuroscience”, says Paweł Świeboda, CEO of EBRAINS and Director General of the HBP.

Katrin Amunts, the Scientific Director of the HBP emphasizes: “The workflow shown here incorporates a range of complementary EBRAINS services, including the multilevel human brain atlas, high-performance computing infrastructure, and, eventually, the Neurorobotics Platform. EBRAINS is the only research infrastructure in Europe that offers this powerful combination of services”.

EBRAINS is a new digital research infrastructure, created by the EU-funded Human Brain Project, to foster brain-related research and to help translate the latest scientific discoveries into innovation in medicine and industry, for the benefit of patients and society. All academic researchers have open access to EBRAINS’ state-of-the-art services and industry researchers have access to the platform under specific agreements.

You can find more information about the Human Brain Project and EBRAINS on its website.

Stay up to date with the most recent automation, machine vision, and robotics news on MVPro. Read the best stories every Friday with our newsletter.