Computers Math

An interdisciplinary team consisting of researchers from the Center for Cognition and Sociality and the Data Science Group within the Institute for Basic Science (IBS) revealed a striking similarity between the memory processing of artificial intelligence (AI) models and the hippocampus of the human brain. This new finding provides a novel perspective on memory consolidation, which is a process that transforms short-term memories into long-term ones, in AI systems.
In the race towards developing Artificial General Intelligence (AGI), with influential entities like OpenAI and Google DeepMind leading the way, understanding and replicating human-like intelligence has become an important research interest. Central to these technological advancements is the Transformer model, whose fundamental principles are now being explored in new depth.
The key to powerful AI systems is grasping how they learn and remember information. The team applied principles of human brain learning, specifically concentrating on memory consolidation through the NMDA receptor in the hippocampus, to AI models.
The NMDA receptor is like a smart door in your brain that facilitates learning and memory formation. When a brain chemical called glutamate is present, the nerve cell undergoes excitation. On the other hand, a magnesium ion acts as a small gatekeeper blocking the door. Only when this ionic gatekeeper steps aside, substances are allowed to flow into the cell. This is the process that allows the brain to create and keep memories, and the gatekeeper’s (the magnesium ion) role in the whole process is quite specific.
The team made a fascinating discovery: the Transformer model seems to use a gatekeeping process similar to the brain’s NMDA receptor. This revelation led the researchers to investigate if the Transformer’s memory consolidation can be controlled by a mechanism similar to the NMDA receptor’s gating process.
In the animal brain, a low magnesium level is known to weaken memory function. The researchers found that long-term memory in Transformer can be improved by mimicking the NMDA receptor. Just like in the brain, where changing magnesium levels affect memory strength, tweaking the Transformer’s parameters to reflect the gating action of the NMDA receptor led to enhanced memory in the AI model. This breakthrough finding suggests that how AI models learn can be explained with established knowledge in neuroscience.
C. Justin LEE, who is a neuroscientist director at the institute, said, “This research makes a crucial step in advancing AI and neuroscience. It allows us to delve deeper into the brain’s operating principles and develop more advanced AI systems based on these insights.”
CHA Meeyoung, who is a data scientist in the team and at KAIST, notes, “The human brain is remarkable in how it operates with minimal energy, unlike the large AI models that need immense resources. Our work opens up new possibilities for low-cost, high-performance AI systems that learn and remember information like humans.”
What sets this study apart is its initiative to incorporate brain-inspired nonlinearity into an AI construct, signifying a significant advancement in simulating human-like memory consolidation. The convergence of human cognitive mechanisms and AI design not only holds promise for creating low-cost, high-performance AI systems but also provides valuable insights into the workings of the brain through AI models. More

Air conditioning and heating systems have contributed considerably to reducing mortality linked to extreme temperatures in Spain, according to a study led by the Barcelona Institute for Global Health (ISGlobal), a centre supported by the “la Caixa” Foundation. The findings, published in Environment International, provide valuable insights for designing policies to adapt to climate change.
Rising temperatures but lower mortality
Spain, like many parts of the world, has experienced rising temperatures in recent decades, with the average annual mean temperature increasing at an average rate of 0.36°C per decade. The warming trend is even more pronounced in the summer months (0.40°C per decade). Surprisingly, this increase in temperature has coincided with a progressive reduction in mortality associated with heat. In addition, cold-related mortality has also decreased.
“Understanding the factors that reduce susceptibility to extreme temperatures is crucial to inform health adaptation policies and to combat the negative effects of climate change,” says first author of the study, Hicham Achebak, researcher at ISGlobal and Inserm (France) and holder of a Marie Sklodowska-Curie Postdoctoral Fellowship from the European Commission.
Effective societal adaptations
In this study, Achebak and colleagues analysed the demographic and socioeconomic factors behind the observed reduction in heat and cold-related mortality, despite rising temperatures. They found that the increase in air conditioning (AC) prevalence in Spain was associated with a reduction in heat-related mortality, while the rise in heating prevalence was associated with a decrease in cold-related mortality. Specifically, AC was found to be responsible for about 28.6% of the decline in deaths due to heat and 31.5% of the decrease in deaths due to extreme heat between the late 1980s and the early 2010s. Heating systems contributed significantly, accounting for about 38.3% of the reduction in cold-related deaths and a substantial 50.8% decrease in extreme cold-related fatalities during the same period. The decrease in mortality due to cold would have been larger had there not been a demographic shift towards a higher proportion of people aged over 65, who are more susceptible to cold temperatures.
The authors conclude that the reduction in heat-related mortality is largely the result of the country’s socioeconomic development over the study period, rather than specific interventions such as heat-wave warning systems.

Four decades of data
For the statistical analysis, the research team collected data on daily mortality (all causes) and weather (temperature and relative humidity) for 48 provinces in mainland Spain and the Balearic Islands, between January 1980 and December 2018. These data were then linked to 14 indicators of context (demographic and socioeconomic variables such as housing, income and education) for these populations over the same period.
Implications for climate adaptation
The results of the study extend previous findings on heat-related mortality in Spain and underscore the importance of air conditioning and heating as effective adaptation measures to mitigate the effects of heat and cold. “However, we observed large disparities in the presence of AC across provinces. AC is still unaffordable for many Spanish households,” says Achebak.
The authors also point out that the widespread use of AC could further contribute to global warming depending on the source of electricity generation, which is why other cooling strategies, such as expanding green and blue spaces in cities, are also needed.
“Our findings have important implications for the development of adaptation strategies to climate change. They also inform future projections of the impact of climate change on human health,” concludes Joan Ballester, ISGlobal researcher and study coordinator. More

Artificial intelligence developed to model written language can be utilized to predict events in people’s lives. A research project from DTU, University of Copenhagen, ITU, and Northeastern University in the US shows that if you use large amounts of data about people’s lives and train so-called ‘transformer models’, which (like ChatGPT) are used to process language, they can systematically organize the data and predict what will happen in a person’s life and even estimate the time of death.
In a new scientific article, ‘Using Sequences of Life-events to Predict Human Lives’, published in Nature Computational Science, researchers have analyzed health data and attachment to the labour market for 6 million Danes in a model dubbed life2vec. After the model has been trained in an initial phase, i.e., learned the patterns in the data, it has been shown to outperform other advanced neural networks (see fact box) and predict outcomes such as personality and time of death with high accuracy.
“We used the model to address the fundamental question: to what extent can we predict events in your future based on conditions and events in your past? Scientifically, what is exciting for us is not so much the prediction itself, but the aspects of data that enable the model to provide such precise answers,” says Sune Lehmann, professor at DTU and first author of the article.
Predictions of time of death
The predictions from Life2vec are answers to general questions such as: ‘death within four years’? When the researchers analyze the model’s responses, the results are consistent with existing findings within the social sciences; for example, all things being equal, individuals in a leadership position or with a high income are more likely to survive, while being male, skilled or having a mental diagnosis is associated with a higher risk of dying. Life2vec encodes the data in a large system of vectors, a mathematical structure that organizes the different data. The model decides where to place data on the time of birth, schooling, education, salary, housing and health.
“What’s exciting is to consider human life as a long sequence of events, similar to how a sentence in a language consists of a series of words. This is usually the type of task for which transformer models in AI are used, but in our experiments we use them to analyze what we call life sequences, i.e., events that have happened in human life,” says Sune Lehmann.
Raising ethical questions
The researchers behind the article point out that ethical questions surround the life2vec model, such as protecting sensitive data, privacy, and the role of bias in data. These challenges must be understood more deeply before the model can be used, for example, to assess an individual’s risk of contracting a disease or other preventable life events.

“The model opens up important positive and negative perspectives to discuss and address politically. Similar technologies for predicting life events and human behaviour are already used today inside tech companies that, for example, track our behaviour on social networks, profile us extremely accurately, and use these profiles to predict our behaviour and influence us. This discussion needs to be part of the democratic conversation so that we consider where technology is taking us and whether this is a development we want,” says Sune Lehmann.
According to the researchers, the next step would be to incorporate other types of information, such as text and images or information about our social connections. This use of data opens up a whole new interaction between social and health sciences.
The research project
The research project ‘Using Sequences of Life-events to Predict Human Lives’ is based on labour market data and data from the National Patient Registry (LPR) and Statistics Denmark. The dataset includes all 6 million Danes and contains information on income, salary, stipend, job type, industry, social benefits, etc. The health dataset includes records of visits to healthcare professionals or hospitals, diagnosis, patient type and degree of urgency. The dataset spans from 2008 to 2020, but in several analyses, researchers focus on the 2008-2016 period and an age-restricted subset of individuals.
Transformer model
A transformer model is an AI, deep learning data architecture used to learn about language and other tasks. The models can be trained to understand and generate language. The transformer model is designed to be faster and more efficient than previous models and is often used to train large language models on large datasets.
Neural networks
A neural network is a computer model inspired by the brain and nervous system of humans and animals. There are many different types of neural networks (e.g. transformer models). Like the brain, a neural network is made up of artificial neurons. These neurons are connected and can send signals to each other. Each neuron receives input from other neurons and then calculates an output passed on to other neurons. A neural network can learn to solve tasks by training on large amounts of data. Neural networks rely on training data to learn and improve their accuracy over time. But once these learning algorithms are fine-tuned for accuracy, they are potent tools in computer science and artificial intelligence that allow us to classify and group data at high speed. One of the most well-known neural networks is Google’s search algorithm.  More

During a chemical reaction, molecules gain energy until they reach what’s known as the transition state — a point of no return from which the reaction must proceed. This state is so fleeting that it’s nearly impossible to observe it experimentally.
The structures of these transition states can be calculated using techniques based on quantum chemistry, but that process is extremely time-consuming. A team of MIT researchers has now developed an alternative approach, based on machine learning, that can calculate these structures much more quickly — within a few seconds.
Their new model could be used to help chemists design new reactions and catalysts to generate useful products like fuels or drugs, or to model naturally occurring chemical reactions such as those that might have helped to drive the evolution of life on Earth.
“Knowing that transition state structure is really important as a starting point for thinking about designing catalysts or understanding how natural systems enact certain transformations,” says Heather Kulik, an associate professor of chemistry and chemical engineering at MIT, and the senior author of the study.
Chenru Duan PhD ’22 is the lead author of a paper describing the work, which appears today in Nature Computational Science. Cornell University graduate student Yuanqi Du and MIT graduate student Haojun Jia are also authors of the paper.
Fleeting transitions
For any given chemical reaction to occur, it must go through a transition state, which takes place when it reaches the energy threshold needed for the reaction to proceed. The probability of any chemical reaction occurring is partly determined by how likely it is that the transition state will form.

“The transition state helps to determine the likelihood of a chemical transformation happening. If we have a lot of something that we don’t want, like carbon dioxide, and we’d like to convert it to a useful fuel like methanol, the transition state and how favorable that is determines how likely we are to get from the reactant to the product,” Kulik says.
Chemists can calculate transition states using a quantum chemistry method known as density functional theory. However, this method requires a huge amount of computing power and can take many hours or even days to calculate just one transition state.
Recently, some researchers have tried to use machine-learning models to discover transition state structures. However, models developed so far require considering two reactants as a single entity in which the reactants maintain the same orientation with respect to each other. Any other possible orientations must be modeled as separate reactions, which adds to the computation time.
“If the reactant molecules are rotated, then in principle, before and after this rotation they can still undergo the same chemical reaction. But in the traditional machine-learning approach, the model will see these as two different reactions. That makes the machine-learning training much harder, as well as less accurate,” Duan says.
The MIT team developed a new computational approach that allowed them to represent two reactants in any arbitrary orientation with respect to each other, using a type of model known as a diffusion model, which can learn which types of processes are most likely to generate a particular outcome. As training data for their model, the researchers used structures of reactants, products, and transition states that had been calculated using quantum computation methods, for 9,000 different chemical reactions.
“Once the model learns the underlying distribution of how these three structures coexist, we can give it new reactants and products, and it will try to generate a transition state structure that pairs with those reactants and products,” Duan says.

The researchers tested their model on about 1,000 reactions that it hadn’t seen before, asking it to generate 40 possible solutions for each transition state. They then used a “confidence model” to predict which states were the most likely to occur. These solutions were accurate to within 0.08 angstroms (one hundred-millionth of a centimeter) when compared to transition state structures generated using quantum techniques. The entire computational process takes just a few seconds for each reaction.
“You can imagine that really scales to thinking about generating thousands of transition states in the time that it would normally take you to generate just a handful with the conventional method,” Kulik says.
Modeling reactions
Although the researchers trained their model primarily on reactions involving compounds with a relatively small number of atoms — up to 23 atoms for the entire system — they found that it could also make accurate predictions for reactions involving larger molecules.
“Even if you look at bigger systems or systems catalyzed by enzymes, you’re getting pretty good coverage of the different types of ways that atoms are most likely to rearrange,” Kulik says.
The researchers now plan to expand their model to incorporate other components such as catalysts, which could help them investigate how much a particular catalyst would speed up a reaction. This could be useful for developing new processes for generating pharmaceuticals, fuels, or other useful compounds, especially when the synthesis involves many chemical steps.
“Traditionally all of these calculations are performed with quantum chemistry, and now we’re able to replace the quantum chemistry part with this fast generative model,” Duan says.
Another potential application for this kind of model is exploring the interactions that might occur between gases found on other planets, or to model the simple reactions that may have occurred during the early evolution of life on Earth, the researchers say.
The research was funded by the U.S. Office of Naval Research and the National Science Foundation. More

Research appearing in ACS Nano, a premier journal on nanoscience and nanotechnology, reveals the ballistic movement of electrons in graphene in real time.
The observations, made at the University of Kansas’ Ultrafast Laser Lab, could lead to breakthroughs in governing electrons in semiconductors, fundamental components in most information and energy technology.
“Generally, electron movement is interrupted by collisions with other particles in solids,” said lead author Ryan Scott, a doctoral student in KU’s Department of Physics & Astronomy. “This is similar to someone running in a ballroom full of dancers. These collisions are rather frequent — about 10 to 100 billion times per second. They slow down the electrons, cause energy loss and generate unwanted heat. Without collisions, an electron would move uninterrupted within a solid, similar to cars on a freeway or ballistic missiles through air. We refer to this as ‘ballistic transport.'”
Scott performed the lab experiments under the mentorship of Hui Zhao, professor of physics & astronomy at KU. They were joined in the work by former KU doctoral student Pavel Valencia-Acuna, now a postdoctoral researcher at the Northwest Pacific National Laboratory.
Zhao said electronic devices utilizing ballistic transport could potentially be faster, more powerful and more energy efficient.
“Current electronic devices, such as computers and phones, utilize silicon-based field-effect transistors,” Zhao said. “In such devices, electrons can only drift with a speed on the order of centimeters per second due to the frequent collisions they encounter. The ballistic transport of electrons in graphene can be utilized in devices with fast speed and low energy consumption.”
The KU researchers observed the ballistic movement in graphene, a promising material for next-generation electronic devices. First discovered in 2004 and awarded the Nobel Prize in Physics in 2010, graphene is made of a single layer of carbon atoms forming a hexagonal lattice structure — somewhat like a soccer net.

“Electrons in graphene move as if their ‘effective’ mass is zero, making them more likely to avoid collisions and move ballistically,” Scott said. “Previous electrical experiments, by studying electrical currents produced by voltages under various conditions, have revealed signs of ballistic transport. However, these techniques aren’t fast enough to trace the electrons as they move.”
According to the researchers, electrons in graphene (or any other semiconductor) are like students sitting in a full classroom, where students can’t freely move around because the desks are full. The laser light can free electrons to momentarily vacate a desk, or ‘hole’ as physicists call them.
“Light can provide energy to an electron to liberate it so that it can move freely,” Zhao said. “This is similar to allowing a student to stand up and walk away from their seat. However, unlike a charge-neutral student, an electron is negatively charged. Once the electron has left its ‘seat,’ the seat becomes positively charged and quickly drags the electron back, resulting in no more mobile electrons — like the student sitting back down.”
Because of this effect, the super-light electrons in graphene can only stay mobile for about one-trillionth of a second before falling back to its seat. This short time presents a severe challenge to observing the movement of the electrons. To address this problem, the KU researchers designed and fabricated a four-layer artificial structure with two graphene layers separated by two other single-layer materials, molybdenum disulphide and molybdenum diselenide.
“With this strategy, we were able to guide the electrons to one graphene layer while keeping their ‘seats’ in the other graphene layer,” Scott said. “Separating them with two layers of molecules, with a total thickness of just 1.5 nanometers, forces the electrons to stay mobile for about 50-trillionths of a second, long enough for the researchers, equipped with lasers as fast as 0.1 trillionth of a second, to study how they move.”
The researchers use a tightly focused laser spot to liberate some electrons in their sample. They trace these electrons by mapping out the “reflectance” of the sample, or the percentage of light they reflect.

“We see most objects because they reflect light to our eyes,” Scott said. “Brighter objects have larger reflectance. On the other hand, dark objects absorb light, which is why dark clothes become hot in the summer. When a mobile electron moves to a certain location of the sample, it makes that location slightly brighter by changing how electrons in that location interact with light. The effect is very small — even with everything optimized, one electron only changes the reflectance by 0.1 part per million.”
To detect such a small change, the researchers liberated 20,000 electrons at once, using a probe laser to reflect off the sample and measure this reflectance, repeating the process 80 million times for each data point. They found the electrons on average move ballistically for about 20-trillionths of a second with a speed of 22 kilometers per second before running into something that terminates their ballistic motion.
The research was funded by a grant from the Department of Energy under the program of Physical Behavior of Materials.
Zhao said currently his lab is working to refine their material design to guide electrons more efficiently to the desired graphene layer, and trying to find ways to make them move longer distances ballistically. More

Artificial Intelligence (AI) and 3D images of the human tongue have revealed that the surface of our tongues are unique to each of us, new findings suggest.
The results offer an unprecedented insight into the biological make-up of our tongue’s surface and how our sense of taste and touch differ from person to person.
The research has huge potential for discovering individual food preferences, developing healthy food alternatives and early diagnosis of oral cancers in the future, experts say.
The human tongue is a highly sophisticated and complex organ. It’s surface is made up of hundreds of small buds — known as papillae — that assist with taste, talking and swallowing.
Of these numerous projections, the mushroom-shaped fungiform papillae hold our taste buds whereas the crown-shaped filiform papillae give the tongue its texture and sense of touch.
The taste function of our fungiform papillae has been well researched but little is known about the difference in shape, size and pattern of both forms of papillae between individuals.
A team of researchers led by the University of Edinburgh’s School of Informatics, in collaboration with the University of Leeds, trained AI computer models to learn from three-dimensional microscopic scans of the human tongue, showing the unique features of papillae.

They fed the data from over two thousand detailed scans of individual papillae — taken from silicone moulds of fifteen people’s tongues — to the AI tool.
The AI models were designed to gain a better understanding of individual features of the participant’s papillae and to predict the age and gender of each volunteer.
The team used small volumes of data to train the AI models about the different features of the papillae, combined with a significant use of topology — an area of mathematics which studies how certain spaces are structured and connected.
This enabled the AI tool to predict the type of papillae to within 85 per cent accuracy and to map the position of filiform and fungiform papillae on the tongue’s surface.
Remarkably, the papillae were also found to be distinctive across all fifteen subjects and individuals could be identified with an accuracy of 48 per cent from a single papilla.
The findings have been published in the journal Scientific Reports.

The study received funding from the United Kingdom Research and Innovation (UKRI) CDT in Biomedical AI and European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program.
Senior author, Professor Rik Sakar, Reader, School of Informatics, University of Edinburgh, said:
“This study brings us closer to understanding the complex architecture of tongue surfaces.
“We were surprised to see how unique these micron-sized features are to each individual. Imagine being able to design personalized food customised to the conditions of specific people and vulnerable populations and thus ensure they can get proper nutrition whilst enjoying their food.
Professor Sakar, added:
“We are now planning to use this technique combining AI with geometry and topology to identify micron-sized features in other biological surfaces. This can help in early detection and diagnosis of unusual growths in human tissues.
Lead author, Rayna Andreeva, PhD student at the Centre for Doctoral Training (CDT) in Biomedical AI, University of Edinburgh, said:
“It was remarkable that the features based on topology worked so well for most types of analysis, and they were the most distinctive across individuals. This needs further study not only for the papillae, but also for other kinds of biological surfaces and medical conditions.” More

While screen time is generally known to affect sleep, new research suggests that interactive engagement, such as texting friends or playing video games, delays and reduces the time spent asleep to a greater extent than passive screen time, like watching television — especially for teens.
The research, which published today (Dec. 13) in the Journal of Adolescent Health, demonstrates that adolescents at age 15 who used screens to communicate with friends or play video games in the hour before bed took 30 minutes longer to fall asleep than if they had refrained from interactive screen time. But it wasn’t just interactive screen time before bed that affected kids’ sleep, researchers said. For each hour during the day that kids spent playing video games beyond their usual amount, their sleep was delayed by about 10 minutes.
“If teens typically play video games for an hour each day, but one day a new game comes out and they play for four hours, that’s three additional hours more than they typically play,” said David Reichenberger, postdoctoral scholar at Penn State and lead author on the study. “So, that means they could have 15 minutes of delayed sleep timing that night. For a child, losing 15 minutes of sleep at night is significant. It’s especially difficult when they have to get up in the morning for school; if they’re delaying their sleep, they can’t make up for it in the morning. Without adequate sleep, kids are at increased risk of obesity, as well as impaired cognition, emotion regulation and mental health.”
The team assessed the daytime screen-based activities of 475 adolescents using daily surveys for three or more days. They asked the teens how many hours they had spent that day communicating with friends by email, instant messaging, texting on the phone or through social media sites. They also asked the kids how many hours they spent playing video games, surfing the internet and watching television or videos. Finally, the researchers asked if the adolescents had participated in any of these activities in the hour before bed.
Next, the team used accelerometers to measure the adolescents’ sleep duration for one week. Reichenberger explained that the devices, typically worn on the wrist, measures a person’s movements. “When the participant is least active, we can infer that they are likely asleep,” Reichenberger said. “It’s more accurate than asking them how many hours they slept.”
The researchers found that the teens spent an average of two hours per day communicating with friends via email, instant messaging, texting on the phone or through social media. They spent slightly less time — about 1.3 hours per day — playing video games, less than an hour per day surfing the internet and about 1.7 hours per day watching television or videos. In the hour before bed, the children communicated or played video games via a phone, computer or tablet 77% of the time and watched television or movies 69% of the time.
Overall, the adolescents slept for an average of 7.8 hours per night. For every hour throughout the day that they used screens to communicate with friends, they fell asleep about 11 minutes later on average. For every hour that they used screens to play video games, they fell asleep about 9 minutes later. Those who talked, texted or played games on a device in the hour before bed lost the most sleep: their sleep onset was about 30 minutes later.

Interestingly, Reichenberger said, the team found no significant associations between passive screen-based activities and subsequent sleep, like browsing the internet and watching television, videos and movies.
“It could be that these more passive activities are less mentally stimulating than interactive activities, like texting and video game playing.” said Anne-Marie Chang, associate professor of biobehavioral health and study co-author.
What can parents do to help protect their teens’ sleep?
“It’s a tricky situation,” Chang said. “These tools are really important to everyone nowadays, so it’s hard to put a limit on them, but if you’re really looking out for an adolescent’s health and well-being, then you might consider limiting the more interactive activities, especially in the hour before bed.”
Other authors on the paper include Lindsay Master, researcher, Penn State; Orfeu Buxton, the Elizabeth Fenton Susman Professor of Biobehavioral Health, Penn State; Gina Marie Mathew, postdoctoral associate, Stony Brook University; Lauren Hale, professor of family, population and preventive medicine, Stony Brook University; and Cynthia Snyder, assistant professor of nursing, Pennsylvania College of Health Sciences.
The National Institutes of Health and National Aeronautics and Space Administration supported this research. More

Neuroengineer Silvestro Micera develops advanced technological solutions to help people regain sensory and motor functions that have been lost due to traumatic events or neurological disorders. Until now, he had never before worked on enhancing the human body and cognition with the help of technology.
Now in a study published in Science Robotics, Micera and his team report on how diaphragm movement can be monitored for successful control of an extra arm, essentially augmenting a healthy individual with a third — robotic — arm.
“This study opens up new and exciting opportunities, showing that extra arms can be extensively controlled and that simultaneous control with both natural arms is possible,” says Micera, Bertarelli Foundation Chair in Translational Neuroengineering at EPFL, and professor of Bioelectronics at Scuola Superiore Sant’Anna.
The study is part of the Third-Arm project, previously funded by the Swiss National Science Foundation (NCCR Robotics), that aims to provide a wearable robotic arm to assist in daily tasks or to help in search and rescue. Micera believes that exploring the cognitive limitations of third-arm control may actually provide gateways towards better understanding of the human brain.
Micera continues, “The main motivation of this third arm control is to understand the nervous system. If you challenge the brain to do something that is completely new, you can learn if the brain has the capacity to do it and if it’s possible to facilitate this learning. We can then transfer this knowledge to develop, for example, assistive devices for people with disabilities, or rehabilitation protocols after stroke.”
“We want to understand if our brains are hardwired to control what nature has given us, and we’ve shown that the human brain can adapt to coordinate new limbs in tandem with our biological ones,” explains Solaiman Shokur, co-PI of the study and EPFL Senior Scientist at the Neuro-X Institute. “It’s about acquiring new motor functions, enhancement beyond the existing functions of a given user, be it a healthy individual or a disabled one. From a nervous system perspective, it’s a continuum between rehabilitation and augmentation.”
To explore the cognitive constraints of augmentation, the researchers first built a virtual environment to test a healthy user’s capacity to control a virtual arm using movement of his or her diaphragm. They found that diaphragm control does not interfere with actions like controlling one’s physiological arms, one’s speech or gaze.

In this virtual reality setup, the user is equipped with a belt that measures diaphragm movement. Wearing a virtual reality headset, the user sees three arms: the right arm and hand, the left arm and hand, and a third arm between the two with a symmetric, six-fingered hand.
“We made this hand symmetric to avoid any bias towards either the left or the right hand,” explains Giulia Dominijanni, PhD student at EPFL’s Neuro-X Institute.
In the virtual environment, the user is then prompted to reach out with either the left hand, the right hand, or in the middle with the symmetric hand. In the real environment, the user holds onto an exoskeleton with both arms, which allows for control of the virtual left and right arms. Movement detected by the belt around the diaphragm is used for controlling the virtual middle, symmetric arm. The setup was tested on 61 healthy subjects in over 150 sessions.
“Diaphragm control of the third arm is actually very intuitive, with participants learning to control the extra limb very quickly,” explains Dominijanni. “Moreover, our control strategy is inherently independent from the biological limbs and we show that diaphragm control does not impact a user’s ability to speak coherently.”
The researchers also successfully tested diaphragm control with an actual robotic arm, a simplified one that consists of a rod that can be extended out, and back in. When the user contracts the diaphragm, the rod is extended out. In an experiment similar to the VR environment, the user is asked to reach and hover over target circles with her left or right hand, or with the robotic rod.
Besides the diaphragm, but not reported in the study, vestigial ear muscles have also been tested for feasibility in performing new tasks. In this approach, a user is equipped with ear sensors and trained to use fine ear muscle movement to control the displacement of a computer mouse.
“Users could potentially use these ear muscles to control an extra limb,” says Shokur, emphasizing that these alternative control strategies may help one day for the development of rehabilitation protocols for people with motor deficiencies.
Part of the third arm project, previous studies regarding the control of robotic arms have been focused on helping amputees. The latest Science Robotics study is a step beyond repairing the human body towards augmentation.
“Our next step is to explore the use of more complex robotic devices using our various control strategies, to perform real-life tasks, both inside and outside of the laboratory. Only then will we be able to grasp the real potential of this approach,” concludes Micera. More