Aurelia Butler

What features does a robotic guide dog need? Ask the blind, say the authors of an award-winning paper. Led by researchers at the University of Massachusetts Amherst, a study identifying how to develop robot guide dogs with insights from guide dog users and trainers won a Best Paper Award at CHI 2024: Conference on Human Factors in Computing Systems (CHI).
Guide dogs enable remarkable autonomy and mobility for their handlers. However, only a fraction of people with visual impairments have one of these companions. The barriers include the scarcity of trained dogs, cost (which is $40,000 for training alone), allergies and other physical limitations that preclude caring for a dog.
Robots have the potential to step in where canines can’t and address a truly gaping need — if designers can get the features right.
“We’re not the first ones to develop guide-dog robots,” says Donghyun Kim, assistant professor in the UMass Amherst Manning College of Information and Computer Science (CICS) and one of the corresponding authors of the award-winning paper. “There are 40 years of study there, and none of these robots are actually used by end users. We tried to tackle that problem first so that, before we develop the technology, we understand how they use the animal guide dog and what technology they are waiting for.”
The research team conducted semistructured interviews and observation sessions with 23 visually impaired dog-guide handlers and five trainers. Through thematic analysis, they distilled the current limitations of canine guide dogs, the traits handlers are looking for in an effective guide and considerations to make for future robotic guide dogs.
One of the more nuanced themes that came from these interviews was the delicate balance between robot autonomy and human control. “Originally, we thought that we were developing an autonomous driving car,” says Kim. They envisioned that the user would tell the robot where they want to go and the robot would navigate autonomously to that location with the user in tow.
This is not the case.

The interviews revealed that handlers do not use their dog as a global navigation system. Instead, the handler controls the overall route while the dog is responsible for local obstacle avoidance. However, even this isn’t a hard-and-fast rule. Dogs can also learn routes by habit and may eventually navigate a person to regular destinations without directional commands from the handler.
“When the handler trusts the dog and gives more autonomy to the dog, it’s a bit delicate,” says Kim. “We cannot just make a robot that is fully passive, just following the handler, or just fully autonomous, because then [the handler] feels unsafe.”
The researchers hope this paper will serve as a guide, not only in Kim’s lab, but for other robot developers as well. “In this paper, we also give directions on how we should develop these robots to make them actually deployable in the real world,” says Hochul Hwang, first author on the paper and a doctoral candidate in Kim’s robotics lab.
For instance, he says that a two-hour battery life is an important feature for commuting, which can be an hour on its own. “About 90% of the people mentioned the battery life,” he says. “This is a critical part when designing hardware because the current quadruped robots don’t last for two hours.”
These are just a few of the findings in the paper. Others include: adding more camera orientations to help address overhead obstacles; adding audio sensors for hazards approaching from the occluded regions; understanding ‘sidewalk’ to convey the cue, “go straight,” which means follow the street (not travel in a perfectly straight line); and helping users get on the right bus (and then find a seat as well).
The researchers say this paper is a great starting point, adding that there is even more information to unpack from their 2,000 minutes of audio and 240 minutes of video data.

Winning the Best Paper Award was a distinction that put the work in the top 1% of all papers submitted to the conference.
“The most exciting aspect of winning this award is that the research community recognizes and values our direction,” says Kim. “Since we don’t believe that guide dog robots will be available to individuals with visual impairments within a year, nor that we’ll solve every problem, we hope this paper inspires a broad range of robotics and human-robot interaction researchers, helping our vision come to fruition sooner.”
Other researchers who contributed to the paper include:
Ivan Lee, associate professor in CICS and a co-corresponding author of the article along with Donghyun, an expert in adaptive technologies and human-centered design; Joydeep Biswas, associate professor at the University of Texas Austin, who brought his experience in creating artificial intelligence (AI) algorithms that allow robots to navigate through unstructured environments; Hee Tae Jung, assistant professor at Indiana University, who brought his expertise in human factors and qualitative research to participatory study with people with chronic conditions; and Nicholas Giudice, a professor at the University of Maine who is blind and provided valuable insight and interpretation of the interviews.
Ultimately, Kim understands that robotics can do the most good when scientists remember the human element. “My Ph.D. and postdoctoral research is all about how to make these robots work better,” Kim adds. “We tried to find [an application that is] practical and something meaningful for humanity.” More

AI-generated videos for short messages are only as well received as manually created ones if they are edited by humans.
News organizations — including Bloomberg, Reuters, and The Economist — have been using AI powered video services to meet growing audience demand for audio-visual material. A study recently published in the journal Journalism now shows that the automated production of news videos is better with human supervision.
Technology providers like Wochit and Moovly are allowing publishers to mass produce videos at scale. But what do audiences think of the results? Researchers led by LMU communication scientist Professor Neil Thurman have found that only automated videos which have been post-edited by humans were as well liked as fully human-made videos.
“Our research shows that, on average, news consumers liked short-form, automated news videos as much as manually made ones, as long as the automation process involved human supervision,” says Neil Thurman, from LMU’s Department of Media and Communication.
Together with Dr. Sally Stares (London School of Economic) and Dr. Michael Koliska (Georgetown University), Thurman evaluated the reactions of 4,200 UK news consumers to human-made, highly-automated, and partly-automated videos that covered a variety of topics including Christiano Ronaldo, Donald Trump, and the Wimbledon tennis championships. The partly-automated videos were post-edited by humans after the initial automation process.
The results show that there were no significant differences in how much news audiences liked the human-made and partly-automated videos overall. By contrast, highly-automated videos were liked significantly less. In other words, the results show that news video automation is better with human supervision.
According to Thurman, “one key takeaway of the study is that video automation output may be best when it comes in a hybrid form, meaning a human-machine collaboration. Such hybridity involves more human supervision, ensuring that automated video production maintains quality standards while taking advantage of computers’ strengths, such as speed and scale.” More

Scientists at the University of Oregon have discovered that colonies of gelatinous sea animals swim through the ocean in giant corkscrew shapes using coordinated jet propulsion, an unusual kind of locomotion that could inspire new designs for efficient underwater vehicles.
The research involves salps, small creatures that look similar to jellyfish that take a nightly journey from the depths of the ocean to the surface. Observing that migration with special cameras helped UO researchers and their colleagues capture the macroplankton’s graceful, coordinated swimming behavior.
“The largest migration on the planet happens every single night: the vertical migration of planktonic organisms from the deep sea to the surface,” said Kelly Sutherland, an associate professor in biology at the UO’s Oregon Institute of Marine Biology, who led the research. “They’re running a marathon every day using novel fluid mechanics. These organisms can be platforms for inspiration on how to build robots that efficiently traverse the deep sea.”
The researchers’ findings were published May 15 in the journal Science Advances. The study included collaborations from Louisiana Universities Marine Consortium, University of South Florida, Roger Williams University, Marine Biological Laboratory and Providence College.
Despite looking similar to jellyfish, salps are barrel-shaped, watery macroplankton that are more closely related to vertebrates like fish and humans, said Alejandro Damian-Serrano, an adjunct professor in biology at the UO. They live far from shore and can live either as solitary individuals or operate in colonies, he said. Colonies consist of hundreds of individuals linked in chains that can be up to several meters long.
“Salps are really weird animals,” Damian-Serrano said. “While their common ancestor with us probably looked like a little boneless fish, their lineage lost a lot of those features and magnified others. The solitary individuals behave like this mothership that asexually breeds a chain of individual clones, cojoined together to produce a colony.”
But the most unique thing about these ocean creatures was found during the researchers’ ocean expeditions: their swimming techniques.

Exploring off the coast of Kailua-Kona, Hawaii, Sutherland and her team developed specialized 3D camera systems to bring their lab underwater. They conducted daytime scuba dives, “immersed in infinite blue,” as Damian-Serrano described, for high visibility investigations.
They also performed nighttime dives, when the black backdrop allowed for high-contrast imaging of the transparent critters. They encountered an immense flurry of different salps that were doing their nightly migration to the surface — and many photobombing sharks, squids and crustaceans, Sutherland noted.
Through imaging and recordings, the researchers noticed two modes of swimming. Where shorter colonies spun around an axis, like a spiraling football, longer chains would buckle and coil like a corkscrew. That’s called helical swimming.
Helical swimming is nothing new in biology, Sutherland said. Many microorganisms also spin and corkscrew through water, but the mechanisms behind the salps’ motion are different. Microbes beat water with hair-like projections or tail whips, but salps swim via jet propulsion, Sutherland said. They have contracting muscle bands, like those in the human throat, that pump water sucked from one side of the body and squirted out the other end to create thrust, Damian-Serrano said.
The researchers also noticed that individual jets contracted at different times, causing the whole colony to steadily travel without pause. The jets were also angled, contributing to the spinning and coil swimming, Sutherland said.
“My initial reaction was really one of wonder and awe,” she said. “I would describe their motion as snake-like and graceful. They have multiple units pulsing at different times, creating a whole chain that moves very smoothly. It’s a really beautiful way of moving.”
Microrobots inspired by microbial swimmers already exist, Sutherland said, but this discovery paves the way for engineers to construct larger underwater vehicles. It may be possible to create robots that are silent and less turbulent when modeled after these efficient swimmers, Damian-Serrano said. A multijet design also may be energetically advantageous for saving fuel, he said.
Beyond microbes, larger organisms like plankton have yet to be described in this way, Sutherland said. With Sutherland’s new and innovative methods of studying sea creatures, scientists might come to realize that helical swimming is more pervasive than previously thought.
“It’s a study that opens up more questions than provides answers,” Sutherland said. “There’s this new way of swimming that hadn’t been described before, and when we started the study we sought to explain how it works. But we found that there are a lot more open questions, like what are the advantages of swimming this way? How many different organisms spin or corkscrew?” More

Need a moment of levity? Try watching videos of astronauts falling on the moon. NASA’s outtakes of Apollo astronauts tripping and stumbling as they bounce in slow motion are delightfully relatable.
For MIT engineers, the lunar bloopers also highlight an opportunity to innovate.
“Astronauts are physically very capable, but they can struggle on the moon, where gravity is one-sixth that of Earth’s but their inertia is still the same. Furthermore, wearing a spacesuit is a significant burden and can constrict their movements,” says Harry Asada, professor of mechanical engineering at MIT. “We want to provide a safe way for astronauts to get back on their feet if they fall.”
Asada and his colleagues are designing a pair of wearable robotic limbs that can physically support an astronaut and lift them back on their feet after a fall. The system, which the researchers have dubbed Supernumerary Robotic Limbs or “SuperLimbs” is designed to extend from a backpack, which would also carry the astronaut’s life support system, along with the controller and motors to power the limbs.
The researchers have built a physical prototype, as well as a control system to direct the limbs, based on feedback from the astronaut using it. The team tested a preliminary version on healthy subjects who also volunteered to wear a constrictive garment similar to an astronaut’s spacesuit. When the volunteers attempted to get up from a sitting or lying position, they did so with less effort when assisted by SuperLimbs, compared to when they had to recover on their own.
The MIT team envisions that SuperLimbs can physically assist astronauts after a fall and, in the process, help them conserve their energy for other essential tasks. The design could prove especially useful in the coming years, with the launch of NASA’s Artemis mission, which plans to send astronauts back to the moon for the first time in over 50 years. Unlike the largely exploratory mission of Apollo, Artemis astronauts will endeavor to build the first permanent moon base — a physically demanding task that will require multiple extended extravehicular activities (EVAs).
“During the Apollo era, when astronauts would fall, 80 percent of the time it was when they were doing excavation or some sort of job with a tool,” says team member and MIT doctoral student Erik Ballesteros. “The Artemis missions will really focus on construction and excavation, so the risk of falling is much higher. We think that SuperLimbs can help them recover so they can be more productive, and extend their EVAs.”
Asada, Ballesteros, and their colleagues will present their design and study this week at the IEEE International Conference on Robotics and Automation (ICRA). Their co-authors include MIT postdoc Sang-Yoep Lee and Kalind Carpenter of the Jet Propulsion Laboratory.

Taking a stand
The team’s design is the latest application of SuperLimbs, which Asada first developed about a decade ago and has since adapted for a range of applications, including assisting workers in aircraft manufacturing, construction, and ship building.
Most recently, Asada and Ballesteros wondered whether SuperLimbs might assist astronauts, particularly as NASA plans to send astronauts back to the surface of the moon.
“In communications with NASA, we learned that this issue of falling on the moon is a serious risk,” Asada says. “We realized that we could make some modifications to our design to help astronauts recover from falls and carry on with their work.”
The team first took a step back, to study the ways in which humans naturally recover from a fall. In their new study, they asked several healthy volunteers to attempt to stand upright after lying on their side, front, and back.
The researchers then looked at how the volunteers’ attempts to stand changed when their movements were constricted, similar to the way astronauts’ movements are limited by the bulk of their spacesuits. The team built a suit to mimic the stiffness of traditional spacesuits, and had volunteers don the suit before again attempting to stand up from various fallen positions. The volunteers’ sequence of movements was similar, though required much more effort compared to their unencumbered attempts.

The team mapped the movements of each volunteer as they stood up, and found that they each carried out a common sequence of motions, moving from one pose, or “waypoint,” to the next, in a predictable order.
“Those ergonomic experiments helped us to model in a straightforward way, how a human stands up,” Ballesteros says. “We could postulate that about 80 percent of humans stand up in a similar way. Then we designed a controller around that trajectory.”
Helping hand
The team developed software to generate a trajectory for a robot, following a sequence that would help support a human and lift them back on their feet. They applied the controller to a heavy, fixed robotic arm, which they attached to a large backpack. The researchers then attached the backpack to the bulky suit and helped volunteers back into the suit. They asked the volunteers to again lie on their back, front, or side, and then had them attempt to stand as the robot sensed the person’s movements and adapted to help them to their feet.
Overall, the volunteers were able to stand stably with much less effort when assisted by the robot, compared to when they tried to stand alone while wearing the bulky suit.
“It feels kind of like an extra force moving with you,” says Ballesteros, who also tried out the suit and arm assist. “Imagine wearing a backpack and someone grabs the top and sort of pulls you up. Over time, it becomes sort of natural.”
The experiments confirmed that the control system can successfully direct a robot to help a person stand back up after a fall. The researchers plan to pair the control system with their latest version of SuperLimbs, which comprises two multijointed robotic arms that can extend out from a backpack. The backpack would also contain the robot’s battery and motors, along with an astronaut’s ventilation system.
“We designed these robotic arms based on an AI search and design optimization, to look for designs of classic robot manipulators with certain engineering constraints,” Ballesteros says. “We filtered through many designs and looked for the design that consumes the least amount of energy to lift a person up. This version of SuperLimbs is the product of that process.”
Over the summer, Ballesteros will build out the full SuperLimbs system at NASA’s Jet Propulsion Laboratory, where he plans to streamline the design and minimize the weight of its parts and motors using advanced, lightweight materials. Then, he hopes to pair the limbs with astronaut suits, and test them in low-gravity simulators, with the goal of someday assisting astronauts on future missions to the moon and Mars.
“Wearing a spacesuit can be a physical burden,” Asada notes. “Robotic systems can help ease that burden, and help astronauts be more productive during their missions.”
This research was supported, in part, by NASA. More

Strongly interacting systems play an important role in quantum physics and quantum chemistry. Stochastic methods such as Monte Carlo simulations are a proven method for investigating such systems. However, these methods reach their limits when so-called sign oscillations occur. This problem has now been solved by an international team of researchers from Germany, Turkey, the USA, China, South Korea and France using the new method of wavefunction matching. As an example, the masses and radii of all nuclei up to mass number 50 were calculated using this method. The results agree with the measurements, the researchers now report in the journal “Nature.”
All matter on Earth consists of tiny particles known as atoms. Each atom contains even smaller particles: protons, neutrons and electrons. Each of these particles follows the rules of quantum mechanics. Quantum mechanics forms the basis of quantum many-body theory, which describes systems with many particles, such as atomic nuclei.
One class of methods used by nuclear physicists to study atomic nuclei is the ab initio approach. It describes complex systems by starting from a description of their elementary components and their interactions. In the case of nuclear physics, the elementary components are protons and neutrons. Some key questions that ab initio calculations can help answer are the binding energies and properties of atomic nuclei and the link between nuclear structure and the underlying interactions between protons and neutrons.
However, these ab initio methods have difficulties in performing reliable calculations for systems with complex interactions. One of these methods is quantum Monte Carlo simulations. Here, quantities are calculated using random or stochastic processes. Although quantum Monte Carlo simulations can be efficient and powerful, they have a significant weakness: the sign problem. It arises in processes with positive and negative weights, which cancel each other. This cancellation leads to inaccurate final predictions.
A new approach, known as wavefunction matching, is intended to help solve such calculation problems for ab initio methods. “This problem is solved by the new method of wavefunction matching by mapping the complicated problem in a first approximation to a simple model system that does not have such sign oscillations and then treating the differences in perturbation theory,” says Prof. Ulf-G. Meißner from the Helmholtz Institute for Radiation and Nuclear Physics at the University of Bonn and from the Institute of Nuclear Physics and the Center for Advanced Simulation and Analytics at Forschungszentrum Jülich. “As an example, the masses and radii of all nuclei up to mass number 50 were calculated — and the results agree with the measurements,” reports Meißner, who is also a member of the Transdisciplinary Research Areas “Modeling” and “Matter” at the University of Bonn.
“In quantum many-body theory, we are often faced with the situation that we can perform calculations using a simple approximate interaction, but realistic high-fidelity interactions cause severe computational problems,” says Dean Lee, Professor of Physics from the Facility for Rare Istope Beams and Department of Physics and Astronomy (FRIB) at Michigan State University and head of the Department of Theoretical Nuclear Sciences.
Wavefunction matching solves this problem by removing the short-distance part of the high-fidelity interaction and replacing it with the short-distance part of an easily calculable interaction. This transformation is done in a way that preserves all the important properties of the original realistic interaction. Since the new wavefunctions are similar to those of the easily computable interaction, the researchers can now perform calculations with the easily computable interaction and apply a standard procedure for handling small corrections — called perturbation theory.
The research team applied this new method to lattice quantum Monte Carlo simulations for light nuclei, medium-mass nuclei, neutron matter and nuclear matter. Using precise ab initio calculations, the results closely matched real-world data on nuclear properties such as size, structure and binding energy. Calculations that were once impossible due to the sign problem can now be performed with wavefunction matching.
While the research team focused exclusively on quantum Monte Carlo simulations, wavefunction matching should be useful for many different ab initio approaches. “This method can be used in both classical computing and quantum computing, for example to better predict the properties of so-called topological materials, which are important for quantum computing,” says Meißner.
The first author is Prof. Dr. Serdar Elhatisari, who worked for two years as a Fellow in Prof. Meißner’s ERC Advanced Grant EXOTIC. According to Meißner, a large part of the work was carried out during this time. Part of the computing time on supercomputers at Forschungszentrum Jülich was provided by the IAS-4 institute, which Meißner heads. More

A team of researchers at Delft University of Technology has developed a drone that flies autonomously using neuromorphic image processing and control based on the workings of animal brains. Animal brains use less data and energy compared to current deep neural networks running on GPUs (graphic chips). Neuromorphic processors are therefore very suitable for small drones because they don’t need heavy and large hardware and batteries. The results are extraordinary: during flight the drone’s deep neural network processes data up to 64 times faster and consumes three times less energy than when running on a GPU. Further developments of this technology may enable the leap for drones to become as small, agile, and smart as flying insects or birds. The findings were recently published in Science Robotics.
Learning from animal brains: spiking neural networks
Artificial intelligence holds great potential to provide autonomous robots with the intelligence needed for real-world applications. However, current AI relies on deep neural networks that require substantial computing power. The processors made for running deep neural networks (Graphics Processing Units, GPUs) consume a substantial amount of energy. Especially for small robots like flying drones this is a problem, since they can only carry very limited resources in terms of sensing and computing.
Animal brains process information in a way that is very different from the neural networks running on GPUs. Biological neurons process information asynchronously, and mostly communicate via electrical pulses called spikes. Since sending such spikes costs energy, the brain minimizes spiking, leading to sparse processing.
Inspired by these properties of animal brains, scientists and tech companies are developing new, neuromorphic processors. These new processors allow to run spiking neural networks and promise to be much faster and more energy efficient.
“The calculations performed by spiking neural networks are much simpler than those in standard deep neural networks.,” says Jesse Hagenaars, PhD candidate and one of the authors of the article, “Whereas digital spiking neurons only need to add integers, standard neurons have to multiply and add floating point numbers. This makes spiking neural networks quicker and more energy efficient. To understand why, think of how humans also find it much easier to calculate 5 + 8 than to calculate 6.25 x 3.45 + 4.05 x 3.45.”
This energy efficiency is further boosted if neuromorphic processors are used in combination with neuromorphic sensors, like neuromorphic cameras. Such cameras do not make images at a fixed time interval. Instead, each pixel only sends a signal when it becomes brighter or darker. The advantages of such cameras are that they can perceive motion much more quickly, are more energy efficient, and function well both in dark and bright environments. Moreover, the signals from neuromorphic cameras can feed directly into spiking neural networks running on neuromorphic processors. Together, they can form a huge enabler for autonomous robots, especially small, agile robots like flying drones.

First neuromorphic vision and control of a flying drone
In an article published in Science Robotics on May 15, 2024, researchers from Delft University of Technology, the Netherlands, demonstrate for the first time a drone that uses neuromorphic vision and control for autonomous flight. Specifically, they developed a spiking neural network that processes the signals from a neuromorphic camera and outputs control commands that determine the drone’s pose and thrust. They deployed this network on a neuromorphic processor, Intel’s Loihi neuromorphic research chip, on board of a drone. Thanks to the network, the drone can perceive and control its own motion in all directions.
“We faced many challenges,” says Federico Paredes-Vallés, one of the researchers that worked on the study, “but the hardest one was to imagine how we could train a spiking neural network so that training would be both sufficiently fast and the trained network would function well on the real robot. In the end, we designed a network consisting of two modules. The first module learns to visually perceive motion from the signals of a moving neuromorphic camera. It does so completely by itself, in a self-supervised way, based only on the data from the camera. This is similar to how also animals learn to perceive the world by themselves. The second module learns to map the estimated motion to control commands, in a simulator. This learning relied on an artificial evolution in simulation, in which networks that were better in controlling the drone had a higher chance of producing offspring. Over the generations of the artificial evolution, the spiking neural networks got increasingly good at control, and were finally able to fly in any direction at different speeds. We trained both modules and developed a way with which we could merge them together. We were happy to see that the merged network immediately worked well on the real robot.”
With its neuromorphic vision and control, the drone is able to fly at different speeds under varying light conditions, from dark to bright. It can even fly with flickering lights, which make the pixels in the neuromorphic camera send great numbers of signals to the network that are unrelated to motion.
Improved energy efficiency and speed by neuromorphic AI
“Importantly, our measurements confirm the potential of neuromorphic AI. The network runs on average between 274 and 1600 times per second. If we run the same network on a small, embedded GPU, it runs on average only 25 times per second, a difference of a factor ~10-64! Moreover, when running the network, , Intel’s Loihi neuromorphic research chip consumes 1.007 watts, of which 1 watt is the idle power that the processor spends just when turning on the chip. Running the network itself only costs 7 milliwatts. In comparison, when running the same network, the embedded GPU consumes 3 watts, of which 1 watt is idle power and 2 watts are spent for running the network. The neuromorphic approach results in AI that runs faster and more efficiently, allowing deployment on much smaller autonomous robots.,” says Stein Stroobants, PhD candidate in the field of neuromorphic drones.
Future applications of neuromorphic AI for tiny robots
“Neuromorphic AI will enable all autonomous robots to be more intelligent,” says Guido de Croon, Professor in bio-inspired drones, “but it is an absolute enabler for tiny autonomous robots. At Delft University of Technology’s Faculty of Aerospace Engineering, we work on tiny autonomous drones which can be used for applications ranging from monitoring crop in greenhouses to keeping track of stock in warehouses. The advantages of tiny drones are that they are very safe and can navigate in narrow environments like in between ranges of tomato plants. Moreover, they can be very cheap, so that they can be deployed in swarms. This is useful for more quickly covering an area, as we have shown in exploration and gas source localization settings.”
“The current work is a great step in this direction. However, the realization of these applications will depend on further scaling down the neuromorphic hardware and expanding the capabilities towards more complex tasks such as navigation.” More

Research at Uppsala University and Karolinska Institutet could pave the way for a prosthetic hand and robot to be able to feel touch like a human hand. Their study has been published in the journal Science. The technology could also be used to help restore lost functionality to patients after a stroke.
“Our system can determine what type of object it encounters as fast as a blindfolded person, just by feeling it and deciding whether it is a tennis ball or an apple, for example,” says Zhibin Zhang, docent at the Department of Electrical Engineering at Uppsala University.
He and his colleague Libo Chen performed the study in close cooperation with researchers from the Signals and Systems Division at Uppsala University, who provided data processing and machine learning expertise, and a group of researchers from the Department of Neurobiology, Care Sciences and Society, Division of Neurogeriatrics at Karolinska Institutet.
Drawing inspiration from neuroscience, they have developed an artificial tactile system that imitates the way the human nervous system reacts to touch. The system uses electrical pulses that process dynamic tactile information in the same way as the human nervous system. “With this technology, a prosthetic hand would feel like part of the wearer’s body,” Zhang explains.
The artificial system has three main components: an electronic skin (e-skin) with sensors that can detect pressure by touch; a set of artificial neurons that convert analogue touch signals into electrical pulses; and a processor that processes the signals and identifies the object. In principle, it can learn to identify an unlimited number of objects, but in their tests the researchers have used 22 different objects for grasping and 16 different surfaces for touching.
“We’re also looking into developing the system so it can feel pain and heat as well. It should also be able to feel what material the hand is touching, for example, whether it is wood or metal,” says Assistant Professor Libo Chen, who led the study.
According to the researchers, interactions between humans and robots or prosthetic hands can be made safer and more natural thanks to tactile feedback. The prostheses can also be given the ability to handle objects with the same dexterity as a human hand.
“The skin contains millions of receptors. Current e-skin technology cannot deliver enough receptors, but this technology makes it possible, so we would like to produce artificial skin for a whole robot,” says Chen.
The technology could also be used medically, for example, to monitor movement dysfunctions caused by Parkinson’s disease and Alzheimer’s disease, or to help patients recover lost functionality after a stroke.
“The technology can be further developed to tell if a patient is about to fall. This information can be then used to either stimulate a muscle externally to prevent the fall or prompt an assistive device to take over and prevent it,” says Zhang. More

Researchers have leveraged deep learning techniques to enhance the image quality of a metalens camera. The new approach uses artificial intelligence to turn low-quality images into high-quality ones, which could make these cameras viable for a multitude of imaging tasks including intricate microscopy applications and mobile devices.
Metalenses are ultrathin optical devices — often just a fraction of a millimeter thick — that use nanostructures to manipulate light. Although their small size could potentially enable extremely compact and lightweight cameras without traditional optical lenses, it has been difficult to achieve the necessary image quality with these optical components.
“Our technology allows our metalens-based devices to overcome the limitations of image quality,” said research team leader Ji Chen from Southeast University in China. “This advance will play an important role in the future development of highly portable consumer imaging electronics and can also be used in specialized imaging applications such as microscopy.”
In Optica Publishing Group journal Optics Letters, the researchers describe how they used a type of machine learning known as a multi-scale convolutional neural network to improve resolution, contrast and distortion in images from a small camera — about 3 cm × 3 cm × 0.5 cm — they created by directly integrating a metalens onto a CMOS imaging chip.
“Metalens-integrated cameras can be directly incorporated into the imaging modules of smartphones, where they could replace the traditional refractive bulk lenses,” said Chen. “They could also be used in devices such as drones, where the small size and lightweight camera would ensure imaging quality without compromising the drone’s mobility.”
Enhancing image quality
The camera used in the new work was previously developed by the researchers and uses a metalens with 1000-nm tall cylindrical silicon nitride nano-posts. The metalens focuses light directly onto a CMOS imaging sensor without requiring any other optical elements. Although this design created a very small camera the compact architecture limited the image quality. Thus, the researchers decided to see if machine learning could be used to improve the images.

Deep learning is a type of machine learning that uses artificial neural networks with multiple layers to automatically learn features from data and make complex decisions or predictions. The researchers applied this approach by using a convolution imaging model to generate a large number of high- and low-quality image pairs. These image pairs were used to train a multi-scale convolutional neural network so that it could recognize the characteristics of each type of image and use that to turn low-quality images into high-quality images.
“A key part of this work was developing a way to generate the large amount of training data needed for the neural network learning process,” said Chen. “Once trained, a low-quality image can be sent from the device to into the neural network for processing, and high-quality imaging results are obtained immediately.”
Applying the neural network
To validate the new deep learning technique, the researchers used it on 100 test images. They analyzed two commonly used image processing metrics: the peak signal-to-noise ratio and the structural similarity index. They found that the images processed by the neural network exhibited a significant improvement in both metrics. They also showed that the approach could rapidly generate high-quality imaging data that closely resembled what was captured directly through experimentation.
The researchers are now designing metalenses with complex functionalities — such as color or wide-angle imaging — and developing neural network methods for enhancing the imaging quality of these advanced metalenses. To make this technology practical for commercial application would require new assembly techniques for integrating metalenses into smartphone imaging modules and image quality enhancement software designed specifically for mobile phones.
“Ultra-lightweight and ultra-thin metalenses represent a revolutionary technology for future imaging and detection,” said Chen. “Leveraging deep learning techniques to optimize metalens performance marks a pivotal developmental trajectory. We foresee machine learning as a vital trend in advancing photonics research.” More