Aurelia Butler

Interactions with voice technology, such as Amazon’s Alexa, Apple’s Siri, and Google Assistant, can make life easier by increasing efficiency and productivity. However, errors in generating and understanding speech during interactions are common. When using these devices, speakers often style-shift their speech from their normal patterns into a louder and slower register, called technology-directed speech.
Research on technology-directed speech typically focuses on mainstream varieties of U.S. English without considering speaker groups that are more consistently misunderstood by technology. In JASA Express Letters, published on behalf of the Acoustical Society of America by AIP Publishing, researchers from Google Research, the University of California, Davis, and Stanford University wanted to address this gap.
One group commonly misunderstood by voice technology are individuals who speak African American English, or AAE. Since the rate of automatic speech recognition errors can be higher for AAE speakers, downstream effects of linguistic discrimination in technology may result.
“Across all automatic speech recognition systems, four out of every ten words spoken by Black men were being transcribed incorrectly,” said co-author Zion Mengesha. “This affects fairness for African American English speakers in every institution using voice technology, including health care and employment.”
“We saw an opportunity to better understand this problem by talking to Black users and understanding their emotional, behavioral, and linguistic responses when engaging with voice technology,” said co-author Courtney Heldreth.
The team designed an experiment to test how AAE speakers adapt their speech when imagining talking to a voice assistant, compared to talking to a friend, family member, or stranger. The study tested familiar human, unfamiliar human, and voice assistant-directed speech conditions by comparing speech rate and pitch variation. Study participants included 19 adults identifying as Black or African American who had experienced issues with voice technology. Each participant asked a series of questions to a voice assistant. The same questions were repeated as if speaking to a familiar person and, again, to a stranger. Each question was recorded for a total of 153 recordings.
Analysis of the recordings showed that the speakers exhibited two consistent adjustments when they were talking to voice technology compared to talking to another person: a slower rate of speech with less pitch variation (more monotone speech).
“These findings suggest that people have mental models of how to talk to technology,” said co-author Michelle Cohn. “A set ‘mode’ that they engage to be better understood, in light of disparities in speech recognition systems.”
There are other groups misunderstood by voice technology, such as second-language speakers. The researchers hope to expand the language varieties explored in human-computer interaction experiments and address barriers in technology so that it can support everyone who wants to use it. More

Researchers in Purdue University’s College of Engineering are developing patented and patent-pending innovations that make 3D microscopes faster to operate and less expensive to manufacture.
Traditional, large depth-of-field 3D microscopes are used across academia and industry, with applications ranging from the life sciences to quality control processes used in semiconductor manufacturing. Song Zhang, professor in Purdue’s School of Mechanical Engineering, said they are too slow to capture 3D images and too expensive to build due to the requirement of a high-precision translation stage.
“Such drawbacks in a microscope slow the measurement process, making it difficult to use for applications that require high speeds, such as in situ quality control,” Zhang said.
Research about the Purdue 3D microscope and its innovations has been published in the peer-reviewed Optics Letters and the August 2023 and March 2024 issues of the peer-reviewed Optics and Lasers in Engineering. The National Science Foundation awarded a grant to conduct the research.
The Purdue innovation
Zhang said the Purdue 3D microscope automatically completes three steps: focusing in on an object, determining the optimal capture process and creating a high-quality 3D image for the end user.
“In contrast, a traditional microscope requires users to carefully follow instructions provided by the manufacturer to perform a high-quality capture,” Zhang said.

Zhang and his colleagues use an electronically tunable lens, or ETL, that changes the focal plane of the imaging system without moving parts. He said using the lens makes the 3D microscope easier to use and less expensive to build.
“Our suite of patents covers methods on how to calibrate the ETL, how to create all-in-focus 3D images quickly and how to speed up the data acquisition process by leveraging the ETL hardware information,” Zhang said. “The end result is the same as a traditional microscope: 3D surface images of a scene. Ours is different because of its high speed and relatively low cost.”
The next developmental steps
Zhang and his team have developed algorithms and created a prototype system in their lab. They are looking to translate their research into a commercial product.
“This will require an industrial partner,” Zhang said. “We are certainly interested in helping this process, including sharing our know-how and research results to make the transition smooth.”
Zhang disclosed the innovations to the Purdue Innovates Office of Technology Commercialization, which has applied for and received patents to protect the multiple pieces of intellectual property. More

With the help of a form of machine learning called deep reinforcement learning (DRL), the EPFL robot notably learned to transition from trotting to pronking — a leaping, arch-backed gait used by animals like springbok and gazelles — to navigate a challenging terrain with gaps ranging from 14-30cm. The study, led by the BioRobotics Laboratory in EPFL’s School of Engineering, offers new insights into why and how such gait transitions occur in animals.
“Previous research has introduced energy efficiency and musculoskeletal injury avoidance as the two main explanations for gait transitions. More recently, biologists have argued that stability on flat terrain could be more important. But animal and robotic experiments have shown that these hypotheses are not always valid, especially on uneven ground,” says PhD student Milad Shafiee, first author on a paper published in Nature Communications.
Shafiee and co-authors Guillaume Bellegarda and BioRobotics Lab head Auke Ijspeert were therefore interested in a new hypothesis for why gait transitions occur: viability, or fall avoidance. To test this hypothesis, they used DRL to train a quadruped robot to cross various terrains. On flat terrain, they found that different gaits showed different levels of robustness against random pushes, and that the robot switched from a walk to a trot to maintain viability, just as quadruped animals do when they accelerate. And when confronted with successive gaps in the experimental surface, the robot spontaneously switched from trotting to pronking to avoid falls. Moreover, viability was the only factor that was improved by such gait transitions.
“We showed that on flat terrain and challenging discrete terrain, viability leads to the emergence of gait transitions, but that energy efficiency is not necessarily improved,” Shafiee explains. “It seems that energy efficiency, which was previously thought to be a driver of such transitions, may be more of a consequence. When an animal is navigating challenging terrain, it’s likely that its first priority is not falling, followed by energy efficiency.”
A bio-inspired learning architecture
To model locomotion control in their robot, the researchers considered the three interacting elements that drive animal movement: the brain, the spinal cord, and sensory feedback from the body. They used DRL to train a neural network to imitate the spinal cord’s transmission of brain signals to the body as the robot crossed an experimental terrain. Then, the team assigned different weights to three possible learning goals: energy efficiency, force reduction, and viability. A series of computer simulations revealed that of these three goals, viability was the only one that prompted the robot to automatically — without instruction from the scientists — change its gait.
The team emphasizes that these observations represent the first learning-based locomotion framework in which gait transitions emerge spontaneously during the learning process, as well as the most dynamic crossing of such large consecutive gaps for a quadrupedal robot.
“Our bio-inspired learning architecture demonstrated state-of-the-art quadruped robot agility on the challenging terrain,” Shafiee says.
The researchers aim to expand on their work with additional experiments that place different types of robots in a wider variety of challenging environments. In addition to further elucidating animal locomotion, they hope that ultimately, their work will enable the more widespread use of robots for biological research, reducing reliance on animal models and the associated ethics concerns. More

Researchers have developed a technique to move objects around with a jet of wind. The new approach makes it possible to manipulate objects at a distance and could be integrated into robots to give machines ethereal fingers.
‘Airflow or wind is everywhere in our living environment, moving around objects like pollen, pathogens, droplets, seeds and leaves. Wind has also been actively used in industry and in our everyday lives — for example, in leaf blowers to clean leaves. But so far, we can’t control the direction the leaves move — we can only blow them together into a pile,’ says Professor Quan Zhou from Aalto University, who led the study.
The first step in manipulating objects with wind is understanding how objects move in the airflow. To that end, a research team at Aalto University recorded thousands of sample movements in an artificially generated airflow and used these to build templates of how objects move on a surface in a jet of air.
The team’s analysis showed that even though the airflow is generally chaotic, it’s still regular enough to move objects in a controlled way in different directions — even back towards the nozzle blowing out the air.
‘We designed an algorithm that controls the direction of the air nozzle with two motors. The jet of air is blown onto the surface from several meters away and to the side of the object, so the generated airflow field moves the object in the desired direction. The control algorithm repeatedly adjusts the direction of the air nozzle so that the airflow moves the objects along the desired trajectory,’ explains Zhou.
‘Our observations allowed us to use airflow to move objects along different paths, like circles or even complex letter-like paths. Our method is versatile in terms of the object’s shape and material — we can control the movement of objects of almost any shape,’ he continues.
The technology still needs to be refined, but the researchers are optimistic about the untapped potential of their nature-inspired approach. It could be used to collect items that are scattered on a surface, such as pushing debris and waste to collection points. It could also be useful in complex processing tasks where physical contact is impossible, such as handling electrical circuits.
‘We believe that this technique could get even better with a deeper understanding of the characteristics of the airflow field, which is what we’re working on next,’ says Zhou. More

Researchers at Aalto University were looking for better ways to instruct dance choreography in virtual reality. The new WAVE technique they developed will be presented in May at the CHI conference, a major venue for human-computer interaction research.
Previous techniques have largely relied on pre-rehearsal and simplification.
‘In virtual reality, it is difficult to visualise and communicate how a dancer should move. The human body is so multi-dimensional, and it is difficult to take in rich data in real time,’ says Professor Perttu Hämäläinen.
The researchers started by experimenting with visualisation techniques familiar from previous dance games. But after several prototypes and stages, they decided to try out the audience wave, familiar from sporting events, to guide the dance.
‘The wave-like movement of the model dancers allows you to see in advance what kind of movement is coming next. And you don’t have to rehearse the movement beforehand,’ says PhD researcher Markus Laattala.
In general, one cannot follow a new choreography in real time because of the delay in human perceptual motor control. The WAVE technique developed by the researchers, on the other hand, is based on anticipating future movement, such as a turn.
‘No one had figured out how to guide a continuous, fluid movement like contemporary dance. In the choreography we implemented, making a wave is communication, a kind of micro-canon in which the model dancers follow the same choreography with a split-second delay,’ says Hämäläinen.

From tai chi to exaggerated movements
A total of 36 people took part in the one-minute dance test, comparing the new WAVE visualization to a traditional virtual version in which there was only one model dancer to follow. The differences between the techniques were clear.
‘This implementation is at least suitable for slow-paced dance styles. The dancer can just jump in and start dancing without having to learn anything beforehand. However, in faster movements, the visuals can get confusing, and further research and development is needed to adapt and test the approach with more dance styles’ says Hämäläinen.
In addition to virtual dance games, the new technique may be applicable to music videos, karaoke, and tai chi.
‘It would be optimal for the user if they could decide how to position the model dancers in a way that suits them. And if the idea were taken further, several dancers could send each other moves in social virtual reality. It could become a whole new way of dancing together’, says Laattala.
‘Current mainstream VR devices only track the movement of the headset and handheld controllers. On the other hand, machine learning data can sometimes be used to infer how the legs move,’ says Hämäläinen.

‘But in dance, inference is more difficult because the movements are stranger than, for example, walking,’ adds Laattala.
On the other hand, if you have a mirror in the real dance space, you can follow the movement of your feet using machine vision. The dancer’s view could be modified using a virtual mirror.
‘A dancer’s virtual performance can be improved by exaggeration, for example by increasing flexibility, height of the jumps, or hip movement. This can make them feel that they are more skilled than they are, which research shows has a positive impact on physical activity motivation,’ says Hämäläinen.
The virtual dance game has been developed using the Magics infrastructure’s motion capture kit, where the model dancer is dressed in a costume with sensors. These have been used to record the dance animation.
The WAVE dance game can be downloaded for Meta Quest 2 and 3 VR devices here: https://github.com/CarouselDancing/WAVE. The Github repository also includes the open source code that anyone can use to develop the game further.
Reference:
Laattala, M., Piitulainen, R., Ady, N., Tamariz, M., & Hämäläinen, P. (2024). Anticipatory Movement Visualization for VR Dancing. ACM SIGCHI Annual Conference on Human Factors in Computing Systems. More