Computers Math

A team of researchers demonstrated that popular robotic household vacuum cleaners can be remotely hacked to act as microphones.
The researchers — including Nirupam Roy, an assistant professor in the University of Maryland’s Department of Computer Science — collected information from the laser-based navigation system in a popular vacuum robot and applied signal processing and deep learning techniques to recover speech and identify television programs playing in the same room as the device.
The research demonstrates the potential for any device that uses light detection and ranging (Lidar) technology to be manipulated for collecting sound, despite not having a microphone. This work, which is a collaboration with assistant professor Jun Han at the University of Singapore was presented at the Association for Computing Machinery’s Conference on Embedded Networked Sensor Systems (SenSys 2020) on November 18, 2020.
“We welcome these devices into our homes, and we don’t think anything about it,” said Roy, who holds a joint appointment in the University of Maryland Institute for Advanced Computer Studies (UMIACS). “But we have shown that even though these devices don’t have microphones, we can repurpose the systems they use for navigation to spy on conversations and potentially reveal private information.”
The Lidar navigation systems in household vacuum bots shine a laser beam around a room and sense the reflection of the laser as it bounces off nearby objects. The robot uses the reflected signals to map the room and avoid collisions as it moves through the house.
Privacy experts have suggested that the maps made by vacuum bots, which are often stored in the cloud, pose potential privacy breaches that could give advertisers access to information about such things as home size, which suggests income level, and other lifestyle-related information. Roy and his team wondered if the Lidar in these robots could also pose potential security risks as sound recording devices in users’ homes or businesses.

advertisement

Sound waves cause objects to vibrate, and these vibrations cause slight variations in the light bouncing off an object. Laser microphones, used in espionage since the 1940s, are capable of converting those variations back into sound waves. But laser microphones rely on a targeted laser beam reflecting off very smooth surfaces, such as glass windows.
A vacuum Lidar, on the other hand, scans the environment with a laser and senses the light scattered back by objects that are irregular in shape and density. The scattered signal received by the vacuum’s sensor provides only a fraction of the information needed to recover sound waves. The researchers were unsure if a vacuum bot’s Lidar system could be manipulated to function as a microphone and if the signal could be interpreted into meaningful sound signals.
First, the researchers hacked a robot vacuum to show they could control the position of the laser beam and send the sensed data to their laptops through Wi-Fi without interfering with the device’s navigation.
Next, they conducted experiments with two sound sources. One source was a human voice reciting numbers played over computer speakers and the other was audio from a variety of television shows played through a TV sound bar. Roy and his colleagues then captured the laser signal sensed by the vacuum’s navigation system as it bounced off a variety of objects placed near the sound source. Objects included a trash can, cardboard box, takeout container and polypropylene bag — items that might normally be found on a typical floor.
The researchers passed the signals they received through deep learning algorithms that were trained to either match human voices or to identify musical sequences from television shows. Their computer system, which they call LidarPhone, identified and matched spoken numbers with 90% accuracy. It also identified television shows from a minute’s worth of recording with more than 90% accuracy.

advertisement

“This type of threat may be more important now than ever, when you consider that we are all ordering food over the phone and having meetings over the computer, and we are often speaking our credit card or bank information,” Roy said. “But what is even more concerning for me is that it can reveal much more personal information. This kind of information can tell you about my living style, how many hours I’m working, other things that I am doing. And what we watch on TV can reveal our political orientations. That is crucial for someone who might want to manipulate the political elections or target very specific messages to me.”
The researchers emphasize that vacuum cleaners are just one example of potential vulnerability to Lidar-based spying. Many other devices could be open to similar attacks such as smartphone infrared sensors used for face recognition or passive infrared sensors used for motion detection.
“I believe this is significant work that will make the manufacturers aware of these possibilities and trigger the security and privacy community to come up with solutions to prevent these kinds of attacks,” Roy said.
This research was partially supported by a grant from Singapore Ministry of Education Academic Research Fund Tier 1 (Award No. R-252-000-A26-133).
The research paper, Spying with Your Robot Vacuum Cleaner: Eavesdropping via Lidar Sensors, Sriram Sami, Yimin Dai, Sean Rui Xiang Tan, Nirupam Roy and Jun Han, was presented on November 18, 2020, at the Association for Computing Machinery, SenSys 2020. More

A new kind of radar could make it possible for self-driving cars to navigate safely in bad weather. Electrical engineers at the University of California San Diego developed a clever way to improve the imaging capability of existing radar sensors so that they accurately predict the shape and size of objects in the scene. The system worked well when tested at night and in foggy conditions.
The team will present their work at the Sensys conference Nov. 16 to 19.
Inclement weather conditions pose a challenge for self-driving cars. These vehicles rely on technology like LiDAR and radar to “see” and navigate, but each has its shortcomings. LiDAR, which works by bouncing laser beams off surrounding objects, can paint a high-resolution 3D picture on a clear day, but it cannot see in fog, dust, rain or snow. On the other hand, radar, which transmits radio waves, can see in all weather, but it only captures a partial picture of the road scene.
Enter a new UC San Diego technology that improves how radar sees.
“It’s a LiDAR-like radar,” said Dinesh Bharadia, a professor of electrical and computer engineering at the UC San Diego Jacobs School of Engineering. It’s an inexpensive approach to achieving bad weather perception in self-driving cars, he noted. “Fusing LiDAR and radar can also be done with our techniques, but radars are cheap. This way, we don’t need to use expensive LiDARs.”
The system consists of two radar sensors placed on the hood and spaced an average car’s width apart (1.5 meters). Having two radar sensors arranged this way is key — they enable the system to see more space and detail than a single radar sensor.

advertisement

During test drives on clear days and nights, the system performed as well as a LiDAR sensor at determining the dimensions of cars moving in traffic. Its performance did not change in tests simulating foggy weather. The team “hid” another vehicle using a fog machine and their system accurately predicted its 3D geometry. The LiDAR sensor essentially failed the test.
Two eyes are better than one
The reason radar traditionally suffers from poor imaging quality is because when radio waves are transmitted and bounced off objects, only a small fraction of signals ever gets reflected back to the sensor. As a result, vehicles, pedestrians and other objects appear as a sparse set of points.
“This is the problem with using a single radar for imaging. It receives just a few points to represent the scene, so the perception is poor. There can be other cars in the environment that you don’t see,” said Kshitiz Bansal, a computer science and engineering Ph.D. student at UC San Diego. “So if a single radar is causing this blindness, a multi-radar setup will improve perception by increasing the number of points that are reflected back.”
The team found that spacing two radar sensors 1.5 meters apart on the hood of the car was the optimal arrangement. “By having two radars at different vantage points with an overlapping field of view, we create a region of high-resolution, with a high probability of detecting the objects that are present,” Bansal said.

advertisement

A tale of two radars
The system overcomes another problem with radar: noise. It is common to see random points, which do not belong to any objects, appear in radar images. The sensor can also pick up what are called echo signals, which are reflections of radio waves that are not directly from the objects that are being detected.
More radars mean more noise, Bharadia noted. So the team developed new algorithms that can fuse the information from two different radar sensors together and produce a new image free of noise.
Another innovation of this work is that the team constructed the first dataset combining data from two radars.
“There are currently no publicly available datasets with this kind of data, from multiple radars with an overlapping field of view,” Bharadia said. “We collected our own data and built our own dataset for training our algorithms and for testing.”
The dataset consists of 54,000 radar frames of driving scenes during the day and night in live traffic, and in simulated fog conditions. Future work will include collecting more data in the rain. To do this, the team will first need to build better protective covers for their hardware.
The team is now working with Toyota to fuse the new radar technology with cameras. The researchers say this could potentially replace LiDAR. “Radar alone cannot tell us the color, make or model of a car. These features are also important for improving perception in self-driving cars,” Bharadia said.
Video: https://www.youtube.com/watch?v=5BrC0Jt4xUc&feature=emb_logo More

A small drone takes a test flight through a space filled with randomly placed cardboard cylinders acting as stand-ins for trees, people or structures. The algorithm controlling the drone has been trained on a thousand simulated obstacle-laden courses, but it’s never seen one like this. Still, nine times out of 10, the pint-sized plane dodges all the obstacles in its path.
This experiment is a proving ground for a pivotal challenge in modern robotics: the ability to guarantee the safety and success of automated robots operating in novel environments. As engineers increasingly turn to machine learning methods to develop adaptable robots, new work by Princeton University researchers makes progress on such guarantees for robots in contexts with diverse types of obstacles and constraints.
“Over the last decade or so, there’s been a tremendous amount of excitement and progress around machine learning in the context of robotics, primarily because it allows you to handle rich sensory inputs,” like those from a robot’s camera, and map these complex inputs to actions, said Anirudha Majumdar, an assistant professor of mechanical and aerospace engineering at Princeton.
However, robot control algorithms based on machine learning run the risk of overfitting to their training data, which can make algorithms less effective when they encounter inputs that differ from those they were trained on. Majumdar’s Intelligent Robot Motion Lab addressed this challenge by expanding the suite of available tools for training robot control policies, and quantifying the likely success and safety of robots performing in novel environments.
In three new papers, the researchers adapted machine learning frameworks from other arenas to the field of robot locomotion and manipulation. They turned to generalization theory, which is typically used in contexts that map a single input onto a single output, such as automated image tagging. The new methods are among the first to apply generalization theory to the more complex task of making guarantees on robots’ performance in unfamiliar settings. While other approaches have provided such guarantees under more restrictive assumptions, the team’s methods offer more broadly applicable guarantees on performance in novel environments, said Majumdar.
In the first paper, a proof of principle for applying the machine learning frameworks, the team tested their approach in simulations that included a wheeled vehicle driving through a space filled with obstacles, and a robotic arm grasping objects on a table. They also validated the technique by assessing the obstacle avoidance of a small drone called a Parrot Swing (a combination quadcopter and fixed-wing airplane) as it flew down a 60-foot-long corridor dotted with cardboard cylinders. The guaranteed success rate of the drone’s control policy was 88.4%, and it avoided obstacles in 18 of 20 trials (90%).

advertisement

The work, published Oct. 3 in the International Journal of Robotics Research, was coauthored by Majumdar; Alec Farid, a graduate student in mechanical and aerospace engineering; and Anoopkumar Sonar, a computer science concentrator from Princeton’s Class of 2021.
When applying machine learning techniques from other areas to robotics, said Farid, “there are a lot of special assumptions you need to satisfy, and one of them is saying how similar the environments you’re expecting to see are to the environments your policy was trained on. In addition to showing that we can do this in the robotic setting, we also focused on trying to expand the types of environments that we could provide a guarantee for.”
“The kinds of guarantees we’re able to give range from about 80% to 95% success rates on new environments, depending on the specific task, but if you’re deploying [an unmanned aerial vehicle] in a real environment, then 95% probably isn’t good enough,” said Majumdar. “I see that as one of the biggest challenges, and one that we are actively working on.”
Still, the team’s approaches represent much-needed progress on generalization guarantees for robots operating in unseen environments, said Hongkai Dai, a senior research scientist at the Toyota Research Institute in Los Altos, California.
“These guarantees are paramount to many safety-critical applications, such as self-driving cars and autonomous drones, where the training set cannot cover every possible scenario,” said Dai, who was not involved in the research. “The guarantee tells us how likely it is that a policy can still perform reasonably well on unseen cases, and hence establishes confidence on the policy, where the stake of failure is too high.”
In two other papers, to be presented Nov. 18 at the virtual Conference on Robot Learning, the researchers examined additional refinements to bring robot control policies closer to the guarantees that would be needed for real-world deployment. One paper used imitation learning, in which a human “expert” provides training data by manually guiding a simulated robot to pick up various objects or move through different spaces with obstacles. This approach can improve the success of machine learning-based control policies.

advertisement

To provide the training data, lead author Allen Ren, a graduate student in mechanical and aerospace engineering, used a 3D computer mouse to control a simulated robotic arm tasked with grasping and lifting drinking mugs of various sizes, shapes and materials. Other imitation learning experiments involved the arm pushing a box across a table, and a simulation of a wheeled robot navigating around furniture in a home-like environment.
The researchers deployed the policies learned from the mug-grasping and box-pushing tasks on a robotic arm in the laboratory, which was able to pick up 25 different mugs by grasping their rims between its two finger-like grippers — not holding the handle as a human would. In the box-pushing example, the policy achieved 93% success on easier tasks and 80% on harder tasks.
“We have a camera on top of the table that sees the environment and takes a picture five times per second,” said Ren. “Our policy training simulation takes this image and outputs what kind of action the robot should take, and then we have a controller that moves the arm to the desired locations based on the output of the model.”
A third paper demonstrated the development of vision-based planners that provide guarantees for flying or walking robots to carry out planned sequences of movements through diverse environments. Generating control policies for planned movements brought a new problem of scale — a need to optimize vision-based policies with thousands, rather than hundreds, of dimensions.
“That required coming up with some new algorithmic tools for being able to tackle that dimensionality and still be able to give strong generalization guarantees,” said lead author Sushant Veer, a postdoctoral research associate in mechanical and aerospace engineering.
A key aspect of Veer’s strategy was the use of motion primitives, in which a policy directs a robot to go straight or turn, for example, rather than specifying a torque or velocity for each movement. Narrowing the space of possible actions makes the planning process more computationally tractable, said Majumdar.
Veer and Majumdar evaluated the vision-based planners on simulations of a drone navigating around obstacles and a four-legged robot traversing rough terrain with slopes as high as 35 degrees — “a very challenging problem that a lot of people in robotics are still trying to solve,” said Veer.
In the study, the legged robot achieved an 80% success rate on unseen test environments. The researchers are working to further improve their policies’ guarantees, as well as assessing the policies’ performance on real robots in the laboratory.
The work was supported in part by the U.S. Office of Naval Research, the National Science Foundation, a Google Faculty Research Award and an Amazon Research Award. More

Movie ratings can determine a movie’s appeal to consumers and the size of its potential audience. Thus, they have an impact on a film’s bottom line. Typically, humans do the tedious task of manually rating a movie based on viewing the movie and making decisions on the presence of violence, drug abuse and sexual content.
Now, researchers at the USC Viterbi School of Engineering, armed with artificial intelligence tools, can rate a movie’s content in a matter of seconds, based on the movie script and before a single scene is shot. Such an approach could allow movie executives the ability to design a movie rating in advance and as desired, by making the appropriate edits on a script and before the shooting of a single scene. Beyond the potential financial impact, such instantaneous feedback would allow storytellers and decision-makers to reflect on the content they are creating for the public and the impact such content might have on viewers.
Using artificial intelligence applied to scripts, Shrikanth Narayanan, University Professor and Niki & C. L. Max Nikias Chair in Engineering, and a team of researchers from the Signal Analysis and Interpretation Lab (SAIL) at USC Viterbi, have demonstrated that linguistic cues can effectively signal behaviors on violent acts, drug abuse and sexual content (actions that are often the basis for a film’s ratings) about to be taken by a film’s characters.
Method:
Using 992 movie scripts that included violent, substance-abuse and sexual content, as determined by Common Sense Media, a non-profit organization that rates and makes recommendations for families and schools, the SAIL research team trained artificial intelligence to recognize corresponding risk behaviors, patterns and language.
The AI tool created receives as input all the script, processes it through a neural network and scans it for semantics and sentiment expressed. In the process, it classifies sentences and phrases as positive, negative, aggressive and other descriptors. The AI tool automatically classifies words and phrases into three categories: violence, drug abuse and sexual content.

advertisement

Victor Martinez, a doctoral candidate in computer science at USC Viterbi and the lead researcher on the study, which will appear in The Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing said, “Our model looks at the movie script, rather than the actual scenes, including e.g. sounds like a gunshot or explosion that occur later in the production pipeline. This has the benefit of providing a rating long before production to help filmmakers decide e.g. the degree of violence and whether it needs to be toned down.”
The research team also includes Narayanan, a professor of electrical and computer engineering, computer science and linguistics, Krishna Somandepalli, a Ph.D. candidate in Electrical and Computing Engineering at USC Viterbi, and Professor Yalda T. Uhls of UCLA’s Department of Psychology. They discovered many interesting connections between the portrayals of risky behaviors.
“There seems to be a correlation in the amount of content in a typical film focused on substance abuse and the amount of sexual content. Whether intentionally or not, filmmakers seem to match the level of substance abuse-related content with sexually explicit content,” said Martinez.
Another interesting pattern also emerged. “We found that filmmakers compensate for low levels of violence with joint portrayals of substance abuse and sexual content,” Martinez said.
Moreover, while many movies contain depictions of rampant drug-abuse and sexual content, the researchers found it highly unlikely for a film to have high levels of all three risky behaviors, perhaps because of Motion Picture Association (MPA) standards.

advertisement

They also found an interesting connection between risk behaviors and MPA ratings. As sexual content increases, the MPA appears to put less emphasis on violence/substance-abuse content. Thus, regardless of violent and substance abuse content, a movie with a lot of sexual content will likely receive an R rating.
Narayanan whose SAIL lab has pioneered the field of media informatics and applied natural language processing in order to bring awareness in the creative community about the nuances of storytelling, calls media “a rich avenue for studying human communication, interaction and behavior, since it provides a window into society.”
“At SAIL, we are designing technologies and tools, based on AI, for all stakeholders in this creative business — the writers, film-makers and producers — to raise awareness about the varied important details associated in telling their story on film,” Narayanan said.
“Not only are we interested in the perspective of the storytellers of the narratives they weave,” Narayanan said, “but also in understanding the impact on the audience and the ‘take-away’ from the whole experience. Tools like these will help raise societally-meaningful awareness, for example, through identifying negative stereotypes.”
Added Martinez: “In the future, I’m interested in studying minorities and how they are represented, particularly in cases of violence, sex and drugs.” More

Researchers at the National Institute of Standards and Technology (NIST) have invented a miniature thermometer with big potential applications such as monitoring the temperature of processor chips in superconductor-based quantum computers, which must stay cold to work properly.
NIST’s superconducting thermometer measures temperatures below 1 Kelvin (minus 272.15 ?C or minus 457.87 ?F), down to 50 milliKelvin (mK) and potentially 5 mK. It is smaller, faster and more convenient than conventional cryogenic thermometers for chip-scale devices and could be mass produced. NIST researchers describe the design and operation in a new journal paper.
Just 2.5 by 1.15 millimeters in size, the new thermometer can be embedded in or stuck to another cryogenic microwave device to measure its temperature when mounted on a chip. The researchers used the thermometer to demonstrate fast, accurate measurements of the heating of a superconducting microwave amplifier.
The technology is a spinoff of NIST’s custom superconducting sensors for telescope cameras, specifically microwave detectors delivered for the BLAST balloon.
“This was a fun idea that quickly grew into something very helpful,” group leader Joel Ullom said. “The thermometer allows researchers to measure the temperature of a wide range of components in their test packages at very little cost and without introducing a large number of additional electrical connections. This has the potential to benefit researchers working in quantum computing or using low-temperature sensors in a wide range of fields.”
The thermometer consists of a superconducting niobium resonator coated with silicon dioxide. The coating interacts with the resonator to shift the frequency at which it naturally vibrates. Scientists suspect this is due to atoms “tunneling” between two sites, a quantum-mechanical effect.
The NIST thermometer is based on a new application of the principle that the natural frequency of the resonator depends on the temperature. The thermometer maps changes in frequency, as measured by electronics, to a temperature. By contrast, conventional thermometers for sub-Kelvin temperatures are based on electrical resistance. They require wiring routed to room-temperature electronics, adding complexity and potentially causing heating and interference.
The NIST thermometer measures temperature in about 5 milliseconds (thousandths of a second), much faster than most conventional resistive thermometers at about one-tenth of a second. The NIST thermometers are also easy to fabricate in only a single process step. They can be mass produced, with more than 1,200 fitting on a 3-inch (approximately 75-millimeter) silicon wafer.

Story Source:
Materials provided by National Institute of Standards and Technology (NIST). Note: Content may be edited for style and length. More

During the first months of the COVID-19 pandemic, Joseph Lee McCauley, a physics professor at the University of Houston, was watching the daily data for six countries and wondered if infections were really growing exponentially. By extracting the doubling times from the data, he became convinced they were.
Doubling times and exponential growth go hand in hand, so it became clear to him that modeling based on past infections is impossible, because the rate changes unforeseeably from day to day due to social distancing and lockdown efforts. And the rate changes differ for each country based on the extent of their social distancing.
In AIP Advances, from AIP Publishing, McCauley explains how he combined math in the form of Tchebychev’s inequality with a statistical ensemble to understand how macroscopic exponential growth with different daily rates arise from person-to-person disease infection.
“Discretized ordinary chemical kinetic equations applied to infected, uninfected, and recovered parts of the population allowed me to organize the data, so I could separate the effects of social distancing and recoveries within daily infection rates,” McCauley said.
Plateauing without peaking occurs if the recovery rate is too low, and the U.S., U.K., and Sweden fall into that category. Equations cannot be iterated to look into the future, because tomorrow’s rate is unknown until it unfolds.
“Modelers tend to misapply the chemical kinetic equations as SIR (Susceptible, Infectious, or Recovered) or SEIR (Susceptible, Exposed, Infectious, or Recovered) models, because they are trying to generate future rates from past rates,” McCauley said. “But the past doesn’t allow you to use equations to predict the future in a pandemic, because social distancing changes the rates daily.”
McCauley discovered he could make a forecast within five seconds via hand calculator that is as good as any computer model by simply using infection rates for today and yesterday.
“Lockdowns and social distancing work,” said McCauley. “Compare Austria, Germany, Taiwan, Denmark, Finland, and several other countries that peaked in early April, with the U.S., U.K., Sweden, and others with no lockdown or half-hearted lockdowns — they’ve never even plateaued, much less peaked.”
He stresses that forecasting cannot foresee peaking or even plateauing. Plateauing does not imply peaking, and if peaking occurs, there is nothing in the data to show when it will happen. It happens when the recovery rate is greater than the rate of new infections.
“Social distancing and lockdowns reduce the infection rate but can’t cause peaking,” McCauley said. “Social distancing and recoveries are two separate terms within the daily kinetic rate equations.”
The implication of this work is that research money could be better spent than on expensive epidemic modeling.
“Politicians should know enough arithmetic to be given instruction on the implications,” McCauley said. “The effect of lockdowns and social distancing show up in the observed doubling times, and there is also a predicted doubling time based on two days, which serves as a good forecast of the future.” More

Pandemics are fueled, in part, by dense populations in large cities where networks of buildings, crowded sidewalks, and public transportation force people into tighter conditions. This contrasts with conditions in rural areas, where there is more space available per person.
According to common sense, being in less crowded areas during a pandemic is safer. But small town mayors want to keep people safe, too, and migration of people from cities to rural towns brings concerns. During the COVID-19 pandemic, closing national borders and borders between states and regions has been prevalent. But does it really help?
In a paper published in Chaos, by AIP Publishing, two researchers decided to put this hypothesis to the test and discover if confinement and travels bans are really effective ways to limit the spread of a pandemic disease. Specifically, they focused on the movement of people from larger cities to smaller ones and tested the results of this one-way migration.
“Instead of taking mobility, or the lack of mobility, for granted, we decided to explore how an altered mobility would affect the spreading,” author Massimiliano Zanin said. “The real answer lies in the sign of the result. People always assume that closing borders is good. We found that it is almost always bad.”
The model used by the authors is simplified, without many of the details that affect migration patterns and disease spread. But their focus on changes in population density indicates travel bans might be less effective than migration of people to less dense areas. The result was reduced spread of disease.
Zanin and collaborator David Papo placed a hypothetical group of people in two locations and assumed their travel was in random movement patterns. They used SIR dynamics, which is common in epidemiological studies of disease movement. SIR stands for susceptible, infected, and recovered — classifications used to label groups in a simulation and track disease spread according to their interactions.
They ran 10,000 iterations of the simulation to determine the resulting disease spread among people in two locations when migration is one way: from dense cities to less dense towns. They also studied the effect of “forced migration,” which moves healthy people out of dense cities at the onset of a pandemic.
The results showed that while movement from big cities to small towns might be slightly less safe for the people in small towns, overall, for a global pandemic situation, this reduction in the density of highly populated areas is better for the majority of all people.
“Collaboration between different governments and administrations is an essential ingredient towards controlling a pandemic, and one should consider the possibility of small-scale sacrifices to reach a global benefit,” Zanin said.

Story Source:
Materials provided by American Institute of Physics. Note: Content may be edited for style and length. More

Computer scientists have devised a new weapon against video game players who cheat. The researchers developed their approach for detecting cheaters using the popular first-person shooter game Counter-Strike. But the mechanism can work for any massively multiplayer online (MMO) game that sends data traffic to a central server. More