Computers Math

Researchers from Carnegie Mellon University, the University Hospital Bonn and the University of Bonn have created an open-source platform known as A-SOiD that can learn and predict user-defined behaviors, just from video. The results of the study have now been published in the journal Nature Methods.
“This technique works great at learning classifications for a variety of animal and human behaviors,” said Eric Yttri, Eberly Family Associate Professor of Biological Sciences at Carnegie Mellon. “This would not only work on behavior but also the behavior of anything if there are identifiable patterns: stock markets, earthquakes, proteomics. It’s a powerful pattern recognition machine.”
Unlike many artificial intelligence (AI) programs, A-SOiD is not a black box. Instead, the researchers allowed the program to re-learn what it did wrong. They first trained the program with a fraction of the dataset, with a focus on the program’s weaker beliefs. If the program was not certain, the algorithm would reinforce the belief of that training data.
Because A-SOiD was taught to focus on the algorithm’s uncertainty rather than treating all data the same, Alex Hsu, a recent Ph.D. alumnus from Carnegie Mellon, said that it avoids common biases found in other AI models.
AI tool does justice to every class in a data set
“It’s a different way of feeding data in,” Hsu said. “Usually, people go in with the entire data set of whatever behaviors they’re looking for. They rarely understand that the data can be imbalanced, meaning there could be a well-represented behavior in their set and a poorly represented behavior in their set. This bias could then propagate from the prediction process to the experimental findings. Our algorithm takes care of data balancing by only learning from weaker. Our method is better at fairly representing every class in a data set.”
Because A-SOiD is trained in a supervised fashion, it can be very precise. If given a dataset, it can determine the difference between a person’s normal shiver and the tremors of a patient with Parkinson’s disease. It also serves as a complementary method to their unsupervised behavior segmentation platform, B-SOiD, released two years ago.

Besides being an effective program, A-SOiD is highly accessible, capable of running on a normal computer and is available as open source on GitHub.
A-SOiD is accessible for everyone in science
Jens Tillmann, a postdoctoral researcher from the University of Bonn at the University Hospital Bonn, said that the idea of having this program open to all researchers was part of its impact.
“This project wouldn’t have been possible without the open science mindset that both of our labs, but also the entire community of neuroethology have shown in recent years,” Tillmann said. “I am excited to be part of this community and look forward to future collaborative projects with other experts in the field.”
Yttri and Martin K. Schwarz, principal investigator at the University Hospital Bonn and member of the Transdisciplinary Research Areas (TRA) “Life & Health” at the University of Bonn, plan on using A-SOiD in their own labs to further investigate the relationship between the brain and behavior. Yttri plans to use A-SOiD in conjunction with other tools to investigate the neural mechanisms underlying spontaneous behaviors. Schwartz will use A-SOiD in conjunction with other behavioral modalities for a fine-grained analysis of known behaviors in social interactions.
Both Yttri and Schwarz said they hope that A-SOiD will be used by other researchers across disciplines and countries.
“A-SOiD is an important development allowing an AI-based entry into behavioral classification and thus an excellent unique opportunity to better understand the causal relationship between brain activity and behavior,” Schwarz said. “We also hope that the development of A-SOiD will serve as an efficient trigger for forthcoming collaborative research projects focusing on behavioral research in Europe but also across the Atlantic.” More

Pollsters trying to predict presidential election results and physicists searching for distant exoplanets have at least one thing in common: They often use a tried-and-true scientific technique called Bayesian inference.
Bayesian inference allows these scientists to effectively estimate some unknown parameter — like the winner of an election — from data such as poll results. But Bayesian inference can be slow, sometimes consuming weeks or even months of computation time or requiring a researcher to spend hours deriving tedious equations by hand.
Researchers from MIT and elsewhere have introduced an optimization technique that speeds things up without requiring a scientist to do a lot of additional work. Their method can achieve more accurate results faster than another popular approach for accelerating Bayesian inference.
Using this new automated technique, a scientist could simply input their model and then the optimization method does all the calculations under the hood to provide an approximation of some unknown parameter. The method also offers reliable uncertainty estimates that can help a researcher understand when to trust its predictions.
This versatile technique could be applied to a wide array of scientific quandaries that incorporate Bayesian inference. For instance, it could be used by economists studying the impact of microcredit loans in developing nations or sports analysts using a model to rank top tennis players.
“When you actually dig into what people are doing in the social sciences, physics, chemistry, or biology, they are often using a lot of the same tools under the hood. There are so many Bayesian analyses out there. If we can build a really great tool that makes these researchers lives easier, then we can really make a difference to a lot of people in many different research areas,” says senior author Tamara Broderick, an associate professor in MIT’s Department of Electrical Engineering and Computer Science (EECS) and a member of the Laboratory for Information and Decision Systems and the Institute for Data, Systems, and Society.
Broderick is joined on the paper by co-lead authors Ryan Giordano, an assistant professor of statistics at the University of California at Berkeley; and Martin Ingram, a data scientist at the AI company KONUX. The paper was recently published in the Journal of Machine Learning Research.
Faster results

When researchers seek a faster form of Bayesian inference, they often turn to a technique called automatic differentiation variational inference (ADVI), which is often both fast to run and easy to use.
But Broderick and her collaborators have found a number of practical issues with ADVI. It has to solve an optimization problem and can do so only approximately. So, ADVI can still require a lot of computation time and user effort to determine whether the approximate solution is good enough. And once it arrives at a solution, it tends to provide poor uncertainty estimates.
Rather than reinventing the wheel, the team took many ideas from ADVI but turned them around to create a technique called deterministic ADVI (DADVI) that doesn’t have these downsides.
With DADVI, it is very clear when the optimization is finished, so a user won’t need to spend extra computation time to ensure that the best solution has been found. DADVI also permits the incorporation of more powerful optimization methods that give it an additional speed and performance boost.
Once it reaches a result, DADVI is set up to allow the use of uncertainty corrections. These corrections make its uncertainty estimates much more accurate than those of ADVI.
DADVI also enables the user to clearly see how much error they have incurred in the approximation to the optimization problem. This prevents a user from needlessly running the optimization again and again with more and more resources to try and reduce the error.

“We wanted to see if we could live up to the promise of black-box inference in the sense of, once the user makes their model, they can just run Bayesian inference and don’t have to derive everything by hand, they don’t need to figure out when to stop their algorithm, and they have a sense of how accurate their approximate solution is,” Broderick says.
Defying conventional wisdom
DADVI can be more effective than ADVI because it uses an efficient approximation method, called sample average approximation, which estimates an unknown quantity by taking a series of exact steps.
Because the steps along the way are exact, it is clear when the objective has been reached. Plus, getting to that objective typically requires fewer steps.
Often, researchers expect sample average approximation to be more computationally intensive than a more popular method, known as stochastic gradient, which is used by ADVI. But Broderick and her collaborators showed that, in many applications, this is not the case.
“A lot of problems really do have special structure, and you can be so much more efficient and get better performance by taking advantage of that special structure. That is something we have really seen in this paper,” she adds.
They tested DADVI on a number of real-world models and datasets, including a model used by economists to evaluate the effectiveness of microcredit loans and one used in ecology to determine whether a species is present at a particular site.
Across the board, they found that DADVI can estimate unknown parameters faster and more reliably than other methods, and achieves as good or better accuracy than ADVI. Because it is easier to use than other techniques, DADVI could offer a boost to scientists in a wide variety of fields.
In the future, the researchers want to dig deeper into correction methods for uncertainty estimates so they can better understand why these corrections can produce such accurate uncertainties, and when they could fall short.
This research was supported by a National Science Foundation CAREER Award and the U.S. Office of Naval Research. More

The electron is the basic unit of electricity, as it carries a single negative charge. This is what we’re taught in high school physics, and it is overwhelmingly the case in most materials in nature.
But in very special states of matter, electrons can splinter into fractions of their whole. This phenomenon, known as “fractional charge,” is exceedingly rare, and if it can be corralled and controlled, the exotic electronic state could help to build resilient, fault-tolerant quantum computers.
To date, this effect, known to physicists as the “fractional quantum Hall effect,” has been observed a handful of times, and mostly under very high, carefully maintained magnetic fields. Only recently have scientists seen the effect in a material that did not require such powerful magnetic manipulation.
Now, MIT physicists have observed the elusive fractional charge effect, this time in a simpler material: five layers of graphene — an atom-thin layer of carbon that stems from graphite and common pencil lead. They report their results in Nature.
They found that when five sheets of graphene are stacked like steps on a staircase, the resulting structure inherently provides just the right conditions for electrons to pass through as fractions of their total charge, with no need for any external magnetic field.
The results are the first evidence of the “fractional quantum anomalous Hall effect” (the term “anomalous” refers to the absence of a magnetic field) in crystalline graphene, a material that physicists did not expect to exhibit this effect.
“This five-layer graphene is a material system where many good surprises happen,” says study author Long Ju, assistant professor of physics at MIT. “Fractional charge is just so exotic, and now we can realize this effect with a much simpler system and without a magnetic field. That in itself is important for fundamental physics. And it could enable the possibility for a type of quantum computing that is more robust against perturbation.”
Ju’s MIT co-authors are lead author Zhengguang Lu, Tonghang Han, Yuxuan Yao, Aidan Reddy, Jixiang Yang, Junseok Seo, and Liang Fu, along with Kenji Watanabe and Takashi Taniguchi at the National Institute for Materials Science in Japan.

A bizarre state
The fractional quantum Hall effect is an example of the weird phenomena that can arise when particles shift from behaving as individual units to acting together as a whole. This collective “correlated” behavior emerges in special states, for instance when electrons are slowed from their normally frenetic pace to a crawl that enables the particles to sense each other and interact. These interactions can produce rare electronic states, such as the seemingly unorthodox splitting of an electron’s charge.
In 1982, scientists discovered the fractional quantum Hall effect in heterostructures of gallium arsenide, where a gas of electrons confined in a two-dimensional plane is placed under high magnetic fields. The discovery later won the group a Nobel Prize in Physics.
“[The discovery] was a very big deal, because these unit charges interacting in a way to give something like fractional charge was very, very bizarre,” Ju says. “At the time, there were no theory predictions, and the experiments surprised everyone.”
Those researchers achieved their groundbreaking results using magnetic fields to slow down the material’s electrons enough for them to interact. The fields they worked with were about 10 times stronger than what typically powers an MRI machine.
In August 2023, scientists at the University of Washington reported the first evidence of fractional charge without a magnetic field. They observed this “anomalous” version of the effect, in a twisted semiconductor called molybdenum ditelluride. The group prepared the material in a specific configuration, which theorists predicted would give the material an inherent magnetic field, enough to encourage electrons to fractionalize without any external magnetic control.

The “no magnets” result opened a promising route to topological quantum computing — a more secure form of quantum computing, in which the added ingredient of topology (a property that remains unchanged in the face of weak deformation or disturbance) gives a qubit added protection when carrying out a computation. This computation scheme is based on a combination of fractional quantum Hall effect and a superconductor. It used to be almost impossible to realize: One needs a strong magnetic field to get fractional charge, while the same magnetic field will usually kill the superconductor. In this case the fractional charges would serve as a qubit (the basic unit of a quantum computer).
Making steps
That same month, Ju and his team happened to also observe signs of anomalous fractional charge in graphene — a material for which there had been no predictions for exhibiting such an effect.
Ju’s group has been exploring electronic behavior in graphene, which by itself has exhibited exceptional properties. Most recently, Ju’s group has looked into pentalayer graphene — a structure of five graphene sheets, each stacked slightly off from the other, like steps on a staircase. Such pentalayer graphene structure is embedded in graphite and can be obtained by exfoliation using Scotch tape. When placed in a refrigerator at ultracold temperatures, the structure’s electrons slow to a crawl and interact in ways they normally wouldn’t when whizzing around at higher temperatures.
In their new work, the researchers did some calculations and found that electrons might interact with each other even more strongly if the pentalayer structure were aligned with hexagonal boron nitride (hBN) — a material that has a similar atomic structure to that of graphene, but with slightly different dimensions. In combination, the two materials should produce a moiré superlattice — an intricate, scaffold-like atomic structure that could slow electrons down in ways that mimic a magnetic field.
“We did these calculations, then thought, let’s go for it,” says Ju, who happened to install a new dilution refrigerator in his MIT lab last summer, which the team planned to use to cool materials down to ultralow temperatures, to study exotic electronic behavior.
The researchers fabricated two samples of the hybrid graphene structure by first exfoliating graphene layers from a block of graphite, then using optical tools to identify five-layered flakes in the steplike configuration. They then stamped the graphene flake onto an hBN flake and placed a second hBN flake over the graphene structure. Finally, they attached electrodes to the structure and placed it in the refrigerator, set to near absolute zero.
As they applied a current to the material and measured the voltage output, they started to see signatures of fractional charge, where the voltage equals the current multiplied by a fractional number and some fundamental physics constants.
“The day we saw it, we didn’t recognize it at first,” says first author Lu. “Then we started to shout as we realized, this was really big. It was a completely surprising moment.”
“This was probably the first serious samples we put in the new fridge,” adds co-first author Han. “Once we calmed down, we looked in detail to make sure that what we were seeing was real.”
With further analysis, the team confirmed that the graphene structure indeed exhibited the fractional quantum anomalous Hall effect. It is the first time the effect has been seen in graphene.
“Graphene can also be a superconductor,” Ju says. “So, you could have two totally different effects in the same material, right next to each other. If you use graphene to talk to graphene, it avoids a lot of unwanted effects when bridging graphene with other materials.”
For now, the group is continuing to explore multilayer graphene for other rare electronic states.
“We are diving in to explore many fundamental physics ideas and applications,” he says. “We know there will be more to come.”
This research is supported in part by the Sloan Foundation, and the National Science Foundation. More

In the blink of an eye, the unruly, superheated plasma that drives a fusion reaction can lose its stability and escape the strong magnetic fields confining it within the donut-shaped fusion reactor. These getaways frequently spell the end of the reaction, posing a core challenge to developing fusion as a non-polluting, virtually limitless energy source.
But a Princeton-led team composed of engineers, physicists, and data scientists from the University and the Princeton Plasma Physics Laboratory (PPPL) have harnessed the power of artificial intelligence to predict — and then avoid — the formation of a specific plasma problem in real time.
In experiments at the DIII-D National Fusion Facility in San Diego, the researchers demonstrated their model, trained only on past experimental data, could forecast potential plasma instabilities known as tearing mode instabilities up to 300 milliseconds in advance. While that leaves no more than enough time for a slow blink in humans, it was plenty of time for the AI controller to change certain operating parameters to avoid what would have developed into a tear within the plasma’s magnetic field lines, upsetting its equilibrium and opening the door for a reaction-ending escape.
“By learning from past experiments, rather than incorporating information from physics-based models, the AI could develop a final control policy that supported a stable, high-powered plasma regime in real time, at a real reactor,” said research leader Egemen Kolemen, associate professor of mechanical and aerospace engineering and the Andlinger Center for Energy and the Environment, as well as staff research physicist at PPPL.
The research opens the door for more dynamic control of a fusion reaction than current approaches, and it provides a foundation for using artificial intelligence to solve a broad range of plasma instabilities, which have long been obstacles to achieving a sustained fusion reaction. The team published their findings in Nature on February 21.
“Previous studies have generally focused on either suppressing or mitigating the effects of these tearing instabilities after they occur in the plasma,” said first author Jaemin Seo, an assistant professor of physics at Chung-Ang University in South Korea who performed much of the work while a postdoctoral researcher in Kolemen’s group. “But our approach allows us to predict and avoid those instabilities before they ever appear.”
Superheated plasma swirling in a donut-shaped device
Fusion takes place when two atoms — usually light atoms like hydrogen — come together to form one heavier atom, releasing a large amount of energy in the process. The process powers the Sun, and, by extension, makes life on Earth possible.

However, getting the two atoms to fuse is tricky, as it takes massive amounts of pressure and energy for the two atoms to overcome their mutual repulsion.
Fortunately for the Sun, its massive gravitational pull and extremely high pressures at its core allow fusion reactions to proceed. To replicate a similar process on the Earth, scientists instead use extremely hot plasma and extremely strong magnets.
In donut-shaped devices known as tokamaks — sometimes referred to as “stars in jars” — magnetic fields struggle to contain plasmas that reach above 100 million degrees Celsius, hotter than the center of the Sun.
While there are many types of plasma instabilities that can terminate the reaction, the Princeton team concentrated on solving tearing mode instabilities, a disturbance in which the magnetic field lines within a plasma actually break and create an opportunity for the plasma’s subsequent escape.
“Tearing mode instabilities are one of the major causes of plasma disruption, and they will become even more prominent as we try to run fusion reactions at the high powers required to produce enough energy,” said Seo. “They are an important challenge for us to solve.”
Fusing artificial intelligence and plasma physics
Since tearing mode instabilities can form and derail a fusion reaction in milliseconds, the researchers turned to artificial intelligence for its ability to quickly process and act in response to new data.

But the process to develop an effective AI controller was not as simple as trying out a few things on a tokamak, where time is limited, and the stakes are high.
Co-author Azarakhsh Jalalvand, a research scholar in Kolemen’s group, compared teaching an algorithm to run a fusion reaction in a tokamak to teaching someone how to fly a plane.
“You wouldn’t teach someone by handing them a set of keys and telling them to try their best,” Jalalvand said. “Instead, you’d have them practice on a very intricate flight simulator until they’ve learned enough to try out the real thing.”
Like developing a flight simulator, the Princeton team used data from past experiments at the DIII-D tokamak to construct a deep neural network capable of predicting the likelihood of a future tearing instability based on real-time plasma characteristics.
They used that neural network to train a reinforcement learning algorithm. Like a pilot trainee, the reinforcement learning algorithm could try out different strategies for controlling plasma, learning through trial and error which strategies worked and which did not within the safety of a simulated environment.
“We don’t teach the reinforcement learning model all of the complex physics of a fusion reaction,” Jalalvand said. “We tell it what the goal is — to maintain a high-powered reaction — what to avoid — a tearing mode instability — and the knobs it can turn to achieve those outcomes. Over time, it learns the optimal pathway for achieving the goal of high power while avoiding the punishment of an instability.”
While the model went through countless simulated fusion experiments, trying to find ways to maintain high power levels while avoiding instabilities, co-author SangKyeun Kim could observe and refine its actions.
“In the background, we can see the intentions of the model,” said Kim, a staff research scientist at PPPL and former postdoctoral researcher in Kolemen’s group. “Some of the chnges that the model wants are too rapid, so we work to smooth and calm the model. As humans, we arbitrate between what the AI wants to do and what the tokamak can accommodate.”
Once they were confident in the AI controller’s abilities, they tested it during an actual fusion experiment at the D-III D tokamak, observing as the controller made real-time changes to certain tokamak parameters to avoid the onset of an instability. These parameters included changing the shape of the plasma and the strength of the beams inputting power into the reaction.
“Being able to predict instabilities ahead of time can make it easier to run these reactions than current approaches, which are more passive,” said Kim. “We no longer have to wait for the instabilities to occur and then take quick corrective action before the plasma becomes disrupted.”
Powering into the future
While the researchers said the work is a promising proof-of-concept demonstrating how artificial intelligence can effectively control fusion reactions, it is only one of many next steps already ongoing in Kolemen’s group to advance the field of fusion research.
The first step is to get more evidence of the AI controller in action at the DIII-D tokamak, and then expand the controller to function at other tokamaks.
“We have strong evidence that the controller works quite well at DIII-D, but we need more data to show that it can work in a number of different situations,” said first author Seo. “We want to work toward something more universal.”
A second line of research involves expanding the algorithm to handle many different control problems at the same time. While the current model uses a limited number of diagnostics to avoid one specific type of instability, the researchers could provide data on other types of instabilities and give access to more knobs for the AI controller to tune.
“You could imagine one large reward function that turns many different knobs to simultaneously control for several types of instabilities,” said co-author Ricardo Shousha, a postdoc at PPPL and former graduate student in Kolemen’s group who provided support for the experiments at DIII-D.
And on the route to developing better AI controllers for fusion reactions, researchers might also gain more understanding of the underlying physics. By studying the AI controller’s decisions as it attempts to contain the plasma, which can be radically different than what traditional approaches might prescribe, artificial intelligence may be not only a tool to control fusion reactions but also a teaching resource.
“Eventually, it may be more than just a one-way interaction of scientists developing and deploying these AI models,” said Kolemen. “By studying them in more detail, they may have certain things that they can teach us too.”
The work was supported by the U.S. Department of Energy’s Office of Fusion Energy Sciences, as well as the National Research Foundation of Korea (NRF). The authors also acknowledge the use of the DIII-D National Fusion Facility, a Department of Energy Office of Science user facility. More

The expression “flawless from every angle” is commonly used to characterize a celebrity’s appearance. This doesn’t simply imply that they appear attractive from a specific viewpoint, but rather that their appeal remains consistent and appealing from various angles and perspectives. Recently, a research team from Pohang University of Science and Technology (POSTECH) has employed metasurface to fabricate angle-dependent holograms with multiple functions, capturing significant interest within the academic community.
A research team comprising Professor Junsuk Rho from the Department of Mechanical Engineering and the Department of Chemical Engineering and PhD candidate Joohoon Kim from the Department of Mechanical Engineering at the POSTECH created metasurface display technology. This technology allows holograms to display multiple images based on the observer’s viewing angle. The findings were recently published in Nano Letters, an international journal focusing on nanoscale research and applications.
Objects can appear distinct depending on the viewer’s position, a concept that can be harnessed in holographic technology to generate cinematic and realistic 3D holograms presenting different images based on the viewing angle. However, the current challenge lies in controlling light dispersion according to the angle, making the application of nano-optics in this context a complex endeavor.
The team addressed this challenge by leveraging metasurfaces, artificial nanostructures capable of precisely manipulating the characteristics of light. These metasurfaces are incredibly thin and lightweight, approximately one-hundredth the thickness of a human hair, making them promising for applications in miniaturized displays such as virtual and augmented reality devices. Through the use of metasurfaces, the team devised a system that controls light to convey only a specific phase of information at a given angle, resulting in diverse images based on the angle of incidence.
In their experiments, the team’s metasurface generated distinct 3D holographic images at angles of both +35 degrees and -35 degrees for left-circular polarization. Remarkably, the team achieved the production of different images for incident light by using a single metasurface, contingent on the specific polarization. Notably, the holographic display demonstrated an extensive viewing angle of 70 degrees (±35 degrees), enabling observers to perceive the three-dimensional hologram from various directions.
Professor Junsuk Rho who led the research explained, “We have successfully achieved an effective display from diverse angles.” He added, “We anticipate this technology will make significant contributions to the commercialization of technology in virtual and augmented reality displays, encrypted imaging, information storage, and other applications.”
The study was conducted with the support from the program of POSCO-POSTECH-RIST Convergence Research Center program, the STEAM Research Program of the National Research Foundation of Korea funded by the Ministry of Science and ICT, and the Alchemist fellowship of the Ministry of Trade, Industry and Energy. More

After a recent car crash, John Murray-Bruce wished he could have seen the other car coming. The crash reaffirmed the University of South Florida assistant professor of computer science and engineering’s mission to create a technology that could do just that: See around obstacles and ultimately expand one’s line of vision.
Using a single photograph, Murray-Bruce and his doctoral student, Robinson Czajkowski, created an algorithm that computes highly accurate, full-color three-dimensional reconstructions of areas behind obstacles — a concept that can not only help prevent car crashes, but help law enforcement experts in hostage situations, search-and-rescue and strategic military efforts.
“We’re turning ordinary surfaces into mirrors to reveal regions, objects and rooms that are outside our line of vision,” Murray-Bruce said. “We live in a 3D world, so obtaining a more complete 3D picture of a scenario can be critical in a number of situations and applications.”
As published in Nature Communications, Czajkowski and Murray-Bruce’s research is the first-of-its-kind to successfully reconstruct a hidden scene in 3D using an ordinary digital camera. The algorithm works by using information from the photo of faint shadows cast on nearby surfaces to create a high-quality reconstruction of the scene. While it is more technical for the average person, it could have broad applications.
“These shadows are all around us,” Czajkowski said. “The fact we can’t see them with our naked eye doesn’t mean they’re not there.”
The idea of seeing around obstacles has been a topic of science-fiction movies and books for decades. Murray-Bruce says this research takes significant strides in bringing that concept to life.
Prior to this work, researchers have only used ordinary cameras to create rough 2D reconstructions of small spaces. The most successful demonstrations of 3D imaging of hidden scene all required specialized, expensive equipment.

“Our work achieves a similar result using far less,” Czajkowski said. “You don’t need to spend a million dollars on equipment for this anymore.”
Czajkowski and Murray-Bruce expect it will be 10 to 20 years before the technology is robust enough to be adopted by law enforcement and car manufacturers. Right now, they plan to continue their research to further improve the technology’s speed and accuracy to expand its applications in the future, including self-driving cars to improve their safety and situational awareness.
“In just over a decade since the idea of seeing around corners emerged, there has been remarkable progress and there is accelerating interest and research activity in the area,” Murray-Bruce said. “This increased activity, along with access to better, more sensitive cameras and faster computing power form the basis for my optimism on how soon this technology will become practical for a wide range of scenarios.”
While the algorithm is still in the development phase, it is available for other researchers to test and reproduce in their own space. More

Fashioned from the same element found in sand and covered by intricate patterns, microchips power smartphones, augment appliances and aid the operation of cars and airplanes. Now, scientists at the U.S. Department of Energy’s (DOE) Princeton Plasma Physics Laboratory (PPPL) are developing computer simulation codes that will outperform current simulation techniques and aid the production of microchips using plasma, the electrically charged state of matter also used in fusion research. These codes could help increase the efficiency of the manufacturing process and potentially stimulate the renaissance of the chip industry in the United States.
“Because devices with microchips are essential to our daily lives, how and where they are made is a matter of national security,” said Igor Kaganovich, a principal research physicist who leads the low-temperature modeling group at PPPL. “Robust and reliable simulation tools that can accurately predict plasma behavior and shorten the manufacturing and design cycle of silicon chips could help the U.S. regain a leadership role in this field and maintain it for decades.”
Picking up the pace
One PPPL research effort involves reducing the time computers need to simulate microchip plasma reactors. This innovation would help private industry use more complex and accurate simulations widely and aid their drive to lower microchip costs. “Companies would like to use simulations to improve their processes, but they typically are computationally expensive,” said Andrew Tasman Powis, co-author of the paper reporting the results in Physics of Plasmas and computational research associate at PPPL. “We are doing our best to counter this trend.”
Physicists usually want simulations to reproduce plasma as accurately as possible, generating virtual pictures that reveal the intricacies of plasma behavior with very fine details. That process requires algorithms, programs following a set of rules, that simulate plasma in very short time increments and in small volumes of space. The catch is that such detailed simulations require powerful computers running for days or weeks at a time. That time frame is too long and too expensive for companies that want to use the simulations to improve their microchip manufacturing processes.
The researchers delved into plasma physics history to find already developed algorithms that might be able to shorten the amount of time necessary to simulate microchip plasma. The researchers found suitable algorithms from the 1980s; when tested, the algorithms demonstrated a capability to model microchip plasma systems in much less time and with only a small reduction in accuracy.
In essence, the researchers found that they could get good simulations even though they were modeling plasma particles within larger spaces and using longer time increments. “This development is important because it could save companies both time and money,” said Haomin Sun, the study’s lead researcher and a former graduate student in Princeton University’s Program in Plasma Physics, based at PPPL. “That means that with the same amount of computational resources, you can create more simulations. More simulations not only allow you to find ways to improve manufacturing, but also to learn more physics in general. We can make more discoveries using our limited resources.”
Related research led by Powis reinforces this possibility. In a paper published in Physics of Plasmas, Powis confirms that computer codes can generate accurate models of plasma particles while using virtual “cells” or small volumes of space that exceed a standard measure in plasma physics known as the Debye length. This development means that the codes can in effect use fewer cells and reduce the need for computing time. “This is good news because reducing the number of cells could lower the computational cost of the simulation and therefore improve performance,” Powis said.

The algorithms can simulate so-called “capacitively coupled plasma reactors,” which create the plasma that engineers use to etch narrow channels in a wafer of silicon. These tiny passageways form the microcircuitry that allows the microchip to function. “We are interested in modeling this process so we can learn how to control the properties of the plasma, predict what they would be like in a new machine, and then predict the etching properties so we can improve the process,” Powis said.
The team plans to test the algorithms further by adding the effects of different kinds of wall and electrode materials. “We want to continue to build confidence in these algorithms so we can be sure the results are accurate,” Powis said.
Recognizing and overcoming inherent limits
Another research effort focuses on errors that can creep into plasma simulations because of the inherent limitations of the simulation methods themselves, which model smaller numbers of plasma particles than are present in real plasma.
“When you simulate plasma, you would ideally like to track every single particle and know where it is at all times,” said Sierra Jubin, graduate student in the Princeton Program in Plasma Physics and lead author of the paper reporting the results in Physics of Plasmas. “But we don’t have infinite computing power, so we can’t do that.”
To get around this difficulty, researchers design code to represent millions of particles as one giant particle. Doing so simplifies the computer’s task, but also amplifies the interactions of the virtual mega-particles. As a result, a change in the proportion of particles moving at one speed versus how many are moving at another — a process known as thermalization — happens more quickly than it does in nature. Essentially, the simulation does not match reality.

“This is a problem because if we don’t address this issue, we won’t be modeling the phenomena as they actually occur in the world,” Jubin said. “And if we want to know how many electrons are moving at a particular speed, generating ions or reactive chemical species that interact with the materials used to make microchips, we won’t be getting an accurate picture.”
To compensate for these computational errors, the researchers found that they could make the mega-particle volumes larger and less dense, muting their interactions and slowing down the changes in particle velocities. “In effect, these results put boundaries on what is possible in microchip plasma simulations, point out constraints that we have to consider, and put forth some solutions,” Jubin said.
Jubin’s findings reinforce the notion that current simulation techniques must be improved. Whether because codes used today require small volume sizes and time increments that together slow simulations or because they produce errors based on computational requirements, scientists need new solutions. “This is actually a paradigm shift in the field,” Kaganovich said, “and PPPL is leading the way.”
This research was supported by PPPL’s laboratory-directed research and development (LDRD) program. Computer calculations were performed at the National Energy Research Scientific Computing Center (NERSC), a DOE user facility located at Lawrence Berkeley National Laboratory, as well as the ANTYA high-powered computing facility at India’s Institute for Plasma Research. The team included researchers from Princeton University, the Swiss Plasma Center at the Ecole Polytechnique Federale de Lausanne, India’s Birla Institute of Technology and Science, India’s Homi Bhabha National Institute, the University of Alberta at Edmonton, Applied Materials, Inc., and China’s Sino-French Institute of Nuclear Engineering and Technology. More

Despite significant progress in developing AI systems that can understand the physical world like humans do, researchers have struggled with modelling a certain aspect of our visual system: the perception of light.
“Determining the influence of light in a given photograph is a bit like trying to separate the ingredients out of an already baked cake.” explains Chris Careaga, a PhD student in the Computational Photography Lab at SFU. The task requires undoing the complicated interactions between light and surfaces in a scene. This problem is referred to as intrinsic decomposition, and has been studied for nearly half a century.
In a new paper published in the journal ACM Transactions on Graphics, researchers in the Computational Photography Lab develop an AI approach to intrinsic decomposition that works on a wide range of images. Their method automatically separates an image into two layers: one with only lighting effects and one with the true colours of objects in the scene. “The main innovation behind our work is to create a system of neural networks that are individually tasked with easier problems. They work together to understand the illumination in a photograph,” Careaga adds.
Although intrinsic decomposition has been studied for decades, SFU’s new invention is the first in the field to accomplish this task for any HD image that a person might take with their camera. “By editing the lighting and colours separately, a whole range of applications that are reserved for CGI and VFX become possible for regular image editing,” says Dr. Ya??z Aksoy, who leads the Computational Photography Lab at SFU. “This physical understanding of light makes it an invaluable and accessible tool for content creators, photo editors, and post-production artists, as well as for new technologies such as augmented reality and spatial computing.”
The group has since extended their intrinsic decomposition approach, applying it to the problem of image compositing: “When you insert an object or person from one image into another, it’s usually obvious that it’s edited since the lighting and colours don’t match” explains Careaga. “Using our intrinsic decomposition technique, we can alter the lighting of the inserted object to make it appear more realistic in the new scene.” In addition to publishing a paper on this, presented at SIGGRAPH Asia last December, the group has also developed a computer interface that allows users to interactively edit the lighting of these “composited” images. S. Mahdi H. Miangoleh, a PhD student in Aksoy’s lab, also contributed to this work.
Aksoy and his team plan to extend their methods to video for use in film post-production, and further develop AI capabilities in terms of interactive illumination editing. They emphasize a creativity-driven approach to AI in film production, aiming to empower independent and low-budget productions. To better understand the challenges in these production settings, the group has developed a computational photography studio at the Simon Fraser University campus where they conduct research in an active production environment. They also produce videos explaining their work which you can check out here: More