Aurelia Butler

Innovative electronics is one of the many applications of modern plastics. Some recent research efforts have used plastic to improve the color realism of display technologies.
Now, in a study recently published in Angewandte Chemie International Edition, researchers from Osaka University and collaborating partners have developed a borane molecule that exhibits unusual light emission upon binding to fluoride. Incorporating their molecule into common plastic is straightforward, resulting in versatile materials for electronic display and chemical sensing applications.
A class of molecules known as triarylboranes (TABs) has photochemical properties that are useful in optics. For example, upon binding to an anion such as fluoride, disruption of the TAB electronic structure often does two things to the light emission: shortens the wavelength (blue-shift) and reduces the intensity (turn-off response). Lengthening the emission wavelength (red-shift) is nearly unprecedented because corresponding design principles are unavailable. Developing a new class of TAB that exhibits a red-shifted sensing response, and can be easily incorporated into plastic electronics and similar technologies, is the problem the researchers aimed to address.
“Our borane-based sensor exhibits a red-shifted response upon binding to an anion such as fluoride,” explains Nae Aota, lead author of the study. “Our method is based on reducing orbital energy gap of the molecule in the ground state and enhancing charge-transfer in the excited state by reversing the role of TAB from electronic acceptor to donor.”
A highlight of the researchers’ work is facile incorporation of a TAB-fluoride into polystyrene and poly(methyl methacrylate) polymer films. The polymer matrix did not impair the red-shifted light emission. In fact, one film exhibited warm white light — a highly desired property that mimics sunlight. Furthermore, the color of the light emission was finely tunable by simply adjusting the quantity of added fluoride.
“We’re excited by the versatility of our thin films,” says Youhei Takeda, senior author. “We can use the bipolarity of the phenazaboride to prepare plastic films ranging from blue to near-infrared, for displays and ultra-sensitive anion detection.”
This work is an important step forward in electronic display technologies. Furthermore, by tuning the selectivity of the TAB to anion binding (i.e., detecting only one type of anion even in the presence of other potentially competing anions), applications to highly sought sensing technologies will be straightforward. More

Researchers at the University of Würzburg have developed a method that can improve the performance of quantum resistance standards. It´s based on a quantum phenomenon called Quantum Anomalous Hall effect.
The precise measurement of electrical resistance is essential in industrial production or electronics — for example, in the manufacture of high-tech sensors, microchips and flight controls. “Very precise measurements are essential here, as even the smallest deviations can significantly affect these complex systems,” explains Professor Charles Gould, a physicist at the Institute for Topological Insulators at the University of Würzburg (JMU). “With our new measurement method, we can significantly improve the accuracy of resistance measurements, without any external magnetic field, using the Quantum Anomalous Hall Effect (QAHE).”
How the New Method Works
Many people may remember the classic Hall effect from their physics lessons: When a current flows through a conductor and it is exposed to a magnetic field, a voltage is created — the so-called Hall voltage. The Hall resistance, obtained by dividing this voltage by current, increases as the magnetic field strength increases. In thin layers and at large enough magnetic fields, this resistance begins to develop discreet steps with values of exactly h/ne2, where h is the Planck’s constant, e is the elementary charge, and n is an integer number. This is known as the Quantum Hall Effect because the resistance depends only on fundamental constants of nature (h and e), which makes it an ideal standard resistor.
The special feature of the QAHE is that it allows the quantum Hall effect to exist at zero magnetic field. “The operation in the absence of any external magnetic field not only simplifies the experiment, but also gives an advantage when it comes to determining another physical quantity: the kilogram. To define a kilogram, one has to measure the electrical resistance and the voltage at the same time,” says Gould “but measuring the voltage only works without a magnetic field, so the QAHE is ideal for this.”
Thus far, the QAHE was measured only at currents which are far too low for practical metrological use. The reason for this is an electric field that disrupts the QAHE at higher currents. The Würzburg physicists have now developed a solution to this problem. “We neutralize the electric field using two separate currents in a geometry we call a multi-terminal Corbino device.,” explains Gould. “With this new trick, the resistance remains quantized to h/e2 up to larger currents, making the resistance standard based on QAHE more robust.”
On the Way to Practical Application
In their feasibility study, the researchers were able to show that the new measurement method works at the precision level offered by basic d.c. techniques. Their next goal is to test the feasibility of this method using more precise metrological tools. To this end, the Würzburg group is working closely with the Physikalisch-Technische Bundesanstalt (German National Metrology Institute, PTB), who specialize in this kind of ultra-precise metrological measurements. Gould also notes: “This method is not limited to the QAHE. Given that conventional Quantum Hall Effect experiences similar electric field driven limitations at sufficiently large currents, this method can also improve the existing state of the art metrological standards, for applications where even larger currents are useful.”
The research was funded by the Free State of Bavaria, the German Research Foundation DFG, the Cluster of Excellence ct.qmat (Complexity and Topology in Quantum Matter) and the European Commission. More

University of Sydney researchers have used an AI-assisted application to help people write cartoon captions for cartoons published in The New Yorker Cartoon Caption Contest.
Twenty participants with little to no experience writing cartoon captions wrote 400 cartoon captions. 200 captions were written with the help from the AI tool, and the remainder were written without assistance.
A second group of 67 people then rated how funny these cartoon captions were. The researchers found jokes written with the help of the tool were found to be significantly funnier than those written without the tool. Comparatively, ratings for the AI assisted captions were almost 30 percent closer to the winning captions in The New Yorker Cartoon Caption Contest.
Participants said the tool helped them piece together humorous narratives and get started, helping to understand nuances and funny elements, and to come up with new ideas.
Almost half, 95 out of the 200 jokes written with the help of AI were also rated as funnier than the original cartoon captions by The New Yorker.
“The AI tool helps people be significantly funnier, but more importantly, it may be a cure for writer’s block,” said Dr Anusha Withana from the School of Computer Science and Digital Sciences Initiative.
AI helps non-native speakers be funny in a new language
Dr Withana and his team conceived the tool to help non-native speakers understand humour in their new language. The results also showed non-native speakers found the tool more helpful, bringing them 43 percent closer to the winning caption.

Born in Sri Lanka and having lived in Japan, Singapore, Germany and now Australia, Dr Withana said understanding local humour could often be a “minefield” for a new arrival.
“In a new country I would often find myself ‘off-key’,” he said. “For example, I once made a sarcastic comment that didn’t go down well in Germany. Here in Australia, it would have gotten a laugh.”
Hasindu Kariyawasam led the research project as an undergraduate research intern.
“Humour is such an important way to relate to others,” he said. “It is also important for emotional wellbeing and creativity, and for managing stress, depression, and anxiety. As a non-native speaker myself, I found the system helped me write jokes more easily, and it made the experience fun.”
How can AI help us understand humour?
The original aspiration for the research was to use technology to help get creative juices flowing and get words down on the page. Alister Palmer, a master’s student and amateur cartoonist conceived the idea to engage more people in cartooning.

The tool works through an algorithm which assesses incongruity. It analyses the words in a description of the cartoon and generates incongruous words as hints for the cartoonist.
For example, in one cartoon where a person is depicted wearing a rabbit suit to the office, the tool suggested the words “rabbit” and “soup” (derived from the incongruity with the word “suit”). One of the pilot study participants came up with the caption “I meant the rabbit soup, not suit.” The winning caption at The New Yorker competition was “It’s not just Henderson. Corporate laid off the entire bunny division.”
Professor Judy Kay said this approach means we can explain how the AI works: “With AI playing a bigger role in our lives, our team wanted to create this tool so that people can feel in control.”
Dr Withana said: “Ultimately, humans are still the ones creating the humour, but this research is a great example of how AI can augment and aid our social interactions.” More

Clear guidelines should be established for the generation and processing of synthetic data to ensure transparency, accountability and fairness, a new study says.
Synthetic data — generated through machine learning algorithms from original real-world data — is gaining prominence because it may provide privacy-preserving alternatives to traditional data sources. It can be particularly useful in situations where the actual data is too sensitive to share, too scarce, or of too low quality.
Synthetic data differs from real-world data as it is generated by algorithmic models known as synthetic data generators, such as Generative Adversarial Networks or Bayesian networks.
The study warns existing data protection laws that only apply to personal data are not well-equipped to regulate the processing of all types of synthetic data.
Laws such as the GDPR only apply to the processing of personal data. The GDPR’s definition of personal data encompasses ‘any information relating to an identified or identifiable natural person’. However, not all synthetic datasets are fully artificial — some may contain personal information or present a risk of re-identification. Fully synthetic datasets are, in principle, exempt from GDPR rules, except when there is a possibility of re-identification.
It remains unclear what level of re-identification risk would be sufficient to trigger their application in the context of fully synthetic data processing. That creates legal uncertainty and practical difficulties for the processing of such datasets.
The study, by Professor Ana Beduschi from the University of Exeter, is published in the journal Big Data and Society.
It says there should be clear procedures for calling to account those responsible for the generation and processing of synthetic data. There should be guarantees synthetic data is not generated and used in ways that bring adverse effects on individuals and society, such as perpetuating existing biases or creating new ones.
Professor Beduschi said: “Clear guidelines for all types of synthetic data should be established. They should prioritise transparency, accountability and fairness. Having such guidelines is especially important as generative AI and advanced language models such as DALL-E 3 and GPT-4 — which can both be trained on and generate synthetic data — may facilitate the dissemination of misleading information and have detrimental effects on society. Adhering to these principles could thus help mitigate potential harm and encourage responsible innovation.
“Accordingly, synthetic data should be clearly labelled as such and that information about its generation should be provided to users.” More

A team of computer scientists at the University of Massachusetts Amherst working on two different problems — how to quickly detect damaged buildings in crisis zones and how to accurately estimate the size of bird flocks — recently announced an AI framework that can do both. The framework, called DISCount, blends the speed and massive data-crunching power of artificial intelligence with the reliability of human analysis to quickly deliver reliable estimates that can quickly pinpoint and count specific features from very large collections of images. The research, published by the Association for the Advancement of Artificial Intelligence, has been recognized by that association with an award for the best paper on AI for social impact.
“DISCount came together as two very different applications,” says Subhransu Maji, associate professor of information and computer sciences at UMass Amherst and one of the paper’s authors. “Through UMass Amherst’s Center for Data Science, we have been working with the Red Cross for years in helping them to build a computer vision tool that could accurately count buildings damaged during events like earthquakes or wars. At the same time, we were helping ornithologists at Colorado State University and the University of Oklahoma interested in using weather radar data to get accurate estimates of the size of bird flocks.”
Maji and his co-authors, lead author Gustavo Pérez, who completed this research as part of his doctoral training at UMass Amherst, and Dan Sheldon, associate professor of information and computer sciences at UMass Amherst, thought they could solve the damaged-buildings-and-bird-flock problems with computer vision, a type of AI that can scan enormous archives of images in search of something particular — a bird, a rubble pile — and count it.
But the team was running into the same roadblocks on each project: “the standard computer visions models were not accurate enough,” says Pérez. “We wanted to build automated tools that could be used by non-AI experts, but which could provide a higher degree of reliability.”
The answer, says Sheldon, was to fundamentally rethink the typical approaches to solving counting problems.
“Typically, you either have humans do time-intensive and accurate hand-counts of a very small data set, or you have computer vision run less-accurate automated counts of enormous data sets,” Sheldon says. “We thought: why not do both?”
DISCount is a framework that can work with any already existing AI computer vision model. It works by using the AI to analyze the very large data sets — say, all the images taken of a particular region in a decade — to determine which particular smaller set of data a human researcher should look at. This smaller set could, for example, be all the images from a few critical days that the computer vision model has determined best show the extent of building damage in that region. The human researcher could then hand-count the damaged buildings from the much smaller set of images and the algorithm will use them to extrapolate the number of buildings affected across the entire region. Finally, DISCount will estimate how accurate the human-derived estimate is.
“DISCount works significantly better than random sampling for the tasks we considered,” says Pérez. “And part of the beauty of our framework is that it is compatible with any computer-vision model, which lets the researcher select the best AI approach for their needs. Because it also gives a confidence interval, it gives researchers the ability to make informed judgments about how good their estimates are.”
“In retrospect, we had a relatively simple idea,” says Sheldon. “But that small mental shift — that we didn’t have to choose between human and artificial intelligence, has let us build a tool that is faster, more comprehensive, and more reliable than either approach alone.” More

A new study published in the Proceedings of the National Academy of Sciences (PNAS) found AI-generated messages made recipients feel more “heard” than messages generated by untrained humans, and that AI was better at detecting emotions than these individuals. However, recipients reported feeling less heard when they learned a message came from AI.
As AI becomes more ubiquitous in daily life, understanding its potential and limitations in meeting human psychological needs becomes more pertinent. With dwindling empathetic connections in a fast-paced world, many are finding their human needs for feeling heard and validated increasingly unmet.
The research conducted by Yidan Yin, Nan Jia, and Cheryl J. Wakslak from the USC Marshall School of Business addresses a pivotal question: Can AI, which lacks human consciousness and emotional experience, succeed in making people feel heard and understood?
“In the context of an increasing loneliness epidemic, a large part of our motivation was to see whether AI can actually help people feel heard,” said the paper’s first author, Yidan Yin, a postdoctoral researcher at the Lloyd Greif Center for Entrepreneurial Studies at USC Marshall.
The team’s findings highlight not only the potential of AI to augment human capacity for understanding and communication, but raises important conceptual questions about the meaning of being heard and practical questions about how best to leverage AI’s strengths to support greater human flourishing.
In an experiment and subsequent follow-up study, “we identified that while AI demonstrates enhanced potential compared to non-trained human responders to provide emotional support, the devaluation of AI responses poses a key challenge for effectively deploying AI’s capabilities,” said Nan Jia, associate professor of strategic management.
The USC Marshall research team investigated people’s feelings of being heard and other related perceptions and emotions after receiving a response from either AI or a human. The survey varied both the actual source of the message and the ostensible source of the message: Participants received messages that were actually generated by an AI or by a human responder, with the information that it was either AI or human generated.

“What we found was that both the actual source of the message and the presumed source of the message played a role,” said Cheryl Wakslak, associate professor of management and organization at USC Marshall. “People felt more heard when they received an AI than a human message, but when they believed a message came from AI this made them feel less heard.”
AI bias
Yin noted that their research “basically finds a bias against AI. It’s useful, but they don’t like it.”
Perceptions about AI are bound to change, added Wakslak, “Of course these effects may change over time, but one of the interesting things we found was that the two effects we observed were fairly similar in magnitude. Whereas there is a positive effect of getting an AI message, there is a similar degree of response bias when a message is identified as coming from AI, leading the two effects to essentially cancel each other out.”
Individuals further reported an “uncanny valley” response — a sense of unease when made aware that the empathetic response originated from AI, highlighting the complex emotional landscape navigated by AI-human interactions.
The research survey also asked participants about their general openness to AI, which moderated some of the effects, explained Wakslak.

“People who feel more positively toward AI don’t exhibit the response penalty as much and that’s intriguing because over time, will people gain more positive attitudes toward AI?” she posed. “That remains to be seen … but it will be interesting to see how this plays out as people’s familiarity and experience with AI grows.”
AI offers better emotional support
The study highlighted important nuances. Responses generated by AI were associated with increased hope and lessened distress, indicating a positive emotional effect on recipients. AI also demonstrated a more disciplined approach than humans in offering emotional support and refrained from making overwhelming practical suggestions.
Yin explained that, “Ironically, AI was better at using emotional support strategies that have been shown in prior research to be empathetic and validating. Humans may potentially learn from AI because a lot of times when our significant others are complaining about something, we want to provide that validation, but we don’t know how to effectively do so.”
Instead of AI replacing humans, the research points to different advantages of AI and human responses. The advanced technology could become a valuable tool, empowering humans to use AI to help them better understand one another and learn how to respond in ways that provide emotional support and demonstrate understanding and validation.
Overall, the paper’s findings have important implications for the integration of AI into more social contexts. Leveraging AI’s capabilities might provide an inexpensive scalable solution for social support, especially for those who might otherwise lack access to individuals who can provide them with such support. However, as the research team notes, their findings suggest that it is critical to give careful consideration to how AI is presented and perceived in order to maximize its benefits and reduce any negative responses. More

New research suggests that materials commonly overlooked in computer chip design actually play an important role in information processing, a discovery which could lead to faster and more efficient electronics. Using advanced imaging techniques, an international team led by Penn State researchers found that the material that a semiconductor chip device is built on, called the substrate, responds to changes in electricity much like the semiconductor on top of it.
The researchers worked with the semiconductor material, vanadium dioxide, which they said shows great potential as an electronic switch. They also studied how vanadium dioxide interacts with the substrate material titanium dioxide and said they were surprised to discover that there seems to be an active layer in the substrate that behaves similarly to the semiconductor material on top of it when the semiconductor switches between an insulator — not letting electricity flow — and a metal — letting electricity flow. The revelation that substrates can play an active role in semiconductor processes is significant for designing future materials and devices, said study lead Venkatraman Gopalan, professor of materials science and engineering and of physics at Penn State.
“New ideas are needed for smaller and faster electronics in order to keep up with Moore’s law,” said Gopalan, the corresponding author of the study in Advanced Materials. “One idea being pursued is materials, such as vanadium dioxide, that can switch between metal — the one state — and insulator — the zero state — states in a trillionth of a second. This is known as undergoing metal-insulator transitions.”
The potential of vanadium dioxide as a metal-to-insulator transistor is well-documented and the material is considered promising for semiconductor technology due to its low energy consumption, Gopalan said. However, the material’s properties are still not fully understood, and until now, it has usually been observed in isolation rather than while functioning in a real device.
Vanadium dioxide has strongly correlated electronic effects, meaning the repulsion between electrons interferes with the device, so cannot be ignored as is currently done in silicon-based electronics. This characteristic can result in materials with novel functionalities such as high-temperature superconductivity and enhanced magnetic properties.
“The underlying physics of this material is less understood, and its performance in a device geometry is even lesser understood,” Gopalan said. “If we can make them work, there will be a renaissance in electronics. In particular, neuromorphic computing — where computer systems that take inspiration from the brains of living systems with neurons — could seriously benefit by using such devices.”
The team investigated vanadium dioxide in a device rather than in isolation, applying a voltage to it to make it switch from an insulating to a conducting state. They used the Advanced Photon Source (APS) at Argonne National Laboratory, which uses powerful X-ray beams to study the behavior and structure of materials on the atomic level. When mapping the spatial and temporal response of the material to the switching event, the researchers observed unexpected changes to the structure of the material and substrate.

“What we found was that as the vanadium dioxide film changes to a metal, the whole film channel bulges, which is very surprising,” Gopalan said. “Normally it is supposed to shrink. So clearly something else was going on in the film geometry that was missed before.”
The APS X-ray penetrated through the vanadium dioxide film and into the titanium dioxide (TiO2) substrate — which is normally considered an electrically and mechanically passive material — that the thin film was grown on.
“We found to our great surprise that this substrate is very much active, jiving and responding in completely surprising ways as the film switches from an insulator to a metal and back, when the electrical pulses arrive,” Gopalan said. “This is like watching the tail wagging the dog, which stumped us for a long while. This surprising and previously overlooked observation completely changes how we need to view this technology.”
To understand these findings, the theory and simulation effort — led by Long-Qing Chen, Hamer Professor of Materials Science and Engineering, professor of engineering science and mechanics and of mathematics at Penn State — developed a theoretical framework to explain the entire process of the film and the substrate bulging instead of shrinking. When their model incorporated naturally occurring missing oxygen atoms in this material of two types, charged and uncharged, the experimental results could be satisfactorily explained.
“These neutral oxygen vacancies hold a charge of two electrons, which they can release when the material switches from an insulator to a metal,” Gopalan said. “The oxygen vacancy left behind is now charged and swells up, leading to the observed surprising swelling in the device. This can also happen in the substrate. All of these physical processes are beautifully captured in the phase-field theory and modelling performed in this work for the first time by the postdoc Yin Shi in Professor Chen’s group.”
Gopalan credited the multidisciplinary team’s combined expertise in material growth, synthesis, structure analysis and synchrotron beamline operation with the new understanding. Using a collaborative approach led by Greg Stone, a physical scientist with the U.S. Army and the lead experimental author, and Yin Chi, postdoctoral scholar at Penn State and the lead theory author, the researchers disentangled the material’s responses and observed them individually using phase field simulations, a simulation that helps scientists understand material changes over time by depicting various states of matter in a virtual setting.

“By bringing these experts together and pooling our understanding of the problem, we were able to go far beyond our individual scope of expertise and discover something new,” said Roman Engel-Herbert, director of the Paul Drude Institute of Solid State Electronics in Berlin, Germany, and co-author of the study whose group grew these films along with Darrell Schlom’s group at Cornell University. “Recognizing the potential of functional materials necessitates an appreciation of their broader context, just as complex scientific challenges can only be solved through widening our individual perspectives.”
The collaboration enabled both a significant amount of progress to happen in a short period of time and work to be done in a shorter period of time, and brought in a variety of perspectives from multiple disciplines.
The responses themselves require further investigation, researchers said, but they believe that understanding them will assist in identifying previously unknown capabilities of vanadium dioxide, including potential yet-to-be discovered phenomena in the TiO2 substrate that was considered passive before this study. The study itself unfolded over 10 years, Gopalan noted, including validating the results.
“This is what it takes to go from interesting science to a working device you can hold in the palm of your hand,” Gopalan said. “Experiments and theory are complex and require large-scale collaborative teams working closely together over an extended period of time to solve difficult problems that could have a large impact. We hope and expect that this will accelerate the progress towards a new generation of electronic devices.”
Prior to his current position, Stone completed a postdoctoral fellowship at Penn State. Along with Gopalan, Engel-Herbert, Chen, Schlom, Stone and Chi, other authors of the paper include Matthew Jerry, graduate student, and Vladimir Stoica, research associate professor, both from Penn State; Hanjong Paik from Cornell University; Zhonghou Cai and Haidan Wen from Argonne National Laboratory, and Suman Datta from the Georgia Institute of Technology. The Department of Energy primarily supported this work. The U.S. National Science Foundation supported the film growth for this study. More

In Star Trek: The Next Generation, Captain Picard and the crew of the U.S.S. Enterprise leverage the holodeck, an empty room capable of generating 3D environments, to prepare for missions and to entertain themselves, simulating everything from lush jungles to the London of Sherlock Holmes. Deeply immersive and fully interactive, holodeck-created environments are infinitely customizable, using nothing but language: the crew has only to ask the computer to generate an environment, and that space appears in the holodeck.
Today, virtual interactive environments are also used to train robots prior to real-world deployment in a process called “Sim2Real.” However, virtual interactive environments have been in surprisingly short supply. “Artists manually create these environments,” says Yue Yang, a doctoral student in the labs of Mark Yatskar and Chris Callison-Burch, Assistant and Associate Professors in Computer and Information Science (CIS), respectively. “Those artists could spend a week building a single environment,” Yang adds, noting all the decisions involved, from the layout of the space to the placement of objects to the colors employed in rendering.
That paucity of virtual environments is a problem if you want to train robots to navigate the real world with all its complexities. Neural networks, the systems powering today’s AI revolution, require massive amounts of data, which in this case means simulations of the physical world. “Generative AI systems like ChatGPT are trained on trillions of words, and image generators like Midjourney and DALLE are trained on billions of images,” says Callison-Burch. “We only have a fraction of that amount of 3D environments for training so-called ’embodied AI.’ If we want to use generative AI techniques to develop robots that can safely navigate in real-world environments, then we will need to create millions or billions of simulated environments.”
Enter Holodeck, a system for generating interactive 3D environments co-created by Callison-Burch, Yatskar, Yang and Lingjie Liu, Aravind K. Joshi Assistant Professor in CIS, along with collaborators at Stanford, the University of Washington, and the Allen Institute for Artificial Intelligence (AI2). Named for its Star Trek forebear, Holodeck generates a virtually limitless range of indoor environments, using AI to interpret users’ requests. “We can use language to control it,” says Yang. “You can easily describe whatever environments you want and train the embodied AI agents.”
Holodeck leverages the knowledge embedded in large language models (LLMs), the systems underlying ChatGPT and other chatbots. “Language is a very concise representation of the entire world,” says Yang. Indeed, LLMs turn out to have a surprisingly high degree of knowledge about the design of spaces, thanks to the vast amounts of text they ingest during training. In essence, Holodeck works by engaging an LLM in conversation, using a carefully structured series of hidden queries to break down user requests into specific parameters.
Just like Captain Picard might ask Star Trek’s Holodeck to simulate a speakeasy, researchers can ask Penn’s Holodeck to create “a 1b1b apartment of a researcher who has a cat.” The system executes this query by dividing it into multiple steps: first, the floor and walls are created, then the doorway and windows. Next, Holodeck searches Objaverse, a vast library of premade digital objects, for the sort of furnishings you might expect in such a space: a coffee table, a cat tower, and so on. Finally, Holodeck queries a layout module, which the researchers designed to constrain the placement of objects, so that you don’t wind up with a toilet extending horizontally from the wall.
To evaluate Holodeck’s abilities, in terms of their realism and accuracy, the researchers generated 120 scenes using both Holodeck and ProcTHOR, an earlier tool created by AI2, and asked several hundred Penn Engineering students to indicate their preferred version, not knowing which scenes were created by which tools. For every criterion — asset selection, layout coherence and overall preference — the students consistently rated the environments generated by Holodeck more favorably.

The researchers also tested Holodeck’s ability to generate scenes that are less typical in robotics research and more difficult to manually create than apartment interiors, like stores, public spaces and offices. Comparing Holodeck’s outputs to those of ProcTHOR, which were generated using human-created rules rather than AI-generated text, the researchers found once again that human evaluators preferred the scenes created by Holodeck. That preference held across a wide range of indoor environments, from science labs to art studios, locker rooms to wine cellars.
Finally, the researchers used scenes generated by Holodeck to “fine-tune” an embodied AI agent. “The ultimate test of Holodeck,” says Yatskar, “is using it to help robots interact with their environment more safely by preparing them to inhabit places they’ve never been before.”
Across multiple types of virtual spaces, including offices, daycares, gyms and arcades, Holodeck had a pronounced and positive effect on the agent’s ability to navigate new spaces.
For instance, whereas the agent successfully found a piano in a music room only about 6% of the time when pre-trained using ProcTHOR (which involved the agent taking about 400 million virtual steps), the agent succeeded over 30% of the time when fine-tuned using 100 music rooms generated by Holodeck.
“This field has been stuck doing research in residential spaces for a long time,” says Yang. “But there are so many diverse environments out there — efficiently generating a lot of environments to train robots has always been a big challenge, but Holodeck provides this functionality.” More