Computers Math

A new accelerator chip called “Hiddenite” that can achieve state-of-the-art accuracy in the calculation of sparse “hidden neural networks” with lower computational burdens has now been developed by Tokyo Tech researchers. By employing the proposed on-chip model construction, which is the combination of weight generation and “supermask” expansion, the Hiddenite chip drastically reduces external memory access for enhanced computational efficiency.
Deep neural networks (DNNs) are a complex piece of machine learning architecture for AI (artificial learning) that require numerous parameters to learn to predict outputs. DNNs can, however, be “pruned,” thereby reducing the computational burden and model size. A few years ago, the “lottery ticket hypothesis” took the machine learning world by storm. The hypothesis stated that a randomly initialized DNN contains subnetworks that achieve accuracy equivalent to the original DNN after training. The larger the network, the more “lottery tickets” for successful optimization. These lottery tickets thus allow “pruned” sparse neural networks to achieve accuracies equivalent to more complex, “dense” networks, thereby reducing overall computational burdens and power consumptions.
One technique to find such subnetworks is the hidden neural network (HNN) algorithm, which uses AND logic (where the output is only high when all the inputs are high) on the initialized random weights and a “binary mask” called a “supermask”. The supermask, defined by the top-k% highest scores, denotes the unselected and selected connections as 0 and 1, respectively. The HNN helps reduce computational efficiency from the software side. However, the computation of neural networks also requires improvements in the hardware components.
Traditional DNN accelerators offer high performance, but they do not consider the power consumption caused by external memory access. Now, researchers from Tokyo Institute of Technology (Tokyo Tech), led by Professors Jaehoon Yu and Masato Motomura, have developed a new accelerator chip called “Hiddenite,” which can calculate hidden neural networks with drastically improved power consumption. “Reducing the external memory access is the key to reducing power consumption. Currently, achieving high inference accuracy requires large models. But this increases external memory access to load model parameters. Our main motivation behind the development of Hiddenite was to reduce this external memory access,” explains Prof. Motomura. Their study will feature in the upcoming International Solid-State Circuits Conference (ISSCC) 2022, an international conference showcasing the pinnacles of achievement in integrated circuits.
“Hiddenite” stands for Hidden Neural Network Inference Tensor Engine and is the first HNN inference chip. The Hiddenite architecture  offers three-fold benefits to reduce external memory access and achieve high energy efficiency. The first is that it offers the on-chip weight generation for re-generating weights by using a random number generator. This eliminates the need to access the external memory and store the weights. The second benefit is the provision of the “on-chip supermask expansion,” which reduces the number of supermasks that need to be loaded by the accelerator. The third improvement offered by the Hiddenite chip is the high-density four-dimensional (4D) parallel processor that maximizes data re-use during the computational process, thereby improving efficiency.
“The first two factors are what set the Hiddenite chip apart from existing DNN inference accelerators,” reveals Prof. Motomura. “Moreover, we also introduced a new training method for hidden neural networks, called ‘score distillation,’ in which the conventional knowledge distillation weights are distilled into the scores because hidden neural networks never update the weights. The accuracy using score distillation is comparable to the binary model while being half the size of the binary model.”
Based on the hiddenite architecture, the team has designed, fabricated, and measured a prototype chip with Taiwan Semiconductor Manufacturing Company’s (TSMC) 40nm process. The chip is only 3mm x 3mm and handles 4,096 MAC (multiply-and-accumulate) operations at once. It achieves a state-of-the-art level of computational efficiency, up to 34.8 trillion or tera operations per second (TOPS) per Watt of power, while reducing the amount of model transfer to half that of binarized networks.
These findings and their successful exhibition in a real silicon chip are sure to cause another paradigm shift in the world of machine learning, paving the way for faster, more efficient, and ultimately more environment-friendly computing.
Story Source:
Materials provided by Tokyo Institute of Technology. Note: Content may be edited for style and length. More

New applications in energy, defense and telecommunications could receive a boost after a team from The University of Texas at Austin created a new type of “nanocrystal gel” — a gel composed of tiny nanocrystals each 10,000 times smaller than the width of a human hair that are linked together into an organized network.
The crux of the team’s discovery is that this new material is easily tunable. That is, it can be switched between two different states by changing the temperature. This means the material can work as an optical filter, absorbing different frequencies of light depending on whether it’s in a gelled state or not. So, it could be used, for example, on the outside of buildings to control heating or cooling dynamically. This type of optical filter also has applications for defense, particularly for thermal camouflage.
The gels can be customized for these wide-ranging applications because both the nanocrystals and the molecular linkers that connect them into networks are designer components. Nanocrystals can be chemically tuned to be useful for routing communications through fiber optic networks or keep the temperature of space craft steady on remote planetary bodies. Linkers can be designed to cause gels to switch based on ambient temperature or detection of environmental toxins.
“You could shift the apparent heat signature of an object by changing the infrared properties of its skin,” said Delia Milliron, professor and chair of the McKetta Department of Chemical Engineering in the Cockrell School of Engineering. “It could also be useful for telecommunications which all use infrared wavelengths.”
The new research is published in the recent issue of the journal Science Advances.
The team, led by graduate students Jiho Kang and Stephanie Valenzuela, did this work through the university’s Center for Dynamics and Control of Materials, a National Science Foundation Materials Research Science and Engineering Center that brings together engineers and scientists from across campus to collaborate on materials science research.
The lab experiments allowed the team to see the material change back and forth between its two states of gel and not-gel (that is, free-floating nanocrystals suspended in liquid) that they triggered by specific temperature changes.
Supercomputer simulations done at UT’s Texas Advanced Computing Center helped them to understand what was happening in the gel at the microscopic level when heat was applied. Based on theories of chemistry and physics, the simulations revealed the types of chemical bonds that hold the nanocrystals together in a network, and how those bonds break when hit with heat, causing the gel to break down.
This is the second unique nanocrystal gel created by this team, and they continue to pursue advances in this arena. Kang is currently working to create a nanocrystal gel that can change between four states, making it even more versatile and useful. That gel would be a blend of two different types of nanocrystals, each able to switch between states in response to chemical signals or temperature changes. Such tunable nanocrystal gels are called “programmable” materials.
Story Source:
Materials provided by University of Texas at Austin. Note: Content may be edited for style and length. More

Our face can unlock a smartphone, provide access to a secure building and speed up passport control at airports, verifying our identity for numerous purposes.
An international team of researchers from Australia, New Zealand and India has taken facial recognition technology to the next level, using a person’s expression to manipulate objects in a virtual reality setting without the use of a handheld controller or touchpad.
In a world first study led by the University of Queensland, human computer interaction experts used neural processing techniques to capture a person’s smile, frown and clenched jaw and used each expression to trigger specific actions in virtual reality environments.
One of the researchers involved in the experiment, University of South Australia’s Professor Mark Billinghurst, says the system has been designed to recognise different facial expressions via an EEG headset.
“A smile was used to trigger the ‘move’ command; a frown for the ‘stop’ command and a clench for the ‘action’ command, in place of a handheld controller performing these actions,” says Prof Billinghurst.
“Essentially we are capturing common facial expressions such as anger, happiness and surprise and implementing them in a virtual reality environment.”
The researchers designed three virtual environments — happy, neutral and scary — and measured each person’s cognitive and physiological state while they were immersed in each scenario. More

Commercially viable biofuel crops are vital to reducing greenhouse gas emissions, and a new tool developed by the Center for Advanced Bioenergy and Bioproducts Innovation (CABBI) should accelerate their development — as well as genetic editing advances overall.
The genomes of crops are tailored by generations of breeding to optimize specific traits, and until recently breeders were limited to selection on naturally occurring diversity. CRISPR/Cas9 gene-editing technology can change this, but the software tools necessary for designing and evaluating CRISPR experiments have so far been based on the needs of editing in mammalian genomes, which don’t share the same characteristics as complex crop genomes.
Enter CROPSR, the first open-source software tool for genome-wide design and evaluation of guide RNA (gRNA) sequences for CRISPR experiments, created by scientists at CABBI, a Department of Energy-funded Bioenergy Research Center (BRC). The genome-wide approach significantly shortens the time required to design a CRISPR experiment, reducing the challenge of working with crops and accelerating gRNA sequence design, evaluation, and validation, according to the study published in BMC Bioinformatics.
“CROPSR provides the scientific community with new methods and a new workflow for performing CRISPR/Cas9 knockout experiments,” said CROPSR developer Hans Müller Paul, a molecular biologist and Ph.D. student with co-author Matthew Hudson, Professor of Crop Sciences at the University of Illinois Urbana-Champaign. “We hope that the new software will accelerate discovery and reduce the number of failed experiments.”
To better meet the needs of crop geneticists, the team built software that lifts restrictions imposed by other packages on design and evaluation of gRNA sequences, the guides used to locate targeted genetic material. Team members also developed a new machine learning model that would not avoid guides for repetitive genomic regions often found in plants, a problem with existing tools. The CROPSR scoring model provided much more accurate predictions, even in non-crop genomes, the authors said.
“The goal was to incorporate features to make life easier for the scientist,” Müller Paul said. More

Understanding the nitty gritty of how molecules interact with each other in the real, messy, dynamic environment of a living body is a challenge that must be overcome in order to understand a host of diseases, such as Alzheimer’s.
Until now, researchers could capture the motion of a single molecule, and they could capture its rotation — how it tumbles as it bumps into surrounding molecules — but only by compromising 3D resolution.
Now, the lab of Matthew Lew, assistant professor of electrical and systems engineering at the McKelvey School of Engineering at Washington University in St. Louis, has developed an imaging method that provides an unprecedented look at a molecule as it spins and rolls through liquid, providing the most comprehensive picture yet of molecular dynamics collected using optical microscopes.
The research was published in a special issue of the Journal of Physical Chemistry B. The Feb. 17, 2022, Festschrift is dedicated to Nobel laureate William E. (W.E.) Moerner, an imaging pioneer, Washington University alumnus and mentor to more than 100 students over the years, including Lew.
Moerner was the first person to observe optical signatures of a single molecule; previously, researchers weren’t sure it was even possible to measure such signals.
Now Lew’s lab is the first to be able to visualize the orientation and direction of a molecule’s rotational movement — how it spins and wobbles — while it’s in a liquid system. More

You won’t see them move no matter how closely you watch.
You won’t hear their vibrations, even with your ear pressed to the cool sandstone.
But new research shows that the red rock towers found in Southern Utah and throughout the Colorado Plateau are in constant motion, vibrating with their own signature rhythms as unique as their dramatic profiles against the depth of the blue desert sky.
University of Utah researchers know well how rock towers and arches shimmy, twist and sway in response to far-off earthquakes, wind and even ocean waves. Their latest research compiles a first-of-its-kind dataset to show that the dynamic properties, i.e. the frequencies at which the rocks vibrate and the ways they deform during that vibration, can be largely predicted using the same mathematics that describe how beams in built structures resonate.
Knowing these properties is crucial to understanding the seismic stability of a rock tower and its susceptibility to hazardous vibrations. But it’s tough to get the needed data, partly because getting to the base of the towers often requires traveling through treacherous terrain — and then someone has to climb them to place a seismometer at the top.
With the help of experienced climbers, though, University of Utah researchers have now measured the dynamic properties of 14 rock towers and fins in Utah, creating a unique dataset with a variety of heights and tower shapes. More

Engineers at Caltech have developed an approach for quantum storage could help pave the way for the development of large-scale optical quantum networks.
The new system relies on nuclear spins — the angular momentum of an atom’s nucleus — oscillating collectively as a spin wave. This collective oscillation effectively chains up several atoms to store information.
The work, which is described in a paper published on February 16 in the journal Nature, utilizes a quantum bit (or qubit) made from an ion of ytterbium (Yb), a rare earth element also used in lasers. The team, led by Andrei Faraon (BS ’04), professor of applied physics and electrical engineering, embedded the ion in a transparent crystal of yttrium orthovanadate (YVO4) and manipulated its quantum states via a combination of optical and microwave fields. The team then used the Yb qubit to control the nuclear spin states of multiple surrounding vanadium atoms in the crystal.
“Based on our previous work, single ytterbium ions were known to be excellent candidates for optical quantum networks, but we needed to link them with additional atoms. We demonstrate that in this work,” says Faraon, the co-corresponding author of the Nature paper.
The device was fabricated at the Kavli Nanoscience Institute at Caltech, and then tested at very low temperatures in Faraon’s lab.
A new technique to utilize entangled nuclear spins as a quantum memory was inspired by methods used in nuclear magnetic resonance (NMR). More

Musicians are helping scientists analyze data, teach protein folding and make new discoveries through sound.
A team of researchers at the University of Illinois Urbana-Champaign is using sonification — the use of sound to convey information — to depict biochemical processes and better understand how they happen.
Music professor and composer Stephen Andrew Taylor; chemistry professor and biophysicist Martin Gruebele; and Illinois music and computer science alumna, composer and software designer Carla Scaletti formed the Biophysics Sonification Group, which has been meeting weekly on Zoom since the beginning of the pandemic. The group has experimented with using sonification in Gruebele’s research into the physical mechanisms of protein folding, and its work recently allowed Gruebele to make a new discovery about the ways a protein can fold.
Taylor’s musical compositions have long been influenced by science, and recent works represent scientific data and biological processes. Gruebele also is a musician who built his own pipe organ that he plays and uses to compose music. The idea of working together on sonification struck a chord with them, and they’ve been collaborating for several years. Through her company, Symbolic Sound Corp., Scaletti develops a digital audio software and hardware sound design system called Kyma that is used by many musicians and researchers, including Taylor.
Scaletti created an animated visualization paired with sound that illustrated a simplified protein-folding process, and Gruebele and Taylor used it to introduce key concepts of the process to students and gauge whether it helped with their understanding. They found that sonification complemented and reinforced the visualizations and that, even for experts, it helped increase intuition for how proteins fold and misfold over time. The Biophysics Sonification Group — which also includes chemistry professor Taras Pogorelov, former chemistry graduate student (now alumna) Meredith Rickard, composer and pipe organist Franz Danksagmüller of the Lübeck Academy of Music in Germany, and Illinois electrical and computer engineering alumnus Kurt Hebel of Symbolic Sound — described using sonification in teaching in the Journal of Chemical Education.
Gruebele and his research team use supercomputers to run simulations of proteins folding into a specific structure, a process that relies on a complex pattern of many interactions. The simulation reveals the multiple pathways the proteins take as they fold, and also shows when they misfold or get stuck in the wrong shape — something thought to be related to a number of diseases such as Alzheimer’s and Parkinson’s. More