Nuclear fusion technology, long touted as an unlimited source of safe, clean energy, took a step toward reality with a machine learning algorithm that molds the fuel in a reactor’s core.
What’s new: Researchers at DeepMind and École Polytechnique Fédérale de Lausanne (EPFL) developed a reinforcement learning algorithm to manipulate hydrogen plasma — an extremely high-energy form of matter — into an optimal shape for energy production.
How it works: Reactors that confine plasma in a chamber known as a tokamak generate energy by pushing its atoms so close together that they fuse. A tokamak uses powerful magnetic coils to compress the plasma, heating it to the neighborhood of 100 million degrees Celsius to overcome the electrostatic force that normally pushes them apart. The authors trained a reinforcement learning model to control the voltage of 19 magnetic coils in a small, experimental tokamak reactor, enabling them to shape the plasma in ways that are consistent with maintaining an ongoing fusion reaction.
- The authors initially trained the algorithm in a simulated tokamak. Its reward function scored how well the plasma shape, position, and current matched the desired configuration.
- The training harnessed maximum a priori policy optimization, an actor-critic algorithm in which an actor learns to take actions that maximize rewards delivered by a critic. The actor, a vanilla neural network, learned how to control the simulated coils based on the current state of the plasma. The critic, a recurrent neural network, learned to predict the reward function’s score after each action.
- At inference, the critic was discarded while the actor continued to choose actions 10,000 times per second.
Results: In experimental runs with the real-world reactor, a previous algorithm controlled the coils to form a preliminary plasma shape before handing off the task to the authors’ model. Plasma can't be observed directly, so the authors calculated its shape and position properties based on measurements of the magnetic field within the tokamak. In five separate experiments, the controller formed the plasma into distinct shapes, such as a conventional elongated shape and a prospective “snowflake” shape, within particular tolerances (2 centimeters root mean squared error for shape, 5 kiloamperes root mean squared error for current passing through the plasma). In a novel feat, the algorithm maintained two separate plasma droplets for 200 milliseconds.
Behind the news: Conventional nuclear energy results from nuclear fission. Scientists have been trying to harness nuclear fusion since the 1950s. Yet no fusion reactor has generated more energy than it consumed. (The U.S. National Ignition Facility came the closest yet last year.) A growing number of scientists are enlisting machine learning to manage the hundreds of factors involved in sustaining a fusion reaction.
- Researchers at the Joint European Torus, another tokamak reactor, trained a variety of deep learning models on sensor data from within the reactor. A convolutional neural network visualized the plasma, reducing the time required to compute its behavior. A recurrent neural network predicted the risk of disruptions such as plasma escaping the magnetic field, which could damage the reactor’s walls. A variational autoencoder identified subtle anomalies in plasma that can cause such disruptions.
- Google AI and the startup TAE Technologies developed algorithms designed to improve fusion reactor performance. For instance, a set of Markov chain Monte Carlo models computes starting conditions that enable plasma to remain stable for longer periods of time.
Why it matters: Plasma in a tokamak, which is several times hotter than the sun and reverts to vapor if its electromagnetic container falters, is continually in flux. This work not only shows that deep learning can shape it in real time, it also opens the door to forming plasma in ways that might yield more energy. The next challenge: Scale up to a reactor large enough to produce meaningful quantities of energy.
We’re thinking: Fusion energy — if it ever works — would be a game changer for civilization. It’s thrilling to see deep learning potentially playing a key role in this technology.