ExaSMR simulation toolkit advances nuclear reactor design

Alternatives to carbon-producing energy sources are becoming ever more imperative as climate change shows its effects on the Earth and in our daily lives. Although fossil fuels still generate much of the electricity in the United States, utilities are increasingly adding renewable sources such as wind and solar to their energy portfolios.

In 2021, 20% of the country’s electricity came from renewables vs. 61% from fossil fuels. But by 2050, both are projected to supply 44% as renewables continue to surge and fossil fuels decline.

Where does that leave old-fashioned nuclear energy?

For the past 20 years, fission reactors have produced a nearly unchanging portion of the nation’s electricity: around 20%. But that percentage could start increasing soon as new design approaches and reactor technologies promise to transform the nuclear power industry.

The advent of small modular reactors (SMRs) and advanced reactor concepts (ARCs) signals a new generation of fission power. Unlike most commercial nuclear reactors today, SMRs are substantially smaller and use standardized designs, thereby reducing construction costs and production time. Meanwhile, ARCs explore new technologies to produce fission power more efficiently and safely. Both efforts use computer simulations to predict the viability of proposed designs and to improve them. But running such fluid dynamics and neutron transport models can be computationally demanding and expensive, thus limiting their use by industry.

Exascale SMR (ExaSMR), which is a high-performance computing (HPC) software project supported by the U.S. Department of Energy’s (DOE’s) Exascale Computing Project (ECP), aims to make large-scale nuclear reactor simulations easier to access, cheaper to run, and more accurate than the current state of the art.

“By accurately predicting the nuclear reactor fuel cycle, ExaSMR reduces the number of physical experiments that reactor designers would perform to justify the fuel use. In large part, that’s what simulation is buying companies: a predictive capability that tells you how certain features will perform so that you don’t need to physically construct or perform as many experiments, which are enormously expensive,” said Steven Hamilton, ExaSMR project leader and R&D scientist in the HPC Methods for Nuclear Applications Group at DOE’s Oak Ridge National Laboratory (ORNL).

The ExaSMR project is working to provide the nuclear industry’s engineers with the highest-resolution simulations of reactor systems to date and in turn help advance the future of fission power.

Coupling physics codes into a more powerful whole

Nuclear reactors generate electricity by splitting uranium nuclei to release energy in fuel rods. Water is heated by this energy release and becomes hot enough to turn into steam that spins electricity-producing turbines. ExaSMR integrates the most reliable computer codes available for modeling the different physics of this operation, thereby creating a toolkit that can predict a reactor design’s entire fission process. This toolkit includes Shift and OpenMC for neutron particle transport and reactor depletion and NekRS for thermal fluid dynamics.

Although most of these codes are already well established in science and industry, the ExaSMR team has given them a complete HPC makeover. For the past 6 years, researchers from ORNL, Argonne National Laboratory (Argonne), the Massachusetts Institute of Technology, and Pennsylvania State University (Penn State) have been optimizing the codes for the new generation of GPU-accelerated, exascale-class supercomputers, such as ORNL’s Frontier and Argonne’s upcoming Aurora.

OpenMC’s development has been led by Paul Romano, and significant GPU-optimization work for Aurora has been conducted by John Tramm; Romano and Tramm are computational scientists at Argonne. Shift was originally authored by Thomas Evans, group leader for ORNL’s HPC Methods for Nuclear Applications Group, and is now optimized for Frontier.

Both codes use Monte Carlo methods—computational techniques that use large numbers of random samples to calculate the probable outcomes of models—to simulate how neutrons that move through the nuclear reactor interact with isotopes, such as uranium, and cause the fission events that create heat in the reactor’s fuel rods. The two codes also model how these isotopes evolve over time, which predicts the reactor’s life span.

NekRS—a computational fluid dynamics solver developed by Elia Merzari, associate professor of nuclear engineering at Penn State—essentially describes how the water will move and behave when heated by the reactor’s fuel cylinders. The ExaSMR team’s ENRICO (Exascale Nuclear Reactor Investigative Code), also developed by Romano, enables OpenMC and NekRS to interact.

“What we’re doing in ExaSMR is a coupled physics simulation between the neutron transport and the fluid dynamics—you have these two physics codes that are talking back and forth to each other,” Hamilton said. “The neutron transport is telling you where the heat is generated. That heat becomes a source term for the fluid dynamics calculation. The fluid dynamics tells you what temperature is resulting from that heat source. And then you can adjust the parameters in the simulation until both the neutron transport and the fluid dynamics are in agreement.”

ExaSMR’s ability to accurately model in high resolution the whole reactor process—thus predicting how much heat the reactor’s fission events will produce, the ability of the reactor to transfer that heat to power generators, and the life expectancy of the entire system—provides engineers with key insights to ensure the safety and efficiency of their reactor designs.

Planning ahead to avoid obstacles

When the ECP and the ExaSMR project started in 2016 to prepare software apps and tools for exascale systems, those supercomputers didn’t exist yet—not even on paper. The team was challenged with determining how to best optimize codes for systems that were years away from being finalized.

“At the beginning of the project, we didn’t even know exactly what the architectures of the exascale machines would look like,” Hamilton said. “It was definitely a challenge to design our codes while looking ahead with confidence that we would be able to run effectively on the upcoming systems.”

The team faced not only the task of coupling these separate codes for their new use-case scenario of large-scale, high-fidelity reactor simulations but also the challenge of adapting them to new computing architectures with yet unknown processors. This uncertainty meant pushing the limits of compilers and software packages by testing use cases that were far beyond what the software had been tested for at the time—and it began an ongoing process of constant communication.

“It required us to interact and iterate with the hardware vendors and the companies that produce the software to make sure that their products can handle our use cases. We have researchers who have been in almost daily contact with people who are writing compilers for the machines and trying to identify issues and implement features that are needed to compile and run our codes,” Hamilton said.

Success at last

The team’s interaction with vendors and developers paid off with substantial improvements in the methods and algorithms used by the codes, yielding large gains in performance. With its initial runs on Frontier, ExaSMR blew past the team’s speedup goals for its codes.

Shift performed SMR simulations on up to 8,192 nodes of Frontier and involved simulating over 250 billion neutron histories per iteration. The performance achieved in these simulations is more than 100× that of the baseline simulations performed on the Titan supercomputer (i.e., the US’s most powerful supercomputer in 2016) and more than double the performance improvement goal of 50× from Titan to Frontier.

NekRS performed SMR simulations on up to 6,400 nodes of Frontier, including the largest reactor fluid-flow simulation performed to date with over 1 billion spatial elements. The peak performance on Frontier reflects a more than 125× improvement over corresponding baseline simulations performed on Titan.

What’s ahead for ExaSMR?

Partnering with Westinghouse, which is a producer of commercial nuclear power technology, the ExaSMR team applied for a DOE Office of Advanced Scientific Computing Research Leadership Computing Challenge grant. Westinghouse wants to evaluate the impact of using higher-enrichment fuel than what is currently used in their reactors. Running ExaSMR on Frontier will allow them to perform high-fidelity simulations to predict how different types of fuels would perform if used in a currently operating reactor.

Likewise, Hamilton wants to apply ExaSMR to current ARC technologies being explored in the power industry, such as those being developed as part of the DOE Office of Nuclear Energy’s Advanced Reactor Demonstration Program. The program works with commercial companies to help speed up the demonstration of advanced reactors by providing initial funding. Two such reactors are slated for near-term deployment by 2027: X-energy’s Xe-100 pebble-bed reactor and TerraPower’s Natrium sodium-cooled fast reactor. Five additional designs from Kairos, Westinghouse, BWX Technologies, Holtec International, and Southern Company are ramping up for longer-term deployment.

Hamilton foresees ExaSMR becoming an indispensable tool for companies that are entering a new era of nuclear power.

“Various companies are exploring different types of reactor designs today, and the high-performance, high-fidelity simulations that we’re developing have a lot of appealing features for designers,” Hamilton said. “It’s unlikely, in the near future, that we’ll have enough confidence in simulations that they would fully replace experiments, but if we can reduce the number of experiments that are performed, then there can be huge financial gains for these companies.”