In concentrated solar tower plants, hundreds of heliostats mirror reflected sunlight up to a central receiver. But aiming these mirrors presents a dilemma: point them all at the center, and risk damaging the receiver with intense heat, or risk losing valuable energy with a more “fuzzy” focus. IMAGE ©PSA
A new study from researchers at CIEMAT’s Plataforma Solar de Almería (PSA) in Spain has unveiled a novel application of artificial intelligence to improve the efficiency and safety of solar tower plant operation using reinforcement learning, a form of artificial intelligence that learns from experience.
An intelligent strategy to fine-tune mirror aim
What makes this approach revolutionary is an optimized heliostat aiming strategy that dynamically adjusts to changing solar conditions in real time, giving the solar field the ability to adapt continuously to changing conditions without human intervention.
AI heliostat aiming has already been carried out using mathematical models. However, these models can struggle with real-world complexities like temporary atmospheric distortion due to imperfections that develop in a mirror, changes in sunlight due to cloud movement, or a gust of wind bringing dust.
The PSA team introduced reinforcement learning (RL) using the Soft Actor-Critic (SAC) algorithm. This fully dynamic system continuously adjusts and improves its aiming strategy in response to real-world feedback, constantly recalculating optimal positions over the total receiver surface, enabling real-time decision-making without predefined aiming points.
The research was published in Applied Energy: Reinforcement learning for heliostat aiming: Improving the performance of Solar Tower plants.
The large group was led by Jose Carballo, Javier Bonilla, Jesús Fernández and Antonio Ávila at CIEMAT in collaboration with Manuel Berenguel and José Domingo Álvarez at the University of Almería, with Nicolas Cruz at the University of Granada.
Solarpaces spoke with two of its co-authors, Jose Carballo and Javier Bonilla in a call from Spain.
“I think this is the first approach that uses an AI agent to optimize the aiming strategy in real time to estimate the optimal points in the area of the receiver, depending on the design of the layout field, the season, the meteorological conditions, and also on the time of the day,” explained Bonilla.
“The main difference of this work is that we can use a continuous approach with AI. To my knowledge, no approaches were using AI for the aiming strategy, at least published works. In general, in CSP, there is not much autonomy and intelligence in the different components. So this is part of CSP Industry 4.0 or digitalization. We are trying to use smart approaches for different cases. This, in our opinion, can be huge because with just the same equipment and the same technology, by using this software, we can optimize the performance of the tower power plant, improving the efficiency and reducing the cost, and this is a great path to keep working on this so we can improve the technology.”
The CIEMAT team modeled their system on the solar tower facility at PSA, selecting 114 heliostats for optimal receiver coverage. The researchers trained neural networks to act as the system’s “brain,” with one network deciding actions (the actor) and another evaluating their effectiveness (the critic).
Training the agent was computationally intensive, taking approximately nine days on PSA’s HELIOSUN workstation with 200 CPUs. The training process involved simulating heliostat field operations over millions of episodes,
For safety, the team implemented constraints that would terminate training episodes if solar elevation dropped below 7°, spillage losses exceeded 60% or flux peaks surpassed 900 kW/m². This taught the AI to operate within safe parameters.
From fixed points to AI-driven varying aim points
“The traditional approach is to use fixed points in the receiver, so there is a group of heliostats always aimed at the same point,” explained Carballo.
“This is by no means an optimal solution because the energy absorbed in the receiver depends on many factors. Our new strategy or methodology considers more than the different meteorological conditions, DNI, elevation, azimuth etcetera. For example, for this study based on the PSA receiver – which is a volumetric receiver open to the air – it also considers the ambient temperature in Almeria because that affects this kind of receiver performance.”
The team conducted three sets of tests with different configurations, based on the PSA test site to evaluate their AI-driven strategy against traditional five-point aiming methods: to evaluate their AI-driven strategy against traditional five-point aiming methods:
In the first test, the AI agent adjusted fixed aiming points within heliostat clusters. This approach extended plant operation hours and increased absorbed power during midday, improving overall efficiency without additional costs. The results were impressive: the AI strategy increased absorbed power, especially midday, and extended plant operation at both ends of the solar day, improving overall efficiency at no additional cost.
In their second experiment, they increased the number of aiming points from 5 to 10, dividing the heliostats into smaller groups. This adjustment tested whether more aiming flexibility would improve system performance. Results were mixed: The additional aiming points allowed greater flexibility but required more training time. Using transfer learning to refine the previously trained agent, they achieved an 8.7% improvement in annual absorbed power compared to the five-point method. However, performance fluctuated during late hours, signaling room for refinement.
For the third test, the team set a reward function designed to shape solar flux distribution. This showed promise in optimizing receiver longevity but faced challenges like premature operation stops due to excessive losses. Despite increasing yearly absorbed energy by 3.9%, it encountered more failures and operational irregularities, suggesting that further optimization is needed.
But too much AI can be mechanically costly
The team had to balance the computation expense against the potential energy gain. So, they trained the AI to consider the cost/benefit of actions that ultimately involve mechanical work—forcing large and heavy heliostats to change the angle that they are tilted.
“The AI agent evaluates the current situation and estimates the optimal points,” said Carballo.
“But we don’t want to be always moving. We want to minimize the movement. So when we are training the agent, we penalize the movement so the agent will learn to move the heliostats as little as possible to obtain the maximum power in the receiver. It weighs the cost of the energy used to move those heliostats, whether that is cost-effective. It’s better to ignore a small gain.”
Another example of this cost balancing is that rather than make a particular heliostat make a major position change to a specific better-aiming point, they might instead select another heliostat already focused nearby.
“We have to take into account the previous aiming point state to generate the new aiming point because we can’t move a lot of heliostats suddenly. The movement has to be soft or gentle,” he added.
Real-World Potential
To date, the system has only been tested in a simulation of the PSA tower, but the idea is to implement this software to run an actual solar tower plant. The researchers believe their AI-driven approach could increase the overall efficiency of solar power plants. It’s very specific and personal for each plant.
“The agent has to be trained on each particular power plant,” explained Bonilla.
“It must be adapted to the plant’s design to optimize a lot depending on each particular plant. We have already trained it for our power plant and are running simulations for estimation. Currently, we are conducting an experimental campaign here at PSA. Also, we aim to refine the training process to better adapt it to different power plants.”
Dynamically optimizing heliostat aiming strategies will maximize energy output at the lowest cost and reduce receiver wear and tear.
This application of reinforcement learning AI to make solar tower power plant operations more intelligent and efficient is among many studies being conducted globally, that work on ways to make concentrated solar technology a viable alternative to fossil fuels to heat industrial processes and to manufacture solar fuels.
Future atmospheric attenuation impacts on central solar tower plants
