Operator Learning and Neural Operators
A conventional neural network maps one finite-dimensional vector to another. In partial differential equation problems, however, both the input and the output are often functions. Examples include taking a spatially varying coefficient field as input and producing the corresponding PDE solution, predicting the solution field at later times from an initial condition, or computing a steady-state or time-dependent solution from an external forcing function. In such settings, the learning target is not an individual solution function, but the solution operator that maps an entire input function to an output function.
This perspective is important because standard grid-dependent neural networks cannot naturally represent continuous physical operators. Refining the grid changes the dimensions of the input and output vectors, while moving observation points or adding new points changes the vector representation of the input itself. If the network size is readjusted to match the resolution, both the number of parameters and the receptive field may change as well. Consequently, even for the same physical problem, one may need to construct a separate model for each grid resolution.
A central property sought by Neural Operators is discretization invariance. A model should be able to accept a function observed at arbitrary input points while using a fixed number of trainable parameters, evaluate the solution at desired output points, and converge to a well-defined continuous operator as the grid is progressively refined.
Architecture of Neural Operators
A representative Neural Operator consists of a lifting stage that maps the input function into a latent representation with a larger channel dimension, a sequence of kernel layers, and a projection stage that maps the latent representation back to the output space of the physical variables. Lifting, projection, channel mixing, bias terms, and nonlinear activations are pointwise operations applied independently at each spatial location. They increase expressive power across channels but do not directly transmit information between distinct spatial points.
Interactions across the full spatial domain are handled by a nonlocal kernel operation. The new feature at a point is formed by integrating features at other points over the entire domain, weighted by a kernel. In computation, for example, the continuous integral can be approximated by a weighted sum over observation points. The kernel may be fixed and depend only on position, conditioned on the input coefficient field, or conditioned on the current latent state so as to represent nonlinear interactions. A Neural Operator layer can therefore be interpreted as combining a nonlocal pathway with a pointwise channel-mixing pathway, followed by a nonlinearity. This structure enables local transformations and long-range information transfer to be carried out simultaneously.
Fourier Neural Operator
FNO implements the nonlocal kernel operation efficiently in the Fourier domain. If the kernel is assumed to depend on the relative displacement between two locations rather than on their absolute coordinates, the kernel integral can be interpreted as a convolution. Since convolution becomes frequency-wise multiplication in the Fourier domain, a global kernel operation can be computed efficiently by transforming the input features with an FFT, applying learnable channel mixing at each frequency, and then returning to physical space with an inverse FFT.
The overall FNO architecture lifts the input function, applies a sequence of Fourier layers, and finally performs a projection. Each Fourier layer contains two pathways. The first is a global spectral pathway that transforms the input into the Fourier domain, applies a complex-valued channel-mixing matrix to each retained frequency mode, and then performs an inverse transform. The second is a local pathway that applies a 1×1 convolution or linear channel mixing independently at each spatial point. The outputs of the two pathways are summed and passed through a nonlinear activation to produce the features for the next layer. It is important to note that the term local pathway here refers to a pointwise channel transformation that acts only at each individual location.
FNO does not learn all Fourier coefficients. Instead, it retains only a prescribed number of modes along each spatial axis. This finite spectral parameterization controls the number of parameters and computational cost, and it efficiently represents the dominant low-frequency structure of a physical system. However, as the number of retained modes decreases, high-frequency information is structurally removed. Increasing the number of modes substantially causes parameter and memory costs to grow rapidly. In particular, the size of the selectable frequency grid grows sharply as the spatial dimension increases.
Limitations of FNO and Motivation for LOGLO-FNO
Standard neural networks generally exhibit spectral bias: they learn low-frequency components first and high-frequency components later. In FNO, this optimization bias is compounded by a structural constraint because the spectral pathway explicitly truncates high-frequency modes. This choice may not be problematic when the inputs and outputs are mostly smooth and dominated by low frequencies. In problems where small spatial scales and abrupt variations are important, however, predictions can become excessively smooth and lose fine-scale structure. Examples include high-Reynolds-number turbulence, compressible flows containing shocks and contact discontinuities, complex wave propagation, and reaction–diffusion patterns.
Moreover, because Fourier basis functions are defined over the entire domain, they are well suited to representing global correlations and large-scale flow structures. Efficiently representing a small vortex or a sharp boundary localized to a particular region, however, may require many frequency components. It can therefore be inefficient to assign both global low-frequency structure and local high-frequency structure to a single spectral branch. LOGLO-FNO begins from this observation and processes global structure, local structure, and high-frequency residuals through separate pathways before combining them.
Architecture of LOGLO-FNO
LOGLO-FNO uses three parallel states. The first is a global branch that receives the entire spatial domain as input. As in standard FNO, this pathway retains only a limited number of Fourier modes and learns long-range interactions and dominant large-scale structures across the full domain.
The second is a local branch that partitions the input domain into small, non-overlapping patches and applies Fourier operations within each patch. Because each patch is much smaller than the full domain, retaining all Fourier modes available within each patch is less costly than retaining every mode over the global domain. This pathway is responsible for locally occurring mid- and high-frequency features, including small vortices, steep gradients, and fine structures near boundaries and discontinuities.
The third is the high-frequency propagation, or HFP, branch. The input is first average-pooled to a lower resolution and then interpolated back to its original resolution, producing a smooth approximation dominated by low frequencies. Subtracting this approximation from the original input extracts a high-frequency residual. The HFP branch propagates this residual directly through a channel MLP without subjecting it to Fourier-mode truncation. It therefore encourages information that would otherwise be removed by the global branch to be preserved through deeper layers of the network.
The three pathways are transformed into latent features under the same lifting framework and interact repeatedly through multiple LOGLO layers. The final output is obtained by projecting the global state. Nevertheless, because the local-branch and HFP-branch features are merged into the global branch in every LOGLO layer, the final global representation incorporates information from all three pathways.
LOGLO Layer
Within each LOGLO layer, the global and local branches each apply a spectral convolution and pointwise 1×1 channel mixing. The two results are activated and then passed through a channel MLP together with a skip or gating pathway. The output of the local branch is reassembled from the patch arrangement into the original spatial layout and added to the global-branch output. The HFP branch is passed through a channel MLP without a Fourier transform and is added at the same fusion point. Finally, a nonlinear activation is applied to the sum of all three pathways to form the global state for the next layer.
A key aspect of this design is that the local branch contributes directly to the global representation at every layer. The global pathway supplies large-scale dynamics and coupling across patches, the local pathway supplements fine-scale structure within each patch, and the HFP pathway bypasses frequency truncation to carry residual information that might otherwise be lost. The three pathways impose different inductive biases and are functionally complementary.
Patch-Based Local Spectral Operator
In the local branch, the full domain is partitioned into a collection of non-overlapping patches. The local kernel operation at a point integrates only over the patch containing that point, so its receptive field is explicitly localized. FFT and inverse FFT operations are applied independently to each patch, but the learnable spectral weights are shared across all patches. Therefore, increasing the number of patches does not create a separate set of parameters for each patch. Instead, the same local operator is repeatedly applied throughout the spatial domain.
Because the patches are small, the local branch can retain all frequency modes available within each patch. A high frequency in the global domain corresponds to a small spatial scale, but the same structure can be represented with relatively few modes in patch coordinates. This makes it possible to represent local fine-scale structure more efficiently than by indiscriminately increasing the number of modes in the global branch. The compensatory effect of the local branch is particularly strong when the global branch retains only a small number of modes.
The use of non-overlapping patches can nevertheless introduce artificial discontinuities or boundary artifacts at patch interfaces. The global branch provides coupling across the full domain and may mitigate this issue, but it cannot be guaranteed to eliminate it completely.
Radially Binned Frequency-Aware Loss
LOGLO-FNO modifies not only the architecture but also the training objective. First, the difference between the prediction and the ground truth is computed in physical space, and this error field is transformed into the Fourier domain. In the two-dimensional Fourier plane, frequency coefficients at similar distances from the origin are grouped into the same radial shell. The error energy within each shell is summed and then averaged over the batch, physical variables, and time dimensions. This produces an error measure indexed by wavenumber magnitude, independent of direction.
Based on radial distance, the frequency domain is divided into low-, mid-, and high-frequency bands, with representative cutoffs at 4 and 12. Low-frequency error is left to be controlled sufficiently by the standard MSE, while the additional frequency loss mainly assigns weight to mid- and high-frequency errors. This prevents optimization from focusing only on high-energy low-frequency components and encourages more accurate reconstruction of non-dominant frequencies that carry less energy but remain physically important.
An advantage of this loss is that it summarizes a two-dimensional spectral error as a one-dimensional error profile indexed by wavenumber magnitude. Radial averaging, however, does not distinguish angles and therefore discards information about anisotropic errors concentrated in particular directions. When directional structure in a flow or wave field is essential, a radial loss alone cannot capture every spectral discrepancy.
Experimental Setup
LOGLO-FNO was evaluated on six PDE problems representing different physical phenomena. The two-dimensional problems were Kolmogorov flow at Reynolds number 5000, compressible Navier–Stokes turbulence, Wave-Gauss with a spatially varying propagation speed, the compressible Euler four-quadrant Riemann problem containing shocks and contact discontinuities, and a nonlinearly coupled reaction–diffusion problem. The three-dimensional problem was a turbulent radiative mixing layer in an astrophysical flow.
The comparison models included the base FNO, Factorized FNO, U-FNO, NO-LIDK with localized integral and differential kernels, U-Net, the latent spectral model LSM, and the transformer-based PDE solver Transolver. Evaluation metrics included RMSE and nRMSE for overall spatial-domain error, as well as boundary error, conservation error, maximum error, and variance-normalized vRMSE. Frequency-domain metrics included low-, mid-, and high-frequency fRMSE, together with MELR and WLR, which quantify deviations of the predicted energy spectrum from the ground truth.
The evaluation was divided into a one-step setting, in which the model directly predicts the state one time step ahead, and an autoregressive rollout setting, in which each model prediction is repeatedly used as the next input. One-step evaluation measures immediate approximation accuracy, whereas autoregressive evaluation assesses stability as small errors accumulate over time.
Results on Kolmogorov Flow
Two-dimensional Kolmogorov flow at Reynolds number 5000 is a fully turbulent problem containing small vortices and a broad frequency range, making it an appropriate test case for the purpose of LOGLO-FNO. In the one-step evaluation, the base FNO achieved an nRMSE of 0.147, whereas LOGLO-FNO achieved 0.107, corresponding to an improvement of approximately 27.21%. Low-, mid-, and high-frequency fRMSE decreased by approximately 11.22%, 11.88%, and 24.64%, respectively. Although the combined NO-LIDK model attained the same nRMSE level, LOGLO-FNO produced lower errors in all three frequency bands. In particular, the largest relative improvement occurred in the high-frequency band, consistent with the intended roles of the local spectral branch and the HFP branch.
In the five-step autoregressive evaluation, the nRMSE of LOGLO-FNO was approximately 18.39% lower than that of the base FNO. Low-, mid-, and high-frequency fRMSE decreased by approximately 16.86%, 21.62%, and 17.26%, respectively. Although the errors of all models increased under repeated prediction, LOGLO-FNO exhibited a lower rate of error growth, indicating that preserving high-frequency and local structure also contributes to long-horizon rollout stability.
In a mode sweep over the number of global Fourier modes, LOGLO-FNO achieved lower bandwise fRMSE than the base FNO for nearly every mode setting. The difference was especially large when very few global modes were retained, because the local branch compensated for the limited expressivity of the global spectral pathway. LOGLO-FNO remained superior as the number of modes increased, with the most pronounced gap appearing in the high-frequency band. These results suggest that separating global and local representations can be more efficient than simply increasing the number of modes in the global branch.
In the analysis of radially binned spectral errors, LOGLO-FNO also produced the lowest error curve over most of the displayed frequency range. The largest differences between models appeared in the transition from low to mid frequencies. This is the range in which interactions between large-scale flow and small-scale structure become prominent, suggesting that the local branch and the frequency-aware loss help reconstruct not only the extreme high-frequency tail but also intermediate scales.
Results on the Compressible Euler Riemann Problem
The compressible Euler four-quadrant Riemann problem is difficult for smooth, low-frequency-oriented models because shocks and contact discontinuities form in the density, pressure, and velocity fields. In one-step evaluation, LOGLO-FNO reduced nRMSE by approximately 27.90% relative to the base FNO. Low-, mid-, and high-frequency fRMSE decreased by approximately 38.56%, 50.71%, and 31.58%, respectively. In the five-step autoregressive evaluation, nRMSE decreased by approximately 31.75%, while the mid-frequency error was reduced by 50%. These results show that LOGLO-FNO is effective not only for turbulence but also for hyperbolic problems involving shocks and discontinuities.
In qualitative comparisons of the density and pressure fields, both the base FNO and LOGLO-FNO reproduced the overall flow structure. However, the base FNO's errors were broadly distributed along high-gradient regions such as shock waves and contact surfaces. In the LOGLO-FNO error maps, both the intensity and spatial extent of the errors in these regions were reduced, indicating that the local branch better preserved sharp structures. Nevertheless, errors near discontinuities did not disappear completely, confirming that extremely abrupt changes remain challenging for Neural Operators.
Ablation Study on the 3D Turbulent Radiative Mixing Layer
The ablation study on the three-dimensional turbulent radiative mixing layer separately evaluated the contributions of the local branch, the HFP branch, and the frequency-aware loss. The base FNO achieved an nRMSE of 0.297, while the complete LOGLO-FNO, using both the local and HFP branches together with the frequency loss, achieved 0.258. Low-, mid-, and high-frequency fRMSE and vRMSE were also lowest for the complete configuration.
When the frequency loss was removed while retaining HFP, nRMSE increased to 0.275. When both the frequency loss and HFP were removed, leaving essentially only the global and local branches, nRMSE was 0.276. Conversely, when the frequency loss was retained but HFP was removed, nRMSE was 0.266, showing that the frequency loss plays a substantial role in reconstructing mid- and high-frequency components. Because this value is still above the complete model's 0.258, HFP provides an additional improvement as well. The local branch alone already outperformed the base FNO, but the best results across all metrics were obtained when all three components were used together. The components therefore act complementarily.
Conclusion
The global branch handles long-range coupling over the full domain and the dominant low-frequency dynamics. The local branch retains all modes within each patch to learn small spatial scales and localized structure. The HFP branch directly propagates a residual obtained after removing a low-frequency approximation, thereby bypassing spectral truncation. A radially binned frequency-aware loss is then added so that optimization explicitly reduces mid- and high-frequency errors.
The experimental results show that this design does more than improve one-step accuracy. It also reduces error accumulation during autoregressive rollouts and better preserves energy spectra and local fine-scale structures in turbulence and shock-dominated problems. In particular, when the number of global modes is small, the local branch compensates for the missing expressivity. This indicates that distributing a limited spectral parameter budget across global and local pathways can be more effective than concentrating it entirely in global modes.
LOGLO-FNO nevertheless leaves several issues unresolved. Non-overlapping patches may create boundary artifacts, and the radial loss cannot distinguish directional spectral errors. The three parallel pathways can increase training time and memory consumption relative to the base FNO, while errors remain near genuine discontinuities such as shocks. Even so, LOGLO-FNO provides a clear architectural strategy for strengthening locality and high-frequency reconstruction while retaining the global efficiency of FNO. It demonstrates the value of separating global and local operations and then recombining them when designing multiscale PDE surrogate models.
Q&A
Q. Is the addition of the local branch and HFP branch itself a unique contribution of this paper? Or is the study of branches designed to process local and high-frequency features already a standard approach in FNO research?
A. The general idea of processing local and high-frequency features through separate pathways cannot be regarded as an entirely original contribution unique to LOGLO-FNO. The original standard FNO has a relatively simple structure combining a global spectral convolution with a pointwise channel-mixing pathway. Subsequent studies, however, have proposed various auxiliary pathways to strengthen locality and multiscale representation in FNOs. For example, NO-LIDK, one of the main competing models considered in the LOGLO-FNO paper, adds localized integral and differential kernels in parallel with the global Fourier branch. Wavelet-based neural operators likewise belong to the broader line of work that separates and processes distinct spatial scales and frequency bands.
At the same time, this should not be viewed as a standard, universally adopted FNO configuration. There is no single agreed-upon design. Local kernels, wavelet bases, U-Net-style multiresolution pathways, adaptive mode selection, and physics-constrained losses are among several competing strategies. LOGLO-FNO is best understood as one particular design that combines local–global parallelization with a spectral loss.
Q. In the LOGLO-FNO architecture, the HFP branch does not apply an FNO operation, presumably because of FNO's characteristic low-frequency bias. However, relying on a simple MLP to process high-frequency information does not seem particularly satisfactory. The ablation results, in which the HFP branch has almost no effect without the frequency-aware loss, appear to support this concern.
A. This criticism is well founded. The HFP branch is closer to a preservation or bypass pathway that prevents an already extracted high-frequency residual from being removed by the global spectral branch. The channel MLP used in HFP is a pointwise operation that mixes only channels at each spatial location. It therefore cannot directly learn phase relationships between different spatial points or transport and deform high-frequency structures across space.
The high-frequency extraction mechanism is also relatively crude. Subtracting an average-pooled and subsequently interpolated signal from the original signal produces a simple high-pass residual, but it is not an ideal frequency decomposition. Depending on the pooling kernel and interpolation scheme, mid-frequency components may leak into the residual, some high-frequency components may be lost, and insufficient anti-aliasing before downsampling may introduce aliasing.
As noted in the question, the ablation results support this interpretation. Without the frequency loss, adding HFP reduces nRMSE only from 0.276 to 0.275, and high-frequency fRMSE only from 0.0264 to 0.0263. These differences are very small. By contrast, when the frequency loss is used, nRMSE decreases from 0.266 in the model without HFP to 0.258 in the complete model. It is therefore more accurate to interpret HFP not as an independently powerful high-frequency learning mechanism, but as an additional representational pathway that becomes useful when the frequency loss supplies a sufficiently strong gradient on high-frequency errors. Put simply, HFP provides the pathway, while the frequency-aware loss provides the training signal that encourages the model to use that pathway.
Q. In the LOGLO-FNO architecture, the global and local branches apply the same type of FNO operation. Do they share the parameters of the kernel and the local operator W? If they do not, the architecture may be highly inefficient because it effectively trains two FNOs.
A. According to the expanded equations in the paper and the official project page, the global and local branches do not share parameters. Thus, although the two branches use operation blocks of the same form, they specialize to their respective scales using different parameters. By contrast, the local spectral weights are shared across all patches. The model therefore does not train a separate FNO for each patch; it repeatedly applies a single local operator to every patch. Even so, the structural inefficiency identified in the question does exist to some extent.
The paper nevertheless claims a reduction of up to 50% in parameters relative to FNO. This should be interpreted as the ability to reduce the number of global spectral modes and reallocate the parameter budget to the local branch while achieving a given level of accuracy. According to experiments in the appendix, under the same Kolmogorov-flow setting, the total training time of LOGLO-FNO was approximately 3.30 times that of the base FNO, while training GPU memory was approximately 2.24 times larger. One-step inference was also approximately 3.33 times slower, and inference memory increased from 10.74 GB to 18.31 GB. Therefore, under the same high-performance configuration, its wall-clock time and memory usage cannot be regarded as comparable to or lower than those of the base FNO.
Q. Are there still essentially no models that handle data containing extreme high-frequency content particularly well, such as two-dimensional Kolmogorov flow at Reynolds number 5000 or the compressible Euler Riemann problem?
A. Yes. At present, there still appears to be no model that handles such problems in a fully satisfactory and generally reliable manner. Some methods, including the one studied here, can produce reasonably strong results for short one-step predictions on particular datasets and within limited parameter ranges. However, a 2025 study of hyperbolic systems observed that FNO, WNO, and multiwavelet operators increasingly oversmoothed solutions and lost important physical structures as boundary conditions became more complex. At the same time, recent work, including 2026 preprints on the Shearlet Neural Operator and the Local–Global Neural Operator, indicates clear ongoing progress.
Operator Learning and Neural Operators
A conventional neural network maps one finite-dimensional vector to another. In partial differential equation problems, however, both the input and the output are often functions. Examples include taking a spatially varying coefficient field as input and producing the corresponding PDE solution, predicting the solution field at later times from an initial condition, or computing a steady-state or time-dependent solution from an external forcing function. In such settings, the learning target is not an individual solution function, but the solution operator that maps an entire input function to an output function.
This perspective is important because standard grid-dependent neural networks cannot naturally represent continuous physical operators. Refining the grid changes the dimensions of the input and output vectors, while moving observation points or adding new points changes the vector representation of the input itself. If the network size is readjusted to match the resolution, both the number of parameters and the receptive field may change as well. Consequently, even for the same physical problem, one may need to construct a separate model for each grid resolution.
A central property sought by Neural Operators is discretization invariance. A model should be able to accept a function observed at arbitrary input points while using a fixed number of trainable parameters, evaluate the solution at desired output points, and converge to a well-defined continuous operator as the grid is progressively refined.
Architecture of Neural Operators
A representative Neural Operator consists of a lifting stage that maps the input function into a latent representation with a larger channel dimension, a sequence of kernel layers, and a projection stage that maps the latent representation back to the output space of the physical variables. Lifting, projection, channel mixing, bias terms, and nonlinear activations are pointwise operations applied independently at each spatial location. They increase expressive power across channels but do not directly transmit information between distinct spatial points.
Interactions across the full spatial domain are handled by a nonlocal kernel operation. The new feature at a point is formed by integrating features at other points over the entire domain, weighted by a kernel. In computation, for example, the continuous integral can be approximated by a weighted sum over observation points. The kernel may be fixed and depend only on position, conditioned on the input coefficient field, or conditioned on the current latent state so as to represent nonlinear interactions. A Neural Operator layer can therefore be interpreted as combining a nonlocal pathway with a pointwise channel-mixing pathway, followed by a nonlinearity. This structure enables local transformations and long-range information transfer to be carried out simultaneously.
Fourier Neural Operator
FNO implements the nonlocal kernel operation efficiently in the Fourier domain. If the kernel is assumed to depend on the relative displacement between two locations rather than on their absolute coordinates, the kernel integral can be interpreted as a convolution. Since convolution becomes frequency-wise multiplication in the Fourier domain, a global kernel operation can be computed efficiently by transforming the input features with an FFT, applying learnable channel mixing at each frequency, and then returning to physical space with an inverse FFT.
The overall FNO architecture lifts the input function, applies a sequence of Fourier layers, and finally performs a projection. Each Fourier layer contains two pathways. The first is a global spectral pathway that transforms the input into the Fourier domain, applies a complex-valued channel-mixing matrix to each retained frequency mode, and then performs an inverse transform. The second is a local pathway that applies a 1×1 convolution or linear channel mixing independently at each spatial point. The outputs of the two pathways are summed and passed through a nonlinear activation to produce the features for the next layer. It is important to note that the term local pathway here refers to a pointwise channel transformation that acts only at each individual location.
FNO does not learn all Fourier coefficients. Instead, it retains only a prescribed number of modes along each spatial axis. This finite spectral parameterization controls the number of parameters and computational cost, and it efficiently represents the dominant low-frequency structure of a physical system. However, as the number of retained modes decreases, high-frequency information is structurally removed. Increasing the number of modes substantially causes parameter and memory costs to grow rapidly. In particular, the size of the selectable frequency grid grows sharply as the spatial dimension increases.
Limitations of FNO and Motivation for LOGLO-FNO
Standard neural networks generally exhibit spectral bias: they learn low-frequency components first and high-frequency components later. In FNO, this optimization bias is compounded by a structural constraint because the spectral pathway explicitly truncates high-frequency modes. This choice may not be problematic when the inputs and outputs are mostly smooth and dominated by low frequencies. In problems where small spatial scales and abrupt variations are important, however, predictions can become excessively smooth and lose fine-scale structure. Examples include high-Reynolds-number turbulence, compressible flows containing shocks and contact discontinuities, complex wave propagation, and reaction–diffusion patterns.
Moreover, because Fourier basis functions are defined over the entire domain, they are well suited to representing global correlations and large-scale flow structures. Efficiently representing a small vortex or a sharp boundary localized to a particular region, however, may require many frequency components. It can therefore be inefficient to assign both global low-frequency structure and local high-frequency structure to a single spectral branch. LOGLO-FNO begins from this observation and processes global structure, local structure, and high-frequency residuals through separate pathways before combining them.
Architecture of LOGLO-FNO
LOGLO-FNO uses three parallel states. The first is a global branch that receives the entire spatial domain as input. As in standard FNO, this pathway retains only a limited number of Fourier modes and learns long-range interactions and dominant large-scale structures across the full domain.
The second is a local branch that partitions the input domain into small, non-overlapping patches and applies Fourier operations within each patch. Because each patch is much smaller than the full domain, retaining all Fourier modes available within each patch is less costly than retaining every mode over the global domain. This pathway is responsible for locally occurring mid- and high-frequency features, including small vortices, steep gradients, and fine structures near boundaries and discontinuities.
The third is the high-frequency propagation, or HFP, branch. The input is first average-pooled to a lower resolution and then interpolated back to its original resolution, producing a smooth approximation dominated by low frequencies. Subtracting this approximation from the original input extracts a high-frequency residual. The HFP branch propagates this residual directly through a channel MLP without subjecting it to Fourier-mode truncation. It therefore encourages information that would otherwise be removed by the global branch to be preserved through deeper layers of the network.
The three pathways are transformed into latent features under the same lifting framework and interact repeatedly through multiple LOGLO layers. The final output is obtained by projecting the global state. Nevertheless, because the local-branch and HFP-branch features are merged into the global branch in every LOGLO layer, the final global representation incorporates information from all three pathways.
LOGLO Layer
Within each LOGLO layer, the global and local branches each apply a spectral convolution and pointwise 1×1 channel mixing. The two results are activated and then passed through a channel MLP together with a skip or gating pathway. The output of the local branch is reassembled from the patch arrangement into the original spatial layout and added to the global-branch output. The HFP branch is passed through a channel MLP without a Fourier transform and is added at the same fusion point. Finally, a nonlinear activation is applied to the sum of all three pathways to form the global state for the next layer.
A key aspect of this design is that the local branch contributes directly to the global representation at every layer. The global pathway supplies large-scale dynamics and coupling across patches, the local pathway supplements fine-scale structure within each patch, and the HFP pathway bypasses frequency truncation to carry residual information that might otherwise be lost. The three pathways impose different inductive biases and are functionally complementary.
Patch-Based Local Spectral Operator
In the local branch, the full domain is partitioned into a collection of non-overlapping patches. The local kernel operation at a point integrates only over the patch containing that point, so its receptive field is explicitly localized. FFT and inverse FFT operations are applied independently to each patch, but the learnable spectral weights are shared across all patches. Therefore, increasing the number of patches does not create a separate set of parameters for each patch. Instead, the same local operator is repeatedly applied throughout the spatial domain.
Because the patches are small, the local branch can retain all frequency modes available within each patch. A high frequency in the global domain corresponds to a small spatial scale, but the same structure can be represented with relatively few modes in patch coordinates. This makes it possible to represent local fine-scale structure more efficiently than by indiscriminately increasing the number of modes in the global branch. The compensatory effect of the local branch is particularly strong when the global branch retains only a small number of modes.
The use of non-overlapping patches can nevertheless introduce artificial discontinuities or boundary artifacts at patch interfaces. The global branch provides coupling across the full domain and may mitigate this issue, but it cannot be guaranteed to eliminate it completely.
Radially Binned Frequency-Aware Loss
LOGLO-FNO modifies not only the architecture but also the training objective. First, the difference between the prediction and the ground truth is computed in physical space, and this error field is transformed into the Fourier domain. In the two-dimensional Fourier plane, frequency coefficients at similar distances from the origin are grouped into the same radial shell. The error energy within each shell is summed and then averaged over the batch, physical variables, and time dimensions. This produces an error measure indexed by wavenumber magnitude, independent of direction.
Based on radial distance, the frequency domain is divided into low-, mid-, and high-frequency bands, with representative cutoffs at 4 and 12. Low-frequency error is left to be controlled sufficiently by the standard MSE, while the additional frequency loss mainly assigns weight to mid- and high-frequency errors. This prevents optimization from focusing only on high-energy low-frequency components and encourages more accurate reconstruction of non-dominant frequencies that carry less energy but remain physically important.
An advantage of this loss is that it summarizes a two-dimensional spectral error as a one-dimensional error profile indexed by wavenumber magnitude. Radial averaging, however, does not distinguish angles and therefore discards information about anisotropic errors concentrated in particular directions. When directional structure in a flow or wave field is essential, a radial loss alone cannot capture every spectral discrepancy.
Experimental Setup
LOGLO-FNO was evaluated on six PDE problems representing different physical phenomena. The two-dimensional problems were Kolmogorov flow at Reynolds number 5000, compressible Navier–Stokes turbulence, Wave-Gauss with a spatially varying propagation speed, the compressible Euler four-quadrant Riemann problem containing shocks and contact discontinuities, and a nonlinearly coupled reaction–diffusion problem. The three-dimensional problem was a turbulent radiative mixing layer in an astrophysical flow.
The comparison models included the base FNO, Factorized FNO, U-FNO, NO-LIDK with localized integral and differential kernels, U-Net, the latent spectral model LSM, and the transformer-based PDE solver Transolver. Evaluation metrics included RMSE and nRMSE for overall spatial-domain error, as well as boundary error, conservation error, maximum error, and variance-normalized vRMSE. Frequency-domain metrics included low-, mid-, and high-frequency fRMSE, together with MELR and WLR, which quantify deviations of the predicted energy spectrum from the ground truth.
The evaluation was divided into a one-step setting, in which the model directly predicts the state one time step ahead, and an autoregressive rollout setting, in which each model prediction is repeatedly used as the next input. One-step evaluation measures immediate approximation accuracy, whereas autoregressive evaluation assesses stability as small errors accumulate over time.
Results on Kolmogorov Flow
Two-dimensional Kolmogorov flow at Reynolds number 5000 is a fully turbulent problem containing small vortices and a broad frequency range, making it an appropriate test case for the purpose of LOGLO-FNO. In the one-step evaluation, the base FNO achieved an nRMSE of 0.147, whereas LOGLO-FNO achieved 0.107, corresponding to an improvement of approximately 27.21%. Low-, mid-, and high-frequency fRMSE decreased by approximately 11.22%, 11.88%, and 24.64%, respectively. Although the combined NO-LIDK model attained the same nRMSE level, LOGLO-FNO produced lower errors in all three frequency bands. In particular, the largest relative improvement occurred in the high-frequency band, consistent with the intended roles of the local spectral branch and the HFP branch.
In the five-step autoregressive evaluation, the nRMSE of LOGLO-FNO was approximately 18.39% lower than that of the base FNO. Low-, mid-, and high-frequency fRMSE decreased by approximately 16.86%, 21.62%, and 17.26%, respectively. Although the errors of all models increased under repeated prediction, LOGLO-FNO exhibited a lower rate of error growth, indicating that preserving high-frequency and local structure also contributes to long-horizon rollout stability.
In a mode sweep over the number of global Fourier modes, LOGLO-FNO achieved lower bandwise fRMSE than the base FNO for nearly every mode setting. The difference was especially large when very few global modes were retained, because the local branch compensated for the limited expressivity of the global spectral pathway. LOGLO-FNO remained superior as the number of modes increased, with the most pronounced gap appearing in the high-frequency band. These results suggest that separating global and local representations can be more efficient than simply increasing the number of modes in the global branch.
In the analysis of radially binned spectral errors, LOGLO-FNO also produced the lowest error curve over most of the displayed frequency range. The largest differences between models appeared in the transition from low to mid frequencies. This is the range in which interactions between large-scale flow and small-scale structure become prominent, suggesting that the local branch and the frequency-aware loss help reconstruct not only the extreme high-frequency tail but also intermediate scales.
Results on the Compressible Euler Riemann Problem
The compressible Euler four-quadrant Riemann problem is difficult for smooth, low-frequency-oriented models because shocks and contact discontinuities form in the density, pressure, and velocity fields. In one-step evaluation, LOGLO-FNO reduced nRMSE by approximately 27.90% relative to the base FNO. Low-, mid-, and high-frequency fRMSE decreased by approximately 38.56%, 50.71%, and 31.58%, respectively. In the five-step autoregressive evaluation, nRMSE decreased by approximately 31.75%, while the mid-frequency error was reduced by 50%. These results show that LOGLO-FNO is effective not only for turbulence but also for hyperbolic problems involving shocks and discontinuities.
In qualitative comparisons of the density and pressure fields, both the base FNO and LOGLO-FNO reproduced the overall flow structure. However, the base FNO's errors were broadly distributed along high-gradient regions such as shock waves and contact surfaces. In the LOGLO-FNO error maps, both the intensity and spatial extent of the errors in these regions were reduced, indicating that the local branch better preserved sharp structures. Nevertheless, errors near discontinuities did not disappear completely, confirming that extremely abrupt changes remain challenging for Neural Operators.
Ablation Study on the 3D Turbulent Radiative Mixing Layer
The ablation study on the three-dimensional turbulent radiative mixing layer separately evaluated the contributions of the local branch, the HFP branch, and the frequency-aware loss. The base FNO achieved an nRMSE of 0.297, while the complete LOGLO-FNO, using both the local and HFP branches together with the frequency loss, achieved 0.258. Low-, mid-, and high-frequency fRMSE and vRMSE were also lowest for the complete configuration.
When the frequency loss was removed while retaining HFP, nRMSE increased to 0.275. When both the frequency loss and HFP were removed, leaving essentially only the global and local branches, nRMSE was 0.276. Conversely, when the frequency loss was retained but HFP was removed, nRMSE was 0.266, showing that the frequency loss plays a substantial role in reconstructing mid- and high-frequency components. Because this value is still above the complete model's 0.258, HFP provides an additional improvement as well. The local branch alone already outperformed the base FNO, but the best results across all metrics were obtained when all three components were used together. The components therefore act complementarily.
Conclusion
The global branch handles long-range coupling over the full domain and the dominant low-frequency dynamics. The local branch retains all modes within each patch to learn small spatial scales and localized structure. The HFP branch directly propagates a residual obtained after removing a low-frequency approximation, thereby bypassing spectral truncation. A radially binned frequency-aware loss is then added so that optimization explicitly reduces mid- and high-frequency errors.
The experimental results show that this design does more than improve one-step accuracy. It also reduces error accumulation during autoregressive rollouts and better preserves energy spectra and local fine-scale structures in turbulence and shock-dominated problems. In particular, when the number of global modes is small, the local branch compensates for the missing expressivity. This indicates that distributing a limited spectral parameter budget across global and local pathways can be more effective than concentrating it entirely in global modes.
LOGLO-FNO nevertheless leaves several issues unresolved. Non-overlapping patches may create boundary artifacts, and the radial loss cannot distinguish directional spectral errors. The three parallel pathways can increase training time and memory consumption relative to the base FNO, while errors remain near genuine discontinuities such as shocks. Even so, LOGLO-FNO provides a clear architectural strategy for strengthening locality and high-frequency reconstruction while retaining the global efficiency of FNO. It demonstrates the value of separating global and local operations and then recombining them when designing multiscale PDE surrogate models.
Q&A
Q. Is the addition of the local branch and HFP branch itself a unique contribution of this paper? Or is the study of branches designed to process local and high-frequency features already a standard approach in FNO research?
A. The general idea of processing local and high-frequency features through separate pathways cannot be regarded as an entirely original contribution unique to LOGLO-FNO. The original standard FNO has a relatively simple structure combining a global spectral convolution with a pointwise channel-mixing pathway. Subsequent studies, however, have proposed various auxiliary pathways to strengthen locality and multiscale representation in FNOs. For example, NO-LIDK, one of the main competing models considered in the LOGLO-FNO paper, adds localized integral and differential kernels in parallel with the global Fourier branch. Wavelet-based neural operators likewise belong to the broader line of work that separates and processes distinct spatial scales and frequency bands.
At the same time, this should not be viewed as a standard, universally adopted FNO configuration. There is no single agreed-upon design. Local kernels, wavelet bases, U-Net-style multiresolution pathways, adaptive mode selection, and physics-constrained losses are among several competing strategies. LOGLO-FNO is best understood as one particular design that combines local–global parallelization with a spectral loss.
Q. In the LOGLO-FNO architecture, the HFP branch does not apply an FNO operation, presumably because of FNO's characteristic low-frequency bias. However, relying on a simple MLP to process high-frequency information does not seem particularly satisfactory. The ablation results, in which the HFP branch has almost no effect without the frequency-aware loss, appear to support this concern.
A. This criticism is well founded. The HFP branch is closer to a preservation or bypass pathway that prevents an already extracted high-frequency residual from being removed by the global spectral branch. The channel MLP used in HFP is a pointwise operation that mixes only channels at each spatial location. It therefore cannot directly learn phase relationships between different spatial points or transport and deform high-frequency structures across space.
The high-frequency extraction mechanism is also relatively crude. Subtracting an average-pooled and subsequently interpolated signal from the original signal produces a simple high-pass residual, but it is not an ideal frequency decomposition. Depending on the pooling kernel and interpolation scheme, mid-frequency components may leak into the residual, some high-frequency components may be lost, and insufficient anti-aliasing before downsampling may introduce aliasing.
As noted in the question, the ablation results support this interpretation. Without the frequency loss, adding HFP reduces nRMSE only from 0.276 to 0.275, and high-frequency fRMSE only from 0.0264 to 0.0263. These differences are very small. By contrast, when the frequency loss is used, nRMSE decreases from 0.266 in the model without HFP to 0.258 in the complete model. It is therefore more accurate to interpret HFP not as an independently powerful high-frequency learning mechanism, but as an additional representational pathway that becomes useful when the frequency loss supplies a sufficiently strong gradient on high-frequency errors. Put simply, HFP provides the pathway, while the frequency-aware loss provides the training signal that encourages the model to use that pathway.
Q. In the LOGLO-FNO architecture, the global and local branches apply the same type of FNO operation. Do they share the parameters of the kernel and the local operator W? If they do not, the architecture may be highly inefficient because it effectively trains two FNOs.
A. According to the expanded equations in the paper and the official project page, the global and local branches do not share parameters. Thus, although the two branches use operation blocks of the same form, they specialize to their respective scales using different parameters. By contrast, the local spectral weights are shared across all patches. The model therefore does not train a separate FNO for each patch; it repeatedly applies a single local operator to every patch. Even so, the structural inefficiency identified in the question does exist to some extent.
The paper nevertheless claims a reduction of up to 50% in parameters relative to FNO. This should be interpreted as the ability to reduce the number of global spectral modes and reallocate the parameter budget to the local branch while achieving a given level of accuracy. According to experiments in the appendix, under the same Kolmogorov-flow setting, the total training time of LOGLO-FNO was approximately 3.30 times that of the base FNO, while training GPU memory was approximately 2.24 times larger. One-step inference was also approximately 3.33 times slower, and inference memory increased from 10.74 GB to 18.31 GB. Therefore, under the same high-performance configuration, its wall-clock time and memory usage cannot be regarded as comparable to or lower than those of the base FNO.
Q. Are there still essentially no models that handle data containing extreme high-frequency content particularly well, such as two-dimensional Kolmogorov flow at Reynolds number 5000 or the compressible Euler Riemann problem?
A. Yes. At present, there still appears to be no model that handles such problems in a fully satisfactory and generally reliable manner. Some methods, including the one studied here, can produce reasonably strong results for short one-step predictions on particular datasets and within limited parameter ranges. However, a 2025 study of hyperbolic systems observed that FNO, WNO, and multiwavelet operators increasingly oversmoothed solutions and lost important physical structures as boundary conditions became more complex. At the same time, recent work, including 2026 preprints on the Shearlet Neural Operator and the Local–Global Neural Operator, indicates clear ongoing progress.