FAN: Fourier Analysis Networks for Modeling Periodic Functions

Traditional neural networks, such as Multi-Layer Perceptrons (MLPs) and Transformers, often struggle to accurately model periodic functions, tending to memorize data rather than understanding underlying periodic principles. The study introduces Fourier Analysis Networks (FAN), a novel architecture that integrates Fourier Series into neural network structures, enabling efficient modeling of periodic phenomena. FAN can seamlessly replace MLPs in various models, offering reduced parameters and computational costs. Extensive experiments demonstrate FAN's effectiveness in tasks like symbolic formula representation, time series forecasting, and language modeling, highlighting its potential as a versatile tool for applications requiring periodicity modeling.