Evaluating Video Generation Models' Understanding of Physical Laws

This study investigates whether scaling video generation models enhances their ability to comprehend fundamental physical laws. Researchers developed a 2D physics simulation to generate training data, enabling quantitative evaluation of model accuracy. They assessed model performance across in-distribution, out-of-distribution, and combinatorial generalization scenarios. Findings indicate that while scaling improves in-distribution and combinatorial generalization performance, it does not significantly enhance out-of-distribution scenarios. The models tend to prioritize visual features like color over physics-based properties, suggesting that scaling alone is insufficient for these models to uncover fundamental physical laws.