Abstract

Implicit neural representation has demonstrated promising results in 3D reconstruction in various scenes. However, existing approaches either struggle to model fast-moving objects or are incapable of handling large-scale camera ego-motion in urban environments. This leads to low-quality synthesized views of the large-scale urban scenes. In this paper, we aim to jointly solve the problems caused by large-scale scenes and fast-moving vehicles, which are more practical and challenging. To this end, we propose a progressive scene graph network architecture to learn the local scene representations of dynamic objects and global urban scenes. The progressive learning architecture dynamically allocates a new local scene graph trained on frames within a temporal window, with the window size automatically determined, allowing us to scale up the representation to large-scale scenes. Besides, according to our observations, fast-moving objects are observed only in a few frames, which leads to a significant decline in reconstruction accuracy for dynamic objects. Therefore, We introduce a foundation-guided object representation that extracts object-centric visual priors and conditions the density and color decoders in normalized object coordinates. We further propose a frequency-progressive regularization strategy that gradually exposes high-frequency positional, directional, and pose encodings during training, reducing overfitting to sparse object observations. Experimental results demonstrate that our method achieves state-of-the-art view synthesis accuracy, object manipulation, and scene roaming ability in various scenes. The code will be open-sourced on https://github.com/dtc111111/prosgnerf.

Fig. 1 Urban scene reconstruction and editing with ProSGNeRF. We show our view synthesis in different time steps (65,262) and scene decomposition results. Our approach significantly improves the view synthesis performance in real-world urban scenes containing multiple dynamic objects and large-scale camera ego-motion. We highlight and enlarge objects in the first column of images, providing the corresponding object PSNR in the top left corner. Scene PSNR is provided in the second column.

Fig. 2 The isometric view of the proposed method, ProSGNeRF. We employed a 2D segmentation network, SAM, to preprocess the training data and generate accurate masks for dynamic objects. We propose a progressive neural scene graph architecture that dynamically allocates local neural scene graph (box). The entire scene is decomposed into three parts: background, dynamic objects, and far-field. We design separate networks for background and objects and introduce a far-field loss for regularization. Nodes li represent individual dynamic objects. Fbkg models the static background scene and Fobj models movable foreground objects in local object-centric coordinate frames.

https://doi.org/10.1007/s11263-026-02962-5