S²PT: Spatio-Sequential Point Transformer for Efficient 3D Scene Understanding

Abstract. Efficient processing of large-scale 3D point clouds acquired from Terrestrial or Airborne Laser Scanning (TLS/ALS) presents a significant computational challenge. While transformer-based architectures excel at modeling the global context crucial for interpreting these complex scenes, their quadratic computational complexity makes them infeasible for direct application on massive point sets. To address this scalability bottleneck, we propose the Spatio-Sequential Point Transformer (S²PT), a novel hierarchical architecture for efficient and effective large-scale point cloud processing. Our approach begins by serializing the point cloud into an ordered sequence, which enables the use of attention with linear complexity. This not only circumvents the quadratic bottleneck of standard transformers but also establishes a global receptive field at every layer. To compensate for potential information loss during serialization, we further introduce a novel Spatio-Sequential Positional Encoding (S²PE) that synergistically combines 3D local geometric features with 1D sequential order information, enhancing the model’s spatial awareness. Experiments on multiple benchmarks demonstrate that S²PT achieves performance comparable to state-of-the-art methods while being significantly more efficient during training and inference, offering a promising path towards scalable representation learning for large-scale 3D scenes.