Accurate interpretation and segmentation of 3D point clouds in real-world urban environments is a critical challenge in geospatial analysis, particularly due to the complexity of real-world scenes, inevitable data uncertainties, and potential annotation errors. This paper proposes a confidence-aware deep learning framework to refine the segmentation accuracy of real-world point cloud data. By incorporating multi-source information, such as aerial imagery, and embedding geospatial prior knowledge, this framework models data uncertainty through point-wise confidence scores. Besides, we design an iterative online learning strategy, allowing the network to improve both its predictions and the quality of training labels. Extensive experiments on large-scale airborne laser-scanned data demonstrate that our framework effectively enhances training data by reducing label noise and improving annotation quality, which leads to more robust, generalizable model performance. Our source code is publicly available at <code>https://github.com/AutumnMoon00/RefineNet</code>.