Thermal infrared (TIR) imaging provides a unique capability to reveal surface heat-transfer patterns of buildings and supports applications such as energy leakage detection, insulation inspection, and building energy monitoring. However, large-scale TIR image analysis by deep learning is still constrained by the lack of reliable annotations, as TIR images often exhibit blurred textures and weak boundaries, which makes manual labeling inconsistent and time-consuming. To address this challenge, we propose <strong>ThermalAssist</strong>, a geometry-aware and gradient-enhanced framework designed to assist thermal anomaly labeling in UAV-based TIR images. By combining sparse labeling, dense correspondence, and label propagation, the framework efficiently transfers labels across overlapping views while maintaining geometric and semantic consistency. Experiments on the TBBR (Thermal Bridges on Building Rooftops) dataset demonstrate that ThermalAssist achieves an F1-score of up to <strong>0.87</strong> and a mean IoU of <strong>0.69</strong>, effectively helping reduce missing annotations and inconsistent boundaries. Compared with the tracking-based SAMURAI method, our approach shows greater robustness under low-texture and low-overlap conditions. This work establishes a foundation for a broader thermal annotation system and validates an important step toward scalable, reliable, and more intelligent labeling of thermal anomaly imagery.