Enhancing Data Quality in Crowdsourcing for Tree Outline Acquisition in Aerial Imagery via CNN-Based Real-Time Feedback

Abstract. We propose a method to improve data quality in paid crowdsourcing by leveraging CNN-based real-time feedback. Data acquired through paid crowdsourcing often suffers from inconsistencies or inaccuracies as workers prioritize task completion speed over precision to maximize earnings. To address this issue, we developed a lightweight, two-branch CNN that evaluates and provides quality feedback on polygon acquisitions of tree outlines in aerial imagery. As workers modify their polygons, the CNN predicts a quality score, displayed as a traffic light signal (red, yellow, green), indicating whether adjustments are needed. Our study compares a test group receiving this feedback with a control group without feedback. Results show that the test group achieves a notably higher average Intersection over Union (IoU) score as well as a lower standard deviation, indicating improved quality and consistency. By integrating the results of multiple workers, the test group achieves even better data quality with fewer samples than the control group. This approach reduces the need for redundant data acquisition, demonstrating its potential for time and cost savings in large-scale data collection campaigns.