Reproducing Geospatial Crowdsourcing: How Consistent Is the Crowd?

Abstract. This paper investigates the long-term consistency and reliability of paid geospatial crowdsourcing on the online platform Microworkers.com. Over a five-month period, we conducted three crowdsourcing campaigns, each representing a task typical for remote sensing, i.e., pixel classification, point selection, and geometric outline acquisition, to assess whether repeated worker participation enhances data quality and reproducibility. Beyond individual task performance, we examine the broader question of whether crowdsourcing campaigns can yield reproducible results over extended periods. Despite the large and heterogeneous workforce of Microworkers.com, a substantial share of tasks was completed by recurring workers who consistently outperformed one-time participants. Furthermore, across all campaigns, data quality remained largely stable, with only minor variability between epochs. Additionally performed statistical analyses confirm that reproducible outcomes are achievable, highlighting the potential of reliable and reproducible crowdsourcing results for geospatial data acquisition.