CHALLENGES AND RECOMMENDATIONS FOR 3D PLANT PHENOTYPING IN AGRICULTURE USING TERRESTRIAL LASERS SCANNERS

: Active sensing with LiDAR, and terrestrial laser scanners (TLS) in particular, are increasingly being used in plant phenotyping for assessing structural or 3D geometrical plant traits. Although these technologies provide the unprecedented possibility for remote, non-destructive, automatable, and efficient estimation of plant geometry, their deployment does not come without challenges. In this publication, we present a systematic overview of all challenges impacting TLS-based 3D plant phenotyping. We provide actionable recommendations for the end users of the technology, as well as the research questions and possible directions that can contribute the most to resolving these challenges. We specifically focus on TLSs, as we detected a lack in the existing literature dedicated to this sensing system providing a unique compromise between data quality and resolution vs. measurement efficiency and covered volume. The presented discussions are based on the literature review and our own experience in estimating the structural traits of sugar beet and wheat in plant phenotyping experiments.


INTRODUCTION
Plant breeding is a time-intensive process that can take up to 20 years to establish a new crop variety.With the onset of global warming, there is an urgent need for more adaptable and resilient crop species that can cope with environmental changes.To address this issue, agriculture is undergoing a digital revolution, facilitated by the deployment of sensor technologies and advanced data processing pipelines (Araus and Kefauver, 2018).Assessing structural or 3D geometrical traits such as the shape and size of plants and plant organs is important for breeders to evaluate physical adaptations to changes in their genetics, environment, and management practices.To enable that, scientists and industry are developing strategies to quantify observable cues related to the plant structure.Remote sensing gained a particular momentum as it is non-destructive, remote, and with the possibility of automation.
Strategies for observing plant geometry can be primarily divided into passive and active sensing, where passive sensing relies primarily on RGB cameras and structure from motion workflows.It is better established than active sensing, primarily due to the low cost, high availability, and high developmental stage of RGB cameras and image processing allowing for e.g.high acquisition speed and resolution.However, active remote sensing has some compelling advantages.For example, active remote sensing with LiDAR (Light Detection and Ranging) and different scanning platforms; e.g.static and mobile terrestrial, UAV and airborne; demonstrated compelling potential for 3D phenotyping, due to different and often complementary characteristics to cameras.LiDAR is: 1) illumination invariant and well suited for diurnal observations, 2) plant structure can be directly estimated from observed point clouds with high accuracy, 3) it provides accurate information on the vertical distribution of data samples, 4) gives more information about lower canopy structure and 5) information about soil (Jin et al., 2021).
Terrestrial laser scanners (TLSs) are portable stationary LiDAR systems with the possibility of achieving the highest meas-urement quality and resolution of all LiDAR systems, while still retaining reasonable data acquisition efficiency.They are present in plant phenotyping since their appearance in the early 2000s and they had an eminent role, especially in the field of forestry (Jin et al., 2021).They are widely available, allowing for worldwide applicability necessary for untangling genotypeenvironment-management interaction.Despite mentioned advantages of active sensing with LiDAR, its deployment in plant phenotyping comes with certain challenges.Existing literature reviews discuss some of these challenges in a general case of LiDAR technology, e.g., (Jin et al., 2021), missing specificities of the TLSs.Hence, our work provides a systematic overview of the challenges of using TLSs for 3D plant phenotyping in agricultural applications, together with recommendations on how to mitigate some of them.The work is based on our own experiences of using TLSs and other scanning systems for the phenotyping of sugar beet and wheat.The main goal of the article is to elucidate bottlenecks that require further research efforts and to give clear recommendations to the end users.

STATE OF THE ART
Current 3D measuring devices enable the parameterization of plant properties related to their geometry, either as indirect measurements of shape and size or as descriptors of the overall structure, such as leaf area and canopy volume.These parameters can be linked to the main biological traits of interest like biomass, yield, and plant stress reaction (Paulus et al., 2014c).
Commonly, 3D measuring devices come with the trade-off of reachable resolution and data quality vs. measurable volume and throughput.As plants depict a permanently deforming object -the change in size has to be taken into account, as well as the needed resolution for plant organ description and needed throughput for assuring generalizable results for a plant population (Paulus, 2019).Various active sensing technologies can be used to measure the 3D geometry at different scales in the lab, the greenhouse, and the field, but each comes with its own advantages and limitations.
On one side, there are laborious and accurate measurements in controlled laboratory conditions with laser triangulation and structured light scanners capable of estimating, e.g.exact shape and size of strawberry flower calyx (Paulus, 2019).On the other side, there are highly efficient moving platforms for high-throughput in-field plant phenotyping such as UAVs and ground-based robots often equipped with automotive LiDAR, capable of estimating crop height and biomass at large scale (Jin et al., 2021).The measurement quality and volume of these two phenotyping strategies are drastically different.Laboratory-like measurements are hardly adaptable for higher throughput measurements, e.g. of numerous plant units in the greenhouses, and they cannot be used for in-field phenotyping in an uncontrolled environment.On the other hand, mobile mapping platforms and robots allowed unprecedented efficiency.However, the number of plant traits that can be extracted from these measurements is drastically lower than in the case of laboratory analysis, and their quality is inferior.TLSs with their following characteristics lie somewhere between these two poles: measurement accuracy, spatial and temporal measurement resolution, related sensitivity towards detecting subtle changes, and measurement efficiency.A TLS device combines the advantage of a resolution of up to 1 mm at distances of 10 m, measurement accuracy on a level of a few millimeters, and a measurable volume and reach from tenths of meters up to kilometers (Vosselman and Maas, 2010).Because of that, TLSs are successfully utilized for highly demanding laboratory phenotyping, greenhouse phenotyping with a high amount of plants, and without an automated plant mover or convey belt, as well as for in-field phenotyping (Jin et a l., 2021).By now, an abundance of relevant structural traits was extracted from TLS point clouds (Tab.1).
Generally, for in-field u se, T LS p oint c louds a re primarily utilized for estimating canopy height, growth rates, and other canopy-related properties (Friedli et al., 2016).For larger species, such as maize, the automatic extraction of traits describing the geometry of individual plants or their organs was demonstrated as well (Jin et al., 2021).More challenging plant types are primarily investigated in a higher level of detail in the lab or in the greenhouse experiments, with significantly reduced measurement volume (Paulus, 2019).Acquiring this information from TLS point clouds is not trivial, there are no standardized approaches, and further progress would enhance information extraction.Each TLS-based phenotyping study tackles the related challenges in a specific way, and the complexity varies depending on the use case.However, some of the challenges are common.Hence, in the following section, we present a structured overview of these challenges, focusing on the generalizable aspects, possible remedies, and further work necessary to resolve them.

CHALLENGES AND RECOMMENDATIONS
Many of the trait extraction workflows presented in the literature presume unobstructed access to each individual plant from multiple different viewpoints, from relatively high proximity (a few meters distance or less), with no strong time limitations for scanning (Lumme et al., 2008).This assures that the acquired point clouds have high resolution, high redundancy of data, almost complete coverage of each individual plant, and the most favorable scanning geometry allowing for minimal uncertainty related to the measurement process.However, high throughput phenotyping with an increased measurement volume in the greenhouse or field lacks that luxury.Wider areas with more plants need to be covered in a limited time.This requires compromises, e.g.reduced resolution and non-resolved occlusions.Making these compromises poses challenges, as most of the state-of the-art algorithms are developed for working with ideal or close-to-ideal point clouds.
Fig. 1 shows the 2D case of the ideal point cloud and all deviations that occur in TLS-based 3D plant phenotyping: variable noise due to measurement uncertainty and its functional relationship with measurement configuration and the intensity of the laser beam (Vosselman and Maas, 2010); missing data due to mutual occlusion of plants; non-uniform sampling due to scanning pattern; outliers due to mixed pixels and motion; re-occurrence, fracturing, and non-rigid transformations due to point cloud registration errors and plant movements.
Standard solutions for some of these distortions are adequate for the lab or greenhouse measurements with higher data redundancy.They are commonly implemented in dedicated point cloud processing solutions, like open source software Cloud-Compare, or point cloud libraries, like Open3D and PCL (Point Cloud Library) for Python.For noise reduction, there are different strategies relying on local surface fitting or simultaneously denoising and subsampling (Zhou et al., 2022), while for outlier removal, the standard algorithms are Statistical-Outlier-Removal (SOR) and Radius-Outlier-Removal (ROR).
The problem of non-uniform sampling can be reduced by regular spatial subsampling.However, these algorithms work only in a case of sufficient resolution, redundancy, and data quality.
For more challenging use cases and other distortions, such standard solutions are lacking.However, understanding the causes of the distortion can be used for implementing measurement and data processing strategies that can mitigate them.Fig. 2 summarizes the main causes of distortions, the effects they have on the point clouds, and some remedies that can be used to mitigate these effects.They are primarily grouped into causes related to the data acquisition principles (blue), properties of the laser beams (orange), and plants (green).

Noise
Outliers Non-uniform sampling

Missing data
Variable noise
Figure 1.Sketch of a targeted leaf geometry, ideal point cloud and all deviations from ideal in TLS phenotyping.
However, they are all interrelated and multiple causes have comparable and overlaying effects on point clouds.This in turn means that certain solutions can account for multiple causes, resulting in relatively complex ties between all the elements in the graph.Hence, the separation presented in Fig. 2 is neither strict nor definite, a nd t he m ulticolor fi elds in dicate multiple connections between the elements.The following subsections will discuss the individual causes and related coping strategies.

Scanning pattern and measurement configuration
One of the main causes of TLS point cloud deviations (Fig. 1) is the combination of scanning pattern and suboptimal measurement configuration.T LSs a re c ommonly m ounted o n tripods aside and somewhat above the areas that are densely populated with plants (Fig. 3, left).Such configuration h as undesirable consequences.One issue is varying distances and angles of incidence (AOI) across the measurement volume, reaching greater than desired values for both quantities (Fig. 3, middle).Another is that different plants are sampled with different levels of detail, as TLSs scan with a fixed a ngular resolution, causing irregular sampling in Euclidean space (Fig. 3, right).However, such configurations a re o ften n ecessary for in-greenhouse and in-field use due to the often limited mounting possibilities.The combination of this unfavorable measurement configuration and scanning pattern is the direct cause of the majority of the deviations presented in Fig. 1. High distances and AOIs cause higher noise, lower resolution, and an increase in the laser beam footprint size (Vosselman and Maas, 2010), which also increases the occurrence of mixed pixels (Sec.3.3).Their substantial variation causes variable noise and non-uniform sampling density.Finally, high AOIs also cause more missing data due to higher occlusion between the plants, which especially comes prominent in the later growth stages.
One simple recommendation to minimize these effects is to limit the maximal magnitude and variability of the distances and AOIs, either during measurements or in point cloud preprocessing.There is no exact recommendation, as the optimal solution depends on the variables such as measured volume, scanning resolution, and laser beam footprint size.However, maintaining as uniform as possible point cloud properties is important as many of the point cloud processing algorithms have tunable hyperparameters that cannot be optimally selected for such strong variations within a scene.
Secondly, the TLS is ideally placed upside down above the plants at viewpoints with regular spacing, using a dedicated mounting platform instead of a tripod.This solution requires additional investment in the implementation phase but significantly improves the resulting point cloud quality.This assures on average smaller distances and AOIs, their smaller variation within the scene, fewer occlusions, and it reduces the problem of the mixed pixels.One such example solution is the field phenotyping platform (FIP) (Kirchgessner et al., 2016).
Advanced solutions requiring further scientific efforts can be separated into efforts focused on data acquisition and data processing.Regarding data acquisition, it is possible to pose the viewpoint planning problem as a multi-target optimization problem aiming at minimizing undesired extreme values and variations of measured distances, AOIs, and point spacing.Works solving similar optimization problems are presented for 2D cases of scanning urban environments (Jia and Lichti, 2022).However, TLS-based plant phenotyping would require further efforts due to the higher complexity of the observed scene and the information that needs to be retrieved.Such optimizations could be based on end-to-end pipelines combining 3D TLS point cloud simulations, trait extraction algorithms, and heuristic optimization.The building blocks for such approaches are already available, e.g.Crops in Silico project (Marshall-Colon et al., 2017) for simulating 3D plants and Helios project for LiDAR simulations (Bechtold and Höfle, 2016).
Solutions based on data processing require adopting machine learning (ML) algorithms that can learn stochastic properties of the point clouds and locally adapt the surface reconstruction or plant traits extraction.Developing such algorithms is a part of the current research efforts.However, existing solutions are only demonstrated for simplified CV problems usually tackling a subset of the complications presented in Fig. 1.For example, the algorithms commonly tackle separately the problems of shape completion, denoising, upsampling, and 3D reconstruction and they are demonstrated on small-scale point clouds of individual objects with unrealistic properties (e.g.zero mean Gaussian noise).Their transfer to real-world datasets considering scale and complexity, as well as adaptation for the plant phenotyping domain (as they are typically over-fitted for a particular task to improve the results) is yet to be achieved.A few exemptions of such solutions already applied for plant phenotyping will be presented later in the text.
Finally, scanning from mobile platforms, primarily UAVs, allow more homogenous and favorable measurement configurations and scanning patterns.Hence, if this was found to be a limiting factor in a particular plant phenotyping task, a such alternative could be more desirable.However, current state-ofthe-art mobile mapping platforms provide at least an order of magnitude lower point cloud quality and resolution than static TLSs (Jin et al., 2021).

Incomplete coverage and occlusions
As TLSs are stationary portable platforms, the incomplete coverage of 3D measurement subjects such as plants is unavoidable from a single viewpoint.Additionally, in many plant phenotyping cases, plants are densely populated creating occlusions and The errors in registration can cause re-occurrence and displacement (rigid transformation) of parts of the plants, causing fracturing (Fig. 1).Hence, they need to be minimized to avoid systematic biases in extracted geometrical traits.A typical environment in plant phenotyping is unstructured and repetitive, it changes over time and, in the case of in-field scanning, it is without clear landmarks.Such an environment is unfavorable for use of the common point cloud registration approaches, such as the Iterative Closest Point (ICP) algorithm, as they are prone to get stuck in local minima.Hence, a common solution is to use a network of stable and dedicated scanning targets, which will serve as corresponding points and aid registration, e.g. in (Kirchgessner et al., 2016).
However, designing such a network assuring high accuracy is not a trivial task.Many decisions have an impact on the end result, from the selection of the right targets to the network design.For the highest accuracy, it is beneficial to use black-and-white planar targets.Also, selecting the right target design and algorithm for center estimation, and assuring a minimum of 100 points per target can further improve the results (Janßen et al., 2019).Moreover, allowing for a minimum of 4 targets visible in each scan is a good practice that assures sufficient redundancy, and it is advisable that visible targets at each viewpoint enclose a large surface area or 3D volume (Yang et al., 2020).
Advanced solutions for registration will require the development of dedicated algorithms for such repetitive scenes with geometrically complex and dynamic objects.This will likely require some form of scene understanding and prior knowledge, e.g., scene segmentation into stable regions related to soil and unstable regions related to plants and then minimizing different optimization goals for each region.An example of such a solution for UAV data can be found in (Günder et al., 2022), where datasets are registered using locally unique sowing pat-terns in the field due to specific aberrations like missing plants or increased or decreased plant or row spacing.
Alternatively, the accuracy of the registration and number of viewpoints could be reduced if intelligent end-to-end plant traits extraction algorithms could handle missing data, point re-occurrence, and fracturing.This could become possible if sufficient prior knowledge is encoded in the ML models by providing adequate training data.Such an approach was already demonstrated in the agricultural domain for RGB-D cameras.Using a pre-trained fully connected neural network based on encoder-decoder architecture, the authors were capable of inferring a complete 3D shape of bell peppers and strawberries using only partial coverage of the fruit (Magistri et al., 2022).If a comparable approach could be derived for TLS point clouds is yet to be seen.

Laser beam size
As the 2D cross-section of the laser beam is not infinitely small, a single laser beam interacts with the measured object over a certain surface area, often referred to as a laser beam footprint.Depending on the instrument properties, distance, and AOI, the footprint can have different shapes and sizes ranging from a minimum of several millimeters up to some centimeters in diameter for common plant phenotyping settings.This phenomenon induces variable noise, and limits the size of the smallest observable plant detail.However, the main challenge for geometrical reconstruction arises when the laser beam hits more than one surface at the same time, causing mixed pixels (or ghost points) that fall somewhere in between two sampled surfaces (foreground and background) and do not describe any real object.For example, Fig. 4 presents the extreme case of mixed pixels in the case of in-field wheat phenotyping, where the laser beams sample at the same time leaves (foreground) and the soil (background) with different proportions.
As the footprint size grows, mixed pixels form a much higher percentage of total points in the point cloud, which poses two challenges.First, the algorithms for outlier removal based on analysis of local point neighborhoods (e.g.SOR) are less likely to detect mixed pixels and remove them.Second, a successful removal induces large data losses, which can bias the extracted plant traits.For example, as mixed pixels occur on the edges of the measured surfaces, the estimated leaf size can become systematically too small after mixed pixel removal.
If the mixed pixels sample plant and soil, rather than two neighboring leaves, there is a higher chance of detecting them.Namely, multi-return (echo) time-of-flight scanners are capable of separating the signal of a single laser beam on two or more points belonging to different surfaces.However, they are capable of such separation only on the foreground-background distances higher than 0.75 m (Vosselman and Maas, 2010), hence, making this strategy valid only for taller plants and later growth stages.Nevertheless, such scanners also often provide a measure of how much the return signal deviates from the expected (Gaussian) distribution, which is notably higher for the mixed pixels.This can be well utilized for mixed-pixel filtering.TLSs with other measurement principles have alternative mixed-pixel removal strategies that are available in the manufacturer's software during data import or pre-processing.Their mechanism is not disclosed, and their success rate is variable.Moreover, using spectral data, such as laser beam intensity or RGB values (in the case of the colored point clouds), can help in identifying mixed pixels, especially in the plant-soil case.
Simple remedies to mitigate the mixed-pixel effect are: to limit the measurement distance and AOI; select an instrument with a small footprint size and good mixed-pixel filtering; and adjust measurement configuration so that it assures the highest foreground and background distance (often top-down view).More advanced solutions, allowing recovery of sharp object edges and not the removal of the mixed pixels, will require substantial further development in point cloud processing algorithms.They will likely need to rely on ML deconvolution, as the problem is too complex for established signal processing strategies.Examples of such deconvolution algorithms can already be found in CV domain.For example, in (Xiang et al., 2021), the authors presented a deconvolution algorithm based on transformer network architecture capable of reconstructing sharp edges of simple individual objects with high fidelity.Comparable approaches could potentially be used to tackle mixed-pixel problems in the agricultural domain.
Alternatively, an indispensable source of information for this task are RGB images of integrated onboard cameras, as the surface edges are well-preserved.Hence, a reconstruction of edges distorted by mixed pixels could be possible by inferring the correct shape from images.Early implementations of such solutions for RGB-D are already present in CV.In (Metzger et al., 2022), the authors used an approach that combines guided anisotropic diffusion with a deep convolutional network to obtain super-resolution depth (D) images with sharp edges from initial low-resolution D and guiding RGB images.The results were presented on arbitrary scenes, hence, it is to be investigated if the approach is directly applicable to plant phenotyping using TLSs with high-quality integrated RGB cameras.

Laser-matter interaction
The laser beam has a complex interaction with plant tissue, resulting in partial surface reflection, a nd p artial p enetration followed by absorption or transmission of the light in various proportions.For example, studies show that laser scanners are able to measure through the epidermis (Paulus et al., 2014b).This phenomenon has an influence on the quality of geometrical reconstruction, and the exact impact is specific to the properties of a used laser scanner and plant material.Depending on the laser wavelength, more or less of the energy passes the epidermis and is absorbed by the chlorophyll, which influences the intensity, and, hence, the measurement noise.
According to the characteristic plant spectra, the highest reflectance is available in the green a rea 550nm and in the NIR area > 780nm.Hence, the TLSs with the wavelengths in the mentioned spectrum are preferable for 3D plant phenotyping.
The systematic bias due to laser penetration is commonly on the below-millimeter level (Dupuis et al., 2015), while the increase in the noise level can be an order of magnitude larger, depending on the amount of the absorbed light.Hence, if the most unfavorable wavelengths are avoided, this effect is primarily concerning high accuracy demanding in-lab cases.
Related phenomena worth mentioning are water bodies, which act as semi-transparent specular mirroring surfaces, causing either a total reflection of the laser beam in the direction of the line of sight or away from the TLS's photodetector; or strong absorption of the laser beam (Vosselman and Maas, 2010).Because of that, scanning wet surfaces can induce increased measurement noise, outliers, or data gaps (Fig. 1).Hence, it is strongly advisable to avoid scanning plants when they are notably saturated with water droplets, e.g.due to rain or dew, which particularly concerns field phenotyping.

External excitation
One of the most detrimental factors to point cloud quality is plant movement by external influences.Person movements, moving devices, and particularly wind cause air motion and this results in a position change of the highly sensitive plant leaves.This can only be neutralized by the exclusion of fast external movement, wind, and air draft, which is feasible in some cases of in-lab and in-greenhouse plant phenotyping but is often unachievable.The effects on the point cloud quality are multiple, covering the whole range presented in Fig. 1.
The main problem is that the time required for a single scan is relatively long, often spawning through several minutes to capture the desired scene.Hence, one solution in severe cases is to consider alternative geometry capturing technologies, capable of covering the whole plant(s) with a single snapshot.The main candidate is using high-quality depth cameras, which allow for degraded, but still comparable (millimeter-level) data quality (Frangez et al., 2022).However, they only cover distances of several meters and are only suitable for indoor or overnight use, as they are sensitive to external illumination.
The only simple recommendations for the end-users are: selecting faster-scanning instruments, carefully selecting measurement time windows, using wind sensors to monitor the airflow, and repeating the measurements if a certain threshold is breached during a single scan.More advanced solutions will require domain-adapted ML approaches that will be able to correct point clouds based on some additional source of information.This could be: RGB images that are taken while scanning (see Sec. 3.3), scans from the preceding or following epochs (more in Sec.3.6), or prior knowledge encoded in the ML models (as the example in Sec.3.2).

Plant egomotion
Plants are dynamic objects, moving over time due to growth, or physiologically reacting to changes in the environment such as sun-angle alterations due to heliotropism or variations in gasses and humidity concentration in the air due to adaptable turgor.Often the goal is to understand these plant dynamics, or at least to minimize the impact of these movements on the captured geometry.Comparably to the external excitation, this requires high data acquisition speed and increased temporal resolution.If this is not met, the effects such as the re-occurrence and nonrigid transformations can affect the point cloud quality (Fig. 1).
An example of points re-occurrence due to plants' own movement or egomotion is presented in Fig. 5 for sugar beet leaf movement over the course of 1 h.We can observe two distinct cases, one where the point clouds of two epochs that are close in time (10 min) are still partially overlapping, and one where there is a clear separation between them.These two cases require different treatments.The first case (Fig. 5, c) can happen within "a single measurement epoch" while capturing a complete point cloud from multiple viewpoints.In such a case, it will often be desired and advisable to treat a point cloud as a single surface.To reconstruct a surface without bias due to point re-occurrence, it is possible, for example, to eliminate the redundant points using the Hidden Point Removal algorithm (Katz et al., 2007).In the second case (Fig. 5, d) it is necessary to treat the point clouds as separate measurement epochs and perform adequate point cloud registration.
There is an unavoidable trade-off between temporal resolution vs. point cloud quality, spatial resolution, and measured volume, as shorter measurement epochs require fewer viewpoints and shorter scanning times.So planning an adequate measurement setup considering plant egomotion requires prior knowledge of plant behavior and desired plant traits to be extracted.Only then setup design can be posed as a well-defined optimization problem and can be resolved, e.g. with the approaches discussed in Sec.3.1.Simple recommendations for the end users, apart from complete setup optimization, are again selecting instruments with high measurement rates, and adaptation of scanning resolution based on the traits of interest.For example, canopy and foliage-related traits (Tab. 1) typically do not require high spatial resolution.
More advanced solutions for increasing temporal resolution without information and quality loss would require algorithms capable of fusing sparse low-resolution but frequently taken point clouds with RGB images.Examples of such solutions exist in other domains.For example in (Zhu et al., 2023), the authors fused sparse and low-frequency LiDAR data with high resolution and frequency RGB images using new network architecture for simultaneous scene-flow estimation and sensor data integration.The approach was used in geomonitoring, for tracking mud and debris flow.Comparable solutions could be derived for plant phenotyping.Alternatively, there are algorithms capable of inferring plant structure in the intermediate steps between actual measurement epochs, however currently only in 2D (Yasrab et al., 2021) relying on GANs (Generative Adversarial Networks).Such solutions would allow asynchronous observations of different plants, increasing the temporal resolution without loss in comparability, which is relevant for TLS-based plant phenotyping due to limited acquisition speed.
Another issue related to plant egomotion is the registration of point clouds from different epochs in a way that they can cope with changes in the shape of each individual plant and infer the traits such as leaf area or inclination change as a function of time.Time series measurements can not be simply mapped onto each other to measure differences in plant traits.As the plants move, defining correct point correspondences, and hence, tracking the corresponding parts of the stems, leaves, and other plant organs is not trivial.Therefore, the correspondences have to be established and plants have to be semantically and geometrically modeled (Paulus et al., 2014a), e.g.their organs need to be identified, segmented, and parameterized so that the plant can be represented as a system of elementary parts that can be uniquely identified and traced over time.
There are several ways to approach this issue.The first one is a reconstruction of plants using models, such as functionalstructural-plant-models FSPM (Henke et al., 2016).Such approaches are advantageous as they spare steps of plant organ classification and parameterization (Paulus et al., 2013) as the models themselves provide organ affiliation.The second one is by plant skeleton reconstruction, detecting corresponding keypoints, e.g.tips of the leaves, and mapping them between the epochs using non-rigid transformation equations (Chebrolu et al., 2021).These algorithms are proven to work well with nearly ideal plant point clouds, so it is necessary to take appropriate measures to mitigate all point cloud distortions from Fig. 1 to assure a higher success rate.
Finally, with some additional scientific efforts, emerging ML approaches could be re-applied for tracking plant dynamics.
The applicable approaches are the ones providing the solutions to the scene-flow estimation problem with some adaptations, e.g.ones capable of simultaneous segmentation and rigid-body motion estimation of different scene parts (Huang et al., 2021) or ones describing non-rigid-body registration (Huang et al., 2022).The latter algorithms were demonstrated on humanmotion tracking, which could be directly transferable to plant phenotyping, and tracking the behavior of movable household objects, which is an overly simplified task for a direct transfer.Such approaches could even help improve corrupted scan epochs by inferring the true plant structure from other epochs through spatiotemporal 3D modeling.

CONCLUSION
This article provides a systematic overview of the challenges related to plant phenotyping using TLSs.We give a simple and actionable recommendation to the end users that can partially mitigate them and suggest research directions that can further help overcome these challenges.The insights presented in this article are based on the literature review and own experiences in sugar beet and wheat phenotyping.
The causes of these challenges were classified into three main categories related to the data acquisition principle, properties of the laser beam, and plant behavior.The three specific challenges that have the highest impact on the quality of the point cloud and the subsequent estimation of structural traits are: non-uniform point cloud properties due to specific scanning patterns; mixed pixels due to an unfavorable ratio of the laser beam footprint size and the size of plant structural elements; the external excitation causing sudden plant motion.These challenges can be seen as general constraints of TLS.However, their impact on the results of plant phenotyping is notably higher than in the other application domains.For example, in 3D representation of urban environments the objects are commonly static and the elements of the surrounding are typically much larger than a laser beam footprint size, reducing the relevance of mixed-pixels and non-uniform scanning pattern.
The main actionable recommendations are careful TLS selection, investment in the dedicated supporting infrastructure, informed viewpoint planning focusing on achieving optimal distribution of top-down views, and restriction in the variability of measurement configuration.The exact implementation of these recommendations will depend on the necessary trade-off of the following conflicting goals: spatial resolution, temporal resolution, throughput or measurement volume, and data quality.These goals can only be correctly optimized once the exact objectives of plant phenotyping (e.g.exact structural traits of interest) are defined and the properties of the measurement equipment and measurement subject are fixed and known.
There is evidence in the literature that the breakthrough in resolving the challenges will primarily come from adopting emerging solutions for point cloud processing based on machine learning developed by the computer vision (CV) community.The biggest benefits can be expected from algorithms dedicated to shape completion, surface reconstruction, RGB + LiDAR data integration and non-rigid and multi-body registration (spatiotemporal 3D modeling).
However, adopting these solutions poses a significant challenge on its own due to the necessity of bridging a domain gap.It requires collecting an abundance of diverse datasets with ground truth information for training and fine-tuning the algorithms, which is either costly or even completely infeasible to acquire in the case of 3D plant phenotyping.Hence, bridging the gap will almost certainly require further development of domainspecific synthetic data generation relying on the projects such as Crops in Silico and Helios LiDAR simulations.

Figure 3 .
Figure 3. Properties of wheat phenotyping TLS point cloud (single viewpoint): distances, angles of incidence, and number of neighbors in 0.1 m radius (sampling density).

Figure 4 .
Figure 4. TLS point cloud of wheat: line-of-sight direction (top), side view exposing mixed pixels in black rectangle (bottom)

Table 1
. Literature summary of structural traits observed by TLSs for: L-lab, G-greenhouse, F-field (PAI, PAD, PAVD-plant area index, density, and volume density; PLA-plant leaf area; LA, LAD, LAI-leaf area, density and index).
Figure 2. Challenges of TLS-based 3D plant phenotyping; grouped by causes related to the data acquisition principles (blue), properties of the laser beams (orange), and of the measured subject, i.e. plants (green)obstructions to lines of sight, making the data gaps unavoidable.Hence, scanning from multiple viewpoints is necessary and it even helps to mitigate the negative effects of the unfavorable measurement configuration (Sec.3.1).As the point clouds taken from each viewpoint are stored in local coordinate systems (CS), it is necessary to perform registration and transform them into a common global CS.