In the agricultural imaging realm, working with multiple sensors at a time is often common since each sensor give access to a specific plant information, either related to the architectural, structural, biophysical, or reflectance properties of the plant. And interpreting the data collected from these sensors is required to compute what we usually call traits (i.e., plant assessments).
Now if we want to access more granular assessments, one way of doing it is to combine data from multiple sensors to superpose them and access more insightful and accurate information for agricultural research.
Enter the fascinating world of the pinhole camera model – a simplified yet powerful concept that forms the bedrock of computer vision. In this blog post, we'll explore the wonders of the pinhole camera model and its indispensable role in simulating the behavior of a camera for deeper granularity of phenotypic assessments in agricultural research and production.
At Hiphen we have experienced working with an array of cutting-edge industrial sensors, from RGB cameras to LiDAR, multispectral, thermal sensors and more, each providing valuable data to compute essential traits. Therefore, combining the data from these multiple sensors is crucial for gaining a comprehensive understanding of how plants are behaving in their environment. However, to achieve this, it is essential to be able to simulate and understand the geometric representation of a camera.
Early diagram of the pinhole camera model.
The pinhole camera model is a mathematical representation that describes how light from a three-dimensional scene interacts with an ideal pinhole camera to form a two-dimensional image. In this model, the camera aperture is represented as a point, and light travels in straight lines through the tiny hole before reaching the image sensor or film.
In agricultural imaging projects, researchers employ various sensors simultaneously to access diverse plant information. For example, RGB cameras provide valuable colour data, while 3D sensors allow for structural property assessments such as biovolume and height. By combining data from these sensors, researchers can achieve a more accurate and comprehensive understanding of plant traits. For instance, while LiDAR provides an excellent source of information for structural and architectural plant properties, it cannot provide data on leaf colour or temperature amongst other. However, by integrating LiDAR data with thermal data, researchers can precisely measure leaf temperature in a high-throughput fashion and with excellent repeatability. This level of data fusion enhances the accuracy and efficiency of plant trait assessments, revolutionizing digital phenotyping in agriculture and boosting agricultural research worldwide.
Digital phenotyping Data Fusion in practice. Leaf temperature can now be calculated precisely from imagery.
To superimpose data from different sensors, it's crucial to understand the geometric representation of the camera and ensure perfect alignment. The pinhole camera model allows researchers to put the 2D scene, captured by a thermal camera for example, into perspective to merge it with a 3D point cloud data for instance. To apply this methodology to phenotyping applications, researchers need to consider the sensors' positions on a 3-dimensional plane in comparison to the plant or tree being measured. Validating the alignment is a crucial step and involves extracting and comparing points from each image to ensure accurate layering.
Diagram illustrating the accurate superposition of 3D (point cloud) data with 2D (Thermal) data.
The pinhole camera model has played a crucial role in digital phenotyping evolution, enabling plant phenotyping experts to combine data from different sensors to achieve a deeper granularity of plant trait assessment. By fusing information from various sensors like LiDAR and thermal cameras, researchers can gain a comprehensive understanding of plant health, stress tolerance and resilience, and understand genotypes behavior in their environment globally. It also gives access to traits assessments that we can't imagine having access to before. Such traits and mathematical calculations can be easily implemented into Hiphen's PhenoStation® for phenotyping in controlled conditions but could also be adapted to PhenoMobile® for field-focused phenotyping projects. So, with the latest advancements in data fusion and sensor technology, digital phenotyping is poised to lead the way in shaping tomorrow's agriculture.
Grab a time from one of our experts' calendar to discuss about your project.
Sincerely,
Your Hiphen Team.
Topic brought to you by Matthew Cassidy – R&D Engineer @Hiphen.
In the digital agriculture realm, generating accurate plot patterns of your field trial is crucial for optimal crop monitoring to understand plant dynamics through time. To achieve this, the accurate and precise mapping of your plots, also known as plot mapping or parcellaire generation, plays a pivotal role. In recent years, the integration of deep learning techniques has revolutionized the process, enabled automated plot map generation, and offered a myriad of benefits for agricultural research and production. In this blog post, we will explore the importance of plot mapping, the advantages of using deep learning for automation, and how it impacts the agricultural ecosystem.
Traditionally, creating plot maps was a labor-intensive task involving manual measurement and mapping techniques. However, with the advent of deep learning algorithms, this process has been significantly streamlined. Deep learning models can automatically segment and delineate agricultural land into distinct plots, saving valuable time and effort for farmers, agronomists, and researchers. By automating this process, researchers can allocate their resources more efficiently and focus on other critical aspects of field trial management.
Big plots trial – Automating plot mapping with digital tools is highly helpful.
Small plots trial – Plot map still could be done manually but automation make it even more easy.
One of the remarkable advantages of using deep learning algorithms for plot map generation is the ability to achieve higher precision compared to manual mapping methods. Deep learning models can analyze aerial or satellite imagery, topographic data, and other relevant information to accurately identify plot boundaries.
Example of automated plot mapping of salad field from UAV imagery.
Accurate plot maps generated through deep learning algorithms have a direct impact on the processing of phenotypic data collected for each plot. Phenotypic data is the result of plant assessments like canopy development, stress and disease resilience, yield predictions and more, is essential for agricultural research and breeding programs. By having precise plot boundaries, researchers can associate specific phenotypic data with corresponding plots, enabling more accurate analysis and interpretation of plant assessment distribution within the entire field trial. This granular information facilitates the identification of patterns, trends, and correlations, ultimately leading to informed decision-making for crop research and production.
To achieve reliable and versatile deep learning models for plot map generation, robust training datasets are paramount. High-quality datasets that encompass diverse geographical regions, crop types, and plot patterns variations and orientation are essential for training models to identify plot boundaries effectively within images captured in various environments. This emphasizes the need for collaboration and data sharing within the agricultural community to develop comprehensive datasets that can improve the accuracy and applicability of deep learning models for plot map generation.
Set of images of plots in various conditions used for training DL models.
At Hiphen, we specialize in developing digital plant assessment solutions for agricultural research and production. We understand the significance of plot map generation and the transformative potential of deep learning in the agricultural sector. Our expertise lies in leveraging cutting-edge technologies to empower our clients with accurate and automated plot mapping alongside data acquisition and processing, enabling them to make better decisions and adapt to ever-changing environmental conditions. Through our tailored AI solutions, we strive to revolutionize the agriculture industry, promote sustainable practices, and enhance crop researchers productivity.
View of Hiphen's Cloverfield™ Data Platform.
Plot map generation holds immense importance for agricultural research and production. By utilizing deep learning algorithms, we can automate the process, saving time and improving precision. Accurate plot maps enable efficient resource allocation and precise data processing. However, it is vital to build robust training datasets to ensure the reliability and versatility of deep learning models. With Hiphen's AI methodologies, the agricultural industry can embrace innovation, make informed decisions, and adapt to the evolving needs.
Sincerely,
Your Hiphen Team.
Topic brought to you by Rhianna MCANENY – Image Processing Specialist @Hiphen.
In the agricultural realm, image analysis plays a pivotal role in understanding crop health, detecting issues, and making data-driven decisions. Among the critical factors that significantly impact the effectiveness of image analysis is the management of contrast and brightness. In this blog post, we will delve into the significance of contrast and brightness in agricultural image analysis and how they enhance the quality and usability of agronomic information.
Contrast within an image refers to the variation in brightness or color between different parts, creating visual distinctions that capture the viewer's attention. It is a key element in visual composition, facilitating the differentiation of various elements. In agricultural image analysis, contrast is essential for accurately identifying and interpreting crop health, disease symptoms, stress indicators, and other important characteristics.
To monitor and comprehend contrast in an image, a histogram is a commonly used visual tool. The histogram represents the distribution of brightness or color levels throughout the image. When considering brightness contrast, the histogram provides insights into how brightness values are distributed across the tonal scale, ranging from the darkest tones (blacks) to the brightest tones (whites). A well-balanced brightness histogram exhibits an extended distribution across the tonal scale, indicating good contrast within the image.
Example of an histogram graph representing the light distribution of an image.
Phenotyping, the process of measuring and analyzing plant traits, heavily relies on accurate and detailed imagery. Having well-contrasted images for phenotyping is crucial for a comprehensive interpretation of the information contained within each pixel. By accessing the full color range of an image, the richness of information increases, enabling the extraction of valuable agronomic insights. Thus, the quality of the information derived from crop images is highly dependent on the quality of the provided images.
Depending on the specific traits of interest or desired outputs, the level of contrast needed in images may vary. At Hiphen, we specialize in accompanying and guiding our clients in defining the exact level of contrast required for optimal data processing and a seamless phenotyping experience. By fine-tuning contrast levels, we enhance the accuracy and reliability of trait analysis, ultimately empowering better decision-making for researchers.
Image with an overall bad contrast
Image with an overall good contrast
Working with poorly contrasted images for phenotyping poses challenges and hinders the process. Images with inadequate contrast may be interpreted as underexposed or overexposed, impacting the processing of traits such as green cover, vegetation indices, disease detection, and organ segmentation among others from Hiphen's portfolio. Dealing with such images becomes time-consuming, tedious, and often results in data processing delays, leading to late delivery of results.
At Hiphen, we are constantly improving the automation of data quality checks through our Cloverfield™ platform. By the time you upload your datasets, we can promptly identify if the data is likely to generate processing issues or not. This allows us to inform our clients as soon as possible, in a fully transparent way and via an interactive dashboard, to ensure a frictionless phenotyping experience and to minimize delivery delays caused by poorly contrasted images.
The management of contrast and brightness is of utmost importance in agricultural image analysis. By understanding and optimizing contrast levels, we unlock the full potential of color information, facilitating the extraction of valuable agronomic insights. At Hiphen, we excel in developing AI methodologies for agricultural applications, assisting our clients in making informed decisions and adapting to ever-changing environmental conditions. With our Hiphen Academy, the first e-learning platform dedicated to help acquiring research-grade imagery from drones, we provide the necessary knowledge and skills to enhance image quality and maximize the potential of agricultural image analysis from the PhenoScale product range. Through effective contrast and brightness management, we can improve agricultural practices using digital phenotyping and drive innovation in the industry.
Sincerely,
Your Hiphen Team.
Topic brought to you by Martin GIRARDEY- Image Processing Specialist.
In the realm of phenotyping and agricultural research, image analytics has emerged as a powerful tool for extracting valuable insights. However, beneath the surface lies a crucial element that often goes unnoticed but plays a pivotal role in ensuring accurate analysis and interpretation georeferencing plots. Georeferencing involves assigning geographic coordinates to specific locations within field trials, enabling the reconstruction of Orthomosaic and facilitating precise data processing. In this blog post, we will explore the significance of georeferencing plots and shed light on the importance of coordinate systems, file formats, and advancements in technology for harnessing the full potential of image analytics in phenotyping.
Contrary to popular belief, the Earth is not a perfect sphere. It is slightly flattened at the poles and bulged at the equator, resulting in an irregular shape known as the geoid. Recognizing this, coordinate systems have been designed to account for the specific geodetic deformations in different regions, minimizing errors in positioning and representation. By aligning our data with accurate coordinate systems, we can ensure precise georeferencing and eliminate discrepancies that may arise during image analysis.
Geoid representing the Earth
Georeferencing plots within field trials allow to reconstruct Orthomosaic, a bird's-eye view of the field. An Orthomosaic comprises an RGB image superimposed with the associated plot map, which delineates the division of plots within the entire field trial. With a well-constructed Orthomosaic, we can accurately process data and deliver results through interactive and visual maps that showcase calculated traits for each individual plot. This approach enhances the clarity and interpretability of the obtained data, enabling data-driven decision-making. To ensure optimal plot patterns reconstruction we generally use Ground Control Points (GCP) that are positioned in the field and help identifying the orientation of the field, the origin of plot X1Y1 and make good alignments during the photogrammetry process.
Simplified process of creconstructing the orthomosaic of a field trial with GCPs.
Multiple coordinate systems exist to represent and locate points on the Earth's surface. Two widely used systems are WGS 84 (World Geodetic System 1984) and UTM (Universal Transverse Mercator). WGS 84 is a geodetic coordinate system that provides geographic coordinates, latitude, longitude, and altitude, based on a mathematical model describing the Earth's shape. UTM, on the other hand, is a projected coordinate system that divides the Earth into zones, allowing for more precise measurements within each zone. The choice of coordinate system depends on the specific requirements of the research, but in our experience, UTM coordinate systems have proven to be more efficient and precise as we can see with the example below:
The importance of choosing the right coordinate system: on the left, the length of the plot is 10.79m, while on the right, it is 8.81m.
In addition to coordinate systems, the file format used for storing GPS coordinates is crucial for preserving geographic information. At Hiphen, we recommend specific formats such as shapefile, GeoJSON, geopackage, and KML. These formats ensure that the geographic information contained in the data is preserved and accessible during analysis. Choosing alternative formats like CSV can lead to the loss of vital information regarding the coordinate system used during data acquisition, hindering proper interpretation and analysis.
Thanks to advancements in drone, robotics, and GPS technologies, collecting GPS data has become increasingly automated and efficient. Metadata recorded directly in each image simplifies the process of gathering accurate georeferencing information. By leveraging these technological improvements, researchers can streamline their data collection processes and enhance the accuracy and reliability of plant assessments. This is mostly possible currently thanks to RTK technology. RTK means Real Time Kinematic, and this geolocation technology allows a centimeter-level positioning of the measured object, with real-time synchronization. Such technology is now implemented as standard in most of the latest generation of portable drones with GPS modules mounted on top of the devices, but at Hiphen we also use it as an independent module mounted on a stick, for georeferencing GCPs position, or even mounted on our autonomous PhenoMobile robots.
Field operator collecting GPS coordinates of ground control points with RTK precision device.
At Hiphen, we understand the importance of research-grade data collection and processing. Our experts work closely with researchers to define the best protocols for ensuring high-quality data collection. We navigate through various projected coordinate systems, adapting to the location of the field, to ensure the measurements of plots align accurately with real-world observations. By partnering with Hiphen, researchers can leverage our expertise in georeferencing plots and unlock the full potential of image analytics for phenotyping.
Georeferencing plots for image analytics is an essential but often overlooked aspect of phenotyping. The process of assigning geographic coordinates to field trial locations enables the reconstruction of Orthomosaic, leading to accurate data processing and visualization. Choosing appropriate coordinate systems, file formats, and harnessing advancements in technology empowers researchers to extract meaningful insights from their data. Collaborating with experts like Hiphen ensures that research-grade data collection and processing are achieved, ultimately advancing agricultural research, and propelling the field of phenotyping to new frontiers.
Sincerely,
Your Hiphen Team.
Topic brought to you by Adrien Vielix – Field Acquisition Manager @Hiphen.
Plant phenotyping is a brick in the wall of agricultural research and plant breeding. It helps understanding how a plant is behaving in its environment, by assessing key information related to plant morphology, physiology, biochemistry, yield, and responses to biotic and abiotic stresses. Having access to such information enables us to understand plant dynamics to be able to predict how a variety will perform and produce in a specific environment. Phenotypic assessments, which are accessible by combining Genotypic information x Environment data, are mostly used to improve varietal selection and make it completer and more precise than the genomic approach, which relies on on the study of gene presence and performance through genotypic assessments.
Digital phenotyping is quite a new realm, companies like Hiphen were founded less than 10 years ago. Thus, such technology needs some time to be well-adopted and improved to unleash its full potential. Nonetheless plant phenotyping by itself is not something new, it has existed for ages since agronomists got eyes and can measure plants manually. But most of those manual techniques are either often destructive or time-consuming. However, the rising of digital phenotyping by leveraging new technologies has seen the use of sensors and imaging platforms emerging as a fast and efficient approach to quantitatively assess plant characteristics i.e., phenotypes, in a non-destructive way. And at Hiphen we believe that the use of those sensors to create data fusion enables more accurate and repeatable assessments, in a high-throughput fashion, based on the theory of the 6 dimensions of phenotyping.
Different imaging platforms can be used for field and indoor phenotyping to process data about those 6 dimensions, like Hiphen's Cloverfield, and those platforms are using data collected from various sensors. Commonly used imaging technologies include visible light imaging, thermal imaging, 3D imaging, chlorophyll fluorescence, hyperspectral imaging, and tomographic imaging. So, let's take a deeper dive into the sensor technologies accessible for agricultural imaging nowadays.
Several imaging platforms are utilized for both field-focused and indoor plant phenotyping. These platforms gather data from various sensor types to obtain a comprehensive understanding of plant traits. Some of the commonly used imaging techniques include:
RGB image of plots with emergence issues.
Segmented and annotated RGB image of wheat heads.
Thermal assessment of trees in greenhouse – Here we can see Data fusion in action combining RGB imagery with thermal to estimate precisely leaf temperature.
3D sensing equipment helps accessing information about crops at all plant levels.
NDVI image captured from UAV.
Fluorescence Imaging: Fluorescence imaging involves measuring the light energy emitted when the plant absorbs shorter wavelength radiation, mainly through the chlorophyll complex. The emitted fluorescence is a tiny fraction (<3%) of the total radiation emitted by the light source to the object. The amount of re-emitted light (fluorescence) is a reliable indicator of the plant's ability to use the absorbed light and is used to estimate the overall health status of the plant. Fluorescence imaging is utilized to estimate photosynthetic efficiency and other associated metabolic processes affected by biotic and abiotic stresses. However, this technique does not specify the cause of variations in the plant signal, such as light, temperature, or other environmental factors.
Tomographic Imaging: Other imaging techniques, such as Magnetic Resonance Imaging (MRI), X-ray Computed Tomography (CT), and Positron Emission Tomography (PET), provide high-resolution 3D images of single plants or plant parts. MRI captures 3D images of internal structures, allowing non-invasive quantification of static and dynamic traits, such as structural, biochemical, and temporal changes inside the plant. X-ray CT visualizes the 3D structures of both internal and external plant features at the micro or macro level. These imaging techniques are time-consuming and not suitable for processing substantial amounts of data. Additionally, their large size and weight prevent their use on aerial imaging platforms.
The selection of sensor technologies depends on the specific assessments' researchers aim to perform. Companies like Hiphen are proficient in guiding researchers in identifying the most appropriate sensors and traits for their needs. Data-driven decisions and insightful research based on phenotypic data assessments are key to enhancing agricultural practices, improving crop yields, and ensuring food security for the growing global population.
In conclusion, imaging technologies for plant phenotyping have made significant strides in recent years, and their integration with digital phenotyping has brought new opportunities for advancing agricultural research and plant breeding. As these technologies continue to evolve, we can expect even more sophisticated and efficient ways to understand plant dynamics and harness their full potential to address the challenges of modern agriculture. The future of plant phenotyping is promising, and it holds the key to sustainable and efficient food production for the years to come.
Sincerely,
Your Hiphen Team.
Topic brought to you by Marc LABADIE – Project Leader at Hiphen.
Digital phenotyping has revolutionized the field of agriculture by providing novel ways to monitor and analyze plant growth and development. One fascinating application of digital phenotyping is in orchards, where the accurate counting and classification of fruits plays a crucial role in yield estimation, resource allocation, and overall orchard management. In this blog post, we will explore the significance of digital phenotyping in orchards and focus on a pipeline that employs terrestrial LiDAR scanners for counting and classifying fruits in apple tree orchards.
Digital phenotyping involves the use of advanced technologies, such as remote sensing, computer vision, and machine learning, to extract meaningful information about plants. In orchards, digital phenotyping offers several advantages. It enables growers to monitor the health and productivity of trees, assess the effects of different input products or environmental conditions, and optimize resource allocation to produce varieties with improved yields. Accurate fruit counting and classification is particularly valuable, as it helps estimate crop yields, plan harvesting operations, and make informed decisions regarding fertilization, irrigation, and pest control.
As you may know, imaging solutions for phenotyping are actionable trhough different system types i.e., Vectors. You can discover a detailed list of systems that are suitable for most agricultural applications HERE. For orchards, since they can be quite dense, portable Handheld systems and machinery systems like PhenoMobile are of most interest, even though Drones can be of good help to create some data fusion from a bird eye's view.
In terms of sensors, terrestrial LiDAR scanners have proven to be a very reliable source of information while phenotyping for fruit counting and classification in orchards. These scanners emit laser beams that measure the distance to surrounding objects, creating a detailed 3D representation of the environment. They can be mounted on mobile platforms as mentioned before, such as drones, handheld systems or ground vehicles, and scan the orchard trees from multiple angles. The resulting point cloud data provides a rich source of information that can be processed to extract meaningful information i.e., Plant traits about the behavior of the trees and their organs.
Hiphen's R&D engineer Nathan guilhot, acquiring data on Apple trees, with Hiphen's new handheld system developed with Arvalis.
The process of counting fruits in orchard typically involves a pipeline consisting of 3 main steps: global tree segmentation, fruit semantical segmentation, and fruit clusterization.
Segemnted trees from above.
In red color, we can see all 3D points (from dense cloud) identified as points constituing apple fruits
In this illustration, all points referring to the same apple fruit have been grouped to highlight every single fruit in the tree.
Digital phenotyping, with its ability to automate and phenotype larger orchards at higher speeds, brings significant advantages to fruit counting and classification and orchard management. By employing advanced technologies, such as 3D deep learning and point cloud analysis, researchers can efficiently assess large orchards, and estimate fruit counts. This frictionless phenotyping experience offers researchers the opportunity to make insightful data-driven decisions, optimize resource allocation, make precise yield prediction and enhance overall orchard productivity.
The integration of digital phenotyping into agriculture is a transformative approach, empowering growers with valuable insights and driving sustainable and efficient practices in the management of orchards globally.
Our team is at your disposal should you have any questions, feel free to book a 30min meeting with one of our experts 👉 https://calendly.com/d/gtn-s8d-6fh
Speak soon,
Your Hiphen Team.
Topic brought to you by Nathan Guilhot, R&D Engineer @Hiphen
Spatial correction is a statistical technique employed in plant breeding to mitigate the impact of environmental variability on plant performance. By adjusting observed trait values based on plant positions within the experimental field, spatial correction helps counter the effects of environmental and trial design-related factors, such as soil, climate, diseases, plot patterns, replications, and blocks. This correction ensures fair comparisons among tested varieties and facilitates the selection of those best suited to specific growing conditions. Discover in detail how spatial correction enhances precision and reliability in plant breeding.
Non-corrected data distribution within a field trial.
Corrected data distribution within a field trial.
To apply spatial correction, we can use smooth two-dimensional surfaces to model spatial variation. For example, anisotropic P-splines can be used to distinguish large-scale spatial trends (global trend) from small-scale trends (local trend). The spatial field includes the effects of genotypes, blocks, replications, and/or other sources of spatial variation described by a classical mixed model. Each component of the model has an effective dimension, which is related to variance estimation and helps characterize the importance of model components. An important result of this method is the formal relationship between several definitions of heritability and the effective dimension associated with the genetic component. This method was developed and illustrated by Rodríguez-Álvarez et al. (2018) in their article "Correcting for spatial heterogeneity in plant breeding experiments with P-splines."
Example of spatial trends of adjusted traits by the SpATS model for different modalities of an experiment (Rodríguez-Álvarez et al., 2018)
Spatial correction helps breeders account for the spatial variation that exists within field trials. In agricultural research, field trials are often conducted on large plots of land, and spatial heterogeneity can arise due to differences in soil fertility, microclimate, disease pressure, or other environmental factors. Without proper spatial correction, these variations can introduce bias and confound the estimation of genotypic effects.
By applying spatial correction techniques, breeders can account for the spatial structure within their phenotypic data. This involves modeling and removing the systematic spatial trends present in the field trials, thereby reducing the influence of environmental factors and improving the accuracy of the analysis.
Spatial correction methods can be implemented using various approaches, such as spatial analysis of variance (ANOVA), spatial regression models, or spatial mixed models. These techniques allow breeders to explicitly model the spatial autocorrelation that exists between neighboring plots and estimate the residual variation that is truly attributable to genetic effects.
Incorporating spatial correction in the analysis of phenotypic data also helps in the identification and elimination of outlier observations. Outliers may arise due to localized environmental factors, measurement errors, or other sources of variation. By detecting and removing these outliers, breeders can ensure that their analyses are based on reliable and representative data, leading to more accurate conclusions and better-informed breeding decisions.
Furthermore, spatial correction aids in the integration of multi-environment trials (METs). METs involve evaluating genotypes across different locations or years to assess their performance under diverse environmental conditions. Spatial correction techniques can help harmonize the data collected from different environments, enabling breeders to make valid comparisons and to identify and predict the potential behavior of various genotypes in different environments by getting more stable genetic values, reducing Genotype x Environment interactions, achieve higher adaptive values, and more.
Spatial correction functionnality render in Cloverfield™ Data Platform
In the end, spatial correction is essential for plant breeders to improve the quality and reliability of their phenotypic data analysis. By accounting for spatial variation, breeders can enhance experimental design, strengthen selection processes, increase genetic gain, and facilitate the accurate evaluation of genotypes across diverse environments by helping to calculate narrower confidence intervals and smaller p-values for instance. Incorporating spatial correction techniques into their breeding programs empowers breeders to make more informed and accurate decisions, leading to the development of improved cultivars that pave the way for tomorrow's agriculture.
c) other examples
If you are interested in spatial correction and want to apply it to your breeding experiment trials, at Hiphen, we can help you implement this method and interpret the results. Hiphen has developed the tools for spatial analysis using P-splines and mixed models. We can also advise you on choosing the most suitable experimental design that will help you maximize the value of your field trial.
Representation of fitted spatial trends of an agricultural field.
Spatial correction features are coming to Cloverfield™ in 2024, you will then be able to apply spatial correction on your selected traits to get instant visualization of the corrected data and start making data-driven decisions fast.
Sincerely,
Your Hiphen Team.
Topic brought to you by Don Ced OGOUMOND – Imaging Solutions Specialist @Hiphen.
Last time in the Hiphen blog we were talking about the evolution of drone equipment for agricultural applications, saying that it has become more and more accessible and easier to use through the last decade. Now we’ve reached a new milestone in drone phenotyping, as DJI released a new version of the Mavic range with the Mavic 3M, a promising device that embeds a high-resolution RGB camera combined to a large-specter multispectral camera, and at Hiphen we’ve already tested it. So, evolution or revolution?
A 5MP multispectral camera (560nm > 860nm) combined to 20MP CMOS RGB camera will basically give you access to the most thought-after traits currently, from biochemical traits such as leaf surface to plant count and plot quality traits, you can now access all these measures easily within the same flight.
Figure 1: As we make our detection models evolve, we are now able to extract much more precise information than before.
Here is an example of images you can get with the new Mavic 3M; RGB + Multispectral images are acquired at the same time to speed up plant asssessment:
Figure 2: RGB image of Strawberries from Mavic 3M
Figure 3: NDVI of Strawberries computed from Mavic 3M
Figure 4: GNDVI of Strawberries computed from Mavic 3M
Figure 5: NDRE of Strawberries computed from Mavic 3M
What as evolved a lot from the previous model from the DJI family, combining an RGB + MS camera, is actually the RGB camera. Indeed, the sensor combination of this new Mavic 3M is way better than the DJI Phantom 4 MS, RGB images being sharper, more balanced and more precise with the new CMOS 4/3 sensor thanks to its 20MP resolution (see example below)
Figure 6: RGB image from the Mavic 3M
Figure 7: RGB image from the Phantom 4M
Then the multispectral camera as not really evolved, the specs are almost the same, the slight difference is that the Near-infrared (NIR) filter of the sensor is now calibrated at 860nm vs 840nm on the Phantom 4M. In practice this doesn't change anything to assess the plant reflectance in order to extract NDVI, GNDVI or NDRE indices from your images.
Figure 3: NDVI of Strawberries computed from Mavic 3M
Figure 8: NDVI of Rapeseed computed from Phantom 4M
The blue band is no more included in the MS sensor of the Mavic 3M, indeed this band is not specifically useful in most applications, moreover we can still acces it with the RGB images containing the Red, Green & Blue wavebands.
With this great sensor’s combination, we can now combine RGB + Multispectral image acquisition at the same time, during the same flight, where before 2 separate flights were needed to acquire RGB and MS data. However, you will still need to upload your RGB and MS datasets in 2 different upload sessions to make sure processing will run smoothly.
This new drone also helps us envision merging the best of both worlds (RGB + MS) to dive into much more detailed granularity of information that we can extract from such images. We’ll continue testing and developing modules to access new traits in the future – stay tuned!
Figure 9: The cameras in action – You can now choose to have an instant visualization of the MS camera with a few vegetation indices like NDVI or NDRE while flying over your trials – Or wether to have an instant RGB camera feedback
RTK means Real Time Kinematic, and this geolocation technology allows a centimeter-level positioning of the device, with real-time synchronization, will flying above your trials. Like the latest generation of portable drones from the DJI family, the new Mavic 3M comes as standard with an RTK-precision GPS module mounted on top of the drone that will ensure great image georeferencing and easy alignments while reconstructing your Orthomosaic.
The battery life is also great, DJI mentions 43 mins of flight duration in perfect conditions. We actually measured it flying for 32 mins above a wheat field at 12M high while capturing both RGB and MS images, with a single battery. It is also more silent and compact than previously. in addition it fits easily in a backpack with the 2 batteries that comes as standard with the drone.
To compare it to the previous Phantom 4M, the battery-life is improved due to the lightness of the device and the improved efficiency of the fast-charging lithium batteries. We need to make further testings on this but it seems like we can have a significant 35% to 50% of additionnal battery-life than the Phantom 4M.
With all those features packed into one device, the great surprise is also the price, selling at around 4,300€, the Mavic 3M is giving access to the best drone technology for less money than before if we compare it to another member of the DJI family, the Matrice 300 equipped with two gimbals + two sensors, which we recommended a lot in the past and that costs around +20,000€ for almost the same technology. With the Matrice 300, the RGB camera would be better, however you’ll access the same range of traits with the Matrice than with the Mavic 3M.
Overall, the new Mavic 3M is more precise, more easy to operate, more reliable in terms of image acquisition, faster and less expensive than ever. The combined sensor head will facilitate you accessing most valubale traits for drone phenotyping such as vegetation indices and radiative transfer traits alongside counting and quality traits within one single flight, with less energy and less money spent. It's probably the biggest revolution in drone phenotyping since the establishment of the Mavic range. We strongly recommand this new Mavic 3M for most of your phenotyping applications.
We will soon make further testing of this Mavic 3M in various conditions and update this post with our latest conclusions. In the meantime, feel free to grab a time to discuss drone image analytics for phenotyping with one of our experts using the calendar below!
Sincerely,
Your Hiphen Team.
1/ Choose a day 📆 > 2/ Select a time slot at your convenience 🕒 > 3/ Confirm the meeting ✅ > 4/ You're all set 🎉
Selecting a Drone to Meet Your phenotyping Needs:
As drones have become better, less expensive, and easier to use, more researchers have been incorporating them into their trial assessment programs. The new generation of drones are bringing more capacity to assess new and previously inaccessible traits such as wheat head count, flower/fruit counting and classification, and phenological stage detection.
Many interesting traits can be assessed with multi-spectral sensors, but many of the most exciting and impactful traits that leverage deep learning tools are extracted from imagery captured with high resolution RGB cameras.
So, if you are thinking about leveraging drone technology to extract new traits to enrich your trial data sets, here are 5 drone features to consider for High-throughput Plant Phenotyping (HTPP).
Pixel count is very important for many of the most interesting and important phenotyping operations. More pixels create higher resolution to be able to classify and count small features in images such as counting thousands of heads of wheat in a plot. Generally, a research grade drone starts at 20 megapixels (MP) like on an Autel Evo II Pro or DJI Phantom 4 Pro V2. Sensors up to 100 MP are available for drones and allow higher resolution at higher altitudes which can help shorten flying times.
Pixel count alone does not tell the whole story. Sensor size is even more important than megapixel count for phenotyping.
Resolution without an acceptable sensor size can be misleading. Larger sensors can have larger pixels which allow each pixel to capture more light and thus create a sharper image. With smaller pixels, less light enters each pixel and if there is not enough signal per pixel, cameras “bind” the pixels together reducing the functional resolution. An example that often confuses people looking at drones on the market now is:
Even though the Enterprise Advanced has a 48mp camera, the small pixels size actually yields images less useful than the larger sensor 20mp on the Mavic 2 Pro. Thus, larger sensors are particularly important for advanced deep learning phenotyping from drones for things like plant count or flower/fruit counting and classification.
There are trade offs around senor field of view (FOV). FOV and altitude define your global image footprint. A narrow FOV generates crisp images better suited for fine grained detail required for counting and classification deep learning. But with a smaller footprint you will need to take many pictures of a field (ei higher overlap) which will take more time and data storage space. For phenotyping applications on objects that have significant height or area (corn or tree crops) this smaller FOV also reduces geometric distortion caused by rendering a 3D object in 2D. So, for most detailed applications, a smaller FOV is preferred for plant phenotyping.
High quality optics yield images with higher sharpness, less distortion, less vignetting, and less chromatic aberration. Unfortunately, there are rarely metrics in specifications that allow you to assess the optics quality of sensors. For drones, unfortunately, all you can use as a guide is price. The higher the quality of the optics, the higher the price. Hasselblad and PhaseOne are 2 sensor providers that generally use high quality optics, but there are many others too.
Shutter type also affects image acquisition. Here are the 3 main types of shutters:
So, how does this boil down to the decisions you will be making on which drone best fits your needs?
Many research teams we work with select an entry-level research drone to get started quickly and then move to the advanced drones when they have a season or two of experience. But others realize the added impact that a more advanced drone can bring to their program, want to realize that value quickly, and will start with an advanced drone configuration.
Below are some examples of the rough cost of each equipment option and some of the systems we’ve had the best experience with.
Entry-level Research Grade Drones – fixed sensors:
Advanced Drones – accommodate multiple sensors:
With higher resolution and larger sensors, you can fly at higher altitudes and fly faster and still acquire higher quality images in less time. These drones can carry multiple sensors which can be swapped in and out depending on your needs. This provides flexibility but can make operating more complex because the sensor and the drone are often not integrated. We currently use the DJI Matrice 300 and are very satisfied with its performance, flexibility, and ease of operation. The Matrice 300 costs roughly 10 000$ for the drone alone. Then you will need to add the cost of a gimbal and cameras. For the RGB camera options that really make these drones capable of delivering unique traits, here are several of the most common sensors:
Technical Specs Of The Sensors:
Overall, you need to determine the traits that will add the most value to your research programs and select the right tool for the job.
If you want help thinking through which traits are accessible and which drones and sensors are best suited to your specific needs, simply find a time on our calendar below and we will be happy to discuss your options and help you make the appropriate choice for your needs and budget:
1/ Choose a day 📆 > 2/ Select a time slot at your convenience 🕒 > 3/ Confirm the meeting ✅ > 4/ You're all set 🎉
When you are a company from the seed breeding industry or others, the objective of setting up drone flights for plant phenotyping in a research program is to digitize, simplify and improve the measurement of plants and field with sensors thanks to a precise and unique methodology for all your trials, regardless of their location in the world.
Today, setting up a drone acquisition protocol for agricultural image analytics is a process that some consider long and sometimes expensive. However, companies specialized in image processing for plant phenotyping can accompany the actors of the agricultural ecosystem to set up such projects, and this is where Hiphen can help.
From advice on the most suitable equipment for your project, to flight procedures, data processing and beyond, Hiphen accompanies you all along your phenotyping journey to enable you to access frictionless phenotyping with drones.
Implementing drone flights into a research program is journey that can be divided into 6 mains phases, some of them are recurring, and there are several people involved in such project so first, lets focus on the key players and their roles before to start.
We have listed exhaustively the key players involved in implementing drone flights for plant phenotyping projects in the table below. Each of them are important because it's crucial to understand that the initiative is a chain and not the sum of its links. People from several departments are sometimes involved and providers can also be a part of the project (especially for the drone pilots), so that's why a binding phase needs to take place at the very beginning of the project, as we will see later.
Thus, here are the players that you need to identify within your organization before to start, and their role in the initiative:
| Name | Role |
|---|---|
| Client Management | Decision-makers on client side. They are involved in the major steps of the project. |
| Breeding team | A motivated team with a nominated contact point whose role is to secure ground observation and synchronize observations and flights with the whole team. |
| Project Coordinator (Client Side) | Coordinator of the project on a client side. He is a member of the breeding team and oversees the ground measurements, of the tele pilots planning's, of making the link with Hiphen's project coordinator, of the upload of the datasets to our data platform and of sending updates upon the project's progress or issues. |
| Project Coordinator (Hiphen Side) | Global coordinator of the project, he helps to define the scope of the project and helps to select the device(s)/sensor(s). He also helps to define the traits, the validation process and the phenological stages to fly at with the help of Hiphen's agricultural remote sensing experts. He Generate and send the SOPs and send feedback upon the acquired/uploaded data. Most importantly, he oversees the data processing of the selected traits. |
| Field Acquisition Operator(s) | Whether drone tele pilot with a diploma within the client's organization or external service providers, here also with a contact point involved in the meetings and updates for a better project's coordination. The field acquisition operator(s) oversee acquiring research-grade image datasets following strictly the protocol sent by Hiphen's project coordinator. |
In some cases, people from your organization could cover several roles, but this gives you a precise overview of the resources needed to implement drone flights into your research program.
Now that the key players are identified, it's time to focus on the main steps of the implementation process. Here is the summary of the different phases:
For each phase we several steps to complete and several people involved in, so let's dive deeper into each phase.
It is basically an introductive meeting to let all the stakeholders to get to know each other's and their role in the initiative. As we said before, this step is crucial to make sure everyone in the organization and in the project understand that it's a process that must be followed in the right order to become successful. This is obviously a one-off phase and all the players, and the service providers if some are involved in the project at any time, must take part in this introductive meeting.
It's s also a one-off phase but this one must take place for every trial if there are many located in different places. In this second phase, you must at least:
With this phase, we are aiming at performing all the preparatory and legal steps to be able to trigger drone flights on demand later, whenever you need in the season. Finally, to give a timeframe for this phase, overall, it can last from 2 to 6 months based on our previous experiences. It can be more for some countries since local restrictions apply for drone providers (like DJI in the US for instance) or if you encounter any issues with your civil aviation authority.
With phase number 3 coming up next, we are entering the phases that are recurring every new project during the research program, knowing that a research program can last several years (we usually say that a breeding cycle lasts an average of 7 years).
The first phase which is recurring is the project's definition, and by that we mean that it's time the define the objective and stakes of the project. One project usually refers to one growing season. Here is what you need to do at least:
This third phase is a key pillar of the project and sets the basis of implementing an easy-to-use decision support tool into your research program. Duration of this step is around 2 to 3 weeks depending on your organization.
No surprise here, this phase seals the project for the upcoming season when all the paperwork and payments have been figured out between you and Hiphen. This doesn't take too much time; we consider 2 weeks as a reasonable timing.
The penultimate phase of implementing drone flights for plant phenotyping projects is planning the season:
We consider taking 2 to 3 weeks to plan the season properly before being able to fly the drone for plant phenotyping projects. Also note that this phase and the next one are aiming at giving you access to analytics to support breeding decisions. So, following these steps in the right order will ensure the smooth running of the project.
The final phase of the process, this one objective is to let you collect data for real-time decision thanks to the drone flights and Hiphen's data processing expertise. To make sure the project will be a success, there are a few more steps to complete such as:
Timing this phase is almost impossible since all projects are different and since this phase lasts the entire growing season. So, it depends mostly on the crop(s) concerned and on the number of flights that are needed to compute the selected traits.
Now we have covered the entire process of implementing drone flights for plant phenotyping projects based on our own experience with CGIAR and other institutions and companies. To summarize all the steps and phases, we have put together a more digestible infographic for you to understand the whole process:
Feel free to contact us should you have any question about a specific phenotyping trait through our contact page. We look forward to hearing from you!
Speak soon,
Your Hiphen team.
