Case study Software Services for stakeholders in agriculture

Transcription

1 Case study Software Services for stakeholders in agriculture 1

2 Data and agriculture Challenges Ever more data - Increase agricultural production + 50% by Preserve natural resources Agriculture uses 70% of resources in water/year - Preserve environment 100 millions tons of nitrogen spread each year - Decrease GHG emission by agriculture (17 to 32%) Open databases Climate, Economy, Pedology In-fields observations Humidity, temperature, foliar coverage, nitrogen nutrition Imagery, satellites, drones Biomass, stresses Embedded sensors Yield, chlorophyll 2

3 Numerical technologies Modeling and simulation tools Technologies Data analysis and deep learning Applications - Plant breeding - Farm management - Greenhouse Optimisation and decision aid tools High Performance Computing - Yield forecast - 1st transformation, industrial processes 3

4 Services for every stakeholders in Agriculture Seeds breeding End-user Processing Numerical technologies for agriculture Crop management Yield forecast Ressources management 4

5 Different types of data Climatic data Agronomic data Pedologic data 5

6 Example : farm management Imagery Climatic data Plant modeling Data analytics and optimisation Sowing Irrigation Fertilisation Plant health Yield maps Phenotypic databases 6 Pedologic databases

7 Example: farm management Merge heterogeneous data fluxes Climatic data Real time simulation of plant growth In order to Anticipate phenological stages Real time data Following yield potential and stresses Intra-field input maps Historical data

8 Example : greenhouse management IoT technologies enable real time follow up of greenhouse conditions Simulation tools for plant needs forecast and greenhouse climatic conditions Producer constraints Climatic forecast Optimal management Smart controller dedicated to greenhouse Producer objectives Management techniques under constraints of production objectives and energetic costs

9 Dimension reduction Temporal data: sensors, climate, sampling Spatial data: satellite images, yield maps Extract the meaningful information in a lower dimension space: Principal Hessian Direction: Reduce temporal series to a restricted number of covariables Convolutional Neural Network: Extract features representative of images 9

10 Relational modeling of databases Search without prior knowledge of potentialities for value added services: Graphical relational modeling for - Visualizing structure and content of database - Discover and understand statistical relationships between columns - Modeling and forecast/optimization Learning algorithms of the relational model Example on the database Internet Movie Database (IMDb) From Maier et al. (2014) 10 l

11 Example : Management and processing of sugarbeets Learning of a relational model Harvest date Time of storage in field Sheeting Mulching Sugar content Non valuable sugarbeets Identification of opportunities for maximizing sugar content and reduce occurrences of non valuable sugarbeets Optimization of logistics Extraction machines 11

12 HPC and big data Relational modeling runs conditional independence tests that are very computation intensive Search for one dependency depends on the results for the search of other dependencies Parallelization is not trivial C++/MPI Analysis of parallel efficiency performed by Barcelona Supercomputing Center (BSC) with Extrae tools (POP ) Needs for dynamic load balancing methods in hybrid MPI + OpenMP 12

13 Dynamic Load Balancing Dynamic Load Balancing performed with DLB library from BSC Analysis from BSC for POP ( ) 13

14 Dynamic Load Balancing With Lend-Reclaim mechanism to optimize sequential parts Total performance results Analysis from BSC for POP ( ) 14

15 HPC and HPDA : Stakes and issues for the future Use case of regional scale yield forecast Lots of input data (satellite) Intensive computation of plant growth models 1. Identify crop fields with satellite images 2. Follow crop evolution from satellite images and forecast yield pixel by pixel 15

16 HPC and HPDA : Stakes and issues for the future - IO optimization : Deals with lots of data as input or output Data location One process access a lot of data A lot of processes access a small amount of data Crop identification : Neural networks : multi-layer recursive Follow crop evolution: Data assimilation: Particle filters Multi-spectral values Classification 16

17 Conclusion Big data and HPC technologies at every steps of agricultural product lifecycle Plant breeding, farm management, productions forecast, first processing One challenge is to take into account the heterogeneity of data types Advanced HPC methods required for optimization of complex data mining techniques such as relational modeling A lot of issues for the future around data location and IO at the interface of HPC and HPDA with nice use case from agriculture 17