Comparison of LUCAS program and REQUASUD datasets

Introduction

The aim of the Task 6.3. of the Work Pakage 6 is to compare soil data sets between LUCAS sol and a network of Walloon laboratories REQUASUD. This web page is intended to provide users with the comparison process and the ‘R’ code to perform the analysis. For each section, an ‘R’ code is provided with some explanations.

The comparison is implemented for all possible soil properties that correspond to both LUCAS and REQUASUD datasets.

Main partner : Centre wallon de recherches agronomiques, Belgique

Project and funding : European Joint Programme on Agricultural Soil Management

In collaboration with …

and , for the data

and and , for the script preparation.

if (!op.setup.org) {
# Libraries

library(dplyr)
library(sf)
library(mapview)
library(kableExtra)
library(Hmisc)
library(viridis)
library(dgof)
library(ggplot2)
library(tidyr)
library(conflicted)
library(raster)
library(tmap)
library(psych)
library(DT)
library(ggspatial)
library(terra)
## library(xlsx)
library(readxl)
library(spatstat)

# library(slickR)

# Preventing conflicts

conflict_prefer("select", "dplyr")
conflict_prefer("filter", "dplyr")
conflict_prefer("mutate", "dplyr")
conflict_prefer("describe", "psych")
conflict_prefer("summarize", "dplyr")
conflict_prefer("aes", "ggplot2")
conflict_prefer("theme_classic", "ggplot2")
conflict_prefer("theme_minimal", "ggplot2")
conflict_prefer("arrange", "dplyr")
conflict_prefer("theme_bw", "ggplot2")
}

Technical information

This parametric report is done using rmarkdown approach including scripts in R, a language and environment for statistical computing and graphics (https://cran.r-project.org/). The whole Walloon part of this task, including this report, scripts and main configuration files, was initially cloned from the French git-directory available here : https://forgemia.inra.fr/nicolassaby/ejpsoilwp6lucas . In the near future, the Walloon part of this task will be publicly available also here [LINK TBA] and the Walloon report can be consulted here [LINK TBA].

Session information

The R-scripts were run and the rmarkdown report was generated on a personal computer. Here after, the information linked to the computing session can be consulted in Appendix (Session information).

Loading personal functions

For importing data, processing data analyses and producing graphs and maps, we developped customized functions. These are stored in two R files that are load here. Contents of these files are shown in Appendix (Personal functions).

source('./R/wl_personal_functions.R')
source('./R/wl_personal_functions_fred.R')

Data

For LUCAS and REQUASUD datasets, data are imported from three main types of files : shapefiles (geodata and its associated databases), excel files and csv files.

For LUCAS, we mainly used the available shapefiles. It is important to note that for the 2018 LUCAS data, the attribute table in the shapefile only contains the point ID and geometry. The other variables of interest are in a excel file.

For REQUASUD, source data are imported from csv files. Till now, these data are not fully opensource. We share in this report only statistics and aggregated data for REQUASUD.

Time period checking

The data from the Walloon network of laboratories provided by the REQUASUD unit represent the results of soil analyses for the determination of soil quality from 2008 to 2021. The comparison will therefore be made taking into account the LUCAS campaigns of 2009, 2015 and 2018.

On a human time scale in the absence of erosion or reworking, the particle size fractions of a soil are considered stable. Thus, the 2009 and 2015 LUCAS campaigns for particle size will be compared to particle size data for all years in the REQUASUD dataset. We could have taken all the LUCAS years available to us, however, the particle size was not analyzed in 2018.

For other soil properties, the comparison is made only for the year 2018.

Import data

Setting the main path directories

path0    = "."
path_in  = paste0(path0,"/data-raw")
path_out = paste0(path0,"/data-output")

Import LUCAS data

We have three shapefiles representing the Lucas data sample points for the years 2009, 2015 and 2018 for the European Union member states. The points corresponding to those of the Walloon region are extracted using a shapefile of the Walloon boundaries.

As mentioned above, the attribute table of the LUCAS 2018 shapefile does not contain all the variables of interest, so a join is made with the excel file.

if (op.import.data) {
# Import and read LUCAS data
f_2009 <- paste0(path_in,"/SoilAttr_LUCAS2009/SoilAttr_LUCAS_2009.shp")
f_2015 <- paste0(path_in,"/LUCAS_SOIL_2015/LUCAS_Topsoil_2015_20200323.shp")
f_2018 <- paste0(path_in,"/LUCAS_SOIL_2018/LUCAS-SOIL-2018 .shp")

lucas_eu_2009 <- st_read(f_2009,stringsAsFactors = F)
lucas_eu_2015 <- st_read(f_2015,stringsAsFactors = F)
lucas_eu_2018 <- st_read(f_2018,stringsAsFactors = F)

# Import and read the limits of Walloon
f_limit <- paste0(path_in,"/wallonie/provinces.shp")
wallonie <- st_read(f_limit, stringsAsFactors = F)

# Define LUCAS Walloon data

#-- Convert coordinates L72 to wgs_84
wallonie_wgs84 <- st_transform(wallonie, st_crs(lucas_eu_2015))


#-- Intersection
lucas_wal_2009 <- st_intersection(lucas_eu_2009,wallonie_wgs84)
lucas_wal_2015 <- st_intersection(lucas_eu_2015,wallonie_wgs84)
lucas_wal_2018 <- st_intersection(lucas_eu_2018,wallonie_wgs84)


# Join
f_lucas_eu_2018 <- paste0(path_in,"/LUCAS_SOIL_2018/LUCAS-SOIL-2018.xls")
## excel_lucas_eu_2018 <- read.xlsx2(f_lucas_eu_2018,sheetIndex="LUCAS-SOIL-2018",
##                             header=TRUE,colClasses=c("character",rep("numeric",9),
##                                                      rep("character",8),rep("numeric",2),
##                                                      "character","numeric",rep("character",5)))

excel_lucas_eu_2018 <- readxl::read_excel(f_lucas_eu_2018,
sheet = "LUCAS-SOIL-2018",
col_types = c("text",rep("numeric",9),
             rep("text",8),rep("numeric",2),
             "text","numeric",rep("text",5))
)


lucas_wal_2018 <- left_join(
    lucas_wal_2018, 
    excel_lucas_eu_2018, 
    by = c("POINTID" = "POINTID")
    )
}

Import REQUASUD data

The soil analysis data of Wallonia are imported and attached to an excel file containing the name of the land cover code according to REQAUSUD. The following code is the one that import the “fuzzyfied” data from REQUASUD. But, for this report, we used the original data set, only internally available. The file that is sourced has the same structure of the “fuzzyfied” one.

if (op.import.data) {
if (op.data.file == "fuzzyfied") {
# Import and read REQUASUD data
f_REQ <- paste0(path_in,"/REQ/extraction_a10_2022_corrige.csv")
REQUASUD.1=read.csv2(f_REQ,header = TRUE, dec=",", stringsAsFactors = F,
                   colClasses = c("numeric",
                                  "numeric",
                                  "character","character",
                                  "numeric",
                                  "character","character",
                                  rep("numeric",6),"character",
                                  rep("numeric",5),"character","character",
                                  "numeric","character","character",
                                  rep("numeric",3),"character","character",
                                  "numeric","character","character"
                                  ),
                    na.strings = "NULL", encoding = "latin1")

## import landcover classes
df.lc.class <- read.csv2("./data-raw/REQ/occupation.csv", encoding = "latin1")

REQUASUD.2 <- REQUASUD.1 %>%
left_join(df.lc.class, by = c("occup.sol" = "code"))

REQUASUD.2$sol_pheau[REQUASUD.2$sol_pheau == 0.5] <- NA
#print(qplot(REQUASUD$sol_pheau,geom = "boxplot"))

#-- Convert df to sf
req <- st_as_sf(REQUASUD.2,coords=c("sol_x","sol_y"), crs=31370)
}
if (op.data.file == "clean") {
    source("../walloonlucas_clean/R/clean_data_import.R")
}
}

Coordinates conservation for future representations

This step is just to keep the coordinates of the points of different projection systems in a dataframe for future use. Three systems were chosen: - the World Geodetic System 1984 (used in GPS), - the Belgian Lambert 72 and - the European ETRS89-extended system.

if (op.import.data) {
# LUCAS

#--2009
#----SCR
lucas_wal_2009_l72 <- st_transform(lucas_wal_2009, 31370)
lucas_wal_2009_3035 <- st_transform(lucas_wal_2009, 3035)

#----conservation
lucas_wal_2009$Long_wgs84 <- st_coordinates(lucas_wal_2009)[,1]
lucas_wal_2009$Lat_wgs84 <- st_coordinates(lucas_wal_2009)[,2]

lucas_wal_2009$X_l72 <- st_coordinates(lucas_wal_2009_l72)[,1]
lucas_wal_2009$Y_l72 <- st_coordinates(lucas_wal_2009_l72)[,2]

lucas_wal_2009$X_3035 <- st_coordinates(lucas_wal_2009_3035)[,1]
lucas_wal_2009$Y_3035 <- st_coordinates(lucas_wal_2009_3035)[,2]

#--2015
#----SCR
lucas_wal_2015_l72 <- st_transform(lucas_wal_2015, 31370)
lucas_wal_2015_3035 <- st_transform(lucas_wal_2015, 3035)

#----conservation
lucas_wal_2015$Long_wgs84 <- st_coordinates(lucas_wal_2015)[,1]
lucas_wal_2015$Lat_wgs84 <- st_coordinates(lucas_wal_2015)[,2]

lucas_wal_2015$X_l72 <- st_coordinates(lucas_wal_2015_l72)[,1]
lucas_wal_2015$Y_l72 <- st_coordinates(lucas_wal_2015_l72)[,2]

lucas_wal_2015$X_3035 <- st_coordinates(lucas_wal_2015_3035)[,1]
lucas_wal_2015$Y_3035 <- st_coordinates(lucas_wal_2015_3035)[,2]

#--2018
#----SCR
lucas_wal_2018_l72 <- st_transform(lucas_wal_2018, 31370)
lucas_wal_2018_3035 <- st_transform(lucas_wal_2018, 3035)

#----conservation
lucas_wal_2018$Long_wgs84 <- st_coordinates(lucas_wal_2018)[,1]
lucas_wal_2018$Lat_wgs84 <- st_coordinates(lucas_wal_2018)[,2]

lucas_wal_2018$X_l72 <- st_coordinates(lucas_wal_2018_l72)[,1]
lucas_wal_2018$Y_l72 <- st_coordinates(lucas_wal_2018_l72)[,2]

lucas_wal_2018$X_3035 <- st_coordinates(lucas_wal_2018_3035)[,1]
lucas_wal_2018$Y_3035 <- st_coordinates(lucas_wal_2018_3035)[,2]


#REQUASUD

#--SCR
req_wgs84 <- st_transform(req, 4326)
req_3035 <- st_transform(req, 3035)


#--conservation
req$Long_wgs84 <- st_coordinates(req_wgs84)[,1]
req$Lat_wgs84 <- st_coordinates(req_wgs84)[,2]

req$X_l72 <- st_coordinates(req)[,1]
req$Y_l72 <- st_coordinates(req)[,2]

req$X_3035 <- st_coordinates(req_3035)[,1]
req$Y_3035 <- st_coordinates(req_3035)[,2]
}
rm(list=c("REQUASUD.1","REQUASUD.2"))

Data processing and dataframes

All datafames useful for the comparison are finally produced and saved here.

if (op.import.data) {

# LUCAS
LUCAS2009 <- st_drop_geometry(lucas_wal_2009)
LUCAS2015 <- st_drop_geometry(lucas_wal_2015)
LUCAS2018 <- st_drop_geometry(lucas_wal_2018) %>%
dplyr::select(-"OC (20-30 cm)",
 -"CaCO3 (20-30 cm)",
 -"Ox_Al",
 -"Ox_Fe")

# REQUASUD
REQUASUD <- st_drop_geometry(req)
}

Saving and loading imported and pre-processed data

For lowering the computing time of loading data, after a first run of the previous code, we can save all objects in a .RData file. The next run can start here, loading the objects only by reading this saved file (name of the file is a rmarkdown parameter).

if (op.import.data) {
v.objects <- ls()[!(ls() %in% c("params"))]
save(list = v.objects,
     file = paste0("./data-raw/",op.data.file,".RData")
    )
} else {
load(paste0("./data-raw/",op.data.file,".RData"))
}

Data inventory, units and method check

LUCAS

Soil properties available in LUCAS and their measurement methods from LUCAS Topsoil Survey

The soils are sampled for a 0 to 20 cm increment depth (0-30 cm for the last version).

The data and methods available from LUCAS datasets are:

lucas.methods <- read.csv2(paste0(path_in,"/lucas-data-methods.csv"))

kable(lucas.methods, caption = "Soil properties methods in LUCAS Survey") %>%
  kable_styling(bootstrap_options = "striped", 
  "bordered", 
  full_width = T)
Soil properties methods in LUCAS Survey
Soil.property Method Unit
Particle size distribution : Clay, silt and sand contents Sieving and sedimentation method (FAO/WRB) ISO 11277.1998 (in 2009) - ISO 13320 (in 2015) %
Percentage of coarse fragments (Coarse) Sieving from the required pre-treatment for soil samples subjected to physicochemical analysis ISO 11464:2006 %
pH(H2O) Glass electrode in a 1:5 suspension of soil in H2O ISO 10390.1994 [ - ]
pH(CaCl2) Glass electrode in a 1:5 suspension of soil in CaCl2 ISO 10390.1994 [ - ]
Organic carbon content (OC) Dry combustion ISO 10694.1995 g/kg
Carbonate content (CaCO3) Volumetric method ISO 10693:1995 g/kg
total nitrogen content (N) Determination of total nitrogen — Modified Kjeldahl method ISO 11261.1995 g/kg
Phosphorus content (P) Spectrometric determination of phosphorus soluble in sodium hydrogen carbonate solution ISO 11263.1994 mg/kg
Extractable Potassium content (K) Atomic Absorption Spectrophotometer measurement (USDA, 2004) mg/kg
Cation Exchange Capacity (CEC) Determination of effective cation exchange capacity using barium chloride solution ISO 11260:1994 cmol(+)/kg
Electrical Conductivity (EC) ISO 11265 mS/m

In the next table, we present the 20 first lines of the main LUCAS dataset (2018).

head(LUCAS2018, n = 20) %>%
  datatable(
  options = list(
  pageLength = 5, autoWidth = TRUE, 
  searchHighlight = FALSE), 
  class = 'cell-border stripe', filter = 'bottom'
  )

REQUASUD

The soil properties available at lucas data level are mostly those measured at Requasud network level. However the units and methods may be different.

The methods defined in the Requasud data set for each soil property are quite variable. They are presented in a table in appendix.

Here we present a list of data available in the Requasud dataset:

  • pH in KCl (sol_phkcl) [-]
  • pH in H2O (sol_pheau) [-]
  • organic carbon content (sol_carbone) [% m.s]
  • phosphorus available (sol_phosphore) [mg/100 g TS]
  • potassium available (sol_potassium) [mg/100 g TS]
  • magnesium available (sol_magnesium) [mg/100 g TS]
  • calcium available (sol_calcium) [mg/100 g TS]
  • nitrogen content (sol_azote) [mg/g TS]
  • cation exchange capacity (sol_cec) [meq/100 g TS]
  • particle size distribution (“sol_argile”, “sol_limon” and “sol_sable” are for clay, silt and sand content) [%]
  • carbonate content (sol_carbonate) [%]

In the next table, we present the 20 last lines of the REQUASUD dataset. We removed the geodata.

tail(REQUASUD, n = 20) %>%
    dplyr::select(!c("Long_wgs84","Lat_wgs84","X_l72","Y_l72","X_3035","Y_3035")) %>%
  datatable(
  options = list(
  pageLength = 5, autoWidth = TRUE, 
  searchHighlight = FALSE), 
  class = 'cell-border stripe', filter = 'bottom'
  )

Only the soil properties data available in the two datasets (LUCAS and REQUASUD) are considered for the comparison in this report.

Data conversion

Organic carbon content

To fit the LUCAS OC unit, we multiplied by a factor of 10 to convert from % (Requasud) to [g/kg] (Lucas).

REQUASUD$conv_sol_carbone <- REQUASUD$sol_carbone * 10

Phosphorus content

To convert from mg/100g (Requasud) to mg/kg (Lucas), we will multiply the Requasud data by a 10 factor.

REQUASUD$conv_sol_phosphore <- REQUASUD$sol_phosphore * 10

Potassium content

The same convention as in the case of phosphorus is applied here.

REQUASUD$conv_sol_potassium <- REQUASUD$sol_potassium * 10

Labels change

Parameter names are in French in the Requasud data set. We decide to change the names to make them correspond to those of LUCAS.

data.REQUASUD <- REQUASUD %>%
                    dplyr::select(
                    year = annee, 
                    LC1_Desc=occup.sol,
                    Depth = sol_profondeur,
                    Clay= sol_argile,Silt=sol_limon,Sand=sol_sable,
                    pH_H2O = sol_pheau,
                    OC = conv_sol_carbone,
                    P = conv_sol_phosphore,
                    N = sol_azote,
                    K = conv_sol_potassium,
                    Long_wgs84 = Long_wgs84, Lat_wgs84 =Lat_wgs84,
                    X_l72 = X_l72,Y_l72 =Y_l72, 
                    X_3035 = X_3035,Y_3035 = Y_3035,
                    LC0_Desc=LC0_Desc) %>%
  tibble::rownames_to_column("id")

Details on particles size analyses

The comparison of the granulometry will be done with the data from the LUCAS 2009 and 2015 campaigns and all the data from the REQUASUD dataset. Some filters will be performed on the REQUASUD data, in order to harmonize them better.

#Requasud particles size data 
data.granuloREQUASUD <- f.requasud() %>%
  dplyr::filter(Sand < 100,
                Clay > 0.5)  # le filtre est different sur les vraies données

#Lucas particles size data taking into account years 2009 and 2015
data.granuloLUCAS <- LUCAS2009 %>% 
  dplyr::select(Point_ID = POINT_ID, Clay = clay,
                Silt = silt,Sand = sand, Long_wgs84 = Long_wgs84,
                Lat_wgs84 = Lat_wgs84,X_l72 = X_l72,Y_l72 = Y_l72, 
                X_3035 = X_3035,Y_3035 = Y_3035) %>%
  dplyr::bind_rows(LUCAS2015 %>% 
                     dplyr::select(Point_ID , Clay ,
                                   Silt,Sand, Long_wgs84 ,
                                   Lat_wgs84 ,X_l72 ,Y_l72, 
                                   X_3035,Y_3035))

Details on chemical data

Chemical data comes from the 2018 datasets, excluding the 3 types of land cover : Bareland, Woodland and Wetlands (this choice will be justify here after).

#Requasud chemical property data 
data.chemicalREQUASUD <- f.requasud(2018) %>%
  dplyr:: filter(LC0_Desc != "Woodland", LC0_Desc != "Wetlands" )

#Lucas chemical property data
data.chemicalLUCAS <- LUCAS2018 %>%
  dplyr:: filter(LC0_Desc != "Woodland", LC0_Desc != "Wetlands" )

Monitoring strategy

Sampling strategy and spatial distribution

Number of sites and densities

We provide in the following table the number of sites for both LUCAS and REQUASUD datasets over the study area and their associated densities. The area of Wallonia has been set to 16 845 km².

Note that the comparison of the particles size is done with the data from Lucas (2009 and 2015) and all the data from Requasud. While for the other soil properties, we considered the year 2018.

area <- 16845

# density and number of site table
density_table <- cbind.data.frame(Dataset = c("LUCAS.2015 (particule size)","REQUASUD (particule size)",
                                              "LUCAS.2018","REQUASUD.2018"),
  Nb.site = c(nrow(data.granuloLUCAS),nrow(data.granuloREQUASUD),
                                   nrow(data.chemicalLUCAS),nrow(data.chemicalREQUASUD)
                                   ),
                 Density = c(ceiling((area)/nrow(data.granuloLUCAS)),
                             ceiling((area)/nrow(data.granuloREQUASUD)),
                                     ceiling((area)/nrow(data.chemicalLUCAS)),
                             ceiling((area)/nrow(data.chemicalREQUASUD))
                             )
  )%>% 
  #PRENDRE LE NOMBRE DE POINT PAR KM2 A CHANGER
  kbl(caption = "Number of sites of LUCAS and REQUASUD datasets and their density
      (1 obs for X km²) over study area") %>%
  kable_styling(bootstrap_options = "striped", full_width = F)

density_table
Number of sites of LUCAS and REQUASUD datasets and their density (1 obs for X km²) over study area
Dataset Nb.site Density
LUCAS.2015 (particule size) 113 150
REQUASUD (particule size) 743 23
LUCAS.2018 40 422
REQUASUD.2018 6895 3

Spatial distribution representation

f.wl.spatial(wallonie,
             data.granuloREQUASUD,
             data.granuloLUCAS,
             "Long_wgs84",
             "Lat_wgs84" ,
             coord.crs = 4326,
             data.used = "Data_Particle.size"
)

f.wl.spatial(wallonie,
             data.chemicalREQUASUD,
             data.chemicalLUCAS,
             "Long_wgs84",
             "Lat_wgs84" ,
             coord.crs = 4326,
             data.used = "Data_Chemical.property"
)

Interactive maps (LUCAS only)

m_lucas <- mapview(lucas.sf.l72, col.regions = viridis(3)[1], cex = 2, lwd = 0.5)
  • Individual maps of LUCAS and SIMS spatial points maps.
m_lucas

Sampling point density

We can calculate sampling point density in France for LUCAS in a radius of 10km.

## densité points
w <- as.owin(wallonie) ## creating a window
w.km <- rescale.owin(w, s = 1000) ## change the units in kilometers

## lucas point density
s.lucas  <- as.ppp(lucas_wal_2018_l72) ## points in projected system
marks(s.lucas) <- NULL
s.lucas <- rescale.ppp(s.lucas, s = 1000) # convert units in km
Window(s.lucas) <- w.km ## making the same window

density.lucas <- density.ppp(s.lucas, sigma=10) # calculating point density for LUCAS with 16 km bandwidth
plot(density.lucas, main="Point density of LUCAS samples", las=1)
contour(density.lucas, add=TRUE)

## Requasud point density
s.req  <- as.ppp(req) ## points in projected system
marks(s.req) <- NULL
s.req <- rescale.ppp(s.req, s = 1000) # convert units in km
Window(s.req) <- w.km ## making the same window

density.req <- density.ppp(s.req, sigma=10) # calculating point density for LUCAS with 16 km bandwidth
plot(density.req, main="Point density of REQ samples", las=1)
contour(density.req, add=TRUE)

Main observation, and consequences for the whole following report

We observe that the spatial distribution of the data are not the same in the two data sets.

For the particule size analyses (PSA), the LUCAS points are equally distributed in the whole Walloon region while the REQUASUD points are concentrated around the province of Brabant wallon, whose main laboratory (La Hulpe) is the one in charge with this analysis in the REQUASUD network. This region is also know as the main one for intensive arable farming, in the loamy belt of Belgium.

For the chemical properties, we face also with two contrasted situations : the LUCAS datasets hold a limited number of sampling points, but equally distributed in the Walloon region. On the other side, the REQUASUD datasets hold a huge number of point, but with a high concentration of points in the intesive arable cropping systems of four provinces : East of Hainaut, North of Namur, Brabant wallon and west of Liège. We see also some spots in the grassland farming systems in Luxembourg province and East of the Liège province.

This contrasted situation find its origin in the two contrasting “goals” of these data sets : on the one hand, LUCAS aims at monitor the soil of Europe in general, and in Wallonia for this report ; on the other hand, REQUASUD is a service proposed to farmers, and is done in the framework on some administrative demands or on a voluntary basis.

This contrasted situation has logical consequence on the data analyses that will be done here after. It will be an important source of variablity between the datasets in following comparisons.

Land cover distribution

Full datasets

LUCAS2018$LandCover <- as.factor(LUCAS2018$LC0_Desc)
req$LandCover <- as.factor(req$LC0_Desc)

# Extracting the sites per land cover classes 
    myTabLC <- dplyr::left_join(
    as.data.frame(table(LUCAS2018$LandCover)),
    as.data.frame(table(req$LandCover)),
    by=c("Var1"="Var1")
    )

colnames(myTabLC) <- c("LandCover", "nb.LUCAS2018", "nb.REQUASUD.full")

# Computing percentage
myTabLC$nb.LUCAS2018.perc <- round((myTabLC$nb.LUCAS2018/nrow(LUCAS2018))*100,2)
myTabLC$nb.REQUASUD.2018.perc <- round((myTabLC$nb.REQUASUD.full/nrow(req))*100,2)

# Creating the table
SitesLU <- myTabLC %>%
  kbl(caption = "Number of sites by Land Cover Classes") %>%
  kable_styling(bootstrap_options = "striped",
                full_width = F)

SitesLU
Number of sites by Land Cover Classes
LandCover nb.LUCAS2018 nb.REQUASUD.full nb.LUCAS2018.perc nb.REQUASUD.2018.perc
Bareland 2 13 3.64 0.01
Cropland 33 93388 60.00 84.24
Grassland 5 17415 9.09 15.71
Wetlands 1 NA 1.82 NA
Woodland 14 47 25.45 0.04

Filtered datasets

As REQUASUD is a service mainly proposed to Walloon farmers, two main land covers are dominants : cropland and grassland. Bareland is an uncommon land cover and is considered, in this report, as a farming plot (crop or grassland) that is temporary not covered. For trying to have as comparable as possible datasets, we filtered the data on these three land covers.

For chemical data at least, the REQUASUD dataset is also filtered based on the year (only 2018) and on the sampling depth (15 cm and 25 cm), close to the LUCAS sampling protocol.

This is why the numbers of sampling points in the second table are lower for the REQUASUD column.

data.chemicalLUCAS$LandCover <- as.factor(data.chemicalLUCAS$LC0_Desc)
data.chemicalREQUASUD$LandCover <- as.factor(data.chemicalREQUASUD$LC0_Desc)

# Extracting the sites per land cover classes
    myTabLC <- dplyr::left_join(
    as.data.frame(table(data.chemicalLUCAS$LandCover)),
    as.data.frame(table(data.chemicalREQUASUD$LandCover)),by=c("Var1"="Var1")
    )

colnames(myTabLC) <- c("LandCover", "nb.LUCAS2018", "nb.REQUASUD2018)")

# Computing percentage
myTabLC$nb.LUCAS2018.per <- ceiling((myTabLC$nb.LUCAS2018/nrow(data.chemicalLUCAS))*100)
myTabLC$nb.REQUASUD.2018.per <- ceiling((myTabLC$nb.REQUASUD2018/nrow(data.chemicalREQUASUD))*100)

# Creating the table
SitesLU <- myTabLC %>%
  kbl(caption = "Number of sites by Land Cover Classes") %>%
  kable_styling(bootstrap_options = "striped",
                full_width = F)

SitesLU
Number of sites by Land Cover Classes
LandCover nb.LUCAS2018 nb.REQUASUD2018) nb.LUCAS2018.per nb.REQUASUD.2018.per
Bareland 2 NA 5 NA
Cropland 33 5527 83 81
Grassland 5 1368 13 20

We provide also the graphical representation.

# Vertical format table for graphical representation using ggplot2
TableCover  <- 
  cbind.data.frame(
    LandCover = c(as.factor(myTabLC$LandCover),
                  as.factor(myTabLC$LandCover)),
    Dataset = as.factor(c(rep("REQUASUD",3),
                          rep("LUCAS",3))),
    pc.site =c(myTabLC[,5],
               myTabLC[,4])
    )

# Graphical representation
SitesLU.graph <- ggplot(data=TableCover,
                        aes(x=LandCover, 
                            y=pc.site,
                            fill=Dataset 
                            )
                        ) +
   geom_bar(stat="identity",
            position=position_dodge()
            ) +
   scale_fill_viridis(direction=-1,discrete = TRUE) +
  ylab("Percentage of sites") + 
  theme_minimal()

SitesLU.graph
Repartition of LUCAS and REQUASUD sites according to the land cover classification

Repartition of LUCAS and REQUASUD sites according to the land cover classification

Dominant soil classes

The distribution of dominant soil classes by LUCAS and SIMS can be compared to the frequency of those classes in Belgium.

Dominant soil classes correspond to the FAO dominant STU displayed in the European Soil Database on a grid of 1 km x 1 km from ESDAC.

####    IMPORTANT : THE FIRST BLOCK OF CODE NEED TO BE RUN ONCE TO FIRST SAVE THE RASTER OF DOM SOIL OF THE COUNTRY   #########


## Import European dominant soil classes and reproject the data
# fao.soil <- terra::rast("data/FAO_dominant_soil_Europe.tif")
# crs(fao.soil) <- "epsg:3035" ## projection ETRS89
# names(fao.soil) <- "dominant_soil"
# fao.soil <- terra::project(fao.soil, "epsg:4326")  ## new projection in WGS84

## European countries shapefile
# eu <- terra::vect("data/Europe_countries.shp")
# country <- subset(eu, eu$CNTRYNAME == "France")

## Raster of dominant soil classes in France : cutting the european raster from ESDAC and saving raster
# fao.ext <- crop(fao.soil, country)
# fao.fr <- raster::mask(fao.ext, country)
# writeRaster(fao.fr, "data/FAO_soil_dom.tif")

# Run the previous rows and then just call the saved raster
fao.be <- terra::rast("data/FAO_soil_dom.tif")
f.map.raster.fact(fao.be,
                  variable = "FAO85FU",
                  title = "Main soil classes of Wallonia (FAO)",
                  unit = "FAO soil class"
                  )

## frequency of soil classes in Wallonia from FAO : REFERENCE VALUES
table.freq.class <- terra::freq(fao.be) %>%
  mutate(Total = sum(count)) %>%
  rowwise() %>%
  mutate(Freq.p = round(count/Total*100, digits = 2)) %>%
  dplyr::rename(., "Dominant_soil" = value)


## Extract soil classes at the sampling points
req_wgs84 <- st_transform(req, 4326)
req.terra <- terra::vect(req_wgs84) ## REQ points at the terra format
req.fao <- as.data.frame(terra::extract(fao.be, req.terra)) ## extracting the soil classes values at REQ locations

lucas.terra <- terra::vect(lucas_wal_2018) ## LUCAS points at the terra format
lucas.fao <- as.data.frame(terra::extract(fao.be, lucas.terra)) 

## frequency of soil classes REQ
req.dom.soil <- as.data.frame(req) %>%
  bind_cols(req.fao) %>%
  dplyr::rename("Dominant_soil" = FAO85FU) %>%
  group_by(Dominant_soil) %>%
  dplyr::summarise(., count.req = length(Dominant_soil),
                   freq.req = (length(Dominant_soil)/nrow(req.fao))*100)

## frequency of soil classes LUCAS
lucas.dom.soil <- as.data.frame(lucas_wal_2018) %>%
  bind_cols(lucas.fao) %>%
  dplyr::rename("Dominant_soil" = FAO85FU) %>%
  group_by(Dominant_soil) %>%
  dplyr::summarise(., count.lucas = length(Dominant_soil),
                   freq.lucas = (length(Dominant_soil)/nrow(lucas.fao))*100)


## All frequencies LUCAS and REQ + ref
dom.soil.freq <- table.freq.class %>%
  left_join(req.dom.soil, by = "Dominant_soil") %>%
  left_join(lucas.dom.soil, by = "Dominant_soil") %>%
  dplyr::rename(., "Frequency" = Freq.p)
dom.soil.freq %>%
  datatable(options = list(pageLength = 5, autoWidth = TRUE, searchHighlight = FALSE), class = 'cell-border stripe', filter = 'bottom') %>%
  formatRound(columns=c('freq.req', 'freq.lucas'), digits=3)

Observation

The two major soil units of Wallonia are Orthic Luvisols (Lo, approx. 70.5%), mainly in cropland, and Dystric Cambisols (Bd, approx. 19.5%), mainly in grassland regions.

Comparison

Graphical representation of the difference between distribution of dominant soil classes in Beglium/Wallonia (i.e. Reference frequency) and the soil classes frequencies in LUCAS and REQUASUD data sets.

dom.soil <- dom.soil.freq %>%
  filter(! Dominant_soil %in% c("555","444","333","222", "111","")) %>%
  filter(Frequency > 0.1) %>% ## filter classes with frequency is above 0.1 % in the FAO 
  rowwise() %>%
  mutate(lucas = freq.lucas - Frequency,
         requasud = freq.req - Frequency) %>% ## difference between soil classes frequencies
  pivot_longer(., cols = c("lucas","requasud"), values_to = "Diff.frequency", names_to = "Program") %>%
  unique() %>%
  arrange(Program, Diff.frequency)
dom.soil$Dominant_soil <- factor(dom.soil$Dominant_soil, levels = unique(dom.soil$Dominant_soil))


dom.soil %>%
  group_by(Program) %>%
  dplyr::summarise(median = median(Diff.frequency, na.rm = T),
            min = min(Diff.frequency, na.rm = T),
            max = max(Diff.frequency, na.rm = T)) %>%
  kbl(caption = "Statistics of differences between global frequencies of dominant soil classes and datasets", digits = 3) %>%
  kable_styling(bootstrap_options = "striped", full_width = F)
Statistics of differences between global frequencies of dominant soil classes and datasets
Program median min max
lucas 2.060 -3.936 8.867
requasud -0.491 -20.761 26.155
dom.soil %>%
  ggplot(., aes(x = Dominant_soil, y = Diff.frequency, color = Program)) +
  geom_point(stat = "identity", size = 3) +
  geom_segment( aes(x=Dominant_soil, xend=Dominant_soil, y=0, yend=Diff.frequency), color="grey") +
  coord_flip() +
  geom_vline(xintercept = 0) +
  theme_bw() +
  theme(legend.position = "none")+
  facet_wrap(~ Program) +
  labs(title = "Difference between frequencies of dominant soil classes",
       y = "Reference frequency - Monitoring frequency",
       x = "Dominant soil classes")

High negative of positive values implies either an under- or over-representation of dominant soil classes.

Observations

Statistically, the Orthic luvisol (cropland) is overrepresented, and the Dystric cambisol is underrepresented in the two datasets, but more clearly in the REQUASUD data set. It is linked to the goal of the requasud service, more popular in the cropland area of Wallonia.

Exploratory analysis: summary statistics and land cover comparison

The aim of this part is to compare the datasets regarding their distribution, by providing summary statistics and comparative visual representation of the empirical distribution. Furthermore, we propose an additional application based on comparing the distribution of spatial sites regarding the land cover categories where the sites are located. Hereafter, we estimate all of the summary statistical indicators that will be necessary in the following sections.

Particles size summary statistics

The threshold values for particle size fraction analysis were different between LUCAS and REQUASUD. In Belgium, we traditionally use the same treshold as in France (INRAe, RMSQ / SIMS).

part.frac <- data.frame("Fraction" = c("Sand", "Silt", "Clay"),
                        "SIMS" = c("2 - 0.05 mm", "0.05 - 0.002 mm", "< 0.002 mm"),
                        "LUCAS" = c("2 - 0.063 mm", "0.065 - 0.002 mm", "< 0.002 mm"))
kable(part.frac, caption = "Soil properties methods in LUCAS Survey") %>%
  kable_styling(bootstrap_options = "striped", "bordered", full_width = T)
Soil properties methods in LUCAS Survey
Fraction SIMS LUCAS
Sand 2 - 0.05 mm 2 - 0.063 mm
Silt 0.05 - 0.002 mm 0.065 - 0.002 mm
Clay < 0.002 mm < 0.002 mm

We select clay, silt and sand contents from the LUCAS and REQUASUD datasets to compute summary statistics.

summary_granuloLUCAS <-f.summary(data.granuloLUCAS,param.select = c("Clay","Silt","Sand"))

summary_granuloREQUASUD <- f.summary(data.granuloREQUASUD,param.select = c("Clay","Silt","Sand"))

The reasons of the choice

As the Lucas 2018 particle size data is not available, we will use the particle size data from other available years for comparison.

For the Requasud data, we noted that the particle size data started to be measured recently (see figure below). Since particle size evolves slowly on the human time scale, we used all available data at the Requasud level.

knitr::include_graphics("./data-raw/evolution_granulo_anim.gif")
Animated map showing the clay content mesurement from 2008 to 2021
Animated map showing the clay content mesurement from 2008 to 2021

Central tendency

We estimate central tendency of LUCAS and REQUASUD dataset

f.wl.stat.table(summary_granuloLUCAS,summary_granuloREQUASUD,'central')
Central tendency for LUCAS and REQUASUD datasets
Mean.LUCAS Mean.REQUASUD Median.LUCAS Median.REQUASUD
Clay 19.8 22.3 20.0 22.8
Silt 61.3 65.6 63.5 66.4
Sand 14.7 12.1 12.0 10.9

Measures of spread

In this section, we estimate the range, the quantiles (25% and 75%), the standard deviation, the skewness and kurtosis. These indicators will provide information to qualify the distribution of a soil property either spread or not.

f.wl.stat.table(summary_granuloLUCAS,summary_granuloREQUASUD,'spread')
Spread indicators for LUCAS and REQUASUD datasets
Range.LUCAS Range.REQUASUD Quantile25.LUCAS Quantile25.REQUASUD Quantile75.LUCAS Quantile75.REQUASUD SD.LUCAS SD.REQUASUD Skewness.LUCAS Skewness.REQUASUD Kurtosis.LUCAS Kurtosis.REQUASUD
Clay 37 38.2 17.0 19.5 22.8 25.4 6.4 4.7 -0.7 -0.1 2.3 1.5
Silt 83 78.4 57.2 62.0 71.8 70.0 16.7 7.6 -2.0 -1.5 5.2 7.3
Sand 67 84.7 6.5 8.5 20.0 14.4 12.0 6.6 1.9 3.9 5.3 27.9

Comparison graphical representations

Texture triangle

LUCAS
df.granulo.awc.lucas <- f.soil.awc.ptf(data.granuloLUCAS)
g.tt.lucas <- f.soil.tritext(df.granulo.awc.lucas)


TT.plot(
  class.sys = "BE.TT",
  ## tri.data = rand.text,
  tri.data = na.omit(df.granulo.awc.lucas),
  ## z.name = "OC",
  css.names = c("Clay", "Silt", "Sand"),
  main = "Soil texture triangle (BE) & data lucas",
  tri.sum.tst = FALSE
)

  geo <- TT.geo.get()


  ## Avec densité de probabilité
  kde.res <- TT.kde2d(
    geo = geo,
    tri.data = na.omit(df.granulo.awc.lucas),
    css.names = c("Clay", "Silt", "Sand"),
    tri.sum.tst = FALSE
  ) 

  TT.plot(
    class.sys = "BE.TT",
    ## tri.data = rand.text,
    tri.data = na.omit(df.granulo.awc.lucas),
    ## geo = geo,
    grid.show = FALSE,
    col = "gray",
    ## z.name = "OC",
    css.names = c("Clay", "Silt", "Sand"),
    main = "Soil texture triangle (BE) & data lucas",
    tri.sum.tst = FALSE
  )
  TT.contour(
    x = kde.res,
    geo = geo,
    main = "Probability density estimate of the texture data",
    lwd = 1,
    add = TRUE, # <<-- important
    col = "red"
  ) 

g.tt.lucas %>%
    ggtern::ggsave("./fig/tt_lucas.png", . ,
           width=9, height=7, units="in"
           )

print(g.tt.lucas)

REQUASUD
df.granulo.awc.requasud <- f.soil.awc.ptf(data.granuloREQUASUD)
g.tt.requasud <- f.soil.tritext(df.granulo.awc.requasud,
                 colored.by = "AWC")


  ## Test the data (?? text.tol not working in TT.plot !!!
  ##
  ## TT.data.test(df.granulo.awc.requasud,
  ##         css.names = c("Clay", "Silt", "Sand"),
  ##         text.tol = 0.01
  ##         )

TT.plot(
  class.sys = "BE.TT",
  ## tri.data = rand.text,
  tri.data = df.granulo.awc.requasud,
  ## z.name = "OC",
  css.names = c("Clay", "Silt", "Sand"),
  main = "Soil texture triangle (BE) & data requasud",
  tri.sum.tst = FALSE
)

  kde.res <- TT.kde2d(
    geo = geo,
    tri.data = df.granulo.awc.requasud,
    css.names = c("Clay", "Silt", "Sand"),
    tri.sum.tst = FALSE
  ) 

  TT.plot(
    class.sys = "BE.TT",
    ## tri.data = rand.text,
    tri.data = df.granulo.awc.requasud,
    ## geo = geo,
    grid.show = FALSE,
    col = "gray",
    ## z.name = "OC",
    main = "Soil texture triangle (BE) & data requasud",
    css.names = c("Clay", "Silt", "Sand"),
    tri.sum.tst = FALSE
  )
  TT.contour(
    x = kde.res,
    geo = geo,
    main = "Probability density estimate of the texture data",
    lwd = 1,
    add = TRUE, # <<-- important
    col = "red"
  ) 

  g.tt.requasud %>%
    ggtern::ggsave("./fig/tt_requasud.png", . ,
           width=9, height=7, units="in"
           )

print(g.tt.requasud)

Comparison of the soil classes

Based on the Belgian Soil Texture Triangle
  requasud.texture.class <- df.granulo.awc.requasud
  requasud.texture.class$texture.class <- TT.points.in.classes( 
    tri.data    = df.granulo.awc.requasud, 
    class.sys   = "BE.TT", 
    PiC.type    = "t",
    css.names = c("Clay", "Silt", "Sand"),
    tri.sum.tst = FALSE
  )
  requasud.texture.class$dataset <- "requasud"

  lucas.texture.class <- na.omit(df.granulo.awc.lucas)
  lucas.texture.class$texture.class <- TT.points.in.classes( 
    tri.data    = na.omit(df.granulo.awc.lucas), 
    class.sys   = "BE.TT", 
    PiC.type    = "t",
    css.names = c("Clay", "Silt", "Sand"),
    tri.sum.tst = FALSE
  )
  lucas.texture.class$dataset <- "lucas"

  ## Graphical representation
  df.texture.classes <- rbind(requasud.texture.class[c("texture.class","dataset")],lucas.texture.class[c("texture.class","dataset")]) %>%
    group_by(texture.class,dataset) %>%
    summarise(n = n(),.groups="keep") %>%
    ungroup(texture.class) %>%
    mutate(freq = n / sum(n))
  SitesTT.graph <- ggplot(data = df.texture.classes,
              aes(x = texture.class,
                  y = freq,
                  fill = dataset
                  )
              ) +
    geom_bar(stat = "identity",
         position = position_dodge()
         ) +
    viridis::scale_color_viridis() +
    ylab("Percentage of sites for each sol texture classes") + 
    theme_minimal() + 
    theme(axis.text.x = element_text(angle = 90, vjust = 0.5, hjust=1))
  
  print(SitesTT.graph)

Observation

The A class (Loam) is more frequently present in the REQUASUD dataset while L (Sandy loam) is more frequent in the LUCAS dataset. This is a consequence of the goals of the two contrasted sources of the data, as explained in the beginning of the document.

Based on the USDA soil text. tri., for information / comparison with other countries)
  TT.plot(
    class.sys = "USDA.TT",
    main = "Soil texture triangle (USDA)"
  )

  requasud.texture.class$texture.class <- TT.points.in.classes( 
    tri.data    = df.granulo.awc.requasud, 
    class.sys   = "USDA.TT", 
    PiC.type    = "t",
    css.names = c("Clay", "Silt", "Sand"),
    tri.sum.tst = FALSE
  )
  requasud.texture.class$dataset <- "requasud"

  lucas.texture.class$texture.class <- TT.points.in.classes( 
    tri.data    = na.omit(df.granulo.awc.lucas), 
    class.sys   = "USDA.TT", 
    PiC.type    = "t",
    css.names = c("Clay", "Silt", "Sand"),
    tri.sum.tst = FALSE
  )
  lucas.texture.class$dataset <- "lucas"

  ## Graphical representation
  df.texture.classes <- rbind(requasud.texture.class[c("texture.class","dataset")],lucas.texture.class[c("texture.class","dataset")]) %>%
    group_by(texture.class,dataset) %>%
    summarise(n = n(),.groups="keep") %>%
    ungroup(texture.class) %>%
    mutate(freq = n / sum(n))
  SitesTT.graph <- ggplot(data = df.texture.classes,
              aes(x = texture.class,
                  y = freq,
                  fill = dataset
                  )
              ) +
    geom_bar(stat = "identity",
         position = position_dodge()
         ) +
    viridis::scale_color_viridis() +
    ylab("Percentage of sites for each sol texture classes") + 
    theme_minimal() + 
    theme(axis.text.x = element_text(angle = 90, vjust = 0.5, hjust=1))

  SitesTT.graph

###### no comment

Using boxplots

We present here a comparison of the distributions of the particles size using a boxplot.

data.granuloREQUASUD %>% 
  dplyr::select(Clay, Silt, Sand) %>%
  pivot_longer(cols = Clay:Sand,
               names_to = "property",
               values_to = "value") %>%
  mutate(Dataset = "REQUASUD") %>%
  bind_rows(
    data.granuloLUCAS %>% 
      dplyr::select(Clay, Silt, Sand) %>% 
      pivot_longer(cols = Clay:Sand,
                   names_to = "property",
                   values_to = "value") %>%
      mutate(Dataset = "LUCAS") 
  ) %>%
  mutate(property = factor(property,levels = c("Clay","Silt","Sand"))) %>%
  ggplot(
    aes(x = Dataset ,
        y = value,
        fill = Dataset)) + 
  geom_boxplot() +
  labs(title="Plot of particles size by type of soil monitoring network",x="Soil monitoring", y = "Soil porperty") +
  scale_fill_viridis(direction=-1,discrete = TRUE) +
  theme_classic() +
  facet_wrap( ~ property ,
              scales = "free_y"
  )

Using histograms

Here we can visualize LUCAS and REQUASUD datasets for the following soil properties:

Clay content

We compare the distributions of clay content here between REQUASUD and LUCAS:

f.wl.distr(data.granuloREQUASUD,
           data.granuloLUCAS,
           wl.param = 'Clay',
           wl.bin = 4,
           wl.xlab = "Clay content [%]"
)

Silt content

We compare the distributions of silt content here between REQUASUD and LUCAS:

f.wl.distr(data.granuloREQUASUD,
           data.granuloLUCAS,
           wl.param = 'Silt',
           wl.bin = 6,
           wl.xlab = "Silt content [%]"
)

Sand content

We compare the distributions of sand content here between REQUASUD and LUCAS:

f.wl.distr(data.granuloREQUASUD,
           data.granuloLUCAS,
           wl.param = 'Sand',
           wl.bin = 6,
           wl.xlab = "Sand content [%]"
)

Statistical inferences

We provide as well the KS test that evaluates if LUCAS and REQUASUD represent the same population. The KS is provided with a p-value.

KS.df <- cbind.data.frame(soil.property = c("Clay","Silt","Sand"),
                 KS.test = round(c(
                   as.numeric(ks.test(x=data.granuloREQUASUD$Clay,
                                      y = data.granuloLUCAS$Clay)$statistic),
                   
                   as.numeric(ks.test(x =data.granuloREQUASUD$Silt,
                                     y=data.granuloLUCAS$Silt)["statistic"]),
                   
                   as.numeric(ks.test(x = data.granuloREQUASUD$Sand,
                                      y=data.granuloLUCAS$Sand)["statistic"])),3),
                 
                 p.value = round(c(
                   as.numeric(ks.test(x =data.granuloREQUASUD$Clay,
                                      y = data.granuloLUCAS$Clay)["p.value"]),
                   
                   as.numeric(ks.test(x = data.granuloREQUASUD$Silt,
                                      y = data.granuloLUCAS$Silt)["p.value"]),
                   
                   as.numeric(ks.test(x = data.granuloREQUASUD$Sand,
                                     y = data.granuloLUCAS$Sand)["p.value"])),10))

KS.df %>% 
  kbl(caption = "Kolmogorov-Smirnov (KS) test values and p-value") %>%
  kable_styling(bootstrap_options = "striped", full_width = F)
Kolmogorov-Smirnov (KS) test values and p-value
soil.property KS.test p.value
Clay 0.354 0.0000001
Silt 0.236 0.0010996
Sand 0.287 0.0000307

Comment

The two datasets are different. The clay content shows the highest differences and is higher in the REQUASUD samples.

Chemical property summary statistic

Problem for comparison : few lucas point, lots of requasud points.

  f.wl.property(wallonie,
                 f.requasud(2018),
                 LUCAS2018,
                 property = 'OC',
                 "Long_wgs84",
                 "Lat_wgs84",
                 coord.crs = 4326,
                 title = 'Organic\ncarbon [g/Kg]'
                 )

An idea was to create buffer of X km around each lucas point and compute statistics (mainly “mean”) of the requasud point inside this buffer. As a first approximation, we took a buffer of 5 km.

  req.sf.l72 <- st_as_sf(f.requasud(2018),
                     coords=c(
                         "X_l72",
                         "Y_l72"),
                     crs= 31370
                     )
  lucas.sf.l72 <- st_as_sf(LUCAS2018,
                       coords=c(
                         "X_l72",
                         "Y_l72"),
                       crs= 31370
                       )
  ## Create a buffer
  prem_buffer_5k <- st_buffer(select(lucas.sf.l72,POINTID), 5000)
  ## Plot the map with the buffers
  plot(st_geometry(wallonie))
  plot(st_geometry(req.sf.l72), col = 'red', add = T)
  plot(st_geometry(prem_buffer_5k), add = T, lwd = 2) 

Number of requasud points for each lucas point :

  nreq_per_lucas <- req.sf.l72 %>%
    st_join(prem_buffer_5k, left = FALSE) %>%
    count(POINTID) %>% st_drop_geometry() %>% arrange(-n)

  nreq_per_lucas %>%
  datatable()

Distribution of the number of requasud points for each lucas point.

g <- (ggplot(nreq_per_lucas,aes(x=n)) 
+ geom_boxplot()
+ geom_jitter(aes(y=0))
+ coord_flip()
)
print(g)

These parameters are compared for the year 2018 from the two datasets.

The reasons of the choice

Soil chemical properties are dependent on the land use. In order to better understand and analyze the chemical property data made available, especially those of REQUASUD, we performed a 5Km buffer around each LUCAS point. These have made it possible to select REAQUSUD points for a finer observation of those according to land use.

Comparison of the OC between requasud and lucas.

  meanCreq_per_lucas <- req.sf.l72 %>%
    st_join(prem_buffer_5k, left = FALSE) %>%
    group_by(POINTID) %>%
    summarise(meanOCreq = mean(OC),
              sdOCreq = sd(OC),
              n = length(POINTID)
              ) %>%
    left_join(select(as_tibble(lucas.sf.l72),c(POINTID,OC,LC0_Desc))) %>%
    mutate(err = abs(1 - (OC / meanOCreq)))

  property <- 'OC'
  my.limit <- +Inf
  my.color <- "LC0_Desc"
  g <- meanCreq_per_lucas %>%
    dplyr::filter(OC < my.limit, meanOCreq < my.limit) %>%
    ggplot(aes(x = OC, y = meanOCreq)) +
    geom_abline(intercept = 0, slope = 1, linewidth = 1, color = "darkgrey") +
    geom_pointrange(aes(
      ymin = (meanOCreq - sdOCreq),
      ymax = (meanOCreq + sdOCreq),
      color = get(my.color)),
      size = 0.1) +
    geom_point(
      aes(color = .data[[my.color]]),
      size = 2
    ) +
    coord_cartesian(
      xlim = c(0, min(my.limit,100)),
      ylim = c(0, min(my.limit,100))
    ) +
    labs(title = "Comparaison of continuous data",
         subtitle = "Buffered Requasud data around Lucas",
         x = "OC (LUCAS)",
         y = "OC (Requasud) mean +/- sd"
         ) +
    scale_color_manual(
      values = c(viridis::viridis(5)[5:1]),
      name = "Land Cover",
      drop = FALSE
    ) +
    theme_classic()
  print(g)

Map of the relative differences between datasets for OC based on land use.

  map.data <- lucas.sf.l72 %>%
    left_join(select(as_tibble(meanCreq_per_lucas),
                     c(POINTID,err,LC0_Desc))
              )

  map.diffrel.lucas <- map.data %>%
    ggplot() +
    geom_sf(data = wallonie, fill = NA) +
    geom_sf(aes(color = LC0_Desc, size = err)) + 
    scale_color_manual(
      values = c(viridis::viridis(5)[5:1]),
      name = "Land Cover",
      drop = FALSE
    ) +
    theme_minimal()
  print(map.diffrel.lucas)

Based on this observation, a filter is applied to the data to take only croplands and grasslands at the level of the two data sets.

To do this we have selected pH(H2O), organic carbon, Phosphorus, Nitrogen and Potassium contents from the LUCAS and REQUASUD datasets after the filter, to compute summary statistics.

#Summary statistic
summary_statLUCAS <- f.summary(data.chemicalLUCAS,param.select =c("pH_H2O","OC","P","N","K"))

summary_statREQUASUD <- f.summary(data.chemicalREQUASUD,param.select =c("pH_H2O","OC","P","N","K"))

Central tendency

We estimate central tendency of LUCAS and REQUASUD dataset

f.wl.stat.table(summary_statLUCAS,summary_statREQUASUD,'central')
Central tendency for LUCAS and REQUASUD datasets
Mean.LUCAS Mean.REQUASUD Median.LUCAS Median.REQUASUD
pH_H2O 6.9 7.2 7.0 7.2
OC 19.3 18.7 14.0 13.0
P 66.3 72.9 61.2 66.6
N 2.0 1.8 1.6 1.3
K 206.0 182.0 184.7 171.1

Measures of spread

We do it in the same way for these soil properties.

f.wl.stat.table(summary_statLUCAS,summary_statREQUASUD,'spread')
Spread indicators for LUCAS and REQUASUD datasets
Range.LUCAS Range.REQUASUD Quantile25.LUCAS Quantile25.REQUASUD Quantile75.LUCAS Quantile75.REQUASUD SD.LUCAS SD.REQUASUD Skewness.LUCAS Skewness.REQUASUD Kurtosis.LUCAS Kurtosis.REQUASUD
pH_H2O 2.5 3.4 6.6 6.8 7.4 7.6 0.6 0.6 -0.5 -0.5 -0.6 -0.1
OC 69.5 90.0 12.4 11.3 20.8 20.0 13.0 12.2 2.6 1.8 7.8 2.4
P 148.8 523.0 45.2 44.0 83.4 94.5 31.2 39.6 1.0 1.3 1.4 4.4
N 4.5 6.5 1.5 1.2 2.2 1.9 1.1 1.0 2.0 1.6 2.9 1.4
K 421.7 914.6 132.6 133.0 264.6 215.0 98.2 75.3 0.7 1.8 -0.2 8.4

Comparison graphical representations

Using boxplots

We present here a comparison of the distributions of the soil properties using a boxplot.

data.chemicalREQUASUD %>% 
  dplyr::select(pH_H2O,OC, N, P, K) %>%
  pivot_longer(cols = pH_H2O:K,
               names_to = "property",
               values_to = "value") %>%
  mutate(Dataset = "REQUASUD") %>%
  bind_rows(
    data.chemicalLUCAS %>% 
      dplyr::select(pH_H2O,OC, N, P, K)%>% 
      pivot_longer(cols = pH_H2O:K,
                   names_to = "property",
                   values_to = "value") %>%
      mutate(Dataset = "LUCAS")
  ) %>%
  mutate(property = factor(property,levels = c("pH_H2O","OC","P","N","K"))) %>%
  ggplot(aes(x = Dataset ,
             y = value,
             fill = Dataset)
  ) + 
  
  geom_boxplot()+
  labs(title="Plot of soil property by type of soil monitoring network",x="Soil monitoring", y = "Soil porperty")+
  scale_fill_viridis(direction=-1,discrete = TRUE) +
  theme_classic()+
  facet_wrap( ~ property ,
              scales = "free_y"
  )

Using histograms

Here we can visualize LUCAS and REQUASUD datasets for the following soil properties:

pH(H20)

We compare the distributions of pH content here between REQUASUD and LUCAS:

f.wl.distr(data.chemicalREQUASUD,
           data.chemicalLUCAS,
           wl.param = 'pH_H2O',
           wl.bin = 0.2,
           wl.xlab = "pH_in_H2O [-]"
)

Organic carbon content

We compare the distributions of organic carbon content here between REQUASUD and LUCAS:

f.wl.distr(data.chemicalREQUASUD,
           data.chemicalLUCAS,
           wl.param = 'OC',
           wl.bin = 10,
           wl.xlab = "Organique carbone [g/Kg]"
)

Nitrogen content

We compare the distributions of nitrogen content here between REQUASUD and LUCAS:

f.wl.distr(data.chemicalREQUASUD,
           data.chemicalLUCAS,
           wl.param = 'N',
           wl.bin = 0.5,
           wl.xlab = "Nitrogen [g/Kg]"
)

Potassium content

We compare the distributions of potassium content here between REQUASUD and LUCAS:

f.wl.distr(data.chemicalREQUASUD,
           data.chemicalLUCAS,
           wl.param = 'K',
           wl.bin = 50,
           wl.xlab = "Potassium [mg/Kg]"
)

Phosphorus content

We compare the distributions of phosphorus content here between REQUASUD and LUCAS:

f.wl.distr(data.chemicalREQUASUD,
           data.chemicalLUCAS,
           wl.param = 'P',
           wl.bin = 50,
           wl.xlab = "Phosphorus [mg/Kg]"
)

### Statistical inferences

We provide as well the KS test that evaluates if LUCAS and REQUASUD represent the same population. The KS is provided with a p-value.

KS.df <- cbind.data.frame(soil.property = c("pH_H2O","OC", "P","N","K"),
                 KS.test = round(c(
                   as.numeric(ks.test(x = data.chemicalREQUASUD$pH_H2O,
                       y=data.chemicalLUCAS$pH_H2O)["statistic"]),
                   
                   as.numeric(ks.test(x = data.chemicalREQUASUD$OC,
                                      y= data.chemicalLUCAS$OC)["statistic"]),
                   
                   as.numeric(ks.test(x = data.chemicalREQUASUD$N,
                                    y=data.chemicalLUCAS$N)["statistic"]),
                                    
                   as.numeric(ks.test(x = data.chemicalREQUASUD$P,
                                    y= data.chemicalLUCAS$P)["statistic"]),
                                    
                   as.numeric(ks.test(x = data.chemicalREQUASUD$K,
                                    y= data.chemicalLUCAS$K)["statistic"])),3),
                 
                 p.value = round(c(
                   
                   as.numeric(ks.test(x = data.chemicalREQUASUD$pH_H2O,
                                    y = data.chemicalLUCAS$pH_H2O)["p.value"]),
                   
                   as.numeric(ks.test(x = data.chemicalREQUASUD$OC,
                                      y = data.chemicalLUCAS$OC)["p.value"]),
                   
                   as.numeric(ks.test(x = data.chemicalREQUASUD$N,
                                    y = data.chemicalLUCAS$N)["p.value"]),
                                    
                   as.numeric(ks.test(x = data.chemicalREQUASUD$P,
                                    y = data.chemicalLUCAS$P)["p.value"]),
                                      
                   as.numeric(ks.test(x = data.chemicalREQUASUD$K,
                                    y = data.chemicalLUCAS$K)["p.value"])),10))

KS.df %>% 
  kbl(caption = "Kolmogorov-Smirnov (KS) test values and p-value") %>%
  kable_styling(bootstrap_options = "striped", full_width = F)
Kolmogorov-Smirnov (KS) test values and p-value
soil.property KS.test p.value
pH_H2O 0.219 0.0485967
OC 0.178 0.1586452
P 0.397 0.0000076
N 0.129 0.5189660
K 0.181 0.1455645

General comment on chemical properties

Globally, the distributions are visually closed between REQUASUD and LUCAS dataset. The tests show no significant differences except for the pH_H2O (p<0.05) and for Nitrogen (p<0.001).

Spatial analysis

Data visualisation

Spatial distribution of clay values

f.wl.property (wallonie,
               data.granuloREQUASUD,
               data.granuloLUCAS,
               property = 'Clay',
               "Long_wgs84",
               "Lat_wgs84",
               coord.crs = 4326,
               title = 'Clay [%]'
) 

Spatial distribution of silt values

f.wl.property (wallonie,
               data.granuloREQUASUD,
               data.granuloLUCAS,
               property = 'Silt',
               "Long_wgs84",
               "Lat_wgs84",
               coord.crs = 4326,
               title = 'Silt [%]'
) 

Spatial distribution of sand values

f.wl.property (wallonie,
               data.granuloREQUASUD,
               data.granuloLUCAS,
               property = 'Sand',
               "Long_wgs84",
               "Lat_wgs84",
               coord.crs = 4326,
               title = 'Sand [%]'
)  

Spatial distribution of pH values

f.wl.property (wallonie,
               data.chemicalREQUASUD,
               data.chemicalLUCAS,
               property = 'pH_H2O',
               "Long_wgs84",
               "Lat_wgs84",
               coord.crs = 4326,
               title = 'pH in water [-]'
)  

Spatial distribution of OC values

f.wl.property (wallonie,
               data.chemicalREQUASUD,
               data.chemicalLUCAS,
               property = 'OC',
               "Long_wgs84",
               "Lat_wgs84",
               coord.crs = 4326,
               title = 'Organique\ncarbone [g/Kg]'
)

Spatial distribution of P values

f.wl.property (wallonie,
               data.chemicalREQUASUD,
               data.chemicalLUCAS,
               property = 'P',
               "Long_wgs84",
               "Lat_wgs84",
               coord.crs = 4326,
               title = 'Phophorus [mg/Kg]'
)

Spatial distribution of N values

f.wl.property (wallonie,
               data.chemicalREQUASUD,
               data.chemicalLUCAS,
               property = 'N',
               "Long_wgs84",
               "Lat_wgs84",
               coord.crs = 4326,
               title = 'Nitrogen [g/Kg]'
)

Spatial distribution of K values

f.wl.property (wallonie,
               data.chemicalREQUASUD,
               data.chemicalLUCAS,
               property = 'K',
               "Long_wgs84",
               "Lat_wgs84",
               coord.crs = 4326,
               title = 'Potasium [mg/Kg]'
)

Conclusion

We compareda European-scale soil monitoriring systems (LUCAS) to a dataset of soil analyses mainly done in Walloon farm proposed by a public services for advising in agricultural systems (REQUASUD). We used this dataset because it is available (partner of our research center) and contains a huge quantity of good quality data ((>100k samples).

This divergent status of the data is an issue for the comparison strategy and is also a source of heterogeneity in some soil proporties. But globally, we can observe that the data are closed.

This report is the first of a series that will compare the data from LUCAS and the data from Wallonia. We can already stretch two major recommandations : - increase the number of sampling point of LUCAS - implement a Soil Monitoring network in Wallonia, in the same way as what is recently done in Belgium-Flanders (NAME ? DATE?) or in France (RMQS / SIMS).

Appendix

Session information

 sessioninfo::session_info()
## ─ Session info ────────────────────────────────────────────────────────────────────────────────────
##  setting  value
##  version  R version 4.3.1 (2023-06-16)
##  os       EndeavourOS
##  system   x86_64, linux-gnu
##  ui       X11
##  language (EN)
##  collate  fr_BE.UTF-8
##  ctype    fr_BE.UTF-8
##  tz       Europe/Brussels
##  date     2023-10-05
##  pandoc   3.1.2 @ /usr/bin/ (via rmarkdown)
## 
## ─ Packages ────────────────────────────────────────────────────────────────────────────────────────
##  ! package           * version   date (UTC) lib source
##    abind               1.4-5     2016-07-21 [1] CRAN (R 4.3.0)
##    backports           1.4.1     2021-12-13 [1] CRAN (R 4.3.0)
##    base64enc           0.1-3     2015-07-28 [1] CRAN (R 4.3.0)
##    bayesm              3.1-5     2022-12-02 [1] CRAN (R 4.3.0)
##    bookdown            0.34      2023-05-09 [1] CRAN (R 4.3.0)
##    bslib               0.4.2     2022-12-16 [1] CRAN (R 4.3.0)
##    cachem              1.0.8     2023-05-01 [1] CRAN (R 4.3.0)
##    callr               3.7.3     2022-11-02 [1] CRAN (R 4.3.0)
##    cellranger          1.1.0     2016-07-27 [1] CRAN (R 4.3.0)
##    checkmate           2.2.0     2023-04-27 [1] CRAN (R 4.3.0)
##    class               7.3-22    2023-05-03 [2] CRAN (R 4.3.1)
##    classInt            0.4-9     2023-02-28 [1] CRAN (R 4.3.0)
##    cli                 3.6.1     2023-03-23 [1] CRAN (R 4.3.0)
##    cluster             2.1.4     2022-08-22 [2] CRAN (R 4.3.1)
##    codetools           0.2-19    2023-02-01 [2] CRAN (R 4.3.1)
##    colorspace          2.1-0     2023-01-23 [1] CRAN (R 4.3.1)
##    compositions        2.0-6     2023-04-13 [1] CRAN (R 4.3.0)
##    conflicted        * 1.2.0     2023-02-01 [1] CRAN (R 4.3.0)
##    crayon              1.5.2     2022-09-29 [1] CRAN (R 4.3.0)
##    crosstalk           1.2.0     2021-11-04 [1] CRAN (R 4.3.0)
##    data.table          1.14.8    2023-02-17 [1] CRAN (R 4.3.0)
##    DBI                 1.1.3     2022-06-18 [1] CRAN (R 4.3.0)
##    deldir              1.0-6     2021-10-23 [1] CRAN (R 4.3.0)
##    DEoptimR            1.0-13    2023-05-02 [1] CRAN (R 4.3.0)
##    desc                1.4.2     2022-09-08 [1] CRAN (R 4.3.0)
##    devtools            2.4.5     2022-10-11 [1] CRAN (R 4.3.1)
##    dgof              * 1.4       2022-06-16 [1] CRAN (R 4.3.0)
##    dichromat           2.0-0.1   2022-05-02 [1] CRAN (R 4.3.0)
##    digest              0.6.31    2022-12-11 [1] CRAN (R 4.3.0)
##    dplyr             * 1.1.2     2023-04-20 [1] CRAN (R 4.3.0)
##    DT                * 0.27      2023-01-17 [1] CRAN (R 4.3.0)
##    e1071               1.7-13    2023-02-01 [1] CRAN (R 4.3.0)
##    ellipsis            0.3.2     2021-04-29 [1] CRAN (R 4.3.0)
##    evaluate            0.21      2023-05-05 [1] CRAN (R 4.3.0)
##    fansi               1.0.4     2023-01-22 [1] CRAN (R 4.3.0)
##    farver              2.1.1     2022-07-06 [1] CRAN (R 4.3.0)
##    fastmap             1.1.1     2023-02-24 [1] CRAN (R 4.3.0)
##    foreign             0.8-84    2022-12-06 [2] CRAN (R 4.3.1)
##    Formula             1.2-5     2023-02-24 [1] CRAN (R 4.3.0)
##    fs                  1.6.2     2023-04-25 [1] CRAN (R 4.3.0)
##    generics            0.1.3     2022-07-05 [1] CRAN (R 4.3.0)
##    ggplot2           * 3.4.2     2023-04-03 [1] CRAN (R 4.3.0)
##    ggspatial         * 1.1.8     2023-04-13 [1] CRAN (R 4.3.0)
##  R ggtern            * 3.4.0     <NA>       [?] <NA>
##    glue                1.6.2     2022-02-24 [2] CRAN (R 4.2.0)
##    goftest             1.2-3     2021-10-07 [1] CRAN (R 4.3.0)
##    gridExtra           2.3       2017-09-09 [1] CRAN (R 4.3.0)
##    gtable              0.3.3     2023-03-21 [1] CRAN (R 4.3.0)
##    hexbin              1.28.3    2023-03-21 [1] CRAN (R 4.3.0)
##    highr               0.10      2022-12-22 [1] CRAN (R 4.3.0)
##    Hmisc             * 5.1-0     2023-05-08 [1] CRAN (R 4.3.0)
##    htmlTable           2.4.1     2022-07-07 [1] CRAN (R 4.3.0)
##    htmltools           0.5.5     2023-03-23 [1] CRAN (R 4.3.0)
##    htmlwidgets         1.6.2     2023-03-17 [1] CRAN (R 4.3.0)
##    httpuv              1.6.11    2023-05-11 [1] CRAN (R 4.3.0)
##    httr                1.4.6     2023-05-08 [1] CRAN (R 4.3.0)
##    jquerylib           0.1.4     2021-04-26 [1] CRAN (R 4.3.0)
##    jsonlite            1.8.4     2022-12-06 [1] CRAN (R 4.3.0)
##    kableExtra        * 1.3.4     2021-02-20 [1] CRAN (R 4.3.0)
##    KernSmooth          2.23-21   2023-05-03 [2] CRAN (R 4.3.1)
##    knitr             * 1.42      2023-01-25 [1] CRAN (R 4.3.0)
##    labeling            0.4.2     2020-10-20 [1] CRAN (R 4.3.0)
##    later               1.3.1     2023-05-02 [1] CRAN (R 4.3.0)
##    latex2exp           0.9.6     2022-11-28 [1] CRAN (R 4.3.0)
##    lattice             0.21-8    2023-04-05 [2] CRAN (R 4.3.1)
##    leafem              0.2.0     2022-04-16 [1] CRAN (R 4.3.0)
##    leaflet             2.1.2     2023-03-10 [1] CRAN (R 4.3.0)
##    leaflet.providers   1.9.0     2019-11-09 [1] CRAN (R 4.3.0)
##    leafsync            0.1.0     2019-03-05 [1] CRAN (R 4.3.0)
##    lifecycle           1.0.3     2022-10-07 [2] CRAN (R 4.2.2)
##    lwgeom              0.2-11    2023-01-14 [1] CRAN (R 4.3.0)
##    magrittr            2.0.3     2022-03-30 [2] CRAN (R 4.2.0)
##    mapview           * 2.11.0    2022-04-16 [1] CRAN (R 4.3.0)
##    MASS                7.3-60    2023-05-04 [2] CRAN (R 4.3.1)
##    Matrix              1.5-4.1   2023-05-18 [2] CRAN (R 4.3.1)
##    memoise             2.0.1     2021-11-26 [1] CRAN (R 4.3.0)
##    mgcv                1.8-42    2023-03-02 [2] CRAN (R 4.3.1)
##    mime                0.12      2021-09-28 [1] CRAN (R 4.3.0)
##    miniUI              0.1.1.1   2018-05-18 [1] CRAN (R 4.3.0)
##    mnormt              2.1.1     2022-09-26 [1] CRAN (R 4.3.0)
##    munsell             0.5.0     2018-06-12 [1] CRAN (R 4.3.0)
##    nlme              * 3.1-162   2023-01-31 [2] CRAN (R 4.3.1)
##    nnet                7.3-19    2023-05-03 [2] CRAN (R 4.3.1)
##    pillar              1.9.0     2023-03-22 [1] CRAN (R 4.3.0)
##    pkgbuild            1.4.0     2022-11-27 [1] CRAN (R 4.3.0)
##    pkgconfig           2.0.3     2019-09-22 [2] CRAN (R 4.2.2)
##    pkgload             1.3.2     2022-11-16 [1] CRAN (R 4.3.0)
##    plyr                1.8.8     2022-11-11 [1] CRAN (R 4.3.0)
##    png                 0.1-8     2022-11-29 [1] CRAN (R 4.3.0)
##    polyclip            1.10-4    2022-10-20 [1] CRAN (R 4.3.0)
##    prettydoc         * 0.4.1     2021-01-10 [1] CRAN (R 4.3.0)
##    prettyunits         1.1.1     2020-01-24 [1] CRAN (R 4.3.0)
##    processx            3.8.1     2023-04-18 [1] CRAN (R 4.3.0)
##    profvis             0.3.8     2023-05-02 [1] CRAN (R 4.3.0)
##    promises            1.2.0.1   2021-02-11 [1] CRAN (R 4.3.0)
##    proto               1.0.0     2016-10-29 [1] CRAN (R 4.3.0)
##    proxy               0.4-27    2022-06-09 [1] CRAN (R 4.3.0)
##    ps                  1.7.5     2023-04-18 [1] CRAN (R 4.3.0)
##    psych             * 2.3.3     2023-03-18 [1] CRAN (R 4.3.0)
##    purrr               1.0.1     2023-01-10 [2] CRAN (R 4.2.2)
##    R6                  2.5.1     2021-08-19 [1] CRAN (R 4.3.0)
##    raster            * 3.6-20    2023-03-06 [1] CRAN (R 4.3.0)
##    RColorBrewer        1.1-3     2022-04-03 [1] CRAN (R 4.3.0)
##    Rcpp                1.0.10    2023-01-22 [1] CRAN (R 4.3.0)
##    readxl            * 1.4.2     2023-02-09 [1] CRAN (R 4.3.0)
##    remotes             2.4.2     2021-11-30 [1] CRAN (R 4.3.0)
##    rlang               1.1.1     2023-04-28 [1] CRAN (R 4.3.0)
##    rmarkdown         * 2.21      2023-03-26 [1] CRAN (R 4.3.0)
##    rmdformats        * 1.0.4     2022-05-17 [1] CRAN (R 4.3.0)
##    robustbase          0.95-1    2023-03-29 [1] CRAN (R 4.3.0)
##    rpart             * 4.1.19    2022-10-21 [2] CRAN (R 4.3.1)
##    rprojroot           2.0.3     2022-04-02 [1] CRAN (R 4.3.0)
##    rstudioapi          0.14      2022-08-22 [1] CRAN (R 4.3.0)
##    rvest               1.0.3     2022-08-19 [1] CRAN (R 4.3.0)
##    s2                  1.1.3     2023-04-27 [1] CRAN (R 4.3.0)
##    sass                0.4.6     2023-05-03 [1] CRAN (R 4.3.0)
##    satellite           1.0.4     2021-10-12 [1] CRAN (R 4.3.0)
##    scales              1.2.1     2022-08-20 [1] CRAN (R 4.3.0)
##    sessioninfo         1.2.2     2021-12-06 [1] CRAN (R 4.3.0)
##    sf                * 1.0-14    2023-07-11 [1] CRAN (R 4.3.1)
##    shiny               1.7.4     2022-12-15 [1] CRAN (R 4.3.0)
##    soiltexture       * 1.5.1     2018-09-20 [1] CRAN (R 4.3.0)
##    sp                * 1.6-0     2023-01-19 [1] CRAN (R 4.3.0)
##    spatstat          * 3.0-5     2023-04-22 [1] CRAN (R 4.3.0)
##    spatstat.data     * 3.0-1     2023-03-12 [1] CRAN (R 4.3.0)
##    spatstat.explore  * 3.2-1     2023-05-13 [1] CRAN (R 4.3.0)
##    spatstat.geom     * 3.2-1     2023-05-09 [1] CRAN (R 4.3.0)
##    spatstat.linnet   * 3.1-1     2023-05-15 [1] CRAN (R 4.3.0)
##    spatstat.model    * 3.2-4     2023-05-15 [1] CRAN (R 4.3.0)
##    spatstat.random   * 3.1-5     2023-05-11 [1] CRAN (R 4.3.0)
##    spatstat.sparse     3.0-1     2023-03-12 [1] CRAN (R 4.3.0)
##    spatstat.utils      3.0-3     2023-05-09 [1] CRAN (R 4.3.0)
##    stars               0.6-1     2023-04-06 [1] CRAN (R 4.3.0)
##    stringi             1.7.12    2023-01-11 [1] CRAN (R 4.3.1)
##    stringr             1.5.0     2022-12-02 [1] CRAN (R 4.3.1)
##    svglite             2.1.1     2023-01-10 [1] CRAN (R 4.3.0)
##    systemfonts         1.0.4     2022-02-11 [1] CRAN (R 4.3.0)
##    tensor              1.5       2012-05-05 [1] CRAN (R 4.3.0)
##    tensorA             0.36.2    2020-11-19 [1] CRAN (R 4.3.0)
##    terra             * 1.7-29    2023-04-22 [1] CRAN (R 4.3.0)
##    tibble              3.2.1     2023-03-20 [1] CRAN (R 4.3.0)
##    tidyr             * 1.3.0     2023-01-24 [1] CRAN (R 4.3.0)
##    tidyselect          1.2.0     2022-10-10 [1] CRAN (R 4.3.0)
##    tidyterra         * 0.4.0     2023-03-17 [1] CRAN (R 4.3.0)
##    tmap              * 3.3-3     2022-03-02 [1] CRAN (R 4.3.0)
##    tmaptools           3.1-1     2021-01-19 [1] CRAN (R 4.3.0)
##    units               0.8-2     2023-04-27 [1] CRAN (R 4.3.0)
##    urlchecker          1.0.1     2021-11-30 [1] CRAN (R 4.3.0)
##    usethis             2.1.6     2022-05-25 [1] CRAN (R 4.3.0)
##    utf8                1.2.3     2023-01-31 [1] CRAN (R 4.3.0)
##    vctrs               0.6.2     2023-04-19 [1] CRAN (R 4.3.0)
##    viridis           * 0.6.3     2023-05-03 [1] CRAN (R 4.3.0)
##    viridisLite       * 0.4.2     2023-05-02 [1] CRAN (R 4.3.0)
##    webshot             0.5.4     2022-09-26 [1] CRAN (R 4.3.0)
##    withr               2.5.0     2022-03-03 [1] CRAN (R 4.3.0)
##    wk                  0.7.3     2023-05-06 [1] CRAN (R 4.3.0)
##    xfun                0.39      2023-04-20 [1] CRAN (R 4.3.0)
##    XML                 3.99-0.14 2023-03-19 [1] CRAN (R 4.3.0)
##    xml2                1.3.4     2023-04-27 [1] CRAN (R 4.3.0)
##    xtable              1.8-4     2019-04-21 [1] CRAN (R 4.3.0)
##    yaml                2.3.7     2023-01-23 [1] CRAN (R 4.3.0)
## 
##  [1] /home/fred/R/x86_64-pc-linux-gnu-library/4.3
##  [2] /usr/lib/R/library
## 
##  R ── Package was removed from disk.
## 
## ───────────────────────────────────────────────────────────────────────────────────────────────────

Personal functions

‘./R/wl_personal_functions.R’

cat(readLines('./R/wl_personal_functions.R'), sep = '\n')

## Filter REQUASUD data
f.requasud <- function(f.year = 'all',
                       f.depth = op.depth){
  
  if (f.year == 'all') {
    df <- data.REQUASUD %>% 
      filter(Depth %in% f.depth)
  } else {
    df <- data.REQUASUD %>% 
      filter(year == f.year & Depth %in% f.depth)
    
  }
  
  return(df)
}

get_png <- function(filename) {
    grid::rasterGrob(png::readPNG(filename), interpolate = TRUE,
                     x = unit(1,"npc"),
                     y = unit(-0.15,"npc"),
                     ## width = 0.33,
                     height = 0.12,
                     vjust = 0,
                     hjust = 1
                     )
}
logos.data <- get_png("./img/montage_logos_graph.png")
text.data <- grobTree(textGrob('Data source:',
                               x=unit(0.66, "npc"), y=unit(-0.075,"npc"),
                               hjust=1))

## Graph distribution (histogram)
## ------------------
f.wl.distr <- function(wl.data.req,
                       wl.data.lucas,
                       wl.param,
                       wl.bin,
                       wl.xlab,
                       wl.title = 'Histogram comparing LUCAS and REQUASUD'){
    g <- ggplot() +
        geom_histogram(data = wl.data.req,
                       aes(y = -(..density..), x = get(wl.param), fill = "Requasud"),
                       col = "black", binwidth = wl.bin) +
        geom_histogram(data = wl.data.lucas,
                       aes(y = (..density..), x = get(wl.param), fill = "Lucas"),
                       col = "black", binwidth = wl.bin) +
        labs(fill ="Dataset", x = wl.xlab, y = " ",title=wl.title) +
        scale_fill_viridis(direction=-1,discrete = TRUE) +
        theme_classic() + 
        theme(legend.position = "right",
              plot.margin = unit(c(0.02, 0.02, 0.12, 0.02), "npc")
              ) +
        coord_flip(clip = "off") +
        annotation_custom(text.data) +
        annotation_custom(logos.data) ## , ymin = 0.05, ymax = 0.1, xmin = -7, xmax = -12)
    return(g)
}

### Statistic
## ------------------
f.summary <- function(data.df,
                      param.select){
  summary_stat <- data.df %>% 
    dplyr::select(param.select) %>%
    describe(ranges = T, quant = c(.25,.75))%>%
    as.data.frame()
  return(summary_stat)
  }

### Table of summary
## ------------------
f.wl.stat.table <- function(data.stat.lucas,
                            data.stat.req,
                            show.stat = 'central' ){
  if (show.stat == 'central') {
    stat.table <- cbind.data.frame(data.stat.lucas %>% 
                              dplyr::select(Mean.LUCAS = mean),
                            data.stat.req %>% 
                              dplyr::select(Mean.REQUASUD = mean),
                            data.stat.lucas %>% 
                              dplyr::select(Median.LUCAS = median),
                            data.stat.req %>% 
                              dplyr::select(Median.REQUASUD = median)) %>%
    round(digits = 1) %>%
    kbl(caption ="Central tendency for LUCAS and REQUASUD datasets" ,row.names = T) %>%
    kable_styling(bootstrap_options = "striped", full_width = F)
  }
  else {
    stat.table <- cbind.data.frame(data.stat.lucas %>% 
                                     dplyr::select(Range.LUCAS = range),
                                   data.stat.req %>% 
                                     dplyr::select(Range.REQUASUD = range),
                                   data.stat.lucas %>% 
                                     dplyr::select(Quantile25.LUCAS = Q0.25),
                                   data.stat.req %>% 
                                     dplyr::select(Quantile25.REQUASUD = Q0.25),
                                   data.stat.lucas %>% 
                                     dplyr::select(Quantile75.LUCAS = Q0.75),
                                   data.stat.req %>% 
                                     dplyr::select(Quantile75.REQUASUD = Q0.75),
                                   data.stat.lucas %>% 
                                     dplyr::select(SD.LUCAS = sd),
                                   data.stat.req %>% 
                                     dplyr::select(SD.REQUASUD = sd),
                                   data.stat.lucas %>% 
                                     dplyr::select(Skewness.LUCAS = skew),
                                   data.stat.req %>% 
                                     dplyr::select(Skewness.REQUASUD = skew),
                                   data.stat.lucas %>% 
                                     dplyr::select(Kurtosis.LUCAS = kurtosis),
                                   data.stat.req %>% 
                                     dplyr::select(Kurtosis.REQUASUD = kurtosis)) %>%
      round(digits = 1) %>%
      kbl(caption = "Spread indicators for LUCAS and REQUASUD datasets",row.names = T) %>%
      kable_styling(bootstrap_options = "striped", full_width = F)
    
  }
  return(stat.table)
}


## Spatial distribution
## ------------------


logos.data.map <- grid::rasterGrob(png::readPNG("./img/montage_logos_graph.png"),
                                   interpolate = TRUE,
                     x = unit(0.01,"npc"),
                     y = unit(0.01,"npc"),
                     ## width = 0.33,
                     height = 0.12,
                     vjust = 0,
                     hjust = 0
                     )
text.data.map <- grobTree(textGrob('Data source:',
                                   x=unit(0.01, "npc"),
                                   y=unit(0.15,"npc"),
                               hjust=0,vjust=0))

f.wl.spatial <- function (region,
                          data.spt.req,
                          data.spt.lucas,
                          x.coord,
                          y.coord,
                          coord.crs,
                          data.used
                          ){
    region = st_transform(region,crs = coord.crs)
    
    plot.spatial.dis <- (ggplot(region) +
                         geom_sf(fill='white') +
                         geom_point(data = data.spt.req,
                                    aes(x = .data[[x.coord]],
                                        y = .data[[y.coord]],
                                        fill = "Requasud"),
                                    size = 2,
                                    shape = 23,alpha = 0.2) +
                         geom_point(data = data.spt.lucas,
                                    aes(x = .data[[x.coord]],
                                        y = .data[[y.coord]],
                                        fill = "Lucas"),
                                    size = 2,
                                    shape = 23) +
                         labs(fill = data.used) +
                         scale_fill_viridis(direction=-1,discrete = TRUE) +
                         annotation_scale(location = "bl",
                                          width_hint = 0.1,
                                          pad_x = unit(0.01, "npc"),
                                          pad_y = unit(0.22, "npc")) +
                         annotation_north_arrow(location = "bl", which_north = "true",
                                                pad_x = unit(0.01, "npc"),
                                                pad_y = unit(0.25, "npc"),
                                                style = north_arrow_fancy_orienteering) +
                         coord_sf(clip = "off") +
                         theme_bw() + 
                         theme(legend.position = "right"
                               ) +
                         annotation_custom(text.data.map) +
                         annotation_custom(logos.data.map)
    )
    return(plot.spatial.dis)
    
}


## Spatial distribution by property
## ------------------

f.wl.property <- function (region,
                           data.spt.req,
                           data.spt.lucas,
                           property,
                           x.coord,
                           y.coord,
                           coord.crs,
                           title
){
  region <- st_transform(region,crs = coord.crs)
  
  req.sf <- st_as_sf(data.spt.req,
                     coords=c(x.coord,y.coord),
                     crs= coord.crs)
  
  lucas.sf <- st_as_sf(data.spt.lucas,
                     coords=c(x.coord,y.coord),
                     crs= coord.crs)
  
  map.req <- tm_shape(region) + 
    tm_fill(col = "white") + 
    tm_polygons () +
    tm_shape(req.sf) + 
    tm_dots(property,palette = viridis(14,direction = -1),size = 0.3,colorNA = NULL) +
    tm_legend(legend.outside = TRUE,
              title = paste0("REQUASUD / ", title))
  
  
  map.lucas <- tm_shape(region) + 
    tm_fill(col = "white") + 
    tm_polygons () +
    tm_shape(lucas.sf) + 
    tm_dots(property,palette = viridis(14,direction = -1),size = 0.3,colorNA = NULL) +
    tm_legend(legend.outside = TRUE,
              title = paste0("LUCAS / ", title))
  
  maplucasVSrequausud <- tmap_arrange(map.lucas , map.req)
  
  return(maplucasVSrequausud)
}

‘./R/wl_personal_functions_fred.R’

cat(readLines('./R/wl_personal_functions_fred.R'), sep = '\n')
f.map.raster.fact <- function(raster, variable, title, unit) {
    map <- ggplot() +
        geom_spatraster(data = raster, aes(fill = .data[[variable]])) +
        geom_sf(data = wallonie, fill = NA,
                colour = "black", linewidth = 1) +
        scale_fill_viridis_d(na.value = NA) +
        labs(fill = unit) +
        ggtitle(title) +
        theme_bw()
    return(map)
}

f.map.raster <- function(raster, variable, title, unit) {
    map <- ggplot() +
        geom_spatraster(data = raster, aes(fill = .data[[variable]])) +
        geom_sf(data = wallonie, fill = NA,
                colour = "black", linewidth = 1) +
        scale_fill_viridis_c(na.value = NA) +
        labs(fill = unit) +
        ggtitle(title) +
        theme_bw()
    return(map)
}

  f.soil.awc.ptf <- function(data.granulo, text.tol = 0.01) {
    ## Estimation of available water capacity
    df.granulo.AWC <- data.granulo %>%
      mutate(sum.granulo = ((Clay+Silt+Sand)/100)) %>% 
      ## CHECK IF NEEDED TODO !!!!
      ## mutate(Clay = Clay/10) %>%
      ## mutate(Silt = Silt/10) %>%
      ## mutate(Sand = Sand/10) %>%
      ## Pedotransfer function from Roman Dobarco et al., 2019
      mutate(FC = 0.270 + ((2.45*10^-3)*Clay) + ( (-1.35*10^-3)*Sand) ) %>% 
      mutate(PWP = 0.08 + ((4.01*10^-3)*Clay) + ( (-2.93*10^-4)*Sand)) %>%
      ## simplified estimation of available water capacity
      mutate(AWC = (FC - PWP)*100) %>%
      filter(!is.na(sum.granulo),
             !(sum.granulo < (1-text.tol)),
             !(sum.granulo > (1+text.tol))
             )

    return(df.granulo.AWC)
  }

  f.soil.tritext <- function(data.granulo,
                             colored.by = "AWC"
                             ) {
      TT <- (ggtern(
          data = data.granulo, ggtern::aes(
                                           x = Sand,
                                           y = Clay,
                                           z = Silt
                                       )) +
          geom_point(
              aes(color = .data[["AWC"]]), ## no variable available BUG
              size=2, alpha=0.5
          ) +
          ## scale_color_continuous(
          ##   low="tomato",
          ##   high="dodgerblue"
          ## ) +
          scale_color_viridis_c(option = "plasma", trans = "sqrt") +
          labs(
              title="Soil Texture Triangle",
              subtitle="Difference between field capacity and wilting point",
              color="Available water capacity (vol %)"
          ) +
          ggtern::theme_bw()
      )
  }

Bibliography

Reference of source (Citations) :

  • Panagos Panos. The European soil database (2006) GEO: connexion, 5 (7), pp. 32-33.
  • ESDBv2 Raster Library - a set of rasters derived from the European Soil Database distribution v2.0 (published by the European Commission and the European Soil Bureau Network, CD-ROM, EUR 19945 EN); Marc Van Liedekerke, Arwyn Jones, Panos Panagos ; 2006.

When making reference to the ESDAC

  • Panagos, P., Van Liedekerke, M., Borrelli, P., Köninger, J., Ballabio, C., Orgiazzi, A., Lugato, E., Liakos, L., Hervas, J., Jones, A. Montanarella, L. 2022. European Soil Data Centre 2.0: Soil data and knowledge in support of the EU policies. European Journal of Soil Science, 73(6), e13315. DOI: 10.1111/ejss.13315
  • Panagos P., Van Liedekerke M., Jones A., Montanarella L., “European Soil Data Centre: Response to European policy support and public data requirements”; (2012) Land Use Policy, 29 (2), pp. 329-338. doi:10.1016/j.landusepol.2011.07.003
  • European Soil Data Centre (ESDAC), esdac.jrc.ec.europa.eu, European Commission, Joint Research Centre

  1. CRA-W U7, f.vanwindekens[at]cra.wallonie.be↩︎

  2. CRA-W U7, j.herinckx[at]cra.wallonie.be↩︎

  3. Stagiaire CRA-W - ULg/GbxABT, aaryapo[at]student.uliege.be↩︎

  4. CRA-W U7, b.huyghebaert[at]cra.wallonie.be↩︎