Tutorial 3: Setting up pyWaPOR for your own case study

1. Create a new notebook for your project

After creating an python environment with pywapor and jupyterlab following the step in Tutorial 2: Installing Jupyter Notebook on your local computer, you are now able to open JupyterLab on your PC. In this step, you will create a new jupyter notebook for your own case study:

• First, open Miniforge Prompt and run this command line

mamba activate pywapor_env

• Then, change your working folder. We highly recommend to create a dedicated folder for each new project in a drive where you have sufficient available storage (e.g., D:\\pywapor). If you're in C: drive, you will first need to switch to D: drive using the command below:

D:

cd D:\\pywapor

• Run the command below to start JupyterLab

jupyter lab

Once the JupyterLab interface is opened in your web browser,  Click on the "Python" tile under "Notebook" in "Launcher" tab to create a new notebook.

(© Copyright CodeRefinery team)

You will see a new notebook named "Untitled.ipynb". To change the name of the notebook, right-click on the tab and select "Rename Notebook..."

Enter the name of this new notebook (e.g., "Run pywapor case study.ipynb") and click "Rename".

import pywapor

2. PyWaPOR project set up

To set up pyWaPOR for your own case study you require the following information to configure the set up:

• project folder
• bounding box of the Area-of-Interest (AoI)
• study period, which is usually the start and end of the cropping season

project_folder =  r"D:\\pywapor\\Case_study" #Path to folder
bb = [xmin, ymin, xmax, ymax] #Bounding box of your area
period = ["YYYY-MM-DD", "YYYY-MM-DD"] 
# Set up a project.
project = pywapor.Project(project_folder, bb, period)

2.1. Folder

Replace the value of project_folder in the code above with the absolute path to your project folder. Since you've started Jupyter Notebook in your working folder (e.g., D:\\pywapor), if you use relative path (i.e., "Case_study"), the pywapor.Project function will create a new folder in your working folder named "Case_study" (e.g., D:\\pywapor\Case_study). 

Notes

• When changing Area-of-Interest, Period, or project configuration, it's better to create a separate project folder
  

2.2. Area-of-Interest

Replace the value of bb in the template code above with the a list of minimum longitude, minimum latitude, maximum longitude, and maximum latitude of your study area. If you have a shapefile of your study area (with EPSG:4326 coordinate reference system), the bounding box of your study area can be found from a shapefile extent. Open the shapefile in QGIS. Then, right-click on the shapefile layer in the Layers panel and select 'Properties'. In the 'Information' tab, copy the coordinate of the 'Extent'. See the example below.

Paste the copied coordinates to the code cell in Jupyter Notebook, then reformat the value from xmin, ymin: xmax, ymax to [xmin, ymin, xmax, ymax]

2.3. Period

Replace the value of period in the template code above with the a list of starting date and ending date of your study period in the format ["YYYY-MM-DD", "YYYY-MM-DD"]. 

Depending on the size of your area and the capacity of your computer, long period (a few months to more than a year) might overload your computer processing memory. As we are running the script on a regular laptop (8GB RAM), we can only download and process 1 month of data. Select a month in the middle of your cropping season and add 5 days before and 5 days after to have buffer period for interpolation of dates with missing data. 

For example, if your cropping season is from 2020-10-01 to 2021-04-30, select the middle month from 2021-01-01 to 2021-01-31. Then add 5 days before and after this period. Then, the value of period should be ["2020-12-25", "2021-02-05"].

3. Download input data

After setting up the project, the next step is to configure the input dataset and download data. Copy the example code below to your notebook and run the code cells.

• Configure input dataset

summary = {            
            'elevation': {'COPERNICUS.GLO30'},
            'meteorological': {'GEOS5.inst3_2d_asm_Nx'},
            'optical': {'SENTINEL2.S2MSI2A_R20m'},
            'precipitation': {'CHIRPS.P05'},            
            'solar radiation': {'MERRA2.M2T1NXRAD.5.12.4'},
            'statics': {'STATICS.WaPOR3'},
            'thermal': {'VIIRSL1.VNP02IMG'}, 
            'soil moisture': {'FILE:{folder}{sep}se_root_out*.nc'},             
            '_ENHANCE_': {"bt": ["pywapor.enhancers.dms.thermal_sharpener.sharpen"],},
            '_EXAMPLE_': 'SENTINEL2.S2MSI2A_R20m',
            '_WHITTAKER_': {'SENTINEL2.S2MSI2A_R20m':{'method':'linear'},								
                            'VIIRSL1.VNP02IMG':{'method':'linear'},                                                   
                            },          
            }
project.load_configuration(summary = summary)

• You may configure a custom set of input data sources. Select the source and product code from 

https://www.fao.org/aquastat/py-wapor/data_sources.html

• • Make sure that you have filled in all the categories (i.e.,  elevation, meteorological, optical, precipitation, solar radiation, statistics, and thermal). For the 'soil moisture' category, we will use the output of SERoot model. You may select more than one source of input data. If you do so, the data downloading and processing time will be longer.
  • For example, if you want to use MODIS thermal data instead of VIIRS, the above example can be modified as follows:

  summary = {            
              'elevation': {'COPERNICUS.GLO30'},
              'meteorological': {'GEOS5.inst3_2d_asm_Nx'},
              'optical': {'SENTINEL2.S2MSI2A_R20m'},
              'precipitation': {'CHIRPS.P05'},            
              'solar radiation': {'MERRA2.M2T1NXRAD.5.12.4'},
              'statics': {'STATICS.WaPOR3'},
              'thermal': {'MODIS.MYD11A1.061','MODIS.MOD11A1.061'} 
              'soil moisture': {'FILE:{folder}{sep}se_root_out*.nc'},             
              '_ENHANCE_': {"lst": ["pywapor.enhancers.dms.thermal_sharpener.sharpen"],},
              '_EXAMPLE_': 'SENTINEL2.S2MSI2A_R20m',
              '_WHITTAKER_': {'SENTINEL2.S2MSI2A_R20m':{'method':'linear'},								
                              'MODIS.MYD11A1.061':{'method':'linear'},  
                              'MODIS.MOD11A1.061':{'method':'linear'},                                                  
                              },          
              }
  project.load_configuration(summary = summary)
  

• Set up account credentials to download data

project.set_passwords()

• • Depending on the selected data source and product, the code will prompt you to put in the user accounts and passwords that are required to download the configured input datasets. 

• Download data

datasets = project.download_data()

This code will start the process of downloading input data for pywapor. When python kernel is running the cell, make sure that you have internet connection. 
When you run pywapor for your own case study, the downloading step might take longer due to server or other interruptions not all data may have been downloaded. Check the log of the download section for warnings if there are some input data are not completely downloaded. If the downloading step is finished, you should not receive any warning in the printouts of the downloading step. (i.e., between > DOWNLOADER and < DOWNLOADER) except for the missing se_root_out.nc, which will be created in the next step.

Notes:
Due to sever and internet interruption, you might see some warnings in the printed log that pywapor may continue without certain input data. If this happens, you will need to re-run the downloading step. For that, you will need to restart the kernel by select tab 'Kernel' > 'Restart Kernel'. Remove corrupted files in the input data folders. Then, run all the code cells again.

4. Run SERoot and ETLook models

se_root_in = project.run_pre_se_root()
se_root = project.run_se_root(chunks = {"time": 1, "x": 500, "y": 500})


This code will pre-process input data and run the se_root module to calculate root zone soil moisture, which the an input to calculate evapotranspiration.

• Run etlook model

et_look_in = project.run_pre_et_look()
et_look = project.run_et_look(chunks = {"time_bins": 1, "x": 500, "y": 500})


This code will pre-process input data and run the et_look module to calculate evaporation, transpiration, interception, and net primary production.