{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "colab": { "provenance": [] }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "name": "python" } }, "cells": [ { "id": "cell-000", "cell_type": "markdown", "metadata": { "id": "cell-000" }, "source": [ "# 📙 Notebook 3 — The Groundwater Model: Abstraction & Recharge\n", "# الدفتر ٣ — نموذج المياه الجوفية: السحب والتغذية\n", "\n", "**English** — This is the heart of the tool. For every month we estimate:\n", "\n", "| Output | Formula | Unit |\n", "|---|---|---|\n", "| **Abstraction (mm)** | `AETI / 0.70` (irrigation efficiency) | mm/month |\n", "| **Abstraction (m³)** | `abstraction_mm × 400 m² / 1000` (per 20 m pixel) | m³/month |\n", "| **Effective rainfall** | FAO: `P>75 → 0.8·P−25` ; `P≤75 → 0.6·P−10` (min 0) | mm/month |\n", "| **Recharge** | `max(effective rainfall − AETI, 0)` | mm/month |\n", "\n", "Assumptions: in these arid areas, crop water use (AETI) in irrigated fields is supplied almost entirely by groundwater pumping, with a field irrigation efficiency of 70%. Recharge happens where effective rainfall exceeds what plants consume.\n", "\n", "**العربية** — هذا قلب الأداة. لكل شهر نُقدّر: السحب بالمليمتر (= التبخر-نتح ÷ كفاءة الري 0.70)، والسحب بالمتر المكعب (× مساحة الخلية 400 م²)، والأمطار الفعّالة (معادلة الفاو)، والتغذية الجوفية (= الأمطار الفعّالة − التبخر-نتح، وبحد أدنى صفر). الافتراض: في المناطق الجافة، يأتي استهلاك المحاصيل المروية من ضخ المياه الجوفية بكفاءة ري 70٪.\n", "\n", "**Data in | المدخلات**: WaPOR v3 AETI (20 m, read live from FAO) + CHIRPS daily rainfall.\n", "**Data out | المخرجات**: 3 images per month saved as **assets in your project**, named exactly as the dashboard expects: `abstraction_mm_YYYY_MM`, `abstraction_m3_YYYY_MM`, `recharge_YYYY_MM`.\n" ] }, { "id": "cell-001", "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "cell-001", "outputId": "98e33406-446a-4e9e-f0e7-c88d445e1a1e" }, "source": [ "# @title 🔑 Authenticate with Google Earth Engine | المصادقة مع محرك جوجل الأرضي\n", "# Run this cell, follow the link, and paste nothing — Colab handles it automatically.\n", "import ee\n", "\n", "# ⚠️ CHANGE THIS to your own Earth Engine Cloud project ID\n", "# ⚠️ غيّر هذا إلى معرّف مشروعك الخاص في Earth Engine\n", "PROJECT_ID = \"steel-sonar-428908-v8\" # e.g. \"gw-tool-jordan\"\n", "\n", "ee.Authenticate()\n", "ee.Initialize(project=PROJECT_ID)\n", "print(\"✅ Earth Engine ready! Project:\", PROJECT_ID)" ], "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "✅ Earth Engine ready! Project: steel-sonar-428908-v8\n" ] } ], "execution_count": 1 }, { "id": "cell-002", "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "cell-002", "outputId": "5cf7bf1d-63b4-4e77-d23d-c08cbf822bfc" }, "source": [ "# @title ⚙️ Model configuration | إعدادات النموذج\n", "REGION = \"PAL\" # WaPOR L3 region code\n", "START_MONTH = \"2026-01\" # first month to process\n", "END_MONTH = \"2026-01\" # last month to process\n", "ASSET_FOLDER_NAME = \"GW_Analysis_Jericho\" # output folder name in your assets\n", "\n", "IRRIGATION_EFFICIENCY = 0.70\n", "PIXEL_AREA_M2 = 400 # 20 m x 20 m\n", "SCALE_M = 20\n", "\n", "ASSET_FOLDER = f\"projects/{PROJECT_ID}/assets/{ASSET_FOLDER_NAME}\"\n", "BUCKET = \"fao-gismgr-wapor-3-data\"\n", "PREFIX = f\"DATA/WAPOR-3/MOSAICSET/L3-AETI-M/WAPOR-3.L3-AETI-M.{REGION}.\"\n", "print(\"Outputs will go to:\", ASSET_FOLDER)" ], "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Outputs will go to: projects/steel-sonar-428908-v8/assets/GW_Analysis_Jericho\n" ] } ], "execution_count": 2 }, { "id": "cell-003", "cell_type": "markdown", "metadata": { "id": "cell-003" }, "source": [ "## 1. Input helpers | دوال المدخلات" ] }, { "id": "cell-004", "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "cell-004", "outputId": "6235e423-74b0-4cfb-cefe-0f0b96ee8dda" }, "source": [ "import requests\n", "import pandas as pd\n", "\n", "def list_available_months():\n", " months, token = [], None\n", " while True:\n", " params = {\"prefix\": PREFIX, \"fields\": \"items(name),nextPageToken\"}\n", " if token:\n", " params[\"pageToken\"] = token\n", " r = requests.get(\n", " f\"https://storage.googleapis.com/storage/v1/b/{BUCKET}/o\", params=params\n", " ).json()\n", " months += [i[\"name\"].split(\".\")[-2] for i in r.get(\"items\", [])\n", " if i[\"name\"].endswith(\".tif\")]\n", " token = r.get(\"nextPageToken\")\n", " if not token:\n", " break\n", " return sorted(months)\n", "\n", "def aeti_image(month):\n", " \"\"\"Monthly AETI in mm/month from FAO's bucket.\"\"\"\n", " uri = f\"gs://{BUCKET}/{PREFIX}{month}.tif\"\n", " return ee.Image.loadGeoTIFF(uri).multiply(0.1).rename(\"AETI\")\n", "\n", "def rainfall_image(month, geometry):\n", " \"\"\"Monthly rainfall sum (mm) from CHIRPS daily.\"\"\"\n", " start = ee.Date(month + \"-01\")\n", " return (\n", " ee.ImageCollection(\"UCSB-CHG/CHIRPS/DAILY\")\n", " .filterDate(start, start.advance(1, \"month\"))\n", " .filterBounds(geometry)\n", " .sum()\n", " .rename(\"P\")\n", " )\n", "\n", "available = list_available_months()\n", "months = [m for m in available if START_MONTH <= m <= END_MONTH]\n", "print(f\"{len(months)} months to process ({months[0]} → {months[-1]})\")\n", "\n", "# Study area = footprint of the WaPOR L3 tile\n", "aoi = ee.Image.loadGeoTIFF(f\"gs://{BUCKET}/{PREFIX}{months[0]}.tif\").geometry()" ], "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "1 months to process (2026-01 → 2026-01)\n" ] } ], "execution_count": 3 }, { "id": "cell-005", "cell_type": "markdown", "metadata": { "id": "cell-005" }, "source": [ "## 2. The model | النموذج" ] }, { "id": "cell-006", "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "cell-006", "outputId": "78602f4a-41fe-4041-dd04-b21afcb0166b" }, "source": [ "def compute_month(month):\n", " \"\"\"Return dict of the 3 output images for one month.\"\"\"\n", " aeti = aeti_image(month)\n", " rain = rainfall_image(month, aoi)\n", "\n", " # --- Groundwater abstraction | السحب ---\n", " abst_mm = aeti.divide(IRRIGATION_EFFICIENCY).max(0).rename(\"abstraction_mm\")\n", " abst_m3 = abst_mm.multiply(PIXEL_AREA_M2).divide(1000).rename(\"abstraction_m3\")\n", "\n", " # --- Effective rainfall (FAO method) | الأمطار الفعّالة ---\n", " eff_rain = (\n", " ee.Image(0)\n", " .where(rain.gt(75), rain.multiply(0.8).subtract(25))\n", " .where(rain.lte(75), rain.multiply(0.6).subtract(10))\n", " .max(0)\n", " .rename(\"eff_rain\")\n", " )\n", "\n", " # --- Recharge | التغذية الجوفية ---\n", " recharge = eff_rain.subtract(aeti).max(0).rename(\"recharge\")\n", "\n", " return {\"abstraction_mm\": abst_mm, \"abstraction_m3\": abst_m3, \"recharge\": recharge}\n", "\n", "# quick look at one month before exporting everything\n", "test = compute_month(months[-1])\n", "stats = test[\"abstraction_mm\"].reduceRegion(\n", " ee.Reducer.minMax().combine(ee.Reducer.mean(), None, True),\n", " aoi, 200, maxPixels=1e9).getInfo()\n", "print(f\"Test month {months[-1]} abstraction_mm:\", {k: round(v, 2) for k, v in stats.items()})" ], "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Test month 2026-01 abstraction_mm: {'abstraction_mm_max': 68.71, 'abstraction_mm_mean': 17.75, 'abstraction_mm_min': 0}\n" ] } ], "execution_count": 4 }, { "id": "cell-007", "cell_type": "markdown", "metadata": { "id": "cell-007" }, "source": [ "## 3. Export all months to assets | تصدير كل الأشهر إلى الأصول\n", "\n", "The loop below **skips outputs that already exist**, so you can safely re-run it — for example every few months when FAO publishes new data.\n", "\n", "الحلقة أدناه **تتخطى المخرجات الموجودة مسبقاً**، لذا يمكنك إعادة تشغيلها بأمان — مثلاً كل بضعة أشهر عندما تنشر الفاو بيانات جديدة.\n" ] }, { "id": "cell-008", "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "cell-008", "outputId": "242e49c8-eb3f-4e24-9384-d085a0afec03" }, "source": [ "# Create the output folder if needed\n", "try:\n", " ee.data.createAsset({\"type\": \"Folder\"}, ASSET_FOLDER)\n", " print(\"Created folder:\", ASSET_FOLDER)\n", "except Exception:\n", " print(\"Folder already exists:\", ASSET_FOLDER)\n", "\n", "# Which outputs already exist?\n", "try:\n", " existing = {a[\"id\"].split(\"/\")[-1]\n", " for a in ee.data.listAssets({\"parent\": ASSET_FOLDER})[\"assets\"]}\n", "except Exception:\n", " existing = set()\n", "print(f\"{len(existing)} assets already in the folder\")\n", "\n", "started = 0\n", "for month in months:\n", " date_str = month.replace(\"-\", \"_\") # 2023-06 -> 2023_06\n", " todo = {name: img for name, img in compute_month(month).items()\n", " if f\"{name}_{date_str}\" not in existing}\n", " for name, img in todo.items():\n", " task = ee.batch.Export.image.toAsset(\n", " image=img.clip(aoi),\n", " description=f\"{name}_{date_str}\",\n", " assetId=f\"{ASSET_FOLDER}/{name}_{date_str}\",\n", " scale=SCALE_M,\n", " region=aoi.bounds(1),\n", " maxPixels=1e13,\n", " )\n", " task.start()\n", " started += 1\n", " if todo:\n", " print(f\"{month}: started {len(todo)} exports\")\n", "\n", "print(f\"\\n🚀 {started} export tasks started.\")\n", "print(\"Monitor them in the next cell or at https://code.earthengine.google.com/tasks\")" ], "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Folder already exists: projects/steel-sonar-428908-v8/assets/GW_Analysis_Jericho\n", "288 assets already in the folder\n", "2026-01: started 3 exports\n", "\n", "🚀 3 export tasks started.\n", "Monitor them in the next cell or at https://code.earthengine.google.com/tasks\n" ] } ], "execution_count": 5 }, { "id": "cell-009", "cell_type": "markdown", "metadata": { "id": "cell-009" }, "source": [ "## 4. Monitor the export tasks | متابعة مهام التصدير" ] }, { "id": "cell-010", "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "cell-010", "outputId": "77ee06be-b664-4b33-b5cf-48bc9f9ec29b" }, "source": [ "import collections\n", "\n", "def task_summary(limit=300):\n", " \"\"\"Count the states of your most recent Earth Engine tasks.\"\"\"\n", " ops = ee.data.listOperations()[:limit]\n", " return dict(collections.Counter(op[\"metadata\"][\"state\"] for op in ops))\n", "\n", "print(task_summary())\n", "# Re-run this cell now and then. When everything says SUCCEEDED, move to Notebook 4.\n", "# أعد تشغيل هذه الخلية بين الحين والآخر. عندما تنجح كل المهام انتقل إلى الدفتر ٤." ], "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "{'PENDING': 2, 'RUNNING': 1, 'FAILED': 14}\n" ] } ], "execution_count": 6 }, { "id": "cell-011", "cell_type": "markdown", "metadata": { "id": "cell-011" }, "source": [ "## ⚠️ Model limitations | حدود النموذج\n", "\n", "- Abstraction assumes **all** crop water in the area comes from groundwater irrigation — valid for hyper-arid areas like Al Jafr, but check for areas with surface water sources.\n", "- The irrigation efficiency (0.70) is an assumption — adjust it to local field data if available.\n", "- Recharge here is *potential diffuse recharge*; it ignores wadi/flood focused recharge and lateral flows.\n", "- CHIRPS rainfall is ~5 km resolution — much coarser than the 20 m WaPOR data.\n", "\n", "**Next | التالي**: Notebook 4 — verify the outputs before connecting the dashboard.\n" ] } ] }