Heritage GIS

Automated Drone Image Processing Workflows

Within the broader Photogrammetry & 3D Site Mapping Pipelines framework, automated drone image processing functions as the deterministic ingestion and reconstruction layer for archaeological documentation. This workflow is engineered for field archaeologists, heritage managers, Python GIS developers, and academic research teams requiring reproducible, audit-ready spatial outputs from raw UAV telemetry. The pipeline enforces strict coordinate reference system (CRS) validation, automated failure gates, and deterministic routing to downstream dense reconstruction stages.

Pipeline Architecture & Routing Topology

The orchestration layer operates as a directed acyclic graph (DAG) that standardizes image ingestion, parses EXIF/GPS telemetry, validates spatial metadata, and triggers the Structure-from-Motion (SfM) engine. Routing decisions are made programmatically based on image overlap metrics, GCP availability, and CRS compliance.

import os
import subprocess
import logging
from pathlib import Path
from typing import Optional, List
import exifread
import pyproj
from pyproj import CRS, Transformer

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s | %(levelname)s | %(message)s",
    datefmt="%Y-%m-%d %H:%M:%S"
)

# Version-pinned dependencies (see requirements.txt)
# odm==3.4.1 | pyproj==3.6.1 | exifread==3.0.0 | rasterio==1.3.9

def validate_image_set(image_dir: Path, min_count: int = 50, min_overlap: float = 0.75) -> List[Path]:
    supported_exts = {".JPG", ".jpg", ".TIFF", ".tiff", ".DNG"}
    images = [f for f in image_dir.iterdir() if f.suffix in supported_exts]
    if len(images) < min_count:
        raise ValueError(f"Insufficient imagery: {len(images)} found. Minimum {min_count} required for stable SfM.")
    logging.info(f"Validated {len(images)} images for ingestion.")
    return sorted(images)

def parse_gps_and_convert(image_path: Path, target_epsg: int = 32633) -> dict:
    """Extract WGS84 GPS and project to local UTM zone."""
    with open(image_path, "rb") as f:
        tags = exifread.process_file(f, details=False)
    
    lat = tags.get("GPS GPSLatitude")
    lon = tags.get("GPS GPSLongitude")
    alt = tags.get("GPS GPSAltitude")
    
    if not (lat and lon):
        raise RuntimeError(f"Missing GPS telemetry in {image_path.name}")
    
    # Convert DMS to decimal degrees
    def dms_to_dec(dms):
        vals = dms.values
        return float(vals[0]) + float(vals[1])/60 + float(vals[2])/3600
    
    wgs84_crs = CRS.from_epsg(4326)
    target_crs = CRS.from_epsg(target_epsg)
    transformer = Transformer.from_crs(wgs84_crs, target_crs, always_xy=True)
    
    east, north = transformer.transform(dms_to_dec(lon), dms_to_dec(lat))
    return {"east": east, "north": north, "alt_m": float(alt.values[0]) if alt else None}

def route_alignment(project_dir: Path, gcp_file: Optional[Path] = None, target_epsg: int = 32633) -> str:
    cmd = [
        "odm",
        "--project-path", str(project_dir),
        "--orthophoto-resolution", "5",
        "--dsm", "--dtm",
        "--feature-quality", "ultra",
        "--rerun-all"
    ]
    if gcp_file and gcp_file.exists():
        cmd.extend(["--gcp", str(gcp_file)])
    
    logging.info(f"Routing to SfM engine: {' '.join(cmd)}")
    try:
        result = subprocess.run(
            cmd, 
            check=True, 
            capture_output=True, 
            text=True, 
            timeout=3600
        )
        logging.info("Alignment and sparse cloud generation completed.")
        return result.stdout
    except subprocess.TimeoutExpired:
        raise RuntimeError("SfM alignment exceeded 60-minute timeout. Check image overlap and engine memory limits.")
    except subprocess.CalledProcessError as e:
        logging.error(f"Engine failure (exit {e.returncode}): {e.stderr}")
        raise RuntimeError("Photogrammetric alignment aborted. Verify lens calibration and GCP distribution.")

Geodetic Validation & CRS Enforcement

Geodetic integrity is non-negotiable in heritage contexts. UAV flight controllers typically log GPS coordinates in WGS84 (EPSG:4326) with ellipsoidal heights, which must be rigorously distinguished from orthometric heights (e.g., EGM96/EPSG:5703 or EGM2008/EPSG:3855) before dense reconstruction. Automated workflows must explicitly verify and transform coordinate reference systems using authoritative transformation grids.

The pipeline enforces a mandatory CRS validation gate prior to dense matching. If Ground Control Points (GCPs) are supplied, their coordinates are validated against the target projected CRS (typically UTM zones, e.g., EPSG:32633 for Mediterranean sites). For detailed methodologies on integrating survey-grade control networks and minimizing reprojection residuals, consult the dedicated guide on Aligning photogrammetry point clouds to real-world coordinates.

Transformation logic relies on pyproj (v3.6.1+) and the official EPSG Geodetic Parameter Dataset registry to ensure vertical datum shifts are applied deterministically. Field teams should always log the exact EPSG code used during flight planning and document any local grid transformations (e.g., EPSG:7415 for Irish National Grid) in the project metadata manifest.

Failure Gates & Downstream Routing

The pipeline implements explicit failure gates at three critical junctures:

  1. Ingestion Gate: Rejects datasets with <50 images, <70% forward overlap, or missing EXIF telemetry.
  2. Alignment Gate: Halts execution if the sparse cloud contains <500 tie points or exhibits RMS reprojection error >1.5 pixels.
  3. Geodetic Gate: Blocks dense reconstruction if GCP residuals exceed 2× ground sampling distance (GSD).

Upon successful alignment, outputs are routed to specialized downstream modules. The dense point cloud and orthomosaic are passed to Mesh Generation & Optimization for Ruins for topology simplification and artifact preservation. Concurrently, the aligned imagery bundle is staged for Texture Mapping & UV Alignment Automation to ensure photorealistic surface rendering without geometric distortion.

Routing is managed via a YAML configuration manifest that defines output directories, compression parameters, and archival checksums (SHA-256). This ensures full traceability for heritage compliance audits and academic peer review.

Environment Pinning & Reproducibility

Academic and institutional deployments require strict dependency isolation. The following requirements.txt pins all spatial and photogrammetric libraries to verified stable releases:

# Core Orchestration & Geospatial
pyproj==3.6.1
rasterio==1.3.9
numpy==1.26.4
shapely==2.0.4

# Metadata & EXIF Parsing
exifread==3.0.0
Pillow==10.2.0

# Engine & CLI Wrappers
odm==3.4.1
docker==7.0.0

Deployment should utilize a containerized runtime (Docker/Podman) with the official OpenDroneMap documentation as the baseline configuration. All pipeline executions must be wrapped in a version-controlled environment manager (e.g., conda or uv) to guarantee bitwise reproducibility across field campaigns and institutional servers.