Heritage GIS

Generating Cloud-Optimized GeoTIFF Orthomosaics

A finished site orthomosaic that lives as a plain GeoTIFF is an archival liability: to draw any corner of it, a client must open and parse the whole file, so a 12 GB standing-remains survey stalls QGIS and cannot be served from object storage at all. Converting it to a Cloud-Optimized GeoTIFF (COG) fixes this without changing a single pixel value — the same raster is re-laid-out with internal tiling and embedded overviews so an HTTP range request fetches only the bytes the viewport needs. This guide, part of orthomosaic generation and tiling, gives the exact gdal_translate invocation, compression choices, and the validation that proves the file is genuinely cloud-optimized rather than merely renamed.

Plain GeoTIFF versus Cloud-Optimized GeoTIFF layout A plain strip-organised GeoTIFF compared with a COG that adds internal tiles and overview pyramid for range-request access. Why COG streams and plain GeoTIFF does not Plain GeoTIFF read whole file to draw a corner Internal tiling 512x512 blocks + overview pyramid Range request fetch only visible bytes

Context & When to Use

Reach for a COG whenever an orthomosaic will be hosted rather than hand-delivered: on an S3-compatible archive, behind a heritage web viewer, or shared with a partner who will stream it into QGIS over HTTPS. The trade-off is deliberate. COG conversion spends processing time and a modest storage overhead (the overview pyramid adds roughly 33% to the file) to buy near-instant partial reads and server-free hosting. If your only consumer is a desktop analyst working from a local disk, a tiled internal GeoTIFF is enough and the overviews are optional. Compression choice is the real decision: DEFLATE and LZW are lossless and correct for the master archive record; JPEG compression roughly quarters the size but discards data and must never be applied to a raster you will later measure or re-analyse. For a published visual base map, JPEG at quality 85 is defensible; for the preservation master, stay lossless.

Implementation

The reliable path is a single gdal_translate -of COG, which builds overviews and the tiled layout in one pass. The following wrapper runs the conversion, then re-validates so a broken file never reaches the archive.

# requirements.txt
# rasterio==1.3.9
# rio-cogeo==5.1.1
# GDAL command-line 3.8.4 on PATH
# rasterio==1.3.9, rio-cogeo==5.1.1, GDAL 3.8.4
import subprocess
from pathlib import Path
from rio_cogeo.cogeo import cog_validate

def make_cog(src: Path, dst: Path, *, lossless: bool = True) -> None:
    """Convert a mosaicked GeoTIFF to a validated COG.

    lossless=True -> DEFLATE (preservation master).
    lossless=False -> JPEG q85 with YCbCr (published visual base only).
    substitute your site's EPSG upstream; this step preserves the input CRS.
    """
    creation = [
        "-of", "COG",
        "-co", "BLOCKSIZE=512",          # internal tile edge in pixels
        "-co", "OVERVIEWS=AUTO",         # build pyramid down to a single tile
        "-co", "NUM_THREADS=ALL_CPUS",
    ]
    if lossless:
        creation += ["-co", "COMPRESS=DEFLATE", "-co", "PREDICTOR=2", "-co", "LEVEL=9"]
    else:
        creation += ["-co", "COMPRESS=JPEG", "-co", "QUALITY=85",
                     "-co", "PHOTOMETRIC=YCBCR"]

    subprocess.run(["gdal_translate", *creation, str(src), str(dst)], check=True)

    valid, errors, warnings = cog_validate(str(dst))
    if not valid:
        raise RuntimeError(f"{dst} failed COG validation: {errors}")
    for w in warnings:
        print(f"COG warning: {w}")


if __name__ == "__main__":
    make_cog(Path("site_ortho.tif"), Path("site_ortho_cog.tif"), lossless=True)

The PREDICTOR=2 setting is a horizontal differencing filter that meaningfully improves DEFLATE on continuous-tone imagery; drop it for paletted rasters. BLOCKSIZE=512 matches most web tile requests, keeping range reads efficient without fragmenting the file with tiny blocks. OVERVIEWS=AUTO lets GDAL choose decimation levels down to a thumbnail, which is what a zoomed-out heritage map viewer needs.

If you prefer to build the pyramid explicitly before conversion — useful when you want a specific resampling for categorical overlays — run gdaladdo first:

# GDAL 3.8.4
gdaladdo -r average site_ortho.tif 2 4 8 16 32
gdal_translate -of COG -co COMPRESS=DEFLATE -co PREDICTOR=2 \
  -co OVERVIEWS=FORCE_USE_EXISTING site_ortho.tif site_ortho_cog.tif

Use -r average for continuous imagery and -r nearest for a classified raster where invented intermediate values would be wrong.

Verification

Two independent checks prove the file. First, gdalinfo must report the block size and an overview list:

# GDAL 3.8.4
gdalinfo site_ortho_cog.tif | grep -E "Block|Overviews"

Expected output resembles:

Band 1 Block=512x512 Type=Byte, ColorInterp=Red
  Overviews: 4096x3072, 2048x1536, 1024x768, 512x384, 256x192

Blocks reported as Block=8192x1 mean the file is strip-organised — not a COG — even if the extension says otherwise. Second, run the RFC-level validator, which checks internal byte ordering, not just structure:

# rio-cogeo 5.1.1
rio cogeo validate site_ortho_cog.tif

A pass prints site_ortho_cog.tif is a valid cloud optimized GeoTIFF. Assert both in your promotion gate: gdalinfo showing overviews and rio cogeo validate returning valid. A file that passes structure but fails ordering will still force full-file reads in practice.

Field note. When you rebuild a COG after re-balancing a flight, overwrite the destination path explicitly — GDAL will not silently update an existing file, and a stale site_ortho_cog.tif left in the archive is indistinguishable from the new one until someone measures a wall off last week's colours. Version the filename or delete before rewriting.

Common Errors & Fixes

  • ERROR 1: COG driver does not support update mode — you pointed gdal_translate -of COG at an existing output opened for update, or tried gdaladdo on a finished COG. COGs are write-once; regenerate from the source GeoTIFF instead of editing in place.
  • does not have overviews from rio cogeo validate — the source had none and you used OVERVIEWS=FORCE_USE_EXISTING. Switch to OVERVIEWS=AUTO, or run gdaladdo before translating.
  • ERROR 6: PHOTOMETRIC=YCBCR requires a source raster with 3 bands — you applied the JPEG/YCbCr recipe to a single-band or 4-band (RGBA) raster. Drop the alpha with gdal_translate -b 1 -b 2 -b 3 first, or use DEFLATE for non-RGB data.

Part of Photogrammetry & 3D Site Mapping Pipelines.