Index

Tone mapping (adaptive LUT pipeline)¶

Naming note: this feature is best described as tone-mapping (or "color grading"). "Color-correction" is too specific.

This module is a port of the adaptive LUT grading pipeline from image-tools, exposed as FastAPI endpoints under "/tone-mapping".

Workflows¶

Production workflow:

studio-frontend ← → studio-backend ← → python-backend ↔ mongodb (for config) / aws s3 (for cache/references)

Testing workflow:

image-tools/tonemapping-fe (react app) ← → python-backend ↔ mongodb (for config) / aws s3 (for cache/references)

Testing vs production differences: - image-tools/tonemapping-fe: file I/O, base64 payloads, no "Cache Snapshot" button, no config dropdown/endpoint, all manual adjustments client-side - Production: asset I/O, asset URLs in responses, cache diagnostics, config selection via config_name parameter

What's in this module¶

Manual adjustments: temperature/tint/exposure/brightness/contrast + master offsets + RGB curves.
Adaptive auto-correct: scene analysis → reference matching → adaptive LUT blend → optional preset emission.
Preset application: apply a previously produced adaptive preset (compressed .npz, transported as base64).
Cache diagnostics: inspect the on-disk cluster cache state.

Where things live (python-backend)¶

HTTP API: app/tone_mapping/tone_mapping_router.py
Request/response schemas: app/tone_mapping/tone_mapping_schemas.py
Service layer: app/tone_mapping/tone_mapping_service.py (ToneMappingService)
Workflow logic: app/tone_mapping/tone_mapping_workflow.py (ToneMappingWorkflow)
Helper utilities: app/tone_mapping/tone_mapping_helper.py (ToneMappingHelper)
Reference clustering: app/tone_mapping/tone_mapping_reference_clustering.py (ReferenceClusteringService)
Core primitives: app/tone_mapping/core/* (scene analysis, LUT math, curve math, adjustments)
Cache helpers: app/tone_mapping/cache/*
Preset/metadata I/O: app/tone_mapping/io/*
MongoDB config store: app/tone_mapping/tone_mapping_config_store.py

Data layout on disk¶

This port uses a deployment-friendly data root:

data/tone-mapping/references/ (reference before/after pairs)
data/tone-mapping/cache/
clusters/ (reference pool cache + JSON mirrors)
scene/ (scene cache + LUT artifacts)

See app/tone_mapping/config/paths.py.

Cache and reference syncing: References and cache directories are synced with S3 at server startup (non-blocking). See app/tone_mapping/io/s3_sync.py.

Key environment variables¶

Base64 image payload limits:
TONE_MAPPING_MAX_IMAGE_MB (default 25)
TONE_MAPPING_MAX_IMAGE_PIXELS (default 120000000)
TONE_MAPPING_MAX_IMAGE_DIMENSION (default 20000)
Mongo config store:
MONGODB_URI_STUDIO
MONGODB_TONE_MAPPING_DB (default renderboard)
TONE_MAPPING_CONFIG_COLLECTION (default toneMappingConfig)
MONGODB_TONE_MAPPING_TIMEOUT_MS (default 5000)
Optional S3 sync:
S3_TONE_MAPPING_BUCKET_NAME (default color-grading)
AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION
IMAGE_TOOLS_S3_CONNECT_TIMEOUT, IMAGE_TOOLS_S3_READ_TIMEOUT, IMAGE_TOOLS_S3_MAX_ATTEMPTS

Endpoints¶

See app/tone_mapping/tone_mapping_router.py / tone_mapping_schemas.py.

Auto-correct: POST /tone-mapping/auto-correct
Apply manual adjustments: POST /tone-mapping/adjustments
Apply a serialized adaptive preset: POST /tone-mapping/presets/apply
Cache introspection: GET /tone-mapping/cache
List MongoDB-backed configs (with default fallback): GET /tone-mapping/configs

Sample requests¶

# Manual adjustments
curl -X POST http://localhost:8000/tone-mapping/adjustments \
  -H 'Content-Type: application/json' \
  -d '{
    "image": "data:image/png;base64,...",
    "adjustments": {
      "color_temperature": 12.0,
      "color_tint": -5.0,
      "exposure": 0.25,
      "brightness": 0.10,
      "contrast": 0.15,
      "master_offsets": [0, 0, 0, 0],
      "curve_sliders": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    }
  }'

# Adaptive auto-correct using config "default" (Mongo optional)
curl -X POST http://localhost:8000/tone-mapping/auto-correct \
  -H 'Content-Type: application/json' \
  -d '{
    "image": "data:image/png;base64,...",
    "config_name": "default"
  }'

# Apply an adaptive preset (.npz bytes base64-encoded)
curl -X POST http://localhost:8000/tone-mapping/presets/apply \
  -H 'Content-Type: application/json' \
  -d '{
    "image": "data:image/png;base64,...",
    "preset_data": "<base64 of adaptive_preset.npz>",
    "preset_name": "adaptive_preset.npz"
  }'

Data model (what the endpoints accept/return)¶

Images¶

Requests use a base64 image string. Both raw base64 and data URLs (e.g. data:image/png;base64,...) are accepted (the helper strips an optional header).

Size/abuse guards are applied in app/tone_mapping/tone_mapping_helper.py:

TONE_MAPPING_MAX_IMAGE_MB (base64-length estimate; default 25MB)
TONE_MAPPING_MAX_IMAGE_PIXELS (PIL decompression-bomb guard; default 120M)
TONE_MAPPING_MAX_IMAGE_DIMENSION (default 20000)

Curves¶

The API represents curves using a fixed-length slider array curve_sliders of length 12. Conversion is implemented in app/tone_mapping/ui/curves.py. If curve_sliders is missing or wrong-length, the backend falls back to defaults.

Presets¶

POST /tone-mapping/auto-correct may return preset: { name, data } when options.include_preset=True.
POST /tone-mapping/presets/apply accepts preset_data (base64 of .npz bytes) and optional preset_name.

Auto-correct vs preset application (important distinction)¶

Auto-correct (/tone-mapping/auto-correct) runs the full pipeline:
reference-pair discovery → optional clustering/pool → adaptive LUT selection → application → optional preset emission.
It also returns inferred “manual” adjustment parameters (exposure/temperature/curves, etc.) as a convenience for UI.
Apply preset (/tone-mapping/presets/apply) reuses a previously serialized adaptive preset.
It does not redo reference discovery/clustering.
Applying a preset to the same source image yields the closest match; applying it to other images is effectively “transfer this grade”, and results will vary with content.

Adaptive pipeline inputs (high level)¶

The main knobs are in AutoCorrectOptions (app/tone_mapping/tone_mapping_schemas.py):

LUT selection/blending: grid_size, top_k, temperature, strength
Caching / clustering: use_cache, cluster_references, clusters, max_references
Quality/safety: protect_memory_colors, detect_overcorrection, tonal_separation, normalize_similarity
Performance: fast_mode
Stability: seed (else derived deterministically from the input image)
Preset output: include_preset
Reference source: pairs_dir

pairs_dir is validated to prevent path traversal; it must resolve within REFERENCES_DIR.

default on-disk references roo: data/tone-mapping/references/.

Reference pairs + caching system¶

Reference pairs¶

Reference pairs are discovered by filename convention in a directory:

*-before.<ext> + *-after.<ext>

(see ReferenceSelectionService.detect_pairs_from_directory in app/tone_mapping/pipeline.py).

Reference pool cache (cluster cache)¶

Stores clustered reference pools keyed by the exact list of reference paths + mtimes + target cluster count.
Writes a pickle payload and a JSON mirror for inspection.
Implementation: app/tone_mapping/cache/reference_pool_cache.py.

Scene analysis cache¶

Stores per-image analysis metrics on disk (pickle + JSON mirror) and keeps an in-memory cache per process.
Implementation: app/tone_mapping/cache/scene_analysis_cache.py.

Cache diagnostics endpoint¶

GET /tone-mapping/cache returns:

total_references: count of available *-after.* files under data/tone-mapping/references/
cluster_summary: human-readable per-cluster counts (best-effort)
cache_files: rows describing the current cluster-cache artifacts

(see app/tone_mapping/io/metadata.py).

Important: caches are stored using pickle, so cache directories must be treated as trusted local state.

MongoDB config model ("configs")¶

A "config" is a named, MongoDB-stored set of AutoCorrectOptions that callers can reference via config_name on POST /tone-mapping/auto-correct.

Env wiring (note: this repo uses MONGODB_URI_STUDIO):
MONGODB_URI_STUDIO (required to use Mongo)
MONGODB_TONE_MAPPING_DB (default renderboard)
TONE_MAPPING_CONFIG_COLLECTION (default toneMappingConfig)
MONGODB_TONE_MAPPING_TIMEOUT_MS
If Mongo is missing/unreachable, GET /tone-mapping/configs returns a single built-in default config.
The reserved config name "default" must be unique; duplicates raise DuplicateDefaultConfigError.

S3-backed sync (references + cache)¶

We sync data/tone-mapping/references/ and data/tone-mapping/cache/ with an S3-compatible bucket.

S3 client: app/third_party_services/s3_service.py
Sync logic/decorators: app/tone_mapping/io/s3_sync.py

Parallelism / performance¶

The core pipeline is synchronous and CPU-heavy. The API runs it via thread offloading (asyncio.to_thread(...)) so the FastAPI event loop stays responsive. Running multiple workers (or issuing concurrent requests) enables parallel processing; there is no dedicated batch endpoint.

Known gaps / caveats¶

Asset I/O (Backblaze/B2): input/output image storage is not wired; the API currently expects base64 payloads.
Face detection / "face recognition": not integrated; "memory color" protection is heuristic (skin/sky/grass) rather than explicit face models.
fast_mode: legacy quality knob (reduced scene-analysis fidelity) and likely slated for removal.
Pickle caches: treat cached artifacts as trusted.

Preset I/O (what the files contain)¶

Presets are serialized as compressed NumPy archives (.npz) and transported as base64.

They contain:

the selected LUT tables (and grid sizes)
weights/similarities/indices
confidence
a JSON meta_json payload containing:
preset version
pipeline parameters
selection scene types
optional metadata (including reference paths)

See ToneMappingHelper.serialize_adaptive_preset_to_bytes and app/tone_mapping/io/presets.py.

If you need "manual adjustments" in addition to the LUT preset, persist the adjustments returned by auto-correct alongside the preset blob (today they are separate in the API response).