Business Context
Modern photography workflows often span several disconnected systems.
Apple Photos may hold the authoritative personal library. Lightroom may manage creative edits. Filesystem exports may provide reliable access to originals. Websites and portfolios require another selection and publishing process. Meanwhile, years of photographs accumulate faster than they can be reviewed, tagged, organized, or rediscovered.
The underlying problem was not a lack of storage.
It was the absence of a dependable intelligence and curation layer across the entire photographic workflow.
I created LightKeeper to make a large personal image library more searchable, understandable, and useful without replacing the applications already trusted to manage originals and edits.
Challenge
The platform needed to solve several competing problems at once:
- Preserve Apple Photos and Lightroom as authoritative systems
- Catalog continuously exported originals without modifying them
- Maintain stable identity even when filenames or paths change
- Generate usable previews and extract technical metadata at scale
- Apply AI analysis without sending full-resolution originals to a model
- Keep paid AI processing observable, estimated, and budget-controlled
- Support human review rather than allowing AI to make final creative choices
- Remain useful when some originals are temporarily unavailable locally
- Run safely on self-hosted infrastructure with recoverable persistent data
- Separate creative review from operational processing and maintenance
A conventional photo gallery would not be enough.
The system required the characteristics of a digital asset manager, an AI processing pipeline, a curation workspace, and an operational platform while remaining restrained enough for one person to understand and operate.
My Role
I conceived, architected, and developed LightKeeper as an independent product.
My responsibilities included:
- Product definition and workflow design
- Application and data architecture
- Python package and command-line development
- SQLite schema design and migration strategy
- Web workspace and operations-console design
- Filesystem ingestion and stable source mapping
- Preview generation and EXIF/GPS extraction
- OpenAI integration and structured annotation schemas
- AI cost estimation, usage accounting, and safety controls
- Apple Photos catalog integration and conservative metadata write-back design
- Rootless Podman containerization
- Cloudflare Access deployment
- Backup, restore, health-check, and recovery workflows
- Automated testing and operational documentation
The work combined product thinking, platform engineering, data governance, creative workflow design, and AI systems integration.
Architectural Approach
LightKeeper uses a local-first catalog model.
It does not attempt to become the master repository for every photograph. Instead, it maintains a durable catalog of source identities, previews, metadata, annotations, processing history, curation state, and generated outputs.
The core flow is:
Apple Photos / Lightroom
↓
continuous original export
↓
LightKeeper catalog + stable source identity
↓
previews + EXIF/GPS metadata
↓
AI annotations + search
↓
human review + candidates
↓
original retrieval + website exportThis separation protects the original library while allowing LightKeeper to become the intelligence layer around it.
Apple Photos remains authoritative for Apple Photos assets. Lightroom remains authoritative for Lightroom-managed originals and edits. The production filesystem export remains read-only inside the application container.
Safe and Resumable Ingestion
Continuous photo exports create operational edge cases that simple importers do not handle well.
A file may still be downloading, changing, or being written when a scan begins. Previously cataloged paths should not consume new-work limits. Duplicate content may exist at multiple source locations. A source may disappear temporarily without the photograph itself becoming invalid.
I designed the ingestion process to be idempotent and resumable.
The pipeline:
- Identifies stable, previously unseen source paths.
- Defers files that remain inside a configurable settle interval.
- Hashes eligible files and establishes durable photo and source records.
- Detects duplicate-content sources without silently merging them.
- Generates missing JPEG previews.
- Extracts dimensions, capture date, camera, lens, exposure, and GPS metadata.
- Reports imported, previously cataloged, still-exporting, changed, duplicate, and failed files separately.
Known paths are filtered before hashing and before applying batch limits. This allows repeated scans to remain efficient even while an external exporter is continuously adding photographs.
Human-in-the-Loop AI
AI was introduced as an assistant to creative judgment, not a replacement for it.
LightKeeper uses a two-stage review model:
Broad Scan
A lower-cost first pass generates:
- Visual descriptions
- Mood and style observations
- Subject and composition tags
- Visible-content annotations
- Preliminary website, portfolio, and homepage scores
Candidate Review
Photographs intentionally selected by the user receive a more detailed review covering:
- Strengths and weaknesses
- Composition
- Lighting
- Clarity and texture
- Architecture and visual structure
- Potential use cases
- Refined publication scores
Only generated JPEG previews are sent for analysis. Full-resolution originals remain local.
Paid AI runs require an estimate and explicit approval. Each completed run records the model, duration, failures, token usage, reasoning usage when available, estimated cost, and calculated actual cost.
This made the AI workflow inspectable and financially bounded rather than an opaque background process.
Creative Workspace and Operational Control
The web application deliberately separates photography work from system operations.
The photo workspace supports:
- Browsing and filtering
- Full-screen review
- Search across filenames, annotations, tags, locations, people, dates, cameras, lenses, and notes
- Candidate marking and bulk curation
- Side-by-side comparison
- Editable dates, locations, coordinates, and tags
- Website, portfolio, and homepage scoring
- Contact sheets and candidate exports
- Authenticated access to available originals
A separate operations console manages:
- Background pipeline execution
- Live progress, rates, and failures
- Preview and metadata repair
- Apple Photos export supervision
- AI readiness and credential verification
- Processing history
- Backups and maintenance
- Cost and batch-size safeguards
Catalog jobs are serialized through a single observable runner, while browsing and curation remain available during processing.
This separation prevents operational controls from overwhelming the creative experience while still making system state easy to understand.
Identity, Governance, and Write-Back
Stable identity is essential when multiple systems refer to the same image.
LightKeeper separates a photograph’s internal record from its source mappings. It does not assume that a filename, date, or visual resemblance is sufficient to establish source identity.
For Apple Photos, metadata write-back is only permitted when the catalog has a verified Apple Photos asset UUID. Export filename suffixes are decoded and validated against an Apple Photos catalog before a mapping is created. Malformed, unknown, absent, or conflicting identifiers remain unmapped.
Approved changes can be queued for a Mac-side worker, which applies dates, coordinates, keywords, and captions and records each success or failure for audit and retry.
The Linux deployment never attempts to directly automate the macOS Photos application.
This conservative design favors explainability and recoverability over clever but unreliable matching.
Production Deployment
The production application runs as a rootless Podman container on a private media server and is exposed through Cloudflare Access.
The deployment includes:
- Read-only bind mounting of original photographs
- Persistent SQLite catalog storage
- Persistent previews, exports, configuration, and secrets
- Health endpoints and operational status commands
- Rootless container isolation
- Restricted external access through Cloudflare
- Backup and restore workflows
- Reproducible dependency management
- Automated test and compile validation
Originals, previews, databases, API keys, and runtime outputs are explicitly excluded from source control.
The architecture is intentionally modest: one understandable application, one durable catalog, observable jobs, and recoverable data.
Current Capabilities
LightKeeper currently provides:
- Continuous filesystem ingestion from Apple Photos exports
- Stable photo and source records
- Content hashing and duplicate-source detection
- JPEG preview generation
- EXIF, dimension, capture-date, camera, lens, exposure, and GPS extraction
- Searchable human and AI annotations
- Two-stage AI analysis
- Candidate selection and comparative review
- Website, portfolio, and homepage scoring
- Cost estimation and actual AI usage accounting
- Contact sheets and candidate exports
- Original availability auditing and authenticated retrieval
- Conservative Apple Photos identity mapping
- Queued Apple Photos metadata write-back
- Containerized production deployment
- Backup, restore, health, and maintenance workflows
Planned capabilities include Lightroom source mapping, one-click Lightroom edit handoff, published edit versions, semantic vector search, and visual similarity.
Product Outcomes
The platform transformed a fragmented personal photography workflow into a coherent, inspectable system.
Key outcomes include:
- Established a central catalog without replacing Apple Photos or Lightroom
- Preserved original ownership and nondestructive creative workflows
- Reduced the effort required to rediscover and evaluate older photographs
- Created a repeatable path from ingestion through review and website curation
- Introduced AI analysis without exposing full-resolution originals
- Kept model usage financially bounded through estimates and explicit approval
- Made long-running photo processing safe, resumable, and observable
- Created durable source mappings and audit trails for metadata changes
- Deployed the platform as an accessible, self-hosted production service
- Built a foundation for Lightroom integration and future semantic discovery
Consulting Insight
AI products are often designed around what a model can generate.
LightKeeper was designed around what the user must continue to trust.
That changed the architecture.
Originals remain under the control of established photo applications. AI sees smaller derivatives. Source identity must be proven before metadata is written back. Paid processing is estimated before execution. Every result remains open to human review.
The most useful AI systems do not simply add intelligence.
They introduce intelligence without weakening ownership, control, or recoverability.
Executive Takeaway
LightKeeper demonstrates how AI can be added to a complex personal media workflow without surrendering control of the underlying assets.
By combining a local-first catalog, safe ingestion, structured metadata, observable AI processing, human curation, conservative write-back, and self-hosted operations, the platform turns a passive photo archive into an active creative resource.
The result is not another place to store photographs.
It is a system for understanding what is already there, deciding what matters, and carrying selected work forward with intention.
Reflection
LightKeeper began with a personal frustration: meaningful photographs were becoming harder to find as the library grew.
The obvious answer would have been to build another gallery or upload photographs into another cloud service.
Instead, the project became an exercise in restraint.
The strongest architecture was not the one that tried to own everything. It was the one that respected the systems already doing their jobs well and added the missing capabilities between them.
This project reinforced a principle that applies far beyond photography:
Good platforms do not create value by centralizing every responsibility.
They create value by establishing trustworthy relationships between systems, making hidden information useful, and helping people make better decisions.