Solving the Quiet Friction Behind Cart Abandonment

Signal Extraction Infrastructure

Infrastructure: Amazon S3 → Amazon SQS → Amazon EKS Workers → Amazon Rekognition + Amazon Titan Multimodal Embeddings

The Build: The frontend parses EXIF data client-side during photo selection, capturing GPS coordinates, timestamps, camera models, and exposure settings, before computing a content hash per file. This payload requests a presigned PUT URL, and uploads land in an S3 prefix that emits event notifications to SQS. EKS workers drain the queue, calling DetectFaces with Attributes=ALL, DetectLabels with IMAGE_PROPERTIES for sharpness, brightness, contrast, and dominant colors, and DetectText. In parallel, Titan Multimodal Embeddings G1 produces a 1024-dimensional vector per image, normalized to unit length so similarity can be computed via dot product rather than cosine. All output, including Rekognition metadata, embeddings, and frontend data, is written to Aurora PostgreSQL, using structured columns for queryable fields and JSONB for nested label and face payloads.

The Impact: Rekognition pricing is per-image and independent of complexity. In our testing, a 4.7 MB photo returned in approximately 5 seconds at $0.00475, while a 13 MB frame with 33 detected faces returned in approximately 11 seconds at the same rate. Equivalent extraction through a multimodal LLM benchmarked at roughly twice the per-image cost, an order of magnitude lower throughput, and a bill that varies with subject count and prompt length.

Deterministic Candidate Reduction

Infrastructure: Amazon EKS + Aurora PostgreSQL with pgvector (HNSW)

The Build: Model invocation is gated behind a filtering tier running entirely on indexed metadata. Duplicate content hashes are dropped before analysis. Timestamps outside the event window and coordinates outside the venue cluster are deprioritized. Clustering is computed directly from EXIF GPS with a distance threshold, requiring no external geospatial API call. Exposure values predict unusable frames, and camera model sets the quality threshold per photo rather than applying a global cutoff, so a phone snapshot and a DSLR frame are scored against appropriate curves. A composite score across technical quality, emotional signal, professional merit, and narrative value ranks the survivors.

The Impact: A 150-photo upload reduces to approximately 50 candidates before the first Bedrock call. This is the primary cost control in the system. The agent tier's token consumption scales with candidate count, not upload size.

The Curation Ensemble

Infrastructure: Amazon Bedrock (Claude) + LangGraph on Amazon EKS + Aurora PostgreSQL with pgvector

The Build: Four agents evaluate the same candidate set concurrently, each in its own pod: technical quality, emotional significance, narrative flow, and visual harmony. Each receives identical input, including Rekognition metadata, EXIF data, and embeddings, and applies a distinct evaluation criterion. Agents query pgvector directly during evaluation. Because vectors sit beside relational metadata, a single query can combine HNSW similarity search with hard predicates on venue, timestamp, or face count, with no cross-store join. Final selection is a consensus across all four rather than a single verdict, and per-agent scores are persisted alongside the result.

The Impact: Retaining per-agent scores makes disagreement measurable and attributable. When a selection is wrong, the divergent score identifies which criterion is miscalibrated, allowing each agent's prompt and weighting to be tuned independently rather than adjusting a single combined prompt. Inter-agent agreement is tracked as a first-class metric.

Layout and Conversational Refinement

Infrastructure: Amazon Bedrock + Amazon Location Service + Amazon ElastiCache for Redis + Amazon S3

The Build: Selected images are matched against the template catalog on aspect-ratio fit, composition notes, and Rekognition dominant-color data for facing-page coordination. Amazon Location Service ReverseGeocode resolves coordinates to venue name and place category. The category drives chapter segmentation, and the name is used in caption generation. Because results are persisted, these calls set IntendedUse=Storage. The API defaults to SingleUse, and storage carries a higher rate, so responses are cached in Redis keyed on a geohash prefix rather than exact coordinates, collapsing a venue's worth of photos to a single billable request. The chat surface handles refinement. The conversation agent classifies each request and routes deterministic operations, such as photo swaps, reordering, and frame resizing, to layout code without a model call, reserving Bedrock invocations only for requests that change creative judgment. Session state and undo/redo checkpoints live in Redis. User style preferences and prior book schemas persist in PostgreSQL.

Governance and Guardrails

All user actions and uploads enter through durable SQS queues, with separate queues for ingestion and agent invocation so ingest volume cannot back-pressure the agent tier. AWS Step Functions sequences the multi-stage pipeline with idempotency tokens, retry policies, and dead-letter queues, ensuring a throttled Bedrock call never produces partial state or duplicate work. Layout output is schema-validated before rendering. Invalid structures are trapped and retried rather than passed downstream. Observability is split by layer. Amazon CloudWatch covers EKS scaling, queue depth, pgvector query latency, and S3 throughput, while LLM-specific tracing captures per-phase token attribution, prompt latency, and inter-agent agreement. Amazon Cognito provides identity and API Gateway authorization, AWS Secrets Manager holds credentials, and the processing tier runs entirely in private subnets with no public data-plane exposure.

Acceptance rate, the proportion of AI-selected photos a user retains without override, is the system's primary quality metric, instrumented from the start so that speed and cost gains cannot mask poor selections.