AircraftOps
Search
Documentation
Posts

S3 Object Storage

Dataset → Object Storage

Technical reference for Amazon S3 buckets used by the AircraftOps platform for raw uploads, processed media assets, and audit log storage.

Storage Architecture

Client Upload / Ingestion Pipeline
        ↓
Raw Upload Bucket
        ↓
Image Processing Pipelines
        ↓
Processed Media Bucket
        ↓
Application Delivery / Media Access

Object storage is separated into ingestion-stage and processed-stage buckets to support secure upload workflows, controlled processing pipelines, and long-term media persistence.

RawUploadBucket

Stores user-uploaded or ingestion-stage media files before processing. Uploads to this bucket trigger downstream processing pipelines through event-driven queue notifications.

  • Purpose: temporary staging storage for raw ingestion uploads
  • Event triggers: object-created events publish messages to processing queues
  • Lifecycle policy: automatic expiration of staged uploads after short retention
  • CORS: enabled for direct client upload workflows

ProcessedImagesBucket

Stores processed and validated aircraft media assets after ingestion pipelines complete transformation, validation, and metadata enrichment. This bucket serves as the long-term storage layer for curated platform media.

  • Purpose: long-term storage of processed image assets
  • Access model: restricted public access policies
  • Integration: referenced by image processing and indexing pipelines
  • Retention: persistent storage with retention-protection policies

AuditLogBucket

Stores structured application audit events delivered through Firehose-based analytics pipelines. Audit logs are written in analytics-optimized formats for long-term retention and compliance workflows.

  • Purpose: audit and analytics event storage
  • Data format: columnar analytics-friendly storage format
  • Lifecycle policy: automatic transition to archival storage tiers
  • Integration: downstream analytics querying through data catalog services

Operational Notes

  • Object storage is segmented by ingestion stage and lifecycle responsibility
  • Event-driven notifications trigger processing workflows automatically
  • Lifecycle policies reduce long-term storage costs for temporary ingestion artifacts
  • Processed media storage is retained independently of ingestion pipeline lifecycles
  • Audit storage is optimized for compliance retention and analytics workloads
← Redis Metrics Datastore Typesense Search Datastore →