ICAO Aircraft Ingestion CLI

---

Command-line interface for ingesting, processing, exporting, and synchronizing ICAO aircraft variant, aircraft type, and manufacturer datasets within the AircraftOps ingestion framework.

---

System Responsibilities

• Ingest aircraft variant records from configured ingestion sources
• Deduplicate manufacturer and model combinations
• Generate normalized aircraft type datasets
• Generate normalized manufacturer datasets
• Persist datasets to the AircraftOps repository layer
• Export datasets to storage destinations
• Synchronize records with the search indexing pipeline
• Generate NER rule sets derived from aircraft variant datasets

---

Execution Pipeline

ICAO Ingestion Command
        ↓
Source Load (Live / Local / DynamoDB)
        ↓
Variant Record Normalization
        ↓
Manufacturer / Model Deduplication
        ↓
Aircraft Type Generation
        ↓
Manufacturer Extraction
        ↓
Repository Persistence (Variants, Types, Manufacturers)
        ↓
Optional Search Index Synchronization
        ↓
Optional Rule Generation

All ingestion operations operate through the AircraftOps repository abstraction, ensuring consistent batch writes, pagination-aware scans, and transactional dataset handling across ingestion pipelines.

---

Command Groups

ingest

Run the full ICAO ingestion pipeline, including variant ingestion, deduplication, dataset generation, and persistence.

pipenv run python cli -m icao ingest
    [--source live|local|s3]
    [--save-dynamodb]
    [--save-local]
    [--save-s3]

---

purge

Delete all ICAO variant, aircraft type, and manufacturer records from the primary datastore.

pipenv run python cli -m icao purge

---

typesense-sync

Scan ICAO repository datasets and enqueue indexing jobs into the search indexing pipeline.

pipenv run python cli -m icao typesense-sync

---

export

Export ICAO datasets from the datastore into configured storage destinations.

pipenv run python cli -m icao export
    [--save-local]
    [--save-s3]

---

export-rules

Generate aircraft entity recognition rules derived from variant datasets using sharded model reads and export the generated ruleset.

pipenv run python cli -m icao export-rules
    [--save-local]
    [--save-s3]

---

Data Outputs

The ingestion pipeline produces three normalized datasets that form the canonical ICAO aircraft registry layer within AircraftOps.

Dataset	Description
ICAOTypeVariantModel	Aircraft manufacturer-model variant records
ICAOAircraftTypeModel	Normalized aircraft type entries derived from variants
ICAOManufacturerModel	Normalized manufacturer entities extracted from variant datasets

---

Repository Layer

All persistence operations are performed through the AircraftOps repository layer using model-managed repository objects accessible via:

.objects_sync

The repository layer is responsible for:

• Batch persistence operations
• Pagination-aware scanning
• Conditional updates and upserts
• Bulk deletion workflows
• Export and synchronization pipelines

---

Operational Notes

• The ingestion pipeline supports live ingestion and replay from stored datasets
• Deduplication ensures canonical manufacturer / model normalization across ingestion runs
• Dataset generation operations are deterministic and idempotent
• Search indexing operations are asynchronous via queue-based fanout
• Rule generation operates using sharded model reads to support large variant datasets

---