Typesense Indexing Worker
This Lambda worker processes SQS indexing events and performs batched synchronization operations against the Typesense search cluster. It is responsible for document upserts, document deletions, synonym upserts, and synonym deletions, ensuring search indexes remain consistent with the platform datastore.
- Consume SQS indexing events generated by upstream systems
- Batch documents and synonyms by collection for efficient processing
- Execute Typesense document upsert and delete operations
- Execute Typesense synonym upsert and delete operations
- Log batch-level summaries to simplify debugging and observability
- The Lambda receives a batch of SQS messages.
- Each message is parsed and grouped by:
- Collection name
- Operation type (upsert / delete / synonym actions)
- A batch summary is logged for visibility into indexing workload size.
- Batched operations are executed in strict order:
- Document upserts
- Document deletes
- Synonym upserts
- Synonym deletes
Documents are grouped per collection and imported using the Typesense
bulk import API with the upsert action. This ensures existing
documents are updated while new documents are inserted without requiring
separate existence checks.
Deletions are executed using a filter-based bulk delete query targeting the document ID list. This enables multiple records to be removed in a single request, improving throughput and minimizing indexing latency.
Synonym records are updated individually through the Typesense synonyms API, allowing search behavior adjustments without reindexing entire collections.
Synonym removal operations are processed explicitly to ensure search behavior changes propagate immediately and consistently across query nodes.
This worker forms the dedicated search-indexing execution layer of the platform. By decoupling indexing from core data mutation workflows using SQS, the system achieves independent scaling, controlled retry semantics, and consistent eventual synchronization between the primary datastore and the search engine.