Amazon OpenSearch Service: The Search, Analytics & Vector Intelligence Hub

Use OpenSearch k-NN vector search as the retrieval layer in a RAG pipeline. Amazon Bedrock generates embeddings (via Titan Embeddings or third-party models), stores them in OpenSearch vector search collections, and retrieves semantically similar documents at query time to augment LLM prompts. OpenSearch is the CORRECT service for vector similarity search — not DynamoDB, not RDS.

Firehose delivers streaming data (logs, events, clickstreams) directly to OpenSearch domains without custom code. This is the canonical AWS log analytics architecture: source → Kinesis Data Streams (optional) → Firehose → OpenSearch → Dashboards. Firehose handles buffering, batching, and retry logic automatically.

Lambda functions triggered by DynamoDB Streams, S3 events, or SNS/SQS messages transform and index documents into OpenSearch. Common pattern for keeping OpenSearch in sync with a primary data store (DynamoDB or RDS) — OpenSearch serves as a search replica, not the source of truth.

Use Index State Management (ISM) to automatically migrate aging indices from hot → UltraWarm → cold (S3). For compliance, raw logs are archived to S3 Glacier; OpenSearch cold storage links to S3 Standard. Rehydrate cold indices back to warm/hot for ad-hoc analysis.

CloudWatch receives OpenSearch cluster metrics (ClusterStatus.red/yellow, JVMMemoryPressure, SearchLatency, IndexingLatency, FreeStorageSpace). Set CloudWatch Alarms on these metrics to trigger SNS notifications or Auto Scaling actions. This is the primary monitoring mechanism — NOT VPC Flow Logs or CloudTrail for performance issues.

RDS remains the authoritative OLTP store; OpenSearch provides full-text search capabilities that RDS LIKE queries cannot efficiently serve. Lambda or DMS captures CDC events from RDS and syncs to OpenSearch. This is the correct pattern when questions ask about adding search to an existing RDS application.

SageMaker hosts custom embedding models or reranking models; inference results (embeddings) are stored in OpenSearch for vector search. For Learning to Rank, SageMaker trains ranking models deployed as OpenSearch plugins. This pattern avoids the need to build custom infrastructure for ML-augmented search.

Textract extracts text and structured data from PDFs/images → Bedrock generates embeddings → OpenSearch stores vectors and text for hybrid search (keyword + semantic). This enables enterprise document Q&A and compliance search use cases.

Amazon Cognito User Pools authenticate users; Cognito Identity Pools provide temporary AWS credentials for OpenSearch Dashboards access. This enables corporate SSO without exposing OpenSearch endpoints publicly. Alternative: SAML 2.0 integration for direct IdP federation.

Comprehend extracts entities, sentiment, and key phrases from text documents; these enrichments are stored as additional fields in OpenSearch documents, enabling faceted search by entity type, sentiment filtering, and improved relevance. NOTE: Comprehend does NOT generate embeddings — use Bedrock Titan Embeddings or SageMaker for that.