Cargando...
Cargando...
Stop guessing which Kinesis service to use — master the decision in minutes
Real-time control vs managed delivery vs SQL-powered insight — pick the right tool
| Feature | Kinesis Data Streams Real-time, custom, low-latency streaming | Firehose Fully managed delivery to storage | Analytics SQL or Flink analytics on streams |
|---|---|---|---|
Primary Purpose KDS = ingest & buffer; Firehose = deliver; Analytics = transform & analyze. These are complementary, not interchangeable. | Ingest and buffer real-time streaming data for custom consumers | Load streaming data reliably into AWS storage/analytics destinations | Run real-time SQL or Apache Flink applications on streaming data |
Management Model Firehose is the ONLY fully serverless option here. KDS requires shard management. This distinction is heavily tested. | You manage shards, scaling, and consumer throughput | Fully serverless and managed — zero infrastructure | Managed Apache Flink runtime; you write application code or SQL |
Data Retention CRITICAL TRAP: Firehose has NO replay capability. Only KDS allows consumers to re-read data from the stream. If replay is required, KDS is mandatory. | Default 24 hours; extendable up to 365 days (long-term retention) | No retention — data is delivered to destination, not stored in-stream | No native retention — reads from KDS or Firehose as source |
Replay / Reprocessing Replay = KDS. This is one of the most tested differentiators across SAA-C03, SAP-C02, and DEA-C01. | YES — consumers can replay data within retention window | NO — no replay; data delivered once to destination | NO native replay — depends on source stream (KDS) |
Latency Firehose is NOT real-time — minimum 60-second delivery delay. If the question says 'real-time' processing, Firehose alone is wrong. | Real-time: ~70ms latency (sub-second); Enhanced Fan-Out ~70ms per consumer | Near real-time: minimum 60-second buffer interval (or buffer size threshold) | Real-time to near real-time depending on application and source |
Scaling Model KDS On-Demand mode was introduced to reduce shard management burden — but Firehose still requires zero capacity planning. | Manual or auto scaling via shard splitting/merging; On-Demand mode available | Automatically scales — no capacity planning needed | Scales via Kinesis Processing Units (KPUs) — managed by service |
Supported Destinations Firehose destinations are heavily tested. Know S3, Redshift (via S3 COPY), and OpenSearch as the top three. Redshift is indirect via S3. | Custom consumers (EC2, Lambda, ECS, KCL apps, Analytics, Firehose) | S3, Redshift, OpenSearch, Splunk, HTTP endpoints, Datadog, New Relic, MongoDB, Snowflake, and more | KDS, Firehose, S3, Lambda (as output sinks) |
Data Transformation Firehose can invoke Lambda for lightweight transformation. Analytics is for complex stream processing logic — windowed aggregations, joins, anomaly detection. | None built-in — consumers handle transformation | Lambda-based transformation before delivery; format conversion (JSON→Parquet/ORC) | Full SQL (Apache Flink SQL) or Java/Python/Scala Flink applications |
Ordering Guarantee If ordering matters (e.g., financial transactions, IoT device state), use KDS with a consistent partition key strategy. | Per-shard ordering guaranteed using sequence numbers and partition keys | No strict ordering guarantee across delivery batches | Ordering depends on source stream (KDS provides it) |
Consumer Model Enhanced Fan-Out (EFO) gives each registered consumer dedicated 2 MB/s per shard throughput — critical for high-fan-out architectures. | Multiple consumers (up to 5 standard; Enhanced Fan-Out for dedicated throughput per consumer) | Single delivery stream to one destination (can fan-out via Lambda or multiple streams) | Single application reads from source; can write to multiple outputs |
Pricing Model Firehose has no idle cost for capacity — you pay only for data processed. KDS charges per shard-hour even when idle. Cost optimization: use Firehose for variable/unpredictable traffic. | Per shard-hour + PUT payload units (25KB chunks); Extended retention & EFO add cost | Per GB of data ingested; format conversion and VPC delivery add cost; NO shard costs | Per Kinesis Processing Unit (KPU) per hour; 1 KPU = 1 vCPU + 4 GB memory |
Data Sources / Producers Firehose can use KDS as a source — this is a common pipeline pattern: KDS (ingest/buffer) → Firehose (deliver to S3/Redshift). | Kinesis Agent, KPL, AWS SDK, IoT, CloudWatch Logs, DynamoDB Streams | Direct PUT, KDS as source, MSK (Kafka), CloudWatch Logs, IoT, WAF | KDS or Firehose as primary sources; reference data from S3 |
Built-in Encryption All three support encryption — not a differentiator. Focus on which service meets the functional requirement, not encryption capability. | Server-side encryption with AWS KMS at rest; TLS in transit | S3 SSE, KMS encryption for S3/Redshift destinations; TLS in transit | Encryption at rest for application state; TLS in transit |
Monitoring & Observability IteratorAgeMilliseconds on KDS is the key metric for consumer lag — high value means consumers can't keep up with producers. | CloudWatch metrics per shard (IncomingBytes, GetRecords.IteratorAgeMilliseconds) | CloudWatch metrics (DeliveryToS3.Success, IncomingBytes, ThrottledRecords) | CloudWatch metrics + Apache Flink dashboard for application health |
Use Case Fit | Real-time dashboards, event sourcing, complex custom consumers, replay needed, multiple independent consumers | Log archival to S3, data lake ingestion, Redshift loading, Splunk/OpenSearch delivery | Real-time fraud detection, anomaly detection, windowed aggregations, metric computation |
Serverless / No-Ops | Partial — On-Demand mode reduces ops but still requires monitoring | Fully serverless — zero infrastructure management | Managed runtime — Flink cluster managed by AWS |
Max Record Size Both KDS and Firehose share the 1 MB per-record limit. Records exceeding this must be split by the producer. | 1 MB per record | 1 MB per record | Depends on source stream record size limits |
Throughput Per Shard (KDS) CRITICAL: Standard consumers SHARE the 2 MB/s read throughput per shard. Enhanced Fan-Out gives EACH consumer a dedicated 2 MB/s per shard. | Write: 1 MB/s or 1,000 records/s per shard; Read: 2 MB/s per shard (shared across consumers) | N/A — no shard concept; auto-scales transparently | N/A — throughput governed by source stream |
Typical Exam Scenario Trigger Words Memorize these trigger words — exam scenarios are designed around them. Matching keywords to service is the fastest path to the correct answer. | 'replay', 'reprocess', 'multiple consumers', 'real-time', 'ordering', 'custom processing', 'millisecond latency' | 'load to S3', 'archive logs', 'deliver to Redshift', 'no code', 'managed', 'near real-time', 'Splunk' | 'SQL on streams', 'windowed aggregation', 'anomaly detection', 'real-time metrics', 'Apache Flink' |
Summary
Use Kinesis Data Streams when you need real-time data ingestion with replay capability, multiple independent consumers, or strict ordering guarantees. Use Firehose when you want a zero-management, near-real-time pipeline to deliver data to S3, Redshift, OpenSearch, or third-party destinations without writing consumer code. Use Managed Service for Apache Flink (Analytics) when you need to run SQL queries or complex Flink applications on streaming data for real-time analytics, aggregations, or anomaly detection.
🎯 Decision Tree
Need replay/reprocessing? → KDS. Need to load data into S3/Redshift/OpenSearch with no code? → Firehose. Need SQL or Flink analytics on a stream? → Analytics (MSF). Need both ingest+delivery? → KDS (source) + Firehose (sink). Need multiple independent consumers at scale? → KDS with Enhanced Fan-Out. Need lowest cost for simple log archival? → Firehose direct to S3.
REPLAY = KDS ONLY. If an exam scenario mentions replaying, reprocessing, or re-reading data from a stream, the answer is always Kinesis Data Streams. Firehose delivers data once and has no replay capability — this is the single most tested differentiator across all five certifications.
FIREHOSE IS NEAR REAL-TIME, NOT REAL-TIME. The minimum buffer interval of 60 seconds means Firehose cannot satisfy sub-second or true real-time requirements. If the question specifies 'real-time processing' or 'millisecond latency', eliminate Firehose. This trap appears on CLF-C02, SAA-C03, and DEA-C01.
STANDARD CONSUMERS SHARE 2 MB/s PER SHARD. With multiple consumers reading from the same KDS shard using standard GetRecords, they split the 2 MB/s read bandwidth. Enhanced Fan-Out (EFO) gives EACH registered consumer a dedicated 2 MB/s per shard via HTTP/2 push — critical for high-fan-out architectures. This appears on SAA-C03 and SAP-C02 as a throughput optimization question.
KDS → FIREHOSE IS A VALID AND COMMON PIPELINE. Firehose can use KDS as its source. This pattern gives you: KDS for real-time custom consumers + replay, AND Firehose for automatic delivery to S3/Redshift/OpenSearch — best of both worlds. Exam scenarios describing 'ingest + archive + real-time processing' often expect this combined architecture.
ANALYTICS SERVICE RENAMED: 'Kinesis Data Analytics' is now 'Amazon Managed Service for Apache Flink'. Both names appear in exam questions. The service supports Apache Flink applications (Java, Python, Scala) AND Flink SQL. It replaced the older SQL-only interface. Know both names to avoid being confused by older question phrasing.
FIREHOSE REDSHIFT DELIVERY IS INDIRECT: Firehose does NOT write directly to Redshift tables via JDBC. It first writes data to an S3 bucket, then issues a Redshift COPY command. This means there's an S3 intermediate step — relevant for architecture diagrams and data latency calculations on SAP-C02 and DEA-C01.
KDS ON-DEMAND MODE vs PROVISIONED MODE: On-Demand mode automatically scales shards based on traffic (up to a limit) — no manual shard management. Provisioned mode requires you to specify and manage shard count. On-Demand is more expensive per GB but eliminates operational overhead. Exam may ask which mode fits 'unpredictable traffic patterns'.
The #1 certification trap: Choosing Firehose for a 'real-time streaming' requirement because it has 'Kinesis' in its name and sounds like a streaming service. Firehose is a DELIVERY service with a minimum 60-second buffer — it is near real-time, not real-time. Candidates also frequently confuse Firehose's lack of replay with KDS capabilities. Always ask: Does the scenario need (1) replay? → KDS. (2) Sub-second latency? → KDS. (3) Zero-code delivery to S3/Redshift? → Firehose. (4) SQL analytics on the stream? → Managed Service for Apache Flink.
CertAI Tutor · CLF-C02, DOP-C02, SAA-C03, SAP-C02, DEA-C01 · 2026-02-22
Services
Comparisons