Streaming Into Data Lake

Stream rows continuously into Apache Iceberg tables on S3 Tables (or standard Iceberg on a general purpose bucket) using Amazon Data Firehose with IcebergDestinationConfiguration. Covers the Firehose stream, the optional record-shaping Lambda (ProcessingConfiguration), the Lake Formation grants required for Firehose to write through Glue, and the three-phase CloudFormation deploy that gates the stream on grants existing. Triggers on: stream into data lake, Firehose to S3 Tables, Firehose to Iceberg, real-time ingestion to Iceberg, Firehose IcebergDestinationConfiguration, stream Kinesis to Iceberg, near real-time CDC to data lake, continuous append to Iceberg. Do NOT use for: batch file imports (use ingesting-into-data-lake), creating the destination Iceberg table itself (use creating-data-lake-table), the one-time Lake Formation onboarding (use setting-up-lake-formation), finding tables by fuzzy name (use finding-data-lake-assets), running queries (use querying-data-lake), Glue connections (Firehose does not use them; do not invoke connecting-to-data-source), Kinesis to Lambda to Redshift Data API direct writes (not Iceberg; recommend a different workflow), or producer-side Kafka or MSK setup.

mkdir -p ~/.claude/skills/streaming-into-data-lake
curl -L https://claudskills.com/skills/streaming-into-data-lake/SKILL.md \
  -o ~/.claude/skills/streaming-into-data-lake/SKILL.md

@misc{streaming-into-data-lake-2026,
  author    = {jaingxyz},
  title     = {Streaming Into Data Lake [Claude Code skill]},
  year      = {2026},
  publisher = {ClaudSkills},
  url       = {https://claudskills.com/skills/streaming-into-data-lake/}
}