DEA-C01 — AWS Certified Data Engineer – Associate Cheat Sheet

High-yield DEA-C01 thinking pattern:

1. Ingest: batch files, database CDC, SaaS, stream, events, or messages.
2. Store: Amazon S3 data lake, Amazon Redshift warehouse, operational database, or search/time-series target.
3. Catalog and govern: AWS Glue Data Catalog, AWS Lake Formation, IAM, AWS KMS.
4. Transform: AWS Glue, Amazon EMR, AWS Lambda, Amazon Managed Service for Apache Flink, SQL CTAS, or Redshift SQL.
5. Operate: monitor, retry, validate, secure, optimize, and troubleshoot.

Quick service-selection drills

Use these as fast mental prompts before attempting original practice questions.

If the stem says…

Streaming ingestion

Ingestion design rules

• Separate raw and curated data. Land immutable source data first, then transform.
• Make ingestion idempotent. Assume retries can create duplicate deliveries.
• Capture metadata: source, ingestion timestamp, schema version, batch ID, and lineage.
• Use partition-friendly timestamps. Common S3 partitions include date or hour, but avoid creating excessive tiny partitions.
• Validate early. Use data quality checks before promoting data into curated zones.
• Monitor lag and failures. Streams, DMS tasks, Firehose delivery, and Glue jobs all expose operational signals.

Batch, file, database, SaaS, and stream choices

Kinesis Data Streams vs Kinesis Data Firehose

Streaming traps

• Ordering is usually partition-specific. In Kinesis Data Streams, records with the same partition key go to the same shard, so ordering is per shard.
• Hot shards come from bad partition keys. A timestamp, region, or constant customer type can overload a shard if it concentrates traffic.
• At-least-once delivery means duplicates can happen. Design idempotent consumers and deduplication where needed.
• Firehose buffering affects latency. If a question requires very low latency custom processing, Firehose may not be the best fit.
• SQS is not Kinesis. SQS is excellent for decoupling and retries, but it is not usually the right answer for replayable analytics streams with multiple independent consumers.

File format selection

Partitioning and layout rules

Example S3 layout:

s3://company-data-lake/raw/source=salesforce/object=account/ingest_date=2026-06-18/
s3://company-data-lake/curated/domain=sales/table=orders/order_date=2026-06-18/
s3://company-data-lake/quarantine/source=orders/ingest_date=2026-06-18/

Crawler vs explicit schema

Glue ETL exam points

Illustrative Glue PySpark pattern:

from awsglue.context import GlueContext
from pyspark.context import SparkContext

glue_ctx = GlueContext(SparkContext.getOrCreate())

orders = glue_ctx.create_dynamic_frame.from_catalog(
    database="raw",
    table_name="orders",
    push_down_predicate="ingest_date >= '2026-06-01'"
)

df = orders.toDF().dropDuplicates(["order_id"])

df.write.mode("append") \
    .partitionBy("order_date") \
    .parquet("s3://example-data-lake/curated/orders/")

Example AWS Glue Data Quality rule style:

Rules = [
  IsComplete "order_id",
  IsUnique "order_id",
  ColumnValues "amount" >= 0,
  ColumnExists "order_date"
]

Athena CTAS pattern:

CREATE TABLE curated_orders
WITH (
  format = 'PARQUET',
  partitioned_by = ARRAY['order_date'],
  external_location = 's3://example-data-lake/curated/orders/'
) AS
SELECT
  order_id,
  customer_id,
  amount,
  order_status,
  order_date
FROM raw_orders
WHERE order_date >= DATE '2026-01-01';

Partition repair pattern for Hive-style S3 paths:

MSCK REPAIR TABLE raw_orders;

Common Athena traps:

• Athena queries data in S3; it does not ingest or move data by itself.
• Catalog permissions alone are not enough if S3 or KMS denies access.
• Crawlers update metadata; they do not optimize file format or clean data.
• Partition projection can reduce partition metadata management, but the S3 path pattern must match the table definition.
• Columnar formats help most when queries select only needed columns.

Redshift design points

COPY from S3 pattern:

COPY analytics.orders
FROM 's3://example-data-lake/curated/orders/'
IAM_ROLE 'arn:aws:iam::123456789012:role/RedshiftLoadRole'
FORMAT AS PARQUET;

UNLOAD to S3 pattern:

UNLOAD ('SELECT order_date, SUM(amount) AS revenue FROM analytics.orders GROUP BY order_date')
TO 's3://example-data-lake/exports/revenue/'
IAM_ROLE 'arn:aws:iam::123456789012:role/RedshiftUnloadRole'
FORMAT AS PARQUET;

Redshift troubleshooting shortcuts:

Redshift decision points

Distribution and sort-key intuition

Redshift traps

• Redshift is not the default place for all raw data; S3 is usually the landing zone for a lake.
• Redshift Spectrum still needs access to S3 data and metadata.
• COPY is usually better than row-by-row inserts for large loads.
• Poor distribution keys cause data skew and network redistribution.
• Sort keys help only when query predicates can benefit from them.

Streaming design traps

• Partition keys determine ordering and load distribution. Bad keys create hot partitions.
• Assume at-least-once delivery in many pipelines; design deduplication and idempotent sinks.
• Firehose buffering means it is usually not the answer for the lowest-latency custom consumer requirement.
• Lambda works for lightweight stream processing, not complex stateful analytics.
• Use dead-letter, backup, or quarantine patterns for failed records.
• Monitor consumer lag, delivery failures, throttling, and error logs.

Orchestration exam distinctions

Choose the processing service

AWS Glue high-yield points

Common transformation mistakes

• Choosing Lambda for heavy joins, large file conversions, or long-running Spark work.
• Choosing Glue crawlers when the question asks for transformation logic.
• Choosing EMR when the requirement says serverless and minimal infrastructure management.
• Ignoring incremental processing. If only new data should be processed, look for bookmarks, CDC, timestamps, watermarks, or checkpoints.
• Ignoring bad-record handling. Strong pipelines separate valid records, rejected records, and operational alerts.
• Treating orchestration as transformation. Step Functions and MWAA coordinate work; they are not the processing engine by themselves.

Governance decision table

Least-privilege reminders

• Grant jobs only the S3 prefixes, catalog databases/tables, KMS keys, and logs they need.
• Separate roles for ingestion, transformation, catalog administration, and consumption.
• For cross-account sharing, check all layers: IAM, resource policy, Lake Formation, S3, and KMS.
• Avoid embedding credentials in scripts, notebooks, job parameters, or environment variables when Secrets Manager is appropriate.
• Use encryption in transit and at rest for data pipelines.

IAM, resource policies, KMS, and Lake Formation

Cross-account data access checklist

When a question involves cross-account S3, Glue, Redshift Spectrum, Athena, or Lake Formation, check all of these:

1. Trust policy — can the principal assume the required role?
2. Identity policy — does the role allow required actions?
3. Resource policy — does the bucket, key, queue, or topic allow access?
4. KMS key policy — can the principal encrypt/decrypt with the key?
5. Lake Formation grants — if Lake Formation governs the table, are data permissions granted?
6. Network path — can the service reach the endpoint privately if required?
7. Catalog sharing — is metadata available to the consuming account?

Encryption review

Troubleshooting table

High-yield service map

Pipeline reliability patterns

Service-specific operational points

Monitoring trap list

• A passing pipeline can still produce bad data; monitor quality, not just job success.
• CloudWatch logs may show the error, but you still need retry, alert, and remediation design.
• Duplicate events are normal in many distributed systems; consumers must handle them.
• “Near real time” may require watching consumer lag, iterator age, or delivery delay.
• A job that scans too much data is both slower and more expensive.

Final quick checklist

Before you start a timed mock exam, confirm you can answer these without notes:

• Can I distinguish Kinesis Data Streams, Kinesis Data Firehose, MSK, and SQS?
• Can I choose between Glue, EMR, Lambda, Athena, and Redshift for transformations?
• Can I explain why S3 layout affects Athena and data-lake performance?
• Can I identify when DMS CDC is the right ingestion pattern?
• Can I troubleshoot access failures involving IAM, S3 policies, KMS, and Lake Formation?
• Can I recognize small-file, overpartitioning, schema drift, and hot partition problems?
• Can I design for retries, idempotency, DLQs, checkpoints, and quarantine paths?
• Can I connect monitoring tools to the right failure type?

Next step: use this Cheat Sheet as a checklist, then move into DEA-C01 topic drills and original practice questions with detailed explanations until your mistakes are concentrated in a few identifiable decision areas.

S3 data lake layout

A strong S3 lake design usually has zones:

File format and partition decisions

Partition traps

• Partition by query pattern, not by habit. Date partitions are common, but the best key depends on how users filter data.
• Too many tiny partitions can hurt performance. Hour/minute/customer partitions may create partition explosion.
• Partition columns are often derived from path structure. The catalog must know the partition values.
• Columnar format plus partitions is stronger than either alone.
• Athena cost and speed depend on data scanned. Compress, partition, and select only needed columns.

Schema evolution traps

• A crawler can detect changes, but automatic schema changes can break downstream consumers.
• Adding nullable columns is usually safer than renaming or changing data types.
• Streaming producers and consumers need compatibility controls before bad data reaches storage.
• Catalog schema and physical data must match. A table definition alone does not fix inconsistent files.
• Partition schema drift can cause query errors when old and new files differ.