Data Science & Analytics
Airflow State Store
Persists task and asset state across retries and DAG runs using Airflow 3.3's AIP-103 key/value stores (`task_state_store`, `asset_state_store`) and the crash-safe `ResumableJobMixin`. Use when the user asks about task state store, checkpointing in tasks, persisting state across retries, job IDs surviving worker crashes, watermarks, asset metadata, resumable tasks, crash-safe operators, or "what's new in Airflow 3.3". Also use proactively when reading a DAG that uses Variables or XCom for intra-task coordination state (flag the anti-pattern), or when reviewing any DAG that submits a job to an external system (Databricks, Snowflake, BigQuery, Redshift, Spark, dbt Cloud, EMR, AWS Batch, etc.) and waits for it to finish, whether as one submit-and-wait operator or split into submit + sensor/polling tasks; covers `wait_for_termination`, `deferrable`, `durable`, and hand-rolled polling sensors. State-persistence pieces require Airflow 3.3+; submit+poll architecture guidance applies on any version.
- Publisher / source
- astronomer/agents (found on skills.sh)
- Upstream record
- astronomer/agents
- Version
- Version 3
- Published
- Publication review
- Approved for publication · recorded passed scan 2026-11-03
- Content fingerprint
c67c60b4096e2189d45499fdf5cbfdb6a9f1a87f8c438d09d94ae7b50d6c4514