Making things happen at the right time in the right order is much harder than it ought to be. Dagster is the latest attempt to make it easier.

A DAG is a Directed Acyclic Graph, a fancy name for a sequence of tasks that depend on each other. These types of jobs are the bread and butter of MLOps, controlling data pipelines, transformations and prediction jobs.

The current go-to tool for this kind of thing, Airflow, was first developed by Airbnb. It is a bit tricky to set up and use. Dagster is a replacement. Dagster recently wrote an article about why it's way better than Airflow. Another alternative is Prefect, which also claims to be better than Airflow. I've used Prefect and Airflow, and I preferred Prefect. Maybe Dagster is even better?

🛎️ Why this matters: Better than Airflow. Remember cron? How many task schedulers do we need?