How do I define dependencies between jobs?

[jamiedemaria]

I have two jobs, and I want to run one after the other. How do I do this?

[jamiedemaria]

You can use sensors, specifically run_status_sensors to accomplish this. Relevant documentation can be found here: https://docs.dagster.io/concepts/partitions-schedules-sensors/sensors#run-status-sensors

If you have two jobs, job_1 and job_2 and you want to run job_2 whenever job_1 has run and succeeded, you can write a run_status_sensor like this:

@run_status_sensor(
    run_status=DagsterRunStatus.SUCCESS,
    monitored_jobs=[job_1]
    request_job=job_2,
)
def run_job_2_sensor(context):
        return RunRequest(run_key=None, run_config=run_config)

Note that this kind of dependency is only an ordering dependency - it ensures that job_2 will be run every time job_1 succeeds. You won't be able to pass data from job_1 to job_2

[tjmcdonough]

@jamiedemaria How would you recommend passing data from one job to another? Just calling it directly within a job? Is there a more modern even driven way?

[jamiedemaria]

Another option is to write both jobs as graphs and create a third job that runs the two graphs in series. This requires that the first graph return an output that is used by the second graph:

@op
def op1():
    return 1

@op
def op2(x):
    return x + 1


@graph
def graph_1():
    res = op2(op1())

    return {
        "result": res
    }

@graph
def graph_2(x):
    op2(x)


@job
def job_1():
    graph_2(graph_1())