dbx config should support job_tasks

[ericfeunekes]

Workflow tasks should support job_tasks to enable the latest behaviour in Databricks. Named properties will also need to support job://.

[stephen-bowser]

This would be very helpful for me too. Is this on the roadmap at all?

[jford-ctm]

This would be great if dbx supported the run_job_task type, and it should be able to perform the lookup by Job Name rather than supplying the job_id as per the API.

[nathnx]

This would be great if dbx supported the run_job_task type, and it should be able to perform the lookup by Job Name rather than supplying the job_id as per the API.

To add, this means the job_id value has to be initialised on the workflow prior to retrieval step as the id cannot be pre-defined. In other word, the child job needs to be deployed before the parent job.

[henryhueske]

Is there any update on this topic? In the last days, I was trying to deploy workflows with task types run_job_task and/or for_each_task with the most recent release, but always the deployment failed. It would be great to have the task types included in the dbx deployment.