Using dag_run.conf in an operator

I am passing the following as the json

{“name”: “john”}

when triggering, with this operator:

do_stuff = DatabricksSubmitRunOperator(
task_id=“do_stuff”,
spark_python_task={“python_file”: “…”,
“parameters”: f"{uid}", '\'{{ dag_run.conf["name"] if dag_run else "" }}\''},
existing_cluster_id=cluster_id
)

I’m getting a syntax error, but I am thinking it’s related to escaping characters… I have not used templating before.



Read more here: https://stackoverflow.com/questions/67384983/using-dag-run-conf-in-an-operator

Content Attribution

This content was originally published by Brian at Recent Questions - Stack Overflow, and is syndicated here via their RSS feed. You can read the original post over there.

%d bloggers like this: