Cloud Stack Ninja

I am new to apache airflow. I am trying to send one file from one remote server to another remote server. I have access to both the remote servers and I have no worker running on any of the two remote servers.

As the file size is quite large I cannot download it first to worker server and then transfer it to the another remote server. Is there any workaround to transfer file directly.

I have created a python script which is present in the source server that transfers the file and I connect to this server using SSHOperator. Is there any better way to do it?



Read more here: https://stackoverflow.com/questions/64396525/airflow-ftp-a-file-from-one-remote-server-to-another-remote-server

Content Attribution

This content was originally published by James Maxwell at Recent Questions - Stack Overflow, and is syndicated here via their RSS feed. You can read the original post over there.

%d bloggers like this: