Merouane Benthameur
Sep 9, 2018 · 1 min read

Thanks for the tutorial, I’m facing an issue to setup airflow to submit jobs to remote Hortonworks cluster, I followed the steps you mentioned in your article, I copied all the xml files of cluster into airflow server and I installed Spark, hadoop and Java binaries, but still when I run the job I get these errors :

Exception in createBlockOutputStream
org.apache.hadoop.net.ConnectTimeoutException: 60000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=/172.17.0.2:50010]

for this error I setup the value in the hdfs-site.xml file :

<property>
<name>dfs.client.use.datanode.hostname</name>
<value>true</value>
</property>

but now I get this error :

hdfs.DFSClient: Exception in createBlockOutputStream
java.net.ConnectException: Connection refused

any ideas about this error?

thanks