Python - HDFS with high availability

Owned by Nicolas (Unlicensed)
Dec 07, 2018

Preamble

This article is for using Python on an HDFS with high availability option. The particularity of high availability is to have two namenodes for one HDFS, in case of failure.

Libraries dependency

from hdfs import InsecureClient

Downloading

try:
	client_hdfs = InsecureClient('http://nn1:50070','hdfs')
	ret = client_hdfs.download('my_hdfs_file','my_local_path')
except:
	try:
			client_hdfs = InsecureClient('http://nn2:50070','hdfs')
			ret = client_hdfs.download('my_hdfs_file','my_local_path')
	except:
			print("no namenode available")