Python - Read files from Drill

Gist Page : example-python-read-from-drill

Common part

Libraries dependency

pip install pydrill

from pydrill.client import PyDrill


Each Drillbit IP is working

Default drillbit is nn1

Default port is 31000


# Connecting to drill by providing a drillbit ip and the drill rest api port (31000 by default)
conn = PyDrill(host=os.environ['IP_DRILLBIT'], port=os.environ['DRILL_API_PORT'])

How to read a file from drill with Python ?

Code example

# The file employee.json is installed with drill as an example sample
query = conn.query('SELECT * FROM cp.`employee.json` LIMIT 20', timeout=60)
# Create a pandas DataFrame with the result of the query
df = query.to_dataframe()