Python - Read files from Drill

Owned by Sébastien Collet (Unlicensed)
Last updated: Dec 31, 2018 by Nicolas (Unlicensed)

Gist Page : example-python-read-from-drill

Common part

Libraries dependency

pip install pydrill


from pydrill.client import PyDrill

DRILL REST API

Each Drillbit IP is working

Default drillbit is nn1

Default port is 31000

Connection

# Connecting to drill by providing a drillbit ip and the drill rest api port (31000 by default)
conn = PyDrill(host=os.environ['IP_DRILLBIT'], port=os.environ['DRILL_API_PORT'])

How to read a file from drill with Python ?

Code example

# The file employee.json is installed with drill as an example sample
query = conn.query('SELECT * FROM cp.`employee.json` LIMIT 20', timeout=60)
# Create a pandas DataFrame with the result of the query
df = query.to_dataframe()