SQOOP - Import DB to Hive

Owned by Mélanie Armand (Unlicensed)
Feb 28, 2019
1 min read

Preamble

This article describe how to retrieve a database from SQL to dump it in hive.

Be sure to have enought rights to access database.

Retrieve DB with SQOOP and beeline 

Sqoop Hive query
#Driver
driver=sqlserver                   
#IP or DNS
ip=$SQL_IP
#Port
port=$SQL_PORT
#User
username=$SQL_USER
#Password
password=$SQL_PWD
#Database
database=Jobs
#Folder in HDFS
hdfsdest=/user/hdfs/path_to_empty_folder/$table
 
#Importation of tables
for table in "tbl1" "tbl2" "tbl3"
do
    sqoop import --connect "jdbc:$driver://$ip:$port;DatabaseName=$database"     --username $username --password $password --target-dir $hdfsdest --delete-target-dir --num-mappers 1 --table $table --hive-import --hive-table $table --hive-overwrite --hive-database drive_jobs --hive-drop-import-delims
done