Beruflich Dokumente
Kultur Dokumente
8) Remove .template extension from all the files stored. (Ex : mv hive-env.sh.te
mplate hive-env.sh)
cd $HIVE_HOME/conf
9) Enter into hive console
$hive (Press Enter to hive shell)
Note : If you want to proceed with the default hive metastore with derby db in y
our local machine experimental proceed $hive and start creating your table and t
ry HiveQueryLanguage..(HQL). Go to Step 18
10) Configure the hive metaStore pointing MySQL (Recommended)
This will install the jar for mysql connector through java, cd /usr/share/java/
to see the mysql-connector-java.jar)
sudo apt-get install mysql-server-5.5
sudo apt-get install libmysql-java
sudo ln -s /usr/share/java/mysql.jar /usr/local/hive-0.14/lib/libmysql-java.jar
11) sudo sysv-rc-conf mysql on (Start mysql upon machine start. If sysv-rc-conf
missing in your ubuntu then "sudo apt-get install sysv-rc-conf")
Create the initial database schema using the hive-schema-.sql file located in t
he $HIVE_HOME/scripts/metastore/upgrade/mysql directory.
You also need a MySQL user account for Hive to use to access the metastore. It
is very important to prevent this user account from creating or altering tables
in the metastore database schema. Replace metastorehost(your remote metastore se
rver) with localhost if your metastore server is same.
15) Go to hive prompt and we'll create the table users in the Hive MetaStore to
map data from user.txt
$ hive (Enter)
hive>CREATE TABLE user(id INT, name STRING, City STRING, State STRING, Country S
TRING)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n' STORED AS TEXTFILE;
hive>show tables;
16) The following command maps user.txt data to the users table by loading data
from user.txt.
(LOCAL => keyword needed to load file from local into hive. Not necessary if y
our file is in hdfs)
You can add OVERWRITE prior to 'INTO TABLE' if you want to overwrite existing
user table content
You can use
hadoop fs -put <localfilesystempath> <hdfsfiledirectory>
Load data into hive from your local directory. Remove LOCAL keyword if you file
is in hdfs.
$hive>LOAD DATA LOCAL INPATH '/home/hduser/user.txt' INTO TABLE user;
Note : Load Local command copies the file from local location to /user/hive/ware
house/userdb/user/user.txt. When you execute "DROP TABLE user" the file will als
o be dropped/removed from the hive location /user/hive/warehouse/userdb/user/use
r.txt.
17) Query How many people belong to each state?
$hive>select state, count(state) from user group by state;
18) You can use most of the relational db (SQL) queries in hive
hive>describe user;
hive>show create table user;
19) Drop hive table drops the table and data file from the warehouse (/user/hive
/warehouse/db/table/user.txt). What if you have MapReduce Program reference to t
his data file ? Refer next External Table creation
hive>DROP TABLE user
20) Create external table in hive. So, multiple tables can refer to the same dat
a file and if you drop the table the data file will be available in the same loc
ation.
hive>CREATE EXTERNAL TABLE user(id INT, name STRING, city STRING, state STRING,
country STRING)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n' STORED AS TEXTFILE;
hive>LOAD DATA INPATH '/samples/user.txt' INTO TABLE user;
Note : Load command moves the file from hdfs location to /user/hive/warehouse/us
erdb/user/user.txt. So, file won't be available in original hdfs location. Upon
DROP TABLE user (External created table) the file will be available still in the
same location /user/hive/warehouse/userdb/user/user.txt.Even after table in hiv
e got dropped.
21) Exit from hive
$hive>quit;
22) Hive Log file location :
Note : hive.log path can be find in $HIVE_HOME/conf/hive-log4j.properties. ${jav
a.io.tmpdir}/${user.name}/hive.log (/tmp/hduser/hive.log)
You can do all the DDL (DataDefinitionLanguage commands) you do in SQL in HiveQu
eryLanguage. Refer : Hive DDL Manual