CetnOS 6.7安装Hive 1.2.1

阅读量：6087 次

发布时间：2019-06-20

本文共 7536 字，大约阅读时间需要 25 分钟。

之前已经在（），并正常启动Hadoop集群。

机器名	作用
master	NameNode
slave	DataNode

master

[root@master ~]# wget https://mirrors.tuna.tsinghua.edu.cn/apache/hive/stable/apache-hive-1.2.1-bin.tar.gz[root@master ~]# tar zxvf apache-hive-1.2.1-bin.tar.gz[root@master ~]# mv apache-hive-1.2.1-bin /usr/local/

Hive环境变量

[root@master ~]# vi /etc/profile.d/hive.sh

添加

export HIVE_HOME=/usr/local/apache-hive-1.2.1-binexport PATH=$PATH:$HIVE_HOME/bin

保存退出

[root@master ~]# source /etc/profile.d/hive.sh[root@master ~]# hiveLogging initialized using configuration in jar:file:/usr/local/apache-hive-1.2.1-bin/lib/hive-common-1.2.1.jar!/hive-log4j.propertieshive> quit;[root@master ~]#

安装MySQL

[root@master ~]# rpm -qa |grep mysqlmysql-libs-5.1.73-5.el6_6.x86_64[root@master ~]# yum -y remove mysql-lib*[root@master ~]# yum list |grep mysqlapr-util-mysql.x86_64                      1.3.9-3.el6_0.1               basebacula-director-mysql.x86_64               5.0.0-13.el6                  basebacula-storage-mysql.x86_64                5.0.0-13.el6                  basedovecot-mysql.x86_64                       1:2.0.9-22.el6                basefreeradius-mysql.x86_64                    2.2.6-6.el6_7                 baselibdbi-dbd-mysql.x86_64                    0.8.3-5.1.el6                 basemod_auth_mysql.x86_64                      1:3.0.0-11.el6_0.1            basemysql.x86_64                               5.1.73-7.el6                  basemysql-bench.x86_64                         5.1.73-7.el6                  basemysql-connector-java.noarch                1:5.1.17-6.el6                basemysql-connector-odbc.x86_64                5.1.5r1144-7.el6              basemysql-devel.i686                           5.1.73-7.el6                  basemysql-devel.x86_64                         5.1.73-7.el6                  basemysql-embedded.i686                        5.1.73-7.el6                  basemysql-embedded.x86_64                      5.1.73-7.el6                  basemysql-embedded-devel.i686                  5.1.73-7.el6                  basemysql-embedded-devel.x86_64                5.1.73-7.el6                  basemysql-libs.i686                            5.1.73-7.el6                  basemysql-libs.x86_64                          5.1.73-7.el6                  basemysql-server.x86_64                        5.1.73-7.el6                  basemysql-test.x86_64                          5.1.73-7.el6                  basepcp-pmda-mysql.x86_64                      3.10.9-6.el6                  basephp-mysql.x86_64                           5.3.3-48.el6_8                updatesqt-mysql.i686                              1:4.6.2-28.el6_5              baseqt-mysql.x86_64                            1:4.6.2-28.el6_5              basersyslog-mysql.x86_64                       5.8.10-10.el6_6               basersyslog7-mysql.x86_64                      7.4.10-5.el6                  base[root@master ~]# yum install mysql-server.x86_64[root@master ~]# rpm -qa |grep -i mysqlmysql-5.1.73-7.el6.x86_64mysql-libs-5.1.73-7.el6.x86_64mysql-server-5.1.73-7.el6.x86_64perl-DBD-MySQL-4.013-3.el6.x86_64

将MySQL的JDBC驱动文件mysql-connector-java-5.1.40-bin.jar（该文件可在安装MySQL的lib目录下找到，我是在mysql官网下载的）复制到Hive的依赖库中

[root@master ~]# wget http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.40.tar.gz[root@master ~]#  tar zxvf mysql-connector-java-5.1.40.tar.gz[root@master ~]# cp mysql-connector-java-5.1.40/mysql-connector-java-5.1.40-bin.jar /usr/local/apache-hive-1.2.1-bin/lib/

设置MySQL

[root@master ~]# /etc/init.d/mysqld start[root@master ~]# mysql -u root -pEnter password:Welcome to the MySQL monitor.  Commands end with ; or \g.Your MySQL connection id is 2Server version: 5.1.73 Source distributionmysql> create database hiveDB;

修改root密码

mysql> use mysql;mysql> update user set password=password('root') where user='root';

刷新系统权限表

mysql> FLUSH PRIVILEGES;mysql> quit;

重新登录mysql

[root@master ~]# mysql -u root –pmysql> grant all privileges on hiveDB.* to 'root'@'%' identified by 'root';mysql> flush privileges;mysql> quit;

配置Hive

[root@master ~]# vi /usr/local/apache-hive-1.2.1-bin/conf/hive-env.sh.template

修改

# HADOOP_HOME=${bin}/../../hadoop

为

HADOOP_HOME=/usr/local/hadoop-2.7.2

保存退出

[root@master ~]# mv /usr/local/apache-hive-1.2.1-bin/conf/hive-env.sh.template /usr/local/apache-hive-1.2.1-bin/conf/hive-env.sh[root@master ~]# /usr/local/hadoop-2.7.2/bin/hdfs dfs -ls /16/11/25 19:08:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicableFound 2 itemsdrwx------   - root supergroup          0 2016-11-25 16:36 /tmpdrwxr-xr-x   - root supergroup          0 2016-11-25 01:16 /user

创建数据仓库操作过程中临时数据在HDFS上的转存目录

[root@master ~]# /usr/local/hadoop-2.7.2/bin/hdfs dfs -mkdir /tmp/hive

创建数据仓库操作过程中数据文件在HDFS上的存储目录

[root@master ~]# /usr/local/hadoop-2.7.2/bin/hdfs dfs -mkdir /user/hive[root@master ~]# /usr/local/hadoop-2.7.2/bin/hdfs dfs -mkdir /user/hive/warehouse

文件hive-site.xml内保存Hive运行时所需要的相关配置信息

[root@master ~]# vi /usr/local/apache-hive-1.2.1-bin/conf/hive-site.xml

添加


    
      
       
         
      
       hive.exec.scratchdir
          
      
       /tmp/hive
          
      
       HDFS root scratch dir for Hive jobs which gets created with write all (733) permission. For each connecting user, an HDFS scratch dir: ${hive.exec.scratchdir}/<username> is created, with ${hive.scratch.dir.permission}.
        
       
         
      
       hive.metastore.warehouse.dir
          
      
       hdfs://master:9000/hive/warehouse
          
      
       location of default database for the warehouse
        
       
         
      
       javax.jdo.option.ConnectionDriverName
          
      
       com.mysql.jdbc.Driver
          
      
       Driver class name for a JDBC metastore
        
       
     
      
       javax.jdo.option.ConnectionUserName
          
      
       root
          
      
       Username to use against metastore database
        
       
         
      
       javax.jdo.option.ConnectionPassword
          
      
       root
          
      
       password to use against metastore database
        
       
         
      
       javax.jdo.option.Multithreaded
          
      
       true
          
      
       Set this to true if multiple threads access metastore through JDO concurrently.
        
       
         
      
       javax.jdo.option.ConnectionURL
          
      
       jdbc:mysql://master:3306/hiveDB?createDatabaseIfNotExist=true
          
      
       JDBC connect string for a JDBC metastore

保存退出

属性hive.exec.scratchdir：执行Hive操作访问HDFS时用于存储临时数据的目录，默认为/tmp/目录，通常设置为/tmp/hive/。

属性hive.metastore.warehouse.dir：执行Hive数据仓库操作的数据存储目录，设置为HDFS存储路径hdfs://master_hostname:port/hive/warehouse，这里设为hdfs://master:9000/hive/warehouse。

属性javax.jdo.option.ConnectionDriverName：设置Hive连接MySQL的驱动名称，属性值为com.mysql.jdbc.Driver。

属性javax.jdo.option.ConnectionUserName：Hive连接存储metastore内容的数据库的用户名。

属性javax.jdo.option.ConnectionPassword：Hive连接存储metastore内容的数据库的密码。

属性javax.jdo.option.Multithreaded：是否允许Hive与MySQL之间存在多条连接，设置为true，表示允许。

属性javax.jdo.option.ConnectionURL：设置Hive通过JDBC模式连接MySQL数据库存储metastore内容，属性值为jdbc:mysql://host/database_name?createDatabaseIfNotExist=true，这里设为jdbc:mysql://master:3306/hiveDB?createDatabaseIfNotExist=true。

[root@master ~]# hiveLogging initialized using configuration in jar:file:/usr/local/apache-hive-1.2.1-bin/lib/hive-common-1.2.1.jar!/hive-log4j.propertieshive> SHOW TABLES;OKTime taken: 1.637 secondshive> quit;[root@master ~]#

转载于:https://www.cnblogs.com/Genesis2018/p/9079790.html

你可能感兴趣的文章