CDH6.3.3 安装

测试环境:

192.168.99.142 ORACLE-TEST2

192.168.99.143 ORACLE-TEST3

192.168.99.144 ORACLE-TEST4

Redhat 7.4 

安装包:

jdk-8u231-linux-x64.tar.gz

mysql-5.7.28-1.el7.x86_64.rpm-bundle.tar

cm6.3.3-redhat7.tar.gz

CDH-6.3.3-1.cdh6.3.3.p0.1796617-el7.parcel

manifest.json

注意:没有特殊说明操作只在142上进行

1.禁用防火墙

systemctl stop firewalld.service

systemctl disable firewalld.service

echo "SELINUX=disabled" > /etc/sysconfig/selinux

2.配置NTP

NTP同步采用集群内设置NTP Server的办法,将管理节点做成NTP服务器,集群内的其他节点与管理节点进行时钟同步。

首先在所有节点上安装NTP包:

yum -y install ntp

启动NTP服务,并且查看NTP状态,设置自启动:

service ntpd restart

systemctl enable ntpd.service

3.配置互信

ssh-keygen -N ""

ssh-copy-id 192.168.99.143

ssh-copy-id 192.168.99.144

其他机器也需要执行对另外两台的互信

如果root用户不能远程登录,也需要进行配置:

去其他机器执行以下

[root@ORALCE-TEST3 ~]# who

admin pts/0 2020-03-24 13:42 (10.141.24.13)

vi /etc/securetty 

加入如下内容

pts/1

pts/2

pts/3

4.192.168.99.142 和 192.168.99.143 安装mysql

建议使用rpm 安装,安装过程可能需要json包

yum install -y perl-JSON

创建用户和数据库并授权

create database metastore default character set utf8;

CREATE USER 'hive'@'%' IDENTIFIED BY 'hive';

GRANT ALL PRIVILEGES ON metastore.* TO 'hive'@'%';

Reports Manager

create database rman default character set utf8;

create user 'rman'@'%' identified by 'rman';

grant all privileges on rman.* to 'rman'@'%';

create database sentry default character set utf8;

create user 'sentry'@'%' identified by 'sentry';

grant all privileges on sentry.* to 'sentry'@'%';

Navigator Audit Server

create database nav default character set utf8;

create user 'nav'@'%' identified by 'nav';

grant all privileges on nav.* to 'nav'@'%';

create database navms default character set utf8;

create user 'navms'@'%' identified by 'navms';

grant all privileges on navms.* to 'navms'@'%';

create database cm default character set utf8;

create user 'cm'@'%' identified by 'cm';

grant all privileges on cm.* to 'cm'@'%';

create database scm default character set utf8;

create database oozie default character set utf8;

create user 'oozie'@'%' identified by 'oozie';

grant all privileges on oozie.* to 'oozie'@'%';

create database hue default character set utf8;

create user 'hue'@'%' identified by 'hue';

grant all privileges on hue.* to 'hue'@'%';

Activity Monitor

create database amon default character set utf8; 

CREATE USER 'amon '@'%' IDENTIFIED BY 'amon '; 

GRANT ALL PRIVILEGES ON amon.* TO 'amon'@'%';

FLUSH PRIVILEGES;

4.配置MySQL Connector

安装jdk,配置好环境变量

yum install mysql-connector-java

5.httpd

yum install httpd

systemctl start httpd

systemctl enable httpd.service

6.将cm6.3.3-redhat7.tar.gz解压包、parcel包和manifest.json分别放置到如下路径下

/var/www/html

/var/www/html/cdh6/parcels/6.3.3/

cd /var/www/html/cdh6/parcels/6.3.3/

计算hash值:

sha1sum CDH-6.3.3-1.cdh6.3.3.p0.1796617-el7.parcel

204dc478dbadd93413ff5dc2cd17ae9969ca4b5c  CDH-6.3.3-1.cdh6.3.3.p0.1796617-el7.parcel

正常,如果hash值不是204dc478dbadd93413ff5dc2cd17ae9969ca4b5c,可能文件有损坏,会影响后面安装

vi CDH-6.3.3-1.cdh6.3.3.p0.1796617-el7.parcel.sha

写入哈希值

204dc478dbadd93413ff5dc2cd17ae9969ca4b5c

说明:每个版本hash值不一样,CDH-6.3.3-1.cdh6.3.3.p0.1796617-el7.parcel.sha文件内容是CDH-6.3.3-1.cdh6.3.3.p0.1796617-el7.parcel文件的hash值

且此hash值与manifest.json中,CDH-6.3.3-1.cdh6.3.3.p0.1796617-el7.parcel文件对应的hash值一样

7.yum源配置

cd /etc/yum.repos.d/

vi cdh.repo

写入如下内容:

[cdh]

name=cdh

baseurl=http://192.168.99.142/cm6.3.3

enable=1

gpgcheck=0

8.安装服务

yum install cloudera-manager-daemons cloudera-manager-server

初始化:

/opt/cloudera/cm/schema/scm_prepare_database.sh mysql -h 192.168.99.142 --scm-host 192.168.99.142 scm scm scm

开启服务:

service cloudera-scm-server start

查看日志:

vi /var/log/cloudera-scm-server/cloudera-scm-server.log

查看运行状态:

service cloudera-scm-server.service status -l

9.打开网址:http://192.168.99.142:7180

按提示安装

默认用户名密码都是admin

可能遇到的报错:

报错1:

JAVA_HOME=/usr/lib/jvm/jre-openjdk

Verifying that we can write to /etc/cloudera-scm-server

Creating SCM configuration file in /etc/cloudera-scm-server

Executing: /usr/lib/jvm/jre-openjdk/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/*com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.

[ main] DbCommandExecutor INFO Successfully connected to database.

[ main] DbCommandExecutor ERROR Unable to create/drop a table.

java.sql.SQLException: Statement violates GTID consistency: CREATE TABLE ... SELECT.

解决办法:关闭GTID模式

vi /etc/my.cnf

gtid_mode=OFF

enforce_gtid_consistency=OFF

报错2:

JAVA_HOME=/usr/lib/jvm/jre-openjdk

Verifying that we can write to /etc/cloudera-scm-server

[ main] DbProvisioner ERROR Exception when creating/dropping database with user 'scm' and jdbc url 'jdbc:mysql://192.168.99.141/?useUnicode=true&characterEncoding=UTF-8'

com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Access denied for user 'scm'@'192.168.99.141' to database 'scm'

解决方法:

drop user scm@'%';

drop user scm@'192.168.99.141';

update mysql.user set host = '%' where user = 'root';

然后使用root用户进行初始化:

/opt/cloudera/cm/schema/scm_prepare_database.sh mysql -h 192.168.99.141 -uroot -proot --scm-host 192.168.99.141 scm scm scm

报错3:

ParcelUpdateService:com.cloudera.parcel.components.ParcelDownloaderImpl: Unable to retrieve remote parcel repository manifest

错误处理:

没有创建manifest.json或hash值有误导致的

报错4:

org.apache.hadoop.hive.metastore.HiveMetaException: Failed to retrieve schema tables from Hive Metastore DB,Not supported

原因是缺少JDBC驱动:下载一个:mysql-connector-java-5.1.25.zip

解压到/tmp目录下:

cp /tmp/mysql-connector-java-5.1.25/mysql-connector-java-5.1.25-bin.jar /usr/share/java

cp /usr/share/java/mysql-connector-java-5.1.25-bin.jar /opt/cloudera/parcels/CDH/lib/hive/lib

其他问题:

1.搭建完成后发现143上面的/tmp文件夹总是被撑满,生成很多类似的文件:

mgmt_mgmt-NAVIGATOR-c77e3c9ed97bae1760d495254cb7a0c1_pid2864.hprof

前端修改:Navigator Metadata Server 的 Java 堆栈大小(字节)大小,修改为1-2G

2.Service Monitor的内存溢出

前端控制台修改如下参数,按照提示的建议值进行修改即可,一般1G(可以直接在搜索框进行搜索):

Host Monitor 的 Java 堆栈大小(字节)

Service Monitor 的 Java 堆栈大小(字节)

完结

最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。