配置所有节点的Agent,路径:/opt/cloudera-manager/cm-5.14.1/etc/cloudera-scm-agent/config.ini,将server_host修改成主节点的主机名,如果主节点端口没有自定义,则不用修改。
6.下载mysql-connector-java.jar,并保存到所有主机的/usr/share/Java目录下
7.配置mysql,可以参考
7.1 mysql安装在主节点上,使用mysql命令登录
7.2 新建一个scm用户,并赋予所有权限,密码是scm
mysql> grant all on *.* to 'scm'@'localhost' identified by 'temp' with grant option;
Query OK, 0 rows affected, 1 warning (0.00 sec)
7.3 创建数据库scm
7.4 初始化cloudera manager
[root@hserver1n ~]# /opt/cloudera-manager/cm-5.14.1/share/cmf/schema/scm_prepare_database.sh mysql scm scm scm
JAVA_HOME=/usr/java/jdk1.7.0_80
Verifying that we can write to /opt/cloudera-manager/cm-5.14.1/etc/cloudera-scm-server
Creating SCM configuration file in /opt/cloudera-manager/cm-5.14.1/etc/cloudera-scm-server
Executing: /usr/java/jdk1.7.0_80/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/opt/cloudera-manager/cm-5.14.1/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cloudera-manager/cm-5.14.1/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
Wed Mar 21 14:09:40 CST 2018 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
[ main] DbCommandExecutor INFO Successfully connected to database.
All done, your SCM database is configured correctly!
当看最后一句话,就说明初始化成功了
注意scm账户不能删除,因为cm以后还需要使用此账户,若想创建临时用户,则需要让CM创建用户和数据库,此时可以删除创建的临时用户。
可以使用命令:
/opt/cloudera-manager/cm-5.14.1/share/cmf/schema/scm_prepare_database.sh mysql -utemp -ptemp scm scm scm
8.将CDH安装包移动主节点的/opt/cloudera/parcel-repo目录下
移动的文件有:
将CDH-5.14.0-1.cdh5.14.0.p0.24-el7.parcel.sha1名称改成CDH-5.14.0-1.cdh5.14.0.p0.24-el7.parcel.sha
9.启动主节点的CM Server和所有节点的Agent
路径为:/opt/cloudera-manager/cm-5.14.1/etc/init.d
[root@hserver1n init.d]# ./cloudera-scm-server start
Starting cloudera-scm-server: [ OK ]
[root@hserver1n init.d]# ./cloudera-scm-agent start
Starting cloudera-scm-agent: [ OK ]
10.打开CM管理页面,网址为::7180,
用户名和密码都是admin
11.选择Cloudera Express
12.选择主机
13.群集安装
14.安装完成后,检查主机正确性
其中有两项需要修改,一个是修改swapping阀值,从30修改成10,已最大限度使用内存,第二项是关闭透明大页面压缩功能,提高性能。
15.选择要安装的服务,这里选择核心Hadoop,以后需要安装其它服务时,再根据需要安装
16.角色分配,尽量让角色均衡地分配到所有主机上,以减少某一台主机的压力
17.创建数据库用户,及相关数据库,并配置服务
mysql> grant all on *.* to 'yanggy'@'%' identified by '878963' with grant option;
Query OK, 0 rows affected, 1 warning (0.03 sec)
mysql> create database hive;
Query OK, 1 row affected (0.01 sec)
mysql> create database hue;
Query OK, 1 row affected (0.01 sec)
mysql> create database ooz;
Query OK, 1 row affected (0.00 sec)
18.集群相关服务的配置
HDFS块大小默认是128M,我这里为测试方便,只配置了16M,其它的配置一些数据目录,日志目录,以及端口
19.集群搭建完成,并启动了相关服务