Web浏览HDFS文件
到这一步我们可以通过ip地址访问HDFS文件系统。 [root@localhost hadoop]# ifconfig inet **172.19.4.239** netmask 255.255.255.0 broadcast 172.19.4.255 inet6 fe80::250:56ff:fe8d:1144 prefixlen 64 scopeid 0x20<link> ether 00:50:56:8d:11:44 txqueuelen 1000 (Ethernet) RX packets 1638661 bytes 777256881 (741.2 MiB) RX errors 0 dropped 8125 overruns 0 frame 0 TX packets 333206 bytes 24964212 (23.8 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 我这里的地址为 172.19.4.239 那么外部访问的地址是::50070(默认的端口号)/就可以访问。
命令操作HDFS文件
我们讲到了配置-->启动,现在我们来讲讲对HDFS文件系统的操作,操作HDFS 在linux中的命令: [root@localhost hadoop]# hdfs dfs Usage: hadoop fs [generic options] [-appendToFile <localsrc> ... <dst>] [-cat [-ignoreCrc] <src> ...] [-checksum <src> ...] [-chgrp [-R] GROUP PATH...] [-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...] [-chown [-R] [OWNER][:[GROUP]] PATH...] [-copyFromLocal [-f] [-p] [-l] <localsrc> ... <dst>] [-copyToLocal [-p] [-ignoreCrc] [-crc] <src> ... <localdst>] [-count [-q] [-h] [-v] <path> ...] [-cp [-f] [-p | -p[topax]] <src> ... <dst>] [-createSnapshot <snapshotDir> [<snapshotName>]] [-deleteSnapshot <snapshotDir> <snapshotName>] [-df [-h] [<path> ...]] [-du [-s] [-h] <path> ...] [-expunge] [-find <path> ... <expression> ...] [-get [-p] [-ignoreCrc] [-crc] <src> ... <localdst>] [-getfacl [-R] <path>] [-getfattr [-R] {-n name | -d} [-e en] <path>] [-getmerge [-nl] <src> <localdst>] [-help [cmd ...]] [-ls [-C] [-d] [-h] [-q] [-R] [-t] [-S] [-r] [-u] [<path> ...]] [-mkdir [-p] <path> ...] [-moveFromLocal <localsrc> ... <dst>] [-moveToLocal <src> <localdst>] [-mv <src> ... <dst>] [-put [-f] [-p] [-l] <localsrc> ... <dst>] [-renameSnapshot <snapshotDir> <oldName> <newName>] [-rm [-f] [-r|-R] [-skipTrash] <src> ...] [-rmdir [--ignore-fail-on-non-empty] <dir> ...] [-setfacl [-R] [{-b|-k} {-m|-x <acl_spec>} <path>]|[--set <acl_spec> <path>]] [-setfattr {-n name [-v value] | -x name} <path>] [-setrep [-R] [-w] <rep> <path> ...] [-stat [format] <path> ...] [-tail [-f] <file>] [-test -[defsz] <path>] [-text [-ignoreCrc] <src> ...] [-touchz <path> ...] [-usage [cmd ...]] Generic options supported are -conf <configuration file> specify an application configuration file -D <property=value> use value for given property -fs <local|namenode:port> specify a namenode -jt <local|resourcemanager:port> specify a ResourceManager -files <comma separated list of files> specify comma separated files to be copied to the map reduce cluster -libjars <comma separated list of jars> specify comma separated jar files to include in the classpath. -archives <comma separated list of archives> specify comma separated archives to be unarchived on the compute machines. The general command line syntax is bin/hadoop command [genericOptions] [commandOptions] **建立一个文件夹** [root@localhost hadoop]# hdfs dfs -mkdir /user 16/10/26 04:10:38 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable **查看是否创建成功** [root@localhost hadoop]# hdfs dfs -ls / 16/10/26 04:12:04 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Found 1 items drwxr-xr-x - root supergroup 0 2016-10-26 04:10 /user 简单命令就讲到这里以后会更单独讲解HDFS Mapreduce 案例 首先运行计算就要启动mapreduce,我们刚刚才启动了start-dfs.sh (HDFS文件系统),所以不能计算那么我们把mapreduce的程序也启动起来。 启动mapreduce命令 [root@localhost hadoop]# start-yarn.sh starting yarn daemons starting resourcemanager, logging to /opt/hadoop/hadoop-2.6.0-cdh5.8.2/logs/yarn-root-resourcemanager-localhost.localdomain.out localhost: starting nodemanager, logging to /opt/hadoop/hadoop-2.6.0-cdh5.8.2/logs/yarn-root-nodemanager-localhost.localdomain.out 启动完成以后我们调用"jps"命令看是否真的启动成功了 28113 NodeManager 28011 ResourceManager 28442 Jps 27137 NameNode 27401 SecondaryNameNode 27246 DataNode 可以很明显的看到多出来两个程序。创建测试文件
创建一个测试的数据: vi /opt/test/test.txt 麒麟 小张 张张 果哥 泽安 跨越 天天顺利 泽安 祖渊 张张