二、从MySQL中导到hdfs。
创建mysql表，将其导入到hdfs
mysql> create table test(id int not null primary key auto_increment,name varchar(64) not null,price decimal(10,2), cdate date,version int,comment varchar(255));
Query OK, 0 rows affected (0.10 sec)
mysql> insert into test values(null,'iphone',3900.00,'2012-7-18',1,'8g');
Query OK, 1 row affected (0.04 sec)
mysql> insert into test values(null,'ipad',3200.00,'2012-7-16',2,'16g');
Query OK, 1 row affected (0.00 sec)
mysql> select * from test;
+----+--------+---------+------------+---------+---------+
| id | name | price | cdate | version | comment |
+----+--------+---------+------------+---------+---------+
| 1 | iphone | 3900.00 | 2012-07-18 | 1 | 8g |
| 2 | ipad | 3200.00 | 2012-07-16 | 2 | 16g |
+----+--------+---------+------------+---------+---------+
2 rows in set (0.00 sec)
导入：
[zhouhh@Hadoop48 ~]$ sqoop import --connect jdbc:mysql://Hadoop48/toplists --table test -m 1
Java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Access denied for user ''@'Hadoop48' to database 'toplists'
at org.apache.sqoop.manager.CatalogQueryManager.getColumnNames(CatalogQueryManager.java:162)
给空用户授权
mysql> GRANT ALL PRIVILEGES ON *.* TO ''@'%';
[zhouhh@Hadoop48 ~]$ sqoop import --connect jdbc:mysql://Hadoop48/toplists --username root --table test -m 1
12/07/18 11:10:16 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
12/07/18 11:10:16 INFO tool.CodeGenTool: Beginning code generation
12/07/18 11:10:16 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `index_mapping` AS t LIMIT 1
12/07/18 11:10:16 INFO orm.CompilationManager: HADOOP_HOME is /home/zhoulei/hadoop-1.0.0/libexec/..
注: /tmp/sqoop-zhoulei/compile/2b04bdabb7043e4f75b215d72f65388e/index_mapping.java使用或覆盖了已过时的 API。
注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
12/07/18 11:10:18 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-zhoulei/compile/2b04bdabb7043e4f75b215d72f65388e/index_mapping.jar
12/07/18 11:10:18 WARN manager.MySQLManager: It looks like you are importing from mysql.
12/07/18 11:10:18 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
12/07/18 11:10:18 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
12/07/18 11:10:18 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
12/07/18 11:10:25 INFO mapreduce.ImportJobBase: Beginning import of index_mapping
12/07/18 11:10:26 INFO mapred.JobClient: Running job: job_201207101344_0519
12/07/18 11:10:27 INFO mapred.JobClient: map 0% reduce 0%
12/07/18 11:10:40 INFO mapred.JobClient: map 100% reduce 0%
12/07/18 11:10:45 INFO mapred.JobClient: Job complete: job_201207101344_0519
12/07/18 11:10:45 INFO mapred.JobClient: Counters: 18
12/07/18 11:10:45 INFO mapred.JobClient: Job Counters
12/07/18 11:10:45 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=12083
12/07/18 11:10:45 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
12/07/18 11:10:45 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
12/07/18 11:10:45 INFO mapred.JobClient: Launched map tasks=1
12/07/18 11:10:45 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
12/07/18 11:10:45 INFO mapred.JobClient: File Output Format Counters
12/07/18 11:10:45 INFO mapred.JobClient: Bytes Written=28
12/07/18 11:10:45 INFO mapred.JobClient: FileSystemCounters
12/07/18 11:10:45 INFO mapred.JobClient: HDFS_BYTES_READ=87
12/07/18 11:10:45 INFO mapred.JobClient: FILE_BYTES_WRITTEN=30396
12/07/18 11:10:45 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=28
12/07/18 11:10:45 INFO mapred.JobClient: File Input Format Counters
12/07/18 11:10:45 INFO mapred.JobClient: Bytes Read=0
12/07/18 11:10:45 INFO mapred.JobClient: Map-Reduce Framework
12/07/18 11:10:45 INFO mapred.JobClient: Map input records=2
12/07/18 11:10:45 INFO mapred.JobClient: Physical memory (bytes) snapshot=79167488
12/07/18 11:10:45 INFO mapred.JobClient: Spilled Records=0
12/07/18 11:10:45 INFO mapred.JobClient: CPU time spent (ms)=340
12/07/18 11:10:45 INFO mapred.JobClient: Total committed heap usage (bytes)=56623104
12/07/18 11:10:45 INFO mapred.JobClient: Virtual memory (bytes) snapshot=955785216
12/07/18 11:10:45 INFO mapred.JobClient: Map output records=2
12/07/18 11:10:45 INFO mapred.JobClient: SPLIT_RAW_BYTES=87
12/07/18 11:10:45 INFO mapreduce.ImportJobBase: Transferred 28 bytes in 20.2612 seconds (1.382 bytes/sec)
12/07/18 11:10:45 INFO mapreduce.ImportJobBase: Retrieved 2 records.
检查数据是否导入
[zhouhh@Hadoop48 ~]$ fs -cat /user/zhouhh/test/part-m-00000
1,iphone,3900.00,2012-07-18,1,8g
2,ipad,3200.00,2012-07-16,2,16g
或
[zhouhh@Hadoop48 ~]$ fs -cat test/part-m-00000
1,iphone,3900.00,2012-07-18,1,8g
2,ipad,3200.00,2012-07-16,2,16g