Sqoop1.4.5 导入 hive IOException running import job: java.i(3)

14/09/08 01:28:52 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5
Enter password:
14/09/08 01:28:54 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
14/09/08 01:28:54 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
14/09/08 01:28:55 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
14/09/08 01:28:55 INFO tool.CodeGenTool: Beginning code generation
14/09/08 01:28:55 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `project` AS t LIMIT 1
14/09/08 01:28:55 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `project` AS t LIMIT 1
14/09/08 01:28:55 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/linuxidc/hadoop/hadoop-1.2.1
注: /tmp/sqoop-linuxidc/compile/b281ae9014edf3aae02818af8d90c978/project.java使用或覆盖了已过时的 API。
注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
14/09/08 01:28:56 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-linuxidc/compile/b281ae9014edf3aae02818af8d90c978/project.jar
14/09/08 01:28:56 WARN manager.MySQLManager: It looks like you are importing from mysql.
14/09/08 01:28:56 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
14/09/08 01:28:56 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
14/09/08 01:28:56 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
14/09/08 01:28:56 INFO mapreduce.ImportJobBase: Beginning import of project
14/09/08 01:28:56 INFO mapred.JobClient: Cleaning up the staging area hdfs://linuxidc01:9000/home/linuxidc/hadoop/tmp/mapred/staging/linuxidc/.staging/job_201409072150_0003
14/09/08 01:28:56 ERROR security.UserGroupInformation: PriviledgedActionException as:linuxidc cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory project already exists
14/09/08 01:28:56 ERROR tool.ImportTool: Encountered IOException running import job: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory project already exists
        at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:137)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:973)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:550)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:580)
        at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:186)
        at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:159)
        at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:247)
        at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:665)
        at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:118)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

解决:

Hbase和hive的libthrift版本分别是libthrift-0.8.0.jar,libthrift-0.9.0.jar 

copy libthrift-0.9.0.jar 到sqoop/lib目录下,问题解决。

再次支持导入语句的时候出现目录已经存在的问题

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/9be1acc5482ca6060ff8963d8953850f.html