sqlldr批量导入导出数据测试(2)

Record 1: Rejected - Error on table TT, column OBJECT_TYPE.
 Field in data file exceeds maximum length
 Record 2: Rejected - Error on table TT, column OBJECT_TYPE.
 Field in data file exceeds maximum length
 Record 3: Rejected - Error on table TT, column OBJECT_TYPE.
 Field in data file exceeds maximum length
 Record 4: Rejected - Error on table TT, column OBJECT_TYPE.
 Field in data file exceeds maximum length
 
尝试了好一会儿,最终发现时set linesize的时候长度设置的比较大,在根据逗号','来解析的时候,最后一个字段的长度就包含了剩余的空格,最终加载的时候就会发现它的长度太大了。已经超出了表定义的长度。
 这种情况,我总不能一个一个指定长度吧。
 这时候想到trimspool的功能,尝试果然奏效。
 spooldata.sh的脚本内容如下:
 sqlplus -s $1 <<EOF
 set pages 0
 col object_name format a30
 set linesize 10000
 set trimspool on
 set feedback off
 set colsep ','
 spool $2.lst
 select *from $2 where rownum<20 ;
 spool off;
 EOF
 

再次尝试导入,就没有问题了。
 [ora11g@rac1 sqlldr]$ sqlldr n1/n1 control=sqlldr.ctl data=t.lst
 SQL*Loader: Release 11.2.0.3.0 - Production on Tue May 27 08:14:44 2014
 Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.
 Commit point reached - logical record count 19
 
到此为止,来开始看看direct方式到底有多大的性能提升
 对于将近80万的数据量测试情况如下。
 没有采用direct方式的时候,会有一定的频度(默认50条一次)来加载数据,耗时79秒,基本一秒一万条
 Commit point reached - logical record count 793480
 Commit point reached - logical record count 793544
 Commit point reached - logical record count 793608
 Commit point reached - logical record count 793672
 Commit point reached - logical record count 793736
 Commit point reached - logical record count 793800
 Commit point reached - logical record count 793864
 Commit point reached - logical record count 793928
 Commit point reached - logical record count 793992
 Commit point reached - logical record count 794056
 Commit point reached - logical record count 794120
 Commit point reached - logical record count 794184
 Commit point reached - logical record count 794248
 Commit point reached - logical record count 794312
 Commit point reached - logical record count 794369
 
但是使用了direct=true的时候,速度明显提升,而且输出也很简单,就下面一行。耗时8秒,基本一秒10万条数据。
 8s
 [ora11g@rac1 sqlldr]$ sqlldr n1/n1 direct=true control=sqlldr.ctl data=t.lst   
 SQL*Loader: Release 11.2.0.3.0 - Production on Tue May 27 07:56:31 2014
 Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.
 Load completed - logical record count 794369.

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/8ffbeeca064581c28b55b95724fe4477.html