ORACLE 使用sqluldr2和sqlldr進行導入導出

oracle數據導出工具sqluldr2能夠將數據以csv、txt等格式導出，適用於大批量數據的導出，導出速度很是快。導出後可使用oracle loader工具將數據導入。

簡介：

Sqluldr2：專業用於大數據量導出工具之一，效率比普通導出快70%。 ( Sqlldr：專業用於導入的工具之一，請注意兩個工具的區別。),在使用時,最好用磁盤寫入速度快，網絡好，網速快的作。

工具列表:

針對不一樣平臺用不一樣的導出工具：

Windows:     sqluldr2.exe

Linux(32位)： sqluldr2_linux32_10204.bin

Linux(64位):   sqluldr2_linux64_10204.bin

 

使用說明 （Windows平臺）：

使用sqluldr2的步驟：

1.打開運行àcmd進入到sqluldr2.exe的當前目錄

 

2.參數介紹

User=用戶/密碼@tns

Query=」查詢語句」

File= 導出的路徑

Head= 輸出信息時，yes表示要表頭，no表示不要表頭

 

注意：想查看更多參數，請輸入 sqluldr2 help=yes

 

C:\Users\meng\Desktop\sqluldr2>sqluldr2.exe help=yes


SQL*UnLoader: Fast Oracle Text Unloader (GZIP, Parallel), Release 4.0.1
(@) Copyright Lou Fangxin (AnySQL.net) 2004 - 2010, all rights reserved.


License: Free for non-commercial useage, else 100 USD per server.


Usage: SQLULDR2 keyword=value [,keyword=value,...]


Valid Keywords:
   user    = username/password@tnsname
   sql     = SQL file name
   query   = select statement
   field   = separator string between fields
   record  = separator string between records
   rows    = print progress for every given rows (default, 1000000)
   file    = output file name(default: uldrdata.txt)
   log     = log file name, prefix with + to append mode
   fast    = auto tuning the session level parameters(YES)
   text    = output type (MYSQL, CSV, MYSQLINS, ORACLEINS, FORM, SEARCH).
   charset = character set name of the target database.
   ncharset= national character set name of the target database.
   parfile = read command option from parameter file
   read    = set DB_FILE_MULTIBLOCK_READ_COUNT at session level
   sort    = set SORT_AREA_SIZE at session level (UNIT:MB)
   hash    = set HASH_AREA_SIZE at session level (UNIT:MB)
   array   = array fetch size
   head    = print row header(Yes|No)
   batch   = save to new file for every rows batch (Yes/No)
   size    = maximum output file piece size (UNIB:MB)
   serial  = set _serial_direct_read to TRUE at session level
   trace   = set event 10046 to given level at session level
   table   = table name in the sqlldr control file
   control = sqlldr control file and path.
   mode    = sqlldr option, INSERT or APPEND or REPLACE or TRUNCATE
   buffer  = sqlldr READSIZE and BINDSIZE, default 16 (MB)
   long    = maximum long field size
   width   = customized max column width (w1:w2:...)
   quote   = optional quote string
   data    = disable real data unload (NO, OFF)
   alter   = alter session SQLs to be execute before unload
   safe    = use large buffer to avoid ORA-24345 error (Yes|No)
   crypt   = encrypted user information only (Yes|No)
   sedf/t  = enable character translation function
   null    = replace null with given value
   escape  = escape character for special characters
   escf/t  = escape from/to characters list
   format  = MYSQL: MySQL Insert SQLs, SQL: Insert SQLs.
   exec    = the command to execute the SQLs.
   prehead = column name prefix for head line.
   rowpre  = row prefix string for each line.
   rowsuf  = row sufix string for each line.
   colsep  = separator string between column name and value.
   presql  = SQL or scripts to be executed before data unload.
   postsql = SQL or scripts to be executed after data unload.
   lob     = extract lob values to single file (FILE).
   lobdir  = subdirectory count to store lob files .
   split   = table name for automatically parallelization.
   degree  = parallelize data copy degree (2-128).
   hint    = MySQL SQL hint for the Insert, for example IGNORE.
   unique  = Unique Column List for the MySQL target table.
   update  = Enable MySQL ON DUPLICATE SQL statement(YES/NO).


  for field and record, you can use '0x' to specify hex character code,
  \r=0x0d \n=0x0a |=0x7c ,=0x2c, \t=0x09, :=0x3a, #=0x23, "=0x22 '=0x27

C:\Users\meng\Desktop\sqluldr2>

 

代碼例子1：

sqluldr2.exe USER=用戶/密碼@tnsQUERY="select   /*+ parallel(8)  */  *from cs_XXX dt,cfg_XXX devdim105 wheredt.starttime>=to_date('2012-06-27 00:00:00','yyyy-mm-dd hh24:mi:ss') and dt.starttime<=to_date('2012-06-2700:01:59','yyyy-mm-dd hh24:mi:ss') AND dt.msc= devdim105.mapvalue(+) ANDdevdim105.deviceid=15 " head=yes FILE=F:\cs_XXX_test.csv

 

代碼例子2(這種方式用於導出的查詢sql很長，那麼就把sql寫在123.sql文件裏 )：

sqluldr2.exe USER=用戶/密碼@tns  sql=123.sqlhead=yes FILE=F:\cs_XXX_test.csv

 

 

3.在cmd裏，直接把步驟2的代碼例子1 ，貼進去執行。

 

注：若是執行報報錯，就有多是環境變量path的問題，還有就是plsql須要的oci.dll文件等多個dll文件，請放在sqluldr2的目錄下。

 

4.查看結果F:\cs_XXX_test.csv

 

5.打開csv裏面的內容，就能夠用sqlldr進行 入庫。

 

 

 

 

 

--若是是linux版本

 

[root@localhost export_xdr]# ./sqluldr2_linux64_10204.bin

SQL*UnLoader: Fast Oracle Text Unloader (GZIP, Parallel), Release 4.0.1
(@) Copyright Lou Fangxin (AnySQL.net) 2004 - 2010, all rights reserved.

Usage: SQLULDR2 keyword=value [,keyword=value,...]

Valid Keywords:
   user    = username/password@tnsname
   sql     = SQL file name
   query   = select statement
   field   = separator string between fields
   record  = separator string between records
   rows    = print progress for every given rows (default, 1000000)
   file    = output file name(default: uldrdata.txt)
   log     = log file name, prefix with + to append mode
   fast    = auto tuning the session level parameters(YES)
   text    = output type (MYSQL, CSV, MYSQLINS, ORACLEINS, FORM, SEARCH).
   charset = character set name of the target database.
   ncharset= national character set name of the target database.
   parfile = read command option from parameter file

  for field and record, you can use '0x' to specify hex character code,
  \r=0x0d \n=0x0a |=0x7c ,=0x2c, \t=0x09, :=0x3a, #=0x23, "=0x22 '=0x27

 

[root@localhost export_xdr]# cd /usr/lib/oracle/11.2/client64/lib/
[root@localhost lib]# ls
glogin.sql    libclntsh.so.10.1  libnnz11.so  libocci.so.11.1  libocijdbc11.so  libsqlplus.so  ojdbc6.jar      xstreams.jar
libclntsh.so  libclntsh.so.11.1  libocci.so   libociei.so      libsqlplusic.so  ojdbc5.jar     ottclasses.zip
[root@localhost lib]# ls -l
total 185012
-rw-r--r-- 1 root root       368 Sep 17  2011 glogin.sql
lrwxrwxrwx 1 root root        17 Jan 26 17:08 libclntsh.so -> libclntsh.so.11.1
lrwxrwxrwx 1 root root        12 Jan 26 17:31 libclntsh.so.10.1 -> libclntsh.so
-rw-r--r-- 1 root root  52761218 Sep 17  2011 libclntsh.so.11.1
-rw-r--r-- 1 root root   7955322 Sep 17  2011 libnnz11.so
lrwxrwxrwx 1 root root        15 Jan 26 17:08 libocci.so -> libocci.so.11.1

[root@localhost export_xdr]# history |grep ln
   94  find / -name libclntsh.so.10.1
   95  find / -name libclntsh.so
  116  find / -name libclntsh.so
  127  ln libclntsh.so.10.1 libocci.so
  128  ln libocci.so libclntsh.so.10.1
  130  rm libclntsh.so.10.1 -f
  132  ln libclntsh.so libclntsh.so.10.1
  134  rm libclntsh.so.10.1 -f
  135  ln libclntsh.so.10.1 libclntsh.so

 

./sqluldr2_linux64_10204.bin user=unxx/密碼@192.168.x.x:1521/unxx query="select/*+ parallel(2) */STARTTIME, ENDTIME,phone,imei,rantype,HCITY from v_ps_xxx dt,mv_terminal cc  where endtime_par>=trunc(sysdate)-1 and endtime_par<trunc(sysdate)-1+1/24 and dt.TACID=cc.TACID(+) " head=yes field=0x09 text=txt file='/BigData/export_xdr/exportting.txt.tmp';

 

補充：

兄弟們，在安裝的時候，是否是常常遇到報錯？

好比：

[root@FCJ-2F-21 ~]# ./sqluldr2_linux64_10204.bin
./sqluldr2_linux64_10204.bin: error while loading shared libraries: libclntsh.so.10.1: cannot open shared object file: No such file or directory
[root@FCJ-2F-21 ~]# more /etc/profile

 

方法一：

 

這個so文件，在安裝oracle後，確定是有的，若是沒有能夠用軟鏈接 ln -s xxxx  libclntsh.so.10.1

1.下面只需配置 LD_LIBRARY_PATH

export ORACLE_HOME=/u01/app/Oracle/product/11.2.0/dbhome_1
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH
export PATH=$PATH:$ORACLE_HOME:$ORACLE_HOME/bin

[oracle@FCJ-2F-21 lib]$ more /etc/profile
# /etc/profile

# System wide environment and startup programs, for login setup
# Functions and aliases go in /etc/bashrc

pathmunge () {
    if ! echo $PATH | /bin/egrep -q "(^|:)$1($|:)" ; then
       if [ "$2" = "after" ] ; then
          PATH=$PATH:$1
       else
          PATH=$1:$PATH
       fi
    fi
}

# ksh workaround
if [ -z "$EUID" -a -x /usr/bin/id ]; then
    EUID=`id -u`
    UID=`id -ru`
fi

# Path manipulation
if [ "$EUID" = "0" ]; then
    pathmunge /sbin
    pathmunge /usr/sbin
    pathmunge /usr/local/sbin
fi

# No core files by default
ulimit -S -c 0 > /dev/null 2>&1

if [ -x /usr/bin/id ]; then
    USER="`id -un`"
    LOGNAME=$USER
    MAIL="/var/spool/mail/$USER"
fi

HOSTNAME=`/bin/hostname`
HISTSIZE=1000

if [ -z "$INPUTRC" -a ! -f "$HOME/.inputrc" ]; then
    INPUTRC=/etc/inputrc
fi

export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE INPUTRC

# By default, we want umask to get set. This sets it for login shell
# Current threshold for system reserved uid/gids is 200
# You could check uidgid reservation validity in
# /usr/share/doc/setup-*/uidgid file
if [ $UID -gt 99 ] && [ "`id -gn`" = "`id -un`" ]; then
    umask 002
else
    umask 022
fi

for i in /etc/profile.d/*.sh ; do
    if [ -r "$i" ]; then
        if [ "${-#*i}" != "$-" ]; then
            . $i
        else
            . $i >/dev/null 2>&1
        fi
    fi
done

unset i
unset pathmunge

export LANG=en_US.UTF-8
export ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH
export TNS_ADMIN=/usr/lib/oracle/11.1/client64/
export PATH=$PATH:$ORACLE_HOME:$ORACLE_HOME/bin

 

2.source /etc/profile

 

成功。

 

方法二：（來自其餘網友）

 若是共享庫文件安裝到了/usr/local/lib(不少開源的共享庫都會安裝到該目錄下)或其它"非/lib或/usr/lib"目錄下, 那麼在執行ldconfig命令前, 還要把新共享庫目錄加入到共享庫配置文件/etc/ld.so.conf中, 以下:

# cat /etc/ld.so.conf
include ld.so.conf.d/*.conf
# echo "/usr/local/lib" >> /etc/ld.so.conf
# ldconfig

----實例演示 下載完sqluldr2，文件夾內容以下： sqluldr2_linux32_10204.bin和sqluldr2_linux64_10204.bin分別適用於與linux32位和linux64位操做系統； sqluldr2.exe用於windows平臺。 如下是導出導入過程： 一、首先將sqluldr2.exe複製到到$ORACLE_HOME的bin目錄，便可開始使用： 二、查看help幫助： 三、執行導出數據命令： sqluldr2.exe USER=hh/hh@tiod QUERY="select /*+ parallel(2) */  *from hh.ent_person" table=ent_person head=yes FILE=C:\ent_person.txt p.s. head=yes表示第一行爲表頭；而且query也能夠寫入一個文件而後使用sql選項。 默認分隔字符爲逗號，若是列中有特殊字符，可以使用field選項指定新的分隔字符。 四、數據已經導入到C:\ent_person.txt，幾十萬數據秒速。 五、使用sqlldr進行導入，首先找到$ORACLE_HOME的bin目錄生成的ctl文件，當sqluldr2有table選項會默認生成ctl文件，以用於導入。 六、將數據加載到數據庫中： sqlldr jms/jms@tiod control=ent_person_sqlldr.ctl log=ent_person_sqlldr.log bad=ent_person_sqlldr_bad.log skip=1  errors=5000 rows=5000 bindsize=335542 p.s. skip=1表示跳過第一行，從第二行開始導入。 七、最後查看log是否有數據沒有導入：