TNS-12540: TNS:internal limit restriction exceeded

 

 

   應用程序以及客戶端工具（Toad、PL/SQL Developer等）出現忽然鏈接不上數據庫服務器的狀況，監聽日誌listener.log裏面出現了TSN-12518與TSN-12540錯誤，以下所示，

27-JAN-2015 10:10:19 * (CONNECT_DATA=(SERVICE_NAME=scm2)(CID=(PROGRAM=c:\windows\system32\inetsrv\w3wp.exe)(HOST=POS)(USER=SYSTEM)))

* (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.xxx.xxx)(PORT=1667)) * establish * scm2 * 12518

TNS-12518: TNS:listener could not hand off client connection

TNS-12540: TNS:internal limit restriction exceeded

27-JAN-2015 10:10:19 * (CONNECT_DATA=(SERVICE_NAME=scm2)(CID=(PROGRAM=c:\windows\system32\inetsrv\w3wp.exe)(HOST=CEGWEB1)(USER=NETWO

RK?SERVICE))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.xxx.xxx)(PORT=1284)) * establish * scm2 * 12518

TNS-12518: TNS:listener could not hand off client connection

TNS-12540: TNS:internal limit restriction exceeded

 

通過這幾天的努力，終於解決了這個問題，因此修改、補充了這篇先前的文章，但願能對其餘人有所借鑑。下面整理了一下解決問題的步驟。以下所示：

 

1：首先查看監聽服務的進程是否正常，以下所示，監聽服務徹底正常

ps -ef | grep tnslsnr

oraescm 6669 5979 0 14:03 pts/5 00:00:00 grep tnslsnr

oraescm 29877 1 0 08:12 ? 00:00:11 /u01/app/oracle/product/10.2.0/db_1/bin/tnslsnr LISTENER -inherit

 

2：檢查查看監聽服務的狀態（lsnrctl status）

 

3： 檢查lsnrctl services後， 發現一個奇怪的狀況Dispatcher沒有拒絕監聽的記錄，以下所示

 

4：檢查Dispatcher進程的是否繁忙，結果查詢時一直查詢不出結果，當即取消了當前查詢

 

像平時查詢時（以下所示）的速度是很是快的。若是」busy rate%「比例超過50%通常建議加SHARED SERVER OR DISPATCHER

SQL> select name,(busy/(busy+idle))*100 "busy rate%" from v$dispatcher;

NAME busy rate%

---- ----------

D000 .194393707

D001 .124595175

D002 .102238233

D003 .097416229

D004 .140182022

D005 .114123279

D006 .137454634

D007 .148511476

D008 .148901287

D009 .111103697

10 rows selected.

 

5：檢查會話、進程數量。發現當前會話數爲483，歷史最大值爲720. sessions最大值爲1000 ,processes參數最大值爲870， 應該能夠排除processes與session參數值設置太小致使。

select * from v$resource_limit;

select count(1) from v$session;

SELECT SESSIONS_MAX,SESSIONS_WARNING,SESSIONS_CURRENT,SESSIONS_HIGHWATER    

FROM v$license;

select count(1) from v$process;

select count(1) from v$session;

select busy/(busy+idle) from v$shared_server;

select name,busy/(busy+idle)*100 "Dispatcher Busy Rate" from v$dispatcher;

 

6：檢查服務器CPU、RAM資源。以下所示，CPU利用率很是低。檢查內存，發現物理內存只剩下204M大小，可是Swap還有13729M剩餘。另外告警日誌裏面並無發現任何錯誤信息。

 

在官方文檔查看TSN-12540錯誤信息，以下所示：

 

TNS-12540: TNS:internal limit restriction exceeded

Cause: Too many TNS connections open simultaneously.

Action: Wait for connections to close and re-try.

 

http://docs.oracle.com/cd/B28359_01/network.111/b28316/troublestng.htm#CEGDGHBJ

TNS-12540/ORA-12540: TNS:internal limit restriction exceeded and TNS-00510: Internal limit restriction exceeded

Cause: An internal limit has been exceeded. Possible limits include:

      Number of open connection that Oracle Net can process simultaneously

      Number of memory buffers which can be used simultaneously

      Number of processes a particular database instance is allowed

The first two are examples of hard limits. The third is an example of a limit which can be increased by setting PROCESSES parameter in the database initialization file to a larger value. In this case, a TNS-12500/ORA-12500 error is also returned. In some cases, these errors can be caused by the same conditions which cause TNS-12549/ORA-12549 and TNS-00519 errors.

Action: Perform these steps:

Wait for the open connections to close and retry. If the error persists, then check the sqlnet.log or listener.log file for detailed error stack information.

 

ORA-12540: TNS:internal limit restriction exceeded

Cause: Too many TNS connections open simultaneously.

Action: Wait for connections to close and re-try.

 

This error is not an indication of a network problem unless it occurs on all connections (in other words you have never been able to connect). If so it is a .ora configuration problem.

If you can make connections up to a certain point but then fails it is an indication of a resource limitation at the os level (this sounds like your scenario)

Typical problems are:

       Out of system memory / swap

       Out of process slots in the process table

       Streams resources depleted

       Physical connections allows by the kernal.

       Out of File Handles

可是以下所示，檢查了Linux系統的一些內核參數設置，也沒有發現有設置不當的地方

[orxxm@xxxx ~]$ ulimit -a

core file size          (blocks, -c) 0

data seg size           (kbytes, -d) unlimited

scheduling priority             (-e) 0

file size               (blocks, -f) unlimited

pending signals                 (-i) 193217

max locked memory       (kbytes, -l) 64

max memory size         (kbytes, -m) unlimited

open files                      (-n) 65536

pipe size            (512 bytes, -p) 8

POSIX message queues     (bytes, -q) 819200

real-time priority              (-r) 0

stack size              (kbytes, -s) 4194302

cpu time               (seconds, -t) unlimited

max user processes              (-u) 16384

virtual memory          (kbytes, -v) unlimited

file locks                      (-x) unlimited

 

check to count the number of existing socket handles on your db server (as root):

[oraescm@eellnx18 bdump]$ lsof -i 4 -a | wc -l

473

[oraescm@eellnx18 bdump]$ cat /proc/sys/fs/file-max

2414114

最後，咱們決定增長Dispatcher的數量，從8改成10，嘗試解決問題，結果問題再也不重現。問題解決了。

SQL>alter system set dispatchers='(protocol=TCP)(disp=10)(serv=SCM2)' scope=both;

SQL> show parameter dispatchers

NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

dispatchers                          string      (protocol=TCP)(disp=10)(serv=S

                                                 CM2)

max_dispatchers                      integer

SQL> 

 

205-02-03 添加：

       幾天過去了，悲催的是今天下午又出現了TSN-12540錯誤，糾結啊，這個問題已經把我折騰慘了！

 

參考資料：

 

http://t.askmaclean.com/forum.php?mod=viewthread&tid=1442

http://www.dba-oracle.com/t_ora_12540_tns_internal_limit_restriction_exceeded.htm

http://database.ccidnet.com/art/1105/20060601/569461_1.html

http://blog.itpub.net/81018/viewspace-812609/

http://www.bkjia.com/sjkqy/902979.html

https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=380493197934433&id=340091.1&_afrWindowMode=0&_adf.ctrl-state=6x9o00g2g_49

http://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/