pymssql鏈接Azure SQL Database

使用pymssql訪問Azure SQL Database時遇到「DB-Lib error message 20002, severity 9:\nAdaptive Server connection failed (xxxx:1433)\n')」這樣的錯誤，具體錯誤以下所示：

 

 

# python test.py

Traceback (most recent call last):

  File "src/pymssql.pyx", line 636, in pymssql.connect

  File "src/_mssql.pyx", line 1957, in _mssql.connect

  File "src/_mssql.pyx", line 676, in _mssql.MSSQLConnection.__init__

  File "src/_mssql.pyx", line 1683, in _mssql.maybe_raise_MSSQLDatabaseException

_mssql.MSSQLDatabaseException: (20002, b'DB-Lib error message 20002, severity 9:\nAdaptive Server connection failed (xxxx.database.chinacloudapi.cn:1433)\n')

 

During handling of the above exception, another exception occurred:

 

Traceback (most recent call last):

  File "test.py", line 7, in <module>

    charset="utf8")

  File "src/pymssql.pyx", line 642, in pymssql.connect

pymssql.OperationalError: (20002, b'DB-Lib error message 20002, severity 9:\nAdaptive Server connection failed (xxxx.database.chinacloudapi.cn:1433)\n')

 

 

 

官方文檔關於如何使用pymssql鏈接Azure SQL Database有明確的規定（http://pymssql.org/en/stable/azure.html）：

 

·         Use FreeTDS 0.91 or newer

·         Use TDS protocol 7.1 or newer

·         Make sure FreeTDS is built with SSL support

·         Specify the database name you are connecting to in the database parameter of the relevant connect() call

·         IMPORTANT: Do not use username@server.database.windows.net for the user parameter of the relevant connect() call! You must use the shorter username@server form instead!

·          

 

 

具體翻譯以下：

 

必須使用FreeTDS 0.91 或後續版本

必須是使用TDS protocol 7.1或後續版本

必須確保FreeTDS編譯安裝時支持SSL

在相關的connect（）調用的數據庫參數中指定要鏈接的數據庫名稱

重要提示：相關的connect（）調用時，參數不要使用username@server.database.windows，而是應該使用username@server替代，不然會遇到錯誤（請見下文）

 

 

官方文檔https://www.freetds.org/userguide/ 有介紹TDS協議版本，有興趣能夠了解一下，簡單描述以下所示：

 

Product	TDS Version	Comment
Sybase before System 10, Microsoft SQL Server 6.x	4.2	Still works with all products, subject to its limitations.
Sybase System 10 and above	5.0	Still the most current protocol used by Sybase.
Sybase System SQL Anywhere	5.0 only	Originally Watcom SQL Server, a completely separate codebase. Our best information is that SQL Anywhere first supported TDS in version 5.5.03 using the OpenServer Gateway (OSG), and native TDS 5.0 support arrived with version 6.0.
Microsoft SQL Server 7.0	7.0	Includes support for the extended datatypes in SQL Server 7.0 (such as char/varchar fields of more than 255 characters), and support for Unicode.
Microsoft SQL Server 2000	7.1	Include support for bigint (64 bit integers), variant and collation on all fields. Collation is not widely used.
Microsoft SQL Server 2005	7.2	Includes support for varchar(max), varbinary(max), xml datatypes and MARS[a].
Microsoft SQL Server 2008	7.3	Includes support for time, date, datetime2, datetimeoffset.
Microsoft SQL Server 2012 or 2014	7.4	Includes support for session recovery.
Notes: a. Multiple Active Result Sets.

 

 

關於FreeTDS的安裝，咱們能夠從FreeTDS的從官方網址下載（FreeTDS的官方網址：https://www.freetds.org/），目前的穩定版本爲Stable Release (1.1.15)，下載地址ftp://ftp.freetds.org/pub/freetds/stable/freetds-1.1.15.tar.gz

 

安裝過程：

 

#tar -xzvf freetds-1.1.15.tar.gz

#cd freetds-1.1.15

# ./configure --prefix=/usr/local --with-openssl=/usr/include/openssl --with-tdsver=7.4

#make

#make install

 

驗證過程：

 

# tsql -C

Compile-time settings (established with the "configure" script)

                            Version: freetds v1.1.15

             freetds.conf directory: /usr/local/etc

     MS db-lib source compatibility: no

        Sybase binary compatibility: no

                      Thread safety: yes

                      iconv library: yes

                        TDS version: 7.4

                              iODBC: no

                           unixodbc: no

              SSPI "trusted" logins: no

                           Kerberos: no

                            OpenSSL: yes

                             GnuTLS: no

                               MARS: yes

# pip show pymssql

Name: pymssql

Version: 2.1.4

Summary: DB-API interface to Microsoft SQL Server for Python. (new Cython-based version)

Home-page: http://pymssql.org

Author: Damien Churchill

Author-email: damoxc@gmail.com

License: LGPL

Location: /usr/local/lib/python3.6/site-packages

Requires: 

Required-by: 

 

 

而後我測試驗證pymssql是否能夠訪問Azure SQL Database，可是仍是繼續報這個錯誤，至關納悶，而後我用tsql驗證了一下，確認FreeTDS是能夠訪問Azure SQL Database的。以下所示：

 

#tsql -H eworker.database.chinacloudapi.cn -U EWorkerAdmin -D EWorker -p 1433 -P 5#wJ9p@#T46ncy3Zf

locale is "en_US.UTF-8"

locale charset is "UTF-8"

using default charset "UTF-8"

Setting EWorker as default database in login packet

1> select @@version

2> go

Microsoft SQL Azure (RTM) - 12.0.2000.8 

        Jul  3 2019 10:02:53 

        Copyright (C) 2019 Microsoft Corporation

(1 row affected)

1> 

 

瞬時懵逼了，靜下心來仔細看了官方文檔（這裏就要說到通讀文檔的重要性了，不少時候，咱們去看官方文檔，都是挑一下重要的看或是敷衍了事的看看）,才發現Linux下的pymssql Wheel軟件包捆綁了FreeTDS不支持SSL,並且它是靜態的dll,即便我安裝了支持SSL的FreeTDS，可是pymssql依然不會用到後續安裝的FreeTDS的dll，所以鏈接不到Azure SQL Database:

 

The statically-linked FreeTDS version bundled with our official pymssql Linux Wheel package doesn’t have SSL support so it can’t be used to connect to Azure. Also it doesn’t have Kerberos support so it can’t be used to perform domain logins to SQL Server

 

 

 

另外github上面 https://github.com/pymssql/pymssql/issues/399 也有相關問題的較詳細的介紹:

 

From Ramiro on the ML regarding this issue:

For 2.1.2, I made the decision to stop shipping FreeTDS linked in staticall in pymssql and to depend on external FreeTDS build as DLLs instead.

In our docs (http://pymssql.org/en/v2.1.2/freetds.html#windows) we link to binaries of FreeTDS DLLS I maintain (https://github.com/ramiro/freetds/releases).

But there is more: These FreeDTS DLLs also depend on other DLLs: The OpenSSL ones chosen are these maintained at: http://www.npcglib.org/~stathis/blog/precompiled-openssl/.

(I decided to go with these because it's a binary distribution of OpenSSL which includes binaries built with the full gamut of Visual C versions used to build the official Windwos binaries of our supported Python versions [VC of VS 2008 for Python 2.7, VC of VS2010 for Python 3.3 and 3.4 and VC of VS2015 for Python 3.5] and thus won't risk causing strange errors at runtime because of mixed C runtime DLLs. This is also true for the FreeTDS DLLs above).

So, the situacion on Windows has changed like this:

pymlssql 2.1.1                                           pymssql 2.1.2

+

+------------------------------------------+             +-------------------------------------------+

|

|                                          |             |                                           |

|

| pymssql                                  |             | pymssql                                   |

|

|                                          |             |                                           |

|

|                                          |             |                                           |

|

|                                          |  +------->  |                                           |

|

|                                          |             +-------------------------------------------+

|

|  +-----------------------------------+   |

|

|  | freetds (statically linked)       |   |             +-------------------------------------------+

|

|  |                                   |   |             |                                           |

|

|  |                                   |   |             | freetds (in DLL form)                     |

|

|  +-----------------------------------+   |             |                                           |

+

+------------------------------------------+             +-------------------------------------------+

                                                         +-------------------------------------------+

                                                         |                                           |

                                                         | OpenSSL (in DLL form)                     |

                                                         |                                           |

                                                         +-------------------------------------------+

Unfortunately, as the Python package tooling and file format don't allow to ship third party binaries, these DLLs can't be bundled with the pymssql Wheel binary file. And it woldn't be a good idea anyway (being the package responsible of putting extraneous DLL files on the global site-packages dir of the users' Python installation woldn't be something t be proud of.)

Why this? Because SSL is needed to open secure connections to SQL Server. Particularly it's required when one is trying to use pymssql to connect to Azure 'cloud' SQL Server flavor. And I didn't want to include OpenSSL linked statically with the pymssql binaries because we have a somewhat slow release cycle and in case security adivsores/udpates are published for OpenSSL I didn't want to either a) expose users to keep running older, insecure binaries or b) Have to rush to a new release of pymssql simply because a new version of OpenSSL was released to fix security problems.

I'm still not sure this was a good deicision, and we might revisit it in future releases.

An alternative (which has been kown and used by many in prior releases) is to use pymssql binaries maintained (among many other) by Christoph Gohlke at http://www.lfd.uci.edu/~gohlke/pythonlibs/#pymssql

I hope this helps clifying the situation and the reasoning I followed when making these technical decisiones.

pymssql IS a wrapper for FreeTDS: It can't use other drivers/stacks/libraries.

If you are using an official pymsql 2.1.3 wheel downloaded from pypi then you copy of pymssql includes a statically linked cpy of FreeTDS. But Such copy doesn't include the feature to use TLS-protected connection which are a mandatory requirement to connect to Azure.

To be able to do so you will need to build/use a copy of FreeTDS linked against OpenSSL or similar. and then build pymssql against that version of FreeTDS.

This means your setup should heve never worked.

Pleae post support quetions like this to our mailing list, not here. I'm closing this issue.

 

因此咱們卸載pymssql，而後用pip3從源碼安裝pymmsql，必須先安裝Cython，由於pymssql依賴Cython，另外，必須建立libsybdb.so.5的軟鏈接，不然會報錯（以下所示）

 

[root@db-server python]# pip uninstall pymssql

[root@db-server python]# pip3 install --user Cython

[root@db-server python]# pip3 install --user --no-binary pymssql pymssql

 

[root@db-server python]# python test.py

Traceback (most recent call last):

  File "test.py", line 2, in <module>

    import pymssql

ImportError: libsybdb.so.5: cannot open shared object file: No such file or directory

 

[root@db-server ~]# ls /usr/local/lib/libsybdb.so.5

/usr/local/lib/libsybdb.so.5

[root@db-server ~]# ln -s /usr/local/lib/libsybdb.so.5 /usr/lib/libsybdb.so.5

[root@db-server ~]# ldconfig

 

而後測試驗證確承認以鏈接Azure SQL Database了。問題終於解決。

 

 

 

參考資料:

https://www.freetds.org/userguide/

http://pymssql.org/en/stable/azure.html

https://github.com/pymssql/pymssql/issues/399

https://stackoverflow.com/questions/39395548/how-to-configure-pymssql-with-ssl-support-on-ubuntu/52749308#52749308