centos6u3 安裝 celery 總結

耗時大概6小時.

執行 pip install celery 以後, 在 mac 上 celery 能夠正常運行, 在 centos 6u3 上報錯以下:

Traceback (most recent call last):
  File "/home/work/collie/venv/bin/celery", line 11, in <module>
    sys.exit(main())
  File "/home/work/collie/venv/lib/python2.7/site-packages/celery/__main__.py", line 13, in main
    from celery.bin.celery import main as _main
  File "/home/work/collie/venv/lib/python2.7/site-packages/celery/bin/__init__.py", line 2, in <module>
    from .base import Option
  File "/home/work/collie/venv/lib/python2.7/site-packages/celery/bin/base.py", line 17, in <module>
    from celery import VERSION_BANNER, Celery, maybe_patch_concurrency
  File "/home/work/collie/venv/lib/python2.7/site-packages/celery/local.py", line 509, in __getattr__
    module = __import__(self._object_origins[name], None, None, [name])
  File "/home/work/collie/venv/lib/python2.7/site-packages/celery/app/__init__.py", line 5, in <module>
    from celery import _state
  File "/home/work/collie/venv/lib/python2.7/site-packages/celery/_state.py", line 15, in <module>
    from celery.utils.threads import LocalStack
  File "/home/work/collie/venv/lib/python2.7/site-packages/celery/utils/__init__.py", line 10, in <module>
    from .nodenames import worker_direct, nodename, nodesplit
  File "/home/work/collie/venv/lib/python2.7/site-packages/celery/utils/nodenames.py", line 7, in <module>
    from kombu.entity import Exchange, Queue
  File "/home/work/collie/venv/lib/python2.7/site-packages/kombu/entity.py", line 6, in <module>
    from .abstract import MaybeChannelBound, Object
  File "/home/work/collie/venv/lib/python2.7/site-packages/kombu/abstract.py", line 6, in <module>
    from .connection import maybe_channel
  File "/home/work/collie/venv/lib/python2.7/site-packages/kombu/connection.py", line 15, in <module>
    from kombu import exceptions
  File "/home/work/collie/venv/lib/python2.7/site-packages/kombu/exceptions.py", line 6, in <module>
    from amqp import ChannelError, ConnectionError, ResourceError
  File "/home/work/collie/venv/lib/python2.7/site-packages/amqp/__init__.py", line 47, in <module>
    from .connection import Connection  # noqa
  File "/home/work/collie/venv/lib/python2.7/site-packages/amqp/connection.py", line 29, in <module>
    from . import spec
ImportError: cannot import name spec

遇到這個問題時, 心態很差, 沒有深刻的分析問題, 而是尋求在生產環境服務器上覆現與 mac 同樣的環境, 以求問題能夠本身解決.

最開始覺得是 rabbitMQ 沒有安裝致使, 轉而尋求安裝 rabbitMQ, 生產環境 10KB 的下載速度, 浪費了不少時間.

而後懷疑是 virtualenv 的問題, 而後在另外一臺服務器上採用絕對路徑安裝了 python 2.7.12, pip 以後發現問題依舊.

進一步懷疑是 pip 下載的文件有損, 開始手動下載相關包文件安裝, 問題依舊.

最後回到報錯的信息, 模塊加載失敗, 進入到 amqp 包文件目錄, 解釋器下加載 spec, 沒有報警.

執行 from .connection import Connection, 報錯信息以下:

>>> from connection import Connection
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "connection.py", line 26, in <module>
    from vine import ensure_promise
  File "/usr/local/lib/python2.7/site-packages/vine/__init__.py", line 8, in <module>
    from .abstract import Thenable
  File "/usr/local/lib/python2.7/site-packages/vine/abstract.py", line 8, in <module>
    from .five import with_metaclass
  File "/usr/local/lib/python2.7/site-packages/vine/five.py", line 67, in <module>
    import platform
  File "platform.py", line 16, in <module>
    int, platform.release().split('-')[0].split('.')))
AttributeError: 'module' object has no attribute 'release'

報錯 platform 沒有 release 方法, 這個是不對的, platform 是有這個方法的.

發現緣由, amqp 2.1.2 版本以後, 增長了 platform.py 的同名文件

# platform.py
"""Platform compatibility."""
from __future__ import absolute_import, unicode_literals

import sys
import platform

# Jython does not have this attribute
try:
    from socket import SOL_TCP
except ImportError:  # pragma: no cover

看起來, 當 vine 包中 five.py 執行到 platform.release 時, 解釋器混淆了兩個 platform, 致使報錯.

不過問題到這裏並不能結束, 雖然問題能夠在 centos 6, 非 /usr/bin/ 路徑安裝的 Python 2.7.11 下復現, 爲何 mac 下就沒有這個問題.

mac 下的默認 Python 也不是系統自帶版本, brew install 與手動安裝原理沒有區別, 考慮到 amqp 2.1.2 發佈時間已經一段時間, 而且在 celery 的 github 上也沒有 issue 報出, 還須要想一下, 爲何恰恰在這些機器上出現了這個問題.

順便, 在尋找 amqp 的歷史版本時, 發現無從下手, pypi 頁面只提供最新版, 最後在手動安裝的過程當中, 發現原來 pypi 上是有存儲的.

地址 https://pypi.python.org/simple/amqp/

最後, 直面問題的態度, virtualenv 的原理是否徹底理解了, import 相對路徑絕對路徑是否清楚, 異常信息是否定真看過, 都是些很基礎的事情, 這個須要增強. 項目上線以後須要再總結一下.