pip私有源部署

1 需求分析

私有開發源：開發團隊須要方便的python私有包發佈機制html
私有鏡像源：自建官方源鏡像，提高訪問速度，規避偶然網絡問題，方便離線環境的私有化部署python

2 使用Docker部署PypiServer服務器

2.1 下載PypiServer鏡像

docker pull pypiserver/pypiserver
複製代碼

2.2 生成Auth信息

# 安裝依賴
apt-get install -y apache2-utilssudo pip3 install passlib 
# 生成 htpass 文件
mkdir -p /opt/pypiserver/auth /opt/pypiserver/packages
# 表示全部用戶均可以讀寫但不能執行文件/文件夾
chmod -R 666 /opt/pypiserver/packages
# 會 prompt 密碼輸入，重複兩遍同樣的
cd /opt/pypiserver/auth && htpasswd -sc .htaccess ${username}
複製代碼

2.3 容器部署

docker run -d \
-p ${port}:8080 \ 
--restart=always \ --name=pypiserver \ 
-v /opt/pypiserver/packages/:/data/packages \
-v /opt/pypiserver/auth:/data/auth/ \ 
pypiserver/pypiserver -P /data/auth/.htaccess -a update /data/packages
複製代碼

2.4 Nginx反向代理

使用Docker部署Nginx服務，同時提供HTTPS支持nginx

echo 'server { listen 80; server_name ${sever_name]; rewrite ^(.*)$ https://${server_name}$1 permanent; } server { listen 443 ssl; server_name ${server_name}; #ssl證書文件位置(常見證書文件格式爲：crt/pem) ssl_certificate /etc/nginx/ssl/ps-cert.pem; #ssl證書key位置 ssl_certificate_key /etc/nginx/ssl/ps-cert.key; ssl_session_timeout 10m; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4; ssl_prefer_server_ciphers on; location / { proxy_set_header Host $host; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-For $host; proxy_set_header X-Real-IP $remote_addr; # 此處能夠使用frp作穿透，將內網的服務映射到公網上 proxy_pass http://${public_ip}:${port}; } }' >> /opt/pypi/pypi.conf
複製代碼

部署Nginx容器git

docker run -d \
--restart always \
-v /opt/pypi/pypi.conf:/etc/nginx/conf.d/pypi.conf \
-v /opt/pypi/ssl/ps-cert.pem:/etc/nginx/ssl/ps-cert.pem \
-v /opt/pypi/ssl/ps-cert.key:/etc/nginx/ssl/ps-cert.key \
-p ${port}:80 \
--name=pypi_nginx
nginx
複製代碼

3 安裝bandersnatch本地源同步工具

3.1 本機配置

配置文件github

mkdir -p /opt/bandersnatch/log && touch /opt/bandersnatch/bandersnatch.conf /opt/bandersnatch/bandersnatch-log.conf

echo '[mirror] directory = /opt/bandersnatchjson = false release-files = true cleanup = false master = https://pypi.org timeout = 10 global-timeout = 1800 workers = 3hash-index = false stop-on-error = false storage-backend = filesystem ;log-config = /opt/bandersnatch/bandersnatch-log.conf ; root_uri = https://example.comverifiers = 3 ;keep_index_versions = 0 ;vim: set ft=cfg: ;diff-file = /srv/pypi/mirrored-files ;diff-append-epoch = true [plugins] enabled = all [blacklist] ; https://bandersnatch.readthedocs.io/en/latest/filtering_configuration.html ; https://pypi.org/stats/ [whitelist] packages = cntk tensorflow-gpu tensorflow tensorflow-cpu torch' > /opt/bandersnatch/bandersnatch.conf \
&& echo ' [loggers] keys=root,file [handlers] keys=root,file [formatters] keys=common [logger_root] level=NOTSEThandlers=root [logger_file] level=INFO handlers=file propagate=1qual name=bandersnatch [formatter_common] format=%(asctime)s %(name)-12s: %(levelname)s %(message)s [handler_root] class=StreamHandlerlevel=DEBUGformatter=commonargs=(sys.stdout,) [handler_file] class=handlers.Rotating FileHandlerlevel=INFO formatter=commonargs=('/opt/bandersnatch/log/bandersnatch.log','D',1,'UTF-8') # will manage one file a day' > /opt/bandersnatch/bandersnatch-log.conf
複製代碼

部署容器docker

docker run -d \
--restart=always \ 
--name=bandersnatch \ 
-v /opt/bandersnatch/bandersnatch.conf:/etc/bandersnatch.conf \ 
-v /opt/bandersnatch:/opt/bandersnatch \ 
pypa/bandersnatch bandersnatch mirror
複製代碼

3.2 nginx反向代理配置

使用Docker部署Nginx服務，nginx配置文件以下apache

server {
listen 80;
server_name ${server_name};
rewrite ^(.*)$ https://${server_name}$1 permanent;
}
server {
listen 443 ssl;
server_name ${server_name};
#ssl證書文件位置(常見證書文件格式爲：crt/pem)
ssl_certificate /etc/nginx/ssl/bs-cert.pem;
#ssl證書key位置
ssl_certificate_key /etc/nginx/ssl/bs-cert.key;
ssl_session_timeout 10m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;
ssl_prefer_server_ciphers on;
location / {
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $host;
proxy_set_header X-Real-IP $remote_addr;
# 此處能夠使用frp作穿透，將內網的服務映射到公網上
proxy_pass http://${public_ip}:${port};
}
}
複製代碼

5 參考

PypiServerjson
bandersnatchvim