前言:日常在python中從文件夾中獲取文件名的簡單方法 os.system('ll /data/') 可是當文件夾中含有巨量文件時,這種方式徹底是行不通的;python
在/dd目錄中生成了近6百萬個文件,接下來看看不一樣方法之間的性能對比 快速生成文件的shell腳本 shell
for i in $(seq 1 1000000);do echo text >>$i.txt;done
一、系統命令 ls -l性能
# 系統命令 ls -l import time import subprocess start = time.time() result = subprocess.Popen('ls -l /dd/', stdout=subprocess.PIPE,shell=True) for file in result.stdout: pass print(time.time()-start) # 直接卡死
二、glob 模塊spa
# glob 模塊 import glob import time start = time.time() result = glob.glob("/dd/*") for file in result: pass print(time.time()-start) # 49.60481119155884
三、os.walk 模塊blog
# os.walk 模塊 import os import time start = time.time() for root, dirs, files in os.walk("/dd/", topdown=False): pass print(time.time()-start) # 8.906772375106812
四、os.scandir 模塊排序
# os.scandir 模塊 import os import time start = time.time() path = os.scandir("/dd/") for i in path: pass print(time.time()-start) # 4.118424415588379
五、shell find命令pdo
# shell find命令 import time import subprocess start = time.time() result = subprocess.Popen('find /dd/', stdout=subprocess.PIPE,shell=True) for file in result.stdout: pass print(time.time()-start) # 6.205533027648926
六、shell ls -1 -f 命令 不進行排序class
# shell ls -1 -f 命令 import time import subprocess start = time.time() result = subprocess.Popen('ls -1 -f /dd/', stdout=subprocess.PIPE,shell=True) for file in result.stdout: pass print(time.time()-start) # 3.3476643562316895
七、os.listdirimport
# os.listdir import os import time start = time.time() result = os.listdir('/dd') for file in result: pass print(time.time()-start) # 2.6720399856567383