【轉】使用Django從數據庫中隨機取N條記錄的不一樣方法及其性能實測

【聲明】:本文中的實驗僅限於特定數據庫和特定框架。不一樣數據庫,數據庫服務器的性能,甚至同一個數據庫的不一樣配置都會影響到同一段代碼的性能。具體狀況請在本身的生產環境進行測試。php

這裏(stackoverflow)有一篇關於使用Django隨機獲取記錄的討論。主要意思是說python

1mysql

Record.objects.order_by('?')[:2]sql

這樣獲取2個記錄會致使性能問題,緣由以下:shell

「 對於有着至關多數量記錄的表來講,這種方法異常糟糕。這會致使一個 ORDER BY RAND() 的SQL查詢。舉個栗子,這裏是MYSQL是如何處理這個查詢的(其餘數據庫的狀況也差很少),想象一下當一個表有十億行的時候會怎樣:數據庫

  1. 爲了完成ORDER BY RAND() ,須要一個RAND()列來排序apache

  2. 爲了有RAND()列,須要一個新表,由於現有的表沒有這個列。django

  3. 爲了這個新表,mysql創建了一個帶有新列的,新的臨時表,而且將已有的一百萬行數據複製進去。服務器

  4. 當其新建完了,他如你所要求的,爲每一行運行RAND()函數來填上這個值。是的,你派mysql建立一百萬個隨機數,這要點時間:)app

  5. 幾個小時或幾天後,當他幹完這活,他要排序。是的,你排mysql去排序一個一百萬行的,最糟糕的表(說他最糟糕是由於排序的鍵是隨機的)。

  6. 幾天或者幾星期後,當排序完了,他忠誠地將你實際須要的可憐的兩行抓出來返回給你。作的好。;)

注意:只是稍微說一句,得注意到mysql一開始會試着在內存中建立臨時表。當內存不夠了,他將會把全部東西放在硬盤上,因此你會由於近乎於整個過程當中的I/O瓶頸而雪上加霜。

懷疑者能夠去看看python代碼引發的查詢語句,確認是ORDER BY RAND(), 而後去Google下」order by rand()」(帶上引號)。

一個更好的方式是將這個耗費嚴重的查詢換成3個耗費更輕的:

1

2

3

4

5

6

7

8

last=MyModel.objects.count()-1 

# 這是一個獲取兩個不重複隨機數的簡單方法 

index1=randint(0, last) 

index2=randint(0, last-1) 

ifindex2==index1:  

    index2=last 

MyObj1=MyModel.objects.all()[index1] 

MyObj2=MyModel.objects.all()[index2]

如上Manganeez所說的方法,相應的獲取n條記錄的代碼應該以下:

1

2

sample=random.sample(xrange(Record.objects.count()),n) 

result=[Record.objects.all()[i]foriinsample]

基於Python代碼應該簡潔優雅的想法,如上的代碼彷佛能夠寫成:

1

result=random.sample(Record.objects.all(),n)

就性能問題,請教了stackoverflow上的大神 (雖然被踩和被教育了=。=)

Record.objects.count() 將被轉換成一個至關輕量級的SQL請求:

1

SELECTCOUNT(*)FROMTABLE

Record.objects.all()[0]也會被轉換成一個十分輕量級的SQL請求

1

SELECT*FROMTABLELIMIT 1

Querying all 是一個耗費十分嚴重的請求

1

SELECT*FROMTABLE

一般狀況下Django會不顯示其餘的結果,這樣你不會真正的獲取到全部的記錄。

1

SELECT*FROM table LIMIT20; //orsomething similar

任什麼時候候你將一個Queryset轉換成list的時候,將是資源消耗嚴重的時候。

若是我沒錯的話,在這個例子裏,sample方法將把Queryset轉換成list。

這樣若是你result = random.sample(Record.objects.all(),n) 這樣作的話,所有的Queryset將會轉換成list,而後從中隨機選擇。

想象一下若是你有十億行的數據。你是打算把它存儲在一個有百萬元素的list中,仍是願意一個一個的query?

在上邊Yeo的回答中,freakish回覆道:.count的性能是基於數據庫的。而Postgres的.count爲人所熟知的至關之慢。

某人說過,要知道梨子的滋味,就得變革梨子,親口嘗一嘗。

測試環境:

  • Win8.1 pro x64

  • Wampserver2.4-x64 (apache2.4.4 mysql5.6.12 php5.4.12)

  • Python2.7.5

  • Django1.4.6

在一個已有的測試project中新建一個app,數據庫是MYSQL:

1

D:\PyWorkspace\DjangoTest>python manage.py startapp randomrecords

在models.py中添加模型:

1

2

3

4

5

6

7

8

9

10

classRecord(models.Model): 

    """docstring for Record""" 

 

    id=models.AutoField(primary_key=True) 

    content=models.CharField(max_length=16) 

 

    def__str__(self): 

        return"id:%s content:%s"%(self.id,self.content) 

    def__unicode__(self): 

        returnu"id:%s content:%s"%(self.id,self.content)

添加一萬行數據:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

D:\PyWorkspace\DjangoTest>python manage.py syncdb 

Creating tables ... 

Creating table randomrecords_record 

Installing custom SQL ... 

Installing indexes ... 

Installed0object(s)from0fixture(s) 

 

D:\PyWorkspace\DjangoTest>python manage.py shell 

Python2.7.5(default, May152013,22:44:16) [MSC v.150064bit (AMD64)] on win 

32 

Type"help","copyright","credits"or"license"formore information. 

(InteractiveConsole) 

>>>fromrandomrecords.modelsimportRecord 

>>>foriinxrange(10000): 

...   Record.objects.create(content='c of %s'%i).save()

15分鐘之後我獲得了這個MYSQL表。真的,不騙你,真的是15分鐘。看了記錄才知道 每次save都要調用一次insert和一次update。。。。下次必定用SQL語句初始化。。。。

先寫了個腳本 在manage.py shell中調用了下 結果讓我震驚了。我表示不敢相信 又寫了view 並在settings.py中添加了顯示SQL Query語句的log

這裏是寫的view:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

deftest1(request): 

    start=datetime.datetime.now() 

    result=Record.objects.order_by('?')[:20] 

    l=list(result)# Queryset是惰性的,強制將Queryset轉爲list 

    end=datetime.datetime.now() 

    returnHttpResponse("time:  %s"%(end-start).microseconds/1000)) 

 

deftest2(request): 

    start=datetime.datetime.now() 

    sample=random.sample(xrange(Record.objects.count()),20) 

    result=[Record.objects.all()[i]foriinsample] 

    l=list(result) 

    end=datetime.datetime.now() 

    returnHttpResponse("time:  %s"%(end-start) 

 

deftest3(request): 

    start=datetime.datetime.now() 

    result=random.sample(Record.objects.all(),20) 

    l=list(result) 

    end=datetime.datetime.now() 

    returnHttpResponse("time:  %s"%(end-start)

運行結果以下,第一行是頁面顯示的時間,後邊是Queryset實際調用的SQL語句

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

test1: 

 

time: 0:00:00.012000 

 

(0.009) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` ORDER BY RAND() LIMIT 20; args=() 

[05/Dec/2013 17:48:19] "GET /dbtest/test1 HTTP/1.1" 200 775 

 

test2: 

 

time: 0:00:00.055000 

 

(0.002) SELECT COUNT(*) FROM `randomrecords_record`; args=() 

(0.002) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 6593; args=() 

(0.001) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 2570; args=() 

(0.001) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 620; args=() 

(0.001) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 5814; args=() 

(0.003) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 6510; args=() 

(0.002) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 3536; args=() 

(0.001) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 3362; args=() 

(0.003) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 8948; args=() 

(0.002) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 7723; args=() 

(0.001) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 2374; args=() 

(0.002) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 8269; args=() 

(0.002) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 4370; args=() 

(0.002) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 6953; args=() 

(0.001) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 1441; args=() 

(0.000) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 772; args=() 

(0.002) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 4323; args=() 

(0.002) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 8139; args=() 

(0.002) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 7441; args=() 

(0.001) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 1306; args=() 

(0.001) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 5462; args=() 

[05/Dec/2013 17:50:34] "GET /dbtest/test2 HTTP/1.1" 200 777 

 

test3: 

 

time: 0:00:00.156000 

 

(0.032) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record`; args=() 

[05/Dec/2013 17:51:29] "GET /dbtest/test3 HTTP/1.1" 200 774

使人難以置信的,在10000行的MYSQL表中 方法1的效率是最高的。不管是結果上看(12ms)仍是SQL語句的運行時間上看(9ms)方法1甩了其餘方法一大截

即使數據量增長到21萬,方法1也會比其餘兩種方法快:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

time: 98 

 

(0.094) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` ORDER BY RAND() LIMIT 20; args=() 

[05/Dec/2013 19:18:59] "GET /dbtest/test1 HTTP/1.1" 200 14 

 

time: 0:00:00.668000 

//這裏沒有注意到 掉了一行count語句 

(0.045) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 176449; args=() 

(0.016) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 68082; args=() 

(0.036) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 145571; args=() 

(0.033) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 111029; args=() 

(0.043) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 169675; args=() 

(0.046) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 186234; args=() 

(0.043) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 167233; args=() 

(0.015) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 54404; args=() 

(0.036) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 140395; args=() 

(0.004) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 13128; args=() 

(0.039) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 153695; args=() 

(0.034) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 131863; args=() 

(0.021) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 82785; args=() 

(0.015) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 57253; args=() 

(0.021) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 77836; args=() 

(0.049) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 199567; args=() 

(0.002) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 3867; args=() 

(0.027) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 104470; args=() 

(0.026) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 107058; args=() 

(0.043) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 150979; args=() 

[05/Dec/2013 19:21:33] "GET /dbtest/test2 HTTP/1.1" 200 15 

 

time 0:00:00.781000 

 

[05/Dec/2013 19:23:01] "GET /dbtest/test3 HTTP/1.1" 200 15 

(0.703) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record`; args=() 

[05/Dec/2013 19:23:06] "GET /dbtest/test3 HTTP/1.1" 200 15

數據量再次提高至百萬級別 1066768條數據

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

time:  

0:00:02.197000 

 

(2.193) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` ORDER BY RAND() LIMIT 20; args=() 

[05/Dec/2013 20:00:55] "GET /dbtest/test1 HTTP/1.1" 200 26 

 

time:  

0:00:02.659000 

 

(0.204) SELECT COUNT(*) FROM `randomrecords_record`; args=() 

(0.180) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 703891; args=() 

(0.038) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 156668; args=() 

(0.013) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 50742; args=() 

(0.031) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 121107; args=() 

(0.033) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 130565; args=() 

(0.017) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 66225; args=() 

(0.234) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 922479; args=() 

(0.267) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 1027166; args=() 

(0.189) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 765499; args=() 

(0.009) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 31569; args=() 

(0.233) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 934055; args=() 

(0.264) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 1052741; args=() 

(0.155) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 621692; args=() 

(0.014) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 52388; args=() 

(0.199) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 759669; args=() 

(0.170) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 655598; args=() 

(0.035) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 139709; args=() 

(0.228) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 919480; args=() 

(0.104) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 422051; args=() 

(0.017) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 67549; args=() 

[05/Dec/2013 20:00:45] "GET /dbtest/test2 HTTP/1.1" 200 26 

 

time:  

0:00:19.651000 

 

(3.645) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record`; args=() 

[05/Dec/2013 20:02:50] "GET /dbtest/test3 HTTP/1.1" 200 26

第三種方法所用時間長到使人沒法接受(19.65秒)。但有意思的是 SQL語句所花費的時間「只有」3.6秒。而大部分的時間都用在python上了。

既然第二種方法和第三種方法都須要random.sample 一個百萬個數據的list,那就是說,有大量的時間花費在將SELECT到的結果轉化爲django對象的過程當中了。

此後將再也不測試第三種方法

最後,數據量增長到5,195,536個

隨着表中數據行數的增長,兩個方法的所用的時間都到了一個徹底不能接受的程度。兩種方法所用的時間也幾乎相同。

 

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

time:  

0:00:22.278000 

 

(22.275) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FR 

OM `randomrecords_record` ORDER BY RAND() LIMIT 20; args=() 

[05/Dec/2013 21:46:33] "GET /dbtest/test1 HTTP/1.1" 200 26 

 

time:  

0:00:33.319000 

 

(1.393) SELECT COUNT(*) FROM `randomrecords_record`; args=() 

(3.201) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 4997880; args=() 

(1.229) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 2169311; args=() 

(0.445) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 1745307; args=() 

(1.306) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 3233861; args=() 

(1.881) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 3946647; args=() 

(1.624) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 3534377; args=() 

(1.068) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 1684337; args=() 

(0.902) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 2607361; args=() 

(2.938) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 4872494; args=() 

(0.493) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 851494; args=() 

(3.275) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 5182414; args=() 

(0.946) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 1684670; args=() 

(0.701) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 1819730; args=() 

(0.915) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 1626221; args=() 

(1.809) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 3638682; args=() 

(3.237) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 4801027; args=() 

(1.187) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 1955843; args=() 

(2.736) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 4835733; args=() 

(1.705) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 2756641; args=() 

(0.286) SELECT `randomrecords_record`.`id`, `randomrecords_record`.`content` FRO 

M `randomrecords_record` LIMIT 1 OFFSET 1117426; args=()

值得注意的是,Mysql數據庫有一個特色是,對於一個大表,OFFSET越大,查詢時間越長。或許有其餘方法能夠在offset較大的時候加快select的速度,然而django明顯沒有作到。若是可以減小這種消耗,方法2明顯會優於方法1。

附上三種方法數據量和SQL時間/總時間的數據圖表:

table

20131205220333

總時間

 

最後總結,Django下,使用mysql數據庫,數據量在百萬級如下時,使用

1

Record.objects.order_by('?')[:2]

來獲取隨機記錄序列,性能不會比

1

2

sample = random.sample(xrange(Record.objects.count()),n)  

result = [Record.objects.all()[i]) for i in sample]

差。

相關文章
相關標籤/搜索