蒙特卡洛法—非均勻隨機數的產生

1.反變換法

設需產生分佈函數爲F(x)的連續隨機數X。若已有[0,1]區間均勻分佈隨機數R,則產生X的反變換公式爲：

F(x)=r, 即x=F^-1(r)

反函數存在條件：若是函數y=f(x)是定義域D上的單調函數,那麼f(x)必定有反函數存在,且反函數必定是單調的。分佈函數F(x)爲是一個單調遞增函數，因此其反函數存在。從直觀意義上理解，由於r一一對應着x,而在[0,1]均勻分佈隨機數R≤r的機率P(R≤r)=r。 所以,連續隨機數X≤x的機率P(X≤x)=P(R≤r)=r=F(x)

即X的分佈函數爲F(x)。

例子:下面的代碼使用反變換法在區間[0, 6]上生成隨機數，其機率密度近似爲P(x)=e^-x  

 1 import numpy as np
 2 import matplotlib.pyplot as plt
 3 
 4 # probability distribution we're trying to calculate
 5 p = lambda x: np.exp(-x)
 6 
 7 # CDF of p
 8 CDF = lambda x: 1-np.exp(-x)
 9 
10 # invert the CDF
11 invCDF = lambda x: -np.log(1-x)
12 
13 # domain limits
14 xmin = 0 # the lower limit of our domain
15 xmax = 6 # the upper limit of our domain
16 
17 # range limits
18 rmin = CDF(xmin)
19 rmax = CDF(xmax)
20 
21 N = 10000 # the total of samples we wish to generate
22 
23 # generate uniform samples in our range then invert the CDF
24 # to get samples of our target distribution
25 R = np.random.uniform(rmin, rmax, N)
26 X = invCDF(R)
27 
28 # get the histogram info
29 hinfo = np.histogram(X,100)
30 
31 # plot the histogram
32 plt.hist(X,bins=100, label=u'Samples');
33 
34 # plot our (normalized) function
35 xvals=np.linspace(xmin, xmax, 1000)
36 plt.plot(xvals, hinfo[0][0]*p(xvals), 'r', label=u'p(x)')
37 
38 # turn on the legend
39 plt.legend()
40 plt.show()

 通常來講，直方圖的外廓曲線接近於整體X的機率密度曲線。

 2.舍選抽樣法(Rejection Methold)

用反變換法生成隨機數時，若是求不出F^-1(x)的解析形式或者F(x)就沒有解析形式，則能夠用F^-1(x)的近似公式代替。可是因爲反函數計算量較大，有時也是很不適宜的。另外一種方法是由Von Neumann提出的舍選抽樣法。下圖中曲線w(x)爲機率密度函數，按該密度函數產生隨機數的方法以下：

基本的rejection methold步驟以下：

1. Draw x uniformly from [xmin  xmax]

2. Draw x uniformly from [0, ymax]

3. if y < w(x)，accept the sample, otherwise reject it

4. repeat

即落在曲線w(x)和X軸所圍成區域內的點接受，落在該區域外的點捨棄。

例子:下面的代碼使用basic rejection sampling methold在區間[0, 10]上生成隨機數，其機率密度近似爲P(x)=e^-x  

 1 # -*- coding: utf-8 -*-
 2 '''
 3 The following code produces samples that follow the distribution P(x)=e^−x  
 4 for x=[0, 10] and generates a histogram of the sampled distribution.
 5 '''
 6 import numpy as np
 7 import matplotlib.pyplot as plt
 8 
 9 
10 P = lambda x: np.exp(-x)
11 
12 # domain limits
13 xmin = 0  # the lower limit of our domain
14 xmax = 10 # the upper limit of our domain
15 
16 # range limit (supremum) for y
17 ymax = 1
18 
19 N = 10000    # the total of samples we wish to generate
20 accepted = 0 # the number of accepted samples
21 samples = np.zeros(N)
22 count = 0    # the total count of proposals
23 
24 # generation loop
25 while (accepted < N):
26     
27     # pick a uniform number on [xmin, xmax) (e.g. 0...10)
28     x = np.random.uniform(xmin, xmax)
29     
30     # pick a uniform number on [0, ymax)
31     y = np.random.uniform(0,ymax)
32     
33     # Do the accept/reject comparison
34     if y < P(x):
35         samples[accepted] = x
36         accepted += 1
37     
38     count +=1
39     
40 print count, accepted
41 
42 # get the histogram info
43 # If bins is an int, it defines the number of equal-width bins in the given range 
44 (n, bins)= np.histogram(samples, bins=30) # Returns: n-The values of the histogram,n是直方圖中柱子的高度
45 
46 # plot the histogram
47 plt.hist(samples,bins=30,label=u'Samples')   # bins=30即直方圖中有30根柱子
48 
49 # plot our (normalized) function
50 xvals=np.linspace(xmin, xmax, 1000)
51 plt.plot(xvals, n[0]*P(xvals), 'r', label=u'P(x)')
52 
53 # turn on the legend
54 plt.legend()
55 plt.show()

>>> 

99552 10000

3.推廣的舍取抽樣法

從上圖中能夠看出，基本的rejection methold法抽樣效率很低，由於隨機數x和y是在區間[xmin  xmax]和區間[0  ymax]上均勻分佈的，產生的大部分點不會落在w(x)曲線之下(曲線e^-x的形狀一邊高一邊低，其曲線下的面積佔矩形面積的比例很小，則舍選抽樣效率很低)。爲了改進簡單舍選抽樣法的效率，能夠構造一個新的密度函數q(x)(called a proposal distribution from which we can readily draw samples)，使它的形狀接近p(x)，並選擇一個常數k使得kq(x)≥w(x)對於x定義域內的值都成立。對應下圖，首先從分佈q(z)中生成隨機數z₀，而後按均勻分佈從區間[0   kq(z₀)]生成一個隨機數u_0。 if u₀ > p(z₀) then the sample is rejected,otherwise u₀ is retained.  即下圖中灰色區域內的點都要捨棄。可見，因爲隨機點u₀只出如今曲線kq(x)之下，且在q(x)較大處出現次數較多，從而大大提升了採樣效率。顯然q(x)形狀越接近p(x)，則採樣效率越高。 

 

根據上述思想，也能夠表達採樣規則以下：

1. Draw x from your proposal distribution q(x)

2. Draw y uniformly from [0  1]

3. if y < p(x)/kq(x) , accept the sample, otherwise reject it

4. repeat

下面例子中選擇函數p(x)=1/(x+1)做爲proposal distribution，k=1。曲線1/(x+1)的形狀與e^-x相近。

 1 import numpy as np
 2 import matplotlib.pyplot as plt
 3 
 4 p = lambda x: np.exp(-x)         # our distribution
 5 g = lambda x: 1/(x+1)            # our proposal pdf (we're choosing k to be 1)
 6 CDFg = lambda x: np.log(x +1)    # generates our proposal using inverse sampling
 7 
 8 # domain limits
 9 xmin = 0  # the lower limit of our domain
10 xmax = 10 # the upper limit of our domain
11 
12 # range limits for inverse sampling
13 umin = CDFg(xmin)
14 umax = CDFg(xmax)
15 
16 N = 10000 # the total of samples we wish to generate
17 accepted = 0 # the number of accepted samples
18 samples = np.zeros(N)
19 count = 0 # the total count of proposals
20 
21 # generation loop
22 while (accepted < N):
23     
24     # Sample from g using inverse sampling
25     u = np.random.uniform(umin, umax)
26     xproposal = np.exp(u) - 1
27 
28     # pick a uniform number on [0, 1)
29     y = np.random.uniform(0, 1)
30     
31     # Do the accept/reject comparison
32     if y < p(xproposal)/g(xproposal):
33         samples[accepted] = xproposal
34         accepted += 1
35     
36     count +=1
37     
38 print count, accepted
39 
40 # get the histogram info
41 hinfo = np.histogram(samples,50)
42 
43 # plot the histogram
44 plt.hist(samples,bins=50, label=u'Samples');
45 
46 # plot our (normalized) function
47 xvals=np.linspace(xmin, xmax, 1000)
48 plt.plot(xvals, hinfo[0][0]*p(xvals), 'r', label=u'p(x)')
49 
50 # turn on the legend
51 plt.legend()
52 plt.show()

>>> 

24051 10000

能夠對比基本的舍取法和改進的舍取法的結果，前者產生符合要求分佈的10000個隨機數運算了99552步，後者運算了24051步，能夠看到效率明顯提升。

 

參考：

http://iacs-courses.seas.harvard.edu/courses/am207/blog/lecture-3.html

http://blog.csdn.net/xianlingmao/article/details/7768833

http://blog.sina.com.cn/s/blog_60b44d6a0101l45z.html

http://www.ruanyifeng.com/blog/2015/07/monte-carlo-method.html