{done}GTD190023:【翻譯】Dynamic Programming: First Principles

http://www.flawlessrhetoric.com/Dynamic-Programming-First-Principles

 

Dynamic Programming: First Principles

AlgorithmOct 24, 2017

Dynamic Programming

Introduction

Dynamic Programming is a mathematical tool for finding the optimal algorithm of a problem, often employed in the realms of computer science. 

During the autumn of 1950, Richard Bellman, a tenured professor from Stanford University began working for RAND (Research and Development) Corp, whom suggested he begin work on multistage decision processes. During the 1950s, Charles Erwin Wilson was the Secretary of Defence for the United States, and as RAND was employed by the U.S Air Force, Wilson was virtually their boss. Wilson, originally an electrical engineer, ironically had a hatred for the word ‘research’, and so Bellman had to come up with an alternative name to hide the fact that he was purely researching mathematics. To convey that Bellman’s research was about planning and decision making, he used the word ‘programming’, and prefixed it with ‘dynamic’, to convey that it was multistage and had time variances [1]. Bellman, due to the aforementioned recommendations by RAND, developed an approach that would break down a large or complex problem into a series of smaller problems. Through solving these smaller problems, the optimal solution to the overall problem is discovered. 

The aim of Dynamic Programming is the use of past values for the procurement of a solution to a problem. This is done to avoid unnecessary computation of the same calculation that problems may contain, optimising, enabling computer algorithms to run as efficiently as possible. It aims to find the optimal substructure of a problem (if it exists), and to eliminate any occurrences of overlapping sub problems. 

Dynamic Programming works in contrast to Linear Programming. Whilst it is concerned with functional relations and multi-stage decision processes, Linear Programming is a method used to achieve the best result based upon linear relationships [2].

介紹

動態編程是一種數學工具，用於找到經常使用於計算機科學領域的問題的最優算法。

1950年秋季，斯坦福大學終身教授理查德·貝爾曼（Richard Bellman）開始爲蘭德（研發）公司工做，他建議他開始多階段決策過程工做。五十年代期間，查爾斯·埃爾溫·威爾遜（Charles Erwin Wilson）是美國的國防部長，當蘭德被美國空軍聘用時，威爾遜其實是他們的老闆。威爾遜，原來是一名電氣工程師，諷刺地對「研究」一詞抱有仇恨，因此貝爾曼不得不提出一個替代名字，以隱藏他純粹研究數學的事實。爲了代表貝爾曼的研究是關於規劃和決策的，他使用了「編程」一詞，並以「動態」爲前綴，表達了它是多層次的，有時間差別[1]。貝爾曼，因爲蘭德提出的上述建議，開發出一種將大型或複雜問題分解爲一系列較小問題的方法。經過解決這些較小的問題，發現了總體問題的最優解。

動態規劃的目的是使用過去的價值來採購解決問題的方法。這樣作是爲了不沒必要要的計算問題可能包含的相同計算，優化，使計算機算法儘量高效地運行。它旨在找到問題的最佳子結構（若是存在），並消除任何重疊的子問題的發生。

動態編程與線性規劃相反。雖然它涉及功能關係和多階段決策過程，但線性規劃是一種基於線性關係實現最佳結果的方法[2]。

Optimal Substructures

When Dynamic Programming is applied, breaking up a problem and reconstructing it, we often find the optimal substructure of a problem. A solution is said to have an optimal substructure if it can be defined based upon optimal solutions of its sub problems. 

For example, the shortest path problem has the optimal substructure property. If we have three sequentially placed locations, X, Y, and Z, and we wish to find the optimal distance of X to Z. The solution can be defined as the sum of the optimal solutions between X to Y, and Y to Z, therefore X to Z can be defined based upon the optimal solutions of it’s sub problems. 

最優子結構

當應用動態規劃時，分解問題並進行重構，咱們常常會發現問題的最優子結構。 若是能夠根據其子問題的最佳解決方案來定義一個解決方案，則具備最佳子結構。

例如，最短路徑問題具備最佳子結構屬性。 若是咱們有三個順序放置的位置X，Y和Z，而且咱們但願找到X到Z的最佳距離。解能夠定義爲X到Y和Y到Z之間的最優解的和， 所以X到Z能夠根據其次問題的最優解決定。

Overlapping Sub-Problems

A problem or function contains overlapping problems if it can be broken down into a series of smaller problems, and some are duplicates. The need for the problem to compute the same calculation many times over can cause large increases in the running time of the problem. Dynamic Programming aims to remove the need to compute the same calculations multiple times. 

重疊子問題

問題或功能若是能夠分解成一系列較小的問題，則會包含重疊的問題，有些則是重複的。 問題須要屢次計算相同的計算可能會致使問題的運行時間的大幅度增長。 動態編程旨在消除屢次計算相同計算的需求。

Memoisation

Memoisation , also known as caching, is a technique used in Computer Science, which stores the results of functions and calculations, and uses them if the calculations are needed again. Due to the fact that memory is finite, memoisation is not feasible in all situations, and thus, needs to be used appropriately and sparingly, especially when it comes to large applications. Tail recursion [3], a variant of traditional recursion implements memoisation, which uses memoisation very economically. 

緩存

Memoisation也稱爲緩存，是計算機科學中使用的一種技術，它存儲函數和計算結果，並在須要再次計算的狀況下使用它們。 因爲記憶是有限的，記憶在全部狀況下都是不可行的，所以須要適當和謹慎地使用，特別是在涉及大量應用時。 尾遞歸[3]，傳統遞歸的變體實現了記憶，很是經濟地使用記憶。

Fibonacci: An elementary use of Dynamic Programming

One of the simplest problems that can be optimised with a Dynamic Programming approach is the Fibonacci number. The Fibonacci number (also known as the Fibonacci sequence) is a series of numbers where the leading number is the sum of the previous two numbers (modern interpretation) [4]. 

0,1,1,2,3,5,8,13,21,34…

斐波納契：動態規劃的基本使用

使用動態編程方法能夠優化的最簡單的問題之一就是斐波納契數字。 斐波納契數（也稱爲斐波那契序列）是一系列數字，其中前導數是前兩個數字（現代解釋）的總和[4]。

It is named after Italian mathematician Leonardo of Pisa (commonly known as Fibonacci), whom described the sequence in his book Liber Abaci during 1202 AD. It had previously been described in Indian Mathematics, but had not yet encountered the western world [5]. 

Fibonacci can be described as follows; 

它以意大利數學家萊昂納多比薩（俗稱斐波納契）命名，他在公元1202年在他的書「自由阿巴奇」中描述了序列。 之前曾在印度數學中描述過，但還沒有遇到西方世界[5]。

斐波那契能夠描述以下;

F0 = 0
F1 = 1
Fn = F(n-1) + F(n-2),

For example; 

F3 = F(2) + F(1) F3 = (F(1) + F(0)) + 1 F3 = (1 + 0) + 1 F3 = 2 

This poses a problem. Computing a Fibonacci number greater than two will force overlapping sub problems.

The figure above represents the structure of the Fibonacci sequence of four. The Fibonacci number of four will compute the Fibonacci of two twice. The number of overlapping sub problems will grow exponentially as the Fibonacci number is increased, and thus the running time to compute it. 

We can use Dynamic Programming (and memoisation) to mitigate these unneeded computations, and define the Fibonacci Number of n as follows; 

這是一個問題。 計算大於2的斐波納契數將迫使重疊的子問題。斐波納契四

上圖表示四個斐波納契序列的結構。 斐波那契數四人將計算斐波納契兩次兩次。 隨着斐波納契數量的增長，重疊子問題的數量將呈指數增加，從而計算運行時間。

咱們可使用動態編程（和記憶）來減輕這些沒必要要的計算，而且以下定義n的斐波那契數：

Fibonacci (n, i = 0, a = 0, b = 1 ) { if ( i < n ) Fibonacci( n, i+1, b, a+b ) return b; } 

This approach first checks if we have reached the desired number, if not, it computes the sequence. It will never call a number that has already been calculated, and will instead use memoisation and pass the previous (to the current number we are at) two values to the function to calculate them.

這種方法首先檢查咱們是否已經達到所需的數字，不然，它將計算序列。 它將永遠不會調用已經計算的數字，而是使用memoization並將前一個（到當前數字）傳遞給函數來計算它們的兩個值。

Economic Optimisation: A Canonical example of dynamic programming.

Various economic problems, especially around stage based decision making can be solved through the use of dynamic programming. 

Michael Tick describes the following problem. A corporation owning a number of plants has funds to invest, with each of the plants providing a number of proposals of what returns it can provide depending on how many funds it is allocated. This problem can be solved quite nicely with dynamic programming [6]. 

The total number of solutions is the number of proposals in each plant multiplied, in Ticks case there are 24 possible solutions (3 x 4 x 2). Enumerating of all solutions is infeasible due to the fact that many solutions may not be possible with the funds available, and are not worth calculation, and for problems containing lots of proposals, it may not be computationally feasible. Enumerating over all solutions is also not efficient, we do not look at previous solutions to eliminate possible, inferior (less generated revenue) solutions. 

This is where dynamic programming can come into use. We can employ it to develop a solution that will take into consideration the available funds as well as previously tested solutions. 

經濟優化：動態規劃的典型例子

各類經濟問題，特別是圍繞基於階段的決策能夠經過使用動態規劃來解決。

Michael Tick描述瞭如下問題。一家擁有多家工廠的公司有資金進行投資，每一個工廠都提供了一些能夠根據其分配的資金提供哪些回報的方案。這個問題能夠很好地解決動態規劃問題[6]。

解決方案的總數是每一個工廠中的提案數量相乘，在Ticks案例中有24個可能的解決方案（3 x 4 x 2）。列舉全部解決方案是不可行的，由於許多解決方案可能沒法使用可用的資金，不值得計算，對於包含大量提案的問題，可能沒法在計算上可行。列舉全部解決方案也不是有效的，咱們不考慮之前的解決方案，以消除可能的，較差的（較少的收入）解決方案。

這就是動態規劃可使用的地方。咱們能夠利用它來開發一個考慮到可用資金以及之前測試的解決方案的解決方案。

Solution Design

A solution was developed using dynamic programming to approach this problem, written in C++.

解決方案設計

使用動態編程開發了一種解決方案來解決這個問題，用C ++編寫。

Proposals

Proposals contain three integers, an index (their position in the plant), as well as their cost, and revenue. These are only set in the constructor and can be accessed but not changed.

建議

提案包含三個整數，一個指數（它們在工廠中的位置）以及它們的成本和收入。 這些只能在構造函數中設置，能夠訪問但不能更改。

Plant

Each plant contains a vector of proposals, which it creates when the constructor is invoked. It can return a reference to the vector of it’s proposals, as well as the number of proposals it contains.

廠

每一個工廠都包含一個建議的向量，它在調用構造函數時建立。 它能夠返回對其提案的向量的引用，以及它包含的提議數量。

Stage

This class contains the algorithm. Each state maintains a list of proposals (from the given plant), as well as a map, it’s key the cost, and value the revenue it generates. The stage takes a previous stage (or null if it is the first). 

If it is the first stage, then revenue is just the possible proposals that cost less than the funds available. If we are on any of the stages after the initial, we begin iterating through each of the proposals. If the cost of that proposal is less than or equal to the available funds, then we assign the revenue to a temporary variable, and if any funds remain, we access the revenue from the previous stage for the remaining funds, adding it to the temporary variable. It is then checked against the previous stage’s revenue for that cost. If it is greater, then it is the new revenue for that cost, if not, we get use the previous stage’s revenue for that cost.

階段

該類包含算法。 每一個州保留一份提案清單（來自給定的工廠）以及一張地圖，它是成本的關鍵，並對其產生的收入進行估價。 舞臺須要前一個階段（若是是第一個舞臺，則爲null）。

若是是第一階段，那麼收入只是可能的成本低於可用資金的建議。 若是咱們在初始階段以後的任何一個階段，咱們開始迭代每一個提案。 若是該提案的成本小於或等於可用資金，則咱們將收入分配給臨時變量，若是任何資金仍然存在，則咱們從剩餘資金的前一階段得到收入，將其添加到臨時 變量。 而後根據該成本檢查前一階段的收入。 若是是更大的話，那麼這個成本是新的收入，若是不是，咱們能夠用這個代價來使用前一階段的收入。

This solution uses Tail Recursion. Each stage does its necessary calculations, then passes its information to the next stage which is the natural solution to the problem. Each stage will asses it’s plant’s proposals compared to the available funds, and then pass its information to the next stage. Whilst the approach finds the best solution, it does not return the proposals used, only the possible revenue and the cost it will require. The map of cost to revenue, implements the memoisation aspect of the algorithm. 

此解決方案使用尾遞歸。 每一個階段都進行必要的計算，而後將其信息傳遞給下一個階段，這是解決問題的天然解決方案。 與現有資金相比，每一個階段都會對工廠的建議進行評估，而後將其信息傳遞到下一階段。 雖然該方法找到最佳解決方案，但它不會返回所使用的提案，只能得到可能的收入和成本。 成本收益圖，實現了算法的記憶方面。

if first stage foreach proposal in plant if proposal cost <= available funds if Revenue[proposal cost] < revenue of new proposal Revenue[proposal cost] = revenue of new proposal else foreach proposal in plant if proposal cost <= available funds revenue = proposal revenue remaining funds = funds - proposal cost if remaining funds > 0 (revenue-1,cost-1) = max Stage-1[remaining funds] revenue += revenue-1 cost = propsal cost + cost-1 if Revenue[cost] < revenue Revenue[cost] = revenue 

Whilst solvable, Tick makes several assumptions. Firstly, his solution assumes that each plant will have a proposal enacted upon, and secondly, that we use all the funds with the remainder not included in the revenue created in the end. 

雖然能夠解決，Tick作了幾個假設。 首先，他的解決方案假設每一個工廠都會有一個提出的建議，其次，咱們使用全部的資金，剩下的資金不包括在最終創造的收入中。

Conclusion

Dynamic Programming is an essential tool for solving multistage problems. It helps find the optimal substructure of a problem where possible, and remove any overlapping sub problems. It is applicable for smaller problems, such as Fibonacci, and larger problems such as economic optimisations. This report is by no means an extensive or advanced approach to dynamic programming, and aims only to introduce the concept of dynamic programming, and explain how it works on a rudimentary example.

結論

動態編程是解決多級問題的重要工具。 它有助於在可能的狀況下找到問題的最佳子結構，並刪除任何重疊的子問題。 適用於斐波那契等較小問題，經濟優化等較大問題。 本報告毫不是對動態規劃的普遍或先進的方法，目的只是介紹動態規劃的概念，並以簡單的例子解釋它的工做原理。

References

• [1] R. Bellman, Eye of the hurricane. Singapore: World Scientific, 1984
• [2] S. Dreyfus, 「A Comparison of Linear Programming and Dynamic Programming」, RAND, 1956.
• [3] D. Friedman and M. Want, Essentials of Programming Languages. MIT Press, 1999.
• [4] 「Fibonacci number」, En.wikipedia.org, 2017. [Online]. Available: https://en.wikipedia.org/wiki/Fibonacci_number. [Accessed: 22- Oct- 2017].
• [5] L. Fibonacci and L. Sigler, Fibonacci’s Liber abaci. New York: Springer, 2003.
• [6] M. Tick, 「A Tutorial on Dynamic Programming」, Mat.gsia.cmu.edu, 2017. [Online]. Available: http://mat.gsia.cmu.edu/classes/dynamic/dynamic.html. [Accessed: 30- Oct- 2015]. 
  
  

The source code can be found here.