NEXT社區小課堂 | 第五課：NEO-共識算法dBFT源碼解析

時間 2019-11-13

標籤社區課堂第五 neo 共識算法 dbft 源碼解析简体版

原文原文鏈接

NEXT社區 | 小課堂html

因爲近期NEXT社區加入不少新的小夥伴，有在校大學生，有對區塊鏈感興趣的傳統企業從業者。爲了更方便、更系統的讓NEXT社區的夥伴們瞭解NEO的技術知識，所以咱們開設了小課堂，每週3節，向你們普及NEO相關的知識要點！ 公號：NEONEXTnode

NEXT社區小課堂 | 第五課

NEO-共識機制dBFT源碼解析

1、NEO DBFT共識算法git

dbft改進自算法pbft算法，pbft算法經過屢次網絡請求確認，最終得到多數共識。其缺點在於隨着隨着節點的增長，網絡開銷呈指數級增加，網絡通訊膨脹，難以快速達成一致。neo的解決方案是經過投票選取出必定數量的節點做爲記帳人，由此減小網絡消耗又能夠兼顧交易速度，這也是dbft中d的由來。github

2、代碼結構說明算法

├── Consensus
│   ├── ChangeView.cs           //viewchange 消息
│   ├── ConsensusContext.cs     //共識上下文
│   ├── ConsensusMessage.cs     //共識消息
│   ├── ConsensusMessageType.cs //共識消息類型 ChangeView/PrepareRequest/PrepareResponse
│   ├── ConsensusService.cs     //共識核心代碼    
│   ├── ConsensusState.cs       //節點共識狀態
│   ├── PrepareRequest.cs       //請求消息
│   └── PrepareResponse.cs      //簽名返回消息

3、共識狀態變化流程網絡

一、開啓共識的節點分爲兩大類，非記帳人和記帳人節點，非記帳人的不參與共識，記帳人蔘與共識流程函數

二、選擇議長，Neo議長產生機制是根據當前塊高度和記帳人數量作MOD運算獲得，議長實際上按順序當選區塊鏈

三、節點初始化，議長爲primary節點，議員爲backup節點。ui

四、知足出塊條件後議長髮送PrepareRequestspa

五、議員收到請求後，驗證經過簽名發送PrepareResponse

六、記帳節點接收到PrepareResponse後，節點保存對方的簽名信息，檢查若是超過三分之二則發送 block

七、節點接收到block，PersistCompleted事件觸發後總體從新初始化

4、共識上下文核心成員

public const uint Version = 0;
public ConsensusState State;       //節點當前共識狀態
public UInt256 PrevHash;
public uint BlockIndex;            //塊高度
public byte ViewNumber;            //試圖狀態
public ECPoint[] Validators;       //記帳人     
public int MyIndex;                //當前記帳人次序
public uint PrimaryIndex;          //當前記帳的記帳人
public uint Timestamp;
public ulong Nonce;
public UInt160 NextConsensus;      //共識標識
public UInt256[] TransactionHashes;
public Dictionary<UInt256, Transaction> Transactions;
public byte[][] Signatures;        //記帳人簽名
public byte[] ExpectedView;        //記帳人試圖
public KeyPair KeyPair;

public int M => Validators.Length - (Validators.Length - 1) / 3;   //三分之二數量

ExpectedView 維護視圖狀態中，用於在議長沒法正常工做時從新發起新一輪共識。

Signatures 用於維護共識過程當中的確認狀態。

4、節點共識狀態

[Flags]
internal enum ConsensusState : byte
{
    Initial = 0x00,           //      0
    Primary = 0x01,           //      1
    Backup = 0x02,            //     10
    RequestSent = 0x04,       //    100
    RequestReceived = 0x08,   //   1000
    SignatureSent = 0x10,     //  10000
    BlockSent = 0x20,         // 100000
    ViewChanging = 0x40,      //1000000
}

5、代理節點選擇

neo的dbft算法的delegate部分主要體如今一個函數上面,每次產生區塊後會從新排序資產，選擇前記帳人數量的節點出來參與下一輪共識。

/// <summary>
/// 獲取下一個區塊的記帳人列表
/// </summary>
/// <returns>返回一組公鑰，表示下一個區塊的記帳人列表</returns>
public ECPoint[] GetValidators()
{
      lock (_validators)
      {
           if (_validators.Count == 0)
      {
           _validators.AddRange(GetValidators(Enumerable.Empty<Transaction>()));
      }
           return _validators.ToArray();
      }
}

public virtual IEnumerable<ECPoint> GetValidators(IEnumerable<Transaction> others)
{
      DataCache<UInt160, AccountState> accounts = GetStates<UInt160, AccountState>();
      DataCache<ECPoint, ValidatorState> validators = GetStates<ECPoint, ValidatorState>();
      MetaDataCache<ValidatorsCountState> validators_count = GetMetaData<ValidatorsCountState>();
///更新帳號資產狀況
      foreach (Transaction tx in others)
      {
           foreach (TransactionOutput output in tx.Outputs)
      {
           AccountState account = accounts.GetAndChange(output.ScriptHash, () => new AccountState(output.ScriptHash));
           if (account.Balances.ContainsKey(output.AssetId))
                 account.Balances[output.AssetId] += output.Value;
           else
                 account.Balances[output.AssetId] = output.Value;
           if (output.AssetId.Equals(GoverningToken.Hash) && account.Votes.Length > 0)
           {
                 foreach (ECPoint pubkey in account.Votes)
                      validators.GetAndChange(pubkey, () => new ValidatorState(pubkey)).Votes += output.Value;
                 validators_count.GetAndChange().Votes[account.Votes.Length - 1] += output.Value;
           }
       }
       foreach (var group in tx.Inputs.GroupBy(p => p.PrevHash))
       {
            Transaction tx_prev = GetTransaction(group.Key, out int height);
            foreach (CoinReference input in group)
            {
                  TransactionOutput out_prev = tx_prev.Outputs[input.PrevIndex];
                  AccountState account = accounts.GetAndChange(out_prev.ScriptHash);
                  if (out_prev.AssetId.Equals(GoverningToken.Hash))
                  {
                       if (account.Votes.Length > 0)
                       {
                            foreach (ECPoint pubkey in account.Votes)
                            {
                                ValidatorState validator = validators.GetAndChange(pubkey);
                                validator.Votes -= out_prev.Value;
                                if (!validator.Registered && validator.Votes.Equals(Fixed8.Zero))
                                    validators.Delete(pubkey);
                            }
                       validators_count.GetAndChange().Votes[account.Votes.Length - 1] -= out_prev.Value;
                       }
                  }
                  account.Balances[out_prev.AssetId] -= out_prev.Value;
             }
       }
       switch (tx)
       {
#pragma warning disable CS0612
             case EnrollmentTransaction tx_enrollment:
             validators.GetAndChange(tx_enrollment.PublicKey, () => new ValidatorState(tx_enrollment.PublicKey)).Registered = true;
             break;
#pragma warning restore CS0612
             case StateTransaction tx_state:
                  foreach (StateDescriptor descriptor in tx_state.Descriptors)
                       switch (descriptor.Type)
                       {
                            case StateType.Account:
                                ProcessAccountStateDescriptor(descriptor, accounts, validators, validators_count);
                                break;
                             case StateType.Validator:
                                 ProcessValidatorStateDescriptor(descriptor, validators);
                                 break;
                        }
                  break;
            }
       }
//排序
       int count = (int)validators_count.Get().Votes.Select((p, i) => new
       {
            Count = i,
            Votes = p
       }).Where(p => p.Votes > Fixed8.Zero).ToArray().WeightedFilter(0.25, 0.75, p => p.Votes.GetData(), (p, w) => new
            {
                p.Count,
            Weight = w
       }).WeightedAverage(p => p.Count, p => p.Weight);
       count = Math.Max(count, StandbyValidators.Length);
       HashSet<ECPoint> sv = new HashSet<ECPoint>(StandbyValidators);
       ECPoint[] pubkeys = validators.Find().Select(p => p.Value).Where(p => (p.Registered && p.Votes > Fixed8.Zero) || sv.Contains(p.PublicKey)).OrderByDescending(p => p.Votes).ThenBy(p => p.PublicKey).Select(p => p.PublicKey).Take(count).ToArray();
       IEnumerable<ECPoint> result;
       if (pubkeys.Length == count)
       {
            result = pubkeys;
       }
       else
       {
            HashSet<ECPoint> hashSet = new HashSet<ECPoint>(pubkeys);
            for (int i = 0; i < StandbyValidators.Length && hashSet.Count < count; i++)
                hashSet.Add(StandbyValidators[i]);
            result = hashSet;
       }
       return result.OrderBy(p => p);
}

6、議長選擇

在初始化共識狀態的時候會設置PrimaryIndex，獲知當前議長。原理就是簡單的MOD運算。這裏有分爲兩種狀況，若是節點正常則直接塊高度和記帳人數量mod運算便可，若是存在一場狀況，則須要根據view_number進行調整。

//file /Consensus/ConsensusService.cs   InitializeConsensus方法 
  if (view_number == 0)
     context.Reset(wallet);
  else
     context.ChangeView(view_number);

//file /Consensus/ConsensusContext.cs
public void ChangeView(byte view_number)
{
     int p = ((int)BlockIndex - view_number) % Validators.Length;
     State &= ConsensusState.SignatureSent;
     ViewNumber = view_number;
     PrimaryIndex = p >= 0 ? (uint)p : (uint)(p + Validators.Length);//當前記帳人
     if (State == ConsensusState.Initial)
     {
         TransactionHashes = null;
         Signatures = new byte[Validators.Length][];
     }
     ExpectedView[MyIndex] = view_number;
     _header = null;
}

//file /Consensus/ConsensusContext.cs
public void Reset(Wallet wallet)
{
     State = ConsensusState.Initial;
     PrevHash = Blockchain.Default.CurrentBlockHash;
     BlockIndex = Blockchain.Default.Height + 1;
     ViewNumber = 0;
     Validators = Blockchain.Default.GetValidators();
     MyIndex = -1;
     PrimaryIndex = BlockIndex % (uint)Validators.Length; //當前記帳人
     TransactionHashes = null;
     Signatures = new byte[Validators.Length][];
     ExpectedView = new byte[Validators.Length];
     KeyPair = null;
     for (int i = 0; i < Validators.Length; i++)
     {
         WalletAccount account = wallet.GetAccount(Validators[i]);
         if (account?.HasKey == true)
         {
            MyIndex = i;
            KeyPair = account.GetKey();
            break;
         }
     }
     _header = null;
}

7、狀態初始化

若是是議長則狀態標記爲ConsensusState.Primary，同時改變定時器觸發事件，再上次出塊15s後觸發。議員則設置狀態爲 ConsensusState.Backup，時間調整爲30s後觸發，若是議長不能正常工做，則這個觸發器會開始起做用（具體後邊再詳細分析）。

//file /Consensus/ConsensusContext.cs
private void InitializeConsensus(byte view_number)
{
    lock (context)
    {
        if (view_number == 0)
             context.Reset(wallet);
        else
             context.ChangeView(view_number);
        if (context.MyIndex < 0) return;
        Log($"initialize: height={context.BlockIndex} view={view_number} index={context.MyIndex} role={(context.MyIndex == context.PrimaryIndex ? ConsensusState.Primary : ConsensusState.Backup)}");
        if (context.MyIndex == context.PrimaryIndex)
        {
             context.State |= ConsensusState.Primary;
             if (!context.State.HasFlag(ConsensusState.SignatureSent))
             {
                 FillContext();  //生成mine區塊
             }
             if (context.TransactionHashes.Length > 1)
             {   //廣播自身的交易  
                 InvPayload invPayload = InvPayload.Create(InventoryType.TX, context.TransactionHashes.Skip(1).ToArray());
                 foreach (RemoteNode node in localNode.GetRemoteNodes())
                     node.EnqueueMessage("inv", invPayload);
             }
             timer_height = context.BlockIndex;
             timer_view = view_number;
             TimeSpan span = DateTime.Now - block_received_time;
             if (span >= Blockchain.TimePerBlock)
                timer.Change(0, Timeout.Infinite);
             else
                timer.Change(Blockchain.TimePerBlock - span, Timeout.InfiniteTimeSpan);
        }
        else
        {
             context.State = ConsensusState.Backup;
             timer_height = context.BlockIndex;
             timer_view = view_number;
             timer.Change(TimeSpan.FromSeconds(Blockchain.SecondsPerBlock << (view_number + 1)), Timeout.InfiniteTimeSpan);
        }
    }
}

8、議長髮起請求

議長到了該記帳的時間後，執行下面方法，發送MakePrepareRequest請求，自身狀態轉變爲RequestSent，也設置了30s後重復觸發定時器（一樣也是再議長工做異常時起效）。

//file /Consensus/ConsensusContext.cs
private void OnTimeout(object state)
{
    lock (context)
    {
         if (timer_height != context.BlockIndex || timer_view != context.ViewNumber) return;
         Log($"timeout: height={timer_height} view={timer_view} state={context.State}");
         if (context.State.HasFlag(ConsensusState.Primary) && !context.State.HasFlag(ConsensusState.RequestSent))
         {
             Log($"send perpare request: height={timer_height} view={timer_view}");
             context.State |= ConsensusState.RequestSent;
             if (!context.State.HasFlag(ConsensusState.SignatureSent))
             {
                 context.Timestamp = Math.Max(DateTime.Now.ToTimestamp(), Blockchain.Default.GetHeader(context.PrevHash).Timestamp + 1);
                 context.Signatures[context.MyIndex] = context.MakeHeader().Sign(context.KeyPair);
             }
             SignAndRelay(context.MakePrepareRequest());
             timer.Change(TimeSpan.FromSeconds(Blockchain.SecondsPerBlock << (timer_view + 1)), Timeout.InfiniteTimeSpan);
         }
         else if ((context.State.HasFlag(ConsensusState.Primary) && context.State.HasFlag(ConsensusState.RequestSent)) || context.State.HasFlag(ConsensusState.Backup))
         {
             RequestChangeView();
         }
    }
}

9、議員廣播簽名信息

議員接收到PrepareRequest，會對節點信息進行驗證，自身不存在的交易會去同步交易。交易同步完成後，驗證經過會進行發送PrepareResponse ，包含本身的簽名信息，表示本身已經經過了節點驗證。狀態轉變爲ConsensusState.SignatureSent。

//file /Consensus/ConsensusContext.cs
private void OnPrepareRequestReceived(ConsensusPayload payload, PrepareRequest message)
{
    Log($"{nameof(OnPrepareRequestReceived)}: height={payload.BlockIndex} view={message.ViewNumber} index={payload.ValidatorIndex} tx={message.TransactionHashes.Length}");
    if (!context.State.HasFlag(ConsensusState.Backup) || context.State.HasFlag(ConsensusState.RequestReceived))
        return;
    if (payload.ValidatorIndex != context.PrimaryIndex) return;
    if (payload.Timestamp <= Blockchain.Default.GetHeader(context.PrevHash).Timestamp || payload.Timestamp > DateTime.Now.AddMinutes(10).ToTimestamp())
    {
        Log($"Timestamp incorrect: {payload.Timestamp}");
        return;
    }
    context.State |= ConsensusState.RequestReceived;
    context.Timestamp = payload.Timestamp;
    context.Nonce = message.Nonce;
    context.NextConsensus = message.NextConsensus;
    context.TransactionHashes = message.TransactionHashes;
    context.Transactions = new Dictionary<UInt256, Transaction>();
    if (!Crypto.Default.VerifySignature(context.MakeHeader().GetHashData(), message.Signature, context.Validators[payload.ValidatorIndex].EncodePoint(false))) return;
    context.Signatures = new byte[context.Validators.Length][];
    context.Signatures[payload.ValidatorIndex] = message.Signature;
    Dictionary<UInt256, Transaction> mempool = LocalNode.GetMemoryPool().ToDictionary(p => p.Hash);
    foreach (UInt256 hash in context.TransactionHashes.Skip(1))
    {
    if (mempool.TryGetValue(hash, out Transaction tx))
    if (!AddTransaction(tx, false))
         return;
    }
    if (!AddTransaction(message.MinerTransaction, true)) return;
    if (context.Transactions.Count < context.TransactionHashes.Length)
    {
        UInt256[] hashes = context.TransactionHashes.Where(i => !context.Transactions.ContainsKey(i)).ToArray();
        LocalNode.AllowHashes(hashes);
        InvPayload msg = InvPayload.Create(InventoryType.TX, hashes);
        foreach (RemoteNode node in localNode.GetRemoteNodes())
           node.EnqueueMessage("getdata", msg);
    }
}

//file /Consensus/ConsensusContext.cs
private bool AddTransaction(Transaction tx, bool verify)
{
    if (Blockchain.Default.ContainsTransaction(tx.Hash) ||
        (verify && !tx.Verify(context.Transactions.Values)) ||
        !CheckPolicy(tx))
    {
        Log($"reject tx: {tx.Hash}{Environment.NewLine}{tx.ToArray().ToHexString()}");
        RequestChangeView();
        return false;
    }
    context.Transactions[tx.Hash] = tx;
    if (context.TransactionHashes.Length == context.Transactions.Count)
    {
        if (Blockchain.GetConsensusAddress(Blockchain.Default.GetValidators(context.Transactions.Values).ToArray()).Equals(context.NextConsensus))
        {
            Log($"send perpare response");
            context.State |= ConsensusState.SignatureSent;
            context.Signatures[context.MyIndex] = context.MakeHeader().Sign(context.KeyPair);
            SignAndRelay(context.MakePrepareResponse(context.Signatures[context.MyIndex]));
            CheckSignatures();
        }
        else
        {
            RequestChangeView();
            return false;
        }
    }
    return true;
}

10、共識達成後廣播區塊

其餘節點接收到PrepareResponse後，在本身的簽名列表中記錄對方的簽名信息，再檢查本身的簽名列表是否有超過三分之二的簽名了，有則判斷共識達成開始廣播生成的區塊。狀態狀變爲ConsensusState.BlockSent。

private void CheckSignatures()
{
    if (context.Signatures.Count(p => p != null) >= context.M && context.TransactionHashes.All(p => context.Transactions.ContainsKey(p)))
    {
        Contract contract = Contract.CreateMultiSigContract(context.M, context.Validators);
        Block block = context.MakeHeader();
        ContractParametersContext sc = new ContractParametersContext(block);
        for (int i = 0, j = 0; i < context.Validators.Length && j < context.M; i++)
        if (context.Signatures[i] != null)
    {
        sc.AddSignature(contract, context.Validators[i], context.Signatures[i]);
        j++;
    }
        sc.Verifiable.Scripts = sc.GetScripts();
        block.Transactions = context.TransactionHashes.Select(p => context.Transactions[p]).ToArray();
        Log($"relay block: {block.Hash}");
        if (!localNode.Relay(block))
            Log($"reject block: {block.Hash}");
        context.State |= ConsensusState.BlockSent;
    }
}

11、Blockchain_PersistCompleted後恢復狀態

區塊廣播後，節點接收到這個信息，會保存區塊，保存完成後觸發PersistCompleted事件，最終回到初始狀態，從新開始新一輪的共識。

private void Blockchain_PersistCompleted(object sender, Block block)
{
    Log($"persist block: {block.Hash}");
    block_received_time = DateTime.Now;
    InitializeConsensus(0);
}

12、錯誤分析

若是議長沒法完成記帳任務

這種狀況下議長在超出時間以後依然沒法達成共識，此時議員會增加自身的ExpectedView值，而且廣播出去，若是檢查到三分之二的成員viewnumber都加了1,則在這些相同viewnumber的節點之間開始新一輪共識，從新選擇議長，發起共識。若是此時原來的議長恢復正常，time到時間後會增加自身的viewnumber，慢慢追上當前的視圖狀態。

十3、參考

源碼 :[https://github.com/neo-project/neo/tree/master/neo/Consensus)

官方白皮書:http://docs.neo.org/zh-cn/basic/consensus/consensus.html

DBFT :https://www.jianshu.com/p/2383c7841d41

本文來源：NEOFANS

原文連接：http://neofans.org/

聯繫咱們

微博：https://weibo.com/u/6724929880

官網：https://neonext.club/

QQ羣：612334080

電報：https://t.me/neonextop

twitter：https://twitter.com/NE0NEXT