dpdk在虛擬機上出錯處理

時間  2019-11-06
標籤 dpdk 虛擬 機上 出錯 處理 简体版
原文   原文鏈接

正文html

DPDK介紹見：www.dpdk.orgnode

本文介紹的步驟基本適用於dpdk 1.7.0 - dpdk 2.0.0 各版本。只是setup.sh顯示的菜單有一些小的不一樣； 
一樣的，也適用於ubuntu更高版本（已在ubuntu 12.04+及14.04上驗證過)python

回到頂部linux

1. 所用系統與軟件版本

系統：Ubuntu 12.04.3 LTS 64位， CentOS Linux release 7.0.1406 64位 
dpdk: 1.7.0 (下載頁）git

dpdk 1.7.1 通過試驗，發如今這兩個系統上都有問題， 運行各示例程序都有如下錯誤 
  EAL: Error reading from file descriptorubuntu

這個bug已經由dpdk的開發人員修復，patch內容以下：app

複製代碼

diff --git a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
index d1ca26e..c46a00f 100644
--- a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
+++ b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
@@ -505,14 +505,11 @@ igbuio_pci_probe(struct pci_dev *dev, const struct pci_device_id *id)
         }
         /* fall back to INTX */
     case RTE_INTR_MODE_LEGACY:
-        if (pci_intx_mask_supported(dev)) {
-            dev_dbg(&dev->dev, "using INTX");
-            udev->info.irq_flags = IRQF_SHARED;
-            udev->info.irq = dev->irq;
-            udev->mode = RTE_INTR_MODE_LEGACY;
-            break;
-        }
-        dev_notice(&dev->dev, "PCI INTX mask not supported\n");
+        dev_dbg(&dev->dev, "using INTX");
+        udev->info.irq_flags = IRQF_SHARED;
+        udev->info.irq = dev->irq;
+        udev->mode = RTE_INTR_MODE_LEGACY;
+        break;
         /* fall back to no IRQ */
     case RTE_INTR_MODE_NONE:
         udev->mode = RTE_INTR_MODE_NONE;

複製代碼

在虛擬機裏使用時，打上以上的補丁，或手工修改文件後從新編譯便可。socket

回到頂部工具

2. 虛擬機配置

虛擬機軟件：VMWare WorkStation 10.0.1 build-1379776 
CPU： 2個CPU， 每一個CPU2個核心 
內存： 1GB+ 
網卡：intel網卡*2， 用於dpdk試驗；另外一塊網卡用於和宿主系統進行通訊開發工具

回到頂部

3. Ubuntu 12.04上的配置

3.1 準備

須要安裝gcc及其餘一些小工具等，默認都有了，沒有的話運行sudo apt-get install裝一下。dkdk的一些腳本用到了python，也裝一下。

3.2 經過setup腳本進行配置

首先運行su切換到root權限，root沒有開的話使用

sudo passwd root

來開一下。

dpdk提供了一個方便的配置腳本： <dpdk>/tools/setup.sh，經過它能夠方便地配置環境。 
1） 設置環境變量，這裏是linux 64位的配置

export RTE_SDK=<dpdk主目錄>
export RTE_TARGET=x86_64-native-linuxapp-gcc

2）運行setup.sh，顯示以下

複製代碼

------------------------------------------------------------------------------
 RTE_SDK exported as /home/hack/dpdk-1.7.0
------------------------------------------------------------------------------
----------------------------------------------------------
 Step 1: Select the DPDK environment to build
----------------------------------------------------------
[1] i686-native-linuxapp-gcc
[2] i686-native-linuxapp-icc
[3] x86_64-ivshmem-linuxapp-gcc
[4] x86_64-ivshmem-linuxapp-icc
[5] x86_64-native-bsdapp-gcc
[6] x86_64-native-linuxapp-gcc
[7] x86_64-native-linuxapp-icc

----------------------------------------------------------
 Step 2: Setup linuxapp environment
----------------------------------------------------------
[8] Insert IGB UIO module
[9] Insert VFIO module
[10] Insert KNI module
[11] Setup hugepage mappings for non-NUMA systems
[12] Setup hugepage mappings for NUMA systems
[13] Display current Ethernet device settings
[14] Bind Ethernet device to IGB UIO module
[15] Bind Ethernet device to VFIO module
[16] Setup VFIO permissions

----------------------------------------------------------
 Step 3: Run test application for linuxapp environment
----------------------------------------------------------
[17] Run test application ($RTE_TARGET/app/test)
[18] Run testpmd application in interactive mode ($RTE_TARGET/app/testpmd)

----------------------------------------------------------
 Step 4: Other tools
----------------------------------------------------------
[19] List hugepage info from /proc/meminfo

----------------------------------------------------------
 Step 5: Uninstall and system cleanup
----------------------------------------------------------
[20] Uninstall all targets
[21] Unbind NICs from IGB UIO driver
[22] Remove IGB UIO module
[23] Remove VFIO module
[24] Remove KNI module
[25] Remove hugepage mappings

[26] Exit Script

複製代碼

選擇6， 進行編譯

3）選擇8， 插入igb_uio模塊

4）選擇11，配置大頁內存（非NUMA），選擇後會提示你選擇頁數，輸入64，128什麼的便可

複製代碼

Removing currently reserved hugepages
Unmounting /mnt/huge and removing directory

  Input the number of 2MB pages
  Example: to have 128MB of hugepages available, enter '64' to
  reserve 64 * 2MB pages
Number of pages: 128

複製代碼

選擇19，能夠確認一下大頁內存的配置：

AnonHugePages:         0 kB
HugePages_Total:     128
HugePages_Free:      128
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

5）選擇14， 綁定dpdk要使用的網卡

複製代碼

Network devices using DPDK-compatible driver
============================================
<none>

Network devices using kernel driver
===================================
0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth0 drv=e1000 unused=igb_uio *Active*
0000:02:06.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth1 drv=e1000 unused=igb_uio 
0000:02:07.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth2 drv=e1000 unused=igb_uio 

Other network devices
=====================
<none>

Enter PCI address of device to bind to IGB UIO driver: 0000:02:06.0

複製代碼

綁定好後，選擇13，能夠查看當前的網卡配置：

複製代碼

Network devices using DPDK-compatible driver
============================================
0000:02:06.0 '82545EM Gigabit Ethernet Controller (Copper)' drv=igb_uio unused=e1000
0000:02:07.0 '82545EM Gigabit Ethernet Controller (Copper)' drv=igb_uio unused=e1000

Network devices using kernel driver
===================================
0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth0 drv=e1000 unused=igb_uio *Active*

Other network devices
=====================
<none>

複製代碼

6）選擇18， 運行testpmd測試程序

注意，運行這個測試程序，虛擬機最好提供2個網卡用於dpdk。

  Enter hex bitmask of cores to execute testpmd app on
  Example: to execute app on cores 0 to 7, enter 0xff
bitmask: f

若是沒問題，按回車後會出現如下輸出：

複製代碼

Launching app
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 0 on socket 0
EAL: Detected lcore 3 as core 1 on socket 0
EAL: Support maximum 64 logical core(s) by configuration.
EAL: Detected 4 lcore(s)
EAL: Setting up memory...
EAL: Ask a virtual area of 0xf000000 bytes
EAL: Virtual area found at 0x7fe828000000 (size = 0xf000000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe827c00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe827800000 (size = 0x200000)
EAL: Ask a virtual area of 0x800000 bytes
EAL: Virtual area found at 0x7fe826e00000 (size = 0x800000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fe826800000 (size = 0x400000)
EAL: Requesting 128 pages of size 2MB from socket 0
EAL: TSC frequency is ~3292453 KHz
EAL: Master core 0 is ready (tid=37c79800)
EAL: Core 3 is ready (tid=24ffc700)
EAL: Core 2 is ready (tid=257fd700)
EAL: Core 1 is ready (tid=25ffe700)
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL:   0000:02:01.0 not managed by UIO driver, skipping
EAL: PCI device 0000:02:06.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL:   PCI memory mapped at 0x7fe837c23000
EAL:   PCI memory mapped at 0x7fe837c13000
EAL: PCI device 0000:02:07.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL:   PCI memory mapped at 0x7fe837bf3000
EAL:   PCI memory mapped at 0x7fe837be3000
Interactive-mode selected
Configuring Port 0 (socket 0)
Port 0: 00:0C:29:14:50:CE
Configuring Port 1 (socket 0)
Port 1: 00:0C:29:14:50:D8
Checking link statuses...
Port 0 Link Up - speed 1000 Mbps - full-duplex
Port 1 Link Up - speed 1000 Mbps - full-duplex
Done
testpmd>

複製代碼

輸入start， 開始包轉發

複製代碼

testpmd> start
  io packet forwarding - CRC stripping disabled - packets/burst=32
  nb forwarding cores=1 - nb forwarding ports=2
  RX queues=1 - RX desc=128 - RX free threshold=0
  RX threshold registers: pthresh=8 hthresh=8 wthresh=0
  TX queues=1 - TX desc=512 - TX free threshold=0
  TX threshold registers: pthresh=32 hthresh=0 wthresh=0
  TX RS bit threshold=0 - TXQ flags=0x0

複製代碼

輸入stop，中止包轉發，這時會顯示統計信息

複製代碼

testpmd> stop
Telling cores to stop...
Waiting for lcores to finish...

  ---------------------- Forward statistics for port 0  ----------------------
  RX-packets: 5544832        RX-dropped: 0             RX-total: 5544832
  TX-packets: 5544832        TX-dropped: 0             TX-total: 5544832
  ----------------------------------------------------------------------------

  ---------------------- Forward statistics for port 1  ----------------------
  RX-packets: 5544832        RX-dropped: 0             RX-total: 5544832
  TX-packets: 5544832        TX-dropped: 0             TX-total: 5544832
  ----------------------------------------------------------------------------

  +++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++
  RX-packets: 11089664       RX-dropped: 0             RX-total: 11089664
  TX-packets: 11089664       TX-dropped: 0             TX-total: 11089664
  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Done.

複製代碼

3.3 經過命令配置

最好切換到root權限。

1）編譯dpdk

進入dpdk主目錄<dpdk>，輸入

make install T=x86_64-native-linuxapp-gcc

進行編譯

2）配置大頁內存（非NUMA）

echo 128 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
mkdir /mnt/huge
mount -t hugetlbfs nodev /mnt/huge

能夠用如下命令查看大頁內存狀態：

cat /proc/meminfo | grep Huge

3）安裝igb_uio驅動

modprobe uio
insmod x86_64-native-linuxapp-gcc/kmod/igb_uio.ko

4）綁定網卡

先看一下當前網卡的狀態

複製代碼

 ./tools/dpdk_nic_bind.py --status

Network devices using DPDK-compatible driver
============================================
<none>

Network devices using kernel driver
===================================
0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth0 drv=e1000 unused=igb_uio *Active*

Other network devices
=====================
0000:02:06.0 '82545EM Gigabit Ethernet Controller (Copper)' unused=e1000,igb_uio
0000:02:07.0 '82545EM Gigabit Ethernet Controller (Copper)' unused=e1000,igb_uio

複製代碼

進行綁定：

./tools/dpdk_nic_bind.py -b igb_uio 0000:02:06.0
./tools/dpdk_nic_bind.py -b igb_uio 0000:02:07.0

若是網卡有接口名，如eth1, eth2, 也能夠在-b igb_uio後面使用接口名， 而不使用pci地址。

5） 運行testpmd測試程序

 ./x86_64-native-linuxapp-gcc/app/testpmd -c 0x3 -n 2 -- -i

6）編譯運行其餘示例程序

<dpdk>/examples下面有不少示例程序，這些程序在dpdk編譯時，沒有被編譯。這裏以編譯helloworld爲例，首先要設置環境變量：

export RTE_SDK=<dpdk主目錄>
export RTE_TARGET=x86_64-native-linuxapp-gcc

以後進入<dpdk>/examples/helloworld，運行make，成功會生成build目錄，其中有編譯好的helloworld程序。

回到頂部

4. CentOS 7.0上的配置

4.1 準備

安裝CentOS虛擬機時，若是選擇minimal安裝，還須要安裝其下的基本開發工具集(含gcc，python等）

另外，dpdk提供的dpdk_nic_bind.py腳本中會調用到lspci命令，這個默認沒有安裝，運行如下命令安裝(不安裝此工具則沒法綁定網卡）：

yum install pciutils

ifconfig默認也沒有安裝，若是想用它，應運行：

yum install net-tools

在CentOS上，要綁定給dpdk使用的網卡在綁定前，多是活動的(active)，應將其禁用，不然沒法綁定。禁用的一種方式是運行：

ifconfig eno33554984 down

eno33554984是接口名，如同eth0同樣。

在CentOS上使用setup.sh和經過命令編譯和配置dpdk的過程與Ubuntu同樣，這裏就從略了。

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程