Multi Channel DMA Intel® FPGA IP for PCI Express Design Example User Guide

Download PDF
ID 683517
Date 4/29/2022
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Introduction 2. Design Example Detailed Description 3. Design Example Quick Start Guide 4. User Guide Archives 5. Revision History for the Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide
1. Introduction x
1.1. Terms and Acronyms 1.2. MCDMA IP Modes
2. Design Example Detailed Description x
2.1. Design Example Overview 2.2. Hardware and Software Requirements 2.3. PIO Using MCDMA Bypass Mode 2.4. Avalon-ST Packet Generate/Check 2.5. Avalon-ST Device-side Packet Loopback 2.6. Avalon-MM DMA 2.7. BAM_BAS Traffic Generator and Checker 2.8. External Descriptor Controller
2.1. Design Example Overview x
2.1.1. MCDMA H-Tile Design Examples for Endpoint 2.1.2. MCDMA P-Tile Design Examples for Endpoint 2.1.3. MCDMA F-Tile Design Examples for Endpoint
2.3. PIO Using MCDMA Bypass Mode x
2.3.1. Avalon-ST PIO Using MCDMA Bypass Mode 2.3.2. Avalon-MM PIO Using MCDMA Bypass mode
2.3.1. Avalon-ST PIO Using MCDMA Bypass Mode x
2.3.1.1. Four-Port Avalon-ST PIO Using MCDMA Bypass Mode 2.3.1.2. Single-Port Avalon-ST PIO Using MCDMA Bypass Mode
2.3.1.1. Four-Port Avalon-ST PIO Using MCDMA Bypass Mode x
2.3.1.1.1. Simulation Result 2.3.1.1.2. Hardware Test Result
2.3.2. Avalon-MM PIO Using MCDMA Bypass mode x
2.3.2.1. Simulation Results 2.3.2.2. Hardware Test Results
2.4. Avalon-ST Packet Generate/Check x
2.4.1. Four-Port Avalon-ST Packet Generate/Check 2.4.2. Single-Port Avalon-ST Packet Generate/Check
2.4.1. Four-Port Avalon-ST Packet Generate/Check x
2.4.1.1. Simulation Results 2.4.1.2. Hardware Test Results
2.4.2. Single-Port Avalon-ST Packet Generate/Check x
2.4.2.1. Simulation Waveforms 2.4.2.2. Simulation Log
2.5. Avalon-ST Device-side Packet Loopback x
2.5.1. Simulation Results 2.5.2. Hardware Test Results
2.6. Avalon-MM DMA x
2.6.1. Simulation Results 2.6.2. Hardware Test Results
2.7. BAM_BAS Traffic Generator and Checker x
2.7.1. BAM_BAS Traffic Generator and Checker Example Design Register Map
2.8. External Descriptor Controller x
2.8.1. Registers 2.8.2. Hardware Test Results
2.8.1. Registers x
2.8.1.1. GCSR Registers (Base address 64’h00000) 2.8.1.2. QCSR Registers(Base address 64’h10000)
3. Design Example Quick Start Guide x
3.1. Design Example Directory Structure 3.2. Generating the Example Design using Intel® Quartus® Prime 3.3. Simulating the Design Example 3.4. Compiling the Example Design in Intel® Quartus® Prime 3.5. Running the Design Example Application on a Hardware Setup
3.2. Generating the Example Design using Intel® Quartus® Prime x
3.2.1. Procedure
3.3. Simulating the Design Example x
3.3.1. Testbench Overview 3.3.2. Supported Simulators 3.3.3. Example Testbench Flow for DMA Test with Avalon-ST Packet Generate/Check Design Example 3.3.4. Run the Simulation Script 3.3.5. View the Results
3.5. Running the Design Example Application on a Hardware Setup x
3.5.1. Program the FPGA 3.5.2. Quick Start Guide
3.5.2. Quick Start Guide x
3.5.2.1. Software Test Setup 3.5.2.2. Driver Support 3.5.2.3. MCDMA Custom Driver 3.5.2.4. MCDMA DPDK Poll Mode Driver 3.5.2.5. MCDMA Kernel Mode Character Device Driver 3.5.2.6. MCDMA Kernel Mode Network Device Driver
3.5.2.3. MCDMA Custom Driver x
3.5.2.3.1. Prerequisites 3.5.2.3.2. Software Setup 3.5.2.3.3. Run the Reference Example Application 3.5.2.3.4. Testing Bitstream Configuration beyond 256 Channels 3.5.2.3.5. External Descriptor Mode Verification
3.5.2.3.1. Prerequisites x
3.5.2.3.1.1. Configuration Changes from BIOS 3.5.2.3.1.2. External Packages 3.5.2.3.1.3. Set the Boot Parameters
3.5.2.3.2. Software Setup x
3.5.2.3.2.1. Installing the Linux Kernel Driver 3.5.2.3.2.2. Build and Install User Space Library 3.5.2.3.2.3. Enabling VFs and Create Guest VM by Using QEMU 3.5.2.3.2.4. Establish Communication Between Host and QEMU
3.5.2.3.3. Run the Reference Example Application x
3.5.2.3.3.1. Meta Data Test 3.5.2.3.3.2. BAM Test 3.5.2.3.3.3. BAS Test 3.5.2.3.3.4. Packet Gen Test
3.5.2.4. MCDMA DPDK Poll Mode Driver x
3.5.2.4.1. Prerequisites 3.5.2.4.2. Install PMD and Test Application 3.5.2.4.3. Run the Reference Example Application
3.5.2.4.1. Prerequisites x
3.5.2.4.1.1. Set the Boot Parameters 3.5.2.4.1.2. Configuration Changes from BIOS 3.5.2.4.1.3. Install and Build Testpmd
3.5.2.4.2. Install PMD and Test Application x
3.5.2.4.2.1. Create VM Environment
3.5.2.4.3. Run the Reference Example Application x
3.5.2.4.3.1. Example of Verifying on an AVMM Design 3.5.2.4.3.2. BAM Test 3.5.2.4.3.3. BAS Test
3.5.2.5. MCDMA Kernel Mode Character Device Driver x
3.5.2.5.1. Prerequisites 3.5.2.5.2. Software Setup 3.5.2.5.3. Examples
3.5.2.5.1. Prerequisites x
3.5.2.5.1.1. Host Configuration 3.5.2.5.1.2. Set the Boot Parameters
3.5.2.5.2. Software Setup x
3.5.2.5.2.1. Build and Install Kernel Driver 3.5.2.5.2.2. Build and Install Libmcmem 3.5.2.5.2.3. Build Testapp 3.5.2.5.2.4. Configure Kernel Driver 3.5.2.5.2.5. Configure Libmcmem 3.5.2.5.2.6. Configure Testapp
3.5.2.6. MCDMA Kernel Mode Network Device Driver x
3.5.2.6.1. Build and Install Netdev Driver 3.5.2.6.2. Enable VFs if SRIOV is Supported 3.5.2.6.3. Configure the Number of Channels Supported on the Device 3.5.2.6.4. Configure the MTU Value 3.5.2.6.5. Configure the Device Communication 3.5.2.6.6. Configure Transmit Queue Selection Mechanism 3.5.2.6.7. Test Procedure by Using Name Space Environment 3.5.2.6.8. PIO Test
3.5.2.6.6. Configure Transmit Queue Selection Mechanism x
3.5.2.6.6.1. MCDMA Queue Selection 3.5.2.6.6.2. Transmit Packet Steering (XPS) 3.5.2.6.6.3. Default
1. Introduction
2. Design Example Detailed Description
3. Design Example Quick Start Guide
4. User Guide Archives
5. Revision History for the Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide

2.4.1.2. Hardware Test Results

Note: The PIO test was run with MCDMA H-Tile.
The Custom Driver was used to generate the following output:
Figure 15. PIO Test -o option
Figure 16. H2D Avalon-ST Streaming-t option. Note: This hardware test was run with the Intel® Stratix® 10 GX H-tile PCIe Gen3 x16 configuration. Hardware test result with P-Tile Gen4 x16 may be added in a future release.
Note: In the example above, the perfq_app command transfers 250 MB of total transfer size with payload of 8192 bytes in each descriptor in H2D direction (-t) for four channels. Without -v (data validation) option, the command displays the bandwidth.
The -p option specifies the payload size. The maximum payload size varies depending on the design examples as follows:
  • Loopback: 1 MB
  • Avalon-MM:
    • With validation enabled: ((total available memory) / #channels)
    • With validation disabled: 1 MB
  • Avalon-ST Packet Generate/Check: 1 MB
The -s option specifies the transfer size.
  • For loopback, Avalon-ST and Avalon-MM design examples (except for the Avalon-ST Packet Generate/Check design example), there is no limit on the transfer size.
  • For the Avalon-ST Packet Generate/Check design example, the number of descriptors (transfer size / packet size) should be a modulus of 64 (64 is the default file size).

The -a option specifies the number of threads. For this option, you can provide any number that is a factor of the total number of queues required to distribute the traffic equally among the available cores in the system.

For example, for a system with 64 channels of bidirectional traffic, there is a maximum of 128 possible queues. Hence, the -a option can accept these values: 1,2,4,8,16,32,64,128. If you use -a 128, the 128 queues are distributed among 128 cores. However, if the number of cores in the system is limited, you can use smaller values for a. If you use -a 4, the 128 queues are distributed among 4 cores (with each core handling 32 queues). A higher number of queues per core does lead to a decrease in performance.

Figure 17. H2D Avalon-ST Streaming with Data Validation Enabled-t with -v option. Note: This hardware test was run with the Intel® Stratix® 10 GX H-tile PCIe Gen3 x16 configuration. Hardware test result with P-Tile Gen4 x16 may be added in a future release.
Figure 18. D2H Avalon-ST Streaming-r option. Note: This hardware test was run with the Intel® Stratix® 10 GX H-tile PCIe Gen3 x16 configuration. Hardware test result with P-Tile Gen4 x16 may be added in a future release.
Figure 19. D2H Avalon-ST Streaming with Data Validation Enabled-r with -v option. Note: This hardware test was run with the Intel® Stratix® 10 GX H-tile PCIe Gen3 x16 configuration. Hardware test result with P-Tile Gen4 x16 may be added in a future release.
Figure 20. Bidirectional Avalon-ST Streaming-z option. Note: This hardware test was run with the Intel® Stratix® 10 GX H-tile PCIe Gen3 x16 configuration. Hardware test result with P-Tile Gen4 x16 may be added in a future release.
Figure 21. Bidirectional Avalon-ST Streaming with Data Validation Enabled -z with -v option. Note: This hardware test was run with the Intel® Stratix® 10 GX H-tile PCIe Gen3 x16 configuration. Hardware test result with P-Tile Gen4 x16 may be added in a future release.
Level Two Title