L-Tile and H-Tile Avalon® Memory-mapped+ Intel® FPGA IP for PCI Express* User Guide

Download PDF
ID 683527
Date 3/05/2024
Public
Document Table of Contents
Document Table of Contents x
1. Introduction 2. Quick Start Guide 3. Block and Interface Descriptions 4. Parameters 5. Designing with the IP Core 6. Registers 7. Design Example and Testbench 8. Document Revision History for Intel® L- and H-tile Avalon® Memory-mapped+ IP for PCI Express* User Guide A. Avalon-MM IP Variants Comparison B. Root Port BFM C. BFM Procedures and Functions D. Troubleshooting and Observing the Link Status E. Root Port Enumeration
1. Introduction x
1.1. Features 1.2. Device Family Support 1.3. Release Information 1.4. Recommended Speed Grades 1.5. Resource Utilization 1.6. Transceiver Tiles 1.7. Channel Availability
2. Quick Start Guide x
2.1. Design Components 2.2. Directory Structure 2.3. Generating the Design Example 2.4. Simulating the Design Example 2.5. Compiling the Design Example and Programming the Device 2.6. Installing the Linux Kernel Driver 2.7. Running the Design Example Application 2.8. Ensuring the Design Example Meets Timing Requirements
3. Block and Interface Descriptions x
3.1. Block Descriptions 3.2. Interface Descriptions
3.1. Block Descriptions x
3.1.1. Tags
3.2. Interface Descriptions x
3.2.1. Avalon-MM Interface Summary 3.2.2. Clocks and Resets 3.2.3. System Interfaces
3.2.1. Avalon-MM Interface Summary x
3.2.1.1. Read Data Mover (RDDM) Interfaces 3.2.1.2. Write Data Mover (WRDM) Interfaces 3.2.1.3. Bursting Avalon-MM Slave (BAS) Interface 3.2.1.4. Bursting Avalon-MM Master (BAM) Interface
3.2.1.1. Read Data Mover (RDDM) Interfaces x
3.2.1.1.1. Read Data Mover Avalon-MM Write Master and Conduit 3.2.1.1.2. Read Data Mover Avalon-ST Descriptor Sinks 3.2.1.1.3. Read Data Mover Status Avalon-ST Source
3.2.1.2. Write Data Mover (WRDM) Interfaces x
3.2.1.2.1. Write Data Mover Avalon-MM Read Master and Conduit 3.2.1.2.2. Write Data Mover Avalon-ST Descriptor Sinks 3.2.1.2.3. Write Data Mover Status Avalon-ST Source
3.2.1.4. Bursting Avalon-MM Master (BAM) Interface x
3.2.1.4.1. PCIe Address to Avalon-MM Address Mapping
3.2.3. System Interfaces x
3.2.3.1. Serial Data Interface 3.2.3.2. PIPE Interface 3.2.3.3. Interrupts 3.2.3.4. Configuration Output Interface 3.2.3.5. Flush Requests 3.2.3.6. Reconfiguration 3.2.3.7. Hard IP Status and Link Training Conduit 3.2.3.8. Bus Master Enable (BME) Conduit 3.2.3.9. Test Conduit
3.2.3.3. Interrupts x
3.2.3.3.1. Interrupt Signals Available when the PCIe Hard IP is an Endpoint 3.2.3.3.2. MSI
3.2.3.6. Reconfiguration x
3.2.3.6.1. PCIe Hard IP Reconfiguration 3.2.3.6.2. Transceiver Reconfiguration 3.2.3.6.3. PLL Reconfiguration
4. Parameters x
4.1. Avalon-MM Settings 4.2. Base Address Registers 4.3. Device Identification Registers 4.4. PCI Express and PCI Capabilities Parameters 4.5. Configuration, Debug and Extension Options 4.6. PHY Characteristics 4.7. Example Designs
4.4. PCI Express and PCI Capabilities Parameters x
4.4.1. Device Capabilities 4.4.2. Link Capabilities 4.4.3. MSI and MSI-X Capabilities 4.4.4. Power Management 4.4.5. Vendor Specific Extended Capability (VSEC)
5. Designing with the IP Core x
5.1. Generation 5.2. Simulation 5.3. IP Core Generation Output ( Quartus® Prime Pro Edition) 5.4. Channel Layout and PLL Usage
6. Registers x
6.1. Configuration Space Registers
6.1. Configuration Space Registers x
6.1.1. Register Access Definitions 6.1.2. PCI Configuration Header Registers 6.1.3. PCI Express Capability Structures 6.1.4. Intel Defined VSEC Capability Header 6.1.5. Uncorrectable Internal Error Status Register 6.1.6. Uncorrectable Internal Error Mask Register 6.1.7. Correctable Internal Error Status Register 6.1.8. Correctable Internal Error Mask Register
6.1.4. Intel Defined VSEC Capability Header x
6.1.4.1. Intel Defined Vendor Specific Header 6.1.4.2. Intel Marker 6.1.4.3. JTAG Silicon ID 6.1.4.4. User Configurable Device and Board ID
7. Design Example and Testbench x
7.1. Overview of the Example Design 7.2. Overview of the Testbench
7.1. Overview of the Example Design x
7.1.1. Block Descriptions 7.1.2. Programming Model for the Example Design 7.1.3. DMA Operations Using the Example Design
7.1.1. Block Descriptions x
7.1.1.1. DMA Controller (DMA Example Design Only) 7.1.1.2. Traffic Generator and Checker (BAS Example Design Only) 7.1.1.3. Avalon-MM Address to PCIe Address Mapping 7.1.1.4. BAR Interpreter
7.1.1.1. DMA Controller (DMA Example Design Only) x
7.1.1.1.1. Register Set 7.1.1.1.2. Deadlock Risk and Avoidance 7.1.1.1.3. Interrupts 7.1.1.1.4. Using the DMA Controller
7.1.1.2. Traffic Generator and Checker (BAS Example Design Only) x
7.1.1.2.1. Register Set
7.1.3. DMA Operations Using the Example Design x
7.1.3.1. Read DMA Example 7.1.3.2. Write DMA Example 7.1.3.3. Using the Traffic Generator (BAS Example Design Only) 7.1.3.4. Using the Traffic Checker (BAS Example Design Only)
B. Root Port BFM x
B.1. Root Port BFM Overview B.2. Issuing Read and Write Transactions to the Application Layer B.3. Configuration of Root Port and Endpoint B.4. Configuration Space Bus and Device Numbering B.5. BFM Memory Map
C. BFM Procedures and Functions x
C.1. ebfm_barwr Procedure C.2. ebfm_barwr_imm Procedure C.3. ebfm_barrd_wait Procedure C.4. ebfm_barrd_nowt Procedure C.5. ebfm_cfgwr_imm_wait Procedure C.6. ebfm_cfgwr_imm_nowt Procedure C.7. ebfm_cfgrd_wait Procedure C.8. ebfm_cfgrd_nowt Procedure C.9. BFM Configuration Procedures C.10. BFM Shared Memory Access Procedures C.11. BFM Log and Message Procedures C.12. Verilog HDL Formatting Functions
C.9. BFM Configuration Procedures x
C.9.1. ebfm_cfg_rp_ep Procedure C.9.2. ebfm_cfg_decode_bar Procedure
C.10. BFM Shared Memory Access Procedures x
C.10.1. Shared Memory Constants C.10.2. shmem_write Procedure C.10.3. shmem_read Function C.10.4. shmem_display Verilog HDL Function C.10.5. shmem_fill Procedure C.10.6. shmem_chk_ok Function
C.11. BFM Log and Message Procedures x
C.11.1. ebfm_display Verilog HDL Function C.11.2. ebfm_log_stop_sim Verilog HDL Function C.11.3. ebfm_log_set_suppressed_msg_mask Task C.11.4. ebfm_log_set_stop_on_msg_mask Verilog HDL Task C.11.5. ebfm_log_open Verilog HDL Function C.11.6. ebfm_log_close Verilog HDL Function
C.12. Verilog HDL Formatting Functions x
C.12.1. himage1 C.12.2. himage2 C.12.3. himage4 C.12.4. himage8 C.12.5. himage16 C.12.6. dimage1 C.12.7. dimage2 C.12.8. dimage3 C.12.9. dimage4 C.12.10. dimage5 C.12.11. dimage6 C.12.12. dimage7
D. Troubleshooting and Observing the Link Status x
D.1. Troubleshooting D.2. PCIe Link Inspector Overview
D.1. Troubleshooting x
D.1.1. Simulation Fails to Progress Beyond Polling.Active State D.1.2. Hardware Bring-Up Issues D.1.3. Link Training D.1.4. Use Third-Party PCIe Analyzer
D.2. PCIe Link Inspector Overview x
D.2.1. PCIe* Link Inspector Hardware
D.2.1. PCIe* Link Inspector Hardware x
D.2.1.1. Enabling the PCIe* Link Inspector D.2.1.2. Launching the PCIe* Link Inspector D.2.1.3. The PCIe* Link Inspector LTSSM Monitor D.2.1.4. Accessing the Configuration Space and Transceiver Registers D.2.1.5. Additional Status Commands
D.2.1.3. The PCIe* Link Inspector LTSSM Monitor x
D.2.1.3.1. LTSSM Monitor Registers D.2.1.3.2. ltssm_save2file <file_name> D.2.1.3.3. ltssm_file2console D.2.1.3.4. ltssm_debug_check D.2.1.3.5. ltssm_display <num_states> D.2.1.3.6. ltssm_save_oldstates
D.2.1.4. Accessing the Configuration Space and Transceiver Registers x
D.2.1.4.1. PCIe* Link Inspector Commands D.2.1.4.2. NPDME PLL and Channel Commands D.2.1.4.3. Register Address Map
D.2.1.5. Additional Status Commands x
D.2.1.5.1. Displaying PLL Lock and Calibration Status Registers
1. Introduction
2. Quick Start Guide
3. Block and Interface Descriptions
4. Parameters
5. Designing with the IP Core
6. Registers
7. Design Example and Testbench
8. Document Revision History for Intel® L- and H-tile Avalon® Memory-mapped+ IP for PCI Express* User Guide
A. Avalon-MM IP Variants Comparison
B. Root Port BFM
C. BFM Procedures and Functions
D. Troubleshooting and Observing the Link Status
E. Root Port Enumeration

7.1. Overview of the Example Design

The available example design is for an Endpoint, with a single function and no SR-IOV support.

This DMA example design includes a DMA Controller and an on-chip memory to exercise the Data Movers, and a Traffic Generator and Checker to exercise the Bursting Slave.

The example design also connects the Bursting Master to the on-chip memory to allow high-throughput transfers should the host or some other component of the PCIe system be capable of initiating such transfers (e.g. a Root Complex with a DMA engine).

An address translation module is inserted between the Traffic Generator and the Bursting Slave to show an example of how address translation can be performed between the Avalon® and PCIe address spaces.

The on-chip memory that the Data Movers and the Bursting Master connect to is a dual-port memory to allow full-duplex data movement.

The Bursting Master connects to a BAR Interpreter module, which combines the address and BAR number and allows the Bursting Master to control the DMA Controller, the Traffic Generator and the Address Mapper. The BAR Interpreter also connects the Bursting Master to the dual-port memory.

Figure 26. DMA Example Design

The example design is generated dynamically based on the selected variation of the Intel L-/H-Tile Avalon-MM+ for PCI Express IP. However, some of the user’s parameter selections may need to be overwritten to ensure proper functionality. A warning appears when such a need arises.

The DMA Controller is instantiated for variations that implement at least one of the Data Movers and the Bursting Master. The Traffic Generator and Checker module is instantiated for variations that implement at least the Bursting Slave and the Bursting Master.

You cannot generate an example design for variations that do not implement the Bursting Master as it is necessary for all flavors of the example design. A warning appears to explain that the Bursting Master with appropriate BAR settings must be added in order to generate the example design.

There are three variations currently supported:
  • PIO: This is the variation where only the Bursting Master is enabled. The design example simulation exercises the Bursting Master to perform simple one dword Reads and Writes to the on-chip memory. The BAR Interpreter and on-chip memory are also included in this variation.
  • DMA: This is the variation where the Bursting Master, and both of the Read Data Mover and Write Data Mover are enabled. Software sends instructions via the Bursting Master to the Read or Write Data Movers to initiate DMA Reads or Writes to the system memory. The BAR Interpreter, on-chip memory and DMA Controller are also included.
  • BAS: This is the variation where the Bursting Master and Bursting Slave are enabled.
    • Software writes an instruction via the Bursting Master to the Traffic Generator to generate a block of data, which is then transmitted to the Bursting Slave. Upon receiving the data, the Bursting Slave performs a Memory Write to transfer the data to the PCIe system memory.
    • Software then writes another instruction via the Bursting Master to the Traffic Checker, which issues a Read to the Bursting Slave. The Bursting Slave then forms a Memory Read request to fetch the data from the PCIe system memory.
    The BAR Interpreter, on-chip memory, Traffic Generator and Address Mapper are included in this variation.
Note: Beginning with the 17.1 release, the Quartus® Prime Pro Edition software dynamically generates example designs for the parameters you specify in the parameter editor. Consequently, the Quartus® Prime Pro Edition installation directory no longer provides static example designs for Stratix® 10 devices. Static example designs are available for earlier device families, including Arria® 10 and Cyclone® 10 devices.

The following figure shows the system-level view in Platform Designer of the DMA example design.

Figure 27. Stratix 10 Intel L-/H-Tile Avalon-MM+ for PCI Express IP DMA Example Design in Platform Designer

Section Content
Block Descriptions
Programming Model for the Example Design
DMA Operations Using the Example Design

Level Two Title