by Miguel Rodriguez, PLX Technology
The concept of direct-memory access (DMA) isn’t new; for years now, processors and chipset makers have been implementing DMA as part of their efforts to increase system performance. Similarly, endpoints such as Ethernet and Fibre Channel have been implementing DMA to minimize protocol latencies and maximize data throughput. However, DMA integrated into a PCI Express (PCIe) switch is revolutionary – with significant advantages for control planes, embedded systems, and storage applications.
A communications control plane is not bandwidth intensive; instead, it is latency sensitive. Performance is not measured in Megabytes per second, as is the case in the data plane, but rather in how long a transaction takes to complete. At a high level, the communications control plane is made up of a control card connected to a group of line cards. Depending on the size of the system, the number of line cards can be as many as several dozen. In a PCIe-based control plane, these line cards are connected to the control card through a PCIe switch. Most operations performed through the control plane fall within one of three categories: configuration of the endpoints, status of the devices, and statistics gathering. These operations to the line card are performed by the control processor in a serial manner. That is, the processor writes to and/or reads from the line cards one at a time; often these transactions are to a large number of registers on memory space per line card. This large processor overhead is only magnified in systems with a large number of line cards. Low-latency PCIe switches used in these systems play their part by shortening the round-trip latency of the transactions. However, for systems with a large number of line cards and/or high latency devices on the line cards, alternative methods are employed to reduce the latency.
