Intel Virtualization Technology for Directed I O Architecture Specification Coll. pdf download
Intel Virtualization Technology for Directed I O Architecture Specification Coll. pdf download
https://textbookfull.com/product/intel-virtualization-technology-
for-directed-i-o-architecture-specification-coll/
https://textbookfull.com/product/android-on-x86-an-introduction-
to-optimizing-for-intel-architecture-1st-edition-iggy-krajci/
https://textbookfull.com/product/intel-xeon-phi-coprocessor-
architecture-and-tools-the-guide-for-application-developers-1st-
edition-rezaur-rahman-auth/
https://textbookfull.com/product/choice-architecture-in-
democracies-exploring-the-legitimacy-of-nudging-coll/
Intel Trusted Execution Technology for Server Platforms
A Guide to More Secure Datacenters 1st Edition William
Futral
https://textbookfull.com/product/intel-trusted-execution-
technology-for-server-platforms-a-guide-to-more-secure-
datacenters-1st-edition-william-futral/
https://textbookfull.com/product/strategies-for-palladium-
catalyzed-non-directed-and-directed-c-h-bond-functionalization-
anant-r-kapdi-and-debabrata-maiti-eds/
https://textbookfull.com/product/the-architecture-reference-
specification-book-updated-revised-everything-architects-need-to-
know-every-day-2nd-edition-julia-mcmorrough/
https://textbookfull.com/product/intel-galileo-and-intel-galileo-
gen-2-api-features-and-arduino-projects-for-linux-programmers-
manoel-carlos-ramon/
https://textbookfull.com/product/emerging-technology-and-
architecture-for-big-data-analytics-1st-edition-anupam-
chattopadhyay/
Intel® Virtualization Technology for
Directed I/O
Architecture Specification
June 2016
Intel and Itanium are trademarks of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
Contents
1 Introduction
1.1 Audience ........................................................................................................ 1-1
1.2 Glossary ........................................................................................................ 1-2
1.3 References ..................................................................................................... 1-3
2 Overview
2.1 Intel® Virtualization Technology Overview .......................................................... 2-1
2.2 VMM and Virtual Machines ................................................................................ 2-1
2.3 Hardware Support for Processor Virtualization ..................................................... 2-1
2.4 I/O Virtualization............................................................................................. 2-2
2.5 Intel® Virtualization Technology For Directed I/O Overview ................................... 2-2
2.5.1 Hardware Support for DMA Remapping..................................................... 2-3
2.5.1.1 OS Usages of DMA Remapping................................................... 2-3
2.5.1.2 VMM Usages of DMA Remapping ................................................ 2-4
2.5.1.3 DMA Remapping Usages by Guests............................................. 2-4
2.5.1.4 Interaction with Processor Virtualization...................................... 2-5
2.5.2 Hardware Support for Interrupt Remapping .............................................. 2-6
2.5.2.1 Interrupt Isolation.................................................................... 2-6
2.5.2.2 Interrupt Migration................................................................... 2-6
2.5.2.3 x2APIC Support ....................................................................... 2-6
2.5.3 Hardware Support for Interrupt Posting .................................................... 2-7
2.5.3.1 Interrupt Vector Scalability........................................................ 2-7
2.5.3.2 Interrupt Virtualization Efficiency ............................................... 2-7
2.5.3.3 Virtual Interrupt Migration......................................................... 2-7
3 DMA Remapping
3.1 Types of DMA requests..................................................................................... 3-1
3.2 Domains and Address Translation ...................................................................... 3-1
3.3 Remapping Hardware - Software View................................................................ 3-2
3.4 Mapping Devices to Domains ............................................................................ 3-2
3.4.1 Source Identifier ................................................................................... 3-3
3.4.2 Root-Entry & Extended-Root-Entry .......................................................... 3-3
3.4.3 Context-Entry ....................................................................................... 3-4
3.4.4 Extended-Context-Entry ......................................................................... 3-5
3.5 Hierarchical Translation Structures..................................................................... 3-7
3.6 First-Level Translation...................................................................................... 3-9
3.6.1 Translation Faults ................................................................................ 3-11
3.6.2 Access Rights ..................................................................................... 3-11
3.6.3 Accessed, Extended Accessed, and Dirty Flags ........................................ 3-12
3.6.4 Snoop Behavior................................................................................... 3-13
3.6.5 Memory Typing ................................................................................... 3-13
3.6.5.1 Selecting Memory Type from Page Attribute Table ...................... 3-14
3.6.5.2 Selecting Memory Type from Memory Type Range Registers ........ 3-14
3.6.5.3 Selecting Effective Memory Type.............................................. 3-15
3.7 Second-Level Translation................................................................................ 3-16
3.7.1 Translation Faults ................................................................................ 3-19
3.7.2 Access Rights ..................................................................................... 3-19
3.7.3 Snoop Behavior................................................................................... 3-20
3.7.4 Memory Typing ................................................................................... 3-20
3.8 Nested Translation ........................................................................................ 3-21
3.8.1 Translation Faults ................................................................................ 3-22
3.8.2 Access Rights ..................................................................................... 3-22
3.8.3 Snoop Behavior................................................................................... 3-23
3.8.4 Memory Typing ................................................................................... 3-24
9.11 Interrupt Remapping Table Entry (IRTE) for Posted Interrupts ............................. 9-39
9.12 Posted Interrupt Descriptor (PID) .................................................................... 9-42
10 Register Descriptions
10.1 Register Location........................................................................................... 10-1
10.2 Software Access to Registers .......................................................................... 10-1
10.3 Register Attributes ........................................................................................ 10-2
10.4 Register Descriptions ..................................................................................... 10-3
10.4.1 Version Register.................................................................................. 10-7
10.4.2 Capability Register .............................................................................. 10-8
10.4.3 Extended Capability Register............................................................... 10-13
10.4.4 Global Command Register .................................................................. 10-17
10.4.5 Global Status Register........................................................................ 10-22
10.4.6 Root Table Address Register ............................................................... 10-24
10.4.7 Context Command Register ................................................................ 10-25
10.4.8 IOTLB Registers ................................................................................ 10-28
10.4.8.1 IOTLB Invalidate Register ..................................................... 10-29
10.4.8.2 Invalidate Address Register ................................................... 10-32
10.4.9 Fault Status Register ......................................................................... 10-34
10.4.10Fault Event Control Register ............................................................... 10-36
10.4.11Fault Event Data Register ................................................................... 10-38
10.4.12Fault Event Address Register .............................................................. 10-39
10.4.13Fault Event Upper Address Register ..................................................... 10-40
10.4.14Fault Recording Registers [n] .............................................................. 10-41
10.4.15Advanced Fault Log Register ............................................................... 10-44
10.4.16Protected Memory Enable Register....................................................... 10-45
10.4.17Protected Low-Memory Base Register................................................... 10-47
10.4.18Protected Low-Memory Limit Register .................................................. 10-48
10.4.19Protected High-Memory Base Register .................................................. 10-49
10.4.20Protected High-Memory Limit Register.................................................. 10-50
10.4.21Invalidation Queue Head Register ........................................................ 10-51
10.4.22Invalidation Queue Tail Register .......................................................... 10-52
10.4.23Invalidation Queue Address Register .................................................... 10-53
10.4.24Invalidation Completion Status Register ............................................... 10-54
10.4.25Invalidation Event Control Register ...................................................... 10-55
10.4.26Invalidation Event Data Register ......................................................... 10-56
10.4.27Invalidation Event Address Register ..................................................... 10-57
10.4.28Invalidation Event Upper Address Register ............................................ 10-58
10.4.29Interrupt Remapping Table Address Register......................................... 10-59
10.4.30Page Request Queue Head Register ..................................................... 10-60
10.4.31Page Request Queue Tail Register........................................................ 10-61
10.4.32Page Request Queue Address Register ................................................. 10-62
10.4.33Page Request Status Register ............................................................. 10-63
10.4.34Page Request Event Control Register ................................................... 10-64
10.4.35Page Request Event Data Register ....................................................... 10-65
10.4.36Page Request Event Address Register................................................... 10-66
10.4.37Page Request Event Upper Address Register ......................................... 10-67
10.4.38MTRR Capability Register.................................................................... 10-68
10.4.39MTRR Default Type Register................................................................ 10-69
10.4.40Fixed-Range MTRRs ........................................................................... 10-70
10.4.41Variable-Range MTRRs ....................................................................... 10-72
A Non-Recoverable Fault Reason Encodings ................................................................. 1
Figures
Figure 1-1. General Platform Topology ......................................................................... 1-1
Figure 2-2. Example OS Usage of DMA Remapping ........................................................ 2-3
Figure 2-3. Example Virtualization Usage of DMA Remapping .......................................... 2-4
Figure 2-4. Interaction Between I/O and Processor Virtualization ..................................... 2-5
Figure 3-5. DMA Address Translation ........................................................................... 3-2
Figure 3-6. Requester Identifier Format........................................................................ 3-3
Figure 3-7. Device to Domain Mapping Structures using Root-Table ................................. 3-4
Figure 3-8. Device to Domain Mapping Structures using Extended-Root-Table ................... 3-6
Figure 3-9. Address Translation to a 4-KByte Page......................................................... 3-7
Figure 3-10. Address Translation to a 2-MByte Large Page................................................ 3-8
Figure 3-11. Address Translation to a 1-GByte Large Page ................................................ 3-8
Figure 3-12. Nested Translation with 4-KByte pages .......................................................3-21
Figure 4-13. Device-TLB Operation ................................................................................ 4-1
Figure 5-14. Compatibility Format Interrupt Request........................................................ 5-2
Figure 5-15. Remappable Format Interrupt Request......................................................... 5-3
Figure 5-16. I/OxAPIC RTE Programming ....................................................................... 5-7
Figure 5-17. MSI-X Programming .................................................................................. 5-8
Figure 5-18. Remapping Hardware Interrupt Programming in Intel® 64 xAPIC Mode............ 5-9
Figure 5-19. Remapping Hardware Interrupt Programming in Intel® 64 x2APIC Mode .........5-10
Figure 6-20. Context-cache Invalidate Descriptor ...........................................................6-19
Figure 6-21. PASID-cache Invalidate Descriptor .............................................................6-20
Figure 6-22. IOTLB Invalidate Descriptor.......................................................................6-21
Figure 6-23. Extended IOTLB Invalidate Descriptor .........................................................6-22
Figure 6-24. Device-TLB Invalidate Descriptor................................................................6-24
Figure 6-25. Extended Device-TLB Invalidate Descriptor ..................................................6-25
Figure 6-26. Interrupt Entry Cache Invalidate Descriptor .................................................6-26
Figure 6-27. Invalidation Wait Descriptor ......................................................................6-27
Figure 7-28. Page Request Descriptor ...........................................................................7-15
Figure 7-29. Page Group Response Descriptor................................................................7-19
Figure 7-30. Page Stream Response Descriptor ..............................................................7-20
Figure 8-31. Hypothetical Platform Configuration............................................................. 8-7
Figure 9-32. Root-Entry Format .................................................................................... 9-1
Figure 9-33. Extended-Root-Entry Format ...................................................................... 9-3
Figure 9-34. Context-Entry Format ................................................................................ 9-5
Figure 9-35. Extended-Context-Entry Format.................................................................. 9-8
Figure 9-36. PASID Entry Format .................................................................................9-15
Figure 9-37. PASID-State Entry Format.........................................................................9-17
Figure 9-38. Format for First-Level Paging Entries ..........................................................9-18
Figure 9-39. Format for Second-Level Paging Entries ......................................................9-25
Figure 9-40. Fault-Record Format.................................................................................9-32
Figure 9-41. Interrupt Remap Table Entry Format for Remapped Interrupts .......................9-34
Figure 9-42. Interrupt Remap Table Entry Format for Posted Interrupts.............................9-39
Figure 9-43. Posted Interrupt Descriptor Format ............................................................9-42
Figure 10-44. Version Register ......................................................................................10-7
Figure 10-45. Capability Register ...................................................................................10-8
Figure 10-46. Extended Capability Register ................................................................... 10-13
Figure 10-47. Global Command Register ....................................................................... 10-17
Figure 10-48. Global Status Register ............................................................................ 10-22
Figure 10-49. Root Table Address Register .................................................................... 10-24
Figure 10-50. Context Command Register ..................................................................... 10-25
Figure 10-51. IOTLB Invalidate Register........................................................................ 10-29
Figure 10-52. Invalidate Address Register ..................................................................... 10-32
Figure 10-53. Fault Status Register .............................................................................. 10-34
Tables
Table 1. Glossary .................................................................................................. 1-2
Table 2. References ............................................................................................... 1-3
Table 3. First-level Paging Structures ....................................................................... 3-9
Table 4. Effective Memory Types ............................................................................3-15
Table 5. Second-level Paging Structures ..................................................................3-17
Table 6. Address Fields in Remappable Interrupt Request Format ................................ 5-3
Table 7. Data Fields in Remappable Interrupt Request Format ..................................... 5-4
Table 8. Interrupt Remapping Fault Conditions .......................................................... 5-6
Table 9. Index Mask Programming..........................................................................6-26
Table 10. Interrupt Remapping Fault Conditions .......................................................... 7-1
Table 11. Non-Recoverable Faults for Untranslated Requests Without PASID ................... 7-2
Table 12. Non-Recoverable Faults for Untranslated Requests With PASID ....................... 7-3
Table 13. Non-Recoverable Faults For Translation Requests Without PASID..................... 7-6
Table 14. Non-Recoverable Faults For Translation Requests With PASID ......................... 7-7
Table 15. Non-Recoverable Faults For Translated Requests ........................................... 7-9
Table 16. Recoverable Fault Conditions For Translation Requests ..................................7-10
Table 17. Response Codes.......................................................................................7-20
Table 18. Format of PML4E that references a Page-Directory-Pointer Table ....................9-19
Table 19. Format of PDPE that maps a 1-GByte Page .................................................9-20
Table 20. Format of PDPE that references a Page-Directory Table .................................9-21
Table 21. Format of PDE that maps a 2-MByte Page ..................................................9-22
Table 22. Format of PDE that references a Page Table.................................................9-23
Table 23. Format of PTE that maps a 4-KByte Page ...................................................9-24
Table 24. Format of SL-PML4E referencing a Second-Level-Page-Directory-Pointer Table .9-26
Table 25. Format of SL-PDPE that maps a 1-GByte Page .............................................9-27
Table 26. Format of SL-PDPE that references a Second-Level-Page-Directory .................9-28
Table 27. Format of SL-PDE that maps to a 2-MByte Page ...........................................9-29
Table 28. Format of SL-PDE that references a Second-Level-Page Table ........................9-30
Table 29. Format of SL-PTE that maps 4-KByte Page ..................................................9-31
Table 30. Address Mapping for Fixed-Range MTRRs .................................................. 10-71
Revision History
e ft
IsL
ge
la nk
s Pa y B
i ll
Th o n a
n ti
te
In
1 Introduction
This document describes the Intel® Virtualization Technology for Directed I/O (“Intel® VT for Directed
I/O”); specifically, it describes the components supporting I/O virtualization as it applies to platforms
that use Intel® processors and core logic chipsets complying with Intel® platform specifications.
P ro c e s s o r P ro c e s s o r
S y s te m B u s
N o rth B rid g e
D M A & In te rru p t R e m a p p in g
DRAM
In te g ra te d
D e v ic e s
P C I E x p re s s S o u th P C I, L P C ,
D e v ic e s B rid g e L e g a c y d e v ic e s
The following topics are not covered (or are covered in a limited context):
• Intel® Virtualization Technology for Intel® 64 Architecture. For more information, refer to the
“Intel® 64 Architecture Software Developer's Manual, Volume 3B: System Programming Guide”.
1.1 Audience
This document is aimed at hardware designers developing Intel platforms or core-logic providing
hardware support for virtualization. The document is also expected to be used by Operating System
(OS) and Virtual Machine Monitor (VMM) developers utilizing the I/O virtualization hardware
functions.
1.2 Glossary
The document uses the terms listed in the following table.
Table 1. Glossary
Term Definition
A hardware representation of state that identifies a device and the domain to which the
Context
device is assigned.
Context-
Remapping hardware cache that stores device to domain mappings
cache
Device-TLB A translation cache at the endpoint device (as opposed to in the platform).
DMA Direct Memory Access: Address routed in-bound requests from I/O devices
DMA
The act of translating the address in a DMA request to a host physical address (HPA).
Remapping
A collection of physical, logical, or virtual resources that are allocated to work together.
Domain
Used as a generic term for virtual machines, partitions, etc.
Address in a DMA request: Depending on the software usage and hardware capabilities,
DMA
DMA address can be Guest Physical Address (GPA), Guest Virtual Address (GVA), Virtual
Address
Address (VA), or I/O Virtual Address (IOVA).
First-Level Paging structures used for address translation of DMA requests with Process Address
Paging Space ID (PASID)
GAW Guest Address Width: Physical addressability limit within a partition (virtual machine)
Guest Physical Address: the view of physical memory from software running in a
GPA
partition (virtual machine).
Guest Virtual Address: Processor virtual address used by software running in a partition
GVA
(virtual machine).
HAW Host Address Width: the DMA physical addressability limit for a platform.
Host Physical Address: Physical address used by hardware to access memory and
HPA
memory-mapped resources.
Interrupt Entry Cache: A translation cache in remapping hardware unit that caches
IEC
frequently used interrupt-remapping table entries.
IOVA I/O Virtual Address: Virtual address created by software for use in I/O requests.
Interrupt
The act of translating an interrupt request before it is delivered to the CPU complex.
Remapping
Maximum Guest Address Width: the maximum DMA virtual addressability supported by a
MGAW
remapping hardware implementation.
Table 1. Glossary
Term Definition
Second- Translation caches used by remapping hardware units to cache intermediate (non-leaf)
Level entries of the second-level (SL) paging structures. For hardware supporting 48-bit Guest
Caches Address Width, these include SL-PML4 cache, SL-PDP cache, and SL-PDE cache.
Process Address Space Identifier: DMA requests with virtual address (or guest virtual
PASID address) are tagged with a PASID value that identifies the targeted virtual address
space.
PASID- Remapping hardware cache that caches frequently accessed PASID-table entries used to
cache translate DMA requests with PASID.
Second-
Paging Structures used for address translation of DMA requests without Process Address
Level
Space ID (PASID).
Paging
A 16-bit identification number to identify the source of a DMA or interrupt request. For
Source ID PCI family devices this is the ‘Requester ID’ which consists of PCI Bus number, Device
number, and Function number.
Root- Refers to one or more hardware components that connect processor complexes to the
Complex I/O and memory subsystems. The chipset may include a variety of integrated devices.
Virtual Machine Monitor: a software layer that controls virtualization. Also referred to as
VMM
hypervisor in this document.
1.3 References
Table 2. References
Description
®
Intel 64 Architecture Software Developer's Manuals
http://developer.intel.com/products/processor/manuals/index.htm
PCI-Express Single-Root I/O Virtualization and Sharing (SR-IOV) Specification, Revision 1.0
http://www.pcisig.com/specifications/iov
ACPI Specification
http://www.acpi.info/
2 Overview
This chapter provides a brief overview of Intel® VT, the virtualization software ecosystem it enables,
and hardware support offered for processor and I/O virtualization.
The VMM is a key component of the platform infrastructure in virtualization usages. Intel® VT can
improve the reliability and supportability of virtualization infrastructure software with programming
interfaces to virtualize processor hardware. It also provides a foundation for additional virtualization
support for other hardware components in the platform.
Intel® VT provides hardware support for processor virtualization. For Intel® 64 processors, this
support consists of a set of virtual-machine extensions (VMX) that support virtualization of processor
hardware for multiple software environments by using virtual machines.
Depending on the usage requirements, a VMM may support any of the above models for I/O
virtualization. For example, I/O emulation may be best suited for virtualizing legacy devices. I/O
assignment may provide the best performance when hosting I/O-intensive workloads in a guest.
Using new software interfaces makes a trade-off between compatibility and performance, and device
I/O sharing provides more virtual devices than the number of physical devices in the platform.
DMA remapping provides hardware support for isolation of device accesses to memory, and enables
each device in the system to be assigned to a specific domain through a distinct set of paging
structures. When the device attempts to access system memory, the DMA-remapping hardware
intercepts the access and utilizes the page tables to determine whether the access can be permitted;
it also determines the actual location to access. Frequently used paging structures can be cached in
hardware. DMA remapping can be configured independently for each device, or collectively across
multiple devices.
Domain 1 Domain 2
DMA-Remapping Hardware
Virtual Machine
VM (0) Virtual Machine
VM (n) Virtual Machine
VM (0) Virtual Machine
VM (n)
App App
App App App
App App App
App App App
App
DMA-Remapping Hardware
In this model, the VMM restricts itself to enabling direct assignment of devices to their partitions.
Rather than invoking the VMM for all I/O requests from a partition, the VMM is invoked only when
guest software accesses protected resources (such as configuration accesses, interrupt management,
etc.) that impact system functionality and isolation.
To support direct assignment of I/O devices, a VMM must enforce isolation of DMA requests. I/O
devices can be assigned to domains, and the remapping hardware can be used to restrict DMA from
an I/O device to the physical memory presently owned by its domain. For domains that may be
relocated in physical memory, the remapping hardware can be programmed to perform the necessary
translation.
I/O device assignment allows other I/O sharing usages — for example, assigning an I/O device to an
I/O partition that provides I/O services to other user partitions. Remapping hardware enables
virtualization software to choose the right combination of device assignment and software-based
methods for I/O virtualization.
remapping hardware. Due to the non-restartability of faulting DMA transactions (unlike CPU memory
management virtualization), a VMM cannot perform lazy updates to its shadow remapping structures.
To keep the shadow structures consistent with the guest structures, the VMM may expose virtual
remapping hardware with eager pre-fetching behavior (including caching of not-present entries) or
use processor memory management mechanisms to write-protect the guest remapping structures.
Virtual Machines
Physical Memory
I/O Logical
Devices Processors
DMA CPU Memory
Remapping Virtualization
The VMM manages processor requests to access physical memory via the processor’s memory
management hardware. DMA requests to access physical memory use remapping hardware. Both
processor memory management and DMA memory management are under the control of the VMM.
The interrupt-remapping hardware may be utilized by a Virtual Machine Monitor (VMM) to improve the
isolation of external interrupt requests across domains. For example, the VMM may utilize the
interrupt-remapping hardware to distinguish interrupt requests from specific devices and route them
to the appropriate VMs to which the respective devices are assigned. The VMM may also utilize the
interrupt-remapping hardware to control the attributes of these interrupt requests (such as
destination CPU, interrupt vector, delivery mode etc.).
Another example usage is for the VMM to use the interrupt-remapping hardware to disambiguate
external interrupts from the VMM owned inter-processor interrupts (IPIs). Software may enforce this
by ensuring none of the remapped external interrupts have attributes (such as vector number) that
matches the attributes of the VMM IPIs.
Interrupt remapping enables x2APICs to support the expanded APIC addressability for external
interrupts without requiring hardware changes to interrupt sources (such as I/OxAPICs and MSI/MSI-
X devices).
Each VF requires its own independent interrupt resources, resulting in more interrupt vectors needed
than otherwise required without such I/O virtualization. Without interrupt-posting hardware support,
all interrupt sources in the platform are mapped to the same physical interrupt vector space (8-bit
vector space per logical CPU on Intel®64 processors). For virtualization usages, partitioning the
physical vector space across virtual processors is challenging in a dynamic environment when there is
no static affinity between virtual process and logical processors.
Hardware support for interrupt posting addresses this vector scalability problem by allowing interrupt
requests from device functions assigned to virtual machines to operate in virtual vector space,
thereby scaling naturally with the number of virtual machines or virtual processors.
With hardware support for interrupt posting, interrupts from devices assigned to virtual machines are
posted (recorded) in memory descriptors specified by the VMM, and processed based on the running
state of the virtual processor targeted by the interrupt.
For example, if the target virtual processor is running on any logical processor, hardware can directly
deliver external interrupts to the virtual processor without any VMM intervention. Interrupts received
while the target virtual processor is pre-empted (waiting for its turn to run) can be accumulated in
memory by hardware for delivery when the virtual processor is later scheduled. This avoids disrupting
execution of currently running virtual processors on external interrupts for non-running virtual
machines. If the target virtual processor is halted (idle) at the time of interrupt arrival or if the
interrupt is qualified as requiring real-time processing, hardware can transfer control to VMM,
enabling VMM to schedule the virtual processor and have hardware directly deliver pending interrupts
to that virtual processor.
This target virtual processor state based processing of interrupts reduces overall interrupt latency to
virtual machines and reduces overheads otherwise incurred by the VMM for virtualizing interrupts.
For virtual machines with assigned devices, migrating a virtual processor across logical processors
either incurs the overhead of forwarding interrupts in software (e.g. via VMM generated IPIs), or
complexity to independently migrate each interrupt targeting the virtual processor to the new logical
processor. Hardware support for interrupt posting enables VMM software to atomically co-migrate all
interrupts targeting a virtual processor when the virtual processor is scheduled to another logical
processor.
3 DMA Remapping
This chapter describes the hardware architecture for DMA remapping. The architecture envisions
remapping hardware to be implemented in Root-Complex components, such as the memory controller
hub (MCH) or I/O hub (IOH).
The isolation property of a domain is achieved by blocking access to its physical memory from
resources not assigned to it. Multiple isolated domains are supported in a system by ensuring that all
I/O devices are assigned to some domain (possibly a null domain), and that they can only access the
physical resources allocated to their domain. The DMA remapping architecture facilitates flexible
assignment of I/O devices to an arbitrary number of domains. Each domain has a view of physical
address space that may be different than the host physical address space. Remapping hardware
treats the address in inbound requests as DMA Address. Depending on the software usage model, the
DMA address space may be the Guest-Physical Address (GPA) space of the virtual machine to which
the device is assigned, or application Virtual Address (VA) space defined by the PASID assigned to an
application, or some abstract I/O virtual address (IOVA) space defined by software. In all cases, DMA
remapping transforms the address in a DMA request issued by an I/O device to its corresponding
Host-Physical Address (HPA).
For simplicity, this document refers to address in requests-without-PASID as GPA, and address in
requests-with-PASID as Virtual Address (VA) (or Guest Virtual Address (GVA), if such request is from
a device assigned to a virtual machine). The translated address is referred to as HPA.
Figure 3-5 illustrates DMA address translation. I/O devices 1 and 2 are assigned to domains 1 and 2,
respectively. The software responsible for creating and managing the domains allocates system
physical memory for both domains and sets up the DMA address translation function. DMA address in
requests initiated by devices 1 & 2 are translated to appropriate HPAs by the remapping hardware.
10000h
CPU Assigned to
DM A Domain 1
Mem ory
Mem ory
Management
Managem ent
(G)VA/GPA
Dom ain 2 4000h HPA = HPA = = 4000h
Device 2
3000h 3000h
Assigned to
0h Domain 2
Physical
Memory
The host platform may support one or more remapping hardware units. Each hardware unit supports
remapping DMA requests originating within its hardware scope. For example, a desktop platform may
expose a single remapping hardware unit that translates all DMA transactions at the memory
controller hub (MCH) component. A server platform with one or more core components may support
independent translation hardware units in each component, each translating DMA requests originating
within its I/O hierarchy (such as a PCI-Express root port). The architecture supports configurations in
which these hardware units may either share the same translation data structures (in system
memory) or use independent structures, depending on software programming.
The remapping hardware translates the address in a request to host physical address (HPA) before
further hardware processing (such as address decoding, snooping of processor caches, and/or
forwarding to the memory controllers).
For hardware implementations supporting multiple PCI segment groups, the remapping architecture
requires hardware to expose independent remapping hardware units (at least one per PCI segment
group) for processing requests originating within the I/O hierarchy of each segment group.
For PCI-Express devices, the source-id is the requester identifier in the PCI-Express transaction layer
header. The requester identifier of a device, which is composed of its PCI Bus/Device/Function
number, is assigned by configuration software and uniquely identifies the hardware function that
initiated the request. Figure 3-6 illustrates the requester-id1 as defined by the PCI-Express
Specification.
1
5 87 3 2 0
The following sections describe the data structures for mapping I/O devices to domains.
1. For PCI-Express devices supporting Alternative Routing-ID Interpretation (ARI), bits traditionally
used for the Device Number field in the Requester-id are used instead to expand the Function
Number field.
• Lower Present flag: The lower-present field indicates the lower 64-bits of the extended-root-
entry is present and the lower-context-table pointer (LCTP) field is initialized. Software may Clear
the lower-present field for extended-root-entries corresponding to bus numbers that are either
not present in the platform, or don’t have downstream devices with device numbers 0-15
attached. DMA requests processed through the lower part of an extended-root-entry with the
lower-present field Clear result in translation-fault.
• Lower Context-table pointer: The lower-context-table pointer references the lower-context-
table for devices with device number 0-15, on the bus identified by the referencing extended-
root-entry. Section 3.4.4 describes extended-context-entries in the lower-context-table.
• Upper Present flag: The upper-present field indicates the upper 64-bits of the extended-root-
entry is present and the upper-context-table pointer (UCTP) field is initialized. Software may Clear
the upper-present field for extended-root-entries corresponding to bus numbers that are either
not present in the platform, or don’t have downstream devices with device numbers 16-31
attached. DMA requests processed through the upper part of an extended-root-entry with the
upper-present field Clear result in translation-fault.
• Upper Context-table pointer: The upper-context-table pointer references the upper-context-
table for devices with device number 16-31, on the bus identified by the referencing extended-
root-entry. Section 3.4.4 describes extended-context-entries in the upper-context-table.
3.4.3 Context-Entry
A context-entry maps a specific I/O device on a bus to the domain to which it is assigned, and, in
turn, to the address translation structures for the domain. The context entries are programmed
through memory-resident context-tables. Each root-entry in the root-table contains the pointer to the
context-table for the corresponding bus number. Each context-table contains 256 entries, with each
entry corresponding to a PCI device function on the bus. For a PCI device, the device and function
numbers (lower 8-bits) of source-id are used to index into the context-table. Figure 3-7 illustrates
device to domain mapping through root-table.
(Dev 0, Func 1)
(Bus 0) Root-entry 0
Root-table
Context-entry 255
Address Translation
Structures for Dom ain B
Context-entry 0
Context-table for
Bus 0
Multiple devices may be assigned to the same domain by programming the context-entries for the
devices to reference the same translation structures, and programming them with the same domain
identifier. Section 9.3 provides the exact context-entry format.
3.4.4 Extended-Context-Entry
For implementations supporting Extended-Context-Support (ECS=1 in Extended Capability Register),
when using extended-root-table, each extended-root-entry references a lower-context-table and a
upper-context-table. The Lower-context-table is 4-KByte in size and contains 128 extended-context-
entries corresponding to PCI functions in device range 0-15 on the bus. The Upper-context-table is
also 4-KByte in size and contains 128 extended-context-entries corresponding to PCI functions in
device range 16-31 on the bus. Figure 3-8 illustrates device to domain mapping through extended-
root-table.
E xt-context-entry 127
P A SID -entry P 2
F irst-level P aging
(Bus 0) E xt-root-entry 0 PA S ID -entry P 1 S tructures for
Process P 2
E xten d ed -ro o t-tab le
P AS ID -entry 0
P A S ID -T ab le fo r
D o m ain B
E xt-context-entry 0
U p p er-co n text-tab le
fo r B u s 0
F irst-level P aging
E xt-context-entry 127 S tructures for
Process P 1
Ext-context-entry 0
L o w er-co n text-tab le
fo r B u s 0
Bernard, Sir Thomas, 579 and notes, 580, 582, 585, 595 n., 599.
Bigotry, 198.
Bingen, 751.
Biographia Literaria, 3, 68 n., 74 n., 152 n., 164 n., 174 n., 232 n.,
257, 320 n., 498 n., 607 n., 669 n., 670 n.;
C. ill-used by the printer of, 673, 674;
679, 756 n.
Blake, William, as poet, painter, and engraver, 685 n., 686 n.;
C.’s criticism of his poems and their accompanying illustrations,
686-688;
his Songs of Innocence and Experience, 686 n.
Bloomfield, Robert, 395.
Borrowdale, 431.
Bowles, Rev. William Lisle, C.’s admiration for his poems, 37, 42,
179;
63 n., 76 and note;
C.’s sonnet to, 111 and note;
115;
his sonnets, 177;
his Hope, an Allegorical Sketch, 179, 180;
196, 197, 211;
his translation of Dean Ogle’s Latin Iambics, 374 and note;
school life at Winchester, 374 n.;
C.’s, Southey’s, and Sotheby’s admiration of, and its effect on their
poems, 396;
borrows a line from a poem of C.’s, 396;
his second volume of poems, 403, 404;
637, 638, 650-652.
Box, 631.
Bremhill, 650.
Bridgewater, 164.
Brunton, Elizabeth, 86 n.
Brunton, Louisa, 86 n.
Burnett, George, 74, 121, 140-142, 144-151, 174 n., 325, 467.
Burns, Robert, 196;
C.’s poem on, 206 and note, 207.
Burton, 326.
Buttermere, 393.
Caermarthen, 411.
Carlyon, Clement, M. D., his Early Years and Late Recollections, 258,
298 n.
Cary, Rev. H. F., his translation of the Divina Commedia, 676, 677
and note, 678, 679;
C. introduces himself to, 676 n.;
685, 699;
letters from C., 676, 677, 731, 760.
Castle Spectre, The, a play by Monk Lewis, C.’s criticism of, 236 and
note, 237, 238;
626.
Catania, 458.
Cathloma, 51.
Catholic question, the, letters in the Courier on, 567 and note;
C. proposes to again write for the Courier on, 660, 662;
arrangements for the proposed articles on, 664, 665.
Chantrey, Mr. (afterwards Sir) Francis, R. A., C.’s impressions of, 699;
727.
Character, A, 631 n.
Charity, 110 n.
Christianity, the one true Philosophy (C.’s magnum opus), outline of,
632, 633;
fragmentary remains of, 632 n.;
the sole motive for C.’s wish to live, 668;
J. H. Green helps to lay the foundations of, 679 n.;
694, 753;
plans for, 772, 773.
Chronicle, Morning, 111 n., 114, 116 n., 119 n., 126, 162, 167, 505,
506, 606 n., 615, 616.
Coleridge, Berkeley (son), birth of, 247 and note, 248, 249;
taken with smallpox, 259 n., 260 n.;
262, 267, 272;
death of, 247 n., 282-287, 289.
Coleridge, Colonel James (brother), 7, 54, 56, 61, 306, 724 n., 726
n.;
letter from S. T. C., 61.
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
textbookfull.com