Robert is an independent consultant on the x86 architecture. He can be reached at [email protected].
In the first two installments of this series on Intel's Virtual Mode Extensions (VME), I examined why VME is needed. More specifically, in the January column I explained the performance problems associated with Intel's original v86 mode implementation. I went on to discuss Intel's new Enhanced v86 Mode implementation, in particular, how Ev86 mode fixed many of the performance deficiencies in the first v86-mode implementation.
In the March installment, I went under the hood of VME, examining all of the various components which are used by the VME, and explained how they work. At the end of that column, I presented source code that can be used to model your own Ev86 mode tasks.
Subsequent to those columns being published, Intel released the Intel Architecture Software Developer's Manual, Volume 3. Although six months late, the manual couldn't have been released at a better time because it gave me the opportunity to determine whether Intel had finally documented the missing details of VME or described its many caveats.
Alas, Intel has done neither. The missing details are still missing, and the caveats of VME have never been described. Notwithstanding these omissions, Intel has actually removed some vital VME information.
This month, among other things, I'll show how many references to VMEs remained in the Pentium manuals after Intel's lawyers had forced their removal. These oversights allowed the Appendix H liberation movement to reverse engineer the VME details and release the information six months before Intel.
The Appendix H Liberation Movement
After the creation of the notorious Appendix H, you would have thought Intel had removed every reference to the "secret" VMEs (see my November 1997 column for a description of Appendix H). However, Intel wasn't careful enough. In fact, I found many references to VME scattered throughout the Pentium manuals.
When preparing to help reverse engineer the VME details, I started with the premise that the Pentium manual was virtually silent on the subject of VME. In my research, I found many more references than expected. I extracted all the references and began categorizing them. As I rearranged the quotes into categories, I found the Pentium manual told its own VME story. This story starts by listing some of the problems with the existing v86 mode. Next, the story establishes that new VMEs were implemented to enhance performance. By the end of the story, the manual tells you virtually every aspect of VME. From this information, any capably equipped engineer can write code to figure out the various implementation details.
Here's the Pentium manual's own story of why VME was needed, and how to use it. Unless otherwise noted, all of the following quotes are taken from the Pentium Processor Family Developer's Manual, Volume 3: Architecture and Programming Manual (Intel part number 241430).
- "Many 8086 programs written for nonmultitasking systems set and clear the IF flag to control interrupts. This may cause problems in a multitasking environment. As a result, virtual monitors running on the Intel386 and Intel486 processors require maintaining a virtual interrupt flag in software. All instructions affecting the IF flag trap to the virtual-8086 monitor for emulation on these processors." (Section 22.5.) Trapping to the v86 monitor causes considerable microprocessor time to be spent in the overhead of faulting and returning control to the program under emulation.
- "The Pentium Processor includes extensions to its virtual-8086 mode of operation that improve the performance of applications by eliminating the overhead of faulting to a virtual-8086 monitor for emulation of certain operations." (Chapter 22.)
- "By enabling the virtual mode extensions, the virtual-8086 mode performance of the Pentium processor is significantly improved." (Section 22.10.)
- "Existing interrupt flag sensitive instructions provide significant performance improvement when using the virtual mode extensions of the Pentium processor." (Section 22.1.1.)
- The interrupt flag sensitive instructions which "eliminate the overhead of faulting to a virtual-8086 monitor for emulation" are CLI, STI, PUSHF, POPF, INT n, IRET. (Section 10.1.3.)
- "When IOPL=3, interrupts are serviced by the protected-mode interrupt service routine in a manner compatible with the Intel486 processor. On the Intel386 and Intel486 processors, all INT n instructions running in virtual-8086 mode require interception by the virtual-8086 monitor when IOPL is less than 3. For information on Pentium processor virtual mode extension support of interrupt handling, see Appendix H." (Section 22.6.) This statement is very significant because it essentially tells the reader that INT-n instructions running in Pentium Ev86 mode aren't required to be intercepted by the virtual-8086 monitor.
- Setting CR4.VME=1 "enables support for a virtual interrupt flag in virtual-8086 mode." (Section 10.1.3, Section 22.10.)
- "The Pentium processor TSS contains additional information used in virtual-8086 mode by the virtual mode extensions to the Pentium processor." (Section 23.2.17.1.)
- "The fields of a TSS are divided into two main categories:...(We are only concerned with the second category.) 2. Static fields the processor reads, but does not change. These fields are set up when a task is created. These fields store:...The base address for the I/O permission bit map and interrupt bit map. The base address points to the beginning of the I/O map and the end of the 32-byte interrupt map...See Chapter 22 for more information about the interrupt redirection." (Section 13.1.) This statement clearly tells us that there is a 32-byte interrupt redirection bit map stored in the TSS, whose tail is pointed to by the I/O permission bit map base address.
- The EFLAGS register contains two flag bits (VIF, VIP) used by VME. (Section 3.3.4.1, Section 10.1.1, Section 23.2.9, Section 23.2.9.1, Instruction description for IRET and POPF.)
- VIF and VIP will always be clear on processors that don't support VME. (Intel source code for CPUID detection.)
- "The VIP flag together with the VIF enable each applications program in a multitasking environment to have virtualized versions of the system's IF flag." (Section 10.1.1.)
- "The VIF is a virtual image of IF (the interrupt flag) used with VIP." (Section 10.1.1.)
- CLI, STI, PUSHF, POPF, INT-n, and IRET are all affected by enabling VME. (Instruction description for CLI, STI, PUSHF, POPF, INT-n and IRET.)
- IRETD or POPFD never modifies VIF and VIP. (Instruction description for IRET and POPF.)
- The VIF bit "operates between a STI state for enabling the execution of interrupt instructions initiated by [a] previously written program and a CLI state for disabling the execution of [the] interrupt instructions." (UK Patent Application GB 2 259 794 A, Claim 1.a.i.)
- The VIP bit "operates between a pending state during which interrupt requests are awaiting execution and a nonpending state in which no such interrupt requests are awaiting execution." (UK Patent Application GB 2 259 794 A, Claim 1.a.ii.)
- The Pentium contains the means to let the operating system "(change) the state of the VIF bit of the EFLAGS register without the use of emulation software, so long as the VIP bit is in its nonpending state." In other words, when using Ev86 mode, CLI and STI modify VIF, without blocking or enabling external interrupts, and without modifying the actual interrupt flag (IF). (UK Patent Application GB 2 259 794 A, Claim 1.b.)
- Once VIP is set into EFLAGS while VIF is in the noninterruptible state, the Pentium will respond to an STI instruction, by "automatically executing the STI instruction and awaiting an interrupt request by means of the emulation software without first changing the state of the VIF bit in EFLAGS from a Noninterruptible state to an Interruptible state." (UK Patent Application GB 2 259 794 A, Claim 1.c.)
As I've already mentioned, this information was publicly available at the time Intel was trying to keep it secret. Anybody with the necessary skills could use this information to reverse engineer the remaining details of Intel's VME implementation. In the long run, keeping this information secret hurt the engineering community more than it helped Intel maintain any competitive advantage.
VME Details Finally Released
When the Pentium Pro manuals were finally released in April 1996, Intel released its own description of VME. By this time, my article "Pentium's Virtual Mode Extensions Revealed" (coauthored with Jim Brooks, EE Times, November 11, 1995) describing VME had been publicly available for five months. Upon reading Intel's VME description, I noticed that some of it is fatally flawed. Consider the following excerpt from the Pentium Pro Family Developer's Manual, Volume 3, section 12.3.5:
If the processor receives an interrupt or exception and the VIF flag is clear (maskable hardware interrupts enabled), the processor performs the same operation as it does for method 5 interrupt and exception handling (that is, it redirects handling to the 8080 program's interrupt and exception handlers). The processor also handles interrupts and exceptions in this manner if the VIF flag is set, and the processor receives either an NMI interrupt or an exception (interrupt vectors 0 through 18).
If the processor receives a maskable hardware interrupt (interrupt vector 32 through 255) when the VIF flag is set, processor performs and the interrupt handler software must perform the following operations:
In some way or another, almost every sentence in this quoted text needs correction. The passage should read as follows:
If the processor receives an INT-n instruction, the processor performs the same operation as it does for method 5 interrupt and exception handling (that is, it redirects handling to the 8086 program's interrupt and exception handlers). The processor never handles maskable hardware interrupts, exceptions, or an NMI interrupts in this manner.
If the processor receives a maskable hardware interrupt, exception, or an NMI interrupt when the VIF flag is set, the processor causes a #GP(0). Otherwise, when the VIF flag is clear the processor and interrupt handler software must perform the following operations:
I was so confused by Intel's documentation that I'm still not sure what it was trying to say. Therefore I made my best guess, while making the technical corrections along the way. To determine if Intel has corrected these errors, I checked this same text in the newly released Intel Architecture Software Developer's Manual, Volume 3, discovering that Intel has completely removed this entire section of text, and the detailed explanation that followed.
The following text was removed from Intel's manuals. The text is the continuation of the passage just listed. The text describes the steps which the microprocessor and Ev86 monitor must perform when a maskable hardware interrupt, exception, or an NMI interrupt occurs when EFLAGS.VIF=0.
- The processor makes a call to a protected mode interrupt handler as described in the following steps. These steps are almost identical to those described for method 1 interrupt and exception handling in "Handling a Virtual-8086 Mode Interrupt or Exception Through a Protected-Mode Trap or Interrupt Gate" on page 12-17:
- Switches to 32-bit protected mode and privilege level 0.
- Saves the state of the processor on the privilege-level 0 stack. The states of the EIP, CS, EFLAGS, ESP, SS, ES, DS, FS, and GS registers are saved (see Figure 12-5 on page 12-18). In the EFLAGS image on the stack, the IOPL field is set to 3 and the VIF flag is copied to the IF flag.
- Clears the segment registers.
- Clears the VM flag in the EFLAGS register.
- Begins executing selected the protected-mode interrupt handler.
- The recommended action of the protected-mode interrupt handler is to read the VM flag from the EFLAGS image on the stack. If this flag is set, the handler makes a call to the virtual-8086 monitor.
- The virtual-8086 monitor reads the VIF flag in the EFLAGS register. If the flag is set, the virtual-8086 monitor sets the VIP flag in the EFLAGS register to indicate that there is an interrupt pending and returns to the protected mode handler.
- The protected mode handler executes a return to virtual-8086 mode.
- Upon returning to virtual-8086 mode, the processor continues execution of the 8086 program without handling the interrupt.
When the 8086 program executes the STI instruction to clear the VIF flag, the processor does the following:
- Checks the VIP flag.
- If the VIP flag is clear, the processor clears the VIF flag.
- If the VIP flag is set, the processor generates a general-protection exception (#GP).
- The recommended action of the protected-mode general-protection exception handler is to then call the virtual-8086 monitor and let it handle the pending interrupt.
A typical action of the virtual-8086 monitor is to clear the VIF and VIP flags in the EFLAGS image on the stack and execute a return to the virtual-8086 mode (through the protected-mode exception handler). The next time the processor receives a maskable hardware interrupt, (providing the VIF flag is still clear) it will handle it in the same manner as with method 5 interrupt and exception handling.
Note that the states of the VIF and VIP flags are not modified in real-address mode or during transitions between real-address and protected modes.
More Information: Missing in Action
Intel has never released the algorithms used by the various CPU instructions needed to support VME. Perusal of Intel's Pentium manuals shows that CLI, STI, PUSHF, POPF, INT-n, and IRET have all been modified to support VME. This support is necessary, as all of these instructions have the ability to modify the Interrupt Flag in the EFLAGS register (EFLAGS.IF). With VME enabled, the processor must now act on the Virtual Interrupt Flag (EFLAGS.VIF), instead of the real IF flag.
To the best of my ability, I've made Listing One representative of the algorithms Intel uses in its Pentium VME implementation. I've only included the algorithms for CLI, STI, PUSHF, POPF, and IRET. I never attempted to reverse engineer the INT-n instruction. The INT-n instruction should closely resemble the PUSHF instruction, with the additional support needed for the Interrupt Redirection Bitmap. The code is written in pseudo-C format, but is intended to be human readable.
VME Caveats (When CR4.VME=1)
VME is wonderful for reducing the complexity of the operating system, but has many caveats. Some of the quirks are natural extensions of VME or the underlying architecture, while others make little sense.
- While an Ev86 task is running, the IF-sensitive flags commit fakery to fool the Ev86 task into thinking that it is actually setting and clearing the IF flag. When a fault occurs to the monitor, the monitor sees the actual values of VIF, VIP, and IF unfiltered. This behavior gives the monitor direct control over virtual interrupts, and hardware interrupts.
- Software-generated exceptions are not influenced by the IR bitmap. Instead, these opcodes will always invoke the protected mode exception handler. Software-generated exceptions are as follows:
- INT1, also known as ICEBP. (An undocumented opcode, 0xF1, which generates an INT-1 exception.)
- INT3 (opcode 0xCC).
- INTO (opcode 0xCE).
- BOUND (opcode 0x62).
- All processor exceptions invoke the protected mode exception handler. The IR bitmap never influences processor exceptions.
- Maskable hardware interrupts and NMI interrupts invoke the protected mode exception handler when EFLAGS.VIF=0. When EFLAGS.VIF=1, maskable hardware interrupts and NMI interrupts cause a #GP. The IR bit map never influences these interrupts.
- VIF will never transition from 0 to 1 while a virtual interrupt is pending when IOPL<3. This condition will always cause a general-protection fault (#GP(0)) before VIF is set. This can best be described as follows: While the Ev86 task is in an uninterruptible state (VIF=0), a timer tick occurs, and causes a transition to the protected mode interrupt handler. The timer-tick routine sets VIP on the stack frame, and returns to the Ev86 task. The Ev86 task performs an STI instruction, which will put the program into an interruptible state. Before VIF is actually set, the #GP occurs. (This behavior can be observed by inspecting the EFLAGS image on the CPL-0 stack frame.) This behavioral characteristic is the basis for Intel's British Patent application, which claims "if the VIP bit is in its 'pending' state, then the STI instruction can be executed and an awaiting interrupt is serviced by using the emulation software without first changing the VIF bit from its CLI state to its STI state."
- VIP functions in IOPL=3 in a strange manner. When IOPL<3, and VIP=1, any transition from VIF=0 to VIF=1 will cause a general-protection fault before VIF is actually set. When IOPL=3, this behavior is different. VIF must actually be set before the #GP occurs.
- PUSHFD, POPFD, and IRETD will all generate a general-protection fault in an Ev86 task when IOPL<3.
- The behavior of POPF and IRET is inconsistent in how it handles the trap flag (TF) from the FLAGS image on the stack. When POPF is invoked with TF=1 on the stack image, a general-protection fault occurs. When IRET is invoked under the same condition, the general-protection fault does not occur. It has been recently reported that newer Pentiums do not demonstrate this behavior. If true, then Intel has unceremoniously fixed this behavior, and failed to mention this fact in their errata documents.
- IRET is no longer IOPL-sensitive when CR4.VME=1. Consider that the Ev86 task is running and an interrupt's associated IR bit is set. When this interrupt is invoked, it will cause a fault to the monitor, or the protected mode interrupt will be invoked (depending on IOPL). The interrupt is then reflected back to the Ev86 task. When IOPL=3, the behavior of the fault and subsequent IRET matches non-Ev86 behavior. When IOPL<3, the subsequent IRET from the interrupt service routine doesn't cause a fault back to the monitor, as IRET is no longer IOPL-sensitive. This behavior is inconsistent with its non-Ev86 counterpart.
- When IOPL=3, the CPU never updates VIF, though software can manipulate it in CPL-0. This can lead to the next caveat...
- The CPU will generate a general-protection fault (#GP(0)) whenever VIF=1, and VIP=1 and running in CPL-3. This is regardless of whether or not the code is an Ev86 task.
- In CPL-0, IRETD has the ability to set VIF and VIP, but POPFD does not.
Conclusion
I have described everything I know about VME in this series, but I'm sure that even so, not everything is known and documented. For example, since I wrote my last VME column, I discovered a few more caveats. As time goes on, I'm sure more VME details will be discovered. Yet in these three articles, I've presented many times more information than is contained in Intel's manuals. In the release of its newest architecture manual, Intel has demonstrated that it wishes to say less about VME, thereby removing vital information from the engineers who need it most. Therefore, consider these articles a blessing, and save the source code contained within. You never know when you'll need it -- and you'll never know when Intel will release any more VME details.
DDJ
Listing One
CLI { /* v86 mode */
if (EFLAGS.IOPL == 3) {
then EFLAGS.IF = 0;
else if (CR4.VME == 0) { /* IOPL < 3 */
then #GP(0); /* ERROR CODE = 0 */
else EFLAGS.VIF = 0;
}
}
}
STI { /* v86 mode */
if (EFLAGS.IOPL == 3) {
then EFLAGS.IF = 1;
else if (CR4.VME == 0) { /* IOPL < 3 */
#GP(0); /* ERROR CODE = 0 */
else {
if (EFLAGS.VIP == 1) { /* if VIP already=1, #GP(?) */
then #GP(0); /* ERROR CODE = 0 */
/* Processor never set VIF */
/* before #GP() */
else EFLAGS.VIF = 1;
}
}
}
}
}
PUSHF { /* v86 mode */
if (EFLAGS.IOPL == 3) {
then
if (OperandSize == 32) {
then push(EFLAGS & 0xFCFFFF); /* Clear VM & RF */
else push(FLAGS);
}
/* IOPL < 3 */
else if ((CR4.VME == 0) || (OperandSize == 32)) { /* IOPL < 3 */
then #GP(0); /* ERROR CODE = 0 */
else {
TEMP = FLAGS;
TEMP = TEMP OR 0x3000; /* Set IOPL=3 on stack (dumb?) */
TEMP.IF = EFLAGS.VIF;
push(TEMP);
}
}
}
}
POPF { /* v86 mode */
if (EFLAGS.IOPL == 3) { /* IOPL = 3 */
then if (OperandSize == 32) {
then {
TEMP = pop();
/* Clear these fields from EFLAGS stack image: */
TEMP = TEMP AND NOT 0x1BB02A; /* VIP VIF VM RF IOPL */
/* Keep these fields from current EFLAGS register: */
EFLAGS = EFLAGS AND 0x1B3002; /* VIP VIF VM RF IOPL */
EFLAGS = EFLAGS OR TEMP;
}
else {
TEMP = pop();
/* Clear these fields from EFLAGS stack image: */
TEMP = TEMP AND NOT 0xB02A; /* IOPL */
/* Keep these fields from current EFLAGS register: */
FLAGS = FLAGS AND 0x3002; /* IOPL */
FLAGS = FLAGS OR TEMP;
}
}
else if ((CR4.VME == 0) || (OperandSize == 32)) { /* IOPL < 3 */
then #GP(0);
else if ([SP].TF) { /* If stack image TF=1, then #GP */
then #GP(0);
else if ((VIP == 1) && [SP.IF]) {
then #GP(0);
else {
TEMP = pop();
EFLAGS.VIF = TEMP.IF;
/* Clear these fields from EFLAGS stack image: */
TEMP = AND NOT 0xB22A; /* IOPL, IF */
/* Keep these fields from current EFLAGS reg */
FLAGS = FLAGS AND 0x3202;/* IOPL, IF */
FLAGS = FLAGS OR TEMP;
}
}
}
}
}
}
IRET { /* v86 mode */
if (EFLAGS.IOPL == 3) /* IOPL = 3 */
then if (OperandSize == 32) {
then {
EIP = pop();
CS = pop();
TEMP = pop();
/* Clear these fields from EFLAGS stack image: */
TEMP = TEMP AND NOT 0x1BB02A; /* VIP VIF VM RF IOPL */
/* Keep these fields from current EFLAGS register: */
EFLAGS = EFLAGS AND 0x1B3002; /* VIP VIF VM RF IOPL */
EFLAGS = EFLAGS OR TEMP;
}
else {
IP = pop();
CS = pop();
TEMP = pop();
/* Clear these fields from EFLAGS stack image: */
TEMP = TEMP AND NOT 0xB02A; /* IOPL */
/* Keep these fields from current EFLAGS register: */
FLAGS = FLAGS AND 0x3002; /* IOPL */
FLAGS = FLAGS OR TEMP;
}
}
else if ((CR4.VME == 0) || (OperandSize == 32)) { /* IOPL < 3 */
then #GP(0);
else if ([SP].TF) { /* If stack image TF=1, then #GP */
then #GP(0);
else if ((VIP == 1) && [SP.IF]) {
then #GP(0);
else {
IP = pop();
CS = pop();
TEMP = pop();
EFLAGS.VIF = TEMP.IF;
/* Clear these fields from EFLAGS stack image: */
TEMP = AND NOT 0xB22A; /* IOPL, IF */
/****************************************************/
/* ** NOTE ** The following treatment of TF flag */
/* ********** *MAY BE A BUG* in the Pentium. */
/* POPF uses the FLAGS mask of 0x3202, but IRET */
/* uses 0x3302. Consider that POPF and IRET */
/* #GP on a condition involving TF. So is it */
/* just coincidence that IRET has further */
/* special treatment of TF on IRET, where POPF */
/* does not? */
/****************************************************/
/* Keep these fields from current EFLAGS reg */
FLAGS = FLAGS AND 0x3302;/* IOPL, IF, TF */
FLAGS = FLAGS OR TEMP;
}
}
}
}
}
}
Copyright © 1998, Dr. Dobb's Journal
