The Win95/Win98 Caret Bug

The Win95/Win98 Caret Bug

Chris Branch

If you’re creating a custom control that accepts text input, chances are you’ll need to display a caret. Displaying a caret is fairly simple because Windows provides API functions that handle most of the grunt work. But as I recently found out, these functions can cause problems in some situations. Specifically, a bug in Windows 95 and Windows 98 can prevent the caret from being erased correctly when using SetCaretPos() or HideCaret(). In this article, I’ll provide an analysis of the problem and then describe several ways to avoid it.

Problem Demo

The code in my custom edit control that initially revealed the bug is fairly complex. However, I was able to extract the relevant pieces to create a small Win32 program that elicits the problem. The demonstration program in caretbug.c (Listing 1) displays a window that lets you move a caret using the arrow keys. Each time the user presses an arrow key, I call SetCaretPos() to reposition the caret.

Under Win95 or Win98, the operating system draws the caret as expected in most areas of the window. However, in a band 20 pixels tall near (but slightly below) the top of the client area, the operating system fails to erase the caret properly when moving it to a new position. As you move the caret horizontally, SetCaretPos() leaves a trail of “caret droppings.” For the demo program to work as I’ve described, it’s important that you do not move the window from its initial position. The reason for this will become clear later.

If you examine the source code at the spot where I process the arrow keys (OnKeyDown()), you’ll find that I call InvalidateRect() right before calling SetCaretPos(). The demo program does not actually draw anything, but in my real project I have a legitimate reason for doing this. For the analysis that follows, it’s important to remember that InvalidateRect() does not redraw the window immediately. Instead, the rectangle is added to an update region maintained by the window. At a later time, Windows sends a WM_PAINT to redraw the entire update region.

Finding the Bug

With the aid of SoftICE, I found the source of the problem in the Windows code that actually draws the caret. Internally, ShowCaret(), HideCaret(), and SetCaretPos() eventually call the same internal Windows function to draw or erase the caret. This function, which I’ve named DrawErase(), bitblts the caret to the screen with an XOR operation. Because DrawErase() performs an XOR, calling this function once draws the caret, and calling it a second time erases the caret.

The code for DrawErase() attempts to optimize by skipping unnecessary draw/erase operations. If the area occupied by the caret intersects the current update region, DrawErase() returns immediately without drawing. This makes sense because the caret will need to be redrawn later anyway when the window eventually processes WM_PAINT.

To check the update region, DrawErase() calls a helper function, which I’ve named SkipDrawErase(). Figure 1 contains the pseudo code for SkipDrawErase() that I reverse engineered using SoftICE. If the update region is empty, SkipDrawErase() returns FALSE and drawing proceeds normally. Likewise, if the entire window is invalid the function returns TRUE, which tells DrawErase() to skip the drawing. If a portion of the window has been invalidated, then SkipDrawErase() needs to see if the rectangle occupied by the caret intersects the update region. To do this, it temporarily creates a region for the caret rectangle and then calls CombineRgn() to see if the two regions overlap. The return value from CombineRgn() determines if any overlap exists.

The problem with this code is that there is a mismatch of coordinate systems. The update region (hrgnUpdate) is maintained in screen coordinates, but the temporary caret region (hRgn) is created using client coordinates. Therefore, the result of CombineRgn() is completely meaningless. The end result is that SkipDrawErase() sometimes returns TRUE when the caret is actually outside the update region. If the caret is outside the update region, it won’t be erased while handling WM_PAINT, so you end up with multiple carets.

Since the update region is maintained in screen coordinates, the portion of the window affected by the bug will change as you move the window to different positions on the screen. You can calculate the affected area using the following code:

RECT r = { 0, 0, 300, 20 }; // client area invalidated
POINT p = { 0, 0 };
ClientToScreen(hwnd, &p);
OffsetRect(&r, p.x, p.y);   // client area with caret trouble

Based on this analysis of the problem, it’s clear that you can’t expect reliable results if you call SetCaretPos() while the window has invalid areas. The logical solution is to reorder the code to call SetCaretPos() before calling InvalidateRect(). In theory, this should avoid the problem because the update region is still empty at the time SetCaretPos() is called. In practice, however, this doesn’t work either.

Back in the trenches with SoftICE, I discovered why my revised demo program also failed to work correctly. This time the culprit was HideCaret(). Although you won’t find an explicit call to HideCaret() in my demo program, it turns out that BeginPaint() automatically calls HideCaret(). The code for HideCaret() looks like this:

++pCaret->wHideCount;
if (pCaret->bOnScreen) {
    DrawErase();
    pCaret->bOnScreen = FALSE;
}

When BeginPaint() calls HideCaret(), the update region has not yet been cleared. As a result of the bug, DrawErase() does not erase the caret even though it really is outside the update region. However, notice that the code following the call to DrawErase() changes bOnScreen to indicate that the caret is no longer visible. This causes problems later when the application calls EndPaint() to complete the update. EndPaint() automatically calls ShowCaret() to redisplay the caret. The relevant code for ShowCaret() looks like this:

if (pCaret->wHideCount > 0) {
    --pCaret->wHideCount;
    if (pCaret->wHideCount == 0) {
        pCaret->bOnScreen = TRUE;
        DrawErase();
    }
}

This code essentially performs the reverse of HideCaret(). However, this time the update region is empty because the window is finished processing WM_PAINT. Since the update region is empty, DrawErase() actually bitblts the caret to the screen. The caret was not erased earlier, so this draw operation actually erases the caret (remember, it’s an XOR). The net result is that ShowCaret() erases the caret, but sets bOnScreen to TRUE, which is out of synch with reality. Later when SetCaretPos() is called, it checks bOnScreen to determine if it needs to erase the caret at the current position before moving to the new position. Since the flag is incorrect, SetCaretPos() fails to erase the old caret.

A Workaround

In short, you cannot invalidate a portion of your window while the caret is visible. One solution is to invalidate the entire client area, so the faulty code in SkipDrawErase() is never executed. In my case, this solution was unacceptable because a full redraw was too expensive. A better solution is to bracket the call to InvalidateRect() with HideCaret() and ShowCaret(). For example:

HideCaret(hwnd);
InvalidateRect(hwnd, &r, TRUE);
ShowCaret(hwnd);
SetCaretPos(*x, *y);

This avoids the problem by ensuring the caret is erased before the window is invalidated. In reality, calling ShowCaret() after InvalidateRect() does execute the faulty code. However, any problems are “cancelled” later by BeginPaint() and EndPaint(), so the net result is correct. This is the solution I adopted for my own code.

Summary

This bug is specific to Win95/Win98; NT appears to work correctly. However, the workaround I provided also works correctly on NT, though it isn’t really necessary. This is convenient because you don’t need platform-specific code to avoid problems.

Chris Branch is a software engineer at FSCreations, Inc. in Cincinnati, Ohio. He can be reached at [email protected].

Get Source Code