Underdocumented C++

I call such features "underdocumented C++" with apologies to Andrew Schulman, whose "Undocumented..." and "Unauthorized..." series of books were popular several years ago. A feature with well-understood behavior but ill-defined motivation is, in my opinion, underdocumented. When you are trying to explain programming language behavior to a student, it really helps to know why the language behaves as it does. It helps the student, too.

(By the way, if you think a computer journalist should not use valuable column space to plug his own self-interests and those of his friends and family, you should refer to our heritage and legacy from the early days of computer journalism. Precedent and tradition exists. It is called the "Pournellian imperative.")

Argument-Dependent Name Lookup

The underdocumented feature of the month is called "argument-dependent name lookup," nicknamed "Koenig lookup" after Andrew Koenig, the committee member who invented it. I am saddened to report that neither of the recent excellent C++ books coauthored by Andrew even mentions the feature. Saddened? Yeah, right. Did I mention the name of my book? Oh, yeah, I did.

I searched the Web for a thorough explanation of the feature and found a link (http://www.gotw.ca/publications/ mill02.htm) to Herb Sutter's article in the March 1998 issue of the now-defunct C++ Report, which, to support the subject of Herb's article, explained Koenig lookup sufficiently that I could understand why it is necessary. From that and several other articles, I put together my own description of the feature to include in my book. The treatment here draws on that text and expands considerably upon it. The level of detail here is a bit deep for my book's targeted readership, entry-level C++ programmers.

Ordinary Name Lookup

Understanding Koenig lookup requires an understanding of ordinary name lookup. When the compiler encounters an unqualified identifier in your source code—that is, an identifier with no prefix to specify where the identifier is declared—the compiler launches a lookup for the name starting in the innermost scope. With function names, this lookup includes matching argument types to parameter types to accommodate overloaded function possibilities. The search continues through each successive outer scope until the compiler either finds a name match or hits the wall, which, in this case, is the global scope, at which time the compiler reports the undeclared identifier as a compile-time error.

The nested levels of scope include brace-surrounded statement blocks, function parameter lists, namespaces, class declarations, and the global scope. Some of these things can be within others of these things according to how the program is structured.

Given ordinary name lookup, a function declared in a namespace can be called without the namespace prefix only if the function call itself is within the namespace, perhaps nested several scope levels deep. Programmers don't usually think about compiler behavior when they write code. They think about how the code ought to work, and ordinary name lookup is intuitive. Koenig lookup, or rather its effects, are not. Consequently, it surprises programmers the first time they encounter it.

Koenig Lookup

You might discover Koenig lookup the first time you call a Standard Library function and find that, although you forgot to provide the std:: prefix, the compiler finds the correct function as shown here:

std::vector<int> vint;

// ...

sort(vint.begin(), vint.end());

Even though the sort function is declared within the std:: namespace, you do not need to provide the namespace qualifier when you call it. The behavior is as if the function was in the global namespace. A search of the library headers finds no sort function in the global namespace. This is not the intuitive ordinary name lookup you have come to expect. Your first reaction is to report a bug to the compiler vendor.

The compiler vendor responds patiently, and if tech support is on the ball, you learn that a special circumstance exists wherein the namespace prefix is not required for a function identifier declared in a namespace. To whit: When a function is declared in a namespace and at least one of its parameters is also declared in the namespace, a call to the function does not need to qualify the function name with the namespace as long as it properly qualifies the argument that matches the parameter. Let's step away from the Standard Library functions and examine a generic example that illustrates Koenig lookup. Consider the program in Listing One.

Even though the paycheck function in Listing One is declared inside the personnel namespace, the main function, which is, of course, outside the personnel namespace can call the paycheck function without specifying the namespace—as long as the argument to the function includes namespace information. The empl object is declared as being an object of type personnel::employee, which gives the compiler sufficient namespace information under the rules of Koenig lookup. However, suppose the main function looked like Listing Two.

In Listing Two the call to paycheck is invalid, because the argument, a null address, does not provide the compiler with any namespace information to use in selecting the function to call.

What is the purpose of Koenig lookup? Why is it necessary? What problem does it solve? Consider the following code:

std::cout << "hello";

The line of code just shown is actually C++ shorthand for calling an overloaded operator function. In this case, the overloaded operator function is:

std::operator<<(std::ostream&, const char*).

Without Koenig lookup, the compiler would not know which overloaded operator<< function to call, and there are many such overloaded functions. The Standard Library has its own and you might have them in your own classes, too. Consider the code in Listing Three, which overloads the operator<< function for displaying objects of type employee on std::ostream objects.

In Listing Three, the employee class's overloaded operator function is in the personnel namespace. The two calls to overloaded operator<< functions in the main function call different functions in different namespaces, yet neither call explicitly tells the compiler which function to call. But both calls have arguments that reference objects of classes declared within those namespaces. Remember that the two statements are C++ shorthand for calling the overloaded operator functions. They could be coded like this as well:

operator<<(std::cout, "Employee: ");

operator<<(std::cout, empl);

When the compiler looks for a function to match the first call, it looks in the std namespace because the first argument is in the std namespace. The compiler finds a function there with a parameter list that matches the two argument types, std::ostream and const char*. When the compiler looks for a function to match the second call, it does not find one in the std namespace because the Standard Library does not define an overloaded operator<< function with those two types; the Standard Library does not know about the user-defined employee class, which is the type of the second argument to the function call. So the compiler uses the personnel namespace from that second argument and looks for a matching overloaded operator<< function there, which, of course it finds. The two statements shown above would be compiled this way after the compiler applies Koenig lookup:

std::operator<<(std::cout, "Employee: ");

employee::operator<<(std::cout, empl);

It would be reasonable to expect a programmer to provide the namespace qualification in all cases except for the shorthand notation that supports overloaded operators. Take another look at the two uses of overloaded operator<< in Listing Three. Where would you put the namespace qualification? It would be unreasonable to expect programmers to write code like this:

std::cout std::<< "Employee: ";

std::cout employee::<< empl;

I did not participate in the standardization process, so I can only guess about what happened when the framers of the namespace feature bumped into the problem of overloaded operator functions declared within namespaces. No doubt there was some head-scratching, nail-biting, and hair-pulling among the committee members, after which Mr. Koenig came up with a rather clever solution, and Standard C++ now has Koenig lookup as one of its quiet features. Without Koenig lookup, everything you know about using std::cout, and overloading operator<< would not work.

A side effect is that sometimes functions declared in namespaces behave as if they are in the global namespace because you can call them without specifying their namespaces. This can confuse programmers who subconsciously expect ordinary name lookup and don't include Koenig lookup in their day-to-day thought processes. Which begs the question: Why didn't the committee opt to apply Koenig lookup only to calls to overloaded operator functions? I don't know the answer to that. Perhaps there is a good reason. There must be. Surely, I'm not the first programmer to wonder about that. This is one of those things that happen when programming languages mutate and evolve and are designed in a committee. New constructions build upon old ones, and sometimes things get in the way.

DDJ

Listing One

namespace personnel {
  class employee { };
  void paycheck(employee* emp)
  {
    // ...
  }
}
int main()
{
   personnel::employee empl;
   paycheck(&empl);
   // ...
   return 0;
}

Back to Article

Listing Two

int main()
{
   paycheck(0);
   // ...
   return 0;
}

Back to Article

Listing Three

namespace personnel {
  class employee {
    // ...
  };
  std::ostream& operator<<(std::ostream& os, const employee& em);
}
int main()
{
   personnel::employee empl;
   // ...
   std::cout << "Employee: ";
   std::cout << empl;
   return 0;
}

Back to Article