Andrew Koenig is a former AT&T researcher and programmer. He is author of C Traps and Pitfalls and, with Barbara, coauthor of Ruminations on C++ and Accelerated C++. Barbara Moo is an independent consultant with 20 years of experience in the software field.
Someone once asked us the following question: Suppose the std::string class did not have a swap member, and you wanted to swap the contents of two string variables s and t without having to take the time to copy the strings' constituent characters. Could you do so this way?
// Will this work?
{
unsigned char
temp[sizeof(std::string)];
memcpy(temp, s,
sizeof(sizeof(std::string)));
memcpy(s, t,
sizeof(sizeof(std::string)));
memcpy(t, temp,
sizeof(sizeof(std::string)));
}
The C++ Standard effectively guarantees that any object corresponds to a sequence of unsigned characters, in the sense that we can use memcpy to copy those characters somewhere else and then copy them back to their original location without changing the meaning of the object—provided, of course, that we have done nothing to change the object in the meantime. In other words, if we do this:
// This will work
{
unsigned char
temp[sizeof(std::string)];
memcpy(temp, s,
sizeof(sizeof(std::string)));
memcpy(s, temp,
sizeof(sizeof(std::string)));
}
there is a correspondence between the contents of s and the sequence of unsigned characters that constitute temp. This correspondence ensures that temp captures the entire contents of s, so when we copy s to temp and then copy temp back to s, doing so doesn't change the value of s.
Of course, this second example is useless because if we do anything to change the values of s between the two calls to memcpy, the behavior of the second memcpy is no longer defined. What makes the first example interesting is that we do not do anything overt to change the value of s or t, which leaves open at least the possibility that this example would be a good way of swapping the values of s and t quickly.
If instead of being strings, s and t were of a so-called POD ("plain old data") type—a type that consists entirely of components with no user-defined constructors, copy assignment operators (an operator= that accepts a reference to its own object type), or destructors—this technique would work just fine. An object of POD type necessarily uses the built-in copy assignment operator, and because none of a POD's components have their own copy assignment operators, using memcpy to copy a POD has the same effect as copying its elements. It is true that because std::string has a destructor and copy assignment operator, it is not a POD type—but surely, if we just use memcpy to swap the representations of two string objects, we won't hurt anything. Right?
Our first reaction to this question was that it made our skin crawl. Nevertheless, after much thought, we concluded that it was unlikely in practice that any implementation of std::string would break under this kind of operation. Despite that conclusion, we still believe that our first reaction was right, and that programmers should normally reject programming techniques such as this one on philosophical grounds, without looking at specific technical issues. The rest of this column explains our reasoning.
Abstract versus Concrete Data Structures
We are fond of saying that abstraction is selective ignorance. For example, when we talk about an abstract data structure (or data type), we mean a data structure that lets us selectively ignore some of the aspects of its implementation. In C++, we usually define such data structures by wrapping them in class definitions. Programmers who use a class can ignore its implementation details. Moreover, the class author can concentrate on implementation without having to worry that users might take advantage of implementation details in ways that might make it difficult to improve the implementation in the future.
Not all C++ data structures are abstract. When a data structure is not abstract, we call it concrete. An abstract data structure conceals some of its properties—usually including its implementation details—from its users; a concrete data structure exposes its structure to its users.
If you are reading someone else's code, it can be useful to realize that a class with public data members is usually concrete. If a data member is public, that member grants access to part of the data structure's representation, which means that there is no way to conceal that part of the representation. In contrast, if all access to a data structure is through member functions, those functions can do just about anything. There is no need for them to return values that correspond directly to any part of the underlying data structure. Therefore, it is much easier to conceal a class's implementation from its users if you avoid all public data members.
Of course, when we say that a data structure is abstract or concrete, we are really talking about its author's intentions. Public data members are only one way in which authors can make those intentions clear. Other ways include disclosing (or not disclosing) details of the class's implementation, or saying explicitly how they intend their classes to be used and what kind of abstractions they intend to support.
One widely used example of a concrete data structure is the pair template from the C++ Standard Library. The C++ Standard defines the pair template this way:
template <class T1, class T2>
struct pair {
typedef T1 first_type;
typedef T2 second_type;
T1 first;
T2 second;
pair();
pair(const T1& x, const T2& y);
template<class U, class V> pair(const pair<U, V> &p);
};
Everything is public; there is no abstraction to be found.
In contrast, the standard string type does not include any public data members. Moreover, if we read the description of how the string type works, we find that there are no specific requirements on how to implement it. These facts should suggest to us that string is intended to represent an abstraction: We are told what its properties are and how to use it, but not how it works.
It is because the string class is abstract that our skin crawls when we think of using memcpy on it. When someone writes a class that represents an abstraction, we believe that we should limit ourselves to using its abstract properties and not venture beyond those properties—even if we can convince ourselves that it is safe to do so.
A principle emerges from this discussion: Do not violate abstractions. They cease to be abstractions if we violate them because we are no longer ignoring what we agreed to ignore by using the abstractions in the first place. By violating abstractions, we make our programs depend on implementation details that the implementer assumed we would be ignoring. If those details are different on another implementation, or change in the future on the implementation that we are using, any parts of our program that depend on those details might break without warning.
Another Application of the Principle
So far, we have seen that abstractions conceal information from their users, and for that reason, it is a bad idea to violate them. This principle would have served us well in 1994, when one of us wrote an article in the C++ Report [1] about a popular suggestion at the time. The suggestion was to make it easier to define an assignment operator for a class by having it use the destructor and copy constructor:
class Thing {
public:
Thing& operator=(const Thing& t) {
this->~Thing();
new(this) Thing(t);
return *this;
}
// ...
};
The article started by saying "Yuck!" as a reaction to this technique. It went on to discuss in detail why the technique was unreliable: It did not account for self assignment, it failed if the class had a virtual destructor, and if the class were to use inheritance, assignment would probably leave the destination object with a dynamic type that differed from its static type.
What the article did not do was to explain the "Yuck!" Instead, it justified its search for concrete reasons to avoid the technique by saying, "Gut feelings by themselves are not enough to answer technical questions." This statement is certainly true. Nevertheless, we now believe that the article could have made a more general argument against the technique without looking for specific things that might go wrong. The argument revolves around the C++ notion of an object. This notion is really a kind of abstraction that the language provides for us, and we could have argued that the technique violates this abstraction.
Part of why we say that the C++ notion of an object is an abstraction is that whenever a local variable contains an object, the compiler might have to remember that the object exists, so that it can arrange to destroy the object at the appropriate time. This memory is a part of the implementation that we can ignore under ordinary circumstances. For example:
{
Thing t;
// ...
}
After the definition of t, the compiler has to remember that the local object t exists, so that when the block exits—such as by reaching the } or by throwing an exception—the compiler can arrange to destroy t.
Moreover, C++ has a notion of a partially constructed object. For example, if we were to define a variable of type pair<T1, T2>, and T2's copy constructor were to throw an exception as part of constructing that variable, the compiler would have to remember that the variable's T1 part was already constructed and must be destroyed on the way out of the block.
Because the notion of an object is an abstraction that the language provides, we should avoid violating that abstraction. If we explicitly destroy all or part of a local variable, as would happen with the assignment operator discussed in this article, we are doing exactly that. This fact is all that we should need to know in order to decide to avoid writing an assignment operator that way. If we want to break through the abstraction level and deal with such objects concretely, we need to know their concrete properties—but the language does not define these properties, so we cannot know what they are. The best we can do is to know how a particular implementation provides such properties, which isn't good enough. If we use such concrete properties, we are effectively restricting ourselves to using implementations that provide them in the same way. However, if these properties are part of an abstraction, then there is generally not any guarantee that later versions of the same implementation will provide them in the same way, let alone other implementations. In effect, by breaking through the abstraction level, we make future evolution much more difficult.
Another example of violating abstractions turns out to have been described in the very next issue of C++ Report [2]:
struct Stuff {
int length;
char data[1]; // [sic]
};
This is a common technique among C programmers: Define a structure that ends with what looks like a one-element array, allocate more memory than the structure actually requires, and then use out-of-bounds indices on the array to access that memory. The C++ Report article raised the question of whether this technique should be considered valid in C++ and concluded that it had at least two severe problems.
The first problem is a pragmatic one: If the Stuff structure has any virtual functions, the compiler might put data of its own (such as a virtual-function table pointer) at the end of the structure, and running off the end of data might overwrite the compiler's hidden data.
This problem is really a consequence of a second, more fundamental problem: data is an array, and an array is a kind of abstraction, defined by both C and C++, that provides specific operations. Those operations do not include accessing elements past either end of the array (although you are allowed to compute the address of where the element immediately past the end of the array would be if it existed). Therefore, whenever a program attempts to access an element of data, the implementation is permitted to check whether the element being accessed actually exists, and to stop the program with a diagnostic message if there is no such element.
Because of this restriction, C++ implementers have felt free to insert internal data structures such as virtual-function tables at the end (or even in the middle) of user-defined data structures without worrying about the possibility that programs might intentionally run off the end of arrays that happen to be adjacent to such structures. Violating the array abstraction will be a disaster on such implementations.
Even if our program appears to work, we still should not violate abstractions. Once we break the abstraction layer, we take on the responsibility of proving that our program will work not only on our current implementation today, but also on other implementations we might use in the future. Moreover, we must do so without knowing the details that the implementers quite reasonably hid from us, assuming that we would not try to rely on them.
Discussion
A common thread underlies these examples: If you wish to use an abstraction, you have to restrict yourself to the properties that it defines—otherwise, you're misusing it. The behavior of memcpy is defined on POD structures but not others, so you should restrict yourself to using memcpy on POD structures. For that matter, you should use memcpy only on data structures that you can be confident will remain POD in the future.
Similarly, the notion that a local variable can be a class object is an abstraction. Among other properties, that abstraction includes the notion that the compiler keeps track of the state of construction of such a variable at each point in the program. To use that abstraction appropriately, you should avoid doing anything that might violate its properties, such as writing an assignment operator that explicitly destroys all or part of its object. Our final example involved the array abstraction and again observed that, in order to use the abstraction, we should refrain from trying to access array elements that do not exist.
In all three cases, we tried to achieve the simplicity of using abstractions while violating them at the same time. In all three cases, it was sometimes possible that the resulting programs would appear to work despite these violations. However, whenever we reach through an abstraction layer, we are betting on our knowledge of how the implementation works and how our programs might interact with parts of the implementation about which we know nothing. Moreover, we are also betting that those hidden parts of the implementation will work the same way on other implementations and will remain the same over time.
Such confidence is hard to attain in practice. For example, if we copy the memory that corresponds to the internal representation of an object, are we sure that the copy will remain valid in its new location? Under every implementation? If we have destroyed part of a local variable and an exception occurs, what will happen if the compiler tries to destroy those same parts again? If we try to access a nonexistent element of an array, are we sure that we won't inadvertently overwrite memory that is important to the implementation, and that the implementation won't try to detect our error? If we knew all of the implementation details, we could depend on them. But when we use abstractions, we are choosing to ignore many of those details. Once we have decided to ignore them, it makes no sense to assume that we know what they are.
References
- Koenig, Andrew. "Using Constructors for Assignment," C++ Report 7(2), 1994.
- Koenig, Andrew. "Cheating on Array Bounds," C++ Report 7(3), 1994.
CUJ
