Enhancing Assertions

OK, I admit: I am experiencing "writer's block." The material is here, it's cool, I've had my favorite breakfast (homemade muesli, my personal recipe: 50% oats, 50% nuts, 50% raisins - try to beat that!), my trusty open source editor's caret is blinking invitingly (I know what you're thinking: "Invitingly? Incurable geek!"), yet I can't find a good introductory paragraph. Reason for which I'm resolving my angst, dear reader, by annoying you with these meta-writing quibbles in lieu of that good first paragraph. But then this is a meta-programming column, right? Plus, you're a C++ programmer, so you would hardly mind some extra syntax.

Now that we took the first paragraph problem out of the way, please allow me to introduce my long-time e-mail friend John Torjo, C++ expert, consultant, and "Practical C++" columnist for http://builder.com. John and I are co-authoring this article that describes John's improved assertion framework.

After reading my article on assertions [1], which is mainly the result of the discussion that Jason Shirk and I have had, John found it wanting. More precisely, he found himself wanting more features from an assertion facility. This happened because that article on assertions, and the code accompanying it, was using the Simple World Assumption. This is a highly specialized scientific term that you may or may not know about, so please allow me to detail it a bit.

Under the Simple World Assumption, programmers work normal hours and have reasonable schedules. They have the time and are given incentive to conduct code reviews and write tests. Therefore, whatever failing assertions are planted in the code will be fired by the unit and integration tests. Programmers test the code, debug it, make sure no assertion fails in comprehensive testing scenarios, and finally compile with NDEBUG defined and deliver a small and fast executable to their managers, who, in turn, take care of delivering it to a happy customer base. By the way, under the Simple World Assumption, managers are considered competent people who help the programmers do their job and add no undue stress or overhead. (As it turns out, the Simple World Assumption does not hold in practice. Ahem.)

In a more realistic world, programmers endure significant schedule pressure as the norm, and therefore instead of writing unit tests, they throw the not-so-tested program over the fence to the black-box testing team, which creates bug reports together with the scenarios in which those bugs arise.

Figuring out exactly the sequence of events that leads to the manifestation of a bug is not always easy. A rich state space, threads, or event-driven programming are going to make bug reproducibility quite difficult. Random behavior created by uninitialized variables, ill casts, or buffer overruns only add to the fun. Hey, I almost forgot the various system settings such as installed DLLs and registry settings... (Ever write an application that runs perfectly on your system and fails mysteriously on another?)

A way to help the situation, John says, is to devise a better assertion framework that extends assertion's charter. Specifically:

• There are multiple levels of assertions; design errors come in more shades than just black and white. At one extreme, there are the most obvious checks, the least likely to fail as the application becomes mature. At the other extreme, there are "low-cost, high-utility" checks that you can leave in for the professional testers, beta testers, and sometimes even for the final users of your application. I personally don't enjoy the idea of the final users seeing the assertion messages, but John sure convinced me that that's quite a possible scenario. Besides, read below because there are various ways in which failing checks are reported.
• Displaying messages is not enough, especially when the development and the testing teams are separated. A logging facility would be extremely useful; after a failing run, the developers can take a look at what assertion (or sequence of assertions) failed.
• The quality of the messages is greatly improved, containing detailed information not only about the failed expression, file, and line, but also (under programmer's control) the values of related variables.

All of these extra features come in an industry-strength package, building on the original assertion code, to which John added some tricks also used in the Enforcements article [2] and many of his own. We will detail below how the improved assertion package works.

Providing Extra State Information

When an assertion fails, it means the expression that ASSERT evaluates is false. Often, however, you might be interested in knowing the actual values of key variables that caused the assertion to fail. For example, consider that at a point you are confident that two strings are empty. So you write:

string s1, s2;
...
ASSERT(s1.empty() && s2.empty());

Should this assertion fail, it is very likely you'd want to know what the actual content of myString was; that would provide insight into where it was last updated. John's framework allows you to do that by using the syntax:

SMART_ASSERT(s1.empty() && s2.empty())(s1)(s2);

Notice the use of the parentheses that offer a nice extensible way to provide extra arguments to ASSERT, similar to ENFORCE [2] (yet, of course, ENFORCE would not be the first component that uses operator() in that particular style).

When doing so, if the assertion fails, the message displayed and logged will look like this:

Assertion failed in matrix.cpp: 879412:
Expression: 's1.empty() && s2.empty()'
Values: s1 = "Wake up, Neo"
        s2 = "It's time to reload."

Here's how the magic works. Get ready for something quite tricky, but definitely worth knowing. (To understand and enjoy it, you need to wake up your inner C macro lover.) First, the basic idea is that to grab the name and the value of a variable, you need to use the "stringizing" operator #. In the example above, you need to use the stringizing operator on myString, But the stringizing operator works only when inside a macro, and here's what makes everything so tricky: you need an "infinitely expansible" macro mechanism - a macro that expands into something that can be continued as a macro (so you can collect more variables s3, s4...). But we all know that recursive macros don't work.

However, the trick is doable if you press the gas pedal, open the sunroof, and hold the antenna with your left hand at the same time. Credits are due to Paul Mensonides, the one who (as far as we know) invented this trick. Here's what you need to do:

First, inside your Assert class (defined and used much as the homonym class in my previous article), add two member variables SMART_ASSERT_A and SMART_ASSERT_B. Their type is Assert&.

class Assert
{
    ...
public:
    Assert& SMART_ASSERT_A;
    Assert& SMART_ASSERT_B;
    // whatever member functions
    Assert& print_current_val(bool, const char*);
    ...
};

Make sure you initialize these members with *this. The whole purpose of all this is that, if you have an object of type Assert called obj, you can write obj.SMART_ASSERT_A or obj.SMART_ASSERT_B - and the whole entity will behave just like obj itself, given that the references refer to *this. Second - and here's the cool trick - define two macros SMART_ASSERT_A and SMART_ASSERT_B that kind of "recurse to each other". Here's how:

#define SMART_ASSERT_A(x) SMART_ASSERT_OP(x, B)
#define SMART_ASSERT_B(x) SMART_ASSERT_OP(x, A)
#define SMART_ASSERT_OP(x, next) \
    SMART_ASSERT_A.print_current_val((x), #x).SMART_ASSERT_ ## next

As you can see, when you invoke SMART_ASSERT_A(xyz), it will expand into something that ends with SMART_ASSERT_B. When you invoke SMART_ASSERT_B(xyz) it will expand into something that ends with SMART_ASSERT_A. During the expansion the two macros grab the value xyz you passed and its representation as string.

Yes, it is tricky. One observation that helps with understanding this trick is that, when the preprocessor sees an open parenthesis after SMART_ASSERT_A (or _B), it treats that as a macro invocation. If there's no parenthesis, the preprocessor just leaves the symbol alone. When left alone, the symbol is just a member variable. Here's the SMART_ASSERT definition that starts the tandem action of the two macros:

#define SMART_ASSERT( expr) \
    if ( (expr) ) ; \
    else make_assert( #expr).print_context( __FILE__, __LINE__).SMART_ASSERT_A 

If at this point you say "Aha!" you're saying exactly what I said after looking over the code for the twentieth time. Ok, now let's follow how the macros expand starting from the initial expression:

SMART_ASSERT(s1.empty() && s2.empty())(s1)(s2);

Let's first expand SMART_ASSERT. (We won't necessarily obey the order in which the preprocessor does things, but for clarity's sake, we'll break down the process in bite-sized chunks. Also, we'll take the liberty to reformat the code as we expand.)

if ( (s1.empty() && s2.empty()) ) ; 
else make_assert( "s1.empty() && s2.empty()").
    print_context("matrix.cpp", 879412).SMART_ASSERT_A(s1)(s2);

Now let's expand SMART_ASSERT_A and the resulting SMART_ASSERT_OP:

if ( (s1.empty() && s2.empty()) ) ; 
else make_assert( "s1.empty() && s2.empty()").
    print_context("matrix.cpp", 879412).
    SMART_ASSERT_A.print_current_val((s1), "s1").
    SMART_ASSERT_B(s2);

Notice how SMART_ASSERT_A is not treated as a macro anymore because it is not followed by a (. Expanding SMART_ASSERT_B and the resulting SMART_ASSERT_OP gives us the final answer:

if ( (s1.empty() && s2.empty()) ) ; 
else make_assert( "s1.empty() && s2.empty()").
    print_context("matrix.cpp", 879412).
    SMART_ASSERT_A.print_current_val((s1), "s1").
    SMART_ASSERT_A.print_current_val((s2), "s2").SMART_ASSERT_A;

This, considering that SMART_ASSERT_A is a member variable and that all member functions return a reference to Assert, is a perfectly well formed statement.

Handling and Logging

When an assertion fails, two things happen in sequence:

• The failure information is being logged
• The failure is being handled depending on its level

The two activities are completely orthogonal and can be customized in separation. You can, for example, have a mode in which you never ask for input on error but you still log all errors, which is very useful in automated runs, push installation, or your "innocent user protection program."

You can customize logging by passing your own logger function to the static member function Assert::set_log(void (*assert_handler)(const assert_context &)). You can define your own assertion handler and plug it in (in the time-honored set_unexpected manner) by calling Assert::set_handler(level, void (*assert_handler)(const assert_context &)). assert_context contains the acquired context of the failed assertion, explained below.

Not Just One Type of Assert

As many senior programmers have noticed, an application can have different levels of assert, some more critical than others.

Let's look at how asserts are used. Typically, you use assertions when you assume some situations will never occur (i.e., you expect a size or an index variable to never be negative).

Sometimes you couple assertions with defensive programming (writing your code in such a way that it will "survive" invalid input), but not always. In the latter case, it could be better to throw an exception. Look at the following code:

void install_program( User & user, const char * program_name) {
// only admins can install programs
ASSERT ( user.get_role() == "admin");
...
}

In case the user is not an administrator, you'll prefer to throw an exception (otherwise the user could get away with doing forbidden things). Just change it to:

void install_program( User & user, const char * program_name) {
// only admins can install programs
SMART_ASSERT( user.get_role() == "admin").error();
...
}

We have 4 levels of assertions:

• lvl_warn (it's just a warning, the program can continue without user intervention)
• lvl_debug (it's the default, the usual assert)
• lvl_error (an error)
• lvl_fatal (it's fatal, it's most likely the program/system got unstable).

Each level can be handled differently. Thus, we have a handle per level. The defaults are:

• Warning: dump a message and continue program
• Debug: ask the user what to do (Ignore/Debug/etc.)
• Error: throw an exception (std::runtime_error)
• Fatal: abort the program.

You can rely on the defaults, or, as said above, changing the handler is as easy as:

Assert::set_handler( lvl_error, my_handler);

Here's how you set the level of the assertion:

SMART_ASSERT( user.get_role() == "admin").level( lvl_error);
SMART_ASSERT( user.get_role() == "admin").level( lvl_debug); // this is the default
SMART_ASSERT( user.get_role() == "admin").level( lvl_fatal);
SMART_ASSERT( user.get_role() == "admin").level( lvl_warn);

// shortcuts
SMART_ASSERT( user.get_role() == "admin").error();
SMART_ASSERT( user.get_role() == "admin").debug();
SMART_ASSERT( user.get_role() == "admin").fatal();
SMART_ASSERT( user.get_role() == "admin").warn();

Acquiring Context

When an assertion fails, it is up to you how you'd like it handled. For instance, you can decide to ignore it (if it's a warning), throw an exception, or the like. But most important, in case you decide to display it, you can choose how the data is laid out and what data is shown. You might choose to just show a simple message and have an "Advanced" option similar to Figure A.

You can customize logging in a similar manner. In order to allow this, when an assertion fails, it acquires context: the file, line it occurred on, its level, the expression that evaluated to false, and the values involved in the expression. Grabbing the __FILE__ and __LINE__ context is done in a similar manner to that described in [1].

class assert_context {
public:
    // where the assertion failed: file & line
    std::string get_context_file() const ;
    int get_context_line() const ;

    // get/ set expression
    void set_expr( const std::string & str) ;
    const std::string & get_expr() const ;

    typedef std::pair< std::string, std::string> val_and_str;
    typedef std::vector< val_and_str> vals_array;
    // return values array as a vector of pairs:
    // [Value, corresponding string]
    const vals_array & get_vals_array() const ;
    // adds one value and its corresponding string
    int add_val( const std::string & val, const std::string & str);

    // get/set level of assertion
    void set_level( int nLevel) ;
    int get_level() const ;

    // get/set (user-friendly) message 
    void set_level_msg( const char * strMsg) ;
    const std::string & get_level_msg() const ;
};

Usually, when logging, you'll want to record as much information as possible. Thus, most of the time you'll be happy with the default logger, which writes out everything in the context. Handling is at the other extreme. There are countless ways of handling an assert:

• Just ignore it
• Show only a summary to the user (the file:line it appeared on, and the expression)
• Show all details to the user on the console
• Throw an exception
• Show summary/ details on a UI-dialog.
• Abort with core dump, etc.

As your application reaches the beta state, you'll be happy with the defaults. However, as it matures and reaches a larger number of customers, you'll want finer control. You'll almost certainly want to override the default handlers and provide your own.

A customer-friendly handler could simply look like this:

// show a message box with two buttons: "Ignore" and "Ignore All"
void customerfriendly_handler( const assert_context & ctx) {
    static bool ignore_all = false;
    if ( ignore_all) return;
    std::ostringstream out;
    if ( ctx.msg().size() > 0) out << msg();
    else out << "Expression: '" << ctx.get_expr() << "' failed!";
    int result = message_box( out.str() );
    if ( result == do_ignore_all)
        ignore_all = true;
}
// putting it in place
Assert::set_handler( lvl_debug, customerfriendly_handler);

User-Friendly Messages

As said in the introduction, it's not always you who's debugging your program. Senior programmers keep telling you to document your code. The same goes for asserts. When an assertion fails, you'll want to know what that means. Let's look at a well-behaved ASSERT:

// too many users!
ASSERT(nUsers < 1000);

Should this assertion fail, the more or less informative message "nUsers < 1000" will be displayed and logged. A step forward would be to allow showing an explanatory string, which would give higher-level semantics to the expression:

SMART_ASSERT( nUsers < 1000)(nUsers).msg( "Too many users!");

This is quite a neat solution, because it makes the code more self-documenting and also makes the error message more meaningful.

The msg() member function does not change the level of the assertion. The level() member function sets the level while you can also provide an optional message. Helper functions are provided in order to change the level to warn, debug, error, fatal and eventually set a message at the same time. Take a look:

//  using level()
SMART_ASSERT( nUsers < 1000)(nUsers) .level( lvl_debug, "Too many users!");
SMART_ASSERT( nUsers < 1000)(nUsers) .level( lvl_error, "Too many users!");
// using helpers
SMART_ASSERT( nUsers <= 900)(nUsers) .warn( "Users aproaching max!");
SMART_ASSERT( nUsers < 1000)(nUsers) .debug( "Too many users!");
SMART_ASSERT( nUsers < 1000)(nUsers) .error( "Too many users!");
SMART_ASSERT( nUsers < 1000)(nUsers) .fatal( "Too many users!");

Ignorance Is Bliss

In case you're dealing with someone else's code (which sometimes cannot be altered), and an assertion fails repeatedly, you'll be happy to know there's an "Ignore Always" option. "Ignore Always" works as advertised in [1], but under the hood it uses a different implementation: it remembers the file and line of all failed assertions that were answered with "Ignore Always."

This mechanism makes the assumption that you won't have two SMART_ASSERTs on the same line. The advantage is that interesting levels of granularity - such as "ignore all assertions in file xyz.cpp" - are now possible.

"Ignore All" is useful for non-programmers using your app. In case many assertions fail in a row, the user will prefer using the "Ignore All" option. Also, testers can use "Ignore All" to speed up testing because they know assertions will be logged.

You can define your own handling strategies. For example, John's own complete implementation [3] defines a two-way persistence mechanism that makes "Ignore Always" and "Ignore All" work throughout consecutive runs of your program, which is quite neat.

What About the Release Mode?

The traditional way of using ASSERT is as a debugging tool. The strength of ASSERTs stems from their no-cost promise -- they are not present in the released build (in release mode, it's like they've never been there -- no overhead whatsoever).

However, this is not always the best approach. Sometimes the customer wants a release version of your program (due to efficiency issues). In release mode, all asserts are gone, so bugs are very difficult to track. In the early stages of development, you'll want to keep assertions in release as well. Heck, even Windows NT had a Debug version to make it easier for programmers.

You should also note that in debug mode, it is not necessarily the ASSERTs that bring down the speed of the program, but rather the compiler flags instructing it not to optimize the code. So, keeping ASSERTs in release (optimized) mode might not slow it down much in many cases.

Using SMART_ASSERT, it is very easy to turn on/off SMART_ASSERTs by using the SMART_ASSERT_DEBUG_MODE macro. In case you don't define it, defaults are: SMART_ASSERTs are present in debug mode, and gone in release mode.

If you choose to define it, here is what you need to do:

#define SMART_ASSERT_DEBUG_MODE 0 // SMART_ASSERTs are off (gone)
#define SMART_ASSERT_DEBUG_MODE 1 // SMART_ASSERTs are on (present)

Finally, there are some asserts that you will want present in release mode, even though they might incur some (small) overhead. These are usually the most critical parts of your code. SMART_VERIFY acts like SMART_ASSERT, with two differences: - SMART_VERIFY works in both debug and release modes - SMART_VERIFY's default level is error (lvl_error), while SMART_ASSERT's default level is debug (lvl_debug). This is expectable, since if a SMART_VERIFY fails, it's most likely that the program could crash if it continues on its normal path (therefore, an exception will be thrown). Here's an example:

Widget *p = get_nth_widget( n);
SMART_VERIFY( p) (n).msg( "Widget does not exist");
// if p were null, and we reached this point, the program would most likely crash.
p->print_widget_info();

Conclusion

John's work constructs a full-featured, industrial-strength assertion facility. It remains as true as ever that using assertions is a key ingredient of successful programs (and programmers, for that matter). Now things just got better with a tool that makes it easy for you to define, use, and analyze invariants for your applications. Happy asserting!

Acknowledgements

Many thanks are due to Pavel Vozenilek, who has encouraged John from the beginning to write this library. Thanks to Paul Mensonides for providing the code that allows "chaining" macros. Also, the Boost community deserves credit for testing John's code and giving a lot of positive feedback.

Bibliography and Notes

[1] Andrei Alexandrescu. "Assertions."

[2] Andrei Alexandrescu and Petru Marginean. "Enforcements."

[3] http://www.torjo.com/smart_assert.zip

About the Authors

Andrei Alexandrescu is a Ph.D. student at University of Washington in Seattle, and author of the acclaimed book Modern C++ Design. He may be contacted at www.moderncppdesign.com. Andrei is also one of the featured instructors of The C++ Seminar ().

John Torjo is a freelancer and C++ consultant. He loves C++, generic programming, and streams. He also enjoys writing articles, and can be reached at [email protected].