See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/221108733

Debugging C++ template metaprograms

Conference Paper · January 2006

DOI: 10.1145/1173706.1173746 · Source: DBLP

CITATIONS READS
24 2,084

3 authors:

Zoltán Porkoláb József Mihalicza

Eötvös Loránd University Eötvös Loránd University
103 PUBLICATIONS 280 CITATIONS 10 PUBLICATIONS 43 CITATIONS

SEE PROFILE SEE PROFILE

Ádám Sipos
University of Miskolc
9 PUBLICATIONS 54 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Non-intrusive Testing View project

CodeCompass View project

All content following this page was uploaded by József Mihalicza on 29 May 2014.

The user has requested enhancement of the downloaded file.

 Debugging C++ Template Metaprograms

Zoltán Porkoláb József Mihalicza Ádám Sipos

Eötvös Loránd University, Eötvös Loránd University, Eötvös Loránd University,
Dept. of Programming Languages Dept. of Programming Languages Dept. of Programming Languages
and Compilers and Compilers and Compilers
H-1117 Pázmány Péter sétány 1/C H-1117 Pázmány Péter sétány 1/C H-1117 Pázmány Péter sétány 1/C
Budapest, Hungary Budapest, Hungary Budapest, Hungary
[email protected] [email protected] [email protected]

Abstract Strongly-typed programming languages, good programming con-

Template metaprogramming is an emerging new direction in C++ ventions, and many other techniques can help avoid such frustrating
programming for executing algorithms in compilation time. De- situations. Nevertheless in many cases we have made some error(s),
spite all of its already proven benefits and numerous successful ap- and we have to fix it. For this we have to recognise that the bug ex-
plications, it is yet to be accepted in industrial projects. One reason ists, isolate its nature, and find the place of the error to apply the
is the lack of professional software tools supporting the develop- fix. This procedure is called debugging.
ment of template metaprograms. A strong analogue exists between Debuggers are software tools to help the debugging process.
traditional runtime programs and compile-time metaprograms. This The main objective of a debugger is to help us understand the hid-
connection presents the possibility for creating development tools den sequence of events that led to the error. In most cases this
similar to those already used when writing runtime programs. This means following the program’s control flow, retrieving information
paper introduces Templight, a debugging framework that reveals on memory locations, and showing the execution context. Debug-
the steps executed by the compiler during the compilation of C++ gers also offer advanced functionality to improve efficiency of the
programs with templates. Templight’s features include following debugging process. These include stopping the execution on a cer-
the instantiation chain, setting breakpoints, and inspecting metapro- tain breakpoint, continuing the running step by step, step into, step
gram information. This framework aims to take a step forward to out, or step over functions, etc. Still, debugging can be one of the
help template metaprogramming become more accepted in the soft- most difficult and frustrating tasks for a programmer.
ware industry. The more complex the language environment we work in,
the more sophisticated the debugging toolset we need. In object-
Categories and Subject Descriptors D.3.2 [Programming Lan- oriented languages like C++, a good debugger has to explore the
guages]: Language Classification – C++; D.2.5 [Testing and De- attributes of objects, handle virtual function calls, explore overload-
bugging]: Tracing ing situations. Debuggers available for object-oriented languages
successfully tackle these challenges.
General Terms Languages Templates are also key language elements for the C++ pro-
Keywords C++, template metaprogramming, debugging gramming language [16]. They are essential for capturing com-
monalities of abstractions without performance penalties in run-
time [18]. Generic programming [14], a recently emerged pro-
1. Introduction gramming paradigm for writing highly reusable components – usu-
Programming is a human activity to understand a problem, make ally libraries – is implemented in C++ with heavy use of tem-
design decisions, and express our intentions for the computer. In plates [20]. The Standard Template Library – the most notable
most cases the last step is writing code in a certain programming example – is now an unavoidable part of most professional C++
language. The compiler then tries to interpret the source code programs [8]. Another important application field of templates is
through lexical, syntactic, and semantic analysis. In case the source the C++ template metaprogramming. Template metaprogramming
code is syntactically correct, the compiler takes further steps to is a technique in which template definitions are used to control
generate runnable code. the compilation process so that during the process of compiling
However, in numerous cases the code accepted by the compiler a (meta)program is executed [21]. The output of this process is still
will not work as we had expected, and intended. The causes vary checked by the compiler and run as an ordinary program.
from simple typos – that (unfortunately) do not affect the syntax – Template metaprograming are proved to be a Turing-complete
to serious design problems. There are various methods to decrease sublanguage of C++ [4]. We write metaprograms for various pur-
the possibility of writing software diverging from its specification. poses. Expression templates [22] replace runtime computations
with compile-time activities to enhance runtime performance.
Static interface checking increases the ability of the compile-time
to check the requirements against template parameters, i.e. they
Permission to make digital or hard copies of all or part of this work for personal or form constraints on template parameters [10, 12]. Subtype rela-
classroom use is granted without fee provided that copies are not made or distributed tionships could be controlled with metaprograms too [5, 23].
for profit or commercial advantage and that copies bear this notice and the full citation While generic programming quickly became an essential part of
on the first page. To copy otherwise, to republish, to post on servers or to redistribute
to lists, requires prior specific permission and/or a fee.
C++ programs, template metaprogramming was able to break out
GPCE’06 October 22–26, 2006, Portland, Oregon, USA.
from the academic world only recently. Despite the growing num-
Copyright c 2006 ACM 1-59593-237-2/06/0010. . . $5.00. ber of positive examples, developers are still wary of using template
metaprogramming in strict, time-restricted software projects. One template <int N>
reason beside the necessity of highly trained C++ professionals is class Factorial
the lack of supporting software tools for template metaprogram- {
ming. public:
In this article we describe Templight, a debugging framework enum { value = N*Factorial<N-1>::value };
for C++ template metaprograms. The framework reads a C++ };
source and modifies it to emit well-formed messages during com- template<>
pilation. Information gained this way under the ”running” of a class Factorial<1>
metaprogram is used to reconstruct the compilation process. A {
front-end tool enables the user to set breakpoints, gives a step- public:
by-step reconstruction of the instantiation flow of templates, and enum { value = 1 };
provides various other features. };
In Section 2 we give an overview of C++ template metapro- int main ()
gramming compared to runtime programming. The ontology of {
metaprogram errors is presented in Section 3. In Section 4 we dis- const int r = Factorial<5>::value;
cuss the possibility of debugging template metaprograms. Our de- }
bugging framework is described in detail in Section 5 with demon-
strative examples. Limitations and future directions are discussed Conditional statements (and stopping recursion) are solved via
in Section 6. specialisations. Templates can be overloaded and the compiler has
to choose the narrowest applicable template to instantiate. Subpro-
grams in ordinary C++ programs can be used as data via function
2. C++ Template metaprograms pointers or functor classes. Metaprograms are first class citizens in
It is hard to draw the boundary between an ordinary program template metaprograms, as they can be passed as parameters for
using templates and a template metaprogram. We will use the other metaprograms [4].
notion template metaprogram for the collection of templates, their
instantiations, and specialisations, whose purpose is to carry out // accumulate(n,f) := f(0) + f(1) + ... + f(n)
operations in compile-time. Their expected behaviour might be template <int n, template<int> class F>
emitting messages or generating special constructs for the runtime struct Accumulate
execution. Henceforth we will call a runtime program any kind of {
runnable code, including those which are the results of template enum { RET=Accumulate<n-1,F>::RET+F<n>::RET };
metaprograms. };
There is yet another kind of (pre)compile-time programming: struct Accumulate<0,F>
preprocessor metaprogramming ([27]). Later we show that our de- {
bug method works properly together with preprocessor metapro- enum { RET = F<0>::RET };
gramming techniques. };
Executing programs in either way means executing pre-defined template <int n>
actions on certain entities. It is useful to compare those actions struct Square
and entities between runtime and metaprograms. The following {
table describes the parallels between metaprograms’, and runtime enum { RET = n*n };
programs’ entities. };
int main()
{
const int r = Accumulate<3,Square>::RET;
Metaprogram Runtime program
}
(template) class subprogram (function, procedure)
static const and data Data is expressed in runtime programs as constant values or lit-
enum class members (constant, literal) erals. In metaprograms we use static const and enumeration values
symbolic names variable to store quantitative information. Results of computations under the
(typenames, typedefs) execution of a metaprogram are stored either in new constants or
recursive templates, abstract data structures enumerations. Furthermore, execution of metaprograms can cause
typelist new types be created. Types hold information that can influence the
static const initialisation initialisation further run of the metaprogram.
enum definition (but no assignment!) Complex data structures are also available for metaprograms.
type inference Recursive templates are able to store information in various forms,
most frequently as tree structures, or sequences. Tree structures
Table 1. Comparison of runtime and metaprograms
are the favourite implementation form of expression templates
[22]. The canonical examples for sequential data structures are
typelist [2] and the elements of the boost::mpl library [26].
There is a clear relationship between a number of entities. C++ However, there is a fundamental difference between runtime
template metaprogram actions are defined in the form of template programs and C++ template metaprograms: once a certain entity
definitions and are ”executed” when the compiler instantiates them. (constant, enumeration value, type) has been defined, it will be im-
Templates can refer to other templates, therefore their instantiation mutable. There is no way to change its value or meaning. There-
can instruct the compiler to execute other instantiations. This way fore no such thing as a metaprogram assignment exists. In this
we get an instantiation chain very similar to a call stack of a runtime sense metaprograms are similar to pure functional programming
program. Recursive instantiations are not only possible but regular languages, where referential transparency is obtained. That is the
in template metaprograms to model loops. reason why we use recursion and specialisation to implement loops:
we are not able to change the value of any loop variable. Immutabil- Let us further examine the Factorial metaprogram described
ity – as in functional languages – has a positive effect too: unwanted in Section 2, and let us suppose that the template specialisation
side effects do not occur. Factorial<1> has a syntactic error: a semicolon is missing at the
end of the class definition.
3. Ontology of template metaprogram errors template <int N>
The first examples of C++ template metaprograms were written by class Factorial
Erwin Unruh and circulated among members of the ANSI/ISO C++ {
standardisation committee [19]. These programs did not have to be public:
run, their purpose was to generate warnings or error messages when enum { value = N*Factorial<N-1>::value };
being compiled. The most famous example computed the prime };
numbers, and produced the following output as error messages: template<>
class Factorial<1>
conversion from enum to D<2> requested in Prime {
conversion from enum to D<3> requested in Prime public:
conversion from enum to D<5> requested in Prime enum { value = 1 };
conversion from enum to D<7> requested in Prime } // ; missing
conversion from enum to D<11> requested in Prime
conversion from enum to D<13> requested in Prime This is an ill-formed template metaprogram, with diagnostic
conversion from enum to D<17> requested in Prime message. The metaprogram has not been run: no template instan-
conversion from enum to D<19> requested in Prime tiation happened. Another ill-formed template metaprogram with
diagnostic message is shown in the next example. However, it
This is an erroneous C++ program in the sense of ”traditional”
starts to ”run”, i.e. the compiler starts to instantiate the Factorial
programming, since the compiler emits error messages, but as a
classes, but the metaprogram aborts (in compilation time) [15].
template metaprogram it does exactly the right thing. These error
messages were intentional and produced the expected output. In template <int N>
fact, the lack of error messages would denote an error in the prime class Factorial
generator template metaprogram. However, generating other mes- {
sages than primes would be considered erroneous too. public:
This example points out the difference of the notions correct enum { value = N*Factorial<N-1>::value };
and erroneous behaviour between traditional runtime programs };
and template metaprograms. The C++ International Standard de- template<>
fines the notion well-formed and ill-formed programs [3]. A pro- class Factorial<1>
gram is well-formed if it was constructed according to the syntax {
rules, diagnosable semantic rules, and the One Definition Rule. It // public: missing
is ill-formed, if it is not well-formed. In case any ddiagnosable re- enum { value = 1 };
quirements are broken, the compiler ”shall issue a diagnostic mes- };
sage”. However, no such requirements for other forms of ill-formed int main ()
programs exist. Moreover, ”whenever this International Standard {
places a requirement on the execution of a program ... and the data const int f = Fibonacci<4>::value;
encountered during execution do not meet that requirement, the be- const int r = Factorial<5>::value;
havior of the program is undefined and ... places no requirements }
at all on the behavior of the program”. Even well-formed programs
may be a cause of problems if they exceed resource limits. As the full specialisation for Factorial<1> is written in form
The following code demonstrates an ill-formed program, with of a class, the default visibility rule for a class is private.
diagnostic message, since variable i is undefined. This leads to a Thus enum { value=1 } is a private member, so we receive a
compile-time error. The program does not start to run. compile-time error when the compiler tries to acquire the value of
Factorial<1>::value, while Factorial<2> is being instanti-
#include <iostream> ated. The template metaprogram is ill-formed, with no diagnostic
int main () message. However, if this code fragment is part of a bigger program
{ which has a well-formed Fibonacci template, the Fibonacci part
for (i=0; i!=4; ++i) of the metaprogram would be successfully executed before abor-
std::cout << i << std::endl; tion.
} Let us now erase the full specialisation.
The next example compiles but fails to stop running. template <int N>
#include <iostream> struct Factorial
int main () {
{ enum { value = N*Factorial<N-1>::value };
for (int i=0; ; ++i) };
std::cout << i << std::endl;
} // specialisation for N==1 is missing

This example demonstrates an ill-formed program without di- int main ()

agnostic message. It does compile, but implements an endless for {
loop. The cause of the error is the missing loop condition, and this const int r = Factorial<5>::value;
is the bug we will need to find using some debugging method. }
 As the Factorial template has no explicit specialisation, STATIC_ASSERT(sizeof(int)==sizeof(long),
the Factorial<N-1> expression will trigger the instantiations SIZEOF_INT_NOT_EQUAL_TO_SIZEOF_LONG)
of Factorial<1> followed by Factorial<0>, Factorial<-1>
etc. We have written a compile-time infinite recursion. This is an Static assert is a widely used technique for C++ programs using
ill-formed template metaprogram with no diagnostic message. templates [2, 25].
In fact different compilers might react differently to this code.
g++ 3.4 halts the compilation process after the 17 levels of im- 4.2 Concept checking
plicit instantiations is reached, as defined by the C++ standard. One of the most specific characteristics of C++ templates is that
The compiler for MSVC 6 runs until its resources are exhausted we are not able to explicitly specify the requirements for a type
(reached Factorial<-1308> in our test). MSVC 7.1 halted the used as a template argument. In other words: C++ templates are not
compilation reaching a certain recursion depth. The error message constrained. This is an intentional design decision of the language
received was fatal error C1202: recursive type or function depen- [17]. However, there are situations when this feature might cause
dency context too complex. unwanted behaviour of template metaprograms.
Another possible source of error in template metaprogramming The possibilities for specifying requirements for template argu-
world is the exhaustion of the compiler’s resources. Let us suppose ments are researched extensively, and the results are implemented
that we want to calculate Factorial<125>::value somewhere in in libraries like boost::concept [25], or Alexandrescu’s Loki
the program, and that we are using the original flawless Factorial [2]. A template introspection library was proposed in [24].
metaprogram. In the case of a standard-compliant compiler it will Major efforts are underway to extend the C++ language with
still result in a compile-time error, as the compiler will exceed the the explicit language support for constrained generics. Concepts (a
permitted 17 levels of implicit instantiation. However, some com- type system for templates) can reduce the possibility of misusing
pilers, like g++ can be parameterised to accept deeper instanti- templates and enhance better modularity. Early discussions on con-
ation levels. In this case the compiler continues the instantiation cept checking can be found in [10] and [12]. Currently there are
risking that the resources will be exhausted. In that unfortunate sit- two proposals for constrained generics for C++. The proposal of
uation the compiler will crash. Stroustrup and Dos Reis [18] draws motivation from generic pro-
gramming. Another proposal originated from Siek et. al. [13] is
4. Metaprogram debugging based on earlier comparative studies on generic language features
There are certain techniques to reduce the possibility of writing of various programming languages [6]. An experimental compiler
malfunctioning template metaprograms. derivate – ConceptGCC – already supports the latest proposal.
Both proposals are targeting key weaknesses of the C++ tem-
4.1 Static assert plate facilities. Current template instantiation techniques combine
The tool described is a technique used in the case of runtime pro- information from both definition and instantiation context. This re-
grams, called assertation. An assert is a function (usually imple- duces the possible separation of template definitions from its user
mented as a macro) that requires a logic expression as parameter. context. Therefore C++ template definitions are harder (if not im-
If the condition is false, the program is terminated, and an infor- possible) to modularise and check deeply compared to other lan-
mative message is printed to the screen. If an invariant is not met guages like ADA [9].
at a certain point, the program stops immediately, before any other
4.3 Objectives for a metaprogram debugger
(also incorrect) command could modify the variables in a way that
renders the invariant true again. Thus we avoid overlooking a logic As we have seen in Section 3, compiler diagnostics are often use-
error in the program. less when aiming to detect the place and cause of the unwanted be-
On the other hand, a static assert is one of the possible ways to haviour of C++ template metaprograms. In these situations we need
handle a metaprogram situation, when a compile-time requirement to debug our code with different methods. These methods have to
is not met by a type or a value. A static assert is capable of halting include the following major components:
the compilation of a program at the point of the error’s detection,
thus we can avoid an incorrect program to come into being. At the • We have to follow the ”control flow” of the program. In case of
same time, we aspire to create a static assert that contains some template metaprograms, this is equivalent to following the chain
sensible error message, thus it is easier for the programmer to find of the template instantiations. We should give special attention
the bug. to recognising recursive instantiations.
The simplest way to do this is by using a macro as defined in • We need to inform the programmer of the ”variables’” values
[28, 7], whose simplified version is as follows: (i.e. the arguments and static const values of templates being
template <bool> struct STATIC_ASSERT_FAILURE; instantiated). The debugger has to track the values of variables,
template<> struct STATIC_ASSERT_FAILURE<true>{}; and these values have to be made available to the programmer.
template<int x> struct static_assert_test{}; • To increase the efficiency of debugging the user should be
able to set breakpoints – meaning suspending the compilation
#define STATIC_ASSERT( B, error) \ process when templates with certain arguments are instantiated
typedef static_assert_test< \ • The user should be able to control the debugging process to step
sizeof(STATIC_ASSERT_FAILURE<(bool)(B),error>)>\ into, step over, or step out of instantiation events.
static_assert_typedef_;
If the expression B is true, the existing specialisation of Tools we use in everyday programming for debugging are not
STATIC ASSERTION FAILURE is used as the sizeof’s argument. available in the well–known way, when dealing with metapro-
Otherwise the missing specialisation for false causes a compile- grams. We have no command for printing to the screen (in fact
time error. In the error argument a typename has to be provided we have practically no commands at all), we have no framework
that passes messages for the programmer: to manage running code. On the other hand, we still have some
options. Having a set of good debugging tools in the runtime world
struct SIZEOF_INT_NOT_EQUAL_TO_SIZEOF_LONG {}; and a strong analogue between the runtime and compile-time realm
we can attempt to implement a template metaprogram debugging from #line directives into a separate mapping file and delete #line
framework. directives.
A common property of debugging tools is that they analyse a At this point we have a single preprocessed C++ source file,
specific execution of our program. In most cases this execution that we transform into a C++ token sequence. To make our frame-
does not depend on the usage of the tool1 . In such cases it does work as portable and self-containing as possible we apply the
not matter whether we are using the tool on the running program or boost::wave C++ parser. Note that even though boost::wave
on a previously generated trace of its runtime steps. supports preprocessing, we still use the original preprocessor tool
One of the possible solutions is the modification of the C++ of the compilation environment to eliminate the chance of bugs
compiler to serve the information required for debugging purposes. occurring due to different tools being used. Our aim is to insert
With the strong help of compiler vendors, this approach has many warning-generating code fragments at the instrumentation points.
benefits. However, to identify the exact specification for such a As wave does no semantic analysis we can only recognise these
major step earlier experiences are needed. Such results could be places by searching for specific token patterns. We go through the
obtained from more portable, lightweight solutions with the co- token sequence and look for patterns like template keyword + ar-
operation of standard C++ language elements and external tools. bitrary tokens + class or struct keyword + arbitrary tokens + { to
Applying the analogue we can say that if we had a trace file identify template definitions. The end of a template class or func-
that contains all template instantiations, the appropriate template tion is only a } token that can appear in quite many contexts, so
parameters, the inner typedef definitions, timestamps for a pro- we should track all { and } tokens in order to correctly determine
filer etc. step by step along the compilation process then we could where the template contexts actually end. This pattern matching
apply the tools to analyse the behaviour of our template metapro- step is called annotating, its output is an XML file containing an-
grams. notation entries in a hierarchical structure following the scope.
All we need is a utility that generates a detailed trace file about The instrumentation takes this annotation and the single source
the compilation process. and inserts the warning-generating code fragments for each anno-
tation at its corresponding location in the source thus producing a
source that emits warnings at each annotation point during its com-
5. Implementation pilation. The next step is the execution of the compiler to have these
Most compilers generate additional information for debuggers and warning messages generated. The inserted code fragments are in-
profilers. Obviously the simplest way for providing information for tentionally designed to generate warnings that contain enough in-
our debugging framework would be the implementation of another formation about the context and details of the actual event. Since
compiler setting that makes the framework generate the desired the compiler may produce output independently of our instrumen-
trace file. However, an immediate and more portable solution is tation, it is important for debugger warnings to have a distinct for-
to use external tools cooperating with standard C++ language el- mat that differentiates them. This is the step where we ask the com-
ements. An appropriate compiler support could be an ideal long- piler for valuable information from its internals. Here the result is
term solution. simply the build output as a text file. The warning translator takes
Without the modification of the compiler the only way of ob- the build output, looks for the warnings with the aforementioned
taining any information during compilation is generating informa- special format and generates an event sequence with all the de-
tive warning messages that contain the details we are looking for tails. The result is an XML file that lists the events that occurred
[1]. Therefore the task is the instrumentation of the source, i.e. during the compilation in chronological order. The annotations and
its transformation into a functionally equivalent modified form that the events can be paired. Each event signals that the compiler went
triggers the compiler to emit talkative warning messages. The con- through the corresponding annotation point. We can say events are
cept of such instrumentation is a usual idea in the field of debug- actual occurrences of the annotation points in the compilation pro-
gers, profilers and program slicers [11]. Everytime the compiler in- cess.
stantiates a template, defines an inner type etc. the inserted code
fragments generate detailed information on the actual template- 5.2 Preprocessing
related event. Then we have to gather the desired information from The C++ preprocessor is executed in order to instrument all tem-
the corresponding warning messages in the compilation output and plates that appear in the given compilation unit. It is important to
form a trace file. The front-end system uses this trace file to imple- enable the #line directives since they serve the mapping between
ment its various debugging features. the original source file positions and the positions in the single pre-
processed file. By using the same preprocessor as in the normal
5.1 Overview compilation process all possible preprocessor tricks done in the
The input of the process is a C++ source file and the output is a source are processed exactly the same way, and there are no com-
trace file, a list of events like instantiation of template X began, patibility issues even if the code contains preprocessor metapro-
instantiation of template X ended, typedef definition found etc. gramming or any other compiler-dependent technique.
The procedure begins with the execution of the preprocessor with
exactly the same options as if we were to compile the program. As 5.3 #line filtering
a result we acquire a single file, containing all #included template Simple filters are executed that eliminate the #line directives from
definitions and the original code fragment we are debugging. The the preprocessed source. At the same time they generate a map
preprocessor decorates its output with #line directives to mark file that later can be used to retrieve the original positions with the
where the different sections of the file come from. This information position adjuster.
is essential for a precise jump to the original source file positions
as we step through the compilation while debugging in an IDE for 5.4 Annotating
example. To simplify the process we handle the mapping of the The annotator parses the C++ source (using boost::wave) and
locations in the single processed file to the original source files in a looks for special token sequences like
separate thread. Simple filter scripts move the location information
• beginning of scope
1 this immediately leads to a limitation described in Section 6 • end of scope
 original source
templateFunctionBegin =
wave::T_TEMPLATE >>
preprocessor
+(
preprocessed source
*(anychar_p - (
wave::T_SEMICOLON |
#line filter file and line mapping wave::T_LEFTBRACE |
wave::T_RIGHTPAREN
unaltered preprocessed source )) >>
without #line directives wave::T_RIGHTPAREN
);
annotator
templateFunctionDeclaration =
annotation
templateFunctionBegin >> wave::T_SEMICOLON ;
templateFunctionDefinition =
instrumentator line mapping templateFunctionBegin >> wave::T_LEFTBRACE ;

instrumented code typedefDeclaration =

wave::T_TYPEDEF >>
compiler *(anychar_p - (
wave::T_SEMICOLON |
compilation output wave::T_IDENTIFIER
)) >>
warning parser *(wave::T_IDENTIFIER >>
+(anychar_p - (
trace file wave::T_SEMICOLON |
wave::T_IDENTIFIER
position adjust ))
) >>
position correct trace file wave::T_IDENTIFIER >> wave::T_SEMICOLON ;

visualiser The grammar is implemented by boost::spirit rules that oper-

ate on the token sequence of the wave output. The result is an XML
Figure 1. Architecture of debugging framework file that describes the structure of the source file. Consider the fol-
lowing simple example:

• beginning of template class definition (optionally a specialisa-

template<int i>
struct Factorial
tion)
{
• beginning of template function definition (optionally a special- enum { value = Factorial<i-1>::value };
isation) };
• (inner) typedef definition
template<>
We use the following grammar: struct Factorial<1>
openBrace = wave::T_LEFTBRACE ; {
closeBrace = wave::T_RIGHTBRACE ; enum { value = 1 };
};
templateClassBegin = wave::T_TEMPLATE >>
*(anychar_p - ( int main ()
wave::T_SEMICOLON | {
wave::T_LEFTBRACE | return Factorial<5>::value;
wave::T_CLASS | }
wave::T_STRUCT The resulting XML file is the following:
)) >>
+( <?xml version="1.0" standalone="yes"?>
(wave::T_CLASS | wave::T_STRUCT) >> <!-- [annotation] generated by Templight -->
*(anychar_p - ( <FileAnnotation
wave::T_CLASS | beginpos = "test2.cpp.preprocessed.cpp|1|1"
wave::T_STRUCT | endpos = "|1|1">
wave::T_SEMICOLON | <TemplateClassAnnotation
wave::T_LEFTBRACE beginpos = "test2.cpp|1|1"
)) endpos = "test2.cpp|5|2"
) ; afteropenbracepos = "test2.cpp|3|2"
templateClassDeclaration = beforeclosebracepos = "test2.cpp|5|1">
templateClassBegin >> wave::T_SEMICOLON ; <ScopeAnnotation
templateClassDefinition = beginpos = "test2.cpp|4|7"
templateClassBegin >> wave::T_LEFTBRACE ; endpos = "test2.cpp|4|40"/>
 </TemplateClassAnnotation> Templight::ReportTemplateBegin<
<TemplateClassAnnotation _TEMPLIGHT_0s, &_TEMPLIGHT_0s::a
beginpos = "test2.cpp|7|1" >::Value
endpos = "test2.cpp|11|2" };
afteropenbracepos = "test2.cpp|9|2" /* ----------------- end inserted ------------ */
beforeclosebracepos = "test2.cpp|11|1"> enum { value = Factorial<i-1>::value };
<ScopeAnnotation /* ---------------- begin inserted ----------- */
beginpos = "test2.cpp|10|7" struct _TEMPLIGHT_1s { int a; };
endpos = "test2.cpp|10|20"/> enum { _TEMPLIGHT_1 =
</TemplateClassAnnotation> Templight::ReportTemplateEnd<
<ScopeAnnotation _TEMPLIGHT_1s, &_TEMPLIGHT_1s::a
beginpos = "test2.cpp|14|1" >::Value
endpos = "test2.cpp|16|2"/> };
</FileAnnotation> /* ----------------- end inserted ------------ */
};
The outmost FileAnnotation element represents the source file,
while the two TemplateClassAnnotation elements correspond
template<>
to the general Factorial template class definition and its spe-
struct Factorial<1>
cialisation, respectively. The beginpos and endpos attributes
{
show the borders of the definition while afteropenbracepos
/* ---------------- begin inserted ----------- */
and beforeclosebracepos contain the locations where the in-
struct _TEMPLIGHT_2s { int a; };
strumented code fragments should be inserted. Note the presence
enum { _TEMPLIGHT_2 =
of the ScopeAnnotation elements following all ’{’ ... ’}’ pairs
Templight::ReportTemplateBegin<
regardless of their meaning.
_TEMPLIGHT_2s, &_TEMPLIGHT_2s::a
5.5 Instrumenting >::Value
};
The instrumentator takes the annotation file and for each element /* ----------------- end inserted ------------ */
that marks a debug point it inserts a corresponding code frag- enum { value = 1 };
ment that generates a compilation warning containing the desired /* ---------------- begin inserted ----------- */
information. In the current implementation we use a double to struct _TEMPLIGHT_3s { int a; };
unsigned conversion. This construct is proved to emit a warning enum { _TEMPLIGHT_3 =
when compiled by current C++ compilers. However, it is possi- Templight::ReportTemplateEnd<
ble to insert multiple warning generator fragments to cover a wider _TEMPLIGHT_3s, &_TEMPLIGHT_3s::a
range of compilers. At the warning parsing step (in 5.7) the unnec- >::Value
essary warnings are dropped. };
The other output of this step is the line mapping that enables /* ----------------- end inserted ------------ */
the position adjuster to retrieve the original positions from the };
line numbers referring to the instrumented code in the warnings.
Inserted code fragments were defined to be semantically neutral, int main ()
i.e. they must not affect the functionality of the original template {
metaprogram. The result of the previous example’s instrumentation return Factorial<5>::value;
is the following: }
/* ---------------- begin inserted ----------- */ There is a fixed part instrumented to the beginning of the file
namespace Templight { containing the following helper types: ReportTemplateBegin,
template<class C, int C::*> ReportTemplateEnd, ReportTypedef.
struct ReportTemplateBegin { The occurrence of these helper types in a warning message obvi-
static const unsigned Value = -1.0; ously signals that the compiler met an instrumented code fragment.
}; Each instrumentation introduces a new inner class and passes an
template<class C, int C::*> address of its static variable to the helper templates to enforce new
struct ReportTemplateEnd { instantiations each time. Having this parameter omitted we would
static const unsigned Value = -1.0; get the warning only at the first – the only – instance of the helper
}; template. Unfortunately this trick is not always enough to get the
template<class C, int C::*, class Type> desired warnings for each instance (see Section 6).
struct ReportTypedef { typedef instrumentation Some compilers display the typedef
typedef Type Result; identifier rather than the actual type in the warning messages which
static const unsigned Value = -1.0; makes diagnostics more difficult, as described in [1]. To overcome
}; this problem the instrumentation slightly modifies the type of the
} typedef and inserts a forwarding template in order to have the
/* ----------------- end inserted ------------ */ actual type displayed. This forwarding type is ReportTypedef.
template<int i> The following example shows a typedef instrumentation. The
struct Factorial original source is the following:
{
/* ---------------- begin inserted ----------- */ template<int N>
struct _TEMPLIGHT_0s { int a; }; struct Test
enum { _TEMPLIGHT_0 = {
 typedef Int<N> IntType; 5.7 Parsing warnings
}; The warning parser takes the compilation output and looks for our
And the corresponding instrumentation: special warning messages and collects the encoded information
as well as the position and instantiation history of the event. The
template<int N> result is an XML file containing a sequence of events, where all
struct Test events correspond to an element in the annotation file, set up with
{ the actual properties of activation. The resulting event entry of the
// { ... ReportTemplateBegin part ... } warning is the following:

struct _TEMPLIGHT_2s { int a; }; <TemplateBegin>

typedef typename <Position position=
Templight::ReportTypedef< "test2.cpp.patched.cpp|9|1"/>
_TEMPLIGHT_2s, &_TEMPLIGHT_2s::a, <Context context="Factorial<4>"/>
Int<N> <History>
>::Result IntType; <TemplateContext instance="Templight::
}; ReportTemplateBegin<C,__formal>">
<Parameter name="C" value=
// { ... ReportTemplateEnd part ... } "Factorial<4>::_TEMPLIGHT_0s"/>
}; <Parameter name="__formal"
value="pointer-to-member(0x0)"/>
5.6 Compiling </TemplateContext>
<TemplateContext
The C++ compiler is executed on the instrumented source file to get
instance="Factorial<i>">
the talkative error messages. If the original source is ill-formed, the
<Parameter name="i" value="4"/>
compiler may emit compilation errors and warnings independent of
</TemplateContext>
our framework. These messages are preserved since they are also
</History>
converted to events by the warning parser. The compilation may
</TemplateBegin>
stop on serious errors caused by the original source. In this case
the trace contains all the events up to that point. This way it is
The name of the tag describes that this event is the beginning of
still possible to debug the compilation and follow the control up to
a template instantiation, while the child elements contain additional
the point our compilation aborted. This situation is similar to the
details. The Position tag shows the location of the annotation point
crash of an ordinary program, where one can debug the behaviour
in the source, the Context tag shows the actual template context,
to the point of the problematic instruction. Sometimes the running
and the History tag lists the template instantiation backtrace that
program even survives several illegal instructions, exactly the same
was present in the warning.
way as the compiler does not stop at the first error. A fragment of
the previous example’s compilation output:
5.8 Adjusting positions, visualisation
test2.cpp.patched.cpp(1) : warning C4244: The position adjuster corrects the file positions in the trace file
’initializing’ : conversion from ’double’ to using the previously saved line mappings of the #line filter and
’const unsigned int’, possible loss of data instrumentator steps. Here we have a full trace of all the events that
test2.cpp.patched.cpp(9) : see reference to happened during the compilation with exact source file positions.
class template instantiation This is an ideal input for post-processing tools.
’Templight::ReportTemplateBegin<C,__formal>’ A debugger frontend maintains breakpoint positions set by the
being compiled user. Traversing the trace the frontend stops at each breakpoint
with and allows the user to examine the instantiation stack of templates
[ including template names and arguments at each level. Then she
C=Factorial<4>::_TEMPLIGHT_0s, can advance to the next breakpoint or take only one step. In the
__formal=pointer-to-member(0x0) second case she can step into the next template instantiation inside
] the actual one, step over, i.e. skip nested instantiations, or step
test2.cpp.patched.cpp(10) : see reference to out skipping all instantiations until reaching the context where the
class template instantiation actual template was instantiated. Thus the programmer is able to
’Factorial<i>’ being compiled focus on desired portions of the compilation process and can reduce
with debugging effort.
[ A profiler frontend may display the template instantiations
i=4 sorted by their compile time. The compile times can mean only
] local times, where the times of nested instantiations are subtracted,
or full times meaning the full time spent inside the given instantia-
Here we can see the artificially generated warning message that
tions.
comes from our Templight::ReportTemplateBegin template
class showing that it is not a warning of the original compilation,
but an instrumented one. In order to catch the original context an in- 5.9 Remarks
ner class is passed to this template class. As our previous compiler The framework was intentionally designed to be as portable as pos-
output shows it is recognisable that the warning is originated from sible, for this end we tried to use portable and standard-compliant
the Factorial<4> template instance. In this example the compiler tools. All components except for the #line filter scripts are writ-
also helped us by reporting the context, but this backtrace is not al- ten in C++ using the STL, boost and Xerces libraries. The only
ways present. compiler-dependent step in our framework is the warning parser.
6. Limitations and future works not generate any context information. In contrast to the others this
No intervention Note that while in traditional debugging the pro- compiler prints the same warning for each instantiation.
gram is actually running when being debugged, in our case we only Since we do not have semantic information we fall back on us-
traverse the previously generated trace file. As a result we are un- ing mere syntactic patterns. Unfortunately without semantic infor-
able to modify the values of the variables at a given point and see mation there are ambiguous cases where it is impossible to de-
what happens if the variable had that value at that point. A real in- termine the exact role of the tokens. This simply comes from the
teractive template metaprogram debugger can only be implemented environment-dependent nature of the language and from the heav-
by strong intentional support of compilers. On the other hand we ily overloaded symbols. The following line for example can have
can see the whole process and can arbitrarily go back and forth on totally different semantics depending on its environment:
the timeline, which is impossible in immediate debugging. enum { a = b < c > :: d };
Compiler support The process detailed above works only if the If the preceding line is
compiler gives enough information when it meets the instrumented
erroneous code. Unfortunately not all compilers fulfil this criterion enum { b = 1, c = 2, d = 3 };
today. The table below summarises our experiences with some then the < and > tokens are relational operators, and :: stands
compilers: for ’global scope’, while having the following part instead of the
previous line

compiler result template<int>

struct b {
g++ 3.3.5 ok enum { d = 3 };
g++ 4.1.0 ok };
MSVC 7.1 ok enum { c = 2 };
MSVC 8.0 ok
Intel 9.0 no instantiation backtrace the < and > tokens become template parameter list parentheses and
Comeau 4.3.3 no instantiation backtrace :: the dependent name operator. This renders recognising enum
Metrowerks no instantiation backtrace definitions more difficult.
CodeWarrior 9.0 Future works include the implementation of new IDE exten-
Borland 5.6 no warning message at all sions providing better functionality for the debugger frontend. This
Borland 5.8 no instantiation backtrace, includes the placing and removing of breakpoints, following the
but the warning message is printed compilation process, and examination of the instantiation stack.
for each instantiation Another direction is to create a real, interactive debugger, that
can suspend the compilation process at breakpoints, collect infor-
Table 2. Our experiences with different compilers mation, and is even able to modify the compilation process. As far
as we know this can be done only by the modification of the com-
piler. However, the most adequate modifications and the protocol
It is a frequent case when a warning is emitted, but there is no in- describing how the compiler and the outer tools interact are cur-
formation about its context. Comeau 4.3.3., Intel 9.0, and Metrow- rently open.
erks CodeWarrior 9.0 print the double to unsigned warning only
once in the ReportTemplateBegin helper template without refer- 7. Related work
ring to the instantiation environment.
The example shown in subsection 5.5 produces the following Template metaprogramming was first investigated in Veldhuizen’s
output using the Comeau C++ online evaluation compiler: articles [21]. Static interface checking was introduced by Mc-
Namara [10] and Siek [12]. Compile-time assertion appeared in
"ComeauTest.c", line 5: error: Alexandrescu’s work [2]. Vandevoorde and Josuttis introduced the
expression must have integral or enum type concept of a tracer, which is a specially designed class that emits
static const unsigned Value = -1.0; runtime messages when its operations are called [20]. When this
^ type is passed to a template as an argument, the messages show
"ComeauTest.c", line 9: error: in what order and how often the operations of that argument class
expression must have integral or enum type are called. The authors also defined the notion of an archetype for
static const unsigned Value = -1.0; a class whose sole purpose is checking that the template does not
^ set up undesired requirements on its parameters. In their book on
"ComeauTest.c", line 14: error: boost [1] Abrahams and Gurtovoy devoted a whole section to
expression must have integral or enum type diagnostics, where the authors showed methods for generating tex-
static const unsigned Value = -1.0; tual output in the form of warning messages. They implemented
^ the compile-time equivalent of the aforementioned runtime tracer
3 errors detected in (mpl::print, see [26]).
the compilation of "ComeauTest.c". Even though in the preprocessing phase the TIME predefined
macro can be used to determine the exact system time, it is impos-
Similar output is produced by the Intel 9.0 compiler. The mes- sible to get such information during the actual compilation step.
sages are only emitted once for each helper template regardless of Therefore profiling information can only be generated by the mod-
the number of their actual instantiations. ification of the compiler. Using the Templight framework we im-
The most surprising find was that the Borland 5.6 compiler does plemented a light-weight template profiler for the g++ compiler.
not print any warnings to our instrumented statement even with all We modified the compiler to output timestamps when emitting the
warnings enabled. A later version of this compiler (version 5.8) aforementioned warning messages. These timestamps are collected
prints the desired messages, but similarly to many others it does by the warning parser and then inserted to the trace file.
 8. Conclusion [11] Krisztián Pócza, Mihály Biczó, Zoltán Porkoláb: Cross-language Pro-
gram Slicing in the .NET Framework. Journal of .NET Technologies
C++ template metaprogramming is a new, evolving programming Vol. 3., Number 1-3, 2005 pp. 141-150.
technique. Even though it extends traditional runtime programming
with numerous advantages, the lack of development tools retains its [12] Jeremy Siek and Andrew Lumsdaine: Concept checking: Binding
applicability in time-constrained, large-scale industrial projects. parametric polymorphism in C++. In First C++ Template Program-
ming Workshop, October 2000, Erfurt.
In this paper we listed the analogues between the realms of
runtime and compile-time programming. We set up an ontology [13] Jeremy Siek and Andrew Lumsdaine: Essential Language Support
of template metaprogram errors and defined the requirements for for Generic Programming. Proceedings of the ACM SIGPLAN 2005
a template metaprogram debugger. We described our prototype conference on Programming language design and implementation,
New York, NY, USA, pp 73-84.
framework that fulfils these requirements and supplies a portable
solution for debugging template metaprograms in practice. We [14] Jeremy Siek: A Language for Generic Programming. PhD thesis,
showed the details of the framework’s operation step by step Indiana University, August 2005.
through examples. [15] Ádám Sipos: Effective Metaprogramming. M.Sc. Thesis. Budapest,
Our framework is also an experimental tool towards discovering 2006.
the requirements of intentional compiler support for debugging and [16] Bjarne Stroustrup: The C++ Programming Language Special Edition.
profiling template metaprograms in the future. Addison-Wesley (2000)
[17] Bjarne Stroustrup: The Design and Evolution of C++. Addison-
References Wesley (1994)
[1] David Abrahams, Aleksey Gurtovoy: C++ template metaprogram- [18] Gabriel Dos Reis, Bjarne Stroustrup: Specifying C++ concepts.
ming, Concepts, Tools, and Techniques from Boost and Beyond. Proceedings of the 33rd ACM SIGPLAN-SIGACT Symposium on
Addison-Wesley, Boston, 2004. Principles of Programming Languages (POPL), 2006: pp. 295-308.
[2] Andrei Alexandrescu: Modern C++ Design: Generic Programming [19] Erwin Unruh: Prime number computation. ANSI X3J16-94-0075/ISO
and Design Patterns Applied. Addison-Wesley (2001) WG21-462.
[3] ANSI/ISO C++ Committee. Programming Languages – C++. [20] David Vandevoorde, Nicolai M. Josuttis: C++ Templates: The
ISO/IEC 14882:1998(E). American National Standards Institute, Complete Guide. Addison-Wesley (2003)
1998.
[21] Todd Veldhuizen: Using C++ Template Metaprograms. C++ Report
[4] Krzysztof Czarnecki, Ulrich W. Eisenecker: Generative Program- vol. 7, no. 4, 1995, pp. 36-43.
ming: Methods, Tools and Applications. Addison-Wesley (2000)
[22] Todd Veldhuizen: Expression Templates. C++ Report vol. 7, no. 5,
[5] Ulrich W. Eisenecker, Frank Blinn and Krzysztof Czarnecki: A 1995, pp. 26-31.
Solution to the Constructor-Problem of Mixin-Based Programming in
[23] István Zólyomi, Zoltán Porkoláb, Tamás Kozsik: An extension to the
C++. In First C++ Template Programming Workshop, October 2000,
subtype relationship in C++. GPCE 2003, LNCS 2830 (2003), pp.
Erfurt.
209 - 227.
[6] Ronald Garcia, Jaakko Järvi, Andrew Lumsdaine, Jeremy Siek,
[24] István Zólyomi, Zoltán Porkoláb: Towards a template introspection
Jeremiah Willcock: A Comparative Study of Language Support
library. LNCS Vol.3286 pp.266-282 2004.
for Generic Programming. Proceedings of the 18th ACM SIGPLAN
OOPSLA 2003, pp. 115-134. [25] Boost Concept checking.
http://www.boost.org/libs/
[7] Björn Karlsson: Beyond the C++ Standard Library, A Introduction to
concept check/concept check.htm
Boost. Addison-Wesley, 2005.
[26] Boost Metaprogramming library.
[8] David R. Musser and Alexander A. Stepanov: Algorithm-oriented
http://www.boost.org/libs/mpl/doc/index.html
Generic Libraries. Software-practice and experience, 27(7) July 1994,
pp. 623-642. [27] Boost Preprocessor library.
http://www.boost.org/libs/
[9] David R. Musser and Alexander A. Stepanov: The Ada Generic
preprocessor/doc/index.html
Library: Linear List Processing Packages. Springer Verlag, New
York, 1989. [28] Boost Static assertion.
http://www.boost.org/regression-logs/
[10] Brian McNamara, Yannis Smaragdakis: Static interfaces in C++. In
cs-win32 metacomm/doc/html/boost staticassert.html
First C++ Template Programming Workshop, October 2000, Erfurt.

View publication stats