1B] A Bo. 8 8) Logging In C++ Logging isa critical echnique for troubleshooting and maintaining software systems. Petru presents a C++ logging framework that is typesafe, thread-safe, and portable, Septamver 05,2007 [URL btwn comicanlogaingsn-i201804215, Logging is eritical technique for troubleshooting and maintaining software systems. Is simple, provides information without requiring knowledge of programming language, and does not require specialized tools. Logging is 2 useful means to figure out if an application is actually doing what itis ‘supposed to do, Good logging mechanisms can save long debugging sessions and dramatically inezease the maintainability of application. In this article, I present a simple—but highly wseful—logging framework that i typesafe, dreadsafe (at line-level, efficient, portable, fine-rained, compact, and flexible. The complete source code, which works with Visual C-+ 7.1, g++ 33.5, and CC 5.3 on Sun and other platforms is available inthe DJ October 2007 zip file at www dij.comsode, In Part2 ofthis article, I enhance this technique by providing more granularity. ‘The First Step Lo’ takea fist stab ata Log class. Listing One uses an sta: :ostringstrean member variable called "os" to accumulate logged dat. The Get() member unction gives acess to that variable. Afterall data is formatted, Logs destructor persists the output tothe standard erro. You use tog class ike this Log().Gee(1ogiNr9) <¢ “Hello © ¢e usernanes Executing this code erates a tog abject with the gin logging level, fetches its std::stringstrean object, formats and accumulates the use-supplied ata, and finaly, persists the resulting string into the log file using exactly one eal to #peinte(). ‘Why flush during destruction? The last two steps are important because they confer threadsafty tothe logging mechanism, The fprint*() funetion is threadsafe, so even if his lg is used from different thread, the output lines won't be strambled, According 10 _gmuorg/softwarelibefmanualhtm_node/Streams-ané-Threads him: Listing One 11 (og, version 0.4: 2 single Logeing class ‘nun TLogLevel (logeRton, loghARMING, 1ogINFO, 1ogDEBUG, LogDEBUGL, Sogbemuc,Tegoiis, ogoeabs); ‘ punate: Lok; virtual ~to30): Suuttoseringstreank Get(Thogtevel Level = LogiMfo); pubite: static Tloglevel& Reportingteve(): protectes: ita: ortringstresn os; private: Loggeonst toga): opt operator {const Loge): private: ThogLevel messageLevel; % Skdscostringstreand Log: :Get(TLogLevel level) « os ce "=," ce Nowtine()s Os ce" * ce Tostringtlavel) << * Os ce std:satring{level > 10gQOS hessageLevel level) 1 Level = 1ogbEBUG, “\t"); 2 > Logs Report ingteven()) AF (oessagecen < print? (stderr, "#5", os.str.estr(O)5 fFlusn(staere)y > ‘The POSIX Standard requires that by default the stream operations are atomic. .;ssuing two steam operations for the same stream in two threads atthe same time wil cause the operations to be executed asi they were ised sequentially. The buffer operations performed while reading or wmiting are protected from other uses ofthe same steam. To do ths, each stream has an intemal lock object that has to be (implicitly) acquized before any work can be done. Before moving on toa more efficient implementation, let's write code to insert tabs in proportion tothe loging level, and append an std::endl to each hunk of text. This makes the log line oriented and easy to read by both humans and machines. Here's the relevant code: Log: Reportingeven() = Logan; tog) Se (LogD=806 "A Loop with "<< court << * iterations"; fon (int 1 28; tt count; 4) Log() cet logoesust) ce “the counter £ =" <6 45 which outputs = 72:25:29.583 Debus: Loop with 3 Seerations 1 Zasgsid3 eae petugis the counter 4 = 8 1D Eaudsiaalers panes: the counter 4 21 Indentation makes the logging more readable, More leading tabs imply « more detsiled level of logging. A Little Trick So far, the framework: has shown good returns on only minor investments: It offers some nice formating rules (the tabs according withthe logging level and final ste: rend in a small, easy-to-use package. However, the current Log has an efficiency problem: Ifthe logging level is st to actually do nothing, Log accumulates the data interally—just to notice later, during destruction, that no output is required! This single issu is big enough to be a showstopper against Logs use in & production system. ‘You can use a litle tick that makes the code, when logging isnot necessary, almost as fast asthe eode with no logan at all. Logging will hive a cost only ift actually produces output; otherwise, the cost is low (and actually mumeasurable in most cases) This ets you contol the trade-off between fast execution and detailed logging. Let's move the check from the destructor tothe earliest possible time, which is just before the constriction ofthe Log object. In this way, if the logging evel says you should discard the logged data, you won't even create the Log abject. ‘define Lostlevel) \ Af (level > Loge :ReportingLevel()) 5 \ fe Log(). cet Cleves) [Now the first example becomes Loctiogiifo) << “Het20 * cc usernanes and is expanded by the preprocessor to (new lines added for clarity); 4 (LogINFo > Log:Reportingteven()) tog) .et(Loginr0) << “Hello ‘Consequently, the og class becomes simpler asthe nassagetevel member and the test inthe destructor are not needed anymore tog: :=toe() fprinet (stderr, "85", os str().<.strO)5 sflusn(staeen > Logging is much more efficient now. You can add logging liberally to your code without having serous efficiency concers. The only thing to remember {is to pass higher (hat is, more detsled) logging levels to code that’s more heavily execute. Ate applying this trick, macro-elated dangers shouldbe avoided-—we should forget tha the logging code might not be executed a all, subject othe logging level in effect. This s what we actually wanted, and is ectually what makes the code efficient, But as always, "sacroriis" can introduce subtle bugs. In this example: Loctiogrroy << ‘he clients willbe notified only if the logging level detail willbe Logineo and lowe. Probably not what was intended! The correct code shouldbe: unter of * cass Log c “ » explate LopcoutputPelscy>::-L2g0) < outputpolicy:output(as. st); ‘Thats pretty much all that need to be done on Log, You can now provide the FILe* output simply as an implementation ofthe OutputPolicy policy: see Listing Two, Listing Two lass output2FILe // implementation of CutputPoitey pubiic Biatie FILE*® streanQs Static void Output{eonst std::stringk mse); FILE*a output2FILes:strean() state FILE* pstrean = stderr; return pStrea ) Inline void utput2FTLE::Output(corst std:strings sa) < Frte® pstreae Af (pstream) streanO: fprincetestrean, "Wa", meg.c.steO): ‘eusncpstrean): {ypeder Logeoutput2FTLe> FILeLog: ‘deine FILE OGCievel) \ Af (level > FuLeuog: ReportingLevel() || lOutput2FILe::strean()) + \ else FILELSg() -Get{messagetevel) “The code below shows how you can change the output fom the default stderr to some specie file (error checking/handling omitted for brevity): FILE* pFile = fopen("2pplication.208*, lutputS Lt: strean() = piles FILELOS(1ogINFO) «<5 ‘A note for multithreaded applications: The ostput2Fst€ policy implementation is good if you don't set the destination ofthe log concurrently, I, onthe ‘other hand, you plan to dynamically change the logging stream at runtime from acbitary threads, you should use appropriate interlocking wsing your platform's threading facilities, or a more pertable wrapper such as toost threads. Listing Three shows how you can doit using doost threads, Lising Three sinetude lass oveputarnte “ public: Static void output(const sta: strings #58): State void SetStrean(FiLe™ pFile); atic FILER Streantapl(); 8 ° inline FILES Cutput2FILE::streantapl()state FILE® pstrean = stderr; return pstre ? Inline vold output2FTLE: :setstrean(FILE* rite) < boost: mutex: scoped_lock Lock(a): streantnal() = prile; > Inline vois output2FTLE: Output (const std:strings sa) boost: mutex: scoped_lock tock(atx); FILE pstrean = streaninel(); AF (pstreae) fprint(pstrean, "Ss", msg-c_str())s FHun(estrean): ) [Needless to say, interlocked logging will be slower, yet unused logging will run as fast as ever. This is because the test inthe macro is unsynchronized—a benign race condition that does no harm, assuming integers ae assigned atomically (a fair assumption on most platform) ‘Compile-Time Plateau Logging Level ‘Sometimes, you might fel the footprint of the application increased more than you can afford, o thatthe runtime comparison incurred by even unused ogging statements is significant. Lets provide a means to eliminate some ofthe logging (at compile time) by using a preprocessor symbol ‘endef FILE, pax LeveL Iderine FILELOGRSKCLEVEL Logoenucs fender ‘define FILE Loo(ievel) \ Th Clavel > PELELOG RALLEVEL) 5 flse Hf (level > FILELog::ReportingLevel{) || lvtput2eitessStrean() 5 \ else FILELOg() cet (level) ‘his code is interesting in that it combines wo tess. IFyou pass a compile-time constant to &1LE_L06, the frst tests against wo stich constants and any ‘optimizer wil pick that up statically and discard the dead brencs entirely from generated code. This optimization is so widespread, you can safely count ‘on tin most environments you take your code. The sccond test examines the atime logging level, as before, Effectively, #1.€U06,Max_ LEVEL imposes a static plateau on the dynamically allowed range of logging levels: Any logging level above the static plateau is simply eliminated from the code. To ithe bash-3.28 ars main.cpp bash3.26 “/a.one Debut 2922:25:29 64 DEBUGE A Loop with 9 Sterations 1 zasgsiaaleaa penuci: the counter 4 =e 2 22:95:03 60s bebuGE: the Counter {2 2 bash-3.28 gis nainsepp -DFILELOG MAK LEVELS bash3.26 “/a.one Debus 2172:28:31:372 DEBUG: Loop with 3 tterations Final Tips on Using Log ‘Te been using an interesting technique that lets you compare different runs by actually di¢ing the log files from each run, This is especially useful when ‘you have no idea what could be wrong and why you get diferent results when running the same code Cn ifferent platforms (Linux versus Windows). On different versions ofthe same the same platform (RHEL3 vs, RHEL), With different compiler settings (debug versus optimize) Just turn the verbosity to maximum dtal level (remember, because the logging is so light, you already havea lot of log statements on YogpesuGx in Your ode), run the application in te two different contexts, respove the timestamps from the logfiles (using, for intanee, the sed program), and compare the {so outputs. Even though the output files ean be huge, i is the difference between them that matters, and tht difference tends t be small and highly informative Another nice outcome of using this log framework was combining the logging of multiple different applications (or multiple instances ofa single application in a single unique file. Just open the same file from several applications in append mode, an voila—nicely interleaved logs from various processes! The resulted log Ble i ll meaningful aad (oa a decent operating system a! least) preserves log lines a alomui unis Ie done some measurements to actully sce how fast the runtime mechanism is (versus the compile time). The results tumed out to be very good. ‘he same code in three different configurations off INFO. log disabled at compile time, used as # bas line. on-INFO. log enabled at compile time, disabled at runtime (using INFO level of detail. ‘on-DEBUGA. og enabled at runtime and compile time (using DEBUG level of detail. ‘Each configuration was run multiple (20) times to eliminate the possible random errors) Figure 1 and Table | ilastrate the results. As you can see, the runtime performance (25,109 sec) matches the compile time (25.109 sec). That means you really wil pay for logging only if You ate going fo use i [Click image to view at fll size]non. noo 08. n0oa0 88 ononat Figure 1: Log speed tests. ‘onDEBUGS [ofFINFO_[on-INFO [AVERAGE 00:00:28,002 _00:00:25.106 00:00:25,109 [STDEV __o.0000020858 | o,o000008605 | o.n000014926 ‘Table 1 1 found very useful in practice to also output the thread ID from wiere the output comes from along with the timestamp and logging level. This i areal help when reading a logfile that comes from a multithreading application. ‘Petra isa vce president for Morgan Stanley, where he works as a C++ senior programmer in investment banking. He can be contacted at petra [email protected]. Related Article at 2 of this article canbe found here, eras Sovee | Bae, Suter cape 02 UB ah Mei es