Log4C Developers Guide

1.0.6 13 Nov 2006 First Draft: pulls together information from README files and various 'how to ' documents for Log4C. This document isstarted between the release of Log4C 1.1.0 and 1.2 so although much of it is valid for older releases it is more lookingtowards the 1.2 release. Where you see xxx in the text this denotes an item we need to get back to.

This document was created with Open Office 2.0.2 .

Table of Contents

1 Introduction.............................................................................................................................................22 Log4C Concepts and Overview..............................................................................................................33 How to download and build Log4C.......................................................................................................4

3.1 Parsing the Log4C configuration file--the expat dependency ...................................................43.2 Some comments on building Log4C in the corporate factory ..................................................5

3.2.1 Log4C is released under the LGPL license. ................................................................ 53.2.2 Building on different platforms .................................................................................... 5

3.2.2.1 Building On Unix ............................................................................................ 53.2.2.2 Building on Windows ...................................................................................... 63.2.2.3 Building on MAC OS ...................................................................................... 6

3.2.3 Packaging on different platforms ................................................................................. 64 Getting started--running the examples...................................................................................................6

4.1 Running application_1 ..............................................................................................................74.2 Configuring the priority for a category .....................................................................................84.3 Configuring application_1 to log to a set of log files................................................................94.4 What about application_2 ?.....................................................................................................10

5 Using the Log4C API...........................................................................................................................105.1 The helloworld example with Log4C .....................................................................................10

5.1.1 The helloworld code ................................................................................................... 105.1.2 Configuring and running helloworld .......................................................................... 11

5.1.2.1 Configuring helloworld with the configuration file....................................... 125.1.2.2 Configuring helloworld with the environment variables............................... 125.1.2.3 Configuring helloworld via the Log4C API .................................................. 13

5.2 Comments on the the different configuration methods...........................................................135.3 Interaction of the configuration methods ................................................................................145.4 Including Log4C calls into your code ....................................................................................14

5.4.1 Making it easy to remove Log4C calls from your code ............................................. 145.4.2 Other useful Log4C functions .................................................................................... 165.4.3 Multi threaded applications using Log4C .................................................................. 17

5.4.3.1 Why the Log4C API is not fully MT safe ..................................................... 175.4.3.2 How to use Log4C in an MT safe way .......................................................... 18

6 Extending Log4C with your own custom appender and layout types ..................................................187 Troubleshooting and debugging ..........................................................................................................218 Deployment Experience with Log4C....................................................................................................21

8.1 Blame it on the logger ..............................................................................................................218.2 Preparing to move logging from development into a productionenvironment....................................................................................................................................22

9 References............................................................................................................................................24

1 Introduction

Log4C is a C-based logging library. Its goal is provide the C software developer with a configurableand efficient library for logging messages. So Log4C provides the classic logging functionalityrequired by most projects by default while leaving the scope for extensibility for those projects thathave special requirements.

It seems extraordinary that just about every C project ends up developing it's own logging system.While there are environments like embedded systems or operating system kernels that may justifycustomized logging technology (and there are open source projects aimed at those areas) there seems tobe no a priori reason why most user level application and server software written in C cannot share alogging library.

Log4C may be linked with end user applications or other libraries that will eventually be linked intoapplications. The library is released on SourceForge under the LGPL license.

This document is aimed at developers who wish to use Log4C to manage logging in their own project.The kind of information you can find here includes:

a general introduction to Log4C concepts and how Log4C can be useful to you how to download and build Log4C how to use and configure the library via examples how to extend Log4C with your own code for customizing, for example, where messages are

logged and how they are formatted. troubleshooting and debug information deployment experience with Log4C references to useful material

For information relating to Log4C internals the intention is to create a separate documents, “Log4CInternals”. This will help keep the size of each document reasonable and help decouple the evolution ofthose documents.

2 Log4C Concepts and Overview

Log4C follows the overall style of the native Java logging and Log4j utilities. There are threefundamental types of object in Log4C: categories, appenders and layouts.

You can think of these objects as corresponding to the what, where and how of the logging system:categories describe what sub-system the message relates to, appenders determine where the messagegoes and layouts determine how the message is formatted.

So for example a category might be named “httpd.network.connection” that may be logging messagesto the “syslog” appender using the “dated” format. The convention of using dotted names forcategories determines an obvious hierarchy which is used to determine parent and child relationshipsduring logging. These are used because during logging (and this is configurable) a message logged by acategory will also be logged to the appenders of any parent categories.

There is always a root category which represents a conceptual root of the hierarchy of categories.

The categorization of the system make sit easy to select which messages are logged and where—if Iconfigure an appender and priority for a given category then I automatically do it for all it'sdescendants (unless I explicitly say not to). The extreme case of this is the root category—anyconfiguration positioned there will by default apply to all categories in the system. However, if I needto drill down for finer grained control then the categorization allows me to do that.

This means that you should expect to spend some time defining the categories within your system. Asan example, if your system separates into different applications then you should have at least onecategory per application. If your product ships a set of libraries, then each library probably merits it'sown category. Libraries linked in as a dependency on a given library should probably be sub-categoriesof the including library. Within a library you might identify other sub categories, for example ahousekeeping category, a connection manager category, a request processor and so on that correspondto logging messages that will come from those sections of code.

The other concept fundamental to Log4C is the logging priority: each message has an associatedpriority. At a given point a given instance of Log4c has an associated configured priority--onlymessages with a higher priority than the current Log4c priority are sent to an appender for processing.

3 How to download and build Log4C

The README file shipped in the workspace gives a good account of Log4C dependencies and how todo a basic build so we will not repeat all that information here. Here we treat that information thatrequires more detail than the basic build instructions.

3.1 Parsing the Log4C configuration file--the expat dependency

“The expat dependency”--no it's not the latest Robert Ludlum novel...it refers to the fact that Log4Ccan use the expat library (cf. http://expat.sourceforge.net/) as the low level xml parser for it'sconfiguration file. By default when you run configure, it will look for expat and if it finds it it will useit. You can tell configure not to go looking for expat by using the --without-expat option.

In the case where you have specified --without-expat or expat is not found or is the wrong version orfails to compile with a small test program, then Log4C will use a custom XML parser using lex/yacc

code for the lower level xml parsing. This code works ok with the current configuration file andbundled appenders and layouts.

So why do you need to know how Log4C handles it's xml parsing ? Well, as a developer integratingLog4C you will probably prefer the “bundled parser” option as it means you do not pull a dependencyon expat. However from the point of view of maintaining Log4C there is still a dependency on thelex/yacc tools needed to generate the parsing code. So there is a choice to make in terms of whichdependency Log4C manages. The current consensus is that expat is now sufficiently widespread andmature that introducing a dependency on it is a reasonable step.

So as a developer consuming Log4C you can rely on the older bundled code staying in Log4C for theforeseeable future. But you should not expect bug fixes therein nor that that code will evolve to takeaccount of potential enhancements to the configuration file. So the message is that you should start toprepare for handling the dependency on the expat library, either by bundling it in your delivery withLog4C or ensuring that it is available natively on your supported platforms or by specifying it as arequirement to your end users.

3.2 Some comments on building Log4C in the corporate factory

The Log4C source code is free, which is good, but there is still a cost in integrating it into your project.Here are some points to consider on this.

3.2.1 Log4C is released under the LGPL license.

See the COPYING file in the release for the full license but this means a few things:

you will almost certainly require a legal review inside your company to approve your use ofLog4C.

you are required by the LGL license to ship along with your project a copy of the Log4C sourceas used in your project. You are also required to ship a copy of the LGL license. Typically oneships a file called something like, ThirdPartyLicenses which lists all the open sourcecomponents used by your project and includes all the required licenses for those open sourcecomponents.

Ideally the changes you make to Log4C will be minor porting, bug fixes or integration issues requiredto integrate with your project. Ideally, you would invest some time to work with the Log4C maintainersto integrate your changes back into the Log4C repository in a way that was useful to the wholecommunity. Thus, while one might go with some 'quick fixes' in order to integrate Log4C, expect tospend some time refactoring or generalizing the code in order to have it accepted back into the publicrepository.

A general aspect of integrating an open source project is that, for various reasons, you may wish tomake it easy to remove the dependency. We discuss this in more detail in the section titled “IntegratingLog4C calls into your code”.

3.2.2 Building on different platforms

3.2.2.1 Building On Unix

On Unix systems (certainly for Linux, Solaris and HP-UX) you should be able to use the providedautoconf tools to build Log4C as described in the README. One tip for managing different platformsis not to compile the code in the source directories, but rather to create a directory per platform, run theconfigure script therein to generate the Makefiles and then build the library, tests, examples and doc inper-platform directory. This allows you to use the same source code to maintain binaries for multipleplatforms at the same time. So an example compile session might run:

On my Solaris x86 build machine:[ mySol a r i s x86Bui l dMa c hi ne ] $ t a r - z xvf l og4c - 1. 0. 12. t a r . gz[ mySol a r i s x86Bui l dMa c hi ne ] $ mkdi r bui l d- Sol a r i s - x86; c d bui l d- Sol a r i s - x86[ mySol a r i s x86Bui l dMa c hi ne ] $ . . / l og4c - 1. 0. 12/ c onf i gur e - - pr e f i x=/ pa t h/ of / i ns t a l l a t i on[ mySol a r i s x86Bui l dMa c hi ne ] $ ma ke

then on the Solaris sparc build machine:[ mySol a r i s Spa r c Bui l dMa c hi ne ] $ mkdi r bui l d- Sol a r i s - s pa r c ; c d bui l d- Sol a r i s - s pa r c[ mySol a r i s Spa r c Bui l dMa c hi ne ] $ . . / l og4c - 1. 0. 12/ c onf i gur e - pr e f i x=pa t h/ of / i ns t a l l a t i on[ mySol a r i s Spa r c Bui l dMa c hi ne ] $ ma ke

and so on.If you need to use a compile other than gcc then use the CC environment variable as descirbed in theREADME file.

3.2.2.2 Building on Windows

The build for Windows is not yet integrated into the autoconf mechanism (due to limitations inautoconf on windows at the time the integration was done). So a makefile is provided in the msvc6directory for building on Windows.

Note that some of the appenders are not compiled on Windows...for example the syslog appender.See the README in the msvc6 directory for the Windows build details.

3.2.2.3 Building on MAC OS

There are some project files provided for compiling on MAC OS.xxx Not sure of the status of the mac stuff at this time.

3.2.3 Packaging on different platforms

Using the provided makefilery as explained in the section “Building on different platforms” it shouldbe possible to generate the library, tests and examples. However native packaging is provided only forLinux at this time. Even there you will probably wish to customize the naming and so on of thegenerated rpms.

So you should expect to spend some time creating scripts or makefiles that will take the binaries from abuild directory and package them as required for your supported platforms.

Another solution is just to bundle the Log4C library in one of your existing packages or rpms.

4 Getting started--running the examples

Once you have built Log4C you can go into the examples directory and run the examples. There maybe additional information in the README in the examples directory.

For ease of presentation the text below tends to suppose a Unix (or cygwin) environment. We don' ttranslate each command to Windows supposing that the translation of commands, pathnames and so onare fairly obvious. However where there are significant differences for Windows then we try tohighlight them.

4.1 Running application_1

If you have built in a directory separate from the sources, as is recommended, then you need to copythe log4crc file from the source directory into the build directory:

$ c d e xa mpl e s

$ c p . . / l og4c / e xa mpl e s / l og4c r c .

Then run the example application_1 as follows. You will need to put the Log4C library on some pathso the system can find it when application_1 starts up. If you have used the autoconf tools on Unix andthe Makefile in the msvc6 directory that will look like this:

On Li nux/ Sol a r i s : e xpor t LD_LI BRARY_PATH=. . / s r c / l og4c / . l i bs /

On HP- UX: e xpor t SHLI B_PATH=. . / s r c / l og4c / . l i bs /

On Wi ndows : s e t Pa t h=. . \ l og4c ; % Pa t h %

Then you can run application_1 as follows:

$ . / a ppl i c a t i on_1 4

You should see this:

[ s t dout ] ERROR s i x13l og. l og. a pp. a ppl i c a t i on1 - s ome e r r or f r om a pp1 a t l i ne 79 i nf i l e . . / . . / l og4c / e xa mpl e s / a ppl i c a t i on_1. c - l oop 0[ s t dout ] ERROR s i x13l og. l og. a pp. a ppl i c a t i on1 - s ome e r r or f r om a pp1 a t l i ne 79 i nf i l e . . / . . / l og4c / e xa mpl e s / a ppl i c a t i on_1. c - l oop 0[ s t dout ] ERROR s i x13l og. l og. a pp. a ppl i c a t i on1 - s ome e r r or f r om a pp1 a t l i ne 79 i nf i l e . . / . . / l og4c / e xa mpl e s / a ppl i c a t i on_1. c - l oop 1[ s t dout ] ERROR s i x13l og. l og. a pp. a ppl i c a t i on1 - s ome e r r or f r om a pp1 a t l i ne 79 i nf i l e . . / . . / l og4c / e xa mpl e s / a ppl i c a t i on_1. c - l oop 1. . / . . / l og4c / e xa mpl e s / a ppl i c a t i on_1. c

application_1 is a simple example that loops, logging two messages on each loop. In the output abovewe only see the error message logged at priority ERROR, the other message logged at priority DEBUG

is not logged with the default configuration in the l og4c r c file.

But why is the log message being logged twice ?

Thank you for asking. What's happening is that although the application submits the message onlyonce to Log4C, it is passed to all the appenders attached to the originating categorys i x13l og. l og. a pp. a ppl i c a t i on1 AND it's ancestor categories. So, if you look in log4crc you will seethis line:

< c a t e gor y na me =" s i x13l og. l og" pr i or i t y=" e r r or " a ppe nde r =" s t dout " / >

So the stdout appender for the category s i x13l og. l og. a pp. a ppl i c a t i on1 is logging the message andthe stdout appender for the ancestor category s i x13l og. l og is also logging it.

So why this strange behavior with respect to logging messages to multiple appenders ? Well, in thisexample it's hard to see the point. But suppose we have multiple applications and associate categoriess i x13l og. l og. a pp. a ppl i c a t i on2, s i x13l og. l og. a pp. a ppl i c a t i on3, s i x13l og. l og. a pp. a ppl i c a t i on4

, say, with them. Then I can get very tight control over what gets logged using the categorization of themessages: if I set an appender on s i x13l og. l og. a pp for example then I will get the messages from allthe applications. But if I am just interested in messages from a subset of categories then I can configurean appender and logging priority just for them.

So once the application or system designer has done a good job defining the categories available forlogging then Log4C offers the administrator or end user very powerful and easy control over themessages he chooses to log, where they get logged and the format they are logged in. Furthermore,even if the application developer has to go back and introduce new categories then with Log4C that'san easy task. Too often in-house logging mechanisms are hard coded for a specific domain and addingwhat would amount to new categories requires significant additional coding. Not so with Log4C.

You can see that there is additional text being logged in addition to the message from the application.This 'overhead' is added by Log4C and the various parts of the message break down like this:

[ s t dout ] The name of the appender logging the message surrounded by square brackets.ERROR The log level of the message.s i x13l og. l og. a pp. a ppl i c a t i on1 The name of the catgory originating the messages ome e r r or f r om a pp1 a t l i ne 78 i n f i l e . . / . . / l og4c / e xa mpl e s / a ppl i c a t i on_1. c The actual logmessage.

4.2 Configuring the priority for a categoryNow, edit the log4crc file to change the value of the priority field from “error” to “debug”:

< c a t e gor y na me =" " pr i or i t y=" de bug" a ppe nde r =" s t dout " / >

Run application_1 again, you should see this:

[ s t dout ] DEBUG s i x13l og. l og. a pp. a ppl i c a t i on1 - De buggi ng a pp 1 - l oop 0[ s t dout ] DEBUG s i x13l og. l og. a pp. a ppl i c a t i on1 - De buggi ng a pp 1 - l oop 0[ s t dout ] ERROR s i x13l og. l og. a pp. a ppl i c a t i on1 - s ome e r r or f r om a pp1 a t l i ne 79 i nf i l e . . / . . / l og4c / e xa mpl e s / a ppl i c a t i on_1. c - l oop 0[ s t dout ] ERROR s i x13l og. l og. a pp. a ppl i c a t i on1 - s ome e r r or f r om a pp1 a t l i ne 79 i nf i l e . . / . . / l og4c / e xa mpl e s / a ppl i c a t i on_1. c - l oop 0[ s t dout ] DEBUG s i x13l og. l og. a pp. a ppl i c a t i on1 - De buggi ng a pp 1 - l oop 1

[ s t dout ] DEBUG s i x13l og. l og. a pp. a ppl i c a t i on1 - De buggi ng a pp 1 - l oop 1[ s t dout ] ERROR s i x13l og. l og. a pp. a ppl i c a t i on1 - s ome e r r or f r om a pp1 a t l i ne 79 i nf i l e . . / . . / l og4c / e xa mpl e s / a ppl i c a t i on_1. c - l oop 1[ s t dout ] ERROR s i x13l og. l og. a pp. a ppl i c a t i on1 - s ome e r r or f r om a pp1 a t l i ne 79 i nf i l e . . / . . / l og4c / e xa mpl e s / a ppl i c a t i on_1. c - l oop 1

What's happening now is that because the priority of the category has been lowered to DEBUG, thedebug message in the application is now appearing. And, for the reasons explained above for theERROR message, it appears twice.

4.3 Configuring application_1 to log to a set of log filesLogging to stdout is fine but suppose we wish to trap all the messages fors i x13l og. l og. a pp. a ppl i c a t i on1 to a set of log files. To do this we can use the rolling file appender.

Edit this line in log4crc to replace stdout with myrollingfileappender:< c a t e gor y na me =" s i x13l og. l og. a pp. a ppl i c a t i on1" pr i or i t y=" de bug" a ppe nde r =" s t dout " / >

Run application_1 again. This time on stdout you will see:

[s t dout ] DEBUG s i x13l og. l og. a pp. a ppl i c a t i on1 - De buggi ng a pp 1 - l oop 0

[ s t dout ] ERROR s i x13l og. l og. a pp. a ppl i c a t i on1 - s ome e r r or f r om a pp1 a t l i ne 79 i nf i l e . . / . . / l og4c / e xa mpl e s / a ppl i c a t i on_1. c - l oop 0

[ s t dout ] DEBUG s i x13l og. l og. a pp. a ppl i c a t i on1 - De buggi ng a pp 1 - l oop 1

[ s t dout ] ERROR s i x13l og. l og. a pp. a ppl i c a t i on1 - s ome e r r or f r om a pp1 a t l i ne 79 i nf i l e . . / . . / l og4c / e xa mpl e s / a ppl i c a t i on_1. c - l oop 1

The other copies of the log messages are no longer being sent to stdout but rather to themyrollingfileappender appender, which is logging them to a set of log files in the current directory ofthe form myprefix.*. Verify this by dumping one of the log files:

$ c a t mypr e f i x . 0

20061105 16: 29: 33. 530 DEBUG s i x13l og. l og. a pp. a ppl i c a t i on1- De buggi ng a pp 1 - l oop 0

20061105 16: 29: 33. 604 ERROR s i x13l og. l og. a pp. a ppl i c a t i on1- s ome e r r or f r om a pp1 a tl i ne 79i n f i l e . . / . . / l og4c / e xa mpl e s / a ppl i c a t i on_1. c - l oop 0

20061105 16: 29: 36. 605 DEBUG s i x13l og. l og. a pp. a ppl i c a t i on1- De buggi ng a pp 1 - l oop 1

20061105 16: 29: 36. 605 ERROR s i x13l og. l og. a pp. a ppl i c a t i on1- s ome e r r or f r om a pp1 a tl i ne 79 i n f i l e . . / . . / l og4c / e xa mpl e s / a ppl i c a t i on_1. c - l oop 1

You will notice that these log messages also have a date stamp on them. This is becausemyrollingfileappender has been configured in log4crc to use the dated layout (rather than the basiclayout) and this layout inserts a date stamp in the message before logging it.

You will also notice that the unlike the stream appender which is logging to stdout, the rollingfileappender does not prefix it's messages with the name of the appender enclosed in square brackets. Thereason for the change is twofold: first we did not see a real need to log the appender name in eachmessage and secondly it seems better from a design point of view if it is only the layout code whichdetermines the format of the message.

4.4 What about application_2 ?As an application application_2 does the same thing as application_1—it loops logging a fewmessages. However the interesting thing about application_2 is that it links with a library where newappenders and layouts have been defined. So we leave this more advanced topic to a later section.

5 Using the Log4C APIIn the last section we saw how to control the logging output from a Log4C enabled example using theconfiguration file. In this section we see how to “Log4C enable” your applications using the Log4CAPI.

At the top of the main Log4C home page you will see links to the API documentation for the latestreleased Log4C, for example here: http://log4c.sourceforge.net/files.html. You should keep a referenceto this documentation when working with Log4C—it documents parameters, call conventions, returncodes, issues and sometimes has code snippets. If you have checked out the workspace from CVS youwill need to have added –enable-doc to build the documentation, which gets put into the doc directorywhere you build. You are strongly recommended to build and view (and improve!!) the APIdocumentation.

Depending on the utilities available (dyoxygen, latex,...) on your system the API documentation can begenerated in html, pdf and man page format. See 'configure –help' and the README for details onthis.

5.1 The helloworld example with Log4C

As tradition dictates we provide a hello world example to illustrate the bare bones of Log4C. The helloworld code looks as follows:

5.1.1 The helloworld code

# i nc l ude < s t di o. h >

# i nc l ude " l og4c . h"

i nt ma i n( i nt a r gc , c ha r ** a r gv) {

i n t r c = 0;

l og4c _c a t e gor y_t * myc a t = NULL;

i f ( l og4c _i ni t ( ) ) {

p r i nt f ( " l og4c _i ni t ( ) f a i l e d" ) ;

r c = 1;

} e l s e {

myc a t = l og4c _c a t e gor y_ge t ( " l og4c . e xa mpl e s . he l l owor l d" ) ;

l og4c _c a t e gor y_l og( myc a t , LOG4C_PRI ORI TY_ERROR, " He l l o Wor l d! " ) ;

/ * Expl i c i t l y c a l l t he l og4c c l e a nup r out i ne */

i f ( l og4c _f i ni ( ) ) {

pr i nt f ( " l og4c _f i ni ( ) f a i l e d" ) ;

}

}

r e t ur n 0;

}

As you can see there are four calls to the Log4C API here.

1. The l og4c _i ni t ( ) call initializes the library, and reads the configuration files (if any) andenvironment variables. We will see more about the configuration mechanisms below.

2. The l og4c _c a t e gor y_ge t ( ) function returns a reference to a category required for logging.

3. The l og4c _c a t e gor y_l og( ) function logs the age old message .

4. The l og4c _f i ni ( ) tells Log4C to cleanup, freeing memory, closing files and so on.

5.1.2 Configuring and running helloworld

See the section “Running application_1” for information on setting the environment before runninghelloworld. If you have compiled and run the helloworld example as is you will see...no output :-(.

This is because although we have done all the right things, we have not as yet configured an appenderfor our helloworld category. It will be instructive to visit each of the three ways Log4C provides toconfigure it's behavior.

For information, the command line on Linux to compile a Log4C example, supposing that the log4clibrary is already compiled, looks like this:

$ gc c - I . . / . . / l og4c / s r c - I . . / s r c . . / . . / l og4c / e xa mpl e s / he l l owor l d/ he l l owor l d. c - L. . / s r c / l og4c / . l i bs / - l l og4c

5.1.2.1 Configuring helloworld with the configuration file

Copy the log4crc file into the helloworld directory and edit it to add this line:< c a t e gor y na me =" l og4c . e xa mpl e s . he l l owor l d" pr i or i t y=" de bug" a ppe nde r =" s t dout " / >

For completeness we cite here the two other lines relevant to the configuration for helloworld, whichare already in the file:

< a ppe nde r na me =" s t dout " t ype =" s t r e a m" l a yout =" ba s i c " / >

< l a yout na me =" ba s i c " t ype =" ba s i c " / >

Run the helloworld program again. You will now see:[ s t dout ] ERROR l og4c . e xa mpl e s . he l l owor l d - He l l o Wor l d!

Cool. What has happened is that we have configured Log4C to send messages for our helloworldexample, logged with priority DEBUG or above to go to the appender called “stdout”. This appender

is a stream based appender with name stdout, which means the messages will go to...well, stdout. Theappender called “stdout” also specifies it's layout, which is the basic layout type. Of course we couldconfigure the messages to go to any of the existing Log4C appenders and use any of the layouts toformat the message.

By way of building a mental model of what is happening inside the Log4C library note that the factthat the helloworld category is mentioned in the configuration file means that after the call tolog4c _i ni t ( ) , a nicely configured instance of that category already exists inside the library. In the casewhere that category is not mentioned in the configuration file then it is the call tol og4c _c a t e gor y_ge t ( " l og4c . e xa mpl e s . he l l owor l d" ) that creates the (unconfigured) instance on thiscategory.

5.1.2.2 Configuring helloworld with the environment variables

Log4C provides a minimal configuration ability via two environment variables: LOG4C_APPENDERand LOG4C_PRIORITY. Both these variables act only on the root category but provide a quick way toglobally enable logging . This can be useful to take a look at log messages before drilling down into afiner grained logging using the categorization to focus on messages of particular interest.

So, remove the line from the configuration file you added above:< c a t e gor y na me =" l og4c . e xa mpl e s . he l l owor l d" pr i or i t y=" de bug" a ppe nde r =" s t dout " / >

Rerun helloworld to confirm you have no output then rerun the program as follows:e xpor t LOG4C_APPENDER=s t dout ; he l l owor l d

You will now see the log message as before:[ s t dout ] ERROR l og4c . e xa mpl e s . he l l owor l d - He l l o Wor l d!

What is happening here is that at init time Log4C is reading the LOG4C_APPENDER environmentvariable and configuring the stdout appender for the root category as a result. Thus all messages loggedto this instance of the library, at a priority higher than NOTICE (for that is the priority set on the rootcategory in the configuration file) will be sent to stdout.

We could just as well have set LOG4C_APPENDER to be myrollingfileappender, as defined in theconfiguration file, in which case the helloworld message would have been logged to the myprefix.0file. Go on, give it a lash, it works!

5.1.2.3 Configuring helloworld via the Log4C API

The configuration achieved in the above sections via the configuration file or the environmentvariables can also be done manually using the Log4C API. Add the following call to set the appenderfor the helloworld category:

myc a t = l og4c _c a t e gor y_ge t ( " l og4c . e xa mpl e s . he l l owor l d" ) ;

l og4c _c a t e gor y_s e t _a ppe nde r ( myc a t , l og4c _a ppe nde r _ge t ( " s t dout " ) ) ; / * ADD THI S LI NE */

l og4c _c a t e gor y_l og( myc a t , LOG4C_PRI ORI TY_ERROR, " He l l o Wor l d! " ) ;

Recompile and rerun helloworld to see the output:[ s t dout ] ERROR l og4c . e xa mpl e s . he l l owor l d - He l l o Wor l d!

5.2 Comments on the the different configuration methodsThe three configuration methods demonstrated above are likely to be of more or less interest dependingon the nature of your project. Far be it from Log4C developers to adopt the arms dealer pattern oftechnology development, so here are some general observations to help you think about which methodor methods are most appropriate for your scenario:

The environment variables seem to be generally useful as a quick way to get your application orsystem outputting messages.

The configuration file is certainly a very powerful way and easy way to configure logging. It isprobably of most utility in those cases where you are quite close to the final applicationdevelopment as this gives you more say in the nature of the configuration requirements youintroduce to application users.

Using the API to configure Log4C is probably of use to those projects delivering someintermediate technology or framework to other application developers. This is because as aframework or library provider you probably do not wish to contribute to a kind of“configuration file proliferation”. You probably would rather take configuration from your ownconfiguration files, or even programmatically from your application developers and thenconfigure Log4C programmatically. If you use the Log4C configuration file and yourframework has it's own configuration file and your application has it's own configuration filethen it's getting a bit hard for your administrator to keep track of all these configuration files.

5.3 Interaction of the configuration methods

The l og4c _i ni t ( ) function looks for a configuration file first and then reads the environment variables,so for defining the priority and appender for the root category the environment variables takeprecedence. The order that Log4C (up to version 1.0.12) searches for the configuration file is“$LOG4C_RCPATH/log4crc", "$HOME/.log4crc", "./log4crc" . This seems wrong and will probablybe changed or made configurable. Also, it is not clear if there is a well defined design decision aboutwhat happens when loading fails in a file or if, once a file is loaded successfully, we should then stoptrying to load other files. xxx revisit this on the list.

If you configure Log4C objects after the call to l og4c _i ni t ( ) then the new settings you configure takeprecedence.

5.4 Including Log4C calls into your code

5.4.1 Making it easy to remove Log4C calls from your code

For various reasons you may wish to wrap calls to the Log4C API either in macros or function calls.We list some of these here to help you decide if you want to use wrapped or naked calls to Log4C.

You may wish to do this in order, for example, to do some generic preparation of your loggingmessages.

Another reason might be to define some smaller subset of the Log4C API that you wish all yourlibrary writers or application developers to use—by using the interface you define you canensure a consistent usage of Log4C across your system.

Another reason might be that, Log4C being an external open source project, you may wish tomake it easy to remove Log4C from your project, for fear of API stability, quality or licensingissues that may arise later in your project.

As an example, we present one way of wrapping the Log4C calls here. We will modify thehelloworld.c file to remove the hard coded dependency on Log4C and introduce a level of indirectionthat allows an easy decoupling of helloworld from Log4C.

We simply introduce functions that wrap the required Log4C functions: myl og_i ni t ( ) forl og4c _i ni t ( ) , myl og_f i ni ( ) for l og4c _f i ni ( ) and so on. We also need to define our own copies of thelogging priorities. See the file mylog.h for the full definitions. So, for logging single strings we can usea function like this:

s t a t i c LOG4C_I NLI NE voi d myl og_ms g( c ha r *c a t Na me , i nt a _pr i or i t y, c ha r *ms g) {

# i f nde f WI THOUT_LOG4C

l og4c _c a t e gor y_l og( l og4c _c a t e gor y_ge t ( c a t Na me ) , a _pr i or i t y, ms g) ;

# e l s e

pr i nt f ( ms g) ;

# e ndi f

}

You can see that if this code is compiled with -DWITHOUT_LOG4C then Log4C is totally removed.

To use this code, rather than directly calling l og4c _c a t e gor y_l og( ) we would do this:myl og_ms g( " l og4c . e xa mpl e s . he l l owor l d" , MYLOG_PRI ORI TY_ERROR, " He l l o Wor l d! " ) ;

One suggestion by this writer http://jefficus.usask.ca/article.php?story=20040613140849924 is todefine a macro for the logging call which means you need only mention hte category name for yoursubsystem in one place, typically in some include file used by all files in that subsytem.

So your simplified logging call actually looks like this:

MYLOG_MSG( MYLOG_PRI ORI TY_ERROR, " He l l o Wor l d! " ) ;

but at the start of the helloworld.c file you define the category name to use, which is picked up by theMYLOG_MSG macro:

# de f i ne MYLOG_CATEGORY_NAME " l og4c . e xa mpl e s . he l l owor l d"

The MYLOG_MSG macro is defined such that if you do not define a category name then it defaults tothe root category. This means it is safe and, if you want to be lazy, easy to get logging.

The only comment on this handy technique is that if you are paranoid about performance then you maynot want to have the lookup call based on the logger name at each logging call. In this case you woulddo the call to l og4c _c a t e gor y_ge t ( ) (or the wrapped equivalent) at the initialize time of yoursubsystem and then use the actual reference to the category object rather than the category name.

The code to wrap the log call to take a variable number of arguments is a bit more complex than thecode used to log one string shown here—see the function myl og_l og( ) defined in mylog.h for way todo that. The code discussed above is included in helloworld1.c and the file mylog.h. The full code ofhelloworld1.c is included here for convenience:

# i nc l ude < s t di o. h >

# de f i ne MYLOG_CATEGORY_NAME " l og4c . e xa mpl e s . he l l owor l d"

# i nc l ude " myl og. h"

i nt ma i n( i nt a r gc , c ha r ** a r gv) {

i n t r c = 0;

i f ( myl og_i ni t ( ) ) {

p r i nt f ( " myl og_i ni t ( ) f a i l e d" ) ;

r c = 1;

} e l s e {

MYLOGMSG( LOG4C_PRI ORI TY_ERROR, " He l l o Wor l d! " ) ;

/ * Expl i c i t l y c a l l t he l og4c c l e a nup r out i ne */

i f ( myl og_f i ni ( ) ) {

p r i nt f ( " myl og_f i ni ( ) f a i l e d" ) ;

r c = 1;

}

}

r e t ur n r c ;

}

5.4.2 Other useful Log4C functions

See the Log4C API documentation in your workspace (as generated by make if you have done'configure –enable-doc') or here for the last release: http://log4c.sourceforge.net/files.html.

Log4C provides some convenience functions that mean you do not have to specify the logging priorityin the logging call—for example l og4c _c a t e gor y_f a t a l ( ) , l og4c _c a t e gor y_e r r or ( ) and so on. Theseare documented here for the last release: http://log4c.sourceforge.net/category_8h.html.

You can test to see the logging priority of a given category using thelog4c _c a t e gor y_i s _pr i or i t y_e na bl e d( ) function.

Browse the documentation and examples and tests source to see other usage of the Log4C API.

5.4.3 Multi threaded applications using Log4C

Log4C is not Multi thread safe (MT safe). xxx This topic should be discussed on the mailing list andrevisited systematically to see what can be done and to document the MT status of the various APIcalls.

In the meantime, it is possible to use Log4C safely provided you follow some rules related toinitialization and provided the appenders and layouts you use are themselves MT safe.

5.4.3.1 Why the Log4C API is not fully MT safe

A fully MT safe library would be such that multiple threads in the application could safely call any ofthe LOG4C API functions without any need for synchronization on the part of the application. Thereare several places where Log4C does not satisfy this:

l og4c _i ni t ( ) This function must be called successfully just once by the application. If twothreads call this at the same time the behavior is undefined. The problem is that both threadswill start to initialize global objects and the behavior is this case is undefined.

l og4c _f i ni ( ) The same is true of l og4c _f i ni ( ) —this function can safely be called by only onethread at a time. Furthermore when you call l og4c _f i ni ( ) you must be sure that there are nocalls to the logging API in flight—if there are then l og4c _f i ni ( ) goes ahead freeing up Log4Cobjects (like categories, appenders and layouts) so this will probably crash your application.

Readonly type Log4C calls like l og4c _c a t e gor y_i s _pr i or i t y_e na bl e d( ) and so on are MT safebetween themselves and calls to logging functions like l og4c _c a t e gor y_l og( ) but are not MTsafe with respect to l og4c _i ni t ( ) and l og4c _f i ni ( ) .

Issues related to lazy creation and intialization of objects.

Lookup and creation type calls like l og4c _c a t e gor y_ge t ( ) , l og4c _a ppe nde r _ge t ( ) ,l og4c _l a yout _ge t ( ) and so on are not MT safe, even between themselves. The problemis that if two threads call l og4c _c a t e gor y_ge t ( ) at the same time then, if the categoryinstances do not yet exist in the internal store of category instances, then the threadsmay clash as they write to the shared hash that stores the instance references. This isbecause no synchronization is provided on the shared instance hash.

Calls to l og4c _c a t e gor y_l og( ) are not MT safe when called on the same categories. Theproblem is that log4c_category_log() eventually calls l og4c _a ppe nde r _a ppe nd( ) to logthe message. However, if the appender has not already been opened then it callsl og4c _a ppe nde r _ope n( ) , which is not MT safe. The result could be that the two threadswill to open the appender at the same time which will give rise to undefined behavior.

The calls to the currently shipped basic and dated layouts all work in the context of a logging event

object which is allocated by the l og4c _c a t e gor y_l og( ) call on the stack. Thus the current usage of thelayout interface is in fact MT-safe.

5.4.3.2 How to use Log4C in an MT safe way

It can be seen from the last section that if you can ensure the following two conditions then you shouldbe able to use Log4C safely in an MT environment:

1. perform the initialization of Log4C and the creation and intialization of all the log4C loggingobjects before before starting up the multiple threads that call Log4C.

In reality this can be problematic as in order to ensure that “open” has been called on all theappenders in your system you will need to know the appender names up front at init time. Fordevelopers configuring Log4C programatically then this may work—by definition they willhave the names of all the Log4C objects available, otherwise they could not configure thenprogrammatically. For those using the Log4C configuration file where the Log4C init time codetakes care of the creation of the objects there is currently no way to tell that code “call open atinit time”. There seem to be a few possibilities here that require some development work inLog4C xxx:

A way to do that could be to provide an option 'call open at init time' in theconfiguration file.

Provide a function like l og4c _a ppe nde r _ope n_a l l ( ) that would call open on allappenders in Log4C. This way the application writer could call that routine to explicitlyopen all the appenders and avoid race conditions as various threads try to open them.

A third solution would be that log4c guarantees that the open call on an appender iscalled only once in a critical section.

2. Ensure that there are no logging calls in flight when l og4c _f i ni ( ) is called.

6 Extending Log4C with your own custom appender and layouttypes

The application_2 program in the examples shows how to define your own appender and layout types.

First a short word on Log4C terminology. The Log4C library ships with a set of “base types”. Thesebase types are category, appender, layout and rollingpolicy. These base types are hard coded into thelibrary—there is no run time way to control the creation or deletion of these base types (which perhapsthere should be). The base types all define an interface which one can use to create new types from thebase type. For example the stream appender, the stream2 appender and the rollingfile appender are alltypes of appender. The basic and dated layouts are both type of layout. The only rollingpolicy currentlysupplied with Log4C is sizewin, which is a type of rollingpolicy.

Once a base type exists in Log4C it is very easy to add new types—the application_2 example showshow to do this for appenders and layouts. Of course it is possible to add new base types to Log4C but

we leave that more as a topic for the “Log4C Internals” document as it is less common to need to dothat.

Note that it is not possible to extend the set of priorities defined by Log4C. These follow the syslog(3)model and are defined as follows:

t ype de f e num {

/ ** f a t a l */ LOG4C_PRI ORI TY_FATAL = 000,

/ ** a l e r t */ LOG4C_PRI ORI TY_ALERT = 100,

/ ** c r i t */ LOG4C_PRI ORI TY_CRI T = 200,

/ ** e r r or */ LOG4C_PRI ORI TY_ERROR = 300,

/ ** wa r n */ LOG4C_PRI ORI TY_WARN = 400,

/ ** not i c e */ LOG4C_PRI ORI TY_NOTI CE = 500,

/ ** i nf o */ LOG4C_PRI ORI TY_I NFO = 600,

/ ** de bug */ LOG4C_PRI ORI TY_DEBUG = 700,

/ ** t r a c e */ LOG4C_PRI ORI TY_TRACE = 800,

/ ** not s e t */ LOG4C_PRI ORI TY_NOTSET = 900,

/ ** unknown */ LOG4C_PRI ORI TY_UNKNOWN = 1000

} l og4c _pr i or i t y_l e ve l _t ;

So, turning to application_2...if you look at application_2.c you will see that it is very similar toapplication_1.c—it loops logging two messages at each loop. However, it includes an additional callbefore the call to l og4c _i ni t ( ) :

i n i t _e xa mpl e s _l i b( ) ;

This function is defining some new appender and layout types that can be accessed in the configurationfile or programatically. It eventually calls i n i t _e xa mpl e _a ppe nde r s ( ) . A key code snippet from that filelooks like this:

s t a t i c c ons t l og4c _a ppe nde r _t ype _t * c ons t a ppe nde r _t ype s [ ] = {

# i f de f HAVE_SYSLOG_H

& l og4c _a ppe nde r _t ype _s ys l og_l oc a l 0,

& l og4c _a ppe nde r _t ype _s ys l og_us e r ,

# e ndi f

& l og4c _a ppe nde r _t ype _s 13_f i l e ,

& l og4c _a ppe nde r _t ype _s 13_s t de r r

} ;

s t a t i c s i z e _t na ppe nde r _t ype s =

s i z e of ( a ppe nde r _t ype s ) / s i z e of ( a ppe nde r _t ype s [ 0 ] ) ;

i n t i ni t _e xa mpl e _a ppe nde r s ( ) {

i n t r c = 0; i nt i = 0;

f or ( i = 0; i < na ppe nde r _t ype s ; i ++)

l og4c _a ppe nde r _t ype _s e t ( a ppe nde r _t ype s [ i ] ) ;

r e t ur n( r c ) ;

}

The key call is to the (somewhat badly named) l og4c _a ppe nde r _t ype _s e t ( ) function. This function isregistering the appender types defined here with the Log4C code. Thereafter they are available for usein the configuration file or accessible programmatically.

We list here the complete code fragment that deals with the l og4c _a ppe nde r _t ype _s 13_f i l e type:/ *******************************************************************************/

/ *

* The l og4c s t de r r a dds a " [ s t de r r ] " i n f r ont of t he me s s a ge

* i t i s l oggi ng. t hi s one doe s n ' t , a nd l e a ve s t he f or ma t t i ng

* de s c i s i ons up t o t he f or ma t t e r

*/

s t a t i c i nt s 13_s t de r r _a ppe nd( l og4c _a ppe nde r _t * t hi s , c ons t l og4c _l oggi ng_e ve nt _t * a _e ve nt )

{

r e t ur n f pr i nt f ( s t de r r , " % s \ n" ,

a _e ve nt - > e vt _r e nde r e d_ms g) ;

}

s t a t i c i nt s 13_s t de r r _ope n( l og4c _a ppe nde r _t * t hi s )

{

/ * f pr i nt f ( s t de r r , " r unni ng s 13_s t de r r a ppe nde r ope n c omma nd now\ n" ) ; */

r e t ur n 0;

}

c ons t l og4c _a ppe nde r _t ype _t l og4c _a ppe nde r _t ype _s 13_s t de r r = {

" s 13_s t de r r " ,

s 13_s t de r r _ope n,

13_s t de r r _a ppe nd,

NULL,

} ;

/ *******************************************************************************/

The l og4c _a ppe nde r _t ype _s 13_s t de r r object is defining the interface that the core Log4C code will useto open, append and close this appender. As you can see this appender is pretty simple. It's closefunction is NULL—it is safe to do this the core Log4C code will not try to call a NULL function. Thename of this appender is defined in the first field of it's interface object and is ”s 13_s t de r r " . It is thisname that can be used either in the configuration file or programatically in the l og4c _a ppe nde r _ge t ( )

call to obtain a reference to the actual object.

Defining new layout types follows a similar pattern. See the example_formatters.c file for some

example code. Defining new rollingpolicy types--which are used to tell rollingfile appenders underwhich conditions rollover events should occur—proceeds in the same way. Other types ofrollingpolicy it would be interesting to add would be a policy that triggered rollover based on a timeperiod or fixed time basis.

7 Troubleshooting and debuggingIf you have run configure with the –enable-debug option then by setting the environment variablesSD_DEBUG and SD_ERROR to the value “1” you should see some useful internal Log4C tracemessages. The traces are useful for watching how Log4C initializes itself, how it looks for and handlesconfiguration files and environment variables, how it intializes and configures internal objects and forlog time events like rollover events in the rollingfile appender, for example.

Also keep in mind that using the Log4C configuration file allows you complete control over whichappender and layout your categories use. So if you are having problems with some specific appenderor layout you can easily change those to compare the behavior with other appenders or layouts.

8 Deployment Experience with Log4C

This idea of this section is to gather any deployment experience or tips with deploying Log4C inprojects.

8.1 Blame it on the logger

Here is some experience we had integrating Log4C into a new project. By it's nature a new project hasbugs and also makes heavy use of logging to help with the development and debugging phase. What Iobserved was that on at least four occasions our system was crashing with a stack that led to someLog4C code. In none of these cases did the root cause turn out to be Log4C. After the fourth timedebugging someone else's code I thought it worth while to look at the bugs to understand why therewas a tendency to have the core stack terminate (misleadingly) in Log4C. Here are some observationsthat help explain that:

logging messages is relatively expensive--so once you turn on, especially heavy logging, thenyour code spends a lot of it time in there. It would be interesting to run a collector/analyzer toquantify that. So, in the case where there is general memory corruption in your system it isquite likely that when it does crash, it will be in your logging code. This accounts for 2 of the 4bugs I mention that crashed Log4C

Often folks will put debug log statements (good!) without ever testing them (bad!). So someerrors of the form of a mismatch between the format string and the parameter list can easilyarise. These bugs at least have the merit of being easy to spot once they do occur. That wasone of the 4 bugs.

Enabling logging calls obviously triggers an extra depth in the call stack--in the case of Log4Cusing the stream appender this can be up to 10 deep by the time you hit the base write() call towrite to the file. So watch out if your stack size is small, especially if you are on a 64 bit

system (where you double the memory consumed by pointer types) and especially if your codeis already stack hungry. This of course describes exactly the scenario of the 4th bug thatcrashed Log4C. For information, on HP-UX, where we saw this problem, the default stacksize for threads is 64K, which is pretty small compared with defaults of 1M on Solaris andWindows and 2M on Linux.

Another Log4C integrator shared this relevant experience:

on some Operating Systems there will be a maximum file size limit. Iflog4c gets deployed in a production environment and no one watches androtates the log files, a crash and stackdump eventually results from thestandard C library routines that write to a file. This stack dump alwaystraces back through log4c and log4c can get blamed as the cause of thecrashes. Use rotating file appenders or tell your operations team to puttheir own rotators on the files.

8.2 Preparing to move logging from development into a productionenvironment.

At some point your project will actually be ready to ship and run. Here aresome considerations as you migrate logging from a development role to anoperations role. Operators will need to be able to debug problems without recompiling code.This is the most critical difference in how logging will be used once yourcode ships. Here are some tips.

1. Use the configuration file for configuration of categories and appenders, not the API.Operators will want do things like add additional appenders, redirectcategories to different appenders etc. They will not be able to justrecompile the code they way you do as a developer. If the configuration isdone by API then they cannot change anything. If the configuration is doneby the log4crc file then they can change things as needed.

2. Plan ahead for runtime debugging. Remember that once it ships, youwon't be able to just recompile to change or add logging. Make careful useof the different log levels. Keep use of the ERROR level down to theminimum, and use WARNING, INFO, and DEBUG as you get to messages that aremore verbose and less related to known bugs or problems. But don't hesitateto put in lots of DEBUG statements. Your application will be run at ERRORlevel in production until it has a problem. Picture yourself on the phonetelling someone how to change the level to DEBUG in the log4crc and they arein a different city than you. If you have lots of extra DEBUG statementsavailable, it can mean the different between finding the problem just bylooking at logs, or having to reproduce the entire bug from inputs on asystem that simulates production. The first is quick, the second caninvolve many days of setup before you can even properly simulate theproblem.

3. Use a logging hierarchy wisely to aid in pinpoint debugging.Say that you have shipped several program that work together on and endsystem. You can choose a logging hierarchy that makes it easy and flexibleto manage your logging. Say your company name is initech. And that youhave three applications that work together. And in each of theseapplications you have input handling, business logic, and database accesslogic. By using a hierarchy for your categories you can get great runtimeflexibility."company.program.subsystem"Using this you would have logging to categories likeinitech.prog1.dalinitech.prog2.blogicinitech.prog2.ilogicThis allows tremendous runtime flexibility so that operations can manage thelogging in many different ways without asking for recompiles and changes toyour code. In the above example the top level category "initech" would bethe main logging and would remain at ERROR level. If program 3 was having alot of difficulties and crashes, an additional logging would be addedmapping category initech.prog3 to a separate file at DEBUG level until theproblems could be solved. If program 2 was extremely slow and databaseaccess was suspected, logging could be added to category initech.prog2.dalat DEBUG level and put into its own file.

9 Referenceshttp://log4c.sf.net/ This SourceForge site is now the main log4c site.http://expat.sourceforge.net/http://gcc.gnu.orghttp://www.sunfreeware.com/http://hpux.connect.org.uk/Log4c developer mailing listNice 'How to get developing with log4c' articleLog4jPantheios - "The C++ Logging Sweetspot"http://sourceforge.net/projects/log4cpp/