Geeks With Blogs

News View Michael Stephenson's profile on BizTalk Blog Doc View Michael Stephenson's profile on LinkedIn
Michael Stephenson keeping your feet on premise while your heads in the cloud

In this post I wanted to explore the options around building a Windows Azure Service Bus appender for log4net (sample download). This would allow me to publish logging events outside of my application to an internet scale messaging system which can then provide opportunities to centrally store or process messages from all of my applications. This article is part of a few articles I will write while playing with the architectural idea of the centralized logging based on Azure Service Bus but in this case specifically focusing on a log4net appender that can be used by custom applications.

At this point I am going to assume you have some familiarity with log4net and Windows Azure Service Bus but only a basic level should be required.

To begin with in this article I want to start by writing an appender which I will plug into log4net. This will allow me to have a custom widget which I can use to process logging events from log4net and where I can decide what I want to do with them. This is the normal log4net approach and when you configure a logger through your .net configuration file you would normally specify a set of appenders which will process any log events your application creates. By default log4net comes with a set of appenders which you can configure out of the box but also I can write my own by deriving from the AppenderSkeleton class in log4net. This means that I can declare some properties to receive configuration data for my appender and then override a few methods and have quite an easy way to implement my appender.

Once I have written my appender I just need to make sure that the dll containing my appender is in the application bin directory and my configuration file references my appender then I should be good to go.

 

Appender Configuration

Below is an example of the appender configuration you would use for the ServiceBusAppender in your log4net configuration file.

<appender name="AzureServiceBus-Logging-Appender" type="MyLog4net.Azure.ServiceBus.ServiceBusAppender,MyLog4net.Azure.ServiceBus">

<ConnectionStringKey value="Log4netServiceBusConnection"/>

<MessagingEntity value="LoggingTopic"/>

<ApplicationName value="MyLog4net.ConsoleApplication"/>

<EventType value="LoggingEvent"/>

<CorrelationIdPropertyName value="CorrelationId"/>

<Synchronous value="false"/>

<filter type="log4net.Filter.LevelRangeFilter">

<levelMin value="INFO" />

<levelMax value="FATAL" />

</filter>

</appender>

 

There are a couple of key properties here:

  • ConnectionStringKey

This property is a string which points to a connection string you have declared in the config file. The connection string you have declared will be a connection string for a Windows Azure Service Bus Namespace

  • MessagingEntity

This is the name of a queue or topic in the Windows Azure Service bus which you would like to publish the message to

  • ApplicationName

The application name property will be used to indicate on the log message which application published the message.

  • EventType

The event type is used to indicate what type of event it is. An example of where you might use this is if you have multiple loggers using different instances of the appender which are configured differently, then you might use this to indicate the types of events being published. An example might be publishing audit events and standard logging events.

  • CorrelationIdPropertyName

Normally as you process flows across components you would include in this flow some context to relate execution in different places. An example of how you might do this would be to create a unique id when the user first clicks a button and then flow this through a web service call and into another component. From here you would usually use the log4net ThreadContext to set these variables to be available to each component as they begin executing. In this case the correlation id is a useful property to allow us to relate log messages from different components. In the configuration the CorrelationIdPropertyName allows you to specify the name of the log4net property that holds the CorrelationId in this component. This correlation id is then used as a specific element on the published log event.

  • Synchronous

The synchronous property allows you to control if an event is published on a background thread or not. There is a performance gain in publishing on a background thread but the trade of is you can't as easily respond to errors. For normal logging events you're probably happy to swallow the error and continue, but perhaps for audit events you might not want to do this.

 

In addition to the specific configuration properties the rest of the normal log4net stuff pretty much applies. As an example you can use the Appender filtering mechanism to control which messages are processed by the appender.

 

What happens inside the appender?

Inside the appender the code will start by creating an instance of the AzureLoggingEvent which is a type which will encapsulate the data I want to publish and setup the data as appropriate. The appender will then decide if you want to publish synchronously or asynchronously and then call the Internal Append method. In the internal append method we will serialize the AzureLoggingEvent to a JSON string to be the body of the published message. We will then setup the following context properties on the Azure Service Bus Brokered Message:

  • ApplicationName
  • Username
  • MachineName
  • Level
  • EventType

These properties will allow you to do some routing if you have published the message to a topic. Examples of routing uses could include:

  • Creating a higher processing priority for certain messages (perhaps Errors need to be processed quicker)
  • Routing for specific messages to also go to an operator who is looking for activity for a specific user

I'm sure you can think of other examples.

The Appender will then create a MessageingFactory, MessageSender and the other Azure Service Bus objects as required and will send the message. A quick point to note is Ill probably have a look at some point to include the transient error handling block but for now I have just included a basic retry mechanism (very basic).

 

Using the Appender

To illustrate the user of the log4net appender I've included a sample to show you how it can be used. In this sample I want to create three log4net loggers:

  1. ApplicationLogger

The application logger is a basic logger which will just use the console appender and event log appender to record information locally. This is just application diagnostic info.

  1. GlobalLogger

The global logger is intended to be used when the application needs to publish information to the Azure Service Bus for general log events. This logger will process log events asynchronously

  1. AuditLogger

The audit logger will send out audit events using the Azure Service Bus appender but will do it synchronously.

 

The configuration for setting up the appender looks like the below example:

Step 1 - Loggers

The configuration to setup the loggers is below

<root>

<level value="ALL"/>

<appender-ref ref="EventLogAppender"/>

<appender-ref ref="ColoredConsoleAppender"/>

</root>

<logger name="GlobalLogger" additivity="false">

<level value="ALL"/>

<appender-ref ref="EventLogAppender"/>

<appender-ref ref="AzureServiceBus-Logging-Appender"/>

</logger>

<logger name="AuditLogger" additivity="false">

<level value="ALL" />

<appender-ref ref="AzureServiceBus-Audit-Appender"/>

</logger>

There are 3 loggers as described above with the GlobalLogger and AuditLogger being the ones which will send messages to Azure Service Bus.

 

Step 2 - Appenders

The configuration for the appenders is below:

<appender name="AzureServiceBus-Logging-Appender" type="MyLog4net.Azure.ServiceBus.ServiceBusAppender,MyLog4net.Azure.ServiceBus">

<ConnectionStringKey value="Log4netServiceBusConnection"/>

<MessagingEntity value="LoggingTopic"/>

<ApplicationName value="MyLog4net.ConsoleApplication"/>

<EventType value="LoggingEvent"/>

<CorrelationIdPropertyName value="CorrelationId"/>

<Synchronous value="false"/>

<filter type="log4net.Filter.LevelRangeFilter">

<levelMin value="INFO" />

<levelMax value="FATAL" />

</filter>

</appender>

 

<appender name="AzureServiceBus-Audit-Appender" type="MyLog4net.Azure.ServiceBus.ServiceBusAppender,MyLog4net.Azure.ServiceBus">

<ConnectionStringKey value="Log4netServiceBusConnection"/>

<MessagingEntity value="AuditQueue"/>

<ApplicationName value="MyLog4net.ConsoleApplication"/>

<EventType value="AuditEvent"/>

<Synchronous value="true"/>

<CorrelationIdPropertyName value="CorrelationId"/>

<filter type="log4net.Filter.LevelRangeFilter">

<levelMin value="INFO" />

<levelMax value="FATAL" />

</filter>

</appender>

In the appenders you can see how the settings indicate each appender will send to different queues/topics in Azure Service Bus.

 

Step 3 Initialize Log4net

In the sample code when the console application starts I begin by setting up my loggers:

 

static void Initialize()

{

log4net.Config.XmlConfigurator.Configure();

ApplicationLogger = log4net.LogManager.GetLogger(typeof(Program));

AuditLogger = log4net.LogManager.GetLogger("AuditLogger");

GlobalLogger = log4net.LogManager.GetLogger("GlobalLogger");

 

}

Step 4 Setup my Correlation ID

I then setup my correlation id property using the below example. Note that in this sample it's a really basic implementation but if you were flowing the correlation id across components then this would be a little more involved.

log4net.ThreadContext.Properties["CorrelationId"] = Guid.NewGuid();

 

Step 5 Adding the Application Code

After this I will execute some code to log different messages to different loggers in my application.

 

ApplicationLogger.Debug("Program starting");

 

for (int i = 0; i <= 1000; i++)

{

ApplicationLogger.Info("This is info");

GlobalLogger.Info("The application has started");

}

 

//lots of code.....

ApplicationLogger.Debug("Ive done something");

 

try

{

ApplicationLogger.Debug("About to audit");

AuditLogger.Info("About to do something important");

ApplicationLogger.Debug("Audit complete");

//.... doing some work

throw new ApplicationException("This is an error");

}

catch(Exception ex)

{

//... log error locally

ApplicationLogger.Error("There was an error", ex);

//... send error to central error system

GlobalLogger.Error("There was an error", ex);

}

Console.ReadLine();

 

This results in a bunch of messages going to log4net. Some of them will be filtered out because the log4net configuration says we aren't interested in them and some will be sent to Windows Azure Service Bus.

Step 6 Configuring Azure Service Bus

The key thing here is that you can configure it however you like. As long as it matches the configuration of the MessageEntity and Connection string in the log4net configuration it will work fine. For this demo I have created the following queue setup:

 

There is an audit message queue which all events from the audit logger will be sent to. There is also a logging topic for the general logging events. On the loggingtopic I have created some subscriptions to demonstrate how you could filter messages. The first subscription is an all events subscription using the 1=1 routing rule so that it will get all messages. The Just Errors subscription has a rule of Level='Error' so that it will only get errors. You can create filter rules based on any of the properties we earlier published as context properties on the service bus message.

You can see that at this point its quite easy to configure a flexible routing of events depending on what you want to do with the events once they reach service bus.

You are now ready to run the sample and publish logging messages.

 

What does the message look like?

After running the demo you will see that the queue or subscription will have a lot of messages on it. You can use Service Bus Explorer to take a look at the details. The below picture shows you an example of one message.

 

You can see here that there are the context properties I described on the message in the Message Custom Properties box. These are what you can filter on. The message text is also the JSON string I mentioned earlier. It will look something like the below:

{

"Properties":

{

"CorrelationId":"5f4c5936-674b-474a-9b2a-c3d9002cc070",

"log4net:Identity":"",

"log4net:UserName":"CSCMAIN\\Michael",

"log4net:HostName":"CSCMain"

},

"CorrelationId":"5f4c5936-674b-474a-9b2a-c3d9002cc070",

"EventType":"LoggingEvent",

"MachineName":"CSCMAIN",

"ExceptionMessage":"This is an error",

"ThreadName":"10",

"UTCTimeStamp":"2014-01-02T06:22:59.1615766Z",

"TimeStamp":"2014-01-02T06:22:59.1615766+00:00",

"UserName":"CSCMAIN\\Michael",

"RenderedMessage":"There was an error",

"Domain":"MyLog4net.vshost.exe",

"ExceptionString":"System.ApplicationException: This is an error\r\n at MyLog4net.Program.Main(String[] args) in c:\\Users\\Michael\\Documents\\Visual Studio 2013\\Projects\\MyLog4net\\MyLog4net\\Program.cs:line 37",

"Identity":"",

"Level":"ERROR",

"ClassName":"MyLog4net.Program",

"FileName":"c:\\Users\\Michael\\Documents\\Visual Studio 2013\\Projects\\MyLog4net\\MyLog4net\\Program.cs",

"FullInfo":"MyLog4net.Program.Main(c:\\Users\\Michael\\Documents\\Visual Studio 2013\\Projects\\MyLog4net\\MyLog4net\\Program.cs:44)",

"LineNumber":"44",

"MethodName":"Main",

"LoggerName":"GlobalLogger",

"ApplicationName":"MyLog4net.ConsoleApplication"

}

We now have the capability to have some fairly useful logging information centrally published with the ability to process it.

 

What might I do with this now?

Now that we have messages on Azure Service Bus queues the most obvious thing to do with them might be to have a Worker Role or similar which polls the messages and then saves them to a database for operational analysis. I'll probably talk about that in a future post but for now the messages are somewhere safe and somewhere you can do some interesting stuff with them.

 

Download Sample

The sample can be downloaded from the following location:

http://cscblogsamples.blob.core.windows.net/publicblogsamples/MyLog4net.zip

 

Conclusion

As you can see it was really easy to implement this appender. Feel free to get the sample and give it a try or modify it if you choose. The one caveat is that this was only some R&D so I haven't really tried this in a production setup to see how it performs.

 

 

 

Posted on Thursday, January 2, 2014 5:15 AM Azure Service Bus | Back to top


Comments on this post: Log4net Azure Service Bus Appender

# re: Log4net Azure Service Bus Appender
Requesting Gravatar...
Thanks for posting this sample. It's a great start for building a Log4Net adapter for Service Bus.
I'm trying to Sign and register the MyLog4net.Azure.ServiceBus into the GAC.

When doing this, I now get the following error:
log4net:ERROR Could not create Appender [AzureServiceBus-AuditAppender] of type
[MyLog4net.Azure.ServiceBus.ServiceBusAppender,MyLog4net.Azure.ServiceBus]. Reported error follows.
System.IO.FileNotFoundException: Could not load file or assembly 'MyLog4net.Azure.ServiceBus' or one of its dependencies. The system cannot find the file specif
ied.
File name: 'MyLog4net.Azure.ServiceBus'
at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError,
Boolean ignoreCase, Boolean reflectionOnly, StackCrawlMarkHandle stackMark, IntP
tr pPrivHostBinder, Boolean loadTypeFromPartialName, ObjectHandleOnStack type)
at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError,
Boolean ignoreCase, Boolean reflectionOnly, StackCrawlMark& stackMark, IntPtr pP
rivHostBinder, Boolean loadTypeFromPartialName)
at System.RuntimeType.GetType(String typeName, Boolean throwOnError, Boolean
ignoreCase, Boolean reflectionOnly, StackCrawlMark& stackMark)
at System.Type.GetType(String typeName, Boolean throwOnError, Boolean ignoreCase)
at log4net.Util.SystemInfo.GetTypeFromString(Assembly relativeAssembly, String typeName, Boolean throwOnError, Boolean ignoreCase)
at log4net.Util.SystemInfo.GetTypeFromString(String typeName, Boolean throwOnError, Boolean ignoreCase)
at log4net.Repository.Hierarchy.XmlHierarchyConfigurator.ParseAppender(XmlEement appenderElement)
Left by BTSLogging on Nov 26, 2014 6:24 PM

# re: Log4net Azure Service Bus Appender
Requesting Gravatar...
Is there a Windows Service Bus appender? I.e. non-Azure for a regular service bus.
Left by Aaron Shumaker on Feb 24, 2015 7:00 AM

# re: Log4net Azure Service Bus Appender
Requesting Gravatar...
Best app ever please try to use..
snapchat app download
Left by snaphat apk on Sep 22, 2016 6:35 AM

# re: Log4net Azure Service Bus Appender
Requesting Gravatar...
But in my opinion, this app makes batteries wasteful because we have to open it simultaneously with Pokemon Go. Features of this application is quite complete. We can set notifications for pokemon that we wanted.
pokemesh
APP
Left by Alexandr Lukin on Dec 31, 2016 1:52 AM

# re: Log4net Azure Service Bus Appender
Requesting Gravatar...
Showbox is a fabulous and remarkable application to acquire entertainment and recreation just right at your fingertips. Showbox is a wonderful approach to access a wide range of multimedia items that can be executed both on mobile or PC. Showbox for PC .
Left by snow on Jan 19, 2017 5:49 PM

# re: Log4net Azure Service Bus Appender
Requesting Gravatar...
Showbox App is not like other Streaming applications. Showbox
allows it’s users to watch and stream Movies and Serials up to 1080P HD. You can select the quality of the serial ranging from 360P to 1080P which is the maximum. z
Left by Alexandr Lukin on Jan 28, 2017 4:55 AM

# re: Log4net Azure Service Bus Appender
Requesting Gravatar...
But in my opinion, this app makes batteries wasteful because we have to open it simultaneously with Pokemon Go. search [url=google.com]google.com[/url] google.com
Left by google.com on Mar 17, 2017 8:02 PM

# re: Log4net Azure Service Bus Appender
Requesting Gravatar...
Boolean ignoreCase, Boolean reflectionOnly, StackCrawlMark& stackMark, IntPtr pP
rivHostBinder, Boolean loadTypeFromPartialName)
at System.RuntimeType.GetType(String typeName, Boolean throwOnError, Boolean
ignoreCase, Boolean reflectionOnly, StackCrawlMark& stackMark) http://dienlanhphucthinh.com/bao-hanh-may-giat/trung-tam-bao-hanh-may-giat-tai-tp-hcm.html
Left by google.com on Mar 17, 2017 8:04 PM

# Xbox 2 Release Date
Requesting Gravatar...
When it comes to gaming consoles, there is always a big fight between Sony's Playstation and Microsoft's XBox. Now this rivalry is going to increase even more because Microsoft is getting ready to launch it's next console, the Xbox 2. You can check out all the details from my site here Xbox 2 Release Date
Left by Tom Parker on May 09, 2017 9:59 PM

# homefrik
Requesting Gravatar...
stand alone atop of our listing of regular cleaning tasks. https://homefrik.com nastiness might live atop the mattress itself.
Left by dennie on Sep 12, 2017 11:48 PM

# AndroDumpper
Requesting Gravatar...
your wifi or network for any kind of susceptabilities. AndroDumpper for PC Desktop Os which is based on Android.Although it is a really challenging.
Left by martin on Nov 24, 2017 9:44 PM

# re: Log4net Azure Service Bus Appender
Requesting Gravatar...
https://ewww-hotmail-it.com/www-hotmail-it-accedi-italia/your wifi or ntwork for any kind of susceptabilities.
Left by Freezen Tsd on Dec 28, 2017 12:27 AM

# re: Log4net Azure Service Bus Appender
Requesting Gravatar...
GREAT hotmail.it
Left by Freezen Tsd on Dec 28, 2017 12:28 AM

# re: Log4net Azure Service Bus Appender
Requesting Gravatar...
4Liker Facebook is an excellent Android App that is created to offer have sort for your 4liker APK status, photos, as well as also for your follower web pages.
Left by elina on Jan 09, 2018 5:50 PM

# re: Log4net Azure Service Bus Appender
Requesting Gravatar...
From reading this post, I have got some idea about the Windows Azure Service Bus appender for log4net. You have shared the configuration and the properties of the application quite clearly through this article. Thanks for the share. motorized wheelchair
Left by lionel on Jan 16, 2018 10:46 PM

# re: Log4net Azure Service Bus Appender
Requesting Gravatar...
Left by Snaptube For PC on Jan 21, 2018 7:02 PM

# re: Log4net Azure Service Bus Appender
Requesting Gravatar...
Actually, I was looking for a complete information on this topic for a long time and finally got it from here. get foot spa massagers for home.
Left by Lisa Nichols on Feb 23, 2018 10:29 PM

Your comment:
 (will show your gravatar)


Copyright © Michael Stephenson | Powered by: GeeksWithBlogs.net