Alois Kraus

blog

  Home  |   Contact  |   Syndication    |   Login
  133 Posts | 8 Stories | 368 Comments | 162 Trackbacks

News



Archives

Post Categories

Programming

Todays little program has a simple task. It should decode a hex bitmask into its original bits as hex values from which it was composed from. As bonus we can also parse an ETW manifest and print the corresponding keywords. Since most low level tools use hex numbers which have specific bits turned on I often want to know which things were actually enabled. This is useful to e.g. see which providers PerfView enables for .NET specific stuff.

If you check in PerfView the .NET Alloc checkbox I want to see what is enabled.

xperf -Loggers | findstr /i Microsoft-Windows-DotNETRuntime

".NET Common Language Runtime":0xc14fccbd

BitmaskDecoder.exe 0xc14fccbd C:\Windows\Microsoft.NET\Framework64\v4.0.30319\CLR-ETW.man

0x1             GCKeyword
0x4             FusionKeyword
0x8             LoaderKeyword
0x10            JitKeyword
0x20            NGenKeyword
0x80            EndEnumerationKeyword
0x400           SecurityKeyword
0x800           AppDomainResourceManagementKeyword
0x4000          ContentionKeyword
0x8000          ExceptionKeyword
0x10000         ThreadingKeyword
0x20000         JittedMethodILToNativeMapKeyword
0x40000         OverrideAndSuppressNGenEventsKeyword
0x80000         TypeKeyword
0x400000        GCHeapSurvivalAndMovementKeyword
0x1000000       GCHeapAndTypeNamesKeyword
0x40000000      StackKeyword
0x80000000      ThreadTransferKeyword

"Microsoft-Windows-DotNETRuntimePrivate":0x4002000b

BitmaskDecoder.exe 0x4002000b d:\privclr\ClrEtwAll.man


0x1             undocumented
0x2             undocumented
0x8             undocumented
0x20000         MulticoreJitPrivateKeyword
0x40000000      StackKeyword

 

If you on the other hand enable ETW .NET Alloc you get

BitmaskDecoder.exe 0xc16fccbd C:\Windows\Microsoft.NET\Framework64\v4.0.30319\CLR-ETW.man

0x1             GCKeyword
0x4             FusionKeyword
0x8             LoaderKeyword
0x10            JitKeyword
0x20            NGenKeyword
0x80            EndEnumerationKeyword
0x400           SecurityKeyword
0x800           AppDomainResourceManagementKeyword
0x4000          ContentionKeyword
0x8000          ExceptionKeyword
0x10000         ThreadingKeyword
0x20000         JittedMethodILToNativeMapKeyword
0x40000         OverrideAndSuppressNGenEventsKeyword
0x80000         TypeKeyword
0x200000        GCSampledObjectAllocationHighKeyword
0x400000        GCHeapSurvivalAndMovementKeyword
0x1000000       GCHeapAndTypeNamesKeyword
0x40000000      StackKeyword
0x80000000      ThreadTransferKeyword

The only difference between .NET Alloc and ETW .NET Alloc is that PerfView addtionally enables the high sampling rate (max 100 alloc events/s) with GCSampledObjectAllocationHighKeyword. If you want to record longer with less events you can use this mask and replace GCSampledObjectAllocationHighKeyword with GCSampledObjectAllocationLowKeyword to get 20 times less allocation events (max 5 alloc events/s).

This makes it easy to find out what other people think about good ETW settings if you have the manifest for the provider. If you have found the list of hex values you want to combine you can call

BitmaskDecoder.exe -encode 0x1 0x2 0x4 0x80000

Encoded value: 0x80007

to get the corresponding hex mask. You can use it of course also for other bit masks if you omit the ETW manifest file name.

Here is the code for this simple tool:

Update: Now it also supports keywords. Thanks for the suggestion Andre.

You can now also write

BitmaskDecoder.exe -encode C:\Windows\Microsoft.NET\Framework64\v4.0.30319\CLR-ETW.man  JittedMethodILToNativeMapKeyword GCHeapAndTypeNamesKeyword

The executable can also be found here.

using System;
using System.Globalization;
using System.IO;
using System.Linq;
 
namespace BitMaskDecoder
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                if (args.Length == 0)
                {
                    Help();
                    return;
                }
 
                string[] lines = null;
 
                if (args[0] == "-encode")
                {
                    if( args.Length < 2 )
                    {
                        Help();
                        return;
                    }
 
                    int skip = 1;
                    if( File.Exists(args[1]) )
                    {
                        lines = File.ReadAllLines(args[1]);
                        skip++;
                    }
                    ulong res = args.Skip(skip).Select(x => x.TrimStart(LeadingHex))
                                               .Select(str => DecodeKeyword(str,lines))
                                               .Aggregate((x, y) => x | y);
                    Console.WriteLine("Encoded value: 0x{0:X}", res);
                    return;
                }
 
                if (args.Length > 1)
                {
                    lines = File.ReadAllLines(args[1]);
                }
 
                string hex = args[0].TrimStart(LeadingHex);
 
                ulong hexNum = ulong.Parse(hex, System.Globalization.NumberStyles.AllowHexSpecifier);
                for (int i = 0; i < 64; i++)
                {
                    ulong mask = 1ul << i;
                    ulong hexMask = mask & hexNum;
                    if (hexMask != 0)
                    {
                        string value = String.Format("0x{0:X}", hexMask);
                        string searchValue = value + "\"";
                        bool bFound = false;
                        if (lines != null)
                        {
                            string curProvider = null;
                            for (int line = 0; line < lines.Length; line++)
                            {
 
                                string lineStr = lines[line];
                                if( lineStr.Contains("<provider") )
                                {
                                    curProvider = ExtractName(lineStr);
                                }
 
                                // we are after lines like <keyword name="GCKeyword" mask="0x1" message="$(string.RuntimePublisher.GCKeywordMessage)" symbol="CLR_GC_KEYWORD"/>
                                if (lineStr.Contains(searchValue) && lineStr.Contains("keyword"))
                                {
                                    Console.WriteLine("{0,-8}\t{1,-50}\t{2}", value, ExtractName(lineStr), curProvider);
                                    bFound = true;
                                }
                            }
                        }
 
                        if (!bFound)
                        {
                            Console.WriteLine("{0,-8}\t{1}", value, "undocumented");
                        }
                    }
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine("Error: {0} {1}", ex.Message, ex.StackTrace);
            }
        }
 
        static void Help()
        {
            Console.WriteLine("BitmaskDecoder 0xddddd [ETWManifest] or -encode [manifestfile] 0xddd 0xddddd keyword1 keyword2 ....");
            Console.WriteLine("\tDecode a hex value into its bits and print the bits as hex values.");
            Console.WriteLine("\tWhen ETWManifest file is present the file is parsed and the matching lines of the manifest with the keywords are displayed");
        }
 
        static ulong DecodeKeyword(string hexOrKeyword, string[] manifest)
        {
            ulong lret = 0;
            if (!ulong.TryParse(hexOrKeyword, NumberStyles.AllowHexSpecifier, CultureInfo.InvariantCulture, out lret))
            {
                lret = ulong.Parse(ExtractMaskValue(hexOrKeyword, manifest), NumberStyles.AllowHexSpecifier);
            }
            return lret;
        }
 
        static string ExtractName(string line)
        {
            return ExtractAttribute(line, "name");
        }
 
        static string ExtractMaskValue(string keyword, string[] lines)
        {
            string hex = lines.Where(line => line.Contains("<keyword") && line.Contains(keyword))
                              .Select(line => ExtractAttribute(line, "mask"))
                              .FirstOrDefault();
            if( hex == null )
            {
                throw new NotSupportedException(String.Format("The keyword {0} was not found in the manfifest", keyword));
            }
            return hex.TrimStart(LeadingHex);
        }
 
        static string ExtractAttribute(string line, string attribute)
        {
            return new string(line.Substring(line.IndexOf(attribute))
                                            .SkipWhile(c => c != '"') // search first "
                                            .Skip(1)                  // skip "
                                            .TakeWhile(c => c != '"') // take all until next "
                                            .ToArray());
        }
 
        static char[] LeadingHex = new char[] { '0', 'x' };
 
    }
} 

 

 

 



	
posted on Wednesday, March 25, 2015 7:09 AM

Feedback

# re: Which Keywords Has An ETW Provider Enabled? 3/25/2015 8:27 PM André
nice tool :) It would be nice if you can use BitmaskDecoder.exe -encode FusionKeyword JittedMethodILToNativeMapKeyword GCHeapAndTypeNamesKeyword C:\Windows\Microsoft.NET\Framework64\v4.0.30319\CLR-ETW.man to generate the keyword code.

# re: Which Keywords Has An ETW Provider Enabled? 3/26/2015 7:53 AM André
thanks for the update :) This will be added to my toolbox of helpful dev tools.

# re: Which Keywords Has An ETW Provider Enabled? 3/26/2015 4:08 PM Alois Kraus
Glad that you find it useful. Thanks.

Post A Comment
Title:
Name:
Email:
Comment:
Verification: