In my last post I did show how you can consume much less memory if you read values into a list which have often the same value. The IndexedList class uses internally a reference cache similar to the String.Intern method. But what if we do read value types such as int, enum, int64, … into a list which do repeat themself also very often? Can we store an enum which can have four values in less memory than an integer? Sure we can. Lets have a look at the following data Row Values 0 1 1 2 2 3 3 1 4 2 ......
When you want to store millions of strings in a list you will notice in your memory profiler that the biggest memory consumers strings and lists are. If you e.g read many lines from a file with repetitive data inside it you can use a reference cache to consume the actual memory for the same string only once. The CLR offers String.Intern to do exactly that but the string reference cache is cross AppDomain static which is mostly never a good choice. Where could you use this? If you parse log files ......
Last time we did have a look into the issues you get if you create many class instances. The more class instances you have the more complex is your object graph which means more work for the GC. If the GC has more work there is less CPU time for your application left which does make it slower. An easy way to fix this issue is to create a struct which is a value type which is copied by value and does not add complexity to your object graph. Common wisdom suggests that your structs should not get bigger ......
If you want to write scalable applications with a high throughput you need to take care of good data structure design to make your application scale. If you want to read for example 200 MB data from a file and process it you can finish it in 5 or 30s. To be on the 5s side it is educational to see how fast you actually can get. To measure the raw disc performance I do read the file into a pre allocated byte array. [Test] public void ReadFile_Store_In_ByteArray() { var sw = Stopwatch.StartNew(); int ......