Blog Moved To http://weblogs.asp.net/meligy

Last Tuesday night witnessed a relatively new kind event held in Microsoft Egypt building @ smart village. It was said to be "announcing dotNETwork user group" which I thought was some Microsoft Egypt initiative for a new offline user group model of developer community. I was so excited about so, and it was even better!

The Idea

The dotNETwork group is just am offline user group that's done right. It just happened to have Microsoft Egypt host the group gathering, as this is the hardest issue to maintain. Also, Microsoft name made the number of comers reaches over 100 attendee (not an accurate number too!). Note that the gathering/event speaker, Steven Forte, was invited by the group. not by Microsoft :).

This is the real start for offline user groups in Egypt I believe. Before, we used to have smaller user groups here in SilverKey and in ITWorx, we in silverKey started the concept of a public event last year, and recently ITWorx followed with their own event, but a wide spread user group only existed online, like MiddleeastDevelopers yahoo group. This dotNETwork group is the first BIG offline user group, and I'll be proud t try to join this effort.

The Day, Session 1/2: SQL Server 2008

Steven in general is one of the most important speakers in Egypt, having attended almost if not all the previous Microsoft MDC events. everybody already knows him as a superior SQL Server guy who (due to his work nature) can also talk well many other technical topics.

The first session was on SQL Server 2008. According to Steve, this will be launched in the launch event February 29, 2008 although will be released around July! He went on categorizing the new features for SQL Server 2008 as follows:

The enterprise data features, which are more for admins than developers, include the ability to do hot swapping of hardware, even processors! More datat types (which he got back to later), and various reporting and analysis services features, various BI areas. Working on the backup system that's currently slow compared to 3rd party tools that even create way smaller backups and have better options (selective backups).

Also, one other category is predictable performance. Steven mentioned how important the performance show plan is, and went to tell about the "query parallelism" (running the same query twice on different processors or so, and returning the value of the one that's done first), "resource governer" which works on deprioritizing certain queries to allow others to run faster. I understood you can choose prioritization based on some resource pool options like customized connection string, server load, user, time or day,....

Stephen mentioned enhancements to the SQL Server workbench, the reporting of resource over/under utilization, so that an admin may choose to do dynamic allocation for memory, processing, etc. He stated that if you have multiple processors, you can allocate a certain processor to specific SQL tasks.

Speaking of "beyond relational" features, he also mentioned new data types like FileStream, and some essential features like integrating the fullText search with the table itself (It used to be so painful to do DB replication because of missing this before :D), filtered indexes, ability to store "hierarchical" data as hierarchical in your DB, DATE and TIME specific data types in addition to DATETIME, the TIME type is accurate down to nanoseconds; Geometry data type allowing saving geographical information and therefore enabling you to write geography specific where conditions on these :D :), when retrieved in a .NET DataSet or so, this is a string that's in the same format that VirtualEarth uses, if you want to consider integration ;) ;). Also, a DateTimeOffset type which is equivalent to TimeSpan in .NET. Theoretically, this allows you to do better operations on the dates/times, but, the subtraction demo didn't work for this CTP. Stephen asks you to blame Bill Gates :) :) :).

More enhancements were pretty interesting as well, just being able to write "+=" in SQL is sure great :D (it seriously is, especially when promised to work with ALL data types). Another interesting feature he demoed is being able to use "table valued" parameters as SPROC parameters. This is like being able to send a parameter which is actually a series of items each of a complex data type (tupple), just like tables are, and being able to use normal insert,... and other table operations on these (Much like sending temporary table to the SPROC, if I got it right). The most interesting thing is that if you are using .NET 3.5 (ADO.NET 3.5), you can pass LINQ results or DataTable inside a DataSet as table valued parameter!!

Also, the new MERGE feature is pretty magical, especially for dataware housing systems.you can now merge data of different cells in certain column based on whatever criteria you want, or just using the first space in the value of the shortest cell, if that's string type and you are having no condition/pattern. It's just a pattern recognition thing, so, it works for all UTF languages (We're Egyptians you know :D). Actually, this is not the only data ware housing and SSIS features and enhancements. There's also in the BI Workbench some interesting stuff having Fuzzy Lookups and some enhances to cubes (New designer tools, WIZZARDS, Stephens laughs!). The last features he mentioned were being able to use reporting services for native word rendering, note that this means the report rending itself, not just manual exporting, of course with special designer support, and being able to write C# when working with reporting services as part of integration services.

The session was more than great as it had pretty enough information for a "sneak preview" session. MAN, this is my favorite standard for a session, not any less, and Stephen is one of the few who match this standard when we come to SQL :) :) :).

The Day, Session 2/2: ASP.NET Design Patterns: MVC + Database Design Patterns: SCD

MVC Pattern

Starting to talk about many books available for design patterns, Stephen went back to what design patterns are and where they came from. He said they are not as hard as reported by book publishers :D. Of course he defined them as documented general ways to solve common problems (like getting data from db and presenting them on a web page). He also mentioned a situation he had himself with a new Egyptian developer who joined his team recently. The solution hey working on was designed using the MVC pattern, which the developer knew, this made the conversation only goes about JUST the business logic, because the pattern itself is known and used 100 times already, Stephen said.

The history of design patterns was the most interesting part of the session to me. How that this all started in architecture, with European city buildings planning and how the term "design patterns" while planning the city of New York afterwards!!! There was a book on that that SmallTalk guys read and made the first software design patterns starting with it. As SmallTalk is not the language most people are actually using, the Gang Of Four took the SmallTalk work/book (with giving credits and everything :D) and build their own famous reference on. Stephen reminded us that after all, design patterns are just ways to speed up development by preventing us from "re-inventing the wheel". Still, they're just the blueprint/framework of the solution, not the solution itself.

Getting to MVC in particular (with ASP.NET), Stephen said it's one of the oldest (since the days of the SmallTalk work), and it works for having multiple views of the same model. The model is what your DB represents for example, the view is the UI, following the example, the controller is the code you write to show the model on the page.

Actually, this is when I started to not feel OK!! To me, this doesn't sound much different than the MVP pattern which MSDN documentation says it's the model for WebForms Code Behind in ASP.NET. Actually, Stephen made a joke that you shouldn't believe such a documentation when it comes to concepts not specific products, and I thought, most of the guys sitting there seemed to not even know what design patterns are. The beauty of this specific session is that it's really new in public event sessions in Egypt :S, so, we just continued...

Continuing with Stephen, he went on talking about the Model, A model is a response to instruction of change of state (data , user interaction). You know, a search text you enter in a text box is a change in state, that makes the already displayed results (Model) before performing the search become out of sync.One more reason why you shouldn't mix SQL with UI (write SQL in button click or even use SqlDataSrouce) is that you'd be mixing View and Model. One way to do retrieval is using Typed DataSets, Stephen said, the way he use it though, he explained, writing stored procedures in the DB and use them from business classes with static methods (Yeah, I think this part is personal preference too!).

Speaking of the View, he said that this is the easy part. It just all about the display of the data.  Maybe you need to set what to display or hide based on authentication, authorization, but you can use the template view features built into ASP.NET.

Now, we came to the Controller.this is the part that intersects with both the View and Model. When it needs to get some data, it informs the model to get it from there, when it has just "got" the data, it informs the View to display it. At this point of the talk, he suggested having a "Page Controller" (A base class for all pages to inherit from instead of directly inheriting from System.Web.UI.Page) to hold common helper functionality in (I liked him when he said that cause I've been taking that approach always, until I came to SK and found them also accidentally using it as well :D).

Anyway, later he went on implementation issues, how that was hard to implement earlier if you write the code the code in the page code behind, and data access layer issues with he Model as well. He demoed an implementation using ObjectDataSource having the logic and data access in different assembly than the web project, at this point we came to the biggest word and mean reasoning of the MVC pattern and such separation in general, which is TESTABILITY (you know, an MVC session couldn't exist without that word :) :) ).  Explaining this, he clarified "Modularity: Separation between assemblies", and how that this the key for testability, especially that you cannot simulate the Page Life Cycle to do your unit tests.

As one of the biggest issues mentioned nowadays with the new MVC framework for ASP.NET, he went to discuss ViewState. The good of keeping state in a stateless environment like web/HTTP and the evil of page size. He said you can intercept your View (page) to handle storing it on the server and you can do that in your page controller to make it site wide, the audience was very enthusiast about that part, and we took really enough (more than enough to me :D ) time discussing the alternatives and server techniques, etc....

Finally, he named the ASP.NET MVC framework. He didn't say much about it. He just said that Microsoft MEANS IT this time when they say it's should NOT be used in production yet. This is good to get as explicit as this to a naughty boy like me :) :D :D.

Afterwards, we turned to DB patterns. I didn't feel that MVC part was enough and wondered why splitting the session at all. I felt that part was quite little by then, although now I fell it had pretty good amount of information (my hands started to feel tired already from writing :D). I don't know, maybe it's just my high acceptance standard or the fact that he didn't talk about the MVC Framework (which was just released quite close to the session time, he has ALL excuse not to mention it of course), or both, I don't know. It was nice to bring the topic to public talks in Egypt anyway, and he did really well explaining the topics he opened :).

DB Patterns: SCD

Looks like Stephen meant to make this session all about non-opened topics in public Egyptian events :). He is a frequent MDC speaker and he knows there're VERY few other public tech. events in Egypt (Actually, just our SilverKey DemoDay and very recently ITWorx CuttingEdge Club), so, he knows what topics are not on the table and did great bringing them up :) :) :).

The topic of the second part of the session is something that matters most to dataware housing work. He first said that we all are using transactional deign patterns in DB design, you know, the way we use normalization and depend on DB relations.

The design pattern he wanted to show us this time was called "Slowly Changing Dimensions" or SCD. This is yet a different  data model from the transactional one that is used mainly with reports. The idea is simply using flat tables (do I hear somebody yelling "denormlization"?) for storing data. This is good for reports as it is a kind of physical transformation of data, not their views.

The real world situation example he brought to us from his past work experience was a situation in a site which is like news one, he chose to not save the pages with same elements in the same table, instead, he create a table for each page of these. That was in a separate production DB not on the development/staging or even data entry server. This is because the area where this has a benefit is when you need to bring the online transactional data. That's why he called this way "Published SCD".

Working with this way, having two databases one of them for transactional and another for published SCD data brings new challenges though. You need to always keep the two databases in synch.  You can use OLTP or so to do it, but after all, you need to remember that this model is not suitable for time sensitive data (based on time series, for the example he brought up. that was one day).

There are common ways to do synch actually, you can relay on SSIS packages, by the time of the sample story he told us (not very close :) ), they used DTS to do it. He showed us a DEMO on how to automate such sync after every database INSERT, and mentioned that you are likely to get harder challange when it comes to editing (Aka UPDATEing) data on the transactional DB and then finding which part on the published DB matches the UPDATEd data and actually UPATE it.

Later, he went on the value of such. The simplest example is that Amazon and sites of that size are of course using it in their work.

The last demo was doing partitioning for table Index. He mentioned the facts about a clustered and non clustered index and the fight that come to that between developers and DBAs usually for "when to create an index" :D :).

In general, that's as I said, some topic an average developer of even 5 years experience might not have dealt with, and would be too worried and frightened to such a different way of thinking, as most developers have grown up on valuing normalization and speaking of performance without thinking twice. Having heard it from such a domain expert was very useful to everybody there I guess, and sure to me as well :) :) :).

Conclusion

If I were you, I wouldn't have reached this line at all, or at least reached it by skipping all the story above :D, so, if you have read it all (I doubt!), you really deserve a BIG thank you for this :) :) :).

The last thing I want to say in this context is that having a started offline user group not just a public independent/semi-independent event is a step that was waited and worked on for so long. I expect way more to come from this user group and plan to join and share in that great movement myself. I'm as excited as I was with the global ALT.NET movement and plan to do my best to bring the ALT.NET culture to the dotNETwork group. Stephen has already gave me a great help by his excellent selection of topics for his second session and his usual expected outstanding performance in delivering them, so, that was a great start for a great group.

Be with us and keep tuned for more work to come. The dotNETwork group plans to have more events soon and other activities as well. The almost confirmed news is that it'll have another similar event monthly (at least next month), with native Egyptian speakers most likely, to make the move more intimate and more effective/interactive, so, watch the group website and my weblog for updates on this. You may also want to join the Yahoo or facebook groups (see details on the site).