Mother of all Blog Posts

Building expertise in WPF / Silverlight

·         Core Programming Concepts: Declerative programming, Dependency properties, DataBinding, Commands, Triggers, VSM, Control templating, the inheritence model , XAML extensions and control templating

·         Transforms and animations - SVG experience helps

·         Comprehend the core differences in SL and WPF-  lack of Commands and Triggers (Behaviours and VSM are superior anyway), Calling different types of services (WCF, Sockets, Duplex, ADO.NET Data Services, RIA services) à naturally supports good architecture,

·         Silverlight 3 -  navigation , out of browser experience, popups etc

·         MSDN + Adam Nathan book are a great place to kick off

·         Petzold’s WPF 3D Book – A great resource for learning 3D in general – it really mixes well 3D maths with the core programmatic constructs of WPF 3D. There are 3 major limitations to WPF 3D: 1) Retained mode graphics have performance issues. 2) for a high level scene graph, the primitives are still primitive – wouldn’t be great if we had intrinsic data structures for rendering NURBS patches. 3) Point animation of a mesh vertex is on the heap (each interpolation is like writing to the same string object)  à requires annoying work arrounds.

·         Silverlight Animation book – collision detection, kinematrics, particle systems, etc. I like the way that the VSM is leveraged for cartoon like affects and the technique of programmatically manipulating the storyboard as a declarative timer.

·         Silverlight Toolkit / WPF Toolkit – especially DataGrid and Charts

·         Expression

o   Blend - how else would you template your controls. I feel pretty comfortable using this tool(may polish up the data binding side)

o   Encoder - media encoding with markers, leveraging IIS smooth streaming

o   DeepZoom – creating Image pyramids

o   (I have not yet dug deep into expression Design

·         RIA Services – a prescriptive model that takes n-tier development to the next level of abstraction.

·         Blend Behaviours – abstract out triggers and actions into declarative components

·         Perf tools

·         Automation using White

·         SilverlightSpy

·         SL unit testing

·         MVVM – the view has a view model and the view model has a model (whose change notifications it observes). Leverage XAML databinding and DelegateCommands (as the VM is out of the Logical Tree)

·         Composite Application Library – Constructing a mutiview GUI leveraging MVVM, Loading isolated module via IoC and DI, mapping views to regions -  Unity DI Container, DelegateCommands, EventAggreagater, RegionManager etc.

Maths

in order to construct AI algorithms, you need 2 core subsets of knowledge: Programmatic and Mathematic skillsets. Learning Software Engineering through 10 years of practice can yield for me the ability to take a set of algorithms to construct an n-tier solution (with all the –ability adjectives {scalability, maintainability, performability (ha), etc}) while a straight mathematician would flounder on basic programming constructs. By learning the math, I canaggregate knowledge silos - eg in a few hours comprehend the gist of a highly specialized phd thesis that someone spent 5 years writing).  To that affect I’ve gone through the following Schaums Guides (plus watched Jason Gibbs maths training DVD in Linear Algebra - http://www.mathtutordvd.com/ )

·         Advanced Calculus – reading through this felt like sense of achievement one gets from climbing a mountain. I achieved a decent comprehension

·         Probability – slightly simplistic. Stochastic trees are a great mathematical visualization technique for understanding Bayes Theorum. Useful for reasoning under uncertainty.

·         Linear Algebra –different techniques for solving systems of linear equations.

·         Operations Research – allocation of scarce resources

·         Numerical Recipes – function approximation. The notes were slightly sparse.

Putting things into perspective:

·         Larry Gonicks history books – a great way to learn the tapestry of history in a compact and humorous manner

·         The ascent of money – I’ve always enjoyed Nial Ferguson’s geopolitical and socioeconomic analysis. Can we unify behavioral and quantitative finance?

·         Ending Aging – a snapshot of the current state of research into alleviating the affects of the 7 malady categories of aging: - I corresponded with the author during my Bioinformatics days.

·         The trouble with physics – by Lee Smolin. String “Theory” (more like conjencture / cult of Ed Witten) is actually based on another unproven conjecture – supersymmetry. It is not currently falsifiable and has turned into consensus science, drawing away from other potential theories such as Twister theory or Loop Quantum Gravity. M theory is just a vague concept and there are an infinite number of string theories – in programming ,we call this a code smell. While it holds mathematically (and has led to new maths such as Guage theory), it wont be the 1^st mathematically elegant theory that doenst hold in reality.

·         A new kind of science – just like cellular autonoma, Wolfram proposes that complexity evolves out of simple rules. Reads like a pop sci book. Strangely enough, the concepts of quantum or probabilistic states and states that modify rules are not covered!

.NET Parallel Development

·         TPL – a .NET 4 scale up stack for data & task parallelization. Various levels of abstraction – PLINQ, parallel iterators, Tasks (similar to an automated threadpool), futures (tasks that promise to asynchronously return values). + many new thread model synchronization locks. see - http://blogs.msdn.com/pfxteam/default.aspx, http://msdn.microsoft.com/en-us/library/dd460693(VS.100).aspx , http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=348f73fd-593d-4b3c-b055-694c50d2b0f3

·         DryadLINQ – a declarative scale out API of MS Research for scaling processing out to multiple nodes - SEE http://research.microsoft.com/en-us/projects/DryadLINQ/ , http://connect.microsoft.com/site/sitehome.aspx?SiteID=891

·         Accelerator – a non LINQ enabled GPGPU API from ms research which is limited to 2D matrices. It leverages a high level shader to map processing into the GPU - SEE http://research.microsoft.com/en-us/projects/accelerator/default.aspx

·         Axum – a research API based upon the agent paradigm which isolates agent interactions through message channels. Feels clunky to me. SEE http://blogs.msdn.com/maestroteam/default.aspx, http://msdn.microsoft.com/en-us/devlabs/dd795202.aspx

·         Solver Foundation – (horrendous doco) - Optimization API host exposed as a web service, a C# API, or an excel plugin. It suceeds where Excel Services failed. http://code.msdn.microsoft.com/solverfoundation

·         Windows HPC – a grid computing host management environment – includes a prioritized job scheduler and management utilities  - SEE http://msdn.microsoft.com/en-us/library/cc907080(VS.85).aspx , http://www.microsoft.com/hpc/en/us/default.aspx

NET 4

·         Oslo + Quadrant – still half baked.

·         WF 4 – still half baked – currently no state machine or rule engine support evident. Note that persistence and tracking have been abstracted out to Dublin (WAS 2.0) , which wont be released in the NET 4 timeframe.

·         Entity Framework – pretty easy to learn these changes

·         WCF – discovery, REST templates

·         System.Xaml – still half baked

·         What happened to Software Factories (Blueprints)?

·         Azure – what a mess! SQL Data Services doesn’t work, Cloud Workflows were a fork and have been scrapped and Silverlight SMEWAs require live mesh client! Note: Azure is NOT A GRID COMPUTING PLATFORM!!! I’ll wait for v2 thanks.

Work

·         Constructed Silverlight Timeline for Matach

·         Constructed prototype of Silverlight media playback system for Youniversity

·         Completed Amdoc’s staffing workflow system

·         Constructed Southridge WPF labs for MS

·         Constructed WPF control Library  for Elta

·         Attrrnded the Mix conference

·         Taught the Metro WPF course in Israel, Australia, New Zealand, Turkey

·        App Profiling for IFN - fun with Ants profiler!

·         Constructed the Silverlight 3 Courseware Library for MS

·         Constructed WPF GIS  control and architectured Composite client for current customer.

• Ensure your developers build code to meet the spec (the contract of customer requirements) in a timely manner
• Ensure your developers build code changes that don’t break the spec and that pass functional requirements
• Ensure your developers build code which is robust (not fragile) and that meets design and code quality guidelines

• Make sure that the technical analyst provides appropriate specs!
• Keep it deliverable – compile-able. No point building mounds of UML diagrams that don’t stay in synch with the project
• consists of: Use Case Scenarios, Interfaces, Code Contracts, Unit Tests, pre & post conditions.
• To plug in Use Case Scenarios, need a generic baseline architecture.

• Scenarios and Tasks
  • Each use case focuses on describing how an end user or client can achieve a goal.
  • Forces you to think about the spec functional requirements.
  • decompose into Tasks
  • Scenarios and their Tasks map to VSTS.
  • checkin policies force developers to associate each changeset with a task – track progress, avoid unnecessary work.
  • From nouns in scenarios identify entities. Each task maps to a method.
  • Consider Workflows – each scenario maps to a sequential or state machine Workflow and each task maps to an Activity
• VSTS benefits
  • 20 tools in one
  • Integrated work item list – scenarios, bugs, tasks, tests, QoS
  • Source control with Checkin policies
  • Database schema version control, comparison and test data generation
  • Real Project health tracking via reportage
  • Integrated testing environment – 6 types of test: regression, web, load, custom, manual
  • Build & test servers

• Interfaces for Design By Contract
  • define precise verifiable interface specifications for software components
  • In VS, can generate classes from Interfaces, then automatically construct class diagram
  • Each interface method should map to a task
  • The interfaces force the developer to implement all tasks
• Interface Code Contracts
  • See previous post!
  • Applies Preconditions and Postconditions -

• Ensure code standards in [relevant] rules are met – checks: Correctness, Library design , Localization , Naming conventions , Performance , Security
• Analyzes the compiled code for conformance to Microsoft's .NET Framework Design Guidelines.

• Simian is a command line tool - but you can integrate it into Visual Studio via the extensibility of the External Tools menu. Here is an example of the standard output produced by Simian when run against some source code:

Similarity Analyser 2.2.23 - http://www.redhillconsulting.com.au/products/simian/index.html

Copyright (c) 2003-08 RedHill Consulting Pty. Ltd. All rights reserved.

Simian is not free unless used solely for non-commercial or evaluation purposes.

{failOnDuplication=true, ignoreCharacterCase=true, ignoreCurlyBraces=true, ignoreIdentifierCase=true, ignoreModifiers=true, ignoreStringCase=true, threshold=6}

Found 6 duplicate lines in the following files:

 Between lines 201 and 207 in /Users/haruki_zaemon/Projects/redhill/simian/build/dist/src/java/awt/image/WritableRaster.java

 Between lines 1305 and 1311 in /Users/haruki_zaemon/Projects/redhill/simian/build/dist/src/java/awt/image/Raster.java

Found 6 duplicate lines in the following files:

 Between lines 920 and 926 in /Users/haruki_zaemon/Projects/redhill/simian/build/dist/src/com/sun/imageio/plugins/jpeg/JFIFMarkerSegment.java

 Between lines 908 and 914 in /Users/haruki_zaemon/Projects/redhill/simian/build/dist/src/com/sun/imageio/plugins/jpeg/JFIFMarkerSegment.java

• enforce a common set of best practices for layout, readability, maintainability, and documentation of C# source code.
• produce elegant, consistent code that your team members and others who view your code will find highly readable.

• Ensure that all methods work – that changes to code don’t break other methods
• Generate a Test project – each [Task mapped] method should have a postcondition test
• Regression test – attach the util to the DLL
• Automatically generate unit tests using Pex:

• Improved Productivity (satisfy common application concerns out of the box) , Configuration Driven Design
• EntLib application blocks - Caching , Data Access (redundant), Cryptography , Exception Handling , Logging , Security , Validation
• Policy Injection – Aspect oriented programming - separate cross-cutting logic from domain-specific logic. Inversion of Control - instead of the programmer specifying cross-cutting functionality via function calls, instead register desired responses to particular policy rule defined events.
• Eg for a function - inject validation, tracing , exception handling , Authorization, caching, update performance counters. Using the PIAB, such functionality would be introduced through the creation of policies that define the implementation details needed at run time. Policies can be applied to objects through configuration or by decorating classes with attributes. Each policy has a set of matching rules to determine where it should be applied, and a collection of handlers that determine which behaviors should be introduced when invoking methods on the object. The PIAB comes with a number of useful handlers for validation, logging, exception handling, managing performance counters, authorization, and caching. Many of these handlers leverage EntLibs other application blocks. PIAB supports matching rules based on types, assemblies, member names, method signatures, namespaces, parameter types, property types, return types, custom attributes, tags

• Limitations in workflows requiring workarounds:
• Designer is slow – make a coffee slow!
• A setStateActivity is not the end of the Workflow’s current flow!
• In binding, how do you access the properties of dependnecy property sub-properties?
• a dependency property custom activity for receieveactivity binding must be inside the receiveactivity
• In rule condition, you cannot reference a dependency property in a custom activity
• a code activity cannot work on an adjacent activity without explicitly specifying its name
• declarative rules condition editor doesn’t support LINQ eg .contains
• declarative rules condition editor doesn’t support constants
• Copying an activity does not copy name for modification
• No data flow, only control flow à requires a lot of code and tree traversal
• No support for SendActivity custom WCF bindings
• Cannot use a DelayActivity from inside an EventDrivenActivity
• Instance versioning
• RuleActionTrackingEvent only works with PolicyActivity, not IfElseActivity

  Imagine that every project in codeplex had to be decomposed into codeplex hosted unit tested functions which complied to some xunit standard - that would go along way towards code reuse, and move away from reinventing the wheel.

  worked on 2 Silverlight projects recently - see screenshots:

and

About 2 months ago, I had to evaluate MOSS / WSS - compare its features and capacity plan. I'm not sure i want I to go down the Sharepoint path - Its too restrictive, too meta-programmy , it doesnt give you OOTB AJAX enabled WebParts, and is a different path than Silverlight (maybe the next version will be SilverPoint ?). On the flipside, the concepts of pluggable WebParts, InfoPath, Excel Services and Sharepoint Workflows are appealing.

Basically, to learn sharepoint you have to read the following:

• Sharepoint Planning Guide (1500 pages - did this) - http://office.microsoft.com/download/afile.aspx?AssetID=AM101638541033
• Sharepoint Administration (500 pages) - http://office.microsoft.com/en-us/sharepointserver/FX101211721033.aspx
• MOSD doco (500 pages)
• WSS SDK - www.microsoft.com/downloads/details.aspx?FamilyId=05E0DD12-8394-402B-8936-A07FE8AFAFFD
• MOSS SDK - www.microsoft.com/downloads/details.aspx?FamilyId=6D94E307-67D9-41AC-B2D6-0074D6286FA9

Read 3 good books this month:

1) the Emotion Machine - by Marvin Minsky - postulates how emotional states are a mechanism for changing the priority weighting of our cognitive machinary in regards to goals and tasks. A fair bit of conjecture as opposed to experimentally backed up theorum, and some rehash of his previous work 'The Society of Mind'.

2) MultiAgent Systems - Wooldridge - design of distributed workflow services that have different goals and tasks, and often which must compete , communcate and coordinate. Alot of different ideas on modal logic, game theory and auctions. Alot of the communication protocols are redundant with WCF. strangely, no mention of probabibilistic models.

3) Programming the Universe - Seth Loyd - a fascinating introduction to the science of quantum computing. unlike PLINQ where parallel threads must run tasks that doent interfere with each others resources, quantum computing is actually optimized for massively parallel tasks that contain side effects! Eg searching for the 2 prime factors that make up a public encryption key. Postulates the universe may be a quntum computer computing itself - 'it from bit'.

Heres an interesting Wikipedia topic - http://en.wikipedia.org/wiki/List_of_thought_processes

This is a list of thinking styles, methods of thinking (thinking skills), and types of thought.

I wonder how many of these have been abstracted into algorithms and computational models?

I've read 2/3 of Norvig's AI book, and i can see that serious attempts have been made at problem solving, reasoning, planning, reasoning under uncertainty and learning. I've also looked into the problem of Attention.

• Predictive Models -classification, regression, time series analysis, prediction. can be used to forecast explicit values, based on patterns determined from known results.

• Descriptive Models - clustering, summarization, association rules, sequence discovery. describe patterns in existing data, and are generally used to create meaningful subgroups such as demographic clusters.

•  Choosing the right algorithm to use for a specific business task - can be a challenge. While you can use different algorithms to perform the same business task, each algorithm produces a different result, and some algorithms can produce more than one type of result. E.g. you can use the Microsoft Decision Trees algorithm not only for prediction, but also as a way to reduce the number of columns in a dataset, because the decision tree can identify columns that do not affect the final DMM.
• Combining algorithms - can use different algorithms to perform the same business task and each algorithm produces a different result. Lift charts check the accuracy of the DMMs once built on the input data. Use more than one algorithm to produce results and analyze the results for choosing the right one. Different algorithms produce different results. The choosing of the algorithms is based on the accuracy and on the business need. Use algorithms together – use some algorithms to explore data, and then use other algorithms to predict a specific outcome based on that data. E.g. you can use a clustering algorithm, which recognizes patterns, to break data into groups that are more or less homogeneous, and then use the results to create a better decision tree model. Use multiple algorithms within one solution to perform separate tasks - E.g. regression tree algorithm can be used to obtain financial forecasting information, and a rule-based algorithm to perform a market basket analysis.

• The bottom line - Task & algorithms to use