XML/XSLT

A Rant on XSD Schemas

Yow-Hann Lee — Sun, 25 Mar 2007 23:24:00 GMT

Have you ever been frustrated by some of the brittleness in XSD schemas? Granted, this may sound like an oxymoron since XSDs were enforced in the first place to provide strict xml structure.

But let's think about XML based technologies for a moment...

The claim to fame for XML was the flexibility of having variable number of attributes, etc for your business objects. And with your XML, you could perform XPath queries to your heart's content. What was great about XPath was that it is was order independent (unless of course you explicitly called for one return node, SelectNode, in the BCL API and the xml blob consisted of multiple nodes that could be returned with the expression, SelectNodes).

So where did XSD fit in?

Well, in the wild wild west of XML, you could think of it as the cheriff in town.

Unfortunately, there are just some things in XSD that are a little too strict. Thus, making life difficult on the developer.

SCENARIO:

Let's take for instance you want your XSD to govern the following structure:

<AlcoholPercentage>5</AlcoholPercentage>

</Drink>

-- Some other elements in here --

</Food>

-- Some other elements in here --

</Food>

So the rule is that you can only have 1 Drink node but can have multiple Food nodes. It must be flexible enough to be order independent.

-- Some other elements in here --

</Food>

-- Some other elements in here --

</Food>

<AlcoholPercentage>5</AlcoholPercentage>

</Drink>

RULES:

But the rules are as follows:

Drink must occur once
Food can occur a minOccurs="0" and maxOccurs="unbounded"
The two are order independent as shown above

However, because you are performing XPath queries on this data, you do not care what order they come in. The xml structure should be flexible enough so that a developer on another project playing with your data can simply RemoveChild and AppendChild on any nodes without affecting the expected behavior.

So right away, if you were thinking of specifying "<xs:sequence>..." with your complexType and element names underneath, it is ruled out.

Then, perhaps an idea pops up in using an "<xs:any.." tag at the end of the sequence. But that also gets ruled out right away because it will introduce the violation of drink occuring only once.

So then the idea for using a "<xs:choice>" seems appropriate. For the purposes of this simplified version, that would seem like the likely answer. All you have to do is specify two sequences under choice and pick one or the other right?

Well, not exactly. As I mentioned, this example is an oversimplification of the actual scenario. Thus, in the real world, with your complexTypes nested under one another, you would end up with something along the lines of:

So now you're thinking, the xml structure is flawed. You want to do anything ranging from revamping the entire xml blob to minor changes. (i.e. If you really wanted this, why not place Food under a Foods parent node? But let's go back to the original requirement. It must follow this exact xml structure. Sometimes, you don't have the luxury to decide on this.)

And hence the rant on XSD flexibility! Well, perhaps there is some way of circumventing all of the hassles here and still satisfying the requirements. But for simple requirements such as this, the xsd definition should not be so verbose. Also, wasn't the XML vision to have all its technologies play nicely together: xpath, xml, xslt, xsd? XPath is order independent and therefore, there should be a quick and easy way to do it. (yes, there's choice and any, but those don't work as well - see above).

If you do have a simple solution, please do share. =)

A Brief Look at Online IDEs

Yow-Hann Lee — Mon, 19 Mar 2007 09:14:00 GMT

A couple years back, I remember thinking about IDEs as an SaaS. It definitely seemed ridiculous at the time, especially given that online spreadsheets were merely ideas in the works. (NOTE: I'm not saying it's a good idea even today). Also, the thought of having a Visual Studio clone within your browser was a bit much. <insert reference here to classic rule of thumb for creating a desktop driven app vs a client/server app - sorry, I came across an article, amongst a few, way back but I cannot find it now. If anyone knows what I'm referring to, please provide the link.>

Now in 2007, with a few online IDEs on the market, I decided to blog about a couple. I alluded to Yahoo Pipes and their designer in an earlier entry at: http://geekswithblogs.net/yowhann/archive/2007/02/13/106252.aspx. Also, just to add a disclaimer, this analysis excludes websites such as Try Ruby or Run BASIC which provides a quick install free environment and is aimed at getting people to try the language.

There is SednaSpace, an online IDE offered as SaaS. The online IDE was actually built in ASP.NET and the IDE is specialized to produce web apps. And surprise, surprise...the web apps that are generated are in ASP.NET =). While this is a visual programming service, there is an option to download the generated source. After examining the generated code, you quickly realize that it was not structured in a manner that can be easily maintained by developers. But then again, generated code is usually less than ideal. So most of the value in this service comes from the codeless programming aspect.

The SednaSpace designer is very much like the visual designer in VS. Unfortunately, because their aim is to completely abstract code away from the user and keep it primarily visual programming driven, there were some usability issues. Being in a less mature niches, online IDE and visual programming, does not help. And while they do have sparse online documentation at their site, their built-in help within the IDE is non-existent and under construction. (See Below). I imagine there is potential for a following once the service matures and they provide users with adequate quickstart guides and documentation.

The forms are based off of adding eventlisteners and actions to hook up to, much like ASP.NET. Logical expressions can be built using an expression builder that pops up in a separate window. Shown below is a simple graph, button and textbox with a validator (the expression being a value being <= 10).

Then there is CodeIDE (http://www.codeide.com/), which is actually an IDE for multiple languages (Perl, LISP, JavaScript, BASIC, etc). This one is more lightweight than SednaSpace but is limited in IDE features. Shown below is a simple program and its output on the right.

(NOTE: resizing your window while in CodeIDE is not preferred).

Last but not least, there is The Flex Online Compiler, which I have yet to dig into.

I am not going to rank these online IDEs; the purpose of this entry is moreso to bring awareness to a few online IDEs. While it may be possible to monetize this, I am not sure what kind of following it would have. Currently, CodeIDE and SednaSpace are free and I did some some ads within the IDE. So perhaps they intend on monetizing this in a similar fashion as other SaaS.

If the intent is to make this a paid service, it will be a long road ahead in convincing development shops let alone the enterprise of adopting this. While building even something trivial in SednaSpace, there were latency issues. Nevertheless, kudos to these guys for getting this out there. It is not something I would use right now, but it is something that seems like it was loads of fun to build as an SaaS.

Safeguarding Your Website/App Code From User Input: Cross Site Scripting Issues and Anti-XSS Approaches

Yow-Hann Lee — Sun, 04 Feb 2007 13:26:00 GMT

In a previous blog entry, I highlighted a key web development security issue: XSS or Cross Site Scripting. I spoke mainly of case studies from Google websites/services (Security in Software As A Service (GOOGLE) - How Secure is the data and where is the Offline switch?) and the challenges with software as a service. Just so Google is not singled out for case studies on security issues, here is a link for issues in Firefox and the controversy stirred by this last year.

In any case, as a continuation of that and narrowing the focus on XSS this time around, I thought I'd write up a simplified entry on securing your ASP.NET site from XSS.

SCENARIO:

At some point in your ASP.NET code behind, you directly manipulated the user's input (whether it was textbox input, url, or even client cookies). For this example, we'll use a simplified and trivial case below:

string script = Request.QueryString["test"] as string;

But what you may end up with your application's response to the client is the following:

<html><head><script type="text/javascript">alert('42');</script></head></html>

where alert('42'); was the malicious script. So how does this apply to the real-world? Think of web forms..or better yet, think of blogs. Think of the post comments in the blog comments? (Ok, so maybe the first thing that comes to you mind is SQL injection when it comes to persistence of the comments. But XSS will also cross your mind).

Basically, anything in your site that involves outputting user input in some shape or form may be vulnerable. For ASP.NET developers, there is no need to hyperventilate just yet. ValidateRequest method in ASP.NET (PagesSection.ValidateRequest) is the method that handles the validation of your browser's request for dangerous values. As is called out as well on MSDN, this is not a complete blanket approach and you definitely will need to handle for your own specific security issues on top of this as well. Also, turning this feature(whether at the page level <@ page.., at the app level in web.config or worse yet, at the server level with machine.config) is NOT recommended. ASP.NET raises a HttpRequestValidationException if it encounters dangerous input. In these cases, you can refer to my blog entry on IIS custom errors to handle this and display the appropriate error page.

The nice thing about ASP.NET's built in security is that it covers simple cases such as the following:

- Providing script tags in your QueryString (i.e. <script type="text/javascript">alert('42')</script>)

- Providing script tags with URL encode for empty char (i.e. <%8fscript%20type="text/javascript"></script>)

RECOMMENDED RESOURCES AND LIBRARIES/FRAMEWORKS

Obviously, you still need to handle security issues specific to your website or application. So this is where a decent scripting library comes into play. Microsoft provides you with a free Anti-Cross Site Scripting Library in .NET. You can download it here (They also have an archived V1.0 in the MS downloads page. This link is for V1.5 which was published externally on 11/20/2006). This download comes with a decent help document and with accompanying samples as well. Obviously, as with any third party/outside libraries that you adopt into your own projects, you need to evaluate the impact properly. This may not suit the needs of your particular project and it will require a small investment in terms of research/investigation time. Ideally, you will want to become intimately familiar with libraries you use to help abstract your work. (HINT: Here comes another endorsement for Lutz Roeder's .NET Reflector).

The AntiXSS library provides the following functionality:

HtmlAttributeEncode
HtmlEncode
JavaScriptEncode
UrlEncode (NOTE: THIS IS NOT THE SAME AS THE ONE BUILT IN ASP.NET'S Server.UrlEncode)
VisualBasicScriptEncode
XmlAttributeEncode
XmlEncode

A good resource for Script Exploits is: http://msdn2.microsoft.com/en-us/library/w1sw53ds(VS.80).aspx. Happy Securing your ASP.NET applications or web services. Another great paper on XSS can be found at: http://www.technicalinfo.net/papers/CSS.html.

While on the topic of securing your application (and more specifically, the security of your web service), you may want to dive into Microsoft's WSE 3.0. It is a free add-on to VS2005 and adds options to setting the security policies to your web service.

Windows Live Search Mashup Day V3 Summary: Live Search Web Service API & Exposure to MS Knowledge Network

Yow-Hann Lee — Wed, 31 Jan 2007 11:19:00 GMT

I took part in the first ever Search Mashup event last year in Q4 and enjoyed it. The event is an internal event open to everyone who works for/at Microsoft. In the first ever Mashup event, I was fortunate enough to catch the tail end of Tina Wood's (on10 host) appearance. You can catch snippets of the first ever Mashup event on10.net. So the traditional rule of thumb in software is that it generally takes a few releases (3?) before a product has matured and stabilized. Does it apply to events? Well, the V3 has definitely evolved and was "feature rich".

Since I never blogged about the first Windows Live Search Mashup Night, I am going to take this opportunity to showcase some of the Live Developer offerings. In the first Search Mashup, all the buzz was about the released Windows Live Macros (I finally blogged about it earlier this month with a comparison of Google Co-op, entry found here). So I played around with that one. However, for this entry and what most most devs focused on, I will highlight Windows Live Search Web Service API.

Windows Live Search Web Service API

The API is very simple to use and follows a thin layer web service model. You can find the API reference at: http://msdn2.Microsoft.com/en-us/library/bb266180.aspx

Your request and response is broken down into the following:

1. SearchRequest object and

2. SearchResponse object.

You can think of them as HttpWebRequest and HttpWebResponse.

The generated proxy has a MSNSearchService with one public method called Search taking in a SearchRequest object as a parameter.

The SearchRequest contains a Query property which allows you to specify your query exactly as a user would via the Live Search textbox. So while there are no properties to set where you can maintain a site specific search (or even a list of sites to search on as is available in customized search), you can essentially achieve similar behavior by providing it directly in your Query string. For example, in the scenario of a site specific search, you would just specify something like: "site:www.cs.ubc.ca robotics".

If you have also played around with an older version (MSN Search Web Service 0.60 Beta), your experience should be fairly consistent. The ease of use (although some people may want more features/granular control as mentioned above) is part of what makes it so suitable for building during a Mashup event. Generally, people will spend a couple hours at these events. Add to the time allocated for the consumption of food/desserts, and yet you are still able to create functional and creative apps.

Microsoft Knowledge Network

At tonight's Mashup, there was a table setup with an LCD Screen showcasing Microsoft Knowledge Network (Their team blog can be found here). If you haven't seen the KN tie in with Sharepoints, Outlook, Communicator, etc (other Office products).. then you should definitely check it out. It's really sexy stuff for the enterprise! I think only the Unified Communications effort edges this out in terms of what's hot in the Office Division. So yesterday, on the big screen in the cafeteria in Redmond and with Bill Gates in NYC showcasing the Vista Launch, he referenced back to Windows 95 and pointed out all the technological advances made over that time. Just go back a decade and think of the tools used in the Enterprise. What a change?

Lighting, Prizes, Aura, Music (they played one of my favorite tunes circa 2002/2003 - Motorcycle - As The Rush Comes) and Great Food/Desserts (Chocolate Fondue!). All in all, a well done event. And the main thing to get out of these events? Fun!

XSD.EXE And Using Reflector For Your Documentation Needs

Yow-Hann Lee — Mon, 29 Jan 2007 01:19:00 GMT

I enjoy coming across and reading Rick Strahl's blog every now and then. He has posted free tools up on his site and his indepth articles are also invaluable (one of my favorites was one he wrote back in 2005 on ASP.NET Architecture. He is also very capable of creating blog entries that generate an immense amount of discussion.

I came across his latest blog entry on "XSD.EXE and included schemas". In his post, there was mention that "Looking at XSD.EXE I don't see anyway to specify the whole lot of files to import and apparently XSD.EXE doesn't look at the includes which look something like this:".

Sure enough, based on the documentation (see command prompt below), it can be difficult to determine what is/is not supported.

While reading about his troubles, I thought this would be another great opportunity to make use of Lutz Roeder's .NET Reflector (the individual that Carl and Richard on the DNR podcast show have referred to as God). I blogged about the different motivation factors of using Reflector in a previous entry. After loading up XSD.EXE in Reflector and taking a peek at the internals, in the Run method of XsdTool.Xsd, it appeared to support the import of several xsd files as generated classes.

To confirm this, I performed the following steps:

1. Create [XmlSchema1].xsd and [XmlSchema2].xsd

2. Opened up command prompt and executed the following command: xsd XmlSchema1.xsd XmlSchema2.xsd -c

The output file is generated. In this case, it contained two classes.

This may still not fit Rick's requirement and ideally, it would have been great if his first scenario of having one schema with specifying includes had behaved intelligently. However, hopefully, this clarifies some of the question marks with XSD's help. One of the comments in the blog entry mentioned of an alternate tool called CodeXS. I'll have to take a look at it and check out the similarities/differences in code base and functionality.

While on the topic of Reflector, it may be of interest to see which exe and dlls in .NET are managed code with CLI headers and which are not. Not surprisingly, aspnet_regiis, setreg and sn do not contain CLI headers. Meanwhile, it's interesting to note that InstallUtil and WebDev.WebServer are both managed code (WebDev, the ASP.NET 2.0 host, NOT to be confused with WebDAV, IETF approved and original vision for WWW beyond GET and POST methods). See below for a subset of components of interest:

W3C Recommendations Released for XSLT 2.0, XPath 2.0 & XQuery 1.0

Yow-Hann Lee — Thu, 25 Jan 2007 09:48:00 GMT

In the second part of this two part news update series and having to work with XML and XPath these days, I wanted to point out that the World Wide Consortium has released recommendations for XSLT 2.0, XPath 2.0 & XQuery 1.0. The announcement came on two days ago (Monday Jan 22) and you can find it here.

NOTE: Originally, I was thinking about doing a bit on W3C Recommendations and discussing its compliance. I decided to save that can of worms for a future post.

You can find Microsoft XML Team's comments on this release here.

Two Scotts & Formatting

Yow-Hann Lee — Fri, 20 Oct 2006 10:28:00 GMT

Lessons relearned today.

As I learned from work, encoding can be a nightmare. And back in the summer Scott Hanselman did an entire show on internationalization found here at: (http://www.hanselminutes.com/default.aspx?showID=36).

On another note, when creating files such as XML files via code, be mindful of your audience. Doing things such as saving in a specific encoding (i.e UTF-8) or ensuring that your code is well formatted for readers is essential. If you can guarantee that these files will not be viewed by an organic set of eyes, then go ahead and compress the document with a smaller file size. Or else, one day, you may end up eating your own dogfood.

One thing from work that I can talk about since it is public knowledge at Scott Guthrie's site is Product Studio: http://weblogs.asp.net/scottgu/archive/2004/11/03/251930.aspx. Kind of cool to be using an internal product that made it to the big leagues. It becomes a privilege to say, "I used them way back when they were just an internal product" =P.

I haven't been listening to HanselMinutes lately and just noticed a new podcast from Oct 18. Scott interviews Scott Guthrie. How cool is that? Ub3r T1ght