Programming

Developer, Meet Server - Transactional NTFS and WCF

Jason Olson — Wed, 04 Apr 2007 21:19:00 GMT

I recently posted the second screencast in my new "Developer, Meet Server" screencast series. This latest screencast covers using Transactional NTFS in a service-oriented environment via WCF.

Enjoy!

Transactional NTFS and Free Disk Space

Jason Olson — Wed, 04 Apr 2007 21:15:00 GMT

Andrew recently watched my screencasts and had some great questions that came up around how Transactional NTFS behaves with the file system. So, I figured I would answer them here. So, if you want to see the context around his questions, check out his post.

Here's the paragraph with the questions:

I wonder what happens to files that never get committed - ala a long running transaction. They have to be taking up file space. Jason says not even Windows Explorer knows about them, but they have to exist somewhere. What if they overflow? Is there a way to clean house? If the poop is invisible, does it still stink?

Great observations and questions Andrew! In the case of a long running, open transaction, the files will exist on the disk. Hence, they are certainly taking up space on the disk. If you have 256 MB of free space on your disk, and try to write 257 MB of data to the disk within a transaction, you will get an "out of space" error like you would outside of a transaction.

Is there a way to clean house? Yes, rollback or commit the transaction. On rollback, any changes made within the transaction are rolled back, and then consequently removed form the disk.

But what if my server crashes in the middle of a transaction. Would I have turds floating around the file system? No, you would not. When an NTFS-formatted volume is mounted in Vista or Windows Server "Longhorn", any transactions that are "in-doubt" (the transaction manager is no longer around and we have no clue on the status of the transaction) will be rolled back automatically, effectively removing all the changes from disk. So, in that sense, you would not have any file turds you have to clean up after a system crash (like you would have if you were doing a "temp file -> rename" type of methodology.

Now, as is usual in transaction systems, you have to be _very_ aware and _very_ cautious of long-running transactions as it could cause some long-term problems that you don't want to deal with. So, there certainly are scenarios where Transactional NTFS just is not a good fit. As tends to be the case with all technology, make sure you are using the right tool for the right job :).

Announcing: New "Developer, Meet Server" Screencast Series

Jason Olson — Fri, 09 Mar 2007 19:28:00 GMT

Welcome everybody to the inaugural, first ever, grand opening, “opening the doors to the public” celebration day kick-off for the new “Developer, Meet Server” screencast series. I am your sincere, humble (as ever), gracious and wonderful host/chef/tour guide Jason Olson.

In this series of screencasts we will be diving through all the various cool developer-oriented enhancements on the Windows Server "Longhorn" platform that developers can leverage to build robust and powerful applications. Specifically, in this first screencast, we will take a quick peek at Transactional NTFS. In future screencasts, we will go into a more detailed look at Transactional NTFS, so think of this as a quick introduction.

The screencast can be found in one of the main tubes on the Intarw3b known as the ever-lovely Channel 9: http://channel9.msdn.com/Showpost.aspx?postid=289816.

So sit back, grab a cup of coffee/tea/InsertYourFavoriteLocalTastyBeverageHere, and enjoy this screencast.

MSR TechFest - See How Microsoft Research Shaping Is The Future

Jason Olson — Tue, 06 Mar 2007 17:53:00 GMT

[Crosspost from Managed World]

Rory is doing some videos at Channel 9 on the Microsoft Research TechFest event happening at Microsoft this week. If you want to see how Microsoft's own version of Area 51 is shaping and changing the future of technology, you should ABSOLUTELY watch these videos.

The first video has just been released: http://channel9.msdn.com/ShowPost.aspx?PostID=288554.

There is a lot of cool stuff coming out of Microsoft Research. Even Google won't stand up to some of these technologies :). We shall prevail :P. It's important to realize that Microsoft is filled with geeks just like you. We want to create cool stuff that changes the world. And what better way to emphasize that than taking a dive into Microsoft Research :).

Enjoy the video. And feel absolutely free to send this link around. This is truly exciting stuff coming from Microsoft and most geeks should be able to find some type of excitement in it :).

Vista-Only Features for Managed Developers

Jason Olson — Mon, 13 Nov 2006 22:07:00 GMT

[Crosspost from Managed World]

I recently came across a bunch of stuff done by Daniel Moth, a Developer Evangelist for Microsoft UK. If you are a managed developer and want to see the new stuff you can leverage on Vista, be sure to check out his stuff.

If you want to digest some of these new features in Screencast format, be sure to check out some of these screencasts that Daniel Moth has also done:

Enjoy! And thanks go to Daniel for all of these :).

LHS Feature Of The Week #2 - Remote Differential Compression

Jason Olson — Fri, 22 Sep 2006 18:32:00 GMT

Hello everyone, and welcome to the second installment of the “Feature Of The Week” newsletter. This week, we will take a short little peek at a feature coming in Vista and Longhorn Server called Remote Differential Compression.

So, sit back, relax, and enjoy the Longhorn Server Feature Of The Week!

What is Remote Differential Compression? Remote Differential Compression (RDC) allows applications to synchronize data between two computers in an efficient manner. The synchronization efficiency is made possible by using compression techniques to minimize the amount of data sent across the network.

What makes RDC different from other differencing mechanisms? RDC is different from patching-oriented differencing mechanisms, such as Binary Delta Compression (BDC), which are designed to operate only on known versions of a single file. RDC makes no assumptions about file similarity or versioning. Because differences between files are computed on the fly, RDC is ideally suited for synchronizing files that are different or have been updated independently.

What scenarios does this enable? RDC is suitable for applications that move data across a wide area network where the data transmission costs outweigh the CPU cost of signature computation. RDC can also be used on faster networks if the amount of data to be transferred is relatively large and the changes to the data are typically small.

Resources

Documentation
- Windows SDK
- Using Remote Differential Compression
Code
- If you have the latest Windows SDK installed, you can find a simple Client/Server example using RDC in the following directory: %INSTALL_DIR%\Samples\winbase\RDC

Enjoy!

[Crosspost from Managed World]

LHS Feature Of The Week #1 - Transactional NTFS

Jason Olson — Fri, 22 Sep 2006 18:18:00 GMT

Hello everyone, and welcome to the “Feature Of The Week” newsletter. In the Feature Of The Week Newsletter, we visit a new feature coming in Longhorn Server every week. The goal is that these are nice, short, and efficient to minimize the amount of effort required by you, the reader. With every Feature Of The Week, we will provide links to more resources in case you want to spend more time diving into the feature covered.

So, sit back, relax, and enjoy the Longhorn Server Feature Of The Week!

What is Transactional NTFS? Transactional NTFS is an enhancement to NTFS that allows all file operations on an NTFS file system volume to be performed within the scope of a transaction. This is made possible by a new transaction infrastructure in the kernel that allows operating system services to participate in transactions via the new Kernel Transaction Manager.

What scenarios does this enable? Transactional NTFS is also fully capable of communicating with the MS Distributed Transaction Coordinator, meaning that TxF operations can participate in transactions that are leveraging any other Resource Manager that DTC is capable of working with. What does this mean to you? It means that you can now have an application that not only encompasses database calls, but also operations to the file system as well (think of Document Management solutions). With transaction enhancements being made to the new SMB 2.0 protocol, you can also now contain distributed file operations within a transaction as well.

Resources

Documentation
- Windows SDK
Videos/Webcasts
- [Video] Vista Transactional File System
- [Webcast] The Revolutionary Vista TxF Infrastructure
Blogs
- Jim Johnson
- Jason Olson

Enjoy!

Virtual Machine Monitors, Exokernels, and Hypervisors, Oh My!

Jason Olson — Tue, 19 Sep 2006 22:11:00 GMT

For those of you that semi-regularly follow this blog, you may or may not have noticed that I've really been getting into OS Development lately. Of course, the first step is brushing up on x86 assembly language. Outside of that, it's a lot of Research, Research, Research, and more Research. Heck, even if it's not research, I just finished the book "Just For Fun", about Linus Torvalds (very enjoyable read, IMO).

One of the specific topics that has really excited me is the Virtual Machine Monitor architecture (type of Exokernel, Hypervisor, etc.). I figured I would take this opportunity to link out to a bunch of resources that I have found interesting as I read up on this topic (you'll find that every single one of these points to Wikipedia; I'm finding that I use Wikipedia MORE and MORE when doing research; I LOVE it!).

Enjoy :).

Now I just have to see what it would take to get a hold of a box with one of the new Intel VT or AMD chips :). Yum!!!

Writing Your Own OS With Visual Studio 2005

Jason Olson — Mon, 11 Sep 2006 01:29:00 GMT

Before I joined the ranks at Microsoft, a hobby I started to take up was Operating System development. Technically, I didn't get very far yet (I had just started). My "kernel" (if you can even call it that) basically just printed "Hello World" to the screen. But heck, it was booting up from a floppy, loading the kernel into memory, and executing to do that (in a VM of course, I wouldn't even dare run my hobby OS on an actual machine :P). ... (Read More)

Transaction Resources List

Jason Olson — Fri, 08 Sep 2006 13:06:00 GMT

I'll be updating this post over time to reflect new resources that are made available regarding the transactions platform in Windows.

Websites:

[Transactions] Transactions Programming Forum - MSDN
[Transactions] Transaction Management in Windows - Microsoft.com
[TxF] TxF Documentation - MSDN

Blogs:

... (Read More)