Shaun Xu

The Sheep-Pen of the Shaun


News

logo

Shaun, the author of this blog is a semi-geek, clumsy developer, passionate speaker and incapable architect with about 10 years’ experience in .NET and JavaScript. He hopes to prove that software development is art rather than manufacturing. He's into cloud computing platform and technologies (Windows Azure, Amazon and Aliyun) and right now, Shaun is being attracted by JavaScript (Angular.js and Node.js) and he likes it.

Shaun is working at Worktile Inc. as the chief architect for overall design and develop worktile, a web-based collaboration and task management tool, and lesschat, a real-time communication aggregation tool.

MVP

My Stats

  • Posts - 122
  • Comments - 550
  • Trackbacks - 0

Tag Cloud


Recent Comments


Recent Posts


Archives


Post Categories


.NET



DocumentDB, provided by Microsoft Azure, is a low latency, NoSQL document database service for web high-scale gaming, social, and Internet of Things (IoT) app scenarios. Similar as Microsoft Azure Table Storage Service, when using DocumentDB developers don’t have to worry about managing schemas anymore. But additionally, DocumentDB automatically indexes all JSON documents added to the database, letting you use familiar SQL syntax to query them without having to specify the schema or secondary indices up front.

 

At the end of March, Microsoft just announced DocumentDB supports MongoDB protocol in preview. This means we can use existing drivers and tools for MongoDB, leveraging existing MongoDB skills for building application against DocumentDB.

 

Create DocumentDB with MongoDB Protocol

To enable this preview feature, following this blog post to send request. Several days later there will be an email said it's approved.

After that we can create a DocumentDB service with MongoDB Protocol. Login to azure portal and clicked "New" from the left top side, go to "Data + Storage" and search "DocumentDB" we will find "DocumentDB - Protocol Support for MongoDB".

2016-04-13_09-52-51

Select this service and click "Create".

2016-04-13_09-53-52

Next, specify the name and resource group of this service.

2016-04-13_09-54-43

After clicked "Create" it will be creating. This action may take several minutes.

2016-04-13_09-56-08

 

Access DocumentDB through Existing MongoDB Tool

When DocumentDB was firstly released there's no management tool. This is the reason I created a web-based DocumentDB management tool called "My DocumentDB" here and here. But now this DocumentDB service supports MongoDB protocol, we can use client tools for MongoDB to manage DocumentDB. For example, 3T MongoChef, which are are using right now for MongoDB.

First we need to open the DocumentDB service in azure portal, click "All Settings" and select "Connection String", copy the full connection string.

2016-04-13_09-57-19

Next, open MongoChef and click "Connection" on the toolbar, then click "New Connection". In the popup window click "From URI" and paste the connection string we have just copied from the portal.

2016-04-13_09-58-43

We can click "Test Connection" to verify if it works.

2016-04-13_09-59-14

Now we can view and manage our DocumentDB through 3T MongoChef. For example, I created a database with several collections via azure portal.

2016-04-13_10-02-18

In 3T MongoChef, click "Refresh All" in left side context menu it load them.

2016-04-13_10-03-06

Double click on one of the collection will list all documents, for example "teams" collection. We can also create a document in 3T MongoChef.

2016-04-13_10-05-48

And it will be saved on DocumentDB.

2016-04-13_10-06-45

 

Develop with DocumentDB via Existing MongoDB Driver

There are many MongoDB drivers for many program languages. Currently we are using Mongoose in out Node.js backend service and it's easy as well to use Mongoose to connect our DocumentDB.

First, copy the DocumentDB connection string from portal. Then create a Node.js application with NPM packages installed listed below.

{
  "name": "shx-docdb-mongo",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "co": "^4.6.0",
    "lodash": "^4.10.0",
    "mongoose": "^4.4.12"
  }
}

Includes them and refine a collection's schema as below.

const co = require('co');
const mongoose = require('mongoose');
const Schema = mongoose.Schema;

const TeamSchema = new Schema({
    name: {type: String},
    domain: {type: String},
    members_count: {type: Number, default: 0}
});
const TeamModel = mongoose.model('team', TeamSchema);

Then set the connection string to a variant for future use. Note that the connection string we copied from azure portal does NOT contain database name, which means it will connect to the authentication database "admin". So we need to insert the database in connection string, normally before "?ssl=true" part.

For example the original connection string was

"mongodb://your-doc-db-name:your-password@your-user-name:10250/?ssl=true"

We need to change to

"mongodb://your-doc-db-name:your-password@your-user-name:10250/your-database/?ssl=true"

 

Below is the database connect function which I wrapped from Mongoose API and returns Promise, as I'm running this sample in Node.js ES6 Mode.

const connect = (url) => {
    return new Promise((resolve, reject) => {
        mongoose.connect(url, null, (error) => {
            if (error) {
                return reject(error);
            }
            else {
                return resolve();
            }
        });
    });
};

 

Below is the code to connect and disconnect DocumentDB through Mongoose, which is the native MongoDB Node.js driver underlying.

co(function *() {
    yield mongoose.connect(connectionString);
    console.log('DocuemtDB connected through MongoDB protocol');

    yield mongoose.disconnect();
    console.log('DocuemtDB closed');
}).then(() => {
    console.log('All done.');
}).catch((error) => {
    console.log(`ERROR: ` + JSON.stringify(error, null, 2));
});

image

 

CRUD operations are all supported as well. For example, below code will create 3 documents. Note that "_id" was assigned by default.

let teams = [];
let team1 = new TeamModel({
    name: 'shx',
    domain: 'shx',
    members_count: 10
});
teams.push(yield team1.save());

let team2 = new TeamModel({
    name: 'Worktile',
    domain: 'worktile',
    members_count: 5
});
teams.push(yield team2.save());

let team3 = new TeamModel({
    name: 'Microsoft Azure',
    domain: 'azure'
});
teams.push(yield team3.save());
console.log(JSON.stringify(teams, null, 2));

image

The code below updated document and then query all documents. Note that "__v" was NOT updated automatically.

yield TeamModel.update({
    name: 'shx'
}, {
    name: 'Shaun Xu'
}).exec();
let teams = yield TeamModel.find({}).exec();
console.log(JSON.stringify(teams, null, 2));

image

We used "find" method to retrieve all documents. It also support "findOneAndUpdate" as well. Besides, some basic aggregate method are supported, too. Such as "count".

let shx = yield TeamModel.findOneAndUpdate({
    domain: 'shx'
}, {
    name: 'Shaun Ziyan Xu'
}, {
    new: true,
    upsert: false
}).exec();
console.log(JSON.stringify(shx, null, 2));

let result = yield TeamModel.count({}).exec();
console.log(JSON.stringify(result, null, 2));

image

But when I tried map-reduce, it raised an error back to me. Seems that currently DocumentDB does NOT support MongoDB Map-Reduce command. I had sent an email to DocumentDB team and is waiting for their response.

let result = yield TeamModel.mapReduce({
    map: function () {
        emit(this.name, this.members_count);
    },
    reduce: function (key, values) {
        return values;
    }
});
console.log(JSON.stringify(result, null, 2));

image

 

Summary

Why I'm interested in this feature? Well, currently we are using MongoDB which is hosted in several virtual machines and managed by ourselves. But last month we found some database performance issues. As a startup company we don't have enough budget to build a MongoDB expert team. So we have to investigate and tried by ourselves.

After this issue was resolved we found it's necessary to use a self-managed, easy-to-scale NoSQL database service rather than self-host. MongoLab is one of candidate, DocumentDB is the other.

DocumentDB with MongoDB Protocol helps us to migrate our application with very few lines of code changes. Hence if have the same request as our team, you'd better try it as well.

 

Hope this helps,

Shaun

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Xu. This work is licensed under the Creative Commons License.

Comments

Gravatar # re: Play Azure DocumentDB with MongoDB Protocol
Posted by Joe on 4/15/2016 7:58 AM
I don't get this feature to be honest. It doesn't have feature parity with Mongo. So you say today you want to move your Mongo apps over to this, well that's great as long as your apps only use very basic CRUD. If you do anything like aggregation pipeline, use large docs, use geospatial etc. you're stuck, it just won't work on DocumentDB.

So basically this will be ok if you want to build a new system against DocumentDB using a driver you're familiar with instead of learning a new driver. But that's about it.

Until they get feature parity with Mongo, the notion of app migration is a dream.
Gravatar # re: Play Azure DocumentDB with MongoDB Protocol
Posted by deniceochoa on 4/15/2016 11:57 PM
Building such strong foundation for your blog is indeed necessary to reach your target market. - Dr. Chiann F. Gibson DMD

Gravatar # re: Play Azure DocumentDB with MongoDB Protocol
Posted by Shaun Xu on 4/19/2016 9:42 AM
@Joe

I agree with you. Currently there are some Mongo features which DocumentDB doesn't support yet, for example the Map-Reduce. We didn't use other features such as geospatial, GridFS, etc. so we are waiting and to see when Map-Reduce could be ready.

We just in pre-POC phase. DocumentDB is one of our choice. So we are not decided yet to begin move our Mongo app to DocDB, plus this feature is still in Preview. So I personally don't recommend you to use this feature in production.
Post A Comment
Title:
Name:
Email:
Comment:
Verification: