Shaun Xu

The Sheep-Pen of the Shaun


News

logo

Shaun, the author of this blog is a semi-geek, clumsy developer, passionate speaker and incapable architect with about 10 years’ experience in .NET and JavaScript. He hopes to prove that software development is art rather than manufacturing. He's into cloud computing platform and technologies (Windows Azure, Amazon and Aliyun) and right now, Shaun is being attracted by JavaScript (Angular.js and Node.js) and he likes it.

Shaun is working at Worktile Inc. as the chief architect for overall design and develop worktile, a web-based collaboration and task management tool, and lesschat, a real-time communication aggregation tool.

MVP

My Stats

  • Posts - 115
  • Comments - 466
  • Trackbacks - 0

Tag Cloud


Recent Comments


Recent Posts


Archives


Post Categories


.NET



DocumentDB, provided by Microsoft Azure, is a low latency, NoSQL document database service for web high-scale gaming, social, and Internet of Things (IoT) app scenarios. Similar as Microsoft Azure Table Storage Service, when using DocumentDB developers don’t have to worry about managing schemas anymore. But additionally, DocumentDB automatically indexes all JSON documents added to the database, letting you use familiar SQL syntax to query them without having to specify the schema or secondary indices up front.

 

At the end of March, Microsoft just announced DocumentDB supports MongoDB protocol in preview. This means we can use existing drivers and tools for MongoDB, leveraging existing MongoDB skills for building application against DocumentDB.

 

Create DocumentDB with MongoDB Protocol

To enable this preview feature, following this blog post to send request. Several days later there will be an email said it's approved.

After that we can create a DocumentDB service with MongoDB Protocol. Login to azure portal and clicked "New" from the left top side, go to "Data + Storage" and search "DocumentDB" we will find "DocumentDB - Protocol Support for MongoDB".

2016-04-13_09-52-51

Select this service and click "Create".

2016-04-13_09-53-52

Next, specify the name and resource group of this service.

2016-04-13_09-54-43

After clicked "Create" it will be creating. This action may take several minutes.

2016-04-13_09-56-08

 

Access DocumentDB through Existing MongoDB Tool

When DocumentDB was firstly released there's no management tool. This is the reason I created a web-based DocumentDB management tool called "My DocumentDB" here and here. But now this DocumentDB service supports MongoDB protocol, we can use client tools for MongoDB to manage DocumentDB. For example, 3T MongoChef, which are are using right now for MongoDB.

First we need to open the DocumentDB service in azure portal, click "All Settings" and select "Connection String", copy the full connection string.

2016-04-13_09-57-19

Next, open MongoChef and click "Connection" on the toolbar, then click "New Connection". In the popup window click "From URI" and paste the connection string we have just copied from the portal.

2016-04-13_09-58-43

We can click "Test Connection" to verify if it works.

2016-04-13_09-59-14

Now we can view and manage our DocumentDB through 3T MongoChef. For example, I created a database with several collections via azure portal.

2016-04-13_10-02-18

In 3T MongoChef, click "Refresh All" in left side context menu it load them.

2016-04-13_10-03-06

Double click on one of the collection will list all documents, for example "teams" collection. We can also create a document in 3T MongoChef.

2016-04-13_10-05-48

And it will be saved on DocumentDB.

2016-04-13_10-06-45

 

Develop with DocumentDB via Existing MongoDB Driver

There are many MongoDB drivers for many program languages. Currently we are using Mongoose in out Node.js backend service and it's easy as well to use Mongoose to connect our DocumentDB.

First, copy the DocumentDB connection string from portal. Then create a Node.js application with NPM packages installed listed below.

{
  "name": "shx-docdb-mongo",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "co": "^4.6.0",
    "lodash": "^4.10.0",
    "mongoose": "^4.4.12"
  }
}

Includes them and refine a collection's schema as below.

const co = require('co');
const mongoose = require('mongoose');
const Schema = mongoose.Schema;

const TeamSchema = new Schema({
    name: {type: String},
    domain: {type: String},
    members_count: {type: Number, default: 0}
});
const TeamModel = mongoose.model('team', TeamSchema);

Then set the connection string to a variant for future use. Note that the connection string we copied from azure portal does NOT contain database name, which means it will connect to the authentication database "admin". So we need to insert the database in connection string, normally before "?ssl=true" part.

For example the original connection string was

"mongodb://your-doc-db-name:your-password@your-user-name:10250/?ssl=true"

We need to change to

"mongodb://your-doc-db-name:your-password@your-user-name:10250/your-database/?ssl=true"

 

Below is the database connect function which I wrapped from Mongoose API and returns Promise, as I'm running this sample in Node.js ES6 Mode.

const connect = (url) => {
    return new Promise((resolve, reject) => {
        mongoose.connect(url, null, (error) => {
            if (error) {
                return reject(error);
            }
            else {
                return resolve();
            }
        });
    });
};

 

Below is the code to connect and disconnect DocumentDB through Mongoose, which is the native MongoDB Node.js driver underlying.

co(function *() {
    yield mongoose.connect(connectionString);
    console.log('DocuemtDB connected through MongoDB protocol');

    yield mongoose.disconnect();
    console.log('DocuemtDB closed');
}).then(() => {
    console.log('All done.');
}).catch((error) => {
    console.log(`ERROR: ` + JSON.stringify(error, null, 2));
});

image

 

CRUD operations are all supported as well. For example, below code will create 3 documents. Note that "_id" was assigned by default.

let teams = [];
let team1 = new TeamModel({
    name: 'shx',
    domain: 'shx',
    members_count: 10
});
teams.push(yield team1.save());

let team2 = new TeamModel({
    name: 'Worktile',
    domain: 'worktile',
    members_count: 5
});
teams.push(yield team2.save());

let team3 = new TeamModel({
    name: 'Microsoft Azure',
    domain: 'azure'
});
teams.push(yield team3.save());
console.log(JSON.stringify(teams, null, 2));

image

The code below updated document and then query all documents. Note that "__v" was NOT updated automatically.

yield TeamModel.update({
    name: 'shx'
}, {
    name: 'Shaun Xu'
}).exec();
let teams = yield TeamModel.find({}).exec();
console.log(JSON.stringify(teams, null, 2));

image

We used "find" method to retrieve all documents. It also support "findOneAndUpdate" as well. Besides, some basic aggregate method are supported, too. Such as "count".

let shx = yield TeamModel.findOneAndUpdate({
    domain: 'shx'
}, {
    name: 'Shaun Ziyan Xu'
}, {
    new: true,
    upsert: false
}).exec();
console.log(JSON.stringify(shx, null, 2));

let result = yield TeamModel.count({}).exec();
console.log(JSON.stringify(result, null, 2));

image

But when I tried map-reduce, it raised an error back to me. Seems that currently DocumentDB does NOT support MongoDB Map-Reduce command. I had sent an email to DocumentDB team and is waiting for their response.

let result = yield TeamModel.mapReduce({
    map: function () {
        emit(this.name, this.members_count);
    },
    reduce: function (key, values) {
        return values;
    }
});
console.log(JSON.stringify(result, null, 2));

image

 

Summary

Why I'm interested in this feature? Well, currently we are using MongoDB which is hosted in several virtual machines and managed by ourselves. But last month we found some database performance issues. As a startup company we don't have enough budget to build a MongoDB expert team. So we have to investigate and tried by ourselves.

After this issue was resolved we found it's necessary to use a self-managed, easy-to-scale NoSQL database service rather than self-host. MongoLab is one of candidate, DocumentDB is the other.

DocumentDB with MongoDB Protocol helps us to migrate our application with very few lines of code changes. Hence if have the same request as our team, you'd better try it as well.

 

Hope this helps,

Shaun

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Xu. This work is licensed under the Creative Commons License.


Worktile Pro contains three main modules: task, calendar and drive. In calendar module user can create events such as meeting, appointment with participants, location, resources and recurrence. It also supports multiple types of reminders such as Worktile build-in message, email, SMS and phone-call. But some of our customers would like to us it in some third calendar application, such as Mac OS Calendar, Google Calendar, etc.. In this post I will describe how to build a calendar subscription service in Node.js with iCal protocol.

 

Overall Workflow

First of all, let's have a look on how it would be. The events belongs to my Worktile account had been synchronized with the Calendar.app on my MacBook Pro.

2016-03-18_12-19-35

The overall workflow is very simple. First I opened my subscription URL, which begin with WEBCAL protocol in Safari. Then it opened Calendar.app and asked to subscribe this calendar source.

image

When I said "Yes", Calendar.app will use HTTP/HTTPS to replace the WEBCAL protocol to request events from Worktile.

image

Worktile received my request, find events related with me in the timeframe predefined in system. Then it generate a file in iCal format and respond.

image

Calendar.app received this iCal file and parse to events to be shown in the UI. After a period of time (5 minutes I specified) Calendar.app will send the same request to download iCal file, compare and update them.

image

Most of calendar applications support requesting iCal file periodically. So any updates in your system will be upgraded to applications subscribing your service several minutes or hours later.

 

Create iCal File

iCal is a computer file format which allows Internet users to send meeting requests and tasks to other Internet users, via email, or sharing files with an extension of ".ics". It just takes responsible for transmitting calendar-based data, such as events, and intentionally does not describe what to do with that data.

An iCal file is just a plaint text file. The first line must be "BEGIN:VCALENDAR", and the last line must be "END:VCALENDAR". Between them it can contains one or more events encapsulated by "BEGIN:VEVENT" and "END:VEVENT". Each VEVENT has many properties and alarms. It can also contains "To-Do", "Journal", "Free Busy Time" etc. but in this post we only need VCALENDAR, VEVENT and VALARM.

image

Below are some properties we are currently using and support in Worktile Pro.

VCALENDAR Section

CALSCALE Calendar type, default is "GREGORIAN".
VERSION iCal file format version, default is "2.0".
X-WR-CALNAME Calendar name shown in local application such as Calendar.app.
METHOD How this calendar will be subscribed, default is "PUBLIC".
PRODID The unique name of the production which published this calendar. It should following ISO.9070.1991. Worktile Pro Calendar is "-//YC Tech//NONSGML Worktile Pro Calendar//CN".

VEVENT Section

CALSCALE Calendar type, default is "GREGORIAN".
UID Identifier for each event. It should NOT be changed between each subscribe response against the same event.
DTSTAMP Timestamp of event. It should always be date and time.
DTSTART The start time of this event with time specified.
DTSTART;VALUE=DATE The start date of this event when it's a all-day-event.
DTEND The end time of this event with time specified.
DTEND;VALUE=DATE The end date of this event when it's a all-day-event.
SUMMARY Event title.
DESCRIPTION Event description.
LOCATION Event location.
CLASS Classification of this event. In Worktile Pro we use this field to identify whether this is a "PUBLIC" or "PRIVATE" event.
CATEGORIES We use this field to specify which calendar this event belongs to in Worktile system.
ORGANIZER;CN= Organizer of this event. The name will be append to the key and value would be his/her mail address. For example "ORGANIZER;CN=Shaun Xu:shaun@worktile.com".
ATTENDEE;CN= Attendees of this event. Same as organizer but each attendee should be in one line.

The value of DTSTAMP, DTSTART and DTEND should be in ISO format in UTC time. For example, if an event will be started at Mar 18 2016 15:42:40 in Beijing time (UTC+8), the value should be "20160318T074240Z". If this is an all-day-event, the time part will be removed and the value should be "20160318"

VALARM Section

X-WR-ALARMUID Alarm identifier which should NOT be changed against the same alarm.
UID Same as below.
TRIGGER When this alarm should be fired.
ACTION How this alarm will be remind, default is "DISPLAY".

The value of TRIGGER should be in duration format. If it should alarm 5 minutes before the value would be "-PT5M". We can use moment.js to generate the value.

 

In order to make it easier to create iCal file below is the source code I'm using in Worktile Pro.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
(function () {
    'use strict';

    var _ = require('lodash');

    var VCalendar = function (calName, calScale, version, method, prodId) {
        this._headers = [
            {
                key: 'BEGIN',
                value: 'VCALENDAR'
            },
            {
                key: 'CALSCALE',
                value: calScale || 'GREGORIAN'
            },
            {
                key: 'VERSION',
                value: version || '2.0'
            },
            {
                key: 'X-WR-CALNAME',
                value: calName
            },
            {
                key: 'METHOD',
                value: method || 'PUBLISH'
            },
            {
                key: 'PRODID',
                value: prodId
            }
        ];
        this._events = [];
        this._footer = [
            {
                key: 'END',
                value: 'VCALENDAR'
            }
        ];
    };

    var _formatDate = function (date, isAllDay) {
        var noMsDate = new Date(date.getTime());
        noMsDate.setMilliseconds(0);
        var noMsDateISOString = noMsDate.toISOString();
        var icsDateString = noMsDateISOString.replace(/-/g, '').replace(/:/g, '').replace(/\.000/g, '');
        return isAllDay ? icsDateString.substring(0, icsDateString.indexOf('T')) : icsDateString;
    };

    var VEvent = function (id,
                           startDate, endDate, isAllDay,
                           name, description, location, visibility, category,
                           orgnizer, attendees,
                           createdDate, lastModifiedDate) {
        var self = this;
        self._headers = [
            {
                key: 'BEGIN',
                value: 'VEVENT'
            }
        ];
        self._props = [
            {
                key: 'TRANSP',
                value: 'TRANSPARENT'
            },
            {
                key: 'UID',
                value: id
            },
            {
                key: 'DTSTAMP',
                value: _formatDate(startDate, false)
            },
            {
                key: 'DTSTART' + (isAllDay ? ';VALUE=DATE' : ''),
                value: _formatDate(startDate, isAllDay)
            },
            {
                key: 'DTEND' + (isAllDay ? ';VALUE=DATE' : ''),
                value: _formatDate(endDate, isAllDay)
            },
            {
                key: 'SUMMARY',
                value: name
            },
            {
                key: 'DESCRIPTION',
                value: description
            },
            {
                key: 'LOCATION',
                value: location
            },
            {
                key: 'CLASS',
                value: visibility || 'PUBLIC'
            },
            {
                key: 'CATEGORIES',
                value: category
            },
            {
                key: 'STATUS',
                value: 'CONFIRMED'
            },
            {
                key: 'SEQUENCE',
                value: 0
            },
            {
                key: 'CREATED',
                value: _formatDate(createdDate, false)
            },
            {
                key: 'LAST-MODIFIED',
                value: _formatDate(lastModifiedDate, false)
            },
            {
                key: 'ORGANIZER;CN=' + orgnizer.name,
                value: orgnizer.email
            }
        ];
        _.forEach(attendees, function (attendee) {
            self._props.push({
                key: 'ATTENDEE;CN=' + attendee.name,
                value: attendee.email
            });
        });
        self._alarms = [];
        self._footer = [
            {
                key: 'END',
                value: 'VEVENT'
            }
        ];
    };

    VEvent.prototype.addAlarm = function (id, trigger, description, action) {
        var self = this;
        var alarm = [
            {
                key: 'BEGIN',
                value: 'VALARM'
            },
            {
                key: 'X-WR-ALARMUID',
                value: id
            },
            {
                key: 'UID',
                value: id
            },
            {
                key: 'TRIGGER',
                value: trigger
            },
            {
                key: 'DESCRIPTION',
                value: description
            },
            {
                key: 'ACTION',
                value: action || 'DISPLAY'
            },
            {
                key: 'END',
                value: 'VALARM'
            }
        ];
        self._alarms.push(alarm);
    };

    VCalendar.prototype.addEvent = function (id,
                                             startDate, endDate, isAllDay,
                                             name, description, location, visibility, category,
                                             orgnizer, attendees,
                                             createdDate, lastModifiedDate) {
        var self = this;
        var event = new VEvent(id,
            startDate, endDate, isAllDay,
            name, description, location, visibility, category,
            orgnizer, attendees,
            createdDate, lastModifiedDate);
        self._events.push(event);
        return event;
    };

    VCalendar.prototype._toICSLine = function (key, value) {
        if (_.isEmpty(key) || _.isEmpty(value)) {
            return null;
        }
        else {
            value = _.isString(value) ? value : value.toString();

            key = key.replace(/[\\;,\n]/g, function (match) {
                return (match === '\n' ? '\\n' : ('\\' + match));
            });
            value = value.replace(/[\\;,\n]/g, function (match) {
                return (match === '\n' ? '\\n' : ('\\' + match));
            });
            return key + ':' + value;
        }
    };

    VCalendar.prototype.toICSString = function () {
        var self = this;
        var lines = [];
        var line = null;
        _.forEach(self._headers, function (header) {
            line = self._toICSLine(header.key, header.value);
            if (!_.isEmpty(line)) {
                lines.push(line);
            }
        });
        _.forEach(self._events, function (event) {
            _.forEach(event._headers, function (header) {
                line = self._toICSLine(header.key, header.value);
                if (!_.isEmpty(line)) {
                    lines.push(line);
                }
            });
            _.forEach(event._props, function (prop) {
                line = self._toICSLine(prop.key, prop.value);
                if (!_.isEmpty(line)) {
                    lines.push(line);
                }
            });
            _.forEach(event._alarms, function (alarm) {
                _.forEach(alarm, function (prop) {
                    line = self._toICSLine(prop.key, prop.value);
                    if (!_.isEmpty(line)) {
                        lines.push(line);
                    }
                });
            });
            _.forEach(event._footer, function (footer) {
                line = self._toICSLine(footer.key, footer.value);
                if (!_.isEmpty(line)) {
                    lines.push(line);
                }
            });
        });
        _.forEach(self._footer, function (footer) {
            line = self._toICSLine(footer.key, footer.value);
            if (!_.isEmpty(line)) {
                lines.push(line);
            }
        });
        return lines.join('\r\n');
    };

    exports.createNew = function (calName, calScale, version, method, prodId) {
        return new VCalendar(calName, calScale, version, method, prodId);
    };

})();

I will show the code how to use it.

 

Build Subscription Web Service

Now let's create a very simple web service to demonstrate iCal subscription. The code below is a simple web service with one API on top of Node.js and Express.js.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
(() => {
    'use strict';

    const express = require('express');
    const app = express();

    app.get('/api/calendar/subscribe', (req, res) => {
        // calendar subscription code will be here
        res.send('Hello iCal!');
    });

    const port = 22222;
    app.listen(port);
    console.log(`Application started at ${port}`);

})();

 

Add the reference of iCal source code I mentioned above. Now we can generate our iCal content. In Worktile Pro I need to validate the request and find the user it associated, then connect to calendar service to retrieve events in a timeframe (30 days by default). Then push events into iCal file. But in this post I will just create an iCal file directly.

1
2
3
4
5
6
7
8
9
app.get('/api/calendar/subscribe', (req, res) => {
    // for demo purpose just create iCal on the fly
    let iCal = ics.createNew('iCal Demo by Shaun Calendar', null, null, null, '-//Shaun Xu//NONSGML iCal Demo Calendar//EN');
    let iCalString = iCal.toICSString();

    res.set('Content-Type', 'text/calendar;charset=utf-8');
    res.set('Content-Disposition', 'attachment; filename="worktile.pro.calendar.my.ics"');
    res.send(iCalString);
});

Now if we refresh the web browser it will download this empty iCal file. The content would be like this.

BEGIN:VCALENDAR
CALSCALE:GREGORIAN
VERSION:2.0
X-WR-CALNAME:iCal Demo by Shaun Calendar
METHOD:PUBLISH
PRODID:-//Shaun Xu//NONSGML iCal Demo Calendar//EN
END:VCALENDAR

If we opened Calendar.app and clicked File > New Calendar Subscription and paste the URL, it will request iCal file and show events.

image

In the dialog below, make sure you unchecked "Remove: Alerts" and adjust the value of "Auto-refresh".

image

Then you can see the calendar had been subscribed in Calendar.app.

image

 

Now let's change our service implementation to push one event into iCal.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
let iCal = ics.createNew('iCal Demo by Shaun Calendar', null, null, null, '-//Shaun Xu//NONSGML iCal Demo Calendar//EN');
iCal.addEvent(
    '1',
    new Date(2016, 2, 18, 17, 0, 0), new Date(2016, 2, 18, 17, 30, 0), false,
    'Meet with anytao', 'Meet with anytao to talk about Worktile Pro', 'Conf.Rm.1', null, null,
    {
        name: 'Shaun Xu',
        email: 'shaun@worktile.com'
    },
    [
        {
            name: 'Shaun Xu',
            email: 'shaun@worktile.com'
        },
        {
            name: 'anytao',
            email: 'anytao@worktile.com'
        }
    ],
    new Date(), new Date());
let iCalString = iCal.toICSString();

Restart the service and wait at most 5 minutes, we will find Calendar.app had updated.

image

As you can see, the event name, description, location, start and end time, organizer and attendees were all be synchronized.

 

I can also add alarm if the source event contains reminders. For example, the code below I added an alarm which will be fired 5 minutes before it start.

I used "moment.js" to generate alarm trigger value. Just make sure to specify value in minus which means how long *before* the event starts.

1
2
3
4
5
6
let iCal = ics.createNew('iCal Demo by Shaun Calendar', null, null, null, '-//Shaun Xu//NONSGML iCal Demo Calendar//EN');
let event = iCal.addEvent(
        // ... ...
    );
event.addAlarm('1', moment.duration(-5, 'm').toString(), null, null);
let iCalString = iCal.toICSString();

Several minutes later Calendar.app received this update.

image

If we allow Calendar.app to send notification it will display the alarm at 16:55, 5 minutes before it starts.

 

Use "WEBCAL" Protocol

Now we can use "WEBCAL" protocol to let web browser open Calendar.app or any proper application to subscribe. In fact, WEBCAL protocol is just HTTP/HTTPS. When browser detect that it's "WEBCAL", it will try to open a calendar application on local machine and ask to subscribe. We don't need to do anything to support "WEBCAL".

Now let's open Safari and go to the URL we had just implemented, just replace "HTTP" to "WEBCAL".

image

Then Calendar.app will be opened and subscription dialog will be there automatically.

image

And we will subscribe this calendar.

image

Summary

In this post I demonstrated on how to implement a calendar subscription service through iCal format in Node.js. Basically it is just a HTTP/HTTPS endpoint which return a plaint text in iCal format. Client calendar application, such as Calendar.app in Mac, will request this endpoint periodically to update events.

 

Hope this helps,

Shaun

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Xu. This work is licensed under the Creative Commons License.


There is an enhancement in ECMAScript 6 named "Arrow Functions", which likes lambda expression in C#, allow us to define a function within less lines of code. I like this new feature and began to use it in my Node.js application as many as I could. But today when II  was using JavaScript build-in "arguments" variant I found something wrong.

 

Assuming we have a module to add parameters, which is very simple. I'm using Arrow Functions to implement as below.

// calc.js

(() => {
    'use strict';

    exports.add = (x, y) => {
        return x + y;
    };
})();

Then I can use it as below.

// app.js

(() => {
    'use strict';

    const calc = require('./calc.js');

    let x = 2;
    let y = 3;
    let result1 = calc.add(x, y);
    console.log(`${x} + ${y} = ${result1}`);

})();

 

Now I created another method in my module allows user to input multiple numbers to add. In traditional JavaScript way I don't need to define arguments in the function. I can use "arguments" variant, which is a "semi-array" object contains parameters, add each of them and return the summary.

// calc.js
(() => {
    'use strict';

    exports.add = (x, y) => {
        return x + y;
    };

    exports.addMany = () => {
        let args = [].slice.call(arguments);
        let result = 0;
        for (let x of args) {
            result += x;
        }
        return result;
    };

})();

// app.js
(() => {
    'use strict';

    const calc = require('./calc.js');

    let x = 2;
    let y = 3;
    let result1 = calc.add(x, y);
    console.log(`${x} + ${y} = ${result1}`);

    let x1 = 1;
    let x2 = 2;
    let x3 = 3;
    let x4 = 4;
    let x5 = 5;
    let x6 = 6;
    let x7 = 7;
    let result2 = calc.addMany(x1, x2, x3, x4, x5, x6, x7);
    console.log(`result2 = ${result2}`);

})();

 

But when I ran this application I got an error below.

I'm using Node.js v5.7.0 which supports ES6 features.

Screen Shot 2016-03-15 at 15.55.10

 

If we read Arrow Function specification carefully we will find that it captures the "this" value of the enclosing context, so the following code works as expected. This provides convenient to use parent "this" inside arrow function without needing to specify another variant to hold parent's "this" value. But the side effect is, it also captures the "arguments" value from the parent context.

In my code I defined "addMany" function in arrow function mode. It copied "this" from parent context, which is the whole module, as well as "arguments", which is the module loading function arguments.

Screen Shot 2016-03-15 at 16.03.29

To fix this problem, just simply define this function normally as below. It will use its own "this" and "arguments".

exports.addMany = function () {
    let args = [].slice.call(arguments);
    let result = 0;
    for (let x of args) {
        result += x;
    }
    return result;
};

 

Screen Shot 2016-03-15 at 16.05.54

Alternatively, if you are OK to enable one of Node.js ES6 staging features called "Rest Parameters" you can define the function as below, which allows parameters to be passed in as a real array.

exports.addMany = (...args) => {
    let result = 0;
    for (let x of args) {
        result += x;
    }
    return result;
};

Then execute this application with Node.js options called "--harmony_rest_parameters".

Screen Shot 2016-03-15 at 16.10.03

 

Hope this helps,

Shaun

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Xu. This work is licensed under the Creative Commons License.


Features always grows much faster than we expect. When I am build Worktile Pro I created a JavaScript file contains all business logic in class for the task module. But after several months development it became over 7000 lines of code, which is horrible. Last week I decided to split it into multiple files.

It may not be a big problem to split one JavaScript file into multiple, especially in Node.js environment. We can put functions and variables into many files as we wanted and "require" them in the "main" file. But if we want to split a class definition into multiple files that might not work. In JavaScript a class is a function in essential, and it can be defined only in one file. For example, in the code below I defined a class named "MissionService" with some method in file "mission.js".

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
(function () {
    'use strict';

    var MissionService = function ()  {
    };

    MissionService.prototype.createTask = function (taskName) {
        console.log('Task: "' + taskName + '" was created.');
    };

    MissionService.prototype.loadTask = function (taskId) {
        console.log('Task (' + taskId + ') was loaded.');
    };

    MissionService.prototype.updateTask = function (taskId, taskName) {
        console.log('Task (' + taskId + ') was changed to "' + taskName + '".');
    };

    MissionService.prototype.removeTask = function (taskId) {
        console.log('Task (' + taskId + ') was removed.');
    };

    MissionService.prototype.restoreTask = function (taskId) {
        console.log('Task (' + taskId + ') was restoreTask.');
    };

    exports = module.exports = MissionService;

})();

 

First step is to move the class definition into an "index" file, which will "require" all following files later. As you can see this "index.js" file only contains the class definition and exports it.

1
2
3
4
5
6
7
8
9
(function () {
    'use strict';

    var MissionService = function ()  {
    };

    exports = module.exports = MissionService;

})();

 

Now we can create a "partial" class definition file based on the "index" I created. Just exports a function which allow the class can be passed so that I can define its methods through "PartialClass.prototype".

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
// mission.create.js
(function () {
    'use strict';

    exports = module.exports = function (MissionService) {

        MissionService.prototype.createTask = function (taskName) {
            console.log('Task: "' + taskName + '" was created.');
        };

    };
})();

// mission.update.js
(function () {
    'use strict';

    exports = module.exports = function (MissionService) {

        MissionService.prototype.updateTask = function (taskId, taskName) {
            console.log('Task (' + taskId + ') was changed to "' + taskName + '".');
        };

        MissionService.prototype.removeTask = function (taskId) {
            console.log('Task (' + taskId + ') was removed.');
        };

        MissionService.prototype.restoreTask = function (taskId) {
            console.log('Task (' + taskId + ') was restoreTask.');
        };

    };
})();

// mission.find.js
(function () {
    'use strict';

    exports = module.exports = function (MissionService) {

        MissionService.prototype.loadTask = function (taskId) {
            console.log('Task (' + taskId + ') was loaded.');
        };

    };
})();

 

Now back to the "index" file, what we need to do is to "require" this partial class file, put the class we defined into the parameter so that it will attach methods.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
(function () {
    'use strict';

    var MissionService = function ()  {
    };

    require('./mission.create.js')(MissionService);
    require('./mission.update.js')(MissionService);
    require('./mission.find.js')(MissionService);

    exports = module.exports = MissionService;

})();

 

Finally when we want to use this class, just "require" the "index" file and "new" an instance as below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
(function () {
    'use strict';

    var MissionService = require('./index.js');
    var mission = new MissionService();

    mission.createTask('Shaun\'s task.');
    mission.loadTask(1);
    mission.updateTask(1, 'Shaun\'s new task.');
    mission.removeTask(1);
    mission.updateTask(1);

})();

 

Screen Shot 2016-03-07 at 10.34.23

If we have some internal helper functions or variants we can put them into some "shared" files. 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
(function () {
    'use strict';

    exports = module.exports = function (MissionService) {

        MissionService.prototype._log = function (message) {
            console.log(message);
        };

    };
})();

 

Just ensure we "require" them before we "require" PartialClass files that are using them.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
// index.js
(function () {
    'use strict';

    var MissionService = function ()  {
    };

    require('./mission.shared.js')(MissionService);

    require('./mission.create.js')(MissionService);
    require('./mission.update.js')(MissionService);
    require('./mission.find.js')(MissionService);

    exports = module.exports = MissionService;

})();

// mission.create.js
(function () {
    'use strict';

    exports = module.exports = function (MissionService) {

        MissionService.prototype.createTask = function (taskName) {
            this._log('Task: "' + taskName + '" was created.');
        };

    };
})();

// mission.update.js
(function () {
    'use strict';

    exports = module.exports = function (MissionService) {

        MissionService.prototype.updateTask = function (taskId, taskName) {
            this._log('Task (' + taskId + ') was changed to "' + taskName + '".');
        };

        MissionService.prototype.removeTask = function (taskId) {
            this._log('Task (' + taskId + ') was removed.');
        };

        MissionService.prototype.restoreTask = function (taskId) {
            this._log('Task (' + taskId + ') was restoreTask.');
        };

    };
})();

// mission.find.js
(function () {
    'use strict';

    exports = module.exports = function (MissionService) {

        MissionService.prototype.loadTask = function (taskId) {
            this._log('Task (' + taskId + ') was loaded.');
        };

    };
})();

 

At the end, we can put all files into a folder and rename the "index" as "index.js". Now we could require our class by the folder name, which is more friendly.
Screen Shot 2016-03-07 at 10.40.50

 

Hope this helps,

Shaun

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Xu. This work is licensed under the Creative Commons License.


In Worktile Pro we are using MongoDB and Mongoose as the backend database with Mongoose Node.js module for ODM (Object Document Mapping) framework.

Although Mongoose brings "Schema" back it still provides some advantages such as validation, abstraction, reference and default value. In this post I will talk about the default value based on a bug I found when developing Worktile Pro.

 

We can define default value against a model  schema. In the code below I have a blog schema, with default value against "created_by" and "created_at" properties.

1
2
3
4
5
let BlogSchema = new Schema({
    title: {type: String},
    created_by: {type: String, default: 'Shaun Xu'},
    created_at: {type: Number, default: Date.now}
});

If we tried to add a new blog document without specifying them, Mongoose will use default value as below.

1
2
3
4
5
6
7
8
let blog1 = new BlogModel({
    title: 'Implement Schema Inherence in Mongoose'
});

co(function *() {
    let savedBlog1 = yield blog1.save();
    console.log(`Blog: ${JSON.stringify(savedBlog1, null, 2)}`);
});

 

But if we specified "null", they will be marked as "defined" so that Mongoose will set "null" rather than the default value.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
let blog2 = new BlogModel({
    title: 'Angular Toolkits: Customizable Bootstrap Tabs',
    created_by: null,
    created_at: null
});

co(function *() {
    let savedBlog2 = yield blog2.save();
    console.log(`Blog: ${JSON.stringify(savedBlog2, null, 2)}`);
});

 

How about "undefined"? Let's look at the code below. Will Mongoose use default value?

1
2
3
4
5
let blog3 = new BlogModel({
    title: 'Pay Attention to "angular.merge" in 1.4 on Date Properties',
    created_by: undefined,
    created_at: undefined
});

The result was shown below, Mongoose didn't use default value for "blog3".

Screen Shot 2016-01-26 at 17.36.27

This means even though we set property to "undefined", Mongoose will NOT use default value. This is because Mongoose just check if the document object contains properties. If yes, no matter it has value, it's "null", or even it's "undefined", it was defined. So Mongoose will NOT use default value.

 

So finally rule: If you want Mongoose to use default value on a property, do NOT specify it in the document object, even though set it to "null" or "undefined".

 

Hope this helps,

Shaun

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Xu. This work is licensed under the Creative Commons License.