Showing posts with label Node. Show all posts
Showing posts with label Node. Show all posts

Sunday, October 29, 2017

MQTT: Publishing and Subscribing with Node.js

This short tutorial demonstrates how to communicate with MQTT using Node.js. We will:

 

MQTT.js Library

A very popular MQTT library for Node.js is MQTT.js, and for good reasons:

  • it is actively supported
  • it supports most features of MQTT 3.1.1
  • it has good documentation
  • it can be used in async mode
  • and it works in a browser, too!

We'll be using this library.

 

Setup Test Environment

We will be using the same MTT topic from an earlier post: home/living-room/temperature

We need three Terminal windows:

  1. In the first one, we start the broker using the command:
    mosquitto
    Notice I omitted the -v verbose flag.
  2. In the second window, we start a subscriber listening to the topic
    mosquitto_sub -h localhost -t home/living-room/temperature
    Notice that I removed the -d debug flag. This way output will be cleaner.
  3. In the third Terminal window, publish a message using
    mosquitto_pub -d -h localhost -t home/living-room/temperature -m 'Hello, MQTT!'
    and the message should appear in the second window.

This is how those three windows should look:

 

Setup a Node.js Project

Now create a Node.js app using these steps:

  1. Open yet another Terminal window
  2. Create a folder named "MQTT with NodeJS":
    mkdir "MQTT with NodeJS"
  3. Move into that window with:
    cd "MQTT with NodeJS"
  4. Create a simple Node.js project with:
    npm init
  5. Answer some questions that will create a basic package.json
    - give the project the name "mqtt-with-node-js"
  6. Accept the default package.json file
  7. Install MQTT.js using
    npm install mqtt --save

This last command will create a subfolder called node_modules and a package.json file that looks something like this:

{
  "name": "mqtt-with-node-js",
  "version": "1.0.0",
  "description": "Demonstrates how to communicate with MQTT",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [
    "MQTT",
    "NodeJS"
  ],
  "author": "Mike Klepper",
  "license": "ISC",
  "dependencies": {
    "mqtt": "^2.12.0"
  }
}

 

Publishing to a MQTT Topic

Create a new file called publish.js in the same folder as package.js, and edit it to read:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
/**
 * publish.js
 *
 * By: Mike Klepper
 * Date: 29 October 2017
 *
 * This program demonstrates how to connect to MQTT
 *
 * See blog post on patriot-geek.blogspot.com
 * for instructions.
 */

const BROKER_URL = "mqtt://localhost:1883";
const TOPIC_NAME = "home/living-room/temperature";
const CLIENT_ID = "publish.js";

var MQTT = require("mqtt");
var client  = MQTT.connect(BROKER_URL, {clientId: CLIENT_ID});

client.on("connect", onConnected);

function onConnected()
{
  client.publish(TOPIC_NAME, "Hello MQTT from NodeJS!");
  client.end();
}

Here's the walkthrough:

13-15 Declare constants for the broker address, topic, and client name. Note that the address must specify the "mqtt" protocol!
17 Import the MQTT.js package
18 Create a client and connect to the broker
20 Create a listener that waits for CONACK, then calls the callback named onConnected
22 Once we're connected...
24 ...publish a message to the topic and...
25 ...close the connection.

What QoS does the MQTT.js library use when publishing? By default it is QoS 0. We can change that by modifying line 24 to read: 

24
client.publish(TOPIC_NAME, "Hello MQTT from NodeJS!", {qos: 1});

We can further simplify the code by omitting the clientId from the connect statement, in which case that line will read 

18
var client = MQTT.connect(BROKER_URL);

Notice that we assume that this line 

24
client.publish(TOPIC_NAME, "Hello MQTT from NodeJS!");
finishes before we close the connection. 
25
client.end();
Fortunately, the MQTT.js library does handle that situation by default.

While that code was extremely easy to read, it is only "happy path" code - there is no error handling! The library allows for this by providing callbacks for client.publish client.end and these callbacks can be used to catch and handle errors.

 

Subscribing to Single or Multiple Topics

The next program demonstrates how to subscribe to single or multiple topics. First, create a file called subscribe.js and modify it to read:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
/**
 * subscribe.js
 *
 * By: Mike Klepper
 * Date: 29 October 2017
 *
 * This program demonstrates how to subscribe to MQTT
 *
 * See blog post on patriot-geek.blogspot.com
 * for instructions.
 */

const BROKER_URL = "mqtt://localhost:1883";
const TOPIC_NAME = "home/living-room/temperature";
const CLIENT_ID = "subscribe.js";

var MQTT = require("mqtt");
var client  = MQTT.connect(BROKER_URL, {clientId: CLIENT_ID});

client.on("connect", onConnected);
client.on("message", onMessageReceived)

function onConnected()
{
  client.subscribe(TOPIC_NAME);
}

function onMessageReceived(topic, message)
{
  console.log(topic);
  console.log(message.toString());
  console.log("");
}

To test this program, run it with

node subscribe.js
At which point nothing appears to happen! Then go to the third Terminal window and publish a message:
mosquitto_pub -d -h localhost -t home/living-room/temperature -m 'Hello, MQTT!'
And the message should be displayed in the window running subscribe.js!

Walkthrough:

20 - 21 Set event handlers for when the client connects and it receives a message
23 When the client connects...
25 ...have it subscribe to the desired topic
28-33 When client receives a message, print the topic, the message, and a new line.

Why is the .toString() method called in line 31? It isn't necessary for printing the topic, since the topic is a string, whereas the the message is a string buffer and not a string. If we removed the toString(), the output will be 

home/living-room/temperature
<Buffer 48 65 6c 6c 6f 2c 20 4d 51 54 54 21>

Now let's modify this program to subscribe to wildcard topics. Change line 14 to read: 

14
const TOPIC_NAME = "home/living-room/+";

Run the program. In the Third terminal window, enter:

mosquitto_pub -d -h localhost -t home/living-room/temperature -m 'Hello, MQTT! This is the temperature!'

then enter:

mosquitto_pub -d -h localhost -t home/living-room/humidity -m 'Hello, MQTT! This is the humidity!'

In the window with the program running we should see: 

home/living-room/temperature
Hello, MQTT! This is the temperature!

home/living-room/humidity
Hello, MQTT! This is the humidity!

So we have indeed subscribed to a path with a wildcard in it!

Let's experiment. Try this command:

mosquitto_pub -d -h localhost -t home/living-room/temperature/fahrenheit -m 'Hello, MQTT! This is the temperature in Fahrenheit'
This is NOT displayed in the window running the program. This is because the "+" wildcard represents only one level in a topic.

Notice that this program continues to run until we press Ctrl-C. This is a good thing, since we usually want subscribers to continuously listen for incoming messages.

Posted by at 8 comments:
Labels: , , , ,

Tuesday, September 5, 2017

MongoDB: Connecting with Node.js

This tutorial demonstrates how to

 

Introduction

There are two popular libraries for connecting to MongoDB from Node.js:
  • Native Mongo Driver
  • Mongoose

Native Mongo Driver, also called node-mongodb-native, has official support from the creators of MongoDB, and the commands it uses are similar to those used in the MongoDB shell.

Mongoose provides an ODM (Object -> Document Mapping) service. This allows the imposition of schemas on our schemaless MongoDB! It also provides field-level validations, which prevents the insertion of unexpected values in a document's fields. Mongoose is built atop Native Mongo Driver.

We will be using the first one in this tutorial. We will create several JS files that reproduce our "Queer Eye for the Straight Guy" database from a previous post.

First, start the MongoDB daemon in a Terminal window, then start the MongoDB Shell in another Terminal window. If the qeftsg database is still present, remove it using the following commands:

use qeftsg
db.cast.drop();

Now we are ready to recreate the database using Node.js!

 

Creating a Node.js Project

Assuming that Node.js and NPM are installed, open a new Terminal window and create a simple project:

mkdir testproject
cd testproject
npm init

After answering several questions, the following package.js file will be created:

{
  "name": "testproject",
  "version": "1.0.0",
  "description": "Demonstrates how to connect to MongoDB with Node.js",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [
    "MongoDB",
    "Node.js"
  ],
  "author": "Mike Klepper",
  "license": "ISC"
}

Now we install the Native Mongo Driver:

npm install mongodb --save

Once this command has finished running, there are two changes to the TestProject folder:

  1. There is a new folder called node_modules
  2. The project.json file has been modified: a "dependencies" section has been added:
{
  "name": "testproject",
  "version": "1.0.0",
  "description": "Demonstrates how to connect to MongoDB with Node.js",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [
    "MongoDB",
    "Node.js"
  ],
  "author": "Mike Klepper",
  "license": "ISC",
  "dependencies": {
    "mongodb": "^2.2.31"
  }
}

 

Connecting to MongoDB

First, create a file called connect.js in the same folder as package.json, and edit it to read:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
/**
 * connect.js
 *
 * By: Mike Klepper
 * Date: 3 September 2017
 *
 * This program demonstrates how to connect to MongoDB
 *
 * See blog post on patriot-geek.blogspot.com
 * for instructions.
 */

var MongoClient = require("mongodb").MongoClient;

const DATABASE_NAME = "qeftsg";
const URL = "mongodb://localhost:27017/" + DATABASE_NAME;

MongoClient.connect(URL, onConnected);

function onConnected(err, db)
{
  if(!err)
  {
    console.log("Connected to MongoDB!");
    closeConnection(err, db);
  }
  else
  {
    console.log("Problems connecting to the DB: " + err);
  }
}

function closeConnection(err, db)
{
  console.log("Closing connection");
  db.close();
}

Run the program as follows:

node connect.js
The output indicates that we successfully connected: 
Connected to MongoDB!
Closing connection

Great, we've connected to the database!

 

Create

Now let's create the "cast" collection and populate it with the Fab Five. Create a new program called create.js:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
/**
 * create.js
 * 
 * By: Mike Klepper
 * Date: 3 September 2017
 * 
 * This program creates five records in MongoDB
 * 
 * See blog post on patriot-geek.blogspot.com
 * for instructions.
 */

var MongoClient = require("mongodb").MongoClient;

const DATABASE_NAME = "qeftsg";
const URL = "mongodb://localhost:27017/" + DATABASE_NAME;

var ted = {
 name: "Ted Allen",
 description: "Food and Wine Connoisseur",
 specialties: ["Alcohol", "Beverages", "Food Preparation", "Food Presentation"],
 age: 52,
 contactInfo:
 {
  website: "tedallen.net",
  email: "info@tedallen.net"
 }
};

var carson = {
 name: "Carson Kressley",
 description: "Fashion Savant",
 specialties: ["Clothing", "Fashion", "Personal Styling"],
 age: 47,
 contactInfo:
 {
  website: "carsonkressley.com",
  facebook: "carsonkressley"
 }
};

var kyan = {
 name: "Kyan Douglas",
 description: "Grooming Guru",
 specialties: ["Hair", "Grooming", "Personal Hygiene", "Makeup"],
 age: 47,
 contactInfo:
 {
  instagram: "kyandouglas",
  facebook: "kyandouglasactual"
 }
};

var thom = {
 name: "Thom Filicia",
 description: "Design Doctor",
 specialties: ["Interior Design", "Home Organization"],
 age: 48,
 contactInfo:
 {
  facebook: "thomfiliciainc",
  email: "info@thomfilicia.com",
  website: "thomfilicia.com"
 }
};

var jai = {
 name: "Jai Rodriguez",
 description: "Culture Vulture",
 specialties: ["Popular Culture", "Relationships", "Social Interaction"],
 age: 38,
 contactInfo:
 {
  website: "myspace.com/jairodriguezmusic",
  facebook: "JaiRodriguezfanpage"
 }
};

MongoClient.connect(URL, onConnected);

function onConnected(err, db)
{
  if(!err)
  {
    console.log("Connected to MongoDB!");
    insertCast(db, closeConnection);
  }
  else
  {
    console.log("Problems connecting to the DB: " + err);
  }
}

function insertCast(db, callback)
{
  var cast = db.collection("cast");
  cast.insertMany([ted, carson, kyan, thom, jai], onInsertManyComplete);
  callback(null, db);
}

function onInsertManyComplete(err, result)
{
  if(!err)
  {
    console.log("Inserted " + result.result.n + " documents into the document collection");
    console.dir(result);
  }
  else
  {
    console.log("Problems inserting records: " + err);
  }
}

function closeConnection(err, db)
{
  console.log("Closing connection");
  db.close();
}

We run the program by entering node create.js and it outputs the following:

Connected to MongoDB!
Closing connection
Inserted 5 documents into the document collection
{ result: { ok: 1, n: 5 },
  ops: 
   [ { name: 'Ted Allen',
       description: 'Food and Wine Connoisseur',
       specialties: [Object],
       age: 52,
       contactInfo: [Object],
       _id: [Object] },
     { name: 'Carson Kressley',
       description: 'Fashion Savant',
       specialties: [Object],
       age: 47,
       contactInfo: [Object],
       _id: [Object] },
     { name: 'Kyan Douglas',
       description: 'Grooming Guru',
       specialties: [Object],
       age: 47,
       contactInfo: [Object],
       _id: [Object] },
     { name: 'Thom Filicia',
       description: 'Design Doctor',
       specialties: [Object],
       age: 48,
       contactInfo: [Object],
       _id: [Object] },
     { name: 'Jai Rodriguez',
       description: 'Culture Vulture',
       specialties: [Object],
       age: 38,
       contactInfo: [Object],
       _id: [Object] } ],
  insertedCount: 5,
  insertedIds: 
   [ ObjectID { _bsontype: 'ObjectID', id: [Object] },
     ObjectID { _bsontype: 'ObjectID', id: [Object] },
     ObjectID { _bsontype: 'ObjectID', id: [Object] },
     ObjectID { _bsontype: 'ObjectID', id: [Object] },
     ObjectID { _bsontype: 'ObjectID', id: [Object] } ] }

 

Retrieve

Now let's retrieve the records. Create a new file called retrieve.js and add the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
/**
 * retrieve.js
 *
 * By: Mike Klepper
 * Date: 3 September 2017
 *
 * This program retrieves all records in the cast collection
 *
 * See blog post on patriot-geek.blogspot.com
 * for instructions.
 */

var MongoClient = require("mongodb").MongoClient;

const DATABASE_NAME = "qeftsg";
const URL = "mongodb://localhost:27017/" + DATABASE_NAME;

var database = null;
MongoClient.connect(URL, onConnected);

function onConnected(err, db)
{
  if(!err)
  {
    console.log("Connected to MongoDB!");
    database = db;
    findEverybody(db, closeConnection);
  }
  else
  {
    console.log("Problems connecting to the DB: " + err);
  }
}

function findEverybody(db, closeConnection)
{
  var cast = db.collection("cast");
  cast.find({}, {name: 1, age: 1, _id: 0}).toArray(onResultsFound);
}

function onResultsFound(err, docs)
{
  if(!err)
  {
    console.log("Found the following docs:");
    console.dir(docs);
  }
  else
  {
    console.log("Problems retrieving documents: " + err);
  }

  closeConnection(null, database);
}

function closeConnection(err, db)
{
  console.log("Closing connection");
  database.close();
}

Run the program with node retrieve.js and the output is as expected:

Connected to MongoDB!
Found the following records
[ { name: 'Ted Allen', age: 52 },
  { name: 'Carson Kressley', age: 47 },
  { name: 'Kyan Douglas', age: 47 },
  { name: 'Thom Filicia', age: 48 },
  { name: 'Jai Rodriguez', age: 38 } ]
Closing connection

 

Update

To demonstrate how to update a record, we will update Ted's age. Create a file called update.js and add the following to it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
/**
 * update.js
 *
 * By: Mike Klepper
 * Date: 3 September 2017
 *
 * This program update a record in the cast collection
 *
 * See blog post on patriot-geek.blogspot.com
 * for instructions.
 */

var MongoClient = require("mongodb").MongoClient;

const DATABASE_NAME = "qeftsg";
const URL = "mongodb://localhost:27017/" + DATABASE_NAME;

var database = null;
MongoClient.connect(URL, onConnected);

function onConnected(err, db)
{
  if(!err)
  {
    console.log("Connected to MongoDB!");
    database = db;
    updateTed(db, closeConnection);
  }
  else
  {
    console.log("Problems connecting to the DB: " + err);
  }
}

function updateTed(db, callback)
{
  var cast = db.collection("cast");
  cast.updateOne({name: "Ted Allen"}, {$set: {age: 53}}, onUpdate);
}

function onUpdate(err, result)
{
  if(!err)
  {
    console.log("Updated Ted");
    console.log(JSON.stringify(result, null, 5));
  }
  else
  {
    console.log("Problems updating the DB: " + err);
  }

  closeConnection(null, database);
}

function closeConnection(err, db)
{
  console.log("Closing connection");
  database.close();
}

Run the program with node update.js and the output is:

Connected to MongoDB!
Updated Ted
{
     "n": 1,
     "nModified": 1,
     "ok": 1
}
Closing connection

Then rerun retrieve.js and we see that Ted's age has indeed been set to 53:

Connected to MongoDB!
Found the following records
[ { name: 'Ted Allen', age: 53 },
  { name: 'Carson Kressley', age: 47 },
  { name: 'Kyan Douglas', age: 47 },
  { name: 'Thom Filicia', age: 48 },
  { name: 'Jai Rodriguez', age: 38 } ]
Closing connection

 

Delete

Finally, let's delete Jai from the database. Create delete.js and add the following lines:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
/**
 * delete.js
 *
 * By: Mike Klepper
 * Date: 3 September 2017
 *
 * This program deletes a record from the cast collection
 *
 * See blog post on patriot-geek.blogspot.com
 * for instructions.
 */

var MongoClient = require("mongodb").MongoClient;

const DATABASE_NAME = "qeftsg";
const URL = "mongodb://localhost:27017/" + DATABASE_NAME;

var database = null;
MongoClient.connect(URL, onConnected);

function onConnected(err, db)
{
  if(!err)
  {
    console.log("Connected to MongoDB!");
    database = db;
    deleteJai(db, closeConnection);
  }
  else
  {
    console.log("Problems connecting to the DB: " + err);
  }
}

function deleteJai(db, callback)
{
  var cast = db.collection("cast");
  cast.deleteOne({name: "Jai Rodriguez"}, onDeleted);
}

function onDeleted(err, result)
{
  if(!err)
  {
    console.log("Deleted Jai");
    console.log(JSON.stringify(result, null, 5));
  }
  else
  {
    console.log("Problems deleting document: " + err);
  }

  closeConnection(err, database);
}

function closeConnection(err, db)
{
  console.log("Closing connection");
  database.close();
}

Run it, and the output should be:

Connected to MongoDB!
Deleted Jai
{
     "n": 1,
     "ok": 1
}
Closing connection

 

Conclusion

That's the basics of using Native Mongo Driver. There is much more to the Native Mongo Client, including:

  • Batch operations
  • Cursor management
  • Aggregation pipeline
  • Application performance management

A great source for information and tutorials is the official documentation.

Posted by at No comments:
Labels: , , ,
Subscribe to: Posts (Atom)