HOW-TO: SMS via Amazon Web Services

(To be followed in order, as indicated below...)

REST API

Note This section is really long. It starts slow but then speeds up. Bear with it, you'll be better off.

Okay, now the rubber really hits the road. It's time to set up a REST API that will provide all of the previously described operations in one neat set of (hopefully intuitive) URLs. This is going to require quite a bit of code and it's possible you won't fully understand all of it, but I'll do what I can to explain along the way so it doesn't get too confusing. If you know a bit of JavaScript and you understand JSON, this shouldn't be too bad.

If you are logged into AWS via a web browser, don't bother logging out. We're going to pop back in to set up some access credentials soon, so just minimize the browser for now.

Use the login instructions that you received in the AWS EC2 portion of the HOW-TO to log into your EC2 instance. Open a new file called app.js using your text editor. This is where the Node.js and Express code will reside for the REST API. This file should be saved in the same directory you were in when you installed Node.js & Express earlier.

Note Copying/pasting text will work, but this is a HOW-TO and you're supposed to be learning. You'll be much better off in the long run typing out the code by hand.

Start by adding a set of variables that will make the calling of libraries and modules within app.js easier. Some of these libraries and modules are included with Node.js, others were installed back in the Node.js & Express section of the HOW-TO:

    var http        = require('http');        // for making a web server
    var express     = require('express');     // for making routes
    var bodyParser  = require('body-parser'); // for POST body content
    var app         = express();              // builds the app
    

The comments explain what the module or library is for. The app variable will be tapped on the most because it will point to the web server that serves the REST API. The others will pepper the code as we progress through app.js coding.

We're setting up an API, so it's a good idea to give it a version number. This variable will become part of the REST API's URLs:

    var apiVer      = 'v1';                   // API version string
    

We won't get very far without the AWS SDK. It's the glue that connects us to the services offered through AWS, including SNS. The code below sets up a simple variable that will be used to configure the service. For now, the code should be added but commented out. We'll uncomment it when it's time to connect to SNS.

    /*
    var AWS         = require('aws-sdk');     // AWS SDK
    
    // Set up new AWS SNS connection. 
    // Note: only US East region offers SMS at this time!
    AWS.config.update({
        'region': 'us-east-1',                // the AWS region for SNS
        'accessKeyId': 'accessKey',           // your access key
        'secretAccessKey': 'secretAccessKey'  // your secret access key
    });
    var sns = new AWS.SNS(); 
    */
    

Take a moment to examine this block of code. Recall again that SNS can only send SMS messages from the USA and only from one region: "US East (N. Virginia)." "us-east-1" is a shorthand name for the "US East (N. Virginia)" region. The other two items within AWS.config.update() are security keys. Unless you already have a pair from a previous EC2 instance, you're going to need to set those up. We'll get to that. For now, placeholders are used.

The next blocks of code set up support for JSON encoded content in the body portion of REST API requests as well as the web server that will be used to serve our REST API. No, you are not running this Node.js application through a web server like Apache. Instead, it is providing the web server itself. The following code sets up a simple web server at port 3000 in your EC2 instance:

    // Support JSON encoded bodies
    app.use(bodyParser.json());
    app.use(bodyParser.urlencoded({ 
        extended: true 
    }));
    
    // Start a web server at port 3000
    var server = app.listen(3000, function() {
        var host = server.address().address;
        var port = server.address().port;
        console.log('Listening at http://%s:%s', host, port);
    });
    

Now, on to the "routes." The route is the portion of the URL for the REST API that follows the server's domain name and it tells the Node.js application what code it needs to run. Ultimately, a number of routes will be coded to handle the various create/read/update/delete operations in our REST API. A catch-all route will be offered at the end to handle all unrecognized routes.

For now, set up that catch-all route and also a basic route that will accept a request with a GET verb and send a simple string in return.

    // Basic GET route for testing. Returns a JSON response indicating
    // message was received.
    app.get('/', function(req, res) {
        console.log("Got a GET request!");
        res.json({
            success: true,
            message: 'Got it...'
        });
    });

    // Catch-all route. Set HTTP status to 404 and send a JSON response
    // indicating the route is unknown.
    app.get('*', function(req, res) {
        console.log('GET request for unknown route');
        res.status = 404;
        res.json({
            success: false,
            message: 'Unknown command'
        });
    });
    

Note the two arguments to the callback functions in app.get(): "req" and "res." The "req" object contains the items that make up the request, and the "res" object handles our response to the request. In both routes above, "res" sends a JSON response via res.json(), and in the catch-all route "res" also sets the HTTP status to the proper error code. These objects are passed to the callback function in every route.

The callback, by the way, is where you put the code that needs to be run when a route receives a request. This may not seem logical at first if you're a JavaScript developer ("...don't you use callbacks after your code runs?..."), but if you consider that your code is the route's callback, perhaps it'll make more sense.

Also, note the routes themselves, which are given as the first arguments to the app.get() functions. In the first route, "/", points to the base of your API domain's path. So, if your domain is "ec2instance.com", the "/" route is "ec2instance.com/" (with the trailing forward-slash). Any other paths sent in the request URL will fall through to the catch-all, "*", and will get the error response.

That should be all we need to get a basic, testable REST API going. If you have everything typed as it's listed above, save the file and exit the editor. If that was all too much to type, feel free to cheat by copying a pre-typed version. Either way, you need to have this file in your home directory in your EC2 instance before running the following command:

    node app.js
    

If all goes well (as in, you have all of the modules and libraries installed as indicated in the AWS EC2 portion of the HOW-TO and your app.js file contains no errors), you should see the following under the command you just typed:

    Listening at http://:::3000
    

That message indicates you have a Node.js web server operating on your EC2 instance at port 3000. In other words, success! Nice work. Now, it's time for a test. Using whatever testing aparatus you have chosen for this project (again, two nice options are the Postman extension to Chrome or the curl command line utility), hit the following URL with a GET request:

    localhost:3000/
    

Two things should happen. First, back in the terminal where you started your Node.js server, you should see the following message:

    Got a GET request!
    

Then, in whatever application you used to send the GET request to the Node.js server, you should see a JSON response:

    {
      "success": true,
      "message": "Got it..."
    }
    

If you look at the route code in your app.js file, it should be easy to determine where these lines come from. Now, test out the catch-all route by sending a GET request to the following URL:

    localhost:3000/nope
    

There is no route established for "/nope" so this should result in an error. Like with the previous GET request, two things should happen. First, back in the terminal where you started your Node.js server, you should see the following message:

    GET request for unknown route
    

Then, in whatever application you used to send the GET request to the Node.js server, you should see a JSON response:

    {
      "success": false,
      "message": "Unknown command"
    }
    

Huzzah, you made it! One basic REST API that does...well, pretty much nothing useful. However, at this point (assuming you really did succeed in getting it going) you should have a solid, general idea of how the app.js file is structured and what a route looks like in code. That's actually quite a lot of information.

Moving forward, the HOW-TO is going to do a lot less hand-holding. I'll explain new concepts where they are applied, but code walk-throughs will be minimal. We need to speed this up, as I'm sure you're getting impatient by now.

On to the "real" REST API coding...

Authentication

Before connecting to SNS, we need to uncomment the code in our app.js file that sets up our access credentials and replace the placeholders with actual values. Remove the "/*" and "*/" lines around the following code in app.js to uncomment it:

Note Your AWS access credentials are not the same as your AWS login credentials. The access credentials operate at an API level and have nothing to do with logging in as a user.

    var AWS         = require('aws-sdk');     // AWS SDK
    
    // Set up new AWS SNS connection. 
    // Note: only US East region offers SMS at this time!
    AWS.config.update({
        'region': 'us-east-1',                // the AWS region for SNS
        'accessKeyId': 'accessKey',           // your access key
        'secretAccessKey': 'secretAccessKey'  // your secret access key
    });
    var sns = new AWS.SNS(); 
    

If you previously set up an AWS account, you may already have your access credentials saved away somewhere. If so, don't worry about creating new credentials, just use the ones you already have. It will be a pair of key strings: one is an access key ID and the other is a secret access key. You can grab the access key ID from your AWS account at any time, but you only get the secret key when you create the access key ID. So, if you lose the secret key, you need to start from scratch with a new key pair.

If you need a new key pair (you can only have two, so if you already have two you'll need to delete one pair before making a new pair), log into your AWS account if you are not logged in already, then pull down your username menu in the top menu bar and select "Security Credentials." If asked about continuing to security credentials or going into an "IAM Users" page, just continue to your security credentials. You should end up on a page that looks like this:

Click on "Access Keys" to pull down a listing of existing access key IDs.

Again, if you have two access key IDs already, you'll need to delete one of them before creating a new one. Choose wisely. When you're ready to create a new key pair, click the blue "Create New Access Key" button. A key pair will be created and a window will pop up offering you the opportunity to view and download they key pair. At a minimum, view and copy/paste the keys some place before moving forward. Preferably, download the .CSV file with your key pair when it's offered. As soon as you leave this window, your secret access key will no longer be obtainable. I therefore politely but firmly suggest that you click on the "Download Key File" button and copy the .CSV file to a trusted location.

Okay, so now you have your access key ID and secret access key. Take those two strings and put them in their proper locations in the app.js file, within the SNS code block.

    var AWS         = require('aws-sdk');     // AWS SDK
    
    // Set up new AWS SNS connection. 
    // Note: only US East region offers SMS at this time!
    AWS.config.update({
        'region': 'us-east-1',                // the AWS region for SNS
        'accessKeyId': 'accessKey',           // your access key
        'secretAccessKey': 'secretAccessKey'  // your secret access key
    });
    var sns = new AWS.SNS(); 
    

You should now be set up to connect to SNS without errors. Now it's time to set up the routes.

A Few Words about Routes

There are three sets of routes that we'll be creating: Topic routes, Subscription routes, and Message routes. In each case, we're going to do something slightly tricky but not difficult to grasp if you are familiar with JavaScript: we're going to chain the code for the HTTP verbs under each route. What this means is our route code will be structured slightly different than what was shown above for the two GET routes:

    app.route('/sns/' + apiVer + '/api-identifier')
        .get(function(req, res) {

            // GET code goes here

        })
        .post(function(req, res) {

            // POST code goes here

        })
        .put(function(req, res) {

            // PUT code goes here

        })
        .delete(function(req, res) {

            // DELETE code goes here

        });
    

In the code sample above, "api-identifier" is going to be some version of "topic", "subscription", or "message". This means our topic routes will be set up to use a variation of the following post-domain path (remember, "apiVer" is a variable that we set to "v1" earlier in app.js):

    /sns/v1/topic
    

Note that not all HTTP verbs will be used for all routes.

Okay, onward.

Topic Routes

There are two things that our REST API needs to GET as far as topics are concerned: information about individual topics, and full topic listings. By default, if a GET request goes to a topic route, the full topic listing should be returned. However, if a topic ARN is sent (remember those? If not, review the AWS SNS section of the HOW-TO) along with the GET request, that means we want information about an individual topic.

Sending extra information with GET requests outside of the actual URL path (say, in a query string) is frowned upon in REST APIs, so a single GET route path needs to cover both situations described above. How is that possible?

Express, which is what we're using to build our routes, has us covered. If we add a string like ":something?" to the end of our route, we can test for an optional additional element in our route path; if it's found, the value is assigned to the variable "something" because that's what followed ":" in the string. The question mark at the end makes this additional element optional (in other words if this extra bit was ":something" with no question mark at the end, it would be required), so if it is not included in a GET request, we will not receive an error.

In our case, the extra optional element is a topic ARN. Load app.js into your editor, delete the "/" GET route that was included for testing (leave the catch-all route alone), then in its place add the following code:

    // SMS "topic" via SNS
    app.route('/sns/' + apiVer + '/topic/:topicArn?')
        .get(function(req, res) {
            console.log('GET /sns/' + apiVer + '/topic');

            if(req.params.topicArn) {

                // we got a topicArn string in the GET request

            } else {

                // we didn't get a topicArn string

            }
        })
        .post(function(req, res) {
            console.log('POST /sns/' + apiVer + '/topic');

            // other POST code goes here
    
        })
        .put(function(req, res) {
            console.log('PUT /sns/' + apiVer + '/topic');
    
            // other PUT code goes here

        })
        .delete(function(req, res) {
            console.log('DELETE /sns/' + apiVer + '/topic');
    
            // other DELETE code goes here

        });
    

This is the complete chain of HTTP verbed requests that we will be covering for topics in our REST API. Note the if/else statement in the GET code. It tests for a value at "topicArn" in the GET request URL by checking if "req.params.topicArn" has a value, and if it does, it's assumed that the value is a valid topic ARN which can (and will) be looked up and its information returned by the API. If there is no value in "req.params.topicArn," a full list of topics will be returned instead.

Now, let's write the actual code that will be run for GET requests. All access to our SNS items will be handled through the "sns" object that we created earlier in app.js with the following line of code:

    var sns = new AWS.SNS(); 
    

Within this object, a wealth of SNS functionality is available via the AWS SDK. For each function that we'll be calling via the "sns" object, we'll set up a basic JSON structure with the information required by the function, submit it to the function, then return the JSON response to the user. Every call to an AWS SDK will follow that process. The function calls and the response handling for the calls are very consistent, only differing in function name and the JSON structure passed to the function. You should be able to recognize the pattern after adding a few verbs to the route chains.

Note Examples of the JSON structures that are submitted by the user to each REST API route will be given in the walk-through section of the HOW-TO. The JSON structures described below are only embedded within function argument lists and are not the same as the structures users will be sending to REST API routes. User-supplied JSON will contain values that are read into the JSON structures described below.

To grab information about a single topic, the topic ARN is passed to the getTopicAttributes() function via a "TopicArn" attribute (case-sensitive, of course). Adjust the first if/else block in the GET route to call this function with our topic ARN from the GET request URL:

    .get(function(req, res) {
        console.log('GET /sns/' + apiVer + '/topic');

        if(req.params.topicArn) {
            // Return attributes about the topic in topicArn
            sns.getTopicAttributes({
               TopicArn: req.params.topicArn     // required
            }, function(err, result) {
                if (err != null) {
                    console.log(err, err.stack); // an error occurred
                } else {
                    console.log(result);         // successful response
                    res.send(result);            // successful response
                }
            });
        } else {

                // we didn't get a topicArn string

        }
    })
    

The req.params.topicArn variable will be replaced with its value from the URL before being sent off to SNS via the AWS SDK. The callback for getTopicAttributes() checks for an error and, if encountered, reports it; if not, the result from the function is output to the console for server debugging and is also sent to the user.

If no "topicArn" string is encountered in the GET request URL, we need to run the listTopics() function to get the full topic listing. Add code to the second part of the if/else block to do this:

    .get(function(req, res) {
        console.log('GET /sns/' + apiVer + '/topic');

        if(req.params.topicArn) {
            // Return attributes about the topic in topicArn
            sns.getTopicAttributes({
               TopicArn: req.params.topicArn     // required
            }, function(err, result) {
                if (err != null) {
                    console.log(err, err.stack); // an error occurred
                } else {
                    console.log(result);         // successful response
                    res.send(result);            // successful response
                }
            });
        } else {
            // Return a topic listing
            sns.listTopics({}, function(err, result) {
                if (err != null) {
                    console.log(err, err.stack); // an error occurred
                } else {
                    console.log(result);         // successful response
                    res.send(result);            // successful response
                }
            });
        }
    })
    

This function's argument list is empty, but there is one attribute that can be passed if the topic listing is too long:

    {
        NextToken: 'token_value'
    }
    

The listTopics() function only returns a finite number of topics (100 max by default). If there are more topics, a token is returned that points to the next topic that can be listed in a subsequent call to listTopics(). This is similar in nature to SQL's "OFFSET": it allows you to jump past a number of records before building a return set.

Note I will not cover grabbing/using this token in this HOW-TO as it is probably beyond the needs of non-enterprise users, but you are welcome to investigate it on your own.

The POST code is significantly simpler than the GET request code. No if/else blocks, just the straightforward creation of a new topic name using createTopic():

    .post(function(req, res) {
        console.log('POST /sns/' + apiVer + '/topic');
    
        // Send an application/json req: { "Name": "topic_name" }
        sns.createTopic({
            'Name': req.body.Name
        }, function (err, result) {
            if (err !== null) {
                console.log(err, err.stack);
            } else { 
                console.log(result);
                res.send(result);
            }
        });
    })
    

Note the comment near the beginning of the code. The POST body content must include a JSON structure that sets the topic name in a "Name" attribute. How you build this JSON structure is your decision, but however it's done, it needs to be sent in the POST body as raw "application/json" content. Your testing application (Postman, curl, etc.) should cover how this is done. The value will be extracted by reading "req.body.Name," as indicated above.

Note As indicated earlier, req.* contains items in the request regardless of the HTTP verb that is used. req.body.* contains items received in the POST/PUT/DELETE body. req.params.* contains variable items in the GET request URL.

Recall from the AWS SNS section of the HOW-TO that each topic requires a display name. Note that no display name is sent with the POST code described above. In order to set a display name, we need to be able to revise the topic's "DisplayName" attribute.

It makes little sense to create a route that only changes one attribute when a generic attribute-changing route will allow us to change any topic attribute, so that's what we'll build now. In REST APIs, the verb used to alter an existing record is PUT. The SDK function that changes topic attributes is setTopicAttributes() and it takes a JSON structure with three bits of information: the attribute name, the new attribute value, and the topic ARN. Of these three bits of information, the attribute name and topic ARN are required.

    .put(function(req, res) {
        console.log('PUT /sns/' + apiVer + '/topic');
    
        // Change values of topic attributes
        sns.setTopicAttributes({
            AttributeName: req.body.attrName, // required
            AttributeValue: req.body.attrVal,
            TopicArn: req.body.topicArn       // required
        }, function (err, result) {
            if (err !== null) {
                console.log(err, err.stack);
            } else { 
                console.log(result);
                res.send(result);
            }
        });
    })
    

To round out our topic routes, we need to offer a method by which topics can be deleted. The HTTP verb associated with deletions is, understandbly, DELETE. The function that deletes topics is deleteTopic() and it takes a JSON structure with one attribute: the topic ARN.

    .delete(function(req, res) {
        console.log('POST /sns/' + apiVer + '/topic');
    
        // Delete a topic by its ARN
        sns.deleteTopic({
            TopicArn: req.body.topicArn // required
        }, function (err, result) {
            if (err !== null) {
                console.log(err, err.stack);
            } else { 
                console.log(result);
                res.send(result);
            }
        });
    });
    

Hopefully by now you're seeing the pattern with each section of the topic route code. Very little changes other than function name and the JSON structure passed in the argument list of the function. Worth noting is that every function in a non-GET request will return a JSON response containing "ResponseMetadata" describing the transaction and any additional data such as new topic ARN (for createTopic()). These serve as confirmation that the function succeeded. Example:

    {
        ResponseMetadata: {
            RequestId: 'bdcaeca4-bad1-614f-b186-a54eac51fb9e'
        },
        TopicArn: 'arn:aws:sns:us-east-1:[AWS ID string]:testing'
    }
    

Note A walk-through that creates a topic, adjusts its display name, subscribes a phone, sends a message, and deletes the topic, will be offered in the walk-through section of the HOW-TO.

Subscription Routes

I'm going to start powering through code descriptions because enough has been explained already to understand what the code is doing for each route. Exhaustive explanations should not be necessary anymore.

The basic code skeleton for subscriptions routes is similar to that of the topic routes, except this time we're only going to set up the GET and POST verbs. There's very little reason to revise subscriptions from the API side of things, and deletion is handled easier by the user that subscribed to the topic (instructions for unsubscribing are included in the subscription confirmation message that is sent to the user).

    // SMS "subscriptions" via SNS
    app.route('/sns/' + apiVer + '/sub/:topicArn?')
        .get(function(req, res) {
            console.log('GET /sns/' + apiVer + '/sub/' + req.params.topicArn);

            if(req.params.topicArn) {

                // we got a topicArn string in the GET request

            } else {

                // we didn't get a topicArn string

            }
        })
        .post(function(req, res) {
            console.log('POST /sns/' + apiVer + '/sub');
    
            // other POST code goes here

        });
    

The GET route for subscriptions is going to have a twist to it that is similar to the GET route for topics: different results will be sent depending on whether or not a topic ARN is added to the GET request URL. If included, a list of subscriptions for that topic will be returned; if left off, a list of all subscriptions for your topics will be returned.

The listSubscriptionsByTopic() function is used if a topic ARN is passed. Similar to listTopics(), a "NextToken" token can be used to start a subsequent subscription listing if there are too many subscriptions to return with one function call. That's optional. A topic ARN, however, is not.

The listSubscriptions() function is similar to the listSubscriptionsByTopic() function but it does not accept a topic ARN. It does, however, support the "NextToken" token.

Revise the subscription GET route code to utilize the fore-mentioned functions:

    .get(function(req, res) {
        console.log('GET /sns/' + apiVer + '/sub/' + req.params.topicArn);

        if(req.params.topicArn) {
            sns.listSubscriptionsByTopic({
               TopicArn: req.params.topicArn
            }, function(err, result) {
                if (err != null) {
                    console.log(err, err.stack); // an error occurred
                } else {
                    console.log(result);         // successful response
                    res.send(result);            // successful response
                }
            });
        } else {
            sns.listSubscriptions({}, function(err, result) {
                if (err != null) {
                    console.log(err, err.stack); // an error occurred
                } else {
                    console.log(result);         // successful response
                    res.send(result);            // successful response
                }
            });
        }
    })
    

The POST route calls the subscribe() function with a JSON structure containing three items: a topic ARN, a protocol, and an "endpoint." The protocols offered by SNS are all listed in the AWS SNS section of the HOW-TO, and topic ARNs should be very familiar by now. As for endpoints, they're different depending on what protocol you are using in the subscription. For the "sms" protocol, the endpoint is a US phone number starting with 1 (example: for 503-555-1212, the endpoint is "15035551212"). For the "email" protocol, it's a valid email address. We're only concerned with SMS in this HOW-TO so be sure to put a valid phone number for an SMS-capable phone in the JSON data.

    .post(function(req, res) {
        console.log('POST /sns/' + apiVer + '/sub');
    
        sns.subscribe({
            'TopicArn': req.body.topicArn, // covered elsewhere
            'Protocol': req.body.protocol, // for SMS, use 'sms'
            'Endpoint': req.body.endpoint  // for SMS, a phone number starting with 1
        }, function (err, result) {
            if (err !== null) {
                console.log(err, err.stack);
                return;
            } else {
                console.log(result);
                res.send(result);
            }
        });
    });
    

Message Routes

For messages, the list of routes is short and sweet with only one route for the POST HTTP verb. It's so short, I won't even bother with a code skeleton. Let's just get to it.

The POST route calls the publish() function with a JSON structure containing at least two main items: a message, and a topic ARN. According to the AWS SDK, you are also supposed to send a set of message attributes that describe the data type and the message value, but best I can tell these are only required if you are going to send binary data. Other attributes that affect other endpoints such as email addresses can be included, but this HOW-TO only covers SMS notifications so I will not be addressing those subjects here.

    // SMS "message" via SNS
    app.route('/sns/' + apiVer + '/msg')
        .post(function(req, res) {
            console.log('POST /sns/' + apiVer + '/msg');
    
            sns.publish({
                Message: req.body.msg,         // required 
                TopicArn: req.body.topicArn    // publish to this topic
            }, function (err, result) {
                if (err !== null) {
                    console.log(err, err.stack);
                } else { 
                    console.log(result);
                    res.send(result);
                }
            });
        });
    

That Just About Does It

If you've gotten this far, you should have a fully fleshed out app.js file with all of the routes we need for our basic SMS REST API. If this was too much to type, cheat by downloading a pre-typed file, but be sure to enter your access key ID and secret access key in the credentials for AWS before launching the API with Node.js.

Speaking of which, to launch the API, kill the current Node.js process if it is still running (CTRL-C in the terminal where it is running should do the trick) and re-start it:

    node app.js
    

If you want it to run in the background even if you're logged out, that command will change to something like this:

    nohup node app.js &
    

If you go the latter route, standard error messages will not be captured but standard output (like whatever is output by console.log() above) will be captured in a "nohup.out" file in the directory you were in when you launched Node.js.

Go ahead and launch the API using one of the commands above. It's time to test the API.