Creating a SSH Tunnel to an AWS ec2 instance using a NodeJS App — Part I : Using AWS S3 Buckets in NodeJS

Set up a NodeJS background to interact with AWS S3 buckets.

What’s up folks, this is my first time with AWS. I’ve been working with AWS for a few weeks now, thought of sharing some cool stuff I learned. Excited!

In a nutshell, this series will focus on creating an SSH tunnel into a private EC2 instance via another. Instead of accessing the private instance directly through a public server. What we will do is map a local port to the remote port, that the server we want to connect to will be listening for the connection.

We will host the configuration data for such SSH tunnel connections in an AWS S3 bucket, and our application shall write/read data to/from it at execution.

So in this very first episode, we will set up our NodeJS background to interact with AWS S3 buckets.


Introduction to AWS S3 Buckets

Amazon AWS offers a variety of web services including AWS S3 Buckets. It basically gives you the ability to upload or download files programmatically with an API.

You might wanna know the following concepts to follow & thoroughly understand this write up.

  • S3 Bucket: is a storage unit from the S3 service from Amazon. You can store different type of files (data) and have metadata that describes the actual data or file.
  • Access Key ID: Is the identification for a certain access key that allows an application or user to access a set of preconfigured AWS Resource (like an S3 Bucket for example).
  • Secret Access key: Is the secret part of the Access Key ID. Think of it as a password for a specific Access Key ID (they must be used always in pairs)

Alright, before we start coding we have to configure the Amazon Web Console.

Step 1: Get Key Pairs

You can start out by getting a key pair to your default account for AWS console.

In the top bar, click your user account.

ssh tunnel 1

Then, click on “My security credentials” &“Access keys”, after that, click “Create New Access Key”.

ssh tunnel 2

A window will pop up and tell you that the access keys have been created. To see both the access keys click on “Show Access Key”.

ssh tunnel 3

Go ahead and save it for later use.

Step 2: Bucket Creation

In “Services/Storage/S3”, click on “Create Bucket”.

ssh tunnel 4

The Bucket name must be unique, so it's time to bring out your creativity ;)

ssh tunnel 5

Properties and permissions are set at default values.

ssh tunnel 6

Click “Next” 3 times and Voila! your bucket is ready!

We identify the bucket by its “Bucket Name” so keep that in place for later use.

Step 3: NodeJS Set Up

Let’s assume you have already initialized a project and created an “index.js” file in place.

If you haven’t , create a blank project using the command below and fill out the necessary information.

npm init

now create a blank .json file in the root directory (In this case I have named it as “config-json.json” since what I want is to retrieve my stored configuration data for servers from the S3 Bucket.)

Now, in the root folder let’s install the NPM AWS package.

npm install aws-sdk --save

Now, create an “index.js” file in the root folder and put the following content.

const tunnel = require('tunnel-ssh'); const fs = require('fs'); const configjs = require('./config-data'); var AWS = require('aws-sdk'); var s3 = new AWS.S3(); // Bucket names must be unique across all S3 users var params = {Bucket: 'boligmappa-cloud-connect-config', Key: 'config-json.json'}; s3.getObject(params, function(err, data) { if (err) console.log(err, err.stack); // an error occurred else file = fs.createWriteStream('./config-json.json'); s3.getObject(params).createReadStream().pipe(file); // successful response });

That’s it for this episode folks! You can leave any doubts or questions that you may have in the comments area below and I will complement the post.

In the next part, We’ll see how servers can be connected to form the configuration data we retrieve from the S3 Bucket (What we looked at in this episode)through the NodeJS application.

Happy coding!