Upgrading the AWS SDK for JavaScript

Summer season is always a bit slower than the rest of the year. A great opportunity to take a look into some well-deserved maintenance of your code base.

One of the things that is always a nasty chore to perform, during feature sprints, is upgrading dependencies. This is especially true if that dependency update introduces major breaking changes and even more so if that dependency is used throughout your code base.

If your software stack is using Amazon Web Services like S3, CloudWatch or SES, upgrading to version 3 of their JavaScript/NodeJS SDK would be an exemplary exercise. Although the changes are major and literally every call needs to be adapted, it is a pretty straight-forward refactor and will actually improve your code base.

Why upgrade to AWS SDK v3

Assuming you are using AWS SDK for JavaScript version 2, there are several important changes/improvements that warrant an upgrade:

• The AWS SDK allows you to talk to all AWS services, i.e., it is huge. The new SDK v3 is modular, allowing you to only import those parts that are needed by your code.
• Working with Promises was always a hassle with v2, because of the combination of callbacks and event listeners you needed to set up. The new v3 methods all return Promises, so modern syntax features like async/await work naturally.
• TypeScript support was tacked on in v2. The new v3 has first-class language support for TypeScript, because the code itself is generated from TypeScript. No more risk of mismatches between TS and JS.

Modules

All NPM modules in the AWS SDK are now grouped under the @aws-sdk scope, which means they have to be imported separately. Of course, any intelligent IDE will help out, but the docs clearly list the names of the packages you need.

For example, to import the CloudWatch module, you would use:

{
  "dependencies": {
    "@aws-sdk/client-cloudwatch": "^3.145.0"
  }
}

import { CloudWatchClient } from '@aws-sdk/client-cloudwatch'

Clients and Commands

In v2, you would import the aws-sdk package and use that to instantiate clients for the services:

import AWS from 'aws-sdk'
const client = new AWS.CloudWatch()
client.putMetricData({
  //... metric data
}, (err, result) => {
  if (err) {
    // handle err
  } else {
    // handle result
  }
})

Although there is still the old (v2) way of calling methods on the client, this would still need a full import of all methods into an aggregated client class.

import { CloudWatch } from '@aws-sdk/client-cloudwatch'
const client = new CloudWatch({ region: 'eu-west-1'})
const result = await client.putMetricData({
  //... metric data
})

The new v3 SDK allows for a finer-grained way of importing by explicitly sending commands to a base client class. It is also the safe way, as the v2 compatible calls might be removed in future versions of the AWS SDK.

import { CloudWatchClient, PutMetricDataCommand } from '@aws-sdk/client-cloudwatch'
const client = new CloudWatchClient({ region: 'eu-west-1' })
const result = await client.send(new PutMetricDataCommand({
  //... metric data
}))

As you may have noticed from the examples, all calls to send() return a Promise. This also means there are no more Event Listeners, but we'll get to that in the S3 example below.

Higher-Level Libs

In v2, there were a couple of higher-level library functions, for example the S3 Upload. These are now separated into their own modules, in this case @aws-sdk/lib-storage. So, instead of:

import AWS from 'aws-sdk'
const client = new AWS.S3({ region: 'eu-west-1' })
client.upload({
  //...
}, (err, result) => {
  //...
})

You would use:

import { Upload } from '@aws-sdk/lib-storage'
const result = await new Upload({
  //...
}).done()

Example: S3

import { S3Client, HeadObjectCommand, PutObjectCommand, GetObjectCommand } from '@aws-sdk/client-s3'
import { createReadStream, createWriteStream } from 'fs'
import { pipeline } from 'stream/promises'
const client = new S3Client({ region: 'eu-west-1'} )
try {
  const headResult = await client.send(new HeadObjectCommand({
    Bucket: myBucket,
    Key: myPath
  }))
  console.log('file exists, last modified is ' + headResult.LastModified.toUTCString())
} catch (err) {
  if (err.$response?.statusCode === 404) {
    // file not found, upload the new version
    try {
      const putResult = await client.send(new PutObjectCommand({
        Bucket: myBucket,
        Key: myPath,
        Body: createReadStream(localFilePath),
        ContentLength: myFileSize,
        ContentType: myFileType
      }))
      console.log('file successfully uploaded to S3')
    } catch (err) {
      if (err.$response?.statusCode === 403) {
        console.log('no permission to upload a file to S3')
      } else {
        // some other error
        console.log('error uploading file to S3: ' + err.message)
      }
    }
  } else {
    // throw or handle error
    console.log('error connecting to S3: ' + err.message)
  }
}

A couple of things to notice here:

• commands constructors accept the same parameters as the v2 equivalent calls
• calls return Promises
• HTTP status codes are inside the err.$response object
• commands accept (and return) streams in their Body property

Also, since there are no more event listeners, if you want a progress indicator, you will need to use your own custom Transform stream.

import { Transform } from 'stream'

function progressStream(totalBytes) {
  let loadedBytes = 0;
  let reporter = new Transform()
  reporter._transform = function(chunk, encoding, callback) {
    if (chunk) {
      loadedBytes += chunk.length;
      this.emit('sendProgress', {
        loaded: loadedBytes,
        total: totalBytes
      })
    }
    callback(null, chunk)
  }
  return reporter
}

const progressStream = progressStream(myFileSize)
progressStream.on('sendProgress', (progress) => {
  console.log('copied ' + progress.loadedBytes + ' of ' + progress.totalBytes)
})
const putResult = await client.send(new PutObjectCommand({
  Bucket: myBucket,
  Key: myPath,
  Body: createReadStream(localFilePath).pipe(progressStream),
  ContentLength: myFileSize,
  ContentType: myFileType
}))

Conclusion

Of course, this is just a quick overview of what it would take to migrate AWS SDK calls to v3. Depending on your setup, this might pose some challenges, but the TypeScript-backed generated code allows for your IDE to help you out with code completion and type hinting.

We hope this example was useful, and as always, we are available via our Support Channel for any questions you might have.