Blog /

Request logging with CloudWatch

Joris Verbogt

Oct 20 2023

Posted in Engineering & Technology

Centralizing your API logs

CloudWatch is an AWS service that allows you to keep track of logs of all other AWS services, keep an eye on performance metrics and act when certain thresholds are passed.

One of the great features of CloudWatch is that you can also ingest your own metrics and treat them the same as you would with regular metrics of your AWS services, as explained in an earlier blog post

Another useful functionality of CloudWatch is to use it as a centralized, combined log for your ephemeral, auto-scaling instances.

Example

We are going to use the excellent Winston logging utility, which has support for AWS CloudWatch as a logging target:

import winston from 'winston'
import WinstonCloudwatch from 'winston-cloudwatch'

Let's first log our warnings to the local logfile:

const logger = winston.createLogger({
    transports: [
        new winston.transports.Console({
            level: 'warn',
            handleExceptions: true,
            format: winston.format.combine(
              winston.format.splat(),
              winston.format.metadata(),
              winston.format.timestamp({ format: Date }),
              winston.format.colorize(),
              winston.format.printf(info => `${info.timestamp} - ${info.level}: ${info.message} ${util.format(info.metadata)}`)
            )
        })
    ]
})

This will generate a nicely colorized logfile:

Then, for all info level (and higher) messages, let's log to CloudWatch:

const logger = winston.createLogger({
    transports: [
        new winston.transports.Console({
            level: 'warn',
            handleExceptions: true,
            format: winston.format.combine(
              winston.format.splat(),
              winston.format.metadata(),
              winston.format.timestamp({ format: Date }),
              winston.format.colorize(),
              winston.format.printf(info => `${info.timestamp} - ${info.level}: ${info.message} ${util.format(info.metadata)}`)
            )
        }), new WinstonCloudwatch({
            awsRegion: awsRegion,
            logGroupName: 'MY_LOG_GROUP',
            logStreamName: 'MY_SERVICE_LOG',
            level: 'info',
            jsonMessage: true,
            handleExceptions: true,
            format: winston.format.combine(
              winston.format.splat()
            )
        })
    ]
})

As you can see, we are creating one log stream per service, so we can see either a combined view or split out per service within the log group.

Now, let's make sure our requests are logged, for example when using Express and Morgan:

const loggerStream = {
  write: (message) => {
      logger.info(message);
  }
}
app.use(morgan(':remote-addr - - [:date] ":method :url HTTP/:http-version" :status :res[content-length] :response-time ":user-agent" :remote-user', { stream: loggerStream }));

Then finally, in CloudWatch we can see these logs and perform queries:

Conclusion

Although there are many managed solutions out there for logging in NodeJS, this is not always an option. Security policies and supplier selection in your organization may prevent you from sending data to these 3rd parties. When you are running on AWS, CloudWatch is already keeping track of your instances, logs and alarms. If you consider this a safe environment for your data, an obvious choice for centralized logging is through CloudWatch.

As always, we hope you liked this article, and if you have anything to add, we are available via our Support Channel.