Serverless plugins – Automatic generation of monitoring dashboards

No Comments

The Serverless framework defines a meta language on top of the Infrastructure as Code (IaC) services of many Cloud Service Providers (CSP). It simplifies the developer experience when designing scalable serverless APIs and boosts productivity and maintainability by abstracting away and automating the generation of infrastructure components.

Amazon Web Services (AWS) is one of the major cloud provider platforms that Serverless supports. With AWS as target, Serverless applications are translated into one or more CloudFormation stacks. CloudFormation is the AWS service providing a common language for describing and provisioning infrastructure components. Speaking in terms of CloudFormation, each component is a resource, where resources are grouped into deployable units, called stacks. CloudFormation deploys, updates, or roles back (in case of failure) a stack as a whole.

Consider the following example. It is probably the simplest application in Serverless with a single function. The function is triggered whenever an HTTP GET request is received on its associated endpoint “/hello”, as a result of which it invokes the corresponding “SayHelloHandler”.

As compact as it is, the above snippet completely describes an application that can be deployed on AWS cloud. When converted to CloudFormation, Serverless automatically generates a stack with more than 10 resources, spanning over a JSON document of around 340 lines! The resulting stack contains many default settings, which you would have to configure explicitly to get a production-ready application. Nevertheless, there is still a huge advantage in having Serverless automatically generate all the required resources (CloudWatch LogGroups, IAM Roles, S3 Buckets, Lambda functions, etc.), using only a minimum set of configurations.

This is already good, very good actually. But there are still scenarios, where a lot of repetitive configurations are needed. Imagine you have successfully developed a proof of concept for your next big thing and are preparing to do the first productive launch. One of the many non-functional aspects you would then need to consider is how to monitor functions in production. AWS CloudWatch service automatically gathers a wide range of function metrics, namely the number of invocations, duration of each invocation, number of errors, and number of throttles. All you need to do is define a CloudWatch dashboard and include the metrics you are interested in. For each metric in the dashboard you get to choose from a wide range of statistics such as percentiles (p99, p95, …), Min, Max, Sum, and Average.

Serverless does unfortunately not support generation of these dashboards out of the box. A poor man’s approach would be to manually add a dashboard resource and start adding widgets for all metrics and their corresponding statistics per function. Here is an example of such a resource with one widget containing a single statistic (p99) of a single metric (duration) for our previously defined lambda function.

Things start to get really messy when you add more and more functions, metrics, and/or statistics. It is not only hard to get all these configurations right, they are also difficult to maintain. Each time you want to add/remove a function or include/exclude a metric, you will have to modify the dashboards, too.

With Serverless’s main promise in mind, namely abstraction and automatic resource generation, the most natural question would be, how Serverless can be used to do the heavy lifting for us in this and similar scenarios.

Plugins to the rescue

The real power of the Serverless framework shines when it comes to its well-designed modular architecture. It allows the framework’s core functionality to be extended nearly indefinitely by docking-in new plugins.

Serverless defines its features in terms of commands, with package and deploy being probably the two most widely used commands. Each command emits a series of lifecycle events during its execution to expose its state to the outside world. Plugins can be used to extend Serverless in two ways: either by defining a new command and its corresponding lifecycle events, or by hooking into lifecycle events of the existing commands.

Examining the architecture of the Serverless framework is a topic on its own and is beyond the scope of this blog post. Interested readers can refer to the Serverless documentation. In this article we will focus on how to use plugins to automate the generation of CloudWatch dashboards.

CloudWatch dashboards plugin

To get a feeling of how plugins work, let’s take a look at the steps required for defining a new dashboard generator plugin. For the sake of brevity, we focus here on the absolutely necessary features for a simplified version of the plugin. An open source version of the plugin supporting more features is actively maintained on GitHub under serverless-plugin-cloudwatch-dashboard project.

Configuration

The plugin should abstract away all the details of CloudWatch dashboards behind a minimal, user-friendly set of configurations. This allows users to simply configure the desired metrics and their corresponding statistics and let the plugin automate the rest. Following is an example of how such a configuration can look like. It selects p99 and p95 percentiles for duration, invocations, errors, and throttles.

Plugin class

Each plugin is a JavaScript class. Serverless will call the constructor during execution and will inject a serverless object together with some additional options into it. We start by defining a new class for our plugin.

The constructor is expected to define how the plugin extends the framework. Remember from the last section that Serverless can be extended either by adding new commands or by hooking external actions into lifecycle events of existing ones. We do not need a new command for our plugin. We can simply hook the addDashboards function into the point in time (i.e before:package:finalize), when all function definitions are fully translated to a valid, ready-to-deploy CloudFormation template (lines 6 – 7).

Implementation

We are half-way done. All that is left is the implementation 😉

The serverless object that is injected into the constructor has a service property. It contains the user-specified configurations for the plugin (metrics and stats as defined before), and an array of all function definitions (lines 2 – 4). We pass this information to the yet to be defined createDashboards function (line 6). The function is expected to return a list of new CloudWatch dashboard resources, which are then converted into a list of key/value objects (lines 8 – 11) and added to the existing list of all resources (lines 13 – 15).

A CloudWatch dashboard is fairly easy to define; it has a name and a body. The real complication, though, lies in the definition of the body. It is expected to be a JSON formatted string containing a list of widgets, where each widget includes a list of metrics. The structure of a dashboard body is described elaborately in Dashboard body structure and syntax.

To implement createDashboards, we need yet another function for creating widgets.

Given a list of metrics, the above function creates a single widget for each metric. Each widget includes all statistics of its corresponding metric for all functions. The result is a key-value map with metric name as key and the corresponding widget as value.

createDashboards can now easily be implemented by iterating over the list of all widgets and creating a new dashboard for each of them:

Result

Here’s our plugin in action. The screenshot is taken from the durations dashboard and includes p90 and p50 percentiles.
CloudWatch dashboard generated by serverless plugin

Summary

Serverless framework simplifies the developer experience and improves productivity when designing scalable serverless APIs. It drastically reduces the amount of code and the maintenance cost by automating the generation of Infrastructure as Code templates. Serverless’s well-designed modular architecture allows to extend the framework’s core functionality almost indefinitely by docking in new plugins. In this blog post we outlined the structure of a Serverless plugin. We then showed the steps to implement a plugin for generating monitoring dashboards, an open-source implementation of which is actively maintained at the plugin’s GitHub repository.

Were the instructions clear? Did they work for you? I would be very happy to learn about your use cases and get your feedback.

References

[1] Amazon API Reference, Dashboard body structure and syntax
[2] Serverless Framework Documentation, Plugins
[3] Serverless Blog, How To Write Your First Plugin For The Serverless Framework
[4] GitHub, lifecycle-cheat-sheet
[5] GitHub, codecentric/serverless-plugin-cloudwatch-dashboard
[6] GitHub, serverless/plugins
[7] codecentric Blog, serverless

Avatar

Mahdi Ebrahimi is an IT consultant at codecentric AG in Karlsruhe. His main focus is designing and implementing cloud-native, resilient, and scalable distributed systems.

Comment

Your email address will not be published. Required fields are marked *