AWS CDK Part 5: How to create a step function

No Comments

In this blog post we will focus on creating the step function (state machine) that coordinates our Lambda workload. Our Lambdas will read from S3, transform data, and store this into the RDS instance we created in part 3 and part 4 of our blog series.

By the way, if you are curious about reducing VPC costs with Lambda, read more about it at: https://blog.codecentric.de/en/2019/09/reduce-vpc-costs-lambda-step-functions/.

In Part 1, 2, 3 and 4 of this blog series we described how to create a custom VPC including security groups and subnets, with S3 buckets, and an RDS instance. These first steps represent our infrastructure that is the foundation for our new architectural setup with AWS CDK using TypeScript. If you are just beginning to use AWS CDK and want to know how to get started, we recommend you start reading Part 1. This blog post is part five of our six-part blog series on AWS CDK:

From this point onward, we assume you have completed everything discussed in the previous parts, that everything is compiling and you successfully deployed the VPC, S3 bucket, RDS instance, and Lambdas to your AWS account.

cdk-vpc-architecture

The Lambdas

Following up on our ‘simplified’ part 4, we will expand our lambda-stack.ts file and add more Lambdas using our previously defined createVpcLambda function.

//lambda-stack.ts
const stepFunctionA = ApplicationStack.createVpcLambda(this, 'rdsLambdaA', 'triggers/rdsLambdaA.trigger', props, secret, s3AccessRole);
const stepFunctionB = ApplicationStack.createVpcLambda(this, 'rdsLambdaB', 'triggers/rdsLambdaB.trigger', props, secret, s3AccessRole);
const stepFunctionC = ApplicationStack.createVpcLambda(this, 'rdsLambdaC', 'triggers/rdsLambdaC.trigger', props, secret, s3AccessRole);

const taskEventA = new Task(this, 'stepFunctionA', {
    task: new InvokeFunction(stepFunctionA)
});

const taskEventB = new Task(this, 'stepFunctionB', {
    task: new InvokeFunction(stepFunctionB)
});

const taskEventC = new Task(this, 'stepFunctionC', {
    task: new InvokeFunction(stepFunctionC)
});

// step functions here

Next, we will continue by adding our step function definitions. Now we create the file ./lib/application-stack.ts containing the following code:

//lambda-stack.ts
const taskEventA = new Task(this, 'stepFunctionA', {
   task: new InvokeFunction(stepFunctionA)
});

const taskEventB = new Task(this, 'stepFunctionB', {
   task: new InvokeFunction(stepFunctionB)
});

const taskEventC = new Task(this, 'stepFunctionC', {
   task: new InvokeFunction(stepFunctionC)
});

const stepChain = new Parallel(this, 'taskParallelTasks', {}).branch(taskEventA).branch(taskEventB).branch(taskEventC);

const stateMachine = new StateMachine(this, 'StateMachine', {
   definition: stepChain,
   timeout: Duration.minutes(3)
});

As you can see, for each Lambda there is a corresponding taskEvent. Step functions can of course be very complex with decisions and recursive flow, however, for our purpose, we only want to parallelize the Lamba workload. Hence we will add a chain to our flow by calling the branch() function. The statemachine API provides a quit semantic API to communicate your flow. We additionally added a timeout which is valid for the complete workflow.

Packaging the Lambdas

Let us shift our focus back to our CDK infrastructure: we need to include the Lambdas we want to deploy. As shown highlighted in part 4 of the blog, the Lambdas are in mono-repo style. We are managing multiple Lambdas in one directory.

// Directory structure for part 5:
.
├── bin
│   └── part5.ts
├── cdk.json
├── lambdas
│   ├── package-lock.json
│   ├── package.json
│   ├── src
│   │   └── triggers
│   │       ├── rdsLambdaA.ts
│   │       ├── rdsLambdaB.ts
│   │       ├── rdsLambdaC.ts
│   │       └── vpcProcessing.ts
│   ├── tsconfig.json
│   └── yarn.lock
├── lib
│   ├── lambda-stack.ts
│   ├── rds-stack.ts
│   ├── s3-stack.ts
│   └── vpc-stack.ts
├── package-lock.json
├── package.json
├── tsconfig.json
└── yarn.lock

Before we can deploy our Lambda functions to the AWS cloud, we need to package them. In order to do so, run the following commands in your console.

cd lambdas && npm i && npm run build-ts && npm run package-cdk

The result of this is a new folder inside the Lambdas directory called ‘deployment’, which contains the production dependencies (node_modules) and the .js Lambdas needed for a successful deployment to AWS.

Last but not least, let us check if our new setup actually compiles to an updated Cloudformation template. Run the following commands:

npm run build && cdk synth

The console output should log that the stack was synthesized successfully. At this point you could deploy your new stack.

Final build & deploy

We are all set up and ready to deploy our new CDK stack to our AWS cloud. In Part 1 we have already set up our credentials, so this time we can build and deploy by simply running the following commands:

npm run build && cdk synth

After successfully having synthesized the Cloudformation template, you can comfortably check what changed by running the command:

cdk diff --profile sample

Finally, we deploy the changes made to the AWS cloud by running the command

cdk deploy --profile sample

Upon signing in to AWS Cloudformation you should see the Lambda stack being created.

A finished application

There you go! In five steps you have created an elaborate Serverless application stack that you can manage professionally using CDK. If you enter the step functions service in the AWS console, you will get a nice visualisation of our state machine. For any workload you will find a detailed flow log of inputs, outputs, possible errors, and handy links to Cloudwatch for your logs.

We will finish off this blog series with our evaluation of the use of CDK for Serverless in our next blog.

Kevin van Ingen

Kevin has a background in software engineering, economics and information science. After working as a development freelancer and teacher he returned to apply a broad multidisciplinary perspective to development projects.

Maik Kingma

Maik is a full stack developer, with a focus on backend and the Java / Spring environment. He also has experience in DevOps and is currently aspiring to gather knowledge as a software developer / software architect in the AWS Cloud.

Post by Kevin van Ingen

Architecture

AWS CDK Part 6: Lessons learned

Architecture

AWS CDK Part 4: How to create Lambdas

More content about Architecture

Comment

Your email address will not be published. Required fields are marked *