In a previous blog post I covered a basic overview of Custom Resources in AWS Cloud Development Kit (CDK) and in keeping with the same theme, I ran into a scenario which I thought would be more straightforward than it actually was: parsing a response containing a JSON object from a Lambda backed Custom Resource.
The problematic context was using a Custom Resource to fetch a secret from Parameter Store which was a JSON object. This object contained some setting and configuration details for AWS resources being deployed in a CDK Stack. I thought I could simply create a Lambda backed Custom Resource which fetched the secret and returned it, so I could use it in the Stack and pass it to the resources which required that information. It turned out that it was not so simple.
The Problem:
When you deploy your infrastructure with CDK and need to reference data stored in AWS Parameter Store, for instance, you cannot get direct access to the data in raw form at runtime. This is called “synthesis time” in CDK speak and it means that when you run a deployment, CDK synthesizes an AWS CoudFormation template (a YAML file) for the desired state of your infra with placeholder tokens for pieces of data which are not resolved until later in the physical resource creation process.
The two hurdles to get over were, as mentioned:
- The value retrieved at synthesis time during deployment was an unresolved token, not the actual secret value.
- The configuration stored in Parameter Store was not a simple string. It was in the form of a JSON object, and I needed to dig into the object to access specific properties off of it.
This meant that I could not leverage code constructs such as built-in parsing APIs for JSON or simple dot notation for property access with JavaScript, or easily work with in-memory data in an object type form as you normally would when writing application code.
The Solution:
CDK comes with AWS CloudFormation’s Intrinsic Functions, which you can use to work with and parse these unresolved placeholder tokens at synthesis time as if they were the resolved actual values at resource creation time.
In particular, the functions to use are cdk.Fn.split and cdk.Fn.select, which can be used to split the JSON string and select the value. This may not be the most elegant or robust way to handle parsing unresolved objects at synthesis time, but it worked, is practical and saved me further lost time exploring other solutions which just didn’t wind up working out (such as building a custom Lambda to use with a non pre-baked standard Custom Resource which parses and returns the object).
I also used the built-in Custom Resource AwsCustomConstruct constructs provided by CDK to fetch the parameter from Parameter Store. What’s great about these built-in Custom Resources, is that you don’t have to create a Lambda, manage dependencies, or write any custom logic to accomplish common use case tasks. The pre-built Custom Resource is automatically backed by a Lambda or functionality to accomplish a given action (in this case, ‘getParameter’ to retrieve a parameter from AWS Parameter Store). This saves you extra overhead and reduces moving parts in your IaC code.
In case it is helpful to others running into the problem, the code for the approach is below (using this example from the AWS CDK documentation as a reference):
As an example, the kind of data I was storing and retrieving from Parameter Store was something like:
{ "prop1": "prop1Value", "prop2": "prop2value" }
// cdkProject/lib/myStack.ts
import * as cr from "aws-cdk-lib/custom-resources";
const JSON_PARAM_NAME = "/myapp/secrets/jsonsecret";
// Using the built-in AwsCustomResource construct with the action "getParameter":
const getParameter = new cr.AwsCustomResource(this, "GetParameter", {
onUpdate: {
service: "SSM",
action: "getParameter",
parameters: {
Name: JSON_PARAM_NAME,
WithDecryption: true, // Needed for SecureString parameters
},
physicalResourceId: cr.PhysicalResourceId.of(Date.now().toString()), // Forces fetch to parameter store on each deploy
},
policy: cr.AwsCustomResourcePolicy.fromSdkCalls({
resources: [
`arn:aws:ssm:${cdk.Aws.REGION}:${cdk.Aws.ACCOUNT_ID}:parameter${JSON_PARAM_NAME}`,
],
}),
});
// Parameter Store values are by default available on the Parameter.Value property of the response from SSM:
const paramValue = getParameter.getResponseField("Parameter.Value");
// Here we need to use intrinsic functions to dig into the future resolved actual value of the retrieved secret (which is a placeholder token at synth time):
const prop1Split = cdk.Fn.split('"prop1"', paramValue);
const afterKeyPart = cdk.Fn.select(1, prop1Split); // Part after "prop1" key
const afterColon = cdk.Fn.split(':', afterKeyPart);
// the value part after the colon
const valuePart = cdk.Fn.select(1, afterColon);
const valueSplit = cdk.Fn.split('"', valuePart);
const prop1Value = cdk.Fn.select(1, valueSplit); // First split part is the value for the property
new cdk.CfnOutput(this, "TheParamValue", {
value: prop1Value,
description: "Parsed value from the custom resource",
});
After implementing the use of Instrinsic Functions and leveraging the built-in Custom Resource Action CDK provides for fetching data from AWS Parameter Store, I was able to successfully retrieve and insert the important piece of configuration needed for my Resources.
If there are more robust ways to handle JSON object like pieces of data in a context like this, please feel free to leave suggestions in the comments, but this turned out to be a working solution that enabled me to move forward and deploy my properly configured infrastructure.