Testing the CDK

Preface

We’re still new to this CDK business, and as such, we’re not exactly claiming our methods are fully baked. But this is what we’re doing for now. If and when we find a better way, you can expect another snarky blog post with a preface stating that we still don’t know what we’re doing.

pulls collar nervously

How do I CDK?

If I were a lazier developer, I would simply rewrite the existing Terraform CDK tutorial and republish them here to complete my H2 goal of writing two blog posts. But lo! I am not so lazy. I will be as strong as oak holding myself accountable to milestones set 5 months ago that may no longer accurately reflect our team’s priorities! I will instead simply link to them (twice in one paragraph).

As you’ll notice in the tutorials, the CDK expects to operate in an OOP fashion - you know, classes and protection levels and other overly verbose, outdated notions 1. This being the case, we’ve written most of our code within classes that represent individual AWS constructs that we want to conjure up through cloud magic.

Acting as a Unit

There are two general use cases for unit tests in our terraform-modules repository: utility functions that live in their own module (e.g. url/string manipulation) and construct-specific configuration logic 2.

Ah, the peaceful sleep that comes when your code base is filled with easily unit testable, pure functions. I can feel the weight lifting from my shoulders just imagining such a world.

Sigh, why does our code actually have to do stuff?

Our current approach is to offload any helper/configuration/logical functions into either utility libraries or static functions within the construct classes themselves. This is your basic “same input always yields the same output” type testing.

Unfortunately, unit tests are a very small part of our test suite for this repo. Surprising no one, most of the code is acting against AWS APIs (via Terraform CDK APIs 3). So what output do we test here?

Jest About There

If you’ve read at all about the CDK, you’ll know part of the development process is running cdktf synth. What’s that? you haven’t read about the CDK yet? Well, I’ll wait. (Just kidding I wont - what is time these days anyway?).

cdktf synth takes all the TypeScript you’ve written and compiles (transpiles? dogpiles?) it to Terraform-specific JSON, which is then run by/through Terraform to summon your specified demons cloud services of yore. Given the same set of inputs, this synthesizing process will produce the same JSON. It’s not exactly unit-testable, buuuut it’s close to something else. What could it be…🤔

That’s right - React! 4 The Jest framework was created 5 to test that React components will render the expected HTML - and that’s basically what the CDK is doing for us here.

Jest works by first running your test and capturing the JSON output of cdktf synth and saving that output (called a snapshot) to a file. Subsequent test runs make sure that the JSON output matches the saved file. If something changes, your tests wont pass - which may or may not be expected. If you’ve added or changed functionality in your TypeScript, you can ensure that the generated JSON has changed in the expected way, and, if so, you can tell Jest to re-create the snapshot, which will make your tests again pass. However, if you haven’t made any changes to your TypeScript and your tests fail, you know that something else is changing the JSON - most likely some dependency (or even an upgrade of the CDK library). Again, here you can verify that the changes to the JSON won’t have any unintended consequences when it’s run against actual infrastructure.

// an example of a Jest snapshot test

test('renders a dynamo db table with minimal config', () => {
  const app = Testing.app();
  const stack = new TerraformStack(app, 'test');

  // ApplicationDynamoDBTable is one of our custom constructs
  // BASE_CONFIG is just an object constant conforming to an interface
  new ApplicationDynamoDBTable(stack, 'testDynamoDBTable', BASE_CONFIG);

  expect(Testing.synth(stack)).toMatchSnapshot();
});

This is testing level two (snapshot boogaloo), but there’s still a BIG ol’ piece of the testing pizza missing here - is the JSON generated by our TypeScript valid? Does it tell AWS to build the things we want in the way we want them? Do Terraform and AWS talk about us when we’re not around?

Cloudy, with a Chance of Working

Unit tests - can’t argue with those 6. Jest snapshots - a clever solution to testing the consistency of your JSON output. But does it work?

This is by far the most, uh, in-progress part of our testing suite. Here’s what we’ve got so far - in the cdktf.out directory created after running cdktf synth, we:

  1. Run terraform init, then terraform validate to make sure the JSON is valid. If the changes are deemed minimal - e.g. adding tags to a service or changing a CloudWatch threshold - we usually stop here.
  2. If we’re doing larger modifications to a service, or spinning up a new service, we run terraform plan (in our dev account!) to verify the change. Then terraform apply. Then we check the AWS console to make sure things are as expected 7. Finally, we terraform destroy because AWS sure doesn’t need any more money 8.

It’s not exactly smooth, and requires you to have local AWS credentials, but it’s a good amount of local testing before filing a PR. (Of course, we have nothing but trust to ensure the dev filing the PR has done the above due diligence. But also if you don’t trust your teammates…see 8.)

Reality Check

This all probably reads a little rosier than actual practice. We currently have very few unit tests. We could improve here. We also generally don’t do a good job checking our Jest snapshot changes. If the tests fail, we just force new snapshots and roll. Eek. As this library matures further, and becomes the backbone for the majority of our services, we’ll make improvements - both to code quality and testing. I shall demand it so. (And have already started complaining about our spaghetti methods.)

If you’ve read this far about testing infrastructure as code, you are truly a saint. Hopefully this post inspires some improved testing on the infrastructure code 9 at your employer, and gets you a nice little raise in the process.

Until next time, take care of each other, and I’ll see you on the internet.

~ jonathan, backend team



  1. If you’ve ever written a factory factory in OOP you know exactly what I mean. And I’m sorry for your pain. ↩

  2. I try my best not to write such jargon-heavy sentences, but sometimes even I sound like a developer. Sorry. ↩

  3. I know programming has always been about layers of abstraction, but it’s getting a little out of hand. You kids can stay on my lawn (you are the future, after all), but you’re listening to my rap music. ↩

  4. I began my career in front-end dev and miss it and I have feelings about React. FEELINGS. ↩

  5. I mean, I assume this is why it was created. I think I’m right. If I’m not, do not email me. ↩

  6. jk this is the internet people will argue about anything. ↩

  7. We actually don’t do this check. But we could! ↩

  8. If your CEO does the “push dev live!” thing we might be hiring ↩ ↩2

  9. Someday I want to explore the impact of “infrastructure as code”, but alas this is not that day. (Expect to see this footnote regularly.) ↩


Tagged with: #infrastructure, #terraform, #typescript, #testing