Just in Time Cloud Infrastructure

Most cloud engineers are familiar with the paradigm of Infrastructure as Code (IaC), which provides the ability to model and deploy your cloud infrastructure via code, or code-generated configuration files. IaC drastically reduces the reliance on manual processes such as creating separate configurations for multiple environments, helps minimize misconfiguration mistakes through automated testing and version control, and can significantly speed up deployment times through CI/CD pipelines.

Implementing IaC at scale still requires significant investment, and many companies have teams dedicated to the management and deployment of their cloud infrastructure. This can cause friction between the infrastructure and application teams since changes to shared resources can have an impact across teams. Deployment cycles can also be significantly slowed down if complex integration tests are required whenever core components are updated to ensure no unintended consequences occur downstream.

Just in Time Infrastructure (JiTI) which is a term my team created, looks to solve some of these difficulties by tying the infrastructure code more closely to the application code. This serves the purpose of isolating the infrastructure as much as possible, coupling infrastructure versions to application versions, and allowing applications to potentially be shared and scaled across accounts.

This blog post will cover the origination of JiTI and the problems it is designed to solve, dive into the benefits and limitations of utilizing JiTI, and end with a working example.

Origination of Just-in-Time Infrastructure

Just-in-time infrastructure is a model in which the entire infrastructure stack required for an application is written as code and deployed with the same lifecycles alongside the application.

The idea for JiTI came during the development of a data analytics platform for data scientists, many of whom wanted to use the platform with sensitive data. Working with sensitive date made it extremely difficult to gain the sufficient permission and access needed for the platform to be hosted in one centralized location. Therefore, the team decided early on to design the platform to be “dropshippable” (deployed in any cloud account with minimal to no code changes required), and simply deploy the platform directly into any account where the data and the data scientists were already working.

There was just one catch - the team building the platform did not have the resources to manually configure and stand up the platform in dozens of accounts. So, in addition to the data analytics platform, we also built a Python-based cloud resource management framework which has recently been open-sourced by Capital One as Particle Cloud Framework. This framework was built intentionally with JiTI in mind, but the principles of JiTI can be applied with any programming language and framework.

Using Particle Cloud Framework, we were easily able to combine the platform codebase with the infrastructure required to run it, making it extremely simple to scale out (in this case on AWS, but the same can be done on any cloud provider). The infrastructure codebase contained not only the compute and storage resources required, but all components including the VPC, subnets, IAM roles (access permissions), and networking resources. In cases where some of the resources had to be shared with other applications, or we did not have the required permissions to dynamically create a resource, the resource was passed in to the configuration programmatically. Otherwise, all resources were generated during run time.

Combining the platform with its infrastructure code is an extremely powerful concept and ensures the required infrastructure is provisioned not only correctly, but consistently, between development and production environments as well as across accounts. Now, when additional features that require additional infrastructure components are ready to be deployed, users simply update the application, which as part of the update, would also deploy the new required infrastructure.

Benefits of Just-in-Time Infrastructure

While JiTI was initially conceived to simplify the deployment of applications to various accounts and environments, reducing operation overhead, there are many other benefits that come as a result of this infrastructure model.   

Application Isolation

Tying your application and infrastructure code reduces the risk of negatively affecting other applications and vice versa. JiTI ensures you have exactly the same infrastructure in all your development, QA, and production environments. Furthermore, JiTI allows the infrastructure code to be versioned directly alongside your application, enabling the deployment of various versions at the same time with different infrastructure requirements.

Cost Management

JiTI makes resource management much simpler since resources can simply be attributed to unique applications, so there is no risk when deleting rogue or unused infrastructure. Additionally, JiTI simplifies the deployment and tearing down your entire application, providing  cost savings while not utilizing the application (especially for development, test, and QA environments). 

Dropshipable

Deploying an application and its infrastructure together in another account or environment ensures the application works as promised.

Auditability

Since the entire infrastructure is written as code alongside the application, auditing is much easier because JiTI assumes all dependencies are in the same code base.

Security

JiTI ensures that resources related to security such as networking, access control, and identity management are deployed only when they're needed by an application. When the application gets terminated, those vulnerable resources also get terminated reducing the security risk from leftover and outdated resources.

Resiliency

JiTI makes it easy to merely recreate the entirety of the infrastructure if things go wrong, instead of having to manually inspect what might have caused a failure and testing a fix and its potential effects on the rest of the ecosystem.

Self Management

JiTI can enable applications to manage their own infrastructure once deployed. This is useful if an application has a variable or unpredictable workload and can automatically scale the underlying infrastructure itself.

Considerations When Using JiTI

There are some downsides to shifting to a JiTI model with two of the key considerations being governance and permission management.

Governance

Since applications are now tightly coupled with the infrastructure, centralized governance teams will need to incorporate new standards and procedures to ensure security, cost, and regulatory compliance. This can be done passively through tools that actively scan and delete resources that are non compliant, or fine-grained access management that limits the scope of each application.

Permissions

Adopting JiTI may require a new way to manage access management since application teams are deploying the entire infrastructure stack from security rules and subnets, to compute and networking resources. One way to manage this is to create a centralized tool that has the actual permissions to deploy, and before any deployment it could programmatically scan the application code to enforce customized rules (for example prohibit public endpoints).

JiTI Example using Particle Cloud Framework

 Now that the basics of JiTI have been covered, I’d like to go over what this might look like in practice.

This example will be based off an example written using Particle Cloud Framework running on AWS, but again JiTI principles can be applied using your prefered language, framework, and cloud provider. This example will cover the deployment and termination of an application and all infrastructure resources as part of the normal build cycle in development environments

The resources generally required for a simple application running on the cloud include basic networking (VPC, subnet), security (security group), compute instances (EC2), access management (Instance Profile), and the application itself. These resources all have various dependencies on each other, as illustrated in the figure below.

One of the advantages of using Particle Cloud Framework for JiTI is the fact that resource dependency management is handled by default, ensuring resources are both provisioned and terminated in the correct order. For example, the subnet resource takes in the VPC resource as a parent. This creates the VPC first and then passes the necessary parameters to create the subnet resource.

The EC2 particle has the most dependencies, and generally requires the user to lookup and pass in various parameters as well as spend time making sure all dependencies are correct. Having multiple users working in parallel, as well as numerous environments, can make debugging issues difficult and time consuming. This can be avoided entirely simply by programmatically ensuring all infrastructure resources are the same.

In the example below the EC2 instance has all its dependencies mapped out as its parents and also automatically retrieves all required parameters via PCF’s lookup and inherit features. A key takeaway here is that there are no hard coded variables, meaning this configuration is account agnostic and can be run in any account or environment and produce identical results every time.

Now that we have all our resources mapped out, all that is required is wrapping them all in a quasiparticle (PCF collection of resources), setting the desired state to running, and running the apply function. Since Particle Cloud Framework is python based these commands can be triggered anywhere python can be run, including a workstation, a CI/CD server, or even a serverless function. To terminate the entire stack after you are finished with development, simply set the desired state to terminated and apply again. These infrastructure configurations should be stored in the same repo as the application. The application install scripts can be combined with the infrastructure code by adding the steps to install and start the application to the EC2 userdata. You now have the ability to easily version your application and infrastructure code, as well as a simplified development workflow and deployment process.

Conclusion

JiTI looks to build on the core fundamentals of IaC, which include programmability, automation, and reusability, by tying infrastructure more closely to the application. Combining the infrastructure stack with the application codebase allows them to share the same deployment lifecycles yielding numerous benefits including higher resiliency, cost savings, preciser audibility, easier multi-account deployments and smaller operations burden. Before adopting JiTI consider the governance and access restrictions that your environments may require as these may limit the benefits of JiTI and increase the operations burden.

I appreciate you taking the time to learn about JiTI and rethink the typical relationship between applications and cloud infrastructure. This is yet another way the cloud is changing how we think about and develop applications. I hope is this is just the beginning of an ongoing discussion on JiTI and look forward to any and all of your comments and feedback. Feel free to reach out to me personally as well at austen.novis@capitalone.com.