Infrastructure-as-Code (Iac) tools like Terraform are experiencing fast growing adoption by the developer community. This is no surprise because they greatly simplify the operational overhead associated with deploying and managing cloud infrastructure. In support of this growing trend, today Google are pleased to announce Google Cloud CLI’s preview release of Declarative Export for Terraform.
Declarative Export allows you to export the current state of your Google Cloud infrastructure into a descriptive file compatible with Terraform (HCL) or Google’s KRM declarative tooling. Here’s how it works and why this is useful to your day-to-day cloud development workflows.
What problem does this solve?
A common gap in IaC workflows on Google Cloud is how to actually create a HCL Terraform file describing all their Google Cloud resources and their states if one did not begin their cloud journey this way. It would be like trying to draw up the blueprints for the house after the house was already built, step by step, by people in the field without documentation of what happened or in what order.
Declarative Export makes it easier to migrate existing Google Cloud resources to Terraform. It also protects you from incorrect configurations and drift.
To export all resources in a folder to a local directory in Terraform format, run:
$ gcloud beta resource-config bulk-export --folder=12345 \--resource-format=terraform --path=/path/to/dir/
To generate a main.tf file in the current directory using the gcloud default values for zone, region and project run:
$ gcloud alpha resource-config terraform init-provider
This command will generate a main.tf file to configure Google Cloud Terraform Provider.
To generate an import script named import.sh and a module file named modules.tf based on the files exported to a local directory, run:
$ gcloud beta resource-config terraform generate-import \/path/to/dir/--output-script-file=import.sh --output-module-file=modules.tf
With Google Cloud CLI’s Declarative Export feature, you can use any tools of choice to ‘build up’ your cloud infrastructure – Cloud Console, the CLI, your existing gcloud or Ansible scripts, or any other method – and then use Declarative Export to create a Terraform compatible file describing the current state of every resource. Using the earlier house analogy, it is like being able to x-ray the already built home to reproduce the original plans used to construct it.
Benefits over imperative approaches:
Traditionally, developers and operators configure and manage their cloud infrastructure through a series of sequential or procedural steps. For example, the operator might first launch a virtual machine, then add a disk, some firewall and load-balancing rules, and so forth. As the complexity of cloud infrastructure grows, these imperative approaches become both more toilesome and error prone. If a step is done out of order, against the wrong resource, or with the wrong instruction, the infrastructure will not be configured correctly all the way through to the desired end state.
Compounding this complexity is that as development teams grow, so too does the likelihood that different people (or scripts) make changes to the same infrastructure. This drift over time can cause unexpected behaviors across the deployment and make it hard to track what changed much less how to revert back to a good known state.
How IaC helps:
The rise of declarative IaC tools enables developers to instead take a different approach: describe the desired target state of the infrastructure, and then let the IaC tools do the work to get the infrastructure into that state.
Terraform has, for several years, provided this type of IaC solution. Terraform uses a Hashicorp proprietary syntax (HCL) to describe the specific cloud resources and the desired end state. These files can be tweaked to depict desired alterations to the end state and then re-applied by Terraform to effectuate whatever is needed to get the infrastructure into the described state. The tooling itself determines what specific invocations need to be made to get to the goal. This frees the operator from having to manage the many sub-steps and their order.
Additionally, expressing the state of infrastructure as a text file means this file can be checked-in to a repository to gain all the benefits of repository management: diffs, conflict resolution, commit notes, timestamps, and so forth. Now a developer can select a known revision from their history and ‘snap’ to it by applying that specific HCL file using Terraform.
Primary use cases for Declarative Export
The HCL file can be used to run a plan and apply in Terraform. This will put the infrastructure into the described target state.
Google anticipate declarative export being used as follows:
- Resolving infrastructure drift: over time, and especially when multiple people are involved in configuring infrastructure on behalf of an organization, the infrastructure will stray from a known accurate and correct original state. By using Google Cloud CLI’s declarative export to capture the good known state before drift occurs, the operator can later leverage Terraform to get the drifted infrastructure back into the good known state.
- Cloning an environment: complex stacks may take hours or days to properly configure. Rather than redoing all the steps manually, or spending effort to script the steps, you can instead simply export the declarative depiction of the entire environment and then use Terraform to create clones. Example use cases would be to clone to spawn a staging environment or to clone a stack into another region. In these cases you would apply minor tweaks to the HCL file such as changing the zone or region before reapplying it.
- Deploying templatized software stacks: You are a company with pre-defined stacks you either sell as a SaaS product or use for internally approved application instances. You may use the tools of choice to set up the ‘perfect’ Google Cloud infrastructure configuration and then recreate multi-resource instances on demand.
- Troubleshooting and / or reverting in case of emergencies: if you can ascertain when a problem started, you can diff the associated and previous HCL files to see what precise changes might be at cause. Or, you can apply a known to be good HCL to get back into a working state to de-escalate the emergency situation and allow more organized remediation.
- Using Declarative tools for day-to-day workflows: you may find yourself more productive editing HCL files as compared to using the Cloud Console or CLI to effectuate changes across your infrastructure.
Current Limitations with this preview release:
- Many but not all Google Cloud resources are covered. To list all exportable resource types in project my-project in json format, run:
$ gcloud beta resource-config list-resource-types --format=json \--project=my-project
- Manual apply in Terraform. You will simultaneously use gcloud + terraform tools to ‘import’, apply, and run the Terraform file.
What’s next
In the future, these steps will be simplified directly from within the gcloud CLI using gcloud beta resource-config terraform generate-import or similar. Google will also continue to add support for additional resources.
Leaving feedback:
Google would love to hear back from you on how this feature is working for you or for your ideas on improvements they can make.