Skip to main content
search

How to Deploy Infrastructure in CI/CD Using OpenTofu

By February 28, 2024Blog, Community

Contributed by Paweł Piwosz, Spacelift | originally posted on spacelift.io

đź’ˇ Read the second post in this series: Managing Infrastructure as Code (IaC) with OpenTofu

OpenTofu automates many tasks, including creating, changing, and versioning your cloud resources. Although many teams run OpenTofu locally, as they would Terraform (sometimes with wrapper scripts), running OpenTofu in CI/CD can boost your organization’s performance and ensure consistent deployments. 

In this article, we review different approaches to integrating OpenTofu into generic deployment pipelines. Let’s start with the basics.

What is OpenTofu?

OpenTofu is an Infrastructure-as-Code (IaC) tool created to keep the most popular IaC tool as an open-source project. It allows developers and infrastructure teams to define and provision infrastructure using a declarative configuration language.

It is stateful, maintaining a state file to track the current version of your infrastructure. This state allows OpenTofu to determine what changes need to be made to achieve the desired state defined in the configuration files.

At this point, OpenTofu is a drop-in replacement for Terraform, although this could change in the future as both projects evolve separately.

OpenTofu workflow

An OpenTofu workflow typically refers to the sequence of steps that engineers follow to write, plan, and apply infrastructure configurations using OpenTofu.

A standard workflow involves the following stages:

  • Writing the HCL Code
  • Initializing the working directory using “tofu init”
  • Validating the configuration using “tofu validate”
  • Formatting the configuration code using “tofu fmt”
  • Planning the configuration changes using “tofu plan”
  • Applying the configuration changes using “tofu apply”
  • Tearing down the configuration using “tofu destroy

Here’s how to deploy your infrastructure in CI/CD using OpenTofu.

1. Store the OpenTofu code

Should you store OpenTofu code in the same repository as the application code or maintain a separate repository for the infrastructure? This question has no definitive answer, but here are some insights that may help you decide:

  • When combined, the OpenTofu and application code represent one unit, so it’s simple for one team to maintain.
  • Conversely, if you have a dedicated team for managing infrastructure (e.g., a platform team), it is more convenient to maintain a separate repository for infrastructure because it’s a standalone project.
  • When infrastructure code is stored with the application, you may have to deal with additional rules for the pipeline to separate triggers for these code parts, but sometimes (e.g., in the case of serverless apps) changes to either part (app/infra) should trigger the deployment.

There is no right or wrong approach, but whichever you choose, remember to follow the Don’t Repeat Yourself (DRY) principle: Make the infrastructure code modular by logically grouping resources into higher abstractions and reusing these modules.

2. Prepare OpenTofu execution environment

Running OpenTofu locally generally means that all dependencies are already in place: You have the OpenTofu installed and present in the user’s PATH, and providers are already stored in the .terraform directory. OpenTofu uses .terraform directory as well as Terraform to keep backward compatibility. This directory is generated automatically when you run tofu init.

But when you shift Opentofu runs from your local machine to stateless pipelines, this is not the case. However, you can still have a pre-built environment — this will speed up the pipeline execution and provide control over the process.

Docker image with an OpenTofu binary is a popular solution for addressing this issue. Once it’s created, you can execute OpenTofu within a container context with configuration files mounted as a Docker volume.

But the main functionality of OpenTofu is delivered by provider plugins, in the same way as Terraform. It takes time to download the provider: For example, the AWS provider is about 250 MB, and on a large scale, with hundreds of OpenTofu runs per day, this makes a difference.

There are two common ways to address this: Either use a shared cache available to your pipeline workloads or bake provider binaries into the runtime environment (i.e., Docker image).

The critical element of both approaches is the configuration of the plugin cache directory path. By default, OpenTofu looks for plugins and downloads them in the .terraform directory, which is local to the main project directory. But you can override this by leveraging the TF_PLUGIN_CACHE_DIR environment variable to do that.

If supported by your CI/CD tool, the shared cache can significantly reduce the operational burden because all your pipeline runtime environments can use it to get the needed provider versions.

That way, all you have to do is to maintain the provider versions in the shared cache and instruct OpenTofu to use it:

  • Mount the cache directory to the pipeline runtime (i.e., docker container) and specify its internal path.
  • Set the value of the TF_PLUGIN_CACHE_DIR environment variable accordingly.

On the other hand, you can bake the provider binaries into the Docker image and inject the value for the TF_PLUGIN_CACHE_DIR environment variable right into the Dockerfile. This approach takes more operational effort but makes the OpenTofu environment (which works in the same way as a Terraform environment) self-sufficient and stateless. It also allows you to set strict boundaries around permitted provider versions as a security measure.

3. Plan and apply changes

Now, let’s review the ways to automate planning and applying of changes. Although tofu apply can do both, it’s sometimes useful to separate these actions. 

Initialization

CI/CD pipelines generally run in stateless environments. Thus, every subsequent run of OpenTofu looks like a fresh start, so the project needs to be initialized before other actions can be performed.

The usage of the init command in CI/CD differs slightly from its common local usage:

tofu init -input=false

The -input=false option prevents OpenTofu CLI from asking for user actions (it will throw an error if the input was required).

There is also a -no-color option that prevents the usage of color codes in a shell, so the output will look much cleaner if your CI/CD logging system cannot render the terminal formatting.

Another aspect of the init command (Terraform example under the link works in the same way for OpenTofu) that is useful in CI is -backend-config. This option allows you to override the backend configuration in your code or define it if you prefer to use partial configuration, thus creating more uniform pipelines.

For example, here’s how you can use the same code with different roles in different environments on AWS:

tofu init -input=false \

-backend-config="role_arn=arn:aws:iam::012345678901:role/QADeploymentAutomation"

However, I suggest checking in .terraform.lock.hcl, as suggested by OpenTofu (Dependency Lock File): This allows you to control dependencies more thoroughly, without worrying about transferring this file between build stages.

Plan

The tofu plan command helps you validate the changes manually. However, you can use it in automation as well.

By default, OpenTofu prints the plan output in a human-friendly format, but it also supports machine-readable JSON. With additional command line options, you can extend your CI experience. For example, you can use validation conditions to decide whether to apply the changes automatically, or you can parse the plan details and integrate the summary into a Pull Request description. Let’s review a simple example that illustrates this: 

First, save the plan output to the file:

tofu plan -input=false -compact-warnings -out=plan.file

The main point here is the -out option — it tells OpenTofu to save its output into a binary plan file. We will talk about it again later in this section. 

The -compact-warnings option suppresses the warning-level messages produced by OpenTofu

Also, the plan command has a -detailed-exitcode option that returns detailed exit codes when the command exits. For example, you can leverage this in a script that wraps OpenTofu and adds more conditional logic to its execution because CIs will generally fail the pipeline on a command’s non-zero exit code. However, that may add complexity to the pipeline logic. So if you need to get detailed information about the plan, I suggest parsing the plan output.

Once you have a plan file, you can read it in JSON format and parse it. Here is a code snippet that illustrates this:

tofu show -json plan.file| jq -r '([.resource_changes[]?.change.actions?]|flatten)|{"create":(map(select(.=="create"))|length),"update":(map(select(.=="update"))|length),"delete":(map(select(.=="delete"))|length)}'

# Output

{

  "create": 1,

  "update": 0,

  "delete": 0

}

Another way to see the information about changes is to run the plan command with -json option and parse its output to stdout: 

tofu plan -json|jq 'select( .type == "change_summary")|."@message"'

# Output

"Plan: 1 to add, 0 to change, 0 to destroy."

This technique can make your Pull Request messages more informative and improve your collaboration with teammates. 

You can write a custom script/function that sends a Pull Request comment to your VCS using its API. Or, you can try the existing features of your VCS: With GitHub Actions, you can use the action that comments PR to achieve this; for GitLab, there is a built-in functionality that integrates plan results into the Merge Request. 

You can find more information about the specification of the JSON output here: OpenTofu JSON Output Format.

Apply

Once the plan file is ready, and the proposed changes are expected and approved, it’s time to apply them.

Here is how the apply command may look like in automation:

tofu apply -input=false -compact-warnings plan.file

The plan.file is the file we got from the previous plan step.

Alternatively, you might want to omit the planning phase altogether. In that case, the following command will apply the configuration immediately, with no need for a plan:

tofu apply -input=false -compact-warnings -auto-approve

Here, the -auto-approve option tells OpenTofu to create the plan implicitly and skip the interactive approval of that plan before applying.

Whichever approach you choose, keep in mind the destructive nature of the apply command. Hence, the fully automated application of configuration generally works well with environments that tolerate unexpected downtimes, such as development or testing. On the other hand, plan review is recommended for production-grade environments, in which case the `apply` job is configured for a manual trigger.

4. Deal with stateless environments

If you run init, plan, and apply commands in different environments, you need to take care of some artifacts produced by Terraform:

  • The .terraform directory with information about OpenTofu modules, providers, and the state file (even in the case of remote state)
  • The .terraform.lock.hcl file — the dependency lock file that OpenTofu uses to check the integrity of provider versions used for the project: If your VCS does not track it, you’ll need to pass that file to the plan and apply commands to make them work after init.
  • The output file of the plan command is essential for the apply command, so treat it as a vital artifact. This file includes a full copy of the project configuration, the state, and variables passed to the plan command (if any). Therefore, observe the security precautions because sensitive information may be present there.

Make sure to pass them successively between operations: init -> plan -> apply.

There is one shortcut, however. You can execute the init and plan commands within the same step/stage and transfer the artifacts only once — to the apply execution.

5. Use the Command Line and Environments Variables

Last but not least, let’s discuss ways to maximize the advantage of variables when running OpenTofu in CI.

There are two common ways of passing values for the variables used in the configuration:

1. Using a -var-file option with the variable definitions file — a filename ending in .tfvars or .tfvars.json. For example:

tofu apply -var-file=development.tfvars -input=false -no-color -compact-warnings -auto-approve

OpenTofu can also automatically load the variables from files named exactly terraform.tfvars or terraform.tfvars.json: With that approach, you don’t need to specify the tfvar file as a command option explicitly.

2. Using environment variables with the prefix TF_VAR_. Implicitly, OpenTofu always looks for the environment variables (within its process context) with that prefix, so the same “instance_type” variables from the example above can be passed as follows:

export TF_VAR_instance_type=t3.nano

tofu -input=false -no-color -compact-warnings -auto-approve

The latter method is widely used in CI because modern CI/CD tools support the management of the environment variables for automation jobs.

Additionally, OpenTofu supports several configuration parameters in the form of environment variables. These parameters are optional; however, they can simplify the automation management and streamline its code.

  • TF_INPUT — when set to “false” or “0”, this tells OpenTofu to behave the same way as with the -input=false flag;
  • TF_CLI_ARGS — can contain a set of command-line options that will be passed to one or another tofu command. Therefore, the following notation can simplify the execution of apply and plan commands by unifying their options for CI:
export TF_CLI_ARGS="-input=false -no-color -compact-warnings"

tofu plan ...

tofu apply ...

You can leverage this further when using this variable as the environment configuration of stages or jobs in a CI/CD tool.

  • TF_IN_AUTOMATION  — when set to any non-empty value (e.g., “true”), OpenTofu stops suggesting commands run after the one you execute, hence producing less output. 

Automation Principles

Although this article provides a comprehensive overview of possible automation approaches, there is always room for specific and unusual scenarios. Whatever your use case or conditions, keep the following goals in mind when using OpenTofu in CI:

  • Ease of code management
  • A secure and controlled execution environment
  • Coherent runs of init, plan, apply phases
  • Leveraging built-in OpenTofu capabilities

See How It Works

Install OpenTofu and try it out for yourself. You can also find more information on the OpenTofu blog.

Close Menu