Wednesday, 4 May 2022

Terraform State


If we execute terraform apply for the first time, TF will create a resource and assign to it a unique id. It also creates a file terraform.tfstate in the configuration directory. This file is a JSON file which maps real resources into their definitions in the configuration files. It contains all details about the resource (name, type, all its attributes and their values, resource id etc...). It is not possible to disable creating this state file. State is a non-optional feature in Terraform and it is a single source of truth about resources deployed.

If terraform apply hasn't been executed at all terraform show doesn't have where to read the values:

$ terraform show
No state.

If we execute terraform apply command again, TF creates a new state, compares it to the persisted state and  detects that resource has already been provisioned and will perform no action.  

If we change resource configuration and execute terraform plan, TF will create a new state in memory only, compare it to the persisted state and show the differences, what will be changed. The next terraform apply will delete the previous resource, create completely new resource, with new id and then persist this new state in .tfstate file. Configuration and state file are now in sync.

State file also tracks dependencies between resources: there is a "dependencies" attribute of type JSON array in JSON in .tfstate. Resources that are not dependent can be created in parallel, at the same time. If ResourceA depends on ResourceB then TF knows that it needs first to created ResourceB. If we remove both ResourceA and ResourceB from the configuration file and execute terraform apply TF uses information about their dependency written in .tfstate file in order to determine which resource to delete first (ResourceA then ResourceB in this case).

By default, terraform plan (and terraform apply) refreshes Terraform state (state of the infrastructure) before checking for configuration changes. This ensures that any changes that have happened out-of-band (e.g. manually, outside the usual TF workflow) are detected and that infrastructure converges back to what is defined in code. 

If we are certain that there were no changes outside the TF workflow, we can instruct TF to skip this preemptive refresh of the state by specifying -refresh=false:

$ terraform plan -refresh=false

New Terraform Planning Options: -refresh=false, -refresh-only, -replace

Every team member should have the latest .tfstate (state data) before running TF commands and should make sure that noone else runs TF commands at the same time otherwise an unpredictable errors could occur. .tfstate should be saved in remote store (AWS, Google Cloud Storage, HashiCorp Consul, Terraform Cloud) and not locally. This allows state to be shared among team members. 

terraform.tfstate contains values of all attributes that belong to resources. This also includes data considered sensitive (DB passwords, private IPs etc...). This means .tfstate files must be stored at secure storage. 

TF configuration files (.tf) can be stored in repositories like GitHub or BitBucket but state files (.tfstate) must be stored in secure remote backend systems (AWS, Google Cloud Storage etc...).

State files should never be manually modified. But should we need to do that, it should be done via terraform state command.

terraform refresh command syncs TF with the actual, real world infrastructure. If there was any manual change in the infrastructure, made outside Terraform (like manual update), this command will pick it up and update the state file. This command is run automatically within commands terraform plan and apply, prior to creating an execution plan. Using -refresh=false with these commands, prevents this.


Mutable and Immutable Infrastructure


If we configure and provision a local_file resource and then change permissions on it and re-execute terraform apply, we'll see from the output that TF first deletes the previously created file and then creates completely new file, with new permissions. This is an example of immutable infrastructure. Terraform uses this approach for all resources. 

An example of mutable (editable, changeable) infrastructure can be e.g. in-place update of the pool of NGINX servers (which exist for high availability). This is done manually, on each machine, during the maintenance window. This is called in-place update because the underlying infrastructure remains the same but the software and configuration get changed. The problem here might occur when upgrade of one or more servers fails (e.g. disk full, networking issues, incompatible OS versions etc...) in which case our pool of servers will have configuration drift: web servers will have different NGINX versions, OS versions, configurations...which would make their maintenance and issue fixing difficult.

The better approach is provisioning an immutable (non-changeable) infrastructure: spin up new web servers with upgraded versions and then remove old web servers. 

LifeCycle Rules

As mentioned above, if we edit resource configuration and re-execute terraform apply, TF will first delete previously created resource and then create completely new resource, with new attributes. What if we want TF first to create a new resource and then remove the old one? Or if we don't want old resource to be deleted at all?

To change default behaviour we can use lifecycle rules. They are specified in a lifecycle block. lifecycle is a meta-argument (like depends_on for example).

create_before_destroy = true instruct TF to create a new resource before destroying the old one

prevent_destroy = true prevents any changes to take place that would result in destroying the existing resource.

resource "local_file" "foo" {
    filename = "${path.cwd}/temp/foo.txt"
    content = "This is a text content of the foo file!"
    # file_permission = "0700"
    file_permission = "0777"
    lifecycle {
        # create_before_destroy = true
        prevent_destroy = true

If we have prevent_destroy set to true and perform the following sequence of actions:

terraform init
terraform apply
modify file_permission ("0700" -> "0777")
terraform apply

...we'll get the following error:

% terraform apply Refreshing state... [id=db5ca40b5588d44e9ec6c1b4005e11a6fd0c910e]
Error: Instance cannot be destroyed
│   on line 1:
│    1: resource "local_file" "foo" {
│ Resource has lifecycle.prevent_destroy set, but the plan calls for this resource to be destroyed. To avoid this
│ error and continue with the plan, either disable lifecycle.prevent_destroy or reduce the scope of the plan using the -target flag.

This is useful when dealing with resources that must not be accidentally deleted like DB instances. 

terraform destroy can still destroy the resource though. prevent_destroy lifecycle rule prevents destruction that are caused by resource configuration changes and subsequent terraform apply command.

ignore_changes - prevents the resource being updated if any of the attribute from the given list gets updated:

If we have:

resource "local_file" "foo" {
    filename = "${path.cwd}/temp/foo.txt"
    content = "This is a text content of the foo file!"
    file_permission = "0777"

    lifecycle {
        ignore_changes = [
            content, file_permission

...and execute terraform apply, Terraform will create the file:

  # will be created
  + resource "local_file" "foo" {
      + content             = "This is a text content of the foo file!"
      + directory_permission = "0777"
      + file_permission      = "0777"
      + filename             = "/path/to/temp/foo.txt"
      + id                   = (known after apply)

But if we then change content attribute to:

content = "Use this to test ignore_changes rule"

...and execute terraform apply again, no changes in the resource would happen as this attribute is listed within ignore_changes list:

% terraform apply Refreshing state... [id=db5ca40b5511d44e9ec6c1b4005e11a6fd0c910e]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

To ignore changes in any attribute use the following syntax:

ignore_changes = all

Data Sources

TF uses data sources to read attributes of resources which were provisioned outside the TF (e.g. manually). To define data source, a data block is used:

resource "local_file" "foo" {
    filename = "${path.cwd}/temp/foo.txt"
    content = data.local_file.my_data_source.content

data "local_file" "my_data_source" {
    filename = "${path.cwd}/data_source.txt"

While resource infrastructure item (managed resource) can be created, updated and destroyed, data infrastructure item (data resource) can be only read from.

No comments: