Terraform state file keeps track of the infrastructure which is under Terraform's control. Terraform compares resource configuration files against it in order to find out which resource needs to be added, edited or deleted. If state file gets lost, Terraform will try to re-create all resources.
By default Terraform state file (terraform.tfstate) is stored locally, on the machine where we initialize Terraform. But this carries the risk of adding this file (which may contain sensitive data) to the repository and pushing it to remote which can be a security risk or deleting it by chance which can be painful experience - see Lessons learned after losing the Terraform state file | Trying things.
To minimize chances of losing the Terraform state file and enable multiple contributors to work on the same infrastructure in parallel we should define a remote storage for it. We can store it in AWS S3 bucket, Google Cloud etc...but one of the totally free options, which also includes the shared state file locking mechanism, is Terraform Cloud.
Here are the steps which explain how to do it.
Sign Up for HashiCorp Cloud Platform (HCP):
- Go to Terraform Cloud (https://app.terraform.io/) and create an account.
- Create an organization (e.g. terraform-states) and a workspace (e.g. remote-state-demo) within Terraform Cloud. Workspaces are where state files are stored and managed.
Configure Terraform Cloud Backend:
- Add the following backend configuration to e.g. terraform.tf file:
terraform {
backend "remote" {
organization = "terraform-states"
workspaces {
name = "remote-state-demo"
}
}
}
Login to Terraform Cloud:
$ terraform login
Terraform will request an API token for app.terraform.io using your browser.
If login is successful, Terraform will store the token in plain text in
the following file for use by subsequent commands:
/home/<user>/.terraform.d/credentials.tfrc.json
Do you want to proceed?
Only 'yes' will be accepted to confirm.
Enter a value: yes
---------------------------------------------------------------------------------
Terraform must now open a web browser to the tokens page for app.terraform.io.
If a browser does not open this automatically, open the following URL to proceed:
https://app.terraform.io/app/settings/tokens?source=terraform-login
---------------------------------------------------------------------------------
Generate a token using your browser, and copy-paste it into this prompt.
Terraform will store the token in plain text in the following file
for use by subsequent commands:
/home/<user>/.terraform.d/credentials.tfrc.json
Token for app.terraform.io:
Enter a value: Opening in existing browser session.
Retrieved token for user <tf_user>
---------------------------------------------------------------------------------
-
----- -
--------- --
--------- - -----
--------- ------ -------
------- --------- ----------
---- ---------- ----------
-- ---------- ----------
Welcome to HCP Terraform! - ---------- -------
--- ----- ---
Documentation: terraform.io/docs/cloud -------- -
----------
----------
---------
-----
-
New to HCP Terraform? Follow these steps to instantly apply an example configuration:
$ git clone https://github.com/hashicorp/tfc-getting-started.git
$ cd tfc-getting-started
$ scripts/setup.sh
During this process a Terraform Cloud token generation page opens in browser:
terraform login should automatically pick the token and save it but in case this fails, you can copy the token and paste it here:
/home/<user>/.terraform.d/credentials.tfrc.json:
{
"credentials": {
"app.terraform.io": {
"token": "1kLiQ....h3A"
}
}
}
This authentication is necessary for the next step:
Initialize the Backend:
- Run terraform init to initialize the backend configuration
If we don't login to Terraform first we'll get:
$ terraform init
Initializing HCP Terraform...
╷
│ Error: Required token could not be found
│
│ Run the following command to generate a token for app.terraform.io:
│ terraform login
╵
If we're authenticated with Terraform:
$ terraform init
Initializing the backend...
Successfully configured the backend "remote"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Finding latest version of hashicorp/local...
- Installing hashicorp/local v2.5.1...
- Installed hashicorp/local v2.5.1 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Let's assume we have the following resource:
main.tf:
resource "local_file" "foo" {
filename = "${path.cwd}/temp/foo.txt"
content = "This is a text content of the foo file!"
}
We can now see the plan:
$ terraform plan
Running plan in the remote backend. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.
Preparing the remote plan...
To view this run in a browser, visit:
https://app.terraform.io/app/terraform-states/remote-state-demo/runs/run-nbxxG2TBxSYGEgCm
Waiting for the plan to start...
Terraform v1.9.3
on linux_amd64
Initializing plugins and modules...
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# local_file.foo will be created
+ resource "local_file" "foo" {
+ content = "This is a text content of the foo file!"
+ content_base64sha256 = (known after apply)
+ content_base64sha512 = (known after apply)
+ content_md5 = (known after apply)
+ content_sha1 = (known after apply)
+ content_sha256 = (known after apply)
+ content_sha512 = (known after apply)
+ directory_permission = "0777"
+ file_permission = "0777"
+ filename = "/home/tfc-agent/.tfc-agent/component/terraform/runs/run-nbxxG2TBxSYGEgCm/config/temp/foo.txt"
+ id = (known after apply)
}
Plan: 1 to add, 0 to change, 0 to destroy.
Notice that plan is running in remote backend and file path is also the one on the remote Terraform cloud machine. This is because we left our workspace to use organisation's Execution Mode which is Remote - all resources will be created on the remote machine. But this is not what we want, we want remote to contain only state file. Therefore we need to change the setting:
We can now apply the configuration (after executing terraform init so the new Execution Mode gets picked):
$ terraform plan
local_file.foo: Refreshing state... [id=db5ca40b5588d44e9ec6c1b4005e11a6fd0c910e]
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# local_file.foo will be created
+ resource "local_file" "foo" {
+ content = "This is a text content of the foo file!"
+ content_base64sha256 = (known after apply)
+ content_base64sha512 = (known after apply)
+ content_md5 = (known after apply)
+ content_sha1 = (known after apply)
+ content_sha256 = (known after apply)
+ content_sha512 = (known after apply)
+ directory_permission = "0777"
+ file_permission = "0777"
+ filename = "/home/<user>/...hcp-cloud-state-storage-demo/temp/foo.txt"
+ id = (known after apply)
}
Plan: 1 to add, 0 to change, 0 to destroy.
We can now execute terraform apply and changes will be done on the local machine.
If we create a resource on the remote (cloud), we can see it in web console:
If we by mistake create a resource on the remote (cloud), we can delete it by removing it from the state:
$ terraform state list
local_file.foo
$ terraform state rm local_file.foo
Removed local_file.foo
Successfully removed 1 resource instance(s).
All revisions of the state file are listed in Terraform Cloud.
We can also roll back to some of the previous versions:
After this we need to unlock the state file:
