Thursday 19 May 2022

Using AWS S3 as Terraform Backend


Terraform backend is the place where Terraform stores the state file. By default, it's a local storage (local machine) but it can also be a remote one (AWS, GCS etc...).

In Terraform State | My Public Notepad it was discussed why it's better to use a remote Terraform backend rather than a local one or using a version control system (e.g. Git repository).
AWS-based remote backend comprises:
  • S3 bucket which stores TF state file
    • bucket name e.g. tf-state-bucket
    • key (of the stored resource) is the object path where the state file is stored e.g. path/to/terraform.tfstate
    • region e.g. eu-south-1
  • DynamoDB table which implements state locking and consistency checks
    • name e.g. tf-state-locking
    • this table must have a primary (hash) key named lockId
To configure remote backend in Terraform, we need to use terraform block in configuration file. We already mentioned this block in Terraform Providers | My Public Notepad when we wanted to specify exact plugin version. There we used terraform_providers block but here, to specify TF backend, we need to use backend block:
resource "local_file" "foo" {
    filename = "/root/foo.txt"
    content = "This is a content of foo.txt."

It is a good practice to keep terraform block in a separate file e.g.

terraform {
    backend "s3"  {
        bucket = "tf-state-bucket"
        key = "path/to/terraform.tfstate"
        region = "eu-south-1"
        dynamodb_table = "tf-state-locking"

backend block "s3" has 3 mandatory attributes: bucket, key and region. dynamodb_table is an optional argument. 

If we've used terraform init before switching to the remote backend, terraform apply would issue an error stating that backend reinitialization is required. We simply need to re-run terraform init which will migrate pre-existing state from local to a new s3 backend (state file will be copied from a local disk into s3 bucket). After this we can delete local state file:

$ rm -rf terraform.tfstate

Any future executions of terraform plan or apply would be using the state file stored remotely, in s3 bucket. Pulling and pushing the terraform.tfstate file will be automatic. Prior to each of these operations the state lock would be acquired and after them, it would be released. This will keep integrity of the remotely stored state file.


No comments: