Friday 27 May 2022

Importing infrastructure in Terraform

 

Importing Resources


Some resources might be provisioned manually, via AWS Console, or by Ansible. If we want such resources to start being managed by TF we need to import them. The general syntax of the import command is:

terraform import <resource_type>.<resource_name> <attributes>

<attribute> is the resource attribute which can uniquely identify the resource such as ID. 

This command does not update configuration file but it tries to update state file with the details of the infrastructure being imported.

Example:

$ terraform import aws_instance.my-other-server i-0123456789

The first run of this command fails with error:

Error: resource address aws_instance.my-other-server does not exist in the configuration

To fix it, we can manually add it but without filling any details - we keep the resource block empty:

resource "aws_instance" "my-other-server" {
}

terraform import should now run with no errors. This resource is now imported into TF state file.

If we try to run terraform apply now, it would show the error: attributes not defined. This is because our resource has no attributes, it is still empty in the configuration file and we need to assign correct values. 

We can inspect terraform.tfstate and see the values of all attributes that belong to this resource.
Alternatively, we can find these details in AWS Management Console or by using AWS CLI like e.g.:

$ aws ec2 describe-instances

If we want to find a value of some particular attribute:

$ aws ec2 describe-instances --filters "Name=image-id,Values=ami-0123456789" | jq -r '.Reservations[].Instances[].InstanceId'

We should copy them into the resource configuration, e.g.:

resource "aws_instance" "my-other-server" {
    ami = "ami-0123456789"
    instance_type = "t2.micro"
    key_name = "ws"
    vpc_security_group_ids = [ "sg-0123456789" ]
}

This resource can now be fully managed by usual Terraform workflow including terraform apply.
 
 

Importing EC2 Key Pair

 
Let's assume EC2 key pair was created manually in AWS Management Console:
 
 
 
We want to get it under Terraform management (to be a part of our Terraform state). 
 
In our root configuration (e.g. main.tf file) we need to specify this resource and use its AWS Console Name as the value of the key_name attribute:

main.tf:

...

resource "aws_key_pair" "ec2--my-app" {
    key_name = "key-pair--ec2--my-app"
}
 
...

We can then perform the import:

$ terraform import aws_key_pair.ec2--my-app key-pair--ec2--my-app
aws_key_pair.ec2--my-app: Importing from ID "key-pair--ec2--my-app"...
aws_key_pair.ec2--my-app: Import prepared!
  Prepared aws_key_pair for import
aws_key_pair.ec2--my-app: Refreshing state... [id=key-pair--ec2--my-app]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.


terraform plan fails now:

$ terraform plan

│ Error: Missing required argument

│   on main.tf line 16, in resource "aws_key_pair" "ec2--my-app":
│   16: resource "aws_key_pair" "ec2--my-app" {

│ The argument "public_key" is required, but no definition was found.

During manual creation of EC2 key pair in AWS Console we have downloaded the private key so we can get the public key from it:

$ sudo chmod 400 key-pair--ec2--my-app.pem
$ ssh-keygen -y -f key-pair--ec2--my-app.pem > key-pair--ec2--my-app.pub

We can then reference this file in public_key value:
 
 
resource "aws_key_pair" "ec2--my-app" {
    key_name = "key-pair--ec2--my-app"
    public_key = file("./keys/key-pair--ec2--my-slack-app.pub")
}

Now:

$ terraform plan
aws_key_pair.ec2--my-app: Refreshing state... [id=key-pair--ec2--my-app]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create
-/+ destroy and then create replacement

Terraform will perform the following actions:

 ...

  # aws_key_pair.ec2--my-app must be replaced
-/+ resource "aws_key_pair" "ec2--my-app" {
      ~ arn             = "arn:aws:ec2:eu-west-1:036201477220:key-pair/key-pair--ec2--my-app" -> (known after apply)
      ~ fingerprint     = "a1:bc:ab:15:7e:87:d3:3b:e9:33:cd:21:8e:24:e7:8b:7b:ad:be:ad" -> (known after apply)
      ~ id              = "key-pair--ec2--my-app" -> (known after apply)
      + key_name_prefix = (known after apply)
      ~ key_pair_id     = "key-0986398ef799fdd42" -> (known after apply)
      + public_key      = "ssh-rsa AAAAB4NzaC1yc2EAAAADAQABAAABAQCmo/In0KJapZmvLFpBWwoOtf7RXrV4iQPjDcddWzG79q8jJlJKVtG1kI3l9XuU8hzmG0eqpyyhy61Hr9pLFtFWFUDa+RqAHYpUwSWV9a4JXRLwA5lEnxvXfIRGIHx7cALTawiVmVDTFJGqkJUfjWD7jHZTaK8NjOBY9k/IX0E51LayxjWxm2jJ1LJ8TTuSr/NYOpsnBDfmojgU9B3ZWAbvrtFwC6JkRJ0dR3YMx392TA9ky9MM/o/ItpZqOWWG64fDcEqNSUeIYPa+oLLlTyZy8aqwTJfLbV554x7G/U0vrd1H3H58GjANEuJAT7oHo94IcyQdmIgSXwlQtyXDEgbB" # forces replacement
      - tags            = {
          - "Description" = "Key pair used for SSH access"
        } -> null
      ~ tags_all        = {
          - "Description" = "Key pair used for SSH access"
        } -> (known after apply)
        # (1 unchanged attribute hidden)
    }

Plan: 2 to add, 0 to change, 1 to destroy.

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.


So terraform apply will replace the Key Pair we created which is not ideal (how to get hold of private key?). aws_key_pair documentation confirms this limitation when importing the key pair:

The AWS API does not include the public key in the response, so terraform apply will attempt to replace the key pair. There is currently no supported workaround for this limitation.

This brings me to conclusion that if we want to provision EC2 instance via Terraform, the best way to manage its SSH key pair is to create them on the local machine (via 3rd party tool like  ssh-keygen and then use aws_key_pair resource type) rather than create them in AWS Management Console.



No comments: