Tuesday 31 May 2022

Terraform Modules

 

Terraform considers every .tf file in configuration directory as configuration file. This means that we can define all resources in a single .tf file or divide them into multiple .tf files. 
 
In practice, there can be hundreds of resources and both options above prevent reusability. 

Terraform module is:

A module where we run Terraform commands from is called a root module. Every Terraform configuration therefore has a root module.

Terraform commands operate on configuration files in the root module (current (working) directory) but configuration files can load configuration files (other modules) from local or remote sources via module blocks:

module "child_module_local_name" {
   source = ...
   version = ...
   child_module_input_variable_1 = ...
   child_module_input_variable_2 = ...
   child_module_input_variable_3 = ...
   ...    
}

We say that root module calls other modules (child modules) to include their resources into configuration. In the example above, the root module calls a child module and uses child_module_local_name as its local name. It sets child module's input variables and later it can reference ONLY output variables declared in the child module by using the following syntax:

id = module.child_module_local_name.provisioned_resource_id

provided that in the child module, in outputs.tf we have something like:

output "aws_resource_id" {
  description = "The ID of the AWS resource this module creates"
  value       = try(aws_resource.this.id, "")
}


Root module loads a local module if this resides in a local filesystem. 
Root module loads a remote module if this one is a remote resource.

source is mandatory argument and is used for specifying local or remote location of the child module. 

version is used for modules published in remote repositories.

Other arguments are simply input variables for child module where we set thier values thus passing data into child module (this is like calling a function in conventional programming language).

 

Calling local modules

 
Let's see how to load a local module. Let's assume we have the following hierarchy:
 
../my-projects/A/
../my-projects/A/main.tf
../my-projects/A/variables.tf
../my-projects/B/
../my-projects/B/main.tf

To include a module A (in directory A) in a configuration file in module B (in directory B) we can do the following in ../my-projects/B/main.tf:
 
module "project-A" {
    source = "../A"
}
 
Module A is a child module of module B. project-A is the logical name of the module. source is a a required argument in module block. Its value is a relative or an absolute path to the child directory. 

In practice, all reusable modules should be stored in a modules directory, grouped by their projects.

This example shows the project outline and configuration for provisioning resources for application that needs to be deployed in various AWS regions.

Project outline:

../my-projects/modules/
../my-projects/modules/my-app/app_server.tf
../my-projects/modules/my-app/dynamodb_table.tf
../my-projects/modules/my-app/s3_bucket.tf
../my-projects/modules/my-app/variables.tf
 
../my-projects/modules/my-app/app_server.tf:
 
resource "aws_instance" "my_app_server" {
    ami = var.ami
    instance_type = "t2.medium" 
    tags = {
        Name = "${var.app_region}-my-app-server"
    }
    depends_on= [
        aws_dynamodb_table.orders_db,
        aws_s3_bucket.products_data
    ]
}
 
../my-projects/modules/my-app/s3_bucket.tf:
 
resource "aws_s3_bucket" "products_data" {
    bucket = "${var.app_region}-${var.bucket}"
}

../my-projects/modules/my-app/dynamodb_table.tf:
 
resource "aws_dynamodb_table" "orders_db" {
    name = "orders_data" 
    billing_mode = "PAY_PER_REQUEST"
    hash_key = "OrderID"
    attribute {
        name = "OrderID" 
        type = "N"
    }
}

../my-projects/modules/my-app/variables.tf:
 
variable "app_region" {
    type = string
}

variable "bucket" {
    default = "product-manuals"
}

variable "ami" {
    type = string
}


If we want to deploy this infrastructure stack to e.g. eu-west-1 region (Ireland) we can create a directory ../my-projects/my-app-ie/ and in it:
 
../my-projects/my-app-ie/provider.tf:
 
provider "aws" {
    region = "eu-west-1"
}
 
../my-projects/my-app-ie/main.tf:
 
module "my_app_ie" {
    source = "../modules/my-app"
    app_region = "eu-west-1"
    ami = "ami-01234567890"
}
 
We can see that there are only two variables that differentiate deployment to each region. To provision this infrastructure stack in this region we just need to cd into ../my-projects/my-app-ie/ and execute:
 
$ terraform init
$ terraform apply

If we want to deploy it in e.g. Brasil, we'll have:
 
../my-projects/my-app-br/provider.tf:
 
provider "aws" {
    region = "sa-east-1"


../my-projects/my-app-br/main.tf:
 
module "my_app_br" {
    source = "../modules/my-app"
    app_region = "sa-east-1"
    ami = "ami-3456789012"
}
 
The usual practice is that the same variables are defined and set at the parent level so they can be used for setting values for module's variables:
 
 ../my-projects/my-app-br/variables.tf:
 
variable "app_region" {
    type = string
}

variable "bucket" {
    default = "product-manuals"
}

variable "ami" {
    type = string
    default = "ami-123456789"
}

 
 
...and these values are then passed to the module:
 
../my-projects/my-app-br/main.tf:
 
module "my_app_br" {
    source = "../modules/my-app"
    app_region = var.app_region
    ami = var.ami
}
 
We can see that app_region does not have value set in the code. Variables defined at parent level can be set when calling terraform plan or terraform apply, from a command line:
 
$ terraform apply -var app_region=eu-west-1

If we try to pass value for some variable that is not defined at the root/parent level, we'll get the following error:
 

│ Error: Value for undeclared variable

│ A variable named "appregion" was assigned on the command line, but the root module does not declare a variable of that name. To use this value, add a
│ "variable" block to the configuration.



Calling modules from the public registry

 
Apart from provider plugins, Terraform registry also contains modules:



Modules are grouped by the provider for which they are created. There are two types of modules:

  • verified - tested and maintained by Hashicorp
  • community - not validated by Hashicorp
 
Example of verified module: AWS module security-group, used to create EC2-VPC security groups on AWS. 
 

 
 
To use it in our own configuration we can first copy-paste code snippet which can be found under Provision Instructions section:

module "security-group" {
  source  = "terraform-aws-modules/security-group/aws"
  version = "4.9.0"
}


module security-group has ssh submodule which can be used to create predefined security groups like this one which allows inbound SSH:

module "security-group_ssh" {
    source  = "terraform-aws-modules/security-group/aws//modules/ssh"
    version = "4.9.0"
    vpc_id = "vpc-0123456789" 
    ingress_cidr_blocks = [ "10.11.0.0/16" ]
    name = "ssh-access"
}
 
terraform get only downloads module from the registry:
 
$ terraform get

When using 3rd party modules, terraform apply might be provisioning additional resources (on top of those we explicitly add to the configuration), as per module's configuration.
 

Calling modules from another Git repository

 
It is possible to call modules defined in an arbitrary Git repository. 
 
 There are two different ways to write a Git SSH URL for Terraform:

# "scp-style":
git::username@hostname:path

# "URL-style":
git::ssh://username@hostname/path

 
In both of these cases, Terraform is just taking the portion after the git:: prefix (after also removing any //subdir and ?rev=... portions) and passing it to git clone:

git clone username@hostname:path
git clone ssh://username@hostname/path

 
How the rest of this is interpreted is entirely up to git. Notice that the scp-style string uses a colon to separate the path from the hostname, while the URL style uses a slash, as described in the official git documentation.
 
It is recommended using the "URL-style" because it's consistent with the other URL forms accepted in module source addresses and thus probably more familiar/intuitive to readers.

If your SSH server is running on a non-standard TCP port (not port 22) then you can include a port number only with a URL-style address by introducing a colon after the hostname:

# URL-style with port number
git::ssh://username@hostname:port/path

 
 
Let's assume we have TF module in repo ssh://git@git.example.com, in the directory path/to/module/. In order to call this module we need to use the following value for source:

module "child_module_name"  {
    source = "git::ssh://git@git.example.com/org/repo//path/to/module"
}

If using web url and tag v2.1 on the default branch:

source = "git::https://git.example.com/org/repo.git?ref=v2.1"
 
If using some other branch:
 
source = "git::https://git.example.com/org/repo.git?ref=branch-name"
 
If using a module nested in hierarchy:
 
source = "git::https://git.example.com/org/repo.git//path/to/module?ref=branch-name"

It is also possible to specify particular commit id:

source = "git::https://git.example.com/org/repo.git//path/to/module?ref=62d462976d84fdea54b47d80dcabbf680badcad1"


Resources:

 
 

No comments: