Every DevOps and platform job post lists Terraform, and in Indian service-company interviews (Infosys, Wipro, TCS) it is one of the three tools you will certainly be asked about. The questions are concrete: what is in the state file, what happens if two people apply at once, how do you handle dev, staging and prod, what do you do when plan says it will destroy the database. In real work, Terraform is where the biggest mistakes happen: one wrong apply can delete a cluster. Knowing the safe workflow is the difference between a junior and someone trusted with the prod account.

Why infrastructure as code

Click a VPC together in the console and nobody can repeat it in a second account, review it first, or see what it looked like yesterday. IaC fixes all three: infrastructure is files in Git, so it is repeatable (apply the same code to a new account), reviewable (the pull request shows the diff) and recoverable (check out last week's commit and apply it).

Imperative code says what to do, step by step: a shell script of AWS CLI calls, which creates everything twice if run twice. Declarative code says what should exist: "a subnet with this CIDR". The tool works out the API calls, and a second run finds nothing to do. Terraform is declarative.

The core workflow

Terraform code, plan and apply across the top; the state file in S3 and the real AWS account below, with a drifted security group rule

terraform init               # download providers and modules, set up the backend,
                             # write .terraform.lock.hcl (commit it: it pins provider versions)
terraform fmt -check         # formatting; CI fails if a file is not formatted
terraform validate           # syntax and types, no cloud calls
terraform plan -out=tfplan   # refresh state from the cloud, diff against code, save the plan
terraform show -json tfplan  # the same plan as JSON, for scripts and policy tools
terraform apply tfplan       # execute exactly that saved plan, nothing else
terraform destroy            # plan and apply the removal of everything in this state

plan does two comparisons: it reads the real cloud and compares it with the state file (refresh), then compares the refreshed state with your code. The output lists every resource to add, change or destroy, with the attribute that caused it and, for a replacement, "forces replacement". plan -detailed-exitcode returns 0 for no changes, 1 for an error, 2 for changes present.

Always apply a saved plan file. terraform apply without a file plans again; if someone merged in between, you apply a diff nobody reviewed. In prod, CI must never be able to destroy.

HCL: the language

One trimmed root module that shows every block type you must recognise:

terraform {
  required_version = ">= 1.11"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
  backend "s3" {
    bucket       = "acme-tf-state"   # versioning and SSE on, public access blocked
    key          = "prod/network/terraform.tfstate"
    region       = "ap-south-1"
    encrypt      = true
    use_lockfile = true              # S3 native lock, no DynamoDB table needed
  }
}

provider "aws" {
  region = "ap-south-1"
}

variable "environment" {
  type        = string
  description = "dev, staging or prod; used in names and tags"
  validation {
    condition     = contains(["dev", "staging", "prod"], var.environment)
    error_message = "environment must be dev, staging or prod."
  }
}

variable "db_password" {
  type      = string
  sensitive = true   # hidden in plan output; still written to state unless ephemeral
}

variable "subnets" {
  type    = map(string)   # key = short name, value = CIDR
  default = { a = "10.0.1.0/24", b = "10.0.2.0/24" }
}

locals {
  name_prefix = "${var.environment}-orders"
}

resource "aws_vpc" "main" {
  cidr_block           = "10.0.0.0/16"
  enable_dns_hostnames = true
}

resource "aws_subnet" "private" {
  for_each   = var.subnets   # addresses: aws_subnet.private["a"], ["b"]
  vpc_id     = aws_vpc.main.id
  cidr_block = each.value
  tags       = { Name = "${local.name_prefix}-${each.key}" }
}

resource "aws_db_instance" "main" {
  identifier                = "${local.name_prefix}-db"
  engine                    = "postgres"
  instance_class            = var.environment == "prod" ? "db.r6g.large" : "db.t4g.micro"
  allocated_storage         = 20
  username                  = "app"
  password                  = var.db_password
  skip_final_snapshot       = false
  final_snapshot_identifier = "${local.name_prefix}-final"
  depends_on                = [aws_vpc.main]   # only when Terraform cannot see the dependency
  lifecycle {
    prevent_destroy = true        # plan errors instead of destroying this
    ignore_changes  = [password]  # rotated outside Terraform, do not fight it
  }
}

resource "aws_security_group" "api" {
  name   = "${local.name_prefix}-api"
  vpc_id = aws_vpc.main.id
  dynamic "ingress" {
    for_each = [443, 8443]
    content {
      from_port   = ingress.value
      to_port     = ingress.value
      protocol    = "tcp"
      cidr_blocks = ["10.0.0.0/8"]
    }
  }
  lifecycle {
    create_before_destroy = true  # new SG exists before the old one is removed
  }
}

output "vpc_id" {
  value = aws_vpc.main.id
}

• provider: a plugin that knows one API. required_providers pins its version; .terraform.lock.hcl records the exact one.
• resource: one thing Terraform creates and owns, addressed as type.name. References such as aws_vpc.main.id build the dependency graph, so the VPC is created before the subnet. depends_on is only for dependencies that are real but invisible in the arguments.
• data blocks (data "aws_ami" "al2023" {...}) read something that exists without owning it.
• variable: an input with a type, a description and optional validation. Values come from -var and -var-file, then *.auto.tfvars, then terraform.tfvars, then TF_VAR_name environment variables, then the default, highest precedence first.
• locals name an expression once; output exposes a value to other root modules.
• expressions: "${a}-${b}", cond ? x : y, [for k, v in var.subnets : v], functions such as merge, cidrsubnet, toset.
• for_each vs count: count = 3 creates thing[0..2]. Remove the middle item of the source list and [1] changes and [2] disappears: Terraform recreates resources that only moved position. for_each over a map or set gives each instance a stable key, so removing a touches only a. Use count only for a zero-or-one toggle. for_each keys must be known at plan time and cannot be sensitive.
• lifecycle: create_before_destroy for things that must not have a gap (security groups, launch templates); prevent_destroy makes any plan that would destroy the resource fail; ignore_changes leaves listed attributes to another system (an autoscaler's desired_capacity).
• dynamic blocks generate repeated nested blocks from a collection. The docs warn that overuse hurts readability: use them inside modules, and write nested blocks out literally elsewhere.

State: the file that makes Terraform work

State is a JSON file that maps every resource address in your code to the real object in the cloud. Lose it and the next plan wants to create everything again while the originals keep running and costing money.

1. State is remote. A file on one laptop cannot be shared. On AWS: an S3 bucket with versioning (roll back a bad write), encryption and public access blocked.
2. State is locked. Two applies at once both change the cloud and the second state write overwrites the first. use_lockfile = true makes the S3 backend write a .tflock object next to the state; a second apply fails at once. DynamoDB locking (dynamodb_table) still works but is deprecated. If a run dies holding the lock, terraform force-unlock LOCK_ID removes it, after you confirm nothing is running.
3. State is sensitive. It holds every attribute, including database passwords, in plain text. Restrict bucket reads with IAM, log access, never commit .tfstate. Terraform 1.10 added ephemeral values and 1.11 added write-only arguments (password_wo) that are never written to state; prefer them for new secrets.

terraform state list                                  # every address Terraform owns
terraform state show aws_db_instance.main             # stored attributes of one resource
terraform state rm aws_instance.legacy                # forget it; the real object keeps running
terraform import aws_s3_bucket.logs orders-prod-logs  # adopt an existing object; prefer an import block
terraform plan -refresh-only                          # show drift without planning any change

Drift is the gap between state and the cloud: a port opened in the console, a count changed by an autoscaler. Every plan refreshes first, so drift appears as an update back to what the code says; -refresh-only shows only the drift. A nightly plan with -detailed-exitcode that alerts on exit 2 catches drift early.

Modules

A module is a directory of .tf files with inputs, resources and outputs. The directory you run Terraform in is the root module; it calls child modules with a module block. This is how a platform team hands out a VPC or EKS cluster with company defaults built in.

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 6.0"

  name            = "prod-orders"
  cidr            = "10.0.0.0/16"
  azs             = ["ap-south-1a", "ap-south-1b", "ap-south-1c"]
  private_subnets = ["10.0.0.0/20", "10.0.16.0/20", "10.0.32.0/20"]
  public_subnets  = ["10.0.48.0/24", "10.0.49.0/24", "10.0.50.0/24"]

  enable_nat_gateway   = true
  single_nat_gateway   = false   # one NAT per AZ in prod; true in dev to save money
  enable_dns_hostnames = true

  private_subnet_tags = { "kubernetes.io/role/internal-elb" = "1" }
  public_subnet_tags  = { "kubernetes.io/role/elb" = "1" }
}

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 21.0"

  name                                     = "prod-orders"
  kubernetes_version                       = "1.35"   # a version in standard support
  vpc_id                                   = module.vpc.vpc_id
  subnet_ids                               = module.vpc.private_subnets
  enable_cluster_creator_admin_permissions = true

  eks_managed_node_groups = {
    general = { instance_types = ["m6i.large"], min_size = 2, max_size = 10 }
  }
}

source is a local path, a Git URL with a ref= tag, or a registry address; only registry sources take version. Pin with ~> and read the changelog before a major bump: the eks module renamed inputs between major versions, and a blind upgrade plans a cluster replacement. Companies publish their own modules through a private registry or tagged Git repos, each with a README, typed variables, outputs, examples/ and tests/.

One code base, three environments

Dev, staging and prod share code and differ only in values.

Recommend directories plus shared modules:

infra/
  modules/
    vpc/            # variables.tf, main.tf, outputs.tf, README.md, tests/
    eks/
  envs/
    dev/            # backend key dev/terraform.tfstate, account 1111..., small instances
      main.tf       # module "vpc" { source = "../../modules/vpc" ... }
      dev.tfvars
    prod/           # backend key prod/terraform.tfstate, account 2222...

When prod grows, split it into layers with separate states: network, data, cluster, apps. An IAM change should not refresh 400 network resources, and a mistake in one layer cannot destroy another. Layers read each other's outputs with terraform_remote_state or data sources.

CI for Terraform

Nobody applies to prod from a laptop. On a pull request: fmt -check, validate, a linter, a policy scanner, then plan, posted as a comment so reviewers approve the diff and not only the HCL. On merge: apply the saved plan. Credentials come from OIDC, not stored access keys.

name: terraform-prod
on:
  pull_request:
    paths: ["infra/**"]
  push:
    branches: [main]
    paths: ["infra/**"]

permissions:
  id-token: write        # request the GitHub OIDC token
  contents: read
  pull-requests: write   # post the plan as a comment

jobs:
  plan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: aws-actions/configure-aws-credentials@v6.2.3
        with:
          role-to-assume: arn:aws:iam::222222222222:role/terraform-plan   # read-only
          aws-region: ap-south-1
      - uses: hashicorp/setup-terraform@v4
      - run: terraform fmt -check -recursive
      - run: terraform init -input=false
      - run: terraform validate
      - run: tflint --init && tflint
      - run: checkov -d .
      - id: plan
        run: terraform plan -input=false -no-color -out=tfplan
      - run: terraform show -json tfplan > plan.json && python3 plan_guard.py plan.json
      - uses: actions/github-script@v7     # post the plan as a PR comment
        if: github.event_name == 'pull_request'
        with:
          script: |
            await github.rest.issues.createComment({
              owner: context.repo.owner, repo: context.repo.repo,
              issue_number: context.issue.number,
              body: "```\n" + process.env.PLAN.slice(0, 60000) + "\n```"
            });
        env:
          PLAN: ${{ steps.plan.outputs.stdout }}
      - uses: actions/upload-artifact@v4
        with: { name: tfplan, path: tfplan }

  apply:
    needs: plan
    if: github.event_name == 'push'
    runs-on: ubuntu-latest
    environment: prod    # required reviewers on the environment = manual approval
    steps:
      - uses: actions/checkout@v7
      - uses: aws-actions/configure-aws-credentials@v6.2.3
        with:
          role-to-assume: arn:aws:iam::222222222222:role/terraform-apply
          aws-region: ap-south-1
      - uses: hashicorp/setup-terraform@v4
      - uses: actions/download-artifact@v4
        with: { name: tfplan }
      - run: terraform init -input=false && terraform apply -input=false tfplan

By name: tflint catches invalid instance types and unused declarations; checkov flags an open security group or an unencrypted bucket; Infracost posts the monthly cost difference as a PR comment, so a reviewer sees "+412 USD/month" before approving; Atlantis is a self-hosted server that plans on every PR and applies when someone comments atlantis apply; HCP Terraform (Terraform Cloud until April 2024) adds remote runs, state storage and policy as code. The plan role is read-only plus state access; only the apply role writes.

Testing

terraform test runs .tftest.hcl files. Each run block executes a plan or an apply (the default) and checks assert blocks, each with a condition and an error_message. A command = plan test is a unit test with no cloud, for example asserting that aws_db_instance.main.instance_class == "db.t4g.micro" when environment = "dev"; an apply test creates real resources and destroys them at the end.

Terratest is a Go library that applies a module, checks the cloud with the SDK and destroys it; module authors use it.

The dangerous moments

Secrets in Terraform

In order of preference. First, keep the secret out of Terraform: manage_master_user_password = true lets RDS generate the password into Secrets Manager, and the application reads it at runtime. Second, read it from the secret store with a data source so it never sits in a .tfvars file:

data "aws_secretsmanager_secret_version" "db" {
  secret_id = "prod/orders/db"
}

locals {
  db_password = jsondecode(data.aws_secretsmanager_secret_version.db.secret_string)["password"]
}

Third, mark variables and outputs sensitive = true so plan output is redacted; the value is still in state unless it is ephemeral or write-only. Fourth, lock down the state bucket like a secret store, because it is one.

Terraform with EKS, and where GitOps takes over

The eks module creates the control plane, node groups, the OIDC provider for IRSA, access entries mapping IAM roles to Kubernetes groups, and the add-ons (vpc-cni, coredns, kube-proxy, eks-pod-identity-agent). All of that is long-lived AWS API state with IAM in the middle: Terraform's job.

Inside the cluster, stop. A Deployment managed by the Terraform Kubernetes provider changes only when someone runs apply, drifts when a developer uses kubectl, and ties application releases to the VPC pipeline. The boundary that works: Terraform creates the cluster and one bootstrap Helm release, Argo CD, which then syncs everything else from Git: ingress controller, monitoring, applications. Terraform owns "the cluster exists and can be reached"; GitOps owns "what runs on it". An IAM role an application needs stays in Terraform, and its ARN is passed to the chart through values.

The plan that wants to replace the production database. A pull request changes one line in envs/prod/main.tf: storage_encrypted = true on aws_db_instance.main, because a security audit asked for encryption at rest. The plan comment reads:

  # aws_db_instance.main must be replaced
-/+ resource "aws_db_instance" "main" {
      ~ storage_encrypted = false -> true # forces replacement
        identifier        = "orders-prod"
        instance_class    = "db.r6g.large"
    }
Plan: 1 to add, 0 to change, 1 to destroy.

Step 1: read why. -/+ means destroy then create. Encryption is set at creation time in RDS, so the provider marks the attribute as forcing a new resource. Applying this deletes the 50 GB production database and creates an empty encrypted one. If skip_final_snapshot were true there would not even be a snapshot.

Step 2: the guard rails that should have fired. lifecycle { prevent_destroy = true } makes Terraform refuse this plan with an error. The CI script in this lesson's code example parses terraform show -json tfplan, sees actions: ["delete", "create"] with replace_paths: [["storage_encrypted"]] on a protected address, and exits 3. Either one stops the apply.

Step 3: the right procedure. Encryption of an existing RDS instance is done by snapshot, not in place: take a snapshot (about 50 GB, a few minutes), copy the snapshot with --kms-key-id to get an encrypted copy, restore a new instance from it, switch the application's endpoint during a maintenance window, then bring the new instance under Terraform. In HCL that is a new aws_db_instance with snapshot_identifier pointing at the encrypted copy, and the old one removed with a removed block plus destroy = false until the switch is proven, then destroyed on purpose.

Step 4: the numbers for the write-up. Snapshot copy of 50 GB: assume 10 to 15 minutes. Restore: assume 10 to 20 minutes. Downtime is only the endpoint switch plus the application reconnecting, under 5 minutes if the endpoint is behind a DNS CNAME with a 60-second TTL. Compare with the accidental apply: full data loss back to the last automated backup, typically up to 24 hours of orders, and an RTO of an hour or more to restore. That comparison is the answer the interviewer wants when they ask "plan says it will destroy the database, what do you do".