Dependencies for cloudinit_post_nodeadm content result in `Invalid count argument` error

[daniel-palmer-gu]

Description

It would seem that cloudinit_post_nodeadm content can not depend on resources that terraform needs to create. Doing so results in a Invalid count argument error.

Below is an example that adds dummy data to the content. It doesn't make sense but proves the point. If there were a real template that used that data, the result should be the same.

• ✋ I have searched the open/closed issues and my issue is not listed.

Versions

• Module version [Required]: 20.35.0

• Terraform version: Terraform v1.5.7

• Provider version(s):

Terraform v1.5.7
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.94.0
+ provider registry.terraform.io/hashicorp/cloudinit v2.3.6
+ provider registry.terraform.io/hashicorp/null v3.2.3
+ provider registry.terraform.io/hashicorp/random v3.7.1
+ provider registry.terraform.io/hashicorp/time v0.13.0
+ provider registry.terraform.io/hashicorp/tls v4.0.6

Reproduction Code [Required]

module "eks_al2023" {
  source  = "terraform-aws-modules/eks/aws"
  version = "= 20.35.0"

  cluster_name    = "testing-al2023"
  cluster_version = "1.31"

  # EKS Addons
  cluster_addons = {
    coredns                = {}
    eks-pod-identity-agent = {}
    kube-proxy             = {}
    vpc-cni                = {}
  }

  # vpc_id     = module.vpc.vpc_id
  # subnet_ids = module.vpc.private_subnets
  vpc_id = "vpc-myvpc"
  subnet_ids = [
    "subnet-mysubnet1",
    "subnet-mysubnet2",
    "subnet-mysubnet3"
  ]

  eks_managed_node_groups = {
    example = {
      # Starting on 1.30, AL2023 is the default AMI type for EKS managed node groups
      instance_types = ["m6i.large"]

      min_size = 1
      max_size = 1
      # This value is ignored after the initial creation
      # https://github.com/bryantbiggs/eks-desired-size-hack
      desired_size = 1

      # This is not required - demonstrates how to pass additional configuration to nodeadm
      # Ref https://awslabs.github.io/amazon-eks-ami/nodeadm/doc/api/
      cloudinit_pre_nodeadm = [
        {
          content_type = "application/node.eks.aws"
          content      = <<-EOT
            ---
            apiVersion: node.eks.aws/v1alpha1
            kind: NodeConfig
            spec:
              kubelet:
                config:
                  shutdownGracePeriod: 30s
                  featureGates:
                    DisableKubeletCloudCredentialProviders: true
          EOT
        }
      ]
    }
  }

  eks_managed_node_group_defaults = {
    enable_bootstrap_user_data = true
    ami_type               = "AL2023_x86_64_STANDARD"
    cloudinit_post_nodeadm = [{
      content_type = "text/x-shellscript; charset=\"us-ascii\""
      content = random_password.my_password.result
    }]
  }
}

resource "random_password" "my_password" {
  length = 32
}

Steps to reproduce the behavior:

1. Create a fresh main.tf file with the contents above
2. Update vpc_id and subnet_ids with ones you have access to
3. terraform init
4. terraform apply (or plan)

Expected behavior

Plan succeeds and can apply the code

Actual behavior

Plan fails with the error below

Terminal Output Screenshot(s)

Text version of error:

│ Error: Invalid count argument
│ 
│   on .terraform/modules/eks_al2023/modules/_user_data/main.tf line 131, in data "cloudinit_config" "al2023_eks_managed_node_group":
│  131:   count = var.create && local.user_data_type == "al2023" && length(local.nodeadm_cloudinit) > 0 ? 1 : 0
│ 
│ The "count" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use the -target
│ argument to first apply only the resources that the count depends on.

Additional context

Removing the dependency on the random_password and replacing it's contents with "mypassword" allows the plan to succeed.

[daniel-palmer-gu]

Additional additional context:

Due to the nature of the error, this really only matters on the initial deploy of the resources. If the random_password has been created in a previous apply, this error does not present.

I recently started moving over to AL2023, and noticed this issue. When using post_bootstrap_user_data for non-AL2023 AMIs, this error doesn't occur likely due to the difference in the way the user data is prepared.

[bryantbiggs]

this isn't quite valid - cloudinit_post_nodeadm (AL2023) and post_bootstrap_user_data (AL2) are not used when using EKS optimized AMIs within EKS managed node groups

[daniel-palmer-gu]

I'm not sure I am tracking. The output above would indicate that it is trying to use it - no?

Can you provide more information on what is not valid about the example?

[bryantbiggs]

https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html#launch-template-user-data

you can only specify user data that is added *ahead of what EKS MNG adds.

in the case of AL2, the user data supplied is added *before the bootstrap.sh script line is injected by EKS MNG

in the case of AL2023, the user data supplied is merged *before the nodeadm cloud init block added by EKS MNG

therefore, what you posted above and what I've modified below, they result in the same outcome in terms of resources/configuration created:

module "eks_al2023" {
  source  = "terraform-aws-modules/eks/aws"
  version = "= 20.35.0"

  cluster_name    = "testing-al2023"
  cluster_version = "1.31"

  # EKS Addons
  cluster_addons = {
    coredns                = {}
    eks-pod-identity-agent = {}
    kube-proxy             = {}
    vpc-cni                = {}
  }

  # vpc_id     = module.vpc.vpc_id
  # subnet_ids = module.vpc.private_subnets
  vpc_id = "vpc-myvpc"
  subnet_ids = [
    "subnet-mysubnet1",
    "subnet-mysubnet2",
    "subnet-mysubnet3"
  ]

  eks_managed_node_groups = {
    example = {
      # Starting on 1.30, AL2023 is the default AMI type for EKS managed node groups
      instance_types = ["m6i.large"]

      min_size = 1
      max_size = 1
      # This value is ignored after the initial creation
      # https://github.com/bryantbiggs/eks-desired-size-hack
      desired_size = 1

      # This is not required - demonstrates how to pass additional configuration to nodeadm
      # Ref https://awslabs.github.io/amazon-eks-ami/nodeadm/doc/api/
      cloudinit_pre_nodeadm = [
        {
          content_type = "application/node.eks.aws"
          content      = <<-EOT
            ---
            apiVersion: node.eks.aws/v1alpha1
            kind: NodeConfig
            spec:
              kubelet:
                config:
                  shutdownGracePeriod: 30s
                  featureGates:
                    DisableKubeletCloudCredentialProviders: true
          EOT
        }
      ]
    }
  }
}

[daniel-palmer-gu]

Two things I am noticing:

1. In my actual code base, I use the post section to configure some containerd stuff. Theres no reason I couldn't move this code to cloudinit_pre_nodeadm. However, I still get the same error. (see example below)
2. Are you sure that is accurate for the AL2023 case? It looks like user data can add in content after by using cloudinit_post_nodeadm. When I run this and stand up the instance after the password already exists, it shows up as part-003 in the cloud init folder on the instance.

module "eks_al2023" {
  source  = "terraform-aws-modules/eks/aws"
  version = "= 20.35.0"

  cluster_name    = "testing-al2023"
  cluster_version = "1.31"

  # EKS Addons
  cluster_addons = {
    coredns                = {}
    eks-pod-identity-agent = {}
    kube-proxy             = {}
    vpc-cni                = {}
  }

  # vpc_id     = module.vpc.vpc_id
  # subnet_ids = module.vpc.private_subnets
  vpc_id = "vpc-myvpc"
  subnet_ids = [
    "subnet-mysubnet1",
    "subnet-mysubnet2",
    "subnet-mysubnet3"
  ]

  eks_managed_node_groups = {
    example = {
      # Starting on 1.30, AL2023 is the default AMI type for EKS managed node groups
      instance_types = ["m6i.large"]

      min_size = 1
      max_size = 1
      # This value is ignored after the initial creation
      # https://github.com/bryantbiggs/eks-desired-size-hack
      desired_size = 1
    }
  }

  eks_managed_node_group_defaults = {
    enable_bootstrap_user_data = true
    ami_type               = "AL2023_x86_64_STANDARD"
    cloudinit_pre_nodeadm = [{
      content_type = "text/x-shellscript; charset=\"us-ascii\""
      content = random_password.my_password
    }]
  }
}

resource "random_password" "my_password" {
  length           = 32
}

[github-actions]

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

[daniel-palmer-gu]

This is still a valid issue

[github-actions]

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

[daniel-palmer-gu]

This is still a valid issue