feat: Filter the directories/files hashed when using commands with patterns, docs and examples fixes by flora-five · Pull Request #720 · terraform-aws-modules/terraform-aws-lambda

flora-five · 2025-12-18T20:21:53Z

Description

This set of changes does the following:

when hashing source paths with commands and patterns, use the patterns to filter what files are hashed
uses os.path.normpath to normalize other paths used as input to the prepare step
fixes some source paths used in README and examples
updates examples to version 6.x of terraform-aws-vpc to fix warnings about deprecated attributes
hashes again the content of package.py
avoids double encoding of source_path strings, with jsencode, when passed from Terraform to python

Motivation and Context

When commands and patterns are used together, the patterns filter is applied when building the archive, but not during the prepare step. Files that are not included in the archive due to the filter are still hashed. If some of those files are generated, with their content depending on the build environment, then, even if the built archives are the same across different build environments, the calculated hashes are not the same and the module wants to create again the archive and redeploy. Making the hashing step take into account the patterns filter fixes #672.

Hashing again the content of package.py restores the logic that was necessary to disable in #66 to fix #63. This time only the content of package.py is hashed, without its path.

The change to normalize other paths used as input with os.path.normpath is for consistency with existing normalization on some input paths.

Breaking Changes

None, but all the hashes will change due to inclusion of package.py into the hashing.

How Has This Been Tested?

I have updated at least one of the examples/* to demonstrate and validate my change(s)
I have tested and validated these changes using one or more of the provided examples/* projects
I have executed pre-commit run -a on my pull request

antonbabenko

Looks pretty good! Thanks for tackling this issue.

Could you please explain how to test it? Compare this one to the current version of the module and explain what is expected/fixed (changed or not changed).

flora-five · 2025-12-20T10:19:08Z

Thank you for the review and feedback!

The fix can be exemplified with this code:

module "package" {
  source = "../../"

  create_function = false
  runtime         = "python3.12"

  source_path = [
    {
      path     = "app"
      commands = [":zip"]
      patterns = ["!ignore.txt"]
    }
  ]
}

and the following steps.

With the current version of the module:

$ mkdir app; echo "# source" > app/app.py; echo "initial" > app/ignore.txt
$ terraform apply -auto-approve
module.package.data.external.archive_prepare[0]: Reading...
module.package.data.external.archive_prepare[0]: Read complete after 0s [id=-]
module.package.data.aws_region.current: Reading...
module.package.data.aws_caller_identity.current: Reading...
module.package.data.aws_partition.current: Reading...
module.package.data.aws_region.current: Read complete after 0s [id=eu-west-1]
module.package.data.aws_partition.current: Read complete after 0s [id=aws]
module.package.data.aws_caller_identity.current: Read complete after 0s [id=...]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with
the following symbols:
  + create

Terraform will perform the following actions:

  # module.package.local_file.archive_plan[0] will be created
  + resource "local_file" "archive_plan" {
      + content              = (sensitive value)
      + content_base64sha256 = (known after apply)
      + content_base64sha512 = (known after apply)
      + content_md5          = (known after apply)
      + content_sha1         = (known after apply)
      + content_sha256       = (known after apply)
      + content_sha512       = (known after apply)
      + directory_permission = "0755"
      + file_permission      = "0644"
      + filename             = "builds/2cf974ee3774e9a68398177febb760466484c417bb82743c568de960cdbf5506.plan.json"
      + id                   = (known after apply)
    }

  # module.package.null_resource.archive[0] will be created
  + resource "null_resource" "archive" {
      + id       = (known after apply)
      + triggers = {
          + "filename"  = "builds/2cf974ee3774e9a68398177febb760466484c417bb82743c568de960cdbf5506.zip"
          + "timestamp" = "1766224363280384000"
        }
    }

Plan: 2 to add, 0 to change, 0 to destroy.
module.package.local_file.archive_plan[0]: Creating...
module.package.local_file.archive_plan[0]: Creation complete after 0s [id=56f0b1fa178948ab0dd6b5924650a2f7a5f0caa7]
module.package.null_resource.archive[0]: Creating...
module.package.null_resource.archive[0]: Provisioning with 'local-exec'...
module.package.null_resource.archive[0] (local-exec): local-exec: Executing: Suppressed by quiet=true
module.package.null_resource.archive[0]: Creation complete after 0s [id=8964199750125432709]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
$ terraform plan
module.package.data.external.archive_prepare[0]: Reading...
module.package.data.external.archive_prepare[0]: Read complete after 0s [id=-]
module.package.local_file.archive_plan[0]: Refreshing state... [id=56f0b1fa178948ab0dd6b5924650a2f7a5f0caa7]
module.package.null_resource.archive[0]: Refreshing state... [id=8964199750125432709]
module.package.data.aws_region.current: Reading...
module.package.data.aws_partition.current: Reading...
module.package.data.aws_caller_identity.current: Reading...
module.package.data.aws_region.current: Read complete after 0s [id=eu-west-1]
module.package.data.aws_partition.current: Read complete after 0s [id=aws]
module.package.data.aws_caller_identity.current: Read complete after 0s [id=...]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are
needed.
$ unzip -l builds/2cf974ee3774e9a68398177febb760466484c417bb82743c568de960cdbf5506.zip 
Archive:  builds/2cf974ee3774e9a68398177febb760466484c417bb82743c568de960cdbf5506.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        9  01-01-1980 00:00   app.py
---------                     -------
        9                     1 file
$ echo "changed" > app/ignore.txt
$ terraform plan
module.package.data.external.archive_prepare[0]: Reading...
module.package.data.external.archive_prepare[0]: Read complete after 0s [id=-]
module.package.local_file.archive_plan[0]: Refreshing state... [id=56f0b1fa178948ab0dd6b5924650a2f7a5f0caa7]
module.package.null_resource.archive[0]: Refreshing state... [id=8964199750125432709]
module.package.data.aws_region.current: Reading...
module.package.data.aws_partition.current: Reading...
module.package.data.aws_region.current: Read complete after 0s [id=eu-west-1]
module.package.data.aws_caller_identity.current: Reading...
module.package.data.aws_partition.current: Read complete after 0s [id=aws]
module.package.data.aws_caller_identity.current: Read complete after 0s [id=...]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with
the following symbols:
+/- create replacement and then destroy

Terraform will perform the following actions:

  # module.package.local_file.archive_plan[0] must be replaced
+/- resource "local_file" "archive_plan" {
      ~ content              = (sensitive value) # forces replacement
      ~ content_base64sha256 = "5kNo8lNSAogubg0U20HitHBcM9MbBglceOHHcEd1HjI=" -> (known after apply)
      ~ content_base64sha512 = "FHbxf3F60tFrn0ILLSvUIX76GI+Jk3FWzak0xpdBwTfqQMelUynzwoUvwG7N/Ztcbul9XMIszbXxKqPqfypRng==" -> (known after apply)
      ~ content_md5          = "4864952ec7248d906d9464be266c481f" -> (known after apply)
      ~ content_sha1         = "56f0b1fa178948ab0dd6b5924650a2f7a5f0caa7" -> (known after apply)
      ~ content_sha256       = "e64368f2535202882e6e0d14db41e2b4705c33d31b06095c78e1c77047751e32" -> (known after apply)
      ~ content_sha512       = "1476f17f717ad2d16b9f420b2d2bd4217efa188f89937156cda934c69741c137ea40c7a55329f3c2852fc06ecdfd9b5c6ee97d5cc22ccdb5f12aa3ea7f2a519e" -> (known after apply)
      ~ filename             = "builds/2cf974ee3774e9a68398177febb760466484c417bb82743c568de960cdbf5506.plan.json" -> "builds/3cc66934d8b579059f506f93c8ff63885ff7b034f444d466da06a24b8850faf2.plan.json" # forces replacement
      ~ id                   = "56f0b1fa178948ab0dd6b5924650a2f7a5f0caa7" -> (known after apply)
        # (2 unchanged attributes hidden)
    }

  # module.package.null_resource.archive[0] must be replaced
+/- resource "null_resource" "archive" {
      ~ id       = "8964199750125432709" -> (known after apply)
      ~ triggers = { # forces replacement
          ~ "filename"  = "builds/2cf974ee3774e9a68398177febb760466484c417bb82743c568de960cdbf5506.zip" -> "builds/3cc66934d8b579059f506f93c8ff63885ff7b034f444d466da06a24b8850faf2.zip"
          ~ "timestamp" = "1766224363280384000" -> "1766224449996733000"
        }
    }

Plan: 2 to add, 0 to change, 2 to destroy.

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if
you run "terraform apply" now.

We can see that ignore.txt is not included in the archive, but changing its content makes the module want to recreate the archive, with a different name, due to the changed hash.

With the proposed set of changes:

$ mkdir app; echo "# source" > app/app.py; echo "initial" > app/ignore.txt
$ terraform apply -auto-approve
module.package.data.external.archive_prepare[0]: Reading...
module.package.data.external.archive_prepare[0]: Read complete after 0s [id=-]
module.package.data.aws_region.current: Reading...
module.package.data.aws_partition.current: Reading...
module.package.data.aws_caller_identity.current: Reading...
module.package.data.aws_region.current: Read complete after 0s [id=eu-west-1]
module.package.data.aws_partition.current: Read complete after 0s [id=aws]
module.package.data.aws_caller_identity.current: Read complete after 0s [id=...]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with
the following symbols:
  + create

Terraform will perform the following actions:

  # module.package.local_file.archive_plan[0] will be created
  + resource "local_file" "archive_plan" {
      + content              = (sensitive value)
      + content_base64sha256 = (known after apply)
      + content_base64sha512 = (known after apply)
      + content_md5          = (known after apply)
      + content_sha1         = (known after apply)
      + content_sha256       = (known after apply)
      + content_sha512       = (known after apply)
      + directory_permission = "0755"
      + file_permission      = "0644"
      + filename             = "builds/c2b94454086393d6981987a835dbd1e60d7ef7ce3290bac4f23e7bd225366a63.plan.json"
      + id                   = (known after apply)
    }

  # module.package.null_resource.archive[0] will be created
  + resource "null_resource" "archive" {
      + id       = (known after apply)
      + triggers = {
          + "filename"  = "builds/c2b94454086393d6981987a835dbd1e60d7ef7ce3290bac4f23e7bd225366a63.zip"
          + "timestamp" = "1766224679244360000"
        }
    }

Plan: 2 to add, 0 to change, 0 to destroy.
module.package.local_file.archive_plan[0]: Creating...
module.package.local_file.archive_plan[0]: Creation complete after 0s [id=a14a3e217713f8311b22691fe555c72d116f39a1]
module.package.null_resource.archive[0]: Creating...
module.package.null_resource.archive[0]: Provisioning with 'local-exec'...
module.package.null_resource.archive[0] (local-exec): local-exec: Executing: Suppressed by quiet=true
module.package.null_resource.archive[0]: Creation complete after 0s [id=6751666028988243283]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
$ terraform plan
module.package.data.external.archive_prepare[0]: Reading...
module.package.data.external.archive_prepare[0]: Read complete after 0s [id=-]
module.package.local_file.archive_plan[0]: Refreshing state... [id=a14a3e217713f8311b22691fe555c72d116f39a1]
module.package.null_resource.archive[0]: Refreshing state... [id=6751666028988243283]
module.package.data.aws_partition.current: Reading...
module.package.data.aws_caller_identity.current: Reading...
module.package.data.aws_region.current: Reading...
module.package.data.aws_partition.current: Read complete after 0s [id=aws]
module.package.data.aws_region.current: Read complete after 0s [id=eu-west-1]
module.package.data.aws_caller_identity.current: Read complete after 0s [id=...]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are
needed.
$ unzip -l builds/c2b94454086393d6981987a835dbd1e60d7ef7ce3290bac4f23e7bd225366a63.zip 
Archive:  builds/c2b94454086393d6981987a835dbd1e60d7ef7ce3290bac4f23e7bd225366a63.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        9  01-01-1980 00:00   app.py
---------                     -------
        9                     1 file
$ echo "changed" > app/ignore.txt
$ terraform plan
module.package.data.external.archive_prepare[0]: Reading...
module.package.data.external.archive_prepare[0]: Read complete after 0s [id=-]
module.package.local_file.archive_plan[0]: Refreshing state... [id=a14a3e217713f8311b22691fe555c72d116f39a1]
module.package.null_resource.archive[0]: Refreshing state... [id=6751666028988243283]
module.package.data.aws_region.current: Reading...
module.package.data.aws_partition.current: Reading...
module.package.data.aws_caller_identity.current: Reading...
module.package.data.aws_region.current: Read complete after 0s [id=eu-west-1]
module.package.data.aws_partition.current: Read complete after 0s [id=aws]
module.package.data.aws_caller_identity.current: Read complete after 0s [id=...]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are
needed.

We can see that after changing the content of ignore.txt, the plan shows no changes.

I would like to ask what do you think about the hash_extra_paths code and logic? Should it be kept even if it is not used or exposed to the users of the module? On one hand it was implemented in a generic way, seeing that it can do expansion of path references:

# Expand a Terraform path.<cwd|root|module> references                                                             
hash_extra_paths = [p.format(path=tf_paths) for p in hash_extra_paths]

On the other hand, for the goal of hashing the content of package.py, without its path, the hash_internal code and logic is simpler.

This ensures that when sharing code between different platforms/environments, this will not generate unneeded changes. Base64 results in different values for the same data across different platforms. Ties in nicely with terraform-aws-modules#720.

github-actions · 2026-01-20T00:27:28Z

This PR has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this PR will be closed in 10 days

flora-five · 2026-01-20T22:39:05Z

I have updated the changes to apply to version 8.2.0. @antonbabenko could you consider this again, please? Thank you!

antonbabenko

Works great, but hash_extra_paths or hash_internal is not needed. Please simplify, and I will merge it right away.

PS: Thank you for this important fix!

antonbabenko · 2026-01-25T13:52:34Z

package.tf

+    runtime          = var.runtime
+    source_path      = try(tostring(var.source_path), jsonencode(var.source_path))
+    hash_extra       = var.hash_extra
+    hash_extra_paths = jsonencode([])


I think hash_extra_paths is not needed anymore, since there is already hash_internal. Let's drop one and we are good to go with this PR.

…the definition

…tions

…ted to be

Previously, the following paths were normalized: the working directory for commands and the path elements when source_path is a list. With this change, the following paths are normalized too: plain paths given as source_path strings, path values of npm_requirements and pip_requirements.

See PR terraform-aws-modules#66 TODO: remove the code and logic for hash_extra_paths?

…ecated attributes with version 6.x of AWS provider

…sh_internal

flora-five · 2026-01-27T12:31:54Z

Sorry for the delay. I have removed the code for hash_extra_paths.

antonbabenko · 2026-02-01T14:40:56Z

@flora-five Please fix the failing tests (I know this is not related to the PR), and we are good to go!

flora-five · 2026-02-02T16:27:57Z

The additional changes update to the latest version of ruff, which fixes one failure.

The other change that makes the tests pass is to remove python 3.8 and 3.9 from the test matrix, because they are EOL.

Other changes add python 3.13 and 3.14 to the test matrix, update to the latest versions of pytest and poetry, fix repeated need to apply package_dir_poetry from examples/build-package.

## [8.5.0](v8.4.0...v8.5.0) (2026-02-02) ### Features * Filter the directories/files hashed when using commands with patterns, docs and examples fixes ([#720](#720)) ([daa5dfc](daa5dfc))

antonbabenko · 2026-02-02T18:09:50Z

This PR is included in version 8.5.0 🎉

flora-five mentioned this pull request Dec 18, 2025

Filename changes and recreates Lambdas on different platforms (CI/CD vs local) #672

Closed

1 task

flora-five force-pushed the fix/hashing branch from 4776bb9 to 784f889 Compare December 18, 2025 20:39

antonbabenko reviewed Dec 19, 2025

View reviewed changes

github-actions bot added the stale label Jan 20, 2026

flora-five force-pushed the fix/hashing branch from 784f889 to 8be2425 Compare January 20, 2026 20:29

flora-five marked this pull request as ready for review January 20, 2026 22:37

github-actions bot removed the stale label Jan 21, 2026

antonbabenko changed the title ~~fix: Filter the directories/files hashed when using commands with patterns, docs and examples fixes~~ feat: Filter the directories/files hashed when using commands with patterns, docs and examples fixes Jan 25, 2026

antonbabenko requested changes Jan 25, 2026

View reviewed changes

flora-five added 9 commits January 27, 2026 11:54

Change order of notes in README to have the patterns examples follow …

a02e7dc

…the definition

Change the paths in the README examples to match the types of applica…

23e3523

…tions

Move the node-app fixture into the examples folder, where it is expec…

e5da5bc

…ted to be

Avoid double encoding of source_path strings

422af67

Include again into the hash the content of package.py, ignoring its path

2f52e90

See PR terraform-aws-modules#66 TODO: remove the code and logic for hash_extra_paths?

Filter the directories/files hashed when using commands with patterns

757bfe8

Update path to app source

a04f63f

Update to version 6.x of terraform-aws-vpc to fix warnings about depr…

32efef0

…ecated attributes with version 6.x of AWS provider

flora-five force-pushed the fix/hashing branch from 048e18a to a45412c Compare January 27, 2026 11:57

Remove hash_extra_paths, similar functionality is implemented with ha…

eb54586

…sh_internal

flora-five force-pushed the fix/hashing branch from a45412c to eb54586 Compare January 27, 2026 12:28

flora-five requested a review from antonbabenko January 27, 2026 12:31

flora-five added 2 commits February 2, 2026 15:45

Update the examples to use Poetry 2, with Python 3.12 as dependency

b410a50

Run Python tests with the supported versions of Python

5f30f34

flora-five added 2 commits February 2, 2026 15:49

Use the latest versions of pytest and ruff

6b0ed33

Don't share the artifacts directory with the docker build

db8bbf7

flora-five force-pushed the fix/hashing branch from bddb8d7 to db8bbf7 Compare February 2, 2026 15:53

antonbabenko approved these changes Feb 2, 2026

View reviewed changes

antonbabenko merged commit daa5dfc into terraform-aws-modules:master Feb 2, 2026
30 checks passed

flora-five deleted the fix/hashing branch February 3, 2026 08:14

antonbabenko mentioned this pull request Feb 3, 2026

Call to function "fileexists" failed: function returned an inconsistent result. #733

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Filter the directories/files hashed when using commands with patterns, docs and examples fixes#720

feat: Filter the directories/files hashed when using commands with patterns, docs and examples fixes#720
antonbabenko merged 14 commits intoterraform-aws-modules:masterfrom
flora-five:fix/hashing

flora-five commented Dec 18, 2025

Uh oh!

antonbabenko left a comment

Uh oh!

flora-five commented Dec 20, 2025

Uh oh!

github-actions bot commented Jan 20, 2026

Uh oh!

flora-five commented Jan 20, 2026

Uh oh!

antonbabenko left a comment

Uh oh!

antonbabenko Jan 25, 2026

Uh oh!

flora-five commented Jan 27, 2026

Uh oh!

antonbabenko commented Feb 1, 2026

Uh oh!

flora-five commented Feb 2, 2026

Uh oh!

Uh oh!

antonbabenko commented Feb 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

flora-five commented Dec 18, 2025

Description

Motivation and Context

Breaking Changes

How Has This Been Tested?

Uh oh!

antonbabenko left a comment

Choose a reason for hiding this comment

Uh oh!

flora-five commented Dec 20, 2025

Uh oh!

github-actions bot commented Jan 20, 2026

Uh oh!

flora-five commented Jan 20, 2026

Uh oh!

antonbabenko left a comment

Choose a reason for hiding this comment

Uh oh!

antonbabenko Jan 25, 2026

Choose a reason for hiding this comment

Uh oh!

flora-five commented Jan 27, 2026

Uh oh!

antonbabenko commented Feb 1, 2026

Uh oh!

flora-five commented Feb 2, 2026

Uh oh!

Uh oh!

antonbabenko commented Feb 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants