Large module init causes not all releases to be made

[sorliem]

Hello - first, thanks for making this!

When I went to init all my existing 33 modules at v1.0.0, I hit an issue where only 30 Github Releases were created. My PR had 33 files changed to trigger terraform-modules-releaser on each one, but in the "Finding all Terraform modules with corresponding changes" step, it said:

Parsing commit 121524819f8461485f52a0ee9f0e4bace6663c7b: init modules at v1.0.0 (Changed Files = 30)

Looking at the Github api it looks like 30 is the default: https://docs.github.com/en/rest/pulls/pulls?apiVersion=2022-11-28#list-pull-requests-files

This is a bit of an edge case but paging would be helpful here. If I get some time I can take a crack at a fix.

[virgofx]

Hey @sorliem - Thank you for catching this and for the kind words about the project!

While I thought I had pagination implemented for most API calls, I clearly missed this one for pull request files.

As a temporary workaround: any subsequent commit should trigger the releaser on the remaining 3 modules that weren't processed in the first run. Could you let me know if this works for your case?

I'll prioritize adding proper pagination for the PR files endpoint to handle repositories with >30 module changes. I'll ping you once I've released a new version with the fix.

Thanks again for bringing this to our attention - these are exactly the kind of edge cases we need to catch to make the tool more robust! 🙌

[sorliem]

Hi @virgofx - Yep I created another PR with the remaining three modules and it ran fine.

Thanks again!

[virgofx]

Great news! Glad you're all set with the current workaround. This is definitely valid edge case and I'll get this fixed soon time permitting.

[virgofx]

Hi @sorliem - I just did a couple tests and it appears that everything is working correctly and I do see pagination in the code.

Example of draft test: #159

I know it's in the past but I'm wondering if you have any other logs or able to add me with view permission on the repo to see the run/job.

[sorliem]

Hi @virgofx - here's the config for the run that init'd 30 of the 33 modules

2025-01-09T14:47:50.2722794Z ##[group]Run techpivot/terraform-module-releaser@v1
2025-01-09T14:47:50.2723446Z with:
2025-01-09T14:47:50.2723853Z   major-keywords: major update,breaking change
2025-01-09T14:47:50.2724423Z   minor-keywords: feat,feature
2025-01-09T14:47:50.2724893Z   patch-keywords: fix,chore,docs
2025-01-09T14:47:50.2725355Z   default-first-tag: v1.0.0
2025-01-09T14:47:50.2725813Z   terraform-docs-version: v0.19.0
2025-01-09T14:47:50.2726286Z   delete-legacy-tags: false
2025-01-09T14:47:50.2726715Z   disable-wiki: false
2025-01-09T14:47:50.2727138Z   wiki-sidebar-changelog-max: 10
2025-01-09T14:47:50.2727983Z   module-change-exclude-patterns: .gitignore,*.md,*.tftest.hcl,tests/**
2025-01-09T14:47:50.2728827Z   module-asset-exclude-patterns: .gitignore,*.md,*.tftest.hcl,tests/**
2025-01-09T14:47:50.2729487Z   disable-branding: false
2025-01-09T14:47:50.2730118Z   github_token: ***

Other than that I don't have any other relevant logs. Glad to see the test working, that looks right to me!

[virgofx]

Hi @sorliem - Do you know if it showed all 33 modules correctly in the PR comment prior to merging? (or was it 30?). Was the issue that it actually only released 30 of 33 (e.g. only 30 tags/releases) ?

[sorliem]

It showed 30 modules in the github comment 'Release Plan' table and only created 30 releases. I made a follow up PR touching the remaining 3 modules and it created a Release Plan with 3 releases and created 3 Github Releases.