Skip to content

Workload Identity Federation GCP <-> AWS failing #2139

@benkoska

Description

@benkoska

Please make sure you have searched for information in the following guides.

A screenshot that you have tested with "Try this API".

The intended behavior does not directly use a Google API, but is a combination of Google Workload Federation and the AWS counterpart that is used by this package to support Workload Identity Federation.

If you want to test the API call manually, you can run the following commands on the configured EC2 instance (after following the tutorial linked in the gist)

TOKEN=$(curl -sS -m 2 -X PUT "http://169.254.169.254/latest/api/token" \
  -H "X-aws-ec2-metadata-token-ttl-seconds: 21600") || echo "token-failed:$?"
ROLE=$(curl -sS -m 2 -H "X-aws-ec2-metadata-token: $TOKEN" \
  "http://169.254.169.254/latest/meta-data/iam/security-credentials/") || echo "role-failed:$?"
curl --verbose -sS -m 2 -H "X-aws-ec2-metadata-token: $TOKEN"   "http://169.254.169.254/latest/meta-data/iam/security-credentials/$ROLE"

which will correctly return a token, accessKeyID and secretAccessKey as so

> GET /latest/meta-data/iam/security-credentials/<ROLE> HTTP/1.1
> Host: 169.254.169.254
> User-Agent: curl/8.11.1
> Accept: */*
> X-aws-ec2-metadata-token: [....]
> 
* Request completely sent off
< HTTP/1.1 200 OK
< X-Aws-Ec2-Metadata-Token-Ttl-Seconds: 21391
< Content-Type: text/plain
< Accept-Ranges: none
< Last-Modified: Tue, 30 Sep 2025 00:02:53 GMT
< Content-Length: [...]
< Date: Tue, 30 Sep 2025 00:14:07 GMT
< Server: EC2ws
< Connection: close
< 
{
  "Code" : "Success",
  "LastUpdated" : "2025-09-30T00:02:01Z",
  "Type" : "AWS-HMAC",
  "AccessKeyId" : "[....]",
  "SecretAccessKey" : "[...]",
  "Token" : "[...]",
  "Expiration" : "2025-09-30T06:16:41Z"
}

Notably though, one notices that the response from the AWS server includes a Content-Type: text/plain header.

Link to the code that reproduces this issue. A link to a public Github Repository or gist with a minimal reproduction.

https://gist.github.com/benkoska/ec03dae2f37eb7ae051215c188fbf4d8

A step-by-step description of how to reproduce the issue, based on the linked reproduction.

  1. Follow the linked tutorial to configure Workload Identity Federation
  2. Run the given script from an AWS EC2 instance that is configured for Workload Identity Federation
  3. The script will fail and throw an error

A clear and concise description of what the bug is, and what you expected to happen.

The process fails to authenticate and receive the token necessary to run operations in the project (in the example, list all instances in a given zone).

If the same credentials file is used with the gcloud CLI, you are able to authenticate and utilize Workload Identity Federation as intended by executing commands in the project.

A clear and concise description WHY you expect this behavior, i.e., was it a recent change, there is documentation that points to this behavior, etc. **

It is the intended behavior as described in the linked tutorial on Workload Identity Federation and if you run the equivalent gcloud CLI commands, it works as intended.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions