[AuthenticationError] pipeline-deploy fails with "Invalid character '#' in entity name: "#xD"" in STS XML response

[douglaslimaamazon]

Environment information

- @aws-amplify/backend-cli: 1.7.2 (also reproduced with 1.8.0 and 1.8.2)
- @aws-cdk/toolkit-lib: 1.15.0 (via 1.7.2) and 1.16.0 (via 1.8.x)
- Node.js: 20.x (Amplify CodeBuild default)
- Region: us-east-1
- Amplify App ID: xxxxxxxxxxx
- Build image: Standard (8GiB Memory, 4vCPUs)
Binaries:
  Node: 23.11.0
  Yarn: undefined - undefined
  npm: 11.3.0 
  pnpm: undefined - undefined
NPM Packages:
  @aws-amplify/auth-construct: 1.11.2
  @aws-amplify/backend: 1.21.1
  @aws-amplify/backend-ai: Not Found
  @aws-amplify/backend-auth: 1.9.2
  @aws-amplify/backend-cli: 1.8.2
  @aws-amplify/backend-data: 1.6.4
  @aws-amplify/backend-deployer: 2.1.6
  @aws-amplify/backend-function: 1.17.0
  @aws-amplify/backend-output-schemas: 1.8.0
  @aws-amplify/backend-output-storage: 1.3.4
  @aws-amplify/backend-secret: 1.4.2
  @aws-amplify/backend-storage: 1.4.3
  @aws-amplify/cli-core: 2.2.4
  @aws-amplify/client-config: 1.10.1
  @aws-amplify/data-construct: 1.17.0
  @aws-amplify/data-schema: 1.25.3
  @aws-amplify/deployed-backend-client: 1.8.1
  @aws-amplify/form-generator: 1.2.6
  @aws-amplify/model-generator: 1.2.2
  @aws-amplify/platform-core: 1.11.0
  @aws-amplify/plugin-types: 1.12.0
  @aws-amplify/sandbox: 2.2.0
  @aws-amplify/schema-generator: 1.4.1
  @aws-cdk/toolkit-lib: 1.16.0
  aws-amplify: 6.16.3
  aws-cdk-lib: 2.244.0

Describe the bug

npx ampx pipeline-deploy fails during the deploy phase with AuthenticationError: Unable to resolve AWS account to use. The debug output shows the root cause:

Looking up default account ID from STS
AWS SDK Call STSClient: GetCallerIdentityCommand
Unable to determine the default AWS account (Error): [EntityReplacer] Invalid character '#' in entity name: "#xD"
Deserialization error: to see the raw response, inspect the hidden field {error}.$response on this object.
The GetCallerIdentity STS call returns an XML response containing 
 (carriage return entity) that fast-xml-parser's EntitiesParser cannot handle, throwing Invalid character '#' in entity name: "#xD".

Key observations:

• The synth phase succeeds (CDK_DEFAULT_REGION is set, backend synthesizes in ~19s, type checks pass)
• The deploy phase fails because SdkProvider.resolveEnvironment calls STS again independently and the XML parsing fails
• Setting CDK_DEFAULT_ACCOUNT env var does NOT help — the deploy phase ignores it and calls STS internally
• The same code deploys successfully via npx ampx sandbox locally
• The same code deployed successfully on April 15, 2026 — the issue started on April 17 without any code changes
• Clean installs with rm -rf node_modules package-lock.json and pinning @aws-amplify/backend-cli to 1.7.2 (pre-1.8.x) still reproduce the issue
• The bug appears to be in the Amplify/CodeBuild build environment's STS response, not in any specific toolkit-lib version

Reproduction steps

1. Create an Amplify Gen 2 app with pipeline-deploy in amplify.yml
2. Deploy via CodeCommit trigger to Amplify Hosting (us-east-1)
3. Build spec:

version: 1
backend:
  phases:
    preBuild:
      commands:
        - npm install
    build:
      commands:
        - npx ampx pipeline-deploy --branch $AWS_BRANCH --app-id $AWS_APP_ID

4. Build fails with [AuthenticationError] Unable to resolve AWS account to use
5. Adding --debug flag reveals the XML parsing error in the STS response
   Expected behavior: pipeline-deploy should successfully resolve the AWS account and deploy the CloudFormation stack.

Additional context:

• The aws sts get-caller-identity CLI command works fine in the same build environment (used to set CDK_DEFAULT_ACCOUNT)
• The issue is specifically in the AWS SDK JS XML deserializer used by @aws-cdk/toolkit-lib when parsing the STS XML response
• This affects all recent versions of @aws-amplify/backend-cli (tested 1.7.2, 1.8.0, 1.8.2)
• Last successful deploy was April 15, 2026 using cached node_modules from a previous build

[tylerlan-rhizome]

I am also experiencing this issue. It seems to be related to #3135

I tried pinning the amplify backend deps,

  "devDependencies": {
    "@aws-amplify/backend": "1.15.0",
    "@aws-amplify/backend-cli": "1.4.4",

In my amplify.yml, I've set the node version to a recent LTS and I'm running the recommended commands. It's only when the execution reaches npx ampx pipeline-deploy that the error occurs

version: 1
backend:
  phases:
    build:
      commands:
        # Set node version
        - nvm use 22.22.0 || nvm install 22.22.0
        # Install dependencies
        - npm ci --cache .npm --prefer-offline
        # Run amplify backend deployment pipeline
        - npx ampx pipeline-deploy --branch $AWS_BRANCH --app-id $AWS_APP_ID

The error:

Error: [EntityReplacer] Invalid character '#' in entity name: "#xD"
Deserialization error: to see the raw response, inspect the hidden field {error}.$response on this object.

[douglaslimaamazon]

Workaround confirmed: Downgrading to @aws-amplify/backend-cli@1.4.4 (which uses the classic CDK CLI binary instead of @aws-cdk/toolkit-lib) resolves the credential resolution failure in the Amplify Hosting build environment (AL2023, build image 2025-09).

In amplify.yml:

• 'npx --package=@aws-amplify/backend-cli@1.4.4 ampx pipeline-deploy --branch $AWS_BRANCH --app-id $AWS_APP_ID'
  Root cause: @aws-cdk/toolkit-lib@1.16.0 (introduced in @aws-amplify/backend-deployer@2.1.6) fails to resolve credentials from the Amplify Hosting container credential chain. The classic CDK CLI binary used by 1.4.4 handles it correctly. This regression appeared after the Amplify build image updated to Node 22 / image version 2025-09 around mid-April 2026.

Environment: @aws-amplify/backend-cli@1.7.2, @aws-amplify/backend-deployer@2.1.6, @aws-cdk/toolkit-lib@1.16.0, Node v22.18.0, AL2023 Standard/Large compute.

[tylerlan-rhizome]

@douglaslimaamazon I have not seen any improvement yet,

  "dependencies": {
     ...
    "@aws-amplify/adapter-nextjs": "^1.7.2",
    "@aws-amplify/ui-react": "^6.15.3",
    "aws-amplify": "6.16.3",
  },
  "devDependencies": {
     ...
    "@aws-amplify/backend": "1.21.1",
    "@aws-amplify/backend-cli": "1.4.4",
    "aws-cdk-lib": "2.244.0",
  },

[tylerlan-rhizome]

I still see:

Error: [EntityReplacer] Invalid character '#' in entity name: "#xD"`
Deserialization error: to see the raw response, inspect the hidden field {error}.$response on this object.

, and my build fails.

[douglaslimaamazon]

@tylerlan-rhizome The @aws-amplify/backend-cli@1.4.4 workaround specifically addresses the [AuthenticationError] Unable to resolve AWS account error that occurs during the deploy phase (after synthesis succeeds). It bypasses @aws-cdk/toolkit-lib which has a broken credential resolution path.

Your error ([EntityReplacer] Invalid character '#' in entity name: "#xD") is a different issue — it's fast-xml-parser failing to parse an STS XML response containing &#xD; (carriage return entity). This happens earlier in the credential chain.

For your case, try adding this to your build spec before ampx pipeline-deploy:

- |
  for f in $(find node_modules -name "fxp.cjs" -path "*/fast-xml-parser/*"); do
    sed -i "s/throw new Error(\`\[EntityReplacer\]/\/\/throw new Error(\`[EntityReplacer]/g" "$f"
  done

Or patch the source files:

- |
  for f in $(find node_modules -name "EntitiesParser.js" -path "*/fast-xml-parser/*"); do
    sed -i 's/throw new Error/\/\/throw new Error/' "$f"
  done

This comments out the entity validation that rejects # in entity names. The &#xD; is a valid XML numeric character reference that fast-xml-parser incorrectly treats as an invalid entity name.

Alternatively, ensure your package.json has "overrides": { "fast-xml-parser": ">=5.5.6" } — version 5.6.0 handles &#xD; correctly via its num_hex regex, but bundled copies inside @aws-amplify/data-construct and @aws-amplify/graphql-api-construct may still use the vulnerable 4.x version.

[tylerlan-rhizome]

It's funny you should say that since I've already had { "fast-xml-parser": ">=5.5.7" } in my overrides since that vuln was discovered last month, GHSA-8gc5-j5rx-235r

But, I will try upping it to 5.6.0

[tylerlan-rhizome]

Still seeing the same error. I tried the first script you recommended and now I'm trying the other one,

version: 1
backend:
  phases:
    build:
      commands:
        # Set node version
        - nvm use 22.22.0 || nvm install 22.22.0
        # Install dependencies
        - npm ci --cache .npm --prefer-offline
        # Comment out the entity validation that rejects # in entity names
        - |
          for f in $(find node_modules -name "EntitiesParser.js" -path "*/fast-xml-parser/*"); do
            sed -i 's/throw new Error/\/\/throw new Error/' "$f"
          done
        # Run amplify backend deployment pipeline
        - npx --package=@aws-amplify/backend-cli@1.4.4 ampx pipeline-deploy --branch $AWS_BRANCH --app-id $AWS_APP_ID --debug

I tried changing the override to >=5.6.0 but that didn't appear to work either.

{
  "dependencies": {
    ...
    "@aws-amplify/adapter-nextjs": "^1.7.2",
    "@aws-amplify/ui-react": "^6.15.3",
    "aws-amplify": "6.16.3",
  },
  "devDependencies": {
    ...
    "@aws-amplify/backend": "1.22.0",
    "@aws-amplify/backend-cli": "1.4.4",
  },
  "engines": {
    "node": ">=22.22.0"
  },
  "overrides": {
    "handlebars": ">=4.7.9",
    "fast-xml-parser": ">=5.5.6",
    "lodash": ">=4.18.1",
    "@smithy/config-resolver": ">=4.4.0"
  }
}

I am seeing this build error in Amplify now:

SyntaxError: Unexpected end of input
                                    at wrapSafe (node:internal/modules/cjs/loader:1638:18)
                                    at Module._compile (node:internal/modules/cjs/loader:1680:20)
                                    at Object..js (node:internal/modules/cjs/loader:1839:10)
                                    at Module.load (node:internal/modules/cjs/loader:1441:32)
                                    at Function._load (node:internal/modules/cjs/loader:1263:12)
                                    at TracingChannel.traceSync (node:diagnostics_channel:328:14)
                                    at wrapModuleLoad (node:internal/modules/cjs/loader:237:24)
                                    at Module.require (node:internal/modules/cjs/loader:1463:12)
                                    at require (node:internal/modules/helpers:147:16)
                                    at Object.<anonymous> (/codebuild/output/src2577370208/src/rhizome-app/node_modules/@aws-sdk/xml-builder/dist-cjs/xml-parser.js:4:27)
                                    Node.js v22.22.0
[ERROR]: !!! Build failed

[shivennn]

Hi @tylerlan-rhizome I was able to reproduce both the errors at my end with sample app.

I was able to deploy the application successfully using below amplify.yml config.

I have tested the deployment with "@aws-amplify/backend-cli": "1.8.2" and "@aws-amplify/backend-cli": "1.4.4"

version: 1
backend:
  phases:
    build:
      commands:
        # Set node version
        - nvm use 22.22.0 || nvm install 22.22.0
        # Install dependencies
        - npm install
        # Patch all fast-xml-parser fxp.cjs files that throw on 
 in entity names.
        # Replace: if("#"===t[0])throw new Error(...)
        # With:    if("#"===t[0])return t;if(false)throw new Error(...)
        # Using if(false) avoids // comments which break single-line minified files.
        - |
          for fxp in $(find node_modules -name "fxp.cjs" -path "*/fast-xml-parser/*"); do
            sed -i 's/if("#"===t\[0\])throw new Error/if("#"===t[0])return t;if(false)throw new Error/g' "$fxp" && echo "Patched $fxp"
          done
        # Also patch @nodable/entities source (used by non-CJS paths)
        - |
          for f in $(find node_modules -name "EntityDecoder.js" -path "*/@nodable/entities/*"); do
            sed -i 's/throw new Error(`\[EntityReplacer\] Invalid character .#./return name;if(false)throw new Error(`[EntityReplacer] Invalid character x/g' "$f" && echo "Patched $f"
          done
        # Run amplify backend deployment pipeline
        - npx ampx pipeline-deploy --branch $AWS_BRANCH --app-id $AWS_APP_ID
frontend:
  phases:
    preBuild:
      commands:
        - npm install
    build:
      commands:
        - npm run build
  artifacts:
    baseDirectory: .next
    files:
      - '**/*'
  cache:
    paths:
      - .next/cache/**/*
      - .npm/**/*

package.json

{
  "name": "amplify-deployment-issue-17-apr",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@aws-amplify/adapter-nextjs": "^1.7.2",
    "@aws-amplify/ui-react": "^6.15.3",
    "aws-amplify": "6.16.3",
    "next": "14.2.35",
    "react": "^18.3.1",
    "react-dom": "^18.3.1",
    "zod": "3.25.17"
  },
  "devDependencies": {
    "@aws-amplify/backend": "1.22.0",
    "@aws-amplify/backend-cli": "1.8.2",
    "@types/node": "20.19.39",
    "@types/react": "18.3.28",
    "@types/react-dom": "^18.2",
    "aws-cdk-lib": "2.244.0",
    "constructs": "^10.6.0",
    "esbuild": "^0.28.0",
    "tsx": "^4.21.0",
    "typescript": "^5.9.3"
  },
  "engines": {
    "node": ">=22.22.0"
  },
  "overrides": {
    "handlebars": ">=4.7.9",
    "fast-xml-parser": ">=5.6.0",
    "lodash": ">=4.18.1",
    "@smithy/config-resolver": ">=4.4.0"
  }
}

Could you please test the deployment on your end using the amplify.yml configuration shared above and let us know if you're able to move past the issue ?