Vault 2.0.1 UBI image cannot start in rootless environments due to IPC_LOCK requirement

[andy66k]

Description

Upgrading from Vault 1.21.4 to Vault 2.0.1 (official container image based on UBI / Red Hat registry variant) causes the container to fail to start in rootless Podman environments.

The Vault binary (/usr/bin/vault) is shipped with the file capability cap_ipc_lock=ep. In rootless container environments, this capability cannot be applied or used, resulting in the kernel rejecting execution of the binary with Operation not permitted.

As a result, Vault does not start at all and never reaches configuration parsing. Therefore, disable_mlock (or any Vault configuration) has no effect.

This effectively breaks rootless deployments that worked with Vault 1.21.x.

---

To Reproduce

Steps to reproduce the behavior:

• Pull Vault 2.0.1 UBI-based container image:
  podman pull registry.connect.redhat.com/vault:2.0.1-ubi

• Start an interactive shell in a rootless Podman environment:
  podman run --rm -it --entrypoint /bin/sh

• Verify file capabilities of Vault binary:
  getcap /usr/bin/vault

• Attempt to execute Vault binary:
  /usr/bin/vault version

• Observe error:
  sh: /usr/bin/vault: Operation not permitted

• Running the default container also fails:
  podman run

Error:
Vault requires the IPC_LOCK capability.
Please use --cap-add IPC_LOCK or add it to the securityContext capabilities
/usr/bin/vault: Operation not permitted2

Expected behavior

Vault binary should start successfully in rootless container environments, optionally falling back to disable_mlock=true when IPC_LOCK capability is not available.

At minimum, the binary should not fail at execution time due to a required file capability in environments where that capability cannot be granted.

Actual behavior

• /usr/bin/vault cannot be executed at all in rootless Podman.

• Execution fails with:
  Operation not permitted

• Vault never starts, so configuration is never loaded.

• disable_mlock is never evaluated.

• Rootless deployments that worked in Vault 1.21.4 are broken in Vault 2.0.1.

Environment:

• Vault version: 2.0.1
• Previous working version: 1.21.4
• Container runtime: Podman (rootless)
• Base image: registry.connect.redhat.com/vault:2.0.1-ubi
• OS: Linux (RHEL-compatible environment)
• User namespace: enabled (rootless container execution)

Additional context

The Vault 2.0.1 container image ships /usr/bin/vault with the file capability:
cap_ipc_lock=ep

In rootless container environments, this prevents execution of the binary at the kernel level with:
Operation not permitted

This occurs before Vault can read configuration or apply disable_mlock.

As a result, rootless deployments that worked with Vault 1.21.4 are no longer functional with Vault 2.0.1 using the official container image, even when disable_mlock=true is configured.

This appears to be a regression in container compatibility for rootless runtimes.

[dosubot]

Hi @andy66k! I'm Dosu and I'm helping the HashiCorp, an IBM Company team.

This is a known-active issue that's already being tracked. The behavior you're describing matches what's reported in #31970 and #31919 — Vault 2.0.1 moved setcap cap_ipc_lock=+ep from the runtime entrypoint to a Dockerfile RUN step (commit 8b0f0b1), so the capability is now baked into the binary at build time [1]. Any runtime that cannot grant IPC_LOCK to the process (rootless Podman, AWS Fargate, Bottlerocket EKS nodes, etc.) hits EPERM on execve() before Vault ever loads configuration — making disable_mlock irrelevant [2].

Engineering and product security are actively working on a resolution [2]. No fixed image has been released yet for environments that cannot provision IPC_LOCK.

Available workarounds in the meantime:

1. Use the binary from releases.hashicorp.com instead of the container image — the zip distribution does not carry file capabilities [1]:

   ARG TARGETARCH
   ARG VAULT_VERSION=2.0.1
   RUN curl -sL -o /tmp/vault.zip \
         https://releases.hashicorp.com/vault/${VAULT_VERSION}/vault_${VAULT_VERSION}_linux_${TARGETARCH}.zip \
    && unzip -q /tmp/vault.zip -d /usr/local/bin/ \
    && rm /tmp/vault.zip

2. Pin to Vault 1.21.4 (or another 1.x image) with disable_mlock = true configured until a fix ships [1].

3. Strip the capability from the binary in a custom image layer:

   FROM registry.connect.redhat.com/vault:2.0.1-ubi
   USER root
   RUN setcap -r /usr/bin/vault
   USER vault

   This removes the file capability so execution succeeds in rootless mode, but you must set disable_mlock = true in your Vault config since mlock won't be possible.

Since this duplicates #31970 (which is re-opened and actively being addressed), you may want to subscribe there for updates on the fix timeline.

Written by @dosu, approved by heatherezell

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.