Kaniko Self-Build: Deferred for External Solution

[kingdonb]

Kaniko Self-Build: Deferred in Favor of External Images

Decision Summary

Adopted External Solution: Using martizih/kaniko:v1.26.0-debug instead of building Kaniko ourselves.

Priority Shift: Focus on proving multi-arch pipeline functionality rather than Kaniko self-hosting.

Technical Background

The BuildKit Incompatibility Issue

Kaniko cannot build images that use advanced BuildKit features, including itself:

# From Kaniko's deploy/Dockerfile - incompatible with Kaniko
FROM scratch AS kaniko-base-slim
RUN --mount=from=busybox,dst=/usr/ ["busybox", "sh", "-c", "mkdir -p /kaniko && chmod 777 /kaniko"]

This creates a bootstrapping problem:

• ❌ Kaniko can't build Kaniko due to RUN --mount=from= usage
• ❌ Manual BuildKit builds add complexity and maintenance overhead
• ❌ Self-hosting Kaniko provides limited value vs. external maintenance

External Solution Benefits

• ✅ Maintained by community: martizih/kaniko:v1.26.0-debug is actively maintained
• ✅ Multi-arch ready: Available for both amd64 and arm64
• ✅ Zero maintenance overhead: No custom build processes to maintain
• ✅ Faster development: Focus on actual business logic, not tooling self-hosting

Implementation Changes

Files Modified

# Disabled self-build configuration
kaniko/build-config.yaml → kaniko/disabled-build-config.yaml

# Updated pipeline to use external image
.gitlab-ci.yml: gcr.io/kaniko-project/executor:latest → martizih/kaniko:v1.26.0-debug

Pipeline Impact

• No functional changes to pipeline logic
• Better reliability with maintained external image
• Removes BuildKit complexity from our stack

Strategic Rationale

Why External Dependencies Are Better

1. Open Source Ecosystem: Leverage community maintenance like everywhere else
2. Specialization: Let Kaniko experts handle Kaniko builds
3. Resource Focus: Spend time on unique business value, not tooling re-implementation
4. Risk Reduction: Maintained images get security updates and bug fixes

Precedent Everywhere

• We don't build our own Linux distributions
• We don't build our own container runtimes
• We don't build our own CI runners
• We don't need to build our own Kaniko

Future Considerations

If We Ever Need Self-Hosting

1. Hybrid approach ready: Issue Hybrid Build Strategy: Manual BuildKit + Kaniko Pipeline #5 documented manual BuildKit process
2. Community alternatives: Multiple Kaniko maintainers available
3. BuildKit migration: Could migrate entire pipeline if needed

Current Priority: Prove Pipeline Works

🎯 Tonight's Goal: Demonstrate that our multi-arch pipeline can build any Dockerfile using external Kaniko

Success Metrics:

• ✅ Multi-arch builds work with martizih/kaniko:v1.26.0-debug
• ✅ Manifest creation works end-to-end
• ✅ Pipeline handles both simple and complex Dockerfiles
• ✅ Architecture selection works automatically

Status

• 🔴 Kaniko self-build: Deferred (low business value)
• 🟢 Multi-arch pipeline: Active focus
• 🟢 External Kaniko adoption: Complete

Bottom Line: We're in the business of building applications, not rebuilding every tool in our stack. Using maintained external images is the right architectural decision.