Skip to content

Test TLS termination, SSL-redirect, Canary, and CORS#330

Open
Stevenjin8 wants to merge 34 commits intokubernetes-sigs:mainfrom
Stevenjin8:tests/e2e-tls
Open

Test TLS termination, SSL-redirect, Canary, and CORS#330
Stevenjin8 wants to merge 34 commits intokubernetes-sigs:mainfrom
Stevenjin8:tests/e2e-tls

Conversation

@Stevenjin8
Copy link
Contributor

@Stevenjin8 Stevenjin8 commented Feb 2, 2026

What type of PR is this?

/kind test

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

NONE

johananl and others added 29 commits January 21, 2026 18:50
@johananl
Add k8s clients
65fb552 
Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add logger interface
7098cb9 
Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add CRD helpers
ae94714 
Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add Helm helpers
7e76c00 
Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add namespace helpers
0385bd2 
Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add dummy app helpers
f4cfe5f 
We need a dummy workload so that ingresses and gateways have some
service to send test traffic to.

Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add helper for installing Istio
b5b6838 
Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add helpers for Kong
a7ad561 
Kong is both an ingress controller and a GWAPI implementation. Add
helpers for both.

Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add GWAPI helpers
6740ba4 
Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add port forwarding helpers
2d48d39 
We use port forwarding to reach ingress controllers and gateways in an
infrastructure-agnostic way during testing.

Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add ingress-nginx helpers
126c0a6 
Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add helper for creating ingresses
665eaa9 
Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add resource manager
395fded 
To be able to run multiple test cases concurrently on the same cluster,
some resources must be shared. For example, CRDs are cluster-level
rather than namespace-scoped. In addition, deploying multiple instances
of an ingress controller or GWAPI implementation wastes time and
resources even in cases where it's technically possible to do so.

To address these concerns, we add a resource manager. This construct
allows efficient and thread-safe sharing of resources. A shared resource
is created on first usage, returned on subsequent attempts to acquire
the resources and cleaned up when no users remain.

Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add verifier interface and httpGetVerifier implementation
9f95c22 
Verifiers are used to ensure ingress and GWAPI resources process traffic
according to expectations.

Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add main test logic
4f693fa 
Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add some initial test cases
f63093e 
Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add Make targets for e2e
88ed612 
Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Add README for e2e tests
d5c28ed 
Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Install kind to ./bin
7aa6b5f 
We can't use sudo on CI.

Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Use builder pattern for ingresses in test cases
ad1c425 
This allows for more compact test case definitions and fewer indentation
levels.

Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Centralize retry logic
04b9b26 
Rather than implementing bespoke retry logic across the code base,
implement generic retry helpers and reuse them.

This also extracts the retry logic out of verifiers for a better
separation of concerns.

Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@johananl
Use ingress2gateway binary
3221ffb 
Call the binary instead of running Go code directly. This simulates
actual customer usage better and works around concurrency issues caused
by multiple test cases executing in parallel.

Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
@Stevenjin8
Canary tests
c3e0aa8
@Stevenjin8
Merge remote-tracking branch 'up/main' into test/canary
6df4abc
@Stevenjin8
Path rewrite
72043b7
@Stevenjin8
Verify TLS Ingresses and TLS redirects
ae822a8
@Stevenjin8
Merge remote-tracking branch 'up/main' into test/canary
5b2790c
@Stevenjin8
Merge branch 'test/canary' into tests/e2e-tls
04180bf
@Stevenjin8
touchup
8a6932c
@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Feb 2, 2026

@Stevenjin8: The label(s) kind/test cannot be applied, because the repository doesn't have them.

Details

In response to this:

What type of PR is this?

/kind test

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

NONE

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot requested a review from bexxmodd February 2, 2026 20:24
@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Feb 2, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Stevenjin8

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Feb 2, 2026
@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Feb 3, 2026
@k8s-ci-robot k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Feb 4, 2026
@Stevenjin8 Stevenjin8 changed the title Test TLS termination and some SSL-redirect Test TLS termination, SSL-redirect, Canary, and CORS Feb 5, 2026
@johananl
Copy link
Member

johananl commented Feb 6, 2026 •
edited
Loading

Right now make e2e is failing. This is caused by the addition of a new package ./e2e/ingressnginx.

The e2e tests are executed on a single Kind cluster in parallel. To avoid races, we use a resource manager singleton:

ingress2gateway/e2e/resource_manager.go

Lines 23 to 25 in 2802595

var globalResourceManager = &resourceManager{
resources: make(map[string]*resourceState),
}

This assumes all e2e tests are under the same package. The addition of a new sub-package creates two instances of the test framework which in effect duplicates the resource manager. This makes the two e2e package race against the Kind cluster.

To address that, we could do one of the following:

  1. Manage all e2e tests in a single package (we could still separate them by files).
  2. Change the locking mechanism to use e.g. file-based locks which would work across processes.
  3. Disable parallelism.

I suggest we do 1.

@johananl
Copy link
Member

johananl commented Feb 6, 2026

I think you need to rebase. I'm currently seeing many commits already in main in your feature branch.

johananl
johananl reviewed Feb 6, 2026
View reviewed changes
e2e/testutil.go
pfs = append(pfs, pf)
addresses[ingressnginx.NginxIngressClass] = addr
addresses[ingressnginx.NginxIngressClass] = Addresses{HTTP: addr}
if useHTTPS {
Copy link
Member

@johananl johananl Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This switch is becoming big and seems to be largely duplicated code. I wonder if we should make it generic and extract only the provider-specific values to some helper data structure. Please treat as optional since this isn't 100% in scope for this specific PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bexxmodd bexxmodd Awaiting requested review from bexxmodd

@LiorLieberman LiorLieberman Awaiting requested review from LiorLieberman LiorLieberman is a code owner

@robscott robscott Awaiting requested review from robscott robscott is a code owner

@youngnick youngnick Awaiting requested review from youngnick youngnick is a code owner

1 more reviewer

@johananl johananl johananl left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Stevenjin8 @k8s-ci-robot @johananl