v6.7.0

Updated Tests

-services/service_instance_lifecycle.go

• Relaxes the test expectations to support cli v8.

Other Updates

-dora app

• Now accepts connections from all addresses.

v6.6.0

Updated Tests

-apps/syslog_drain.go

• better reflect original intention of the test

-isolation_segments/isolation_segments.go

• Stop hardcoding the protocol

-apps/loggregator.go

• increase the timeout to reduce flakiness of test on cf-for-k8s where logging can be slower

-route_services/route_services.go

• remove obsolete -f flag

-windows/lifecycle.go

• Use the configured timeout, rather than the hardcoded one

Other updates

• add uptime endpoint to dora
• improve sticky session documentation

v6.5.0

Updated Tests

-apps/crashing.go

• Updated crash event tests to work with Eirini 1.9.0.
  -apps/routing_transparency.go
• Updated routing transparency tests to work with Istio.

Helpers updates

• Helpers added for route tests.

-helpers/download/download.go

• Send curl output to file to avoid droplet binary dump in terminal.

Other Updates

• Convert calls leveraging v2 Cloud Controller API endpoints to use the v3 CAPI endpoints, as part of VAT's work to deprecate v2 CAPI.
• Additional tests skipped when running against cf-for-k8s, due to incompatibilities with the platform.

v6.4.0

Updated Tests

-apps/crashing.go

• There are two fixes to test app crashes:

1. The id checking for recovery was giving false positives because the post-crash ID was returning an error message which happened to differ from the pre-crash app's ID. It turns out that that IDs are reused on cf-for-k8s but change for cf-deployment.
2. The two different infrastructure's IDs match different specific regular expressions, so a more general one handles both cases.

• cf-for-k8s is waiting for a new release of eirini that will handle app.crash events, so that test is skipped for cf-for-k8s but the recovery tests can be run.

v6.3.0

Notice:

• Cloud Foundry acceptance tests (CATS) now works with CF for Kubernetes

Updated Tests and Helpers

The following general changes affected multiple files, which we won't list here individually:

• 598ff1d : Always explicitly target the test space

• Multiple commits for refactoring and adding conditional skips to support cf-for-k8s:

  • a1dfe43
  • 440ede1
  • ede3fab
  • c5202cd
  • c8c5e9c
  • 4666bdf
  • 70f85c2
  • 38e9d1f
  • 0b321ae
  • 574d01e
  • c40d715
  • d356dc0
  • 36deda9
  • 90c8dbf
  • 909cb52
  • 0dd10a8
  • bf335f9
  • 4e2128c
  • 8fbb5d5

Some of the assertions have also been changed, with the addition of an informative string to be rendered and emitted when an assertion fails.

Other Updates

-docs/run-on-k8s.md
0d796dd : Covers needed changes for running CATs on CF for Kubernetes.

v6.2.0

Notice:

• This release adds the ability to control which tests will run based on the configuration option infrastructure with possible values kubernetes or vms (default).

Note that CATS is still under construction for Kubernetes environment and we are working towards a running version of CATs in cf-for-k8s

Updated Tests

• See this PR 429 for list of tests that were updated. Note

v6.1.0

Notice:

• With this release, we have officially deprecated the windows2012 stack in CATs after having completed deprecation in cf-deployment and the cf cli for some time now.

Updated Tests

• apps/loggregator.go
  • Update go loggregator library version for grpc cve fix
• internet_dependent/git_buildpack.go
  • Update nodejs-buildpack to latest version
• volume_services/volume_services.go
  • Update volume services tests for cf cli v7

Helpers updates

• helpers/config/config_struct.go
  • Remove support for the windows2012 stack.
• helpers/config/config_test.go
  • Remove extra unit testing for windows stack config

Other Updates

• /assets/*
  • Add go.mod to go apps for paketo buildpack compatibility
  • Update bundler and gem versions for ruby apps for paketo buildpack compatibility

v6.0.0

Notice

💥💥 Cloud Foundry acceptance tests (CATS) now works with cf CLI v7💥💥

This major release of CATS now works with cf CLI v7. As you may be aware, the cf CLI team recently release the much awaited v7 CLI that unlocks new app developer workflows for users who require granular control of their apps and other advanced deployment strategies. You can check out more @ https://docs.cloudfoundry.org/cf-cli/v7.html.

With this release, CATS NO longer supports running with cf CLI v6. Going forward, we will only support running with cf CLI v7. If you're running CATS in your pipelines, we recommend you update your cf CLI to v7 or pin to an older version of CATS < v6.0.0.

It would have not been possible without the hard work of folks on the cf CLI team, V3 Acceleration Team, and Release Integration to come together and make this happen.

New Tests

• apps/piecemeal_app_creation_flow.go
  • c760f93

  cover more granular app commands introduced in the v7 CLI/v3 API

• apps/rolling_deploy.go
  • 311fb61

  cf push --strategy=rolling is a new feature in the V7 CF CLI

Updated Tests

Instead of pointing at each file with a change, we will record the general updates we had to make across multiple suites for the migration to cf CLI v7

• cf push no longer uses the -d flag
• cf buildpack uses 1-indexed references instead of 0-indexed
• use the -space flag instead of a positional arg for cf security groups
• use the -command flag for cf run-task
• healthcheck no longer supports the none type, so we updated to process type
• cf add-network-policy now uses a positional arg instead of the -destination-app flag

Deleted Tests

• windows/set_start_command.go
  • e022acf

  remove windows/set_start_command.go as it tests code that's handled in windows/worker.go

• backend_compatibility/backend_compatibility.go
  • 2e0bdc8

  this test was specifically testing if DEA droplets can be staged onto
  Diego, and moving forward Cloudfoundry will be abandoning DEAs all together.

Helpers updates

• helpers/logs/logs_helper.go
  • f22c809

  v7 no longer hits traffic controller; it uses log cache for all logging. This removes the config setting and switches to cf logs instead of cftail.

Other Updates

• v3/buildpacks.go
  • We discovered capi no-bridge tests are no longer present in CATs, but the configuration and branching logic were not cleaned up.
• backend_compatibility/backend_compatibility.go
  • With the deletion of the only tests in its suite, we also deleted the backend_compatibility flag from the cats configuration interface.
• helpers/config/config.go
  • f22c809

  v7 no longer hits the traffic controller; it uses log cache for all logging. This removes the config setting.

v5.11.0

Updated Tests

-volume_services/volume_services.go

• Simplify setup for volume services test suite by enabling diego_docker feature flag. Skip the volume services test suite if the docker suite is enabled.

Other Updates

-assets/nora/README.md

• Replace defunct ./make_a_nora with cf push nora.

v5.10.0

Other Updates

-helpers/app_helpers/app_helpers.go

• Make the AppReporter more resilient. Print out app logs only when a test fails. If getting logs fails, continue so cleanup proceeds. This fixes a resource leak when getting logs failed.

-Update golang crypto library

• Specify the correct hash for the golang.org/x/crypto library