[DRAFT] ROX-8711 Test suite for deployment bundles

[mtesseract]

Description

This work is part of a proof-of-concept for https://issues.redhat.com/browse/ROX-8711.

The basic idea here is that we would like to execute the Helm chart's test suite also in the case where we pre-render the chart files in "deployment bundle mode".

However, the difficulty is that in general the tests require some modification depending on whether executed in the "Helm chart mode" or in the "deployment bundle mode".

This PR depends on this (draft) PR for helmtest: stackrox/helmtest#7.

The helmtest PR extends the helmtest framework such that tests can be run selectively, depending on a "flavour" value, which is passed in by the user of helmtest and which can be accessed by the leave tests defined in the test suite.

This PR extends the test suite for the securedcluster Helm chart such that in addition to what it currently does it instantiates the chart in the "bundle" mode and runs helmtest with the "bundle" flavour being specified.

(Almost) all currently defined tests are moved into a subdirectory "chart", because they are currently specific for the chart use-case.

A new directory "bundle" is created, which contains (almost) identical basic definitions as the "chart" specific test suite.

One test, cluster-name.yaml, has been moved from the chart directory into a new shared directory. The test in that file has been protected with a condition such that it is only executed in the chart test. A slightly modified copy of this test has been appended in the same file which is only executed in the bundle test suite. This file has been symlinked to the chart and the bundle directory.

Checklist

• Investigated and inspected CI test results
• Unit test and regression tests added
• Evaluated and added CHANGELOG entry if required
• Determined and documented upgrade steps

Testing Performed

This is about extending the test coverage.

[ghost]

Tag for build #63052 is 3.67.x-225-g004316d970.

💻 For deploying this image using the dev scripts, run the following first:

export MAIN_IMAGE_TAG='3.67.x-225-g004316d970'

📦 You can also generate an installation bundle with:

docker run -i --rm stackrox/main:3.67.x-225-g004316d970 central generate interactive > bundle.zip

🕹️ A roxctl binary artifact can be downloaded from CircleCI.

[openshift-ci]

@mtesseract: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci]

@mtesseract: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/gke-upgrade-tests	004316d	link	true	/test gke-upgrade-tests
ci/prow/stackrox_branding-images	004316d	link	true	/test stackrox_branding-images
ci/prow/ui-unit-tests	004316d	link	true	/test ui-unit-tests
ci/prow/integration-unit-tests	004316d	link	true	/test integration-unit-tests
ci/prow/shell-unit-tests	004316d	link	true	/test shell-unit-tests
ci/prow/go-unit-tests-release	004316d	link	true	/test go-unit-tests-release
ci/prow/go-unit-tests	004316d	link	true	/test go-unit-tests
ci/prow/push-images	004316d	link	true	/test push-images
ci/prow/policy-checks	004316d	link	true	/test policy-checks

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.