[POC] ci: skip previously passed workflows on pipeline restart

[saw-jan]

Description

How does it work?

1. First CI pipeline run, save the current pipeline number to s3 using the commit hash as a key
2. Second CI pipeline run (Restart on the same commit), check the previously saved pipeline number in s3
3. If found, get the pipeline info using woodpecker API and save the info to s3 inside the same commit hash key path
4. Then, in each test workflow, check if this was passed previously (if passed, then it will be marked as skip)
5. Each step in the test workflow will be skipped if SKIP_WORKFLOW=true is found in .woodpecker.env

Notes:

• If the pipeline is running with a new commit or there is a change in the commit, then there will be no saved pipeline number for that and the workflows will continue normally as the first run

Implemented workflows

This behaviour is implemented in all the test workflows.

• API tests (local and core)
• web-e2e tests
• cs3-API tests
• wopi-validator tests
• unit tests
• k6 tests

Working example

✔️ When the workflow passed previously (steps skipped):

• Previous pipeline: https://ci.opencloud.rocks/repos/3/pipeline/717/61
• Current restarted pipeline: https://ci.opencloud.rocks/repos/3/pipeline/718/61

❌ When the workflow failed previously (steps run):

• Previous pipeline: https://ci.opencloud.rocks/repos/3/pipeline/717/54
• Current restarted pipeline: https://ci.opencloud.rocks/repos/3/pipeline/718/54

Related Issue

• resolves Restart only one workflow qa#55

Motivation and Context

How Has This Been Tested?

• test environment:

Screenshots (if appropriate):

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Technical debt
• Tests only (no source changes)

Checklist:

• Code changes
• Unit tests added
• Acceptance tests added
• Documentation added

[ScharfViktor]

very impressive, thank you so much 👍

Could you please write the logic in the pr description
that we keep info about previous running in s3 and evaluate it when restarting piplene and decide skip or not

[fschade]

Cool :), quick question, following scenario:

1st run
workflow A is green
workflow B is red

the dev fixes the bug which made pipeline B fail

woodpecker starts a new run
workflow A is skipped (it was green before)
workflow B is green

this sounds really good, but what happens if the fix which made workflow B green introduced a new bug on workflow A?

[ScharfViktor]

then it will be a new $CI_COMMIT_SHA INFO_URL="https://s3.ci.opencloud.eu/public/$CI_REPO_NAME/pipelines/$CI_COMMIT_SHA/pipeline_info.json"
it will not be found (404), and we will run all tests again

[fschade]

Ahhhh thanks @ScharfViktor so, it only applies for restarts? That’s really helpful, thanks for explaining 🤗

[saw-jan]

then it will be a new $CI_COMMIT_SHA INFO_URL="https://s3.ci.opencloud.eu/public/$CI_REPO_NAME/pipelines/$CI_COMMIT_SHA/pipeline_info.json" it will not be found (404), and we will run all tests again

yeah, correct. We only want to restart the failing workflows and the change in commit would mean full CI run

[saw-jan]

Could you please write the logic in the pr description
that we keep info about previous running in s3 and evaluate it when restarting piplene and decide skip or not

Updated the PR description