Skip to content

test: Standardize test skip reasons across test suite#4888

Open
mayankansys wants to merge 5 commits intomainfrom
fix/standardize-test-skip-reasons
Open

test: Standardize test skip reasons across test suite#4888
mayankansys wants to merge 5 commits intomainfrom
fix/standardize-test-skip-reasons

Conversation

@mayankansys
Copy link
Collaborator

Context

Several tests were skipped using @pytest.mark.skip, but the skip reasons were inconsistent and often unclear. Many reasons only described symptoms (for example, “works locally but fails on CI”) without stating whether the root cause was known, partially understood, or unknown. Similar situations were described using different wording across the test suite, making maintenance and investigation harder.

Change Summary

  • Introduced a small set of canonical skip reason string literals in conftest.py.
  • Replaced ad-hoc skip reason strings in test files with these shared constants.
  • Updated existing skipped tests to use the standardized reasons.

Impact

  • Test code only: no impact on runtime or production behavior.
  • Improves readability, searchability, and maintainability of skipped tests.
  • Makes it easier to audit and clean up skipped tests in the future.

@mayankansys mayankansys self-assigned this Feb 4, 2026
@mayankansys mayankansys requested a review from mkundu1 as a code owner February 4, 2026 06:45
Copilot AI review requested due to automatic review settings February 4, 2026 06:45
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR standardizes skip reason strings across the test suite by introducing canonical constants in conftest.py and replacing ad-hoc skip reasons with these shared literals.

Changes:

  • Added three skip reason constants (SKIP_UNKNOWN, SKIP_INVESTIGATING, SKIP_BLOCKED) in tests/conftest.py
  • Updated all test files to import and use these constants instead of custom skip reason strings
  • Categorized existing skipped tests according to the new standardized reasons

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 13 comments.

Show a summary per file
File Description
tests/conftest.py Defines three canonical skip reason constants
tests/test_tui_api.py Replaces "Failing in github" with SKIP_UNKNOWN
tests/test_topy.py Replaces "Unable to read generated python journal" with SKIP_BLOCKED
tests/test_settings_api.py Replaces GitHub issue reference with SKIP_BLOCKED
tests/test_server_meshing_workflow.py Replaces "Failing in Github randomly" with SKIP_UNKNOWN
tests/test_pre_post_session.py Replaces "works fine locally but fails on CI" with SKIP_UNKNOWN
tests/test_new_meshing_workflow.py Replaces GitHub issue and generic failure messages with SKIP_UNKNOWN
tests/test_meshing_workflow.py Replaces "Wait for later implementation" with SKIP_INVESTIGATING
tests/test_mapped_api.py Replaces "Currently only tested in backend" with SKIP_BLOCKED (20 occurrences)
tests/test_launcher_remote.py Replaces GitHub issue reference with SKIP_UNKNOWN
tests/test_flobject.py Replaces GitHub issue reference with SKIP_BLOCKED
tests/test_file_transfer_service.py Replaces verbose skip reasons with SKIP_UNKNOWN and SKIP_BLOCKED
tests/test_field_data.py Replaces GitHub issue reference with SKIP_BLOCKED
tests/test_datamodel_service.py Replaces GitHub issue reference with SKIP_BLOCKED
tests/test_datamodel_api.py Replaces GitHub issue reference with SKIP_BLOCKED (3 occurrences)
tests/test_aero_session.py Replaces "Run this locally only" with SKIP_BLOCKED
tests/parametric/test_parametric_workflow.py Replaces GitHub issue reference with SKIP_BLOCKED

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@prmukherj prmukherj changed the title Standardize test skip reasons across test suite test: Standardize test skip reasons across test suite Feb 5, 2026
@github-actions github-actions bot added the enhancement Improve any current implemented feature label Feb 5, 2026
Copilot AI review requested due to automatic review settings February 5, 2026 11:03
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@mayankansys mayankansys enabled auto-merge (squash) February 5, 2026 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Improve any current implemented feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Standardise pytest skip reasons and document failure understanding

5 participants