Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.claude
22 changes: 15 additions & 7 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,16 +34,22 @@ Node.js compatibility tests: `cd mnode-test && ./docker-test.sh` (requires Docke

### Core Upload Pipeline (src/utils/result-upload/)

The upload flow has two stages handled by two classes:
The upload flow has two stages handled by two classes, with a shared `MarkerParser` instance:

1. **`ResultUploadCommandHandler`** — Orchestrates the overall flow:
1. **`MarkerParser`** — Centralizes all test case marker detection/extraction/matching logic:
- Supports three marker formats: hyphenated (`PRJ-123`), underscore-separated hyphenless (`test_prj123_foo`), and CamelCase hyphenless (`TestPrj123Foo` or `TestFooPrj123`)
- Hyphenless matching (underscore-separated and CamelCase) is gated on `type === 'junit-upload'` and requires the test name to start with `test` (case-insensitive)
- Created by `ResultUploadCommandHandler` and passed to `ResultUploader` — both share one instance
- Also exports a standalone `formatMarker()` function used by parsers

2. **`ResultUploadCommandHandler`** — Orchestrates the overall flow:
- Parses report files using the appropriate parser (JUnit XML or Playwright JSON)
- Detects project code from test case names (or from `--run-url`)
- Detects project code from test case names via `MarkerParser` (or from `--run-url`)
- Creates a new test run (or reuses an existing one if title conflicts)
- Delegates actual result uploading to `ResultUploader`

2. **`ResultUploader`** — Handles the upload-to-run mechanics:
- Fetches test cases from the run, maps parsed results to them via marker matching
3. **`ResultUploader`** — Handles the upload-to-run mechanics:
- Fetches test cases from the run, maps parsed results to them via `MarkerParser` matching
- Validates unmatched/missing test cases (respects `--force`, `--ignore-unmatched`)
- Uploads file attachments concurrently (max 10 parallel), then creates results in batches (max 50 per request)

Expand All @@ -65,14 +71,15 @@ Composable fetch wrappers using higher-order functions:

- `env.ts` — Loads `QAS_TOKEN` and `QAS_URL` from environment variables, `.env`, or `.qaspherecli` (searched up the directory tree)
- `config.ts` — Constants (required Node version)
- `misc.ts` — URL parsing, template string processing (`{env:VAR}`, date placeholders), error handling utilities
- `misc.ts` — URL parsing, template string processing (`{env:VAR}`, date placeholders), error handling utilities. Note: marker-related functions have been moved to `MarkerParser.ts`
- `version.ts` — Reads version from `package.json` by traversing parent directories

## Testing

Tests use **Vitest** with **MSW** (Mock Service Worker) for API mocking. Test files are in `src/tests/`:

- `result-upload.spec.ts` — Integration tests for the full upload flow (both JUnit and Playwright), with MSW intercepting all API calls
- `result-upload.spec.ts` — Integration tests for the full upload flow (both JUnit and Playwright), with MSW intercepting all API calls. Includes hyphenless and CamelCase marker tests (JUnit only)
- `marker-parser.spec.ts` — Unit tests for `MarkerParser` (detection, extraction, matching across all marker formats and command types)
- `junit-xml-parsing.spec.ts` — Unit tests for JUnit XML parser
- `playwright-json-parsing.spec.ts` — Unit tests for Playwright JSON parser
- `template-string-processing.spec.ts` — Unit tests for run name template processing
Expand All @@ -90,3 +97,4 @@ ESM project (`"type": "module"`). TypeScript compiles to `build/`, then `ts-add-
- **Linter**: ESLint with typescript-eslint (config: `eslint.config.mjs`)
- **Formatter**: Prettier (config: `.prettierrc`)
- **Pre-commit hook** (Husky): runs lint-staged (Prettier + ESLint on staged files)
- **Commits**: Do NOT add `Co-Authored-By` lines to commit messages
29 changes: 24 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,17 +181,36 @@ The QAS CLI maps test results from your reports (JUnit XML or Playwright JSON) t

### JUnit XML

Test case names in JUnit XML reports must include a QA Sphere test case marker in the format `PROJECT-SEQUENCE`:
Test case names in JUnit XML reports must include a QA Sphere test case marker. The following marker formats are supported (checked in order):

- **PROJECT** - Your QA Sphere project code
- **SEQUENCE** - Test case sequence number (minimum 3 digits, zero-padded if needed)
#### 1. Hyphenated Marker (all languages)

Format: `PROJECT-SEQUENCE` where **PROJECT** is your QA Sphere project code and **SEQUENCE** is the test case sequence number (minimum 3 digits, zero-padded if needed). The marker can appear anywhere in the test name and is matched case-insensitively.

**Examples:**

- `PRJ-002: Login with valid credentials`
- `Login with invalid credentials: PRJ-1312`

**Note:** The project code in test names must exactly match your QA Sphere project code.
#### 2. Underscore-Separated Hyphenless Marker (pytest, Go, Rust, etc.)

For languages where test names are function identifiers and hyphens are not allowed, the CLI supports hyphenless markers separated by underscores. The test name must start with `test` (case-insensitive).

**Examples (pytest):**

- `test_prj002_login_with_valid_credentials`
- `test_login_with_invalid_credentials_prj1312`

#### 3. CamelCase Hyphenless Marker (Go, Java)

For CamelCase test function names, the CLI detects markers at the start (immediately after the `Test` prefix) or at the end of the name. The test name must start with `Test` (case-insensitive).

**Examples (Go):**

- `TestPrj002LoginWithValidCredentials` (marker at start)
- `TestLoginWithValidCredentialsPrj1312` (marker at end)

**Note:** Hyphenless matching (formats 2 and 3) is only available for `junit-upload`. For `playwright-json-upload`, only the hyphenated format is supported (or test annotations, see below).

### Playwright JSON

Expand All @@ -216,7 +235,7 @@ Playwright JSON reports support two methods for referencing test cases (checked
)
```

2. **Test Case Marker in Name** - Include the `PROJECT-SEQUENCE` marker in the test name (same format as JUnit XML)
2. **Hyphenated Marker in Name** - Include the `PROJECT-SEQUENCE` marker in the test name (same format as JUnit XML format 1). Hyphenless markers are **not** supported for Playwright JSON

## Development (for those who want to contribute to the tool)

Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "qas-cli",
"version": "0.4.5",
"version": "0.4.6",
"description": "QAS CLI is a command line tool for submitting your automation test results to QA Sphere at https://qasphere.com/",
"type": "module",
"main": "./build/bin/qasphere.js",
Expand Down
16 changes: 16 additions & 0 deletions src/tests/fixtures/junit-xml/camelcase-matching-tcases.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
<?xml version="1.0" encoding="utf-8"?>
<testsuites name="Go tests">
<testsuite name="cart_test" errors="0" failures="1" skipped="0" tests="5" time="3.5" timestamp="2026-02-09T12:00:00.000000+00:00" hostname="test-host">
<testcase classname="cart_test" name="TestTest002CartItems" time="0.649">
</testcase>
<testcase classname="cart_test" name="TestTest003CheckoutFlow" time="0.625">
</testcase>
<testcase classname="contents_test" name="TestAboutPageTest004" time="0.988">
</testcase>
<testcase classname="contents_test" name="TestTest006NavbarItems" time="0.584">
<failure message="AssertionError: expected True but got False">AssertionError: expected True but got False</failure>
</testcase>
<testcase classname="contents_test" name="TestMenuContentTest007" time="0.734">
</testcase>
</testsuite>
</testsuites>
16 changes: 16 additions & 0 deletions src/tests/fixtures/junit-xml/hyphenless-matching-tcases.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
<?xml version="1.0" encoding="utf-8"?>
<testsuites name="pytest tests">
<testsuite name="pytest" errors="0" failures="1" skipped="0" tests="5" time="3.5" timestamp="2026-02-09T12:00:00.000000+00:00" hostname="test-host">
<testcase classname="tests.cart_test.TestCart" name="test_test002_cart_items" time="0.649">
</testcase>
<testcase classname="tests.cart_test.TestCart" name="test_test003_checkout_flow" time="0.625">
</testcase>
<testcase classname="tests.contents_test.TestContents" name="test_test004_about_page" time="0.988">
</testcase>
<testcase classname="tests.contents_test.TestContents" name="test_test006_navbar_items" time="0.584">
<failure message="AssertionError: expected True but got False">AssertionError: expected True but got False</failure>
</testcase>
<testcase classname="tests.contents_test.TestContents" name="test_test007_menu_content" time="0.734">
</testcase>
</testsuite>
</testsuites>
235 changes: 235 additions & 0 deletions src/tests/marker-parser.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,235 @@
import { describe, expect, test } from 'vitest'
import { MarkerParser, formatMarker } from '../utils/result-upload/MarkerParser'

const junit = new MarkerParser('junit-upload')
const playwright = new MarkerParser('playwright-json-upload')

describe('formatMarker', () => {
test('pads sequence to 3 digits', () => {
expect(formatMarker('TEST', 2)).toBe('TEST-002')
expect(formatMarker('PRJ', 42)).toBe('PRJ-042')
})

test('does not pad sequences with 3+ digits', () => {
expect(formatMarker('TEST', 123)).toBe('TEST-123')
expect(formatMarker('TEST', 1234)).toBe('TEST-1234')
})

test('instance method matches standalone function', () => {
expect(junit.formatMarker('ABC', 7)).toBe(formatMarker('ABC', 7))
})
})

describe('detectProjectCode', () => {
describe('hyphenated markers (all formats)', () => {
test('at start of name', () => {
expect(junit.detectProjectCode('TEST-002 Cart items')).toBe('TEST')
expect(playwright.detectProjectCode('TEST-002 Cart items')).toBe('TEST')
})

test('at end of name', () => {
expect(junit.detectProjectCode('Cart items TEST-002')).toBe('TEST')
expect(playwright.detectProjectCode('Cart items TEST-002')).toBe('TEST')
})

test('in middle of name', () => {
expect(junit.detectProjectCode('Some TEST-002 thing')).toBe('TEST')
expect(playwright.detectProjectCode('Some TEST-002 thing')).toBe('TEST')
})

test('with alphanumeric project code', () => {
expect(junit.detectProjectCode('BD026-123 something')).toBe('BD026')
})
})

describe('separator-bounded hyphenless (JUnit only)', () => {
test('underscore-separated', () => {
expect(junit.detectProjectCode('test_test002_cart')).toBe('TEST')
})

test('returns null for Playwright', () => {
expect(playwright.detectProjectCode('test_test002_cart')).toBeNull()
})

test('requires name starting with "test"', () => {
expect(junit.detectProjectCode('check_test002_cart')).toBeNull()
})

test('case-insensitive project code', () => {
expect(junit.detectProjectCode('test_bd026_cart')).toBe('BD')
})
})

describe('camelCase start (JUnit only)', () => {
test('marker after Test prefix', () => {
expect(junit.detectProjectCode('TestTest002CartItems')).toBe('TEST')
})

test('returns null for Playwright', () => {
expect(playwright.detectProjectCode('TestTest002CartItems')).toBeNull()
})

test('marker at end of string', () => {
expect(junit.detectProjectCode('TestBd026')).toBe('BD')
})

test('requires name starting with "test"', () => {
expect(junit.detectProjectCode('CheckTest002CartItems')).toBeNull()
})
})

describe('camelCase end (JUnit only)', () => {
test('marker at end of name', () => {
expect(junit.detectProjectCode('TestCartItemsTest002')).toBe('TEST')
})

test('returns null for Playwright', () => {
expect(playwright.detectProjectCode('TestCartItemsTest002')).toBeNull()
})

test('all-uppercase without separator matches via separator pattern', () => {
expect(junit.detectProjectCode('TEST002')).toBe('TEST')
})
})

describe('no match', () => {
test('returns null for unrecognized names', () => {
expect(junit.detectProjectCode('some random test')).toBeNull()
expect(junit.detectProjectCode('test_cart_items')).toBeNull()
})
})
})

describe('extractSeq', () => {
describe('hyphenated markers (all formats)', () => {
test('at start of name', () => {
expect(junit.extractSeq('TEST-002 Cart items', 'TEST')).toBe(2)
expect(playwright.extractSeq('TEST-002 Cart items', 'TEST')).toBe(2)
})

test('at end of name', () => {
expect(junit.extractSeq('Cart items TEST-1234', 'TEST')).toBe(1234)
})

test('in middle of name', () => {
expect(junit.extractSeq('Some TEST-042 thing', 'TEST')).toBe(42)
})
})

describe('separator-bounded hyphenless (JUnit only)', () => {
test('underscore-separated', () => {
expect(junit.extractSeq('test_test002_cart', 'TEST')).toBe(2)
})

test('returns null for Playwright', () => {
expect(playwright.extractSeq('test_test002_cart', 'TEST')).toBeNull()
})

test('requires name starting with "test"', () => {
expect(junit.extractSeq('check_test002_cart', 'TEST')).toBeNull()
})

test('case-insensitive project code match', () => {
expect(junit.extractSeq('test_bd026_cart', 'BD')).toBe(26)
})
})

describe('camelCase start (JUnit only)', () => {
test('marker after Test prefix', () => {
expect(junit.extractSeq('TestTest002CartItems', 'TEST')).toBe(2)
})

test('returns null for Playwright', () => {
expect(playwright.extractSeq('TestTest002CartItems', 'TEST')).toBeNull()
})

test('case-insensitive project code', () => {
expect(junit.extractSeq('TestBd026Something', 'BD')).toBe(26)
})
})

describe('camelCase end (JUnit only)', () => {
test('marker at end of name', () => {
expect(junit.extractSeq('TestCartItemsTest002', 'TEST')).toBe(2)
})

test('returns null for Playwright', () => {
expect(playwright.extractSeq('TestCartItemsTest002', 'TEST')).toBeNull()
})
})

describe('no match', () => {
test('returns null for wrong project code', () => {
expect(junit.extractSeq('TEST-002 Cart items', 'OTHER')).toBeNull()
})

test('returns null for no marker', () => {
expect(junit.extractSeq('test_cart_items', 'TEST')).toBeNull()
})
})
})

describe('nameMatchesTCase', () => {
describe('hyphenated markers (all formats)', () => {
test('case-insensitive match', () => {
expect(junit.nameMatchesTCase('test-002 Cart', 'TEST', 2)).toBe(true)
expect(junit.nameMatchesTCase('TEST-002 Cart', 'TEST', 2)).toBe(true)
expect(playwright.nameMatchesTCase('TEST-002 Cart', 'TEST', 2)).toBe(true)
})

test('marker anywhere in name', () => {
expect(junit.nameMatchesTCase('Cart TEST-002 items', 'TEST', 2)).toBe(true)
expect(junit.nameMatchesTCase('Cart items TEST-002', 'TEST', 2)).toBe(true)
})

test('no match for wrong seq', () => {
expect(junit.nameMatchesTCase('TEST-002 Cart', 'TEST', 3)).toBe(false)
})
})

describe('separator-bounded hyphenless (JUnit only)', () => {
test('underscore-separated', () => {
expect(junit.nameMatchesTCase('test_test002_cart', 'TEST', 2)).toBe(true)
})

test('returns false for Playwright', () => {
expect(playwright.nameMatchesTCase('test_test002_cart', 'TEST', 2)).toBe(false)
})

test('requires name starting with "test"', () => {
expect(junit.nameMatchesTCase('check_test002_cart', 'TEST', 2)).toBe(false)
})

test('no match for wrong seq', () => {
expect(junit.nameMatchesTCase('test_test002_cart', 'TEST', 3)).toBe(false)
})
})

describe('camelCase start (JUnit only)', () => {
test('matches marker after Test prefix', () => {
expect(junit.nameMatchesTCase('TestTest002CartItems', 'TEST', 2)).toBe(true)
})

test('returns false for Playwright', () => {
expect(playwright.nameMatchesTCase('TestTest002CartItems', 'TEST', 2)).toBe(false)
})

test('no match for wrong seq', () => {
expect(junit.nameMatchesTCase('TestTest002CartItems', 'TEST', 3)).toBe(false)
})
})

describe('camelCase end (JUnit only)', () => {
test('matches marker at end', () => {
expect(junit.nameMatchesTCase('TestCartItemsTest002', 'TEST', 2)).toBe(true)
})

test('returns false for Playwright', () => {
expect(playwright.nameMatchesTCase('TestCartItemsTest002', 'TEST', 2)).toBe(false)
})

test('no match for wrong seq', () => {
expect(junit.nameMatchesTCase('TestCartItemsTest002', 'TEST', 3)).toBe(false)
})
})
})
Loading