-
Notifications
You must be signed in to change notification settings - Fork 52
feat: SDK Compliance #397
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
feat: SDK Compliance #397
Conversation
PostHog SDK Compliance ReportSDK: posthog-python
|
| Test | Status | Duration |
|---|---|---|
| Format Validation.Event Has Required Fields | ✅ | 516ms |
| Format Validation.Event Has Uuid | ✅ | 1507ms |
| Format Validation.Event Has Lib Properties | ✅ | 1506ms |
| Format Validation.Distinct Id Is String | ✅ | 1507ms |
| Format Validation.Token Is Present | ✅ | 1506ms |
| Retry Behavior.Retries On 503 | ✅ | 8330ms |
| Retry Behavior.Does Not Retry On 400 | ✅ | 3508ms |
| Retry Behavior.Does Not Retry On 401 | ✅ | 3507ms |
| Retry Behavior.Respects Retry After Header | ❌ | 7509ms |
| Retry Behavior.Implements Backoff | ❌ | 20521ms |
| Deduplication.Generates Unique Uuids | ✅ | 1495ms |
| Deduplication.Preserves Uuid On Retry | ✅ | 6968ms |
| Deduplication.Different Events Have Different Uuids | ✅ | 1503ms |
| Compression.Sends Gzip When Enabled | ✅ | 1506ms |
| Batch Format.Uses Proper Batch Structure | ✅ | 1507ms |
| Error Handling.Does Not Retry On 413 | ✅ | 3512ms |
| Error Handling.Retries On 408 | ❌ | 6505ms |
Failures
retry_behavior.respects_retry_after_header
Retry delay too short: 999ms < 2500ms
retry_behavior.implements_backoff
First retry delay too short: 46ms < 100ms
error_handling.retries_on_408
Expected at least 2 requests, got 1
eli-r-ph
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This LGTM
posthog-python Compliance ReportDate: 2026-01-28 13:42:24 UTC
|
| Test | Status | Duration |
|---|---|---|
| Format Validation.Event Has Required Fields | ✅ | 517ms |
| Format Validation.Event Has Uuid | ✅ | 1507ms |
| Format Validation.Event Has Lib Properties | ✅ | 1507ms |
| Format Validation.Distinct Id Is String | ✅ | 1507ms |
| Format Validation.Token Is Present | ✅ | 1507ms |
| Format Validation.Custom Properties Preserved | ✅ | 1506ms |
| Format Validation.Event Has Timestamp | ✅ | 1507ms |
| Retry Behavior.Retries On 503 | ✅ | 7970ms |
| Retry Behavior.Does Not Retry On 400 | ✅ | 3506ms |
| Retry Behavior.Does Not Retry On 401 | ✅ | 3508ms |
| Retry Behavior.Respects Retry After Header | ❌ | 6629ms |
| Retry Behavior.Implements Backoff | ✅ | 21129ms |
| Retry Behavior.Retries On 500 | ✅ | 6768ms |
| Retry Behavior.Retries On 502 | ✅ | 7117ms |
| Retry Behavior.Retries On 504 | ✅ | 7203ms |
| Retry Behavior.Max Retries Respected | ✅ | 18778ms |
| Deduplication.Generates Unique Uuids | ✅ | 1498ms |
| Deduplication.Preserves Uuid On Retry | ✅ | 7472ms |
| Deduplication.Preserves Uuid And Timestamp On Retry | ✅ | 13545ms |
| Deduplication.Preserves Uuid And Timestamp On Batch Retry | ✅ | 7314ms |
| Deduplication.No Duplicate Events In Batch | ✅ | 1503ms |
| Deduplication.Different Events Have Different Uuids | ✅ | 1507ms |
| Compression.Sends Gzip When Enabled | ✅ | 1507ms |
| Batch Format.Uses Proper Batch Structure | ✅ | 1506ms |
| Batch Format.Flush With No Events Sends Nothing | ✅ | 1005ms |
| Batch Format.Multiple Events Batched Together | ✅ | 1505ms |
| Error Handling.Does Not Retry On 403 | ✅ | 3507ms |
| Error Handling.Does Not Retry On 413 | ✅ | 3509ms |
| Error Handling.Retries On 408 | ❌ | 6511ms |
Failures
retry_behavior.respects_retry_after_header
Retry delay too short: 122ms < 2500ms
error_handling.retries_on_408
Expected at least 2 requests, got 1
Integrate the SDK Test Harness to validate SDK compliance against the shared capture API contract.
This adds:
The test harness runs a mock PostHog server, exercises the SDK through the adapter, and verifies behavior matches the contract defined in CONTRACT.yaml (event format, retries, compression, deduplication, etc.).