quickfix/coherence-leaderboard-bug

[lsabor]

use sum instead of average for calculating score
scale score by 2 to represent the normalized version since h2h scores are recriprocal
and the baseline gets a 0

Summary by CodeRabbit

• Bug Fixes
  • Updated tournament leaderboard score calculation to improve ranking accuracy.

[coderabbitai]

Warning

Rate limit exceeded

@lsabor has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 8 minutes and 46 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📝 Walkthrough

Walkthrough

The pull request modifies the per-user score calculation in the coherence tournament leaderboard update command, changing from an average-based approach to a sum-based approach with a 2.0 multiplier, while keeping the baseline score and aggregation logic intact.

Changes

Cohort / File(s)	Summary
Tournament Leaderboard Scoring scoring/management/commands/update_coherence_tournament_leaderboard.py	Modified per-user score aggregation from average-based calculation to sum-based with 2.0 multiplier, altering score weighting in final leaderboard computation.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Possibly related PRs

• Fix/coherence links switch h2h implementation #4274: Both PRs modify the coherence tournament leaderboard command; the related PR focuses on head-to-head data gathering flow while this PR adjusts the score aggregation methodology.

Suggested reviewers

• hlbmtc

Poem

🐰 Scores now double, hop with cheer,
Sums replace averages here,
Leaderboard rankings bound with might,
A rabbit's touch makes scoring right! ✨

🚥 Pre-merge checks | ✅ 2

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'quickfix/coherence-leaderboard-bug' clearly references a bug fix to the coherence leaderboard scoring calculation, which aligns with the PR's objective of changing the calculation from average to sum with scaling.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch quickfix/coherence-leaderboard-bug

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@scoring/management/commands/update_coherence_tournament_leaderboard.py`:
- Around line 188-191: The runtime TypeError comes from multiplying a Python
list by 2.0 in the expression np.sum(competitor_score_record[uid] * 2.0); update
the computation in the block that builds the leaderboard (referencing
competitor_score_record and uid) to either convert competitor_score_record[uid]
to a NumPy array before doing element-wise multiplication or, simpler, move the
scalar multiplication outside the sum (e.g., 2.0 *
np.sum(competitor_score_record[uid])) and keep the weight sum as
np.sum(competitor_weight_record[uid]); ensure np.sum receives a numeric sequence
or ndarray to avoid the TypeError.

[github-actions]

🧹 Preview Environment Cleaned Up

The preview environment for this PR has been destroyed.

Resource	Status
🌐 Preview App	✅ Deleted
🗄️ PostgreSQL Branch	✅ Deleted
⚡ Redis Database	✅ Deleted
🔧 GitHub Deployments	✅ Removed
📦 Docker Image	⚠️ Retained (auto-cleanup via GHCR policies)

---

Cleanup triggered by PR close at 2026-02-06T21:09:53Z