Skip to content
View buzzedaldrinx86-creator's full-sized avatar

Block or report buzzedaldrinx86-creator

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse

Jared Condon — independent researcher, LLM evaluation and measurement

I check whether the software that scores AI systems measures what it claims to. The work is done with AI coding agents operating under rules I set, and every finding below comes with a reproducer anyone can run.

Papers and software

  • Checking the Check: Controls, Corpus Alignment, and Scale Invariance in Evaluation Software (preprint, 2026; latest version). doi:10.5281/zenodo.22973130
  • Unapplied Invariants in Evaluation Instruments: A Source Audit and a Corrected Detection Null (preprint; submitted to the HAIC 2026 workshop, non-archival). doi:10.5281/zenodo.22837944
  • two-coder-agreement 0.1.0: a standard-library Python calculator for two-coder nominal agreement (kappa, Krippendorff's alpha, Gwet's AC1) with item-clustered bootstrap intervals. doi:10.5281/zenodo.22972845

Defects reported in evaluation frameworks

Framework Report Outcome
TruLens #2830: the answer being graded was never shown to the judge when empty Fixed upstream in #2835
TruLens #2831: negative score from a reply with no score line Fixed upstream in #2833
TruLens #2832: parse-failure value averaged into groundedness Fixed upstream in #2834
lm-evaluation-harness #4238: parser ImportError scored as a wrong answer Fix proposed in #4240
HELM #4340: undefined inputs converted to definite scores My fix PR #4356 open
LightEval #1353: empty gold reference scored as a model miss Fix under review in #1354
Ragas #2964: missing reference scored as a model failure Open
HF Evaluate #801: undefined AUPR collapsed to 0 Open

ORCID: 0009-0002-0063-4014

Popular repositories Loading

  1. helm helm Public

    Forked from stanford-crfm/helm

    Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparen…

    Python

  2. buzzedaldrinx86-creator buzzedaldrinx86-creator Public

    Profile

  3. policyengine-us policyengine-us Public

    Forked from PolicyEngine/policyengine-us

    The PolicyEngine US Python package contains a rules engine of the US tax-benefit system, and microdata generation for microsimulation analysis.

    Python

  4. haystack haystack Public

    Forked from deepset-ai/haystack

    Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing…

    Python

  5. cline cline Public

    Forked from cline/cline

    Autonomous coding agent as an SDK, IDE extension, or CLI assistant.

    TypeScript

  6. session-relay session-relay Public

    Live idea relay between parallel Claude Code sessions: one session's finding reaches the others mid-task. Local, stdlib-only.

    Python