Skip to content

test: run ClickHouse core SQL tests through clickhouse-go - #2141

Open
ccoltx wants to merge 28 commits into
mainfrom
ccoltx/Run-core-SQL-tests
Open

ccoltx wants to merge 28 commits into
mainfrom
ccoltx/Run-core-SQL-tests

Conversation

@ccoltx

@ccoltx ccoltx commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Runs a curated subset of the ClickHouse core SQL tests (tests/queries/0_stateless) through clickhouse-go, nightly and on changes to the harness. Implements ClickHouse/integrations#810.

tests/clickhouse-test-runner/ is a Go program that imitates clickhouse-client well enough for the upstream runner (tests/clickhouse-test) to drive it without changes. It sends every query through clickhouse-go, over the native or the HTTP protocol. It scans each row into ColumnType.ScanType(), as a user does, and writes the values as TabSeparated. So a test passes only when clickhouse-go decodes and scans every value correctly; the upstream .reference files are the oracle.

The goal is to find client bugs, not only to pass tests. The harness already found several (see below).

What is in the PR

  • The program: argument parsing, statement splitting (same-line trailing hints, echo hints, the Tags: header, INSERT ... FORMAT inline data), test hints, session handling over both protocols, and a type-directed TabSeparated renderer that is checked against the server's own output.
  • allowlist.txt: 9,903 tests over both protocols, 129 over native only (WITH TOTALS: the server sends no totals in the HTTP Native output), and 920 known failures in 30 groups. Each group header gives the cause and a small reproduction. KNOWN_FAILURES=<id> runs one group, to check a fix.
  • clickhouse-version: pins the server version and the upstream tests together (26.10.4.207, tests from v26.10.4.207-stable). A new ClickHouse release does not change CI; a bump is a pull request of its own, so other pull requests do not fail because of a release.
  • upstream-sql-tests.yml: nightly, manual (clickhouse_version input to try a version), and on pull requests that change the harness. Native and HTTP, 8 shards each.
  • README.md and AGENTS.md: local runs, how to find the cause of a failure (compare with the real clickhouse-client: scripts/run-with-real-client.sh), the rules for the allowlist, and how to bump the version. tests/clickhouse-test-runner/CLAUDE.md imports AGENTS.md, and the root .claude/CLAUDE.md points to it.

Client bugs that the harness found

All reproduce with the public API only, over both protocols unless noted. None is fixed in this PR; each has a known-failure group. I have not filed issues yet.

Wrong values, no error:

  • DateTime64 outside 1677-09-21 .. 2262-04-11 is wrong when read and when written (int64 nanosecond overflow in ch-go DateTime64.Time and in timeToInt64).
  • Time64 with precision 4, 5, 7 or 8 is cut to milliseconds (ch-go truncateDuration).
  • A Nullable value with a null-map byte other than 1 reads as not NULL (lib/column/nullable.go checks == 1).
  • A NULL in a LowCardinality(Nullable(String)) element of a named tuple reads as "".
  • With extremes = 1, native Rows gives the extremes as data rows.

Session state, native:

  • The pool can open a second connection with MaxOpenConns: 1, without an error, so the session state is lost: release() frees the slot before it returns the connection to the idle list. About 1 to 5 in 2,000 iterations. This can make the nightly run fail intermittently in tests with SET, until it is fixed.
  • The connection closes after any error, also a server exception, so temporary tables are lost.
  • Exec of an INSERT that needs data from the client (INSERT INTO t VALUES) never returns; it waits until the context ends.
  • Query on a statement with no result block (WITH ... INSERT) returns io.EOF and closes the connection.
  • rows.Next() can drop blocks that it received before an exception: it reads the block and error channels in one select, so the error can win (in 4 of 20 runs, 970 or 990 of 1000 rows).

Errors on valid data: a tuple in a tuple, Nullable(Tuple) and Int128 in a tuple cannot be scanned into ScanType(); backtick-quoted tuple element names, some JSON data (stream out of sync), JSON(SHARED ...), some QBit forms, Map keys of type Array/Map/Tuple, Tuple(), AggregateFunction, Geometry. Over HTTP, the server-info query of a new connection runs in the user session (visible in query_log; breaks a session with SET dialect). session_timezone is ignored.

Test plan

  • Unit tests of the harness (go test ./tests/clickhouse-test-runner/), and live tests that compare the renderer with the server's TabSeparated output (CLICKHOUSE_TEST_RUNNER_LIVE=1, run in CI shard 1).
  • Locally against 26.10.4.207: full sweeps of all 12,961 .sql tests over both protocols, with each failure compared against the real clickhouse-client. Then the allowlist three times on a fresh server per protocol: 0 to 9 failures per run, each one triaged (pool race, test isolation, or memory limits that also fail through the real client).
  • This PR runs the new workflow on CI.

Checklist

  • Unit and integration tests covering the common scenarios were added

🤖 Generated with Claude Code


Note

Low Risk
Adds test-only harness and CI; it does not change production driver APIs, though new nightly jobs can fail on known client bugs listed in the allowlist.

Overview
Adds tests/clickhouse-test-runner, a clickhouse-client shim that drives ClickHouse’s upstream tests/clickhouse-test through clickhouse-go (native and HTTP). Queries run via the driver, results are re-encoded as TabSeparated from ScanType() values, and output is compared to upstream .reference files.

The PR wires CI in upstream-sql-tests.yml (nightly, manual version override, path-filtered PRs): Docker ClickHouse pinned in clickhouse-version, sparse upstream checkout, an 8-shard × 2-protocol matrix, harness unit/live tests on shard 1, and RETRY_FAILED=2 for a known native pool race. Supporting pieces include allowlist.txt (passing tests plus known-failure groups), shell scripts to fetch upstream tests and run suites, and AGENTS.md / README.md for triage and allowlist rules. Root CLAUDE.md now points agents at that doc before changing the allowlist or debugging the workflow.

Reviewed by Cursor Bugbot for commit 7402883. Bugbot is set up for automated code reviews on this repo. Configure here.

Add tests/clickhouse-test-runner: a Go program that imitates
clickhouse-client well enough for the upstream runner
(tests/clickhouse-test) to drive it. It runs every query through
clickhouse-go over the native or HTTP protocol, scans each row into
ColumnType.ScanType(), and writes the values as TabSeparated, so the
upstream .reference files check the decoding of the driver.

- allowlist.txt: 9,903 tests over both protocols, 129 over native only,
  and 920 known failures in 30 groups (client bugs and shim gaps), each
  with a reproduction.
- clickhouse-version pins the server version and the upstream tests
  together; a bump is an explicit pull request.
- upstream-sql-tests.yml: nightly, manual, and on changes to the
  harness; native and HTTP, 8 shards each.
- README.md and AGENTS.md describe local runs, the triage rules, how to
  grow the allowlist, and how to bump the version.

Refs ClickHouse/integrations#810

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ccoltx
ccoltx requested review from a team, kavirajk and polyglotAI-bot as code owners November 18, 2026 06:23
Comment thread tests/clickhouse-test-runner/runner.go Resolved
Comment thread tests/clickhouse-test-runner/format.go Resolved
@polyglotAI-bot

polyglotAI-bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Summary
Adds a pinned SQL harness for clickhouse-go's native API over TCP and HTTP; the earlier HTTP row-count failure is addressed, while the multiline-comment and mocked-row findings remain.

⚠️ Majors
  • tests/clickhouse-test-runner/split.go:61 Multiline trailing comments become SQL fragments and stop otherwise valid scripts.
  • tests/clickhouse-test-runner/writerows_test.go:55 Stream-error output checks still use prohibited driver mocks, leaving HTTP preservation unproven.
Final verdict

⚠️ Request changes: fix the two Majors.

Automated static review (no build or test run) by Polyglot AI (gpt-6-astra) of 7402883. Requested by @ccoltx. A maintainer can ask for a new review by mentioning @polyglotAI-bot or requesting a review from it.

@polyglotAI-bot polyglotAI-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested in the review of 77a0c5f: see the summary comment and the inline threads.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread tests/clickhouse-test-runner/format.go Resolved
@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

AI review gate — AI review passed

Clean review from: Cursor Bugbot.

- Split the test selector into several arguments of at most 64 KiB.
  Linux limits one argument to 128 KiB, and one selector for the full
  allowlist is about 360 KB, so a run with one shard failed with
  "Argument list too long". Both run scripts share selectors.sh now.
- Read FORMAT as the output-format clause only where it can be one: at
  the end of a query, before an optional SETTINGS clause, and for an
  INSERT only before any SELECT. `SELECT format FROM t` was read as an
  unsupported format, and `INSERT ... SELECT format ...` could take the
  rest of the script as inline data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread tests/clickhouse-test-runner/split.go Resolved
Comment thread tests/clickhouse-test-runner/split.go Resolved
ccoltx and others added 2 commits November 18, 2026 07:47
- Treat only the bare io.EOF sentinel (ServerEndOfStream) as a statement
  without a result. clickhouse-go wraps read failures, such as a
  disconnect or a truncated block, with %w, and errors.Is matched those
  too: a failed query could then look like an empty result.
- hasWord skips ClickHouse `# ` and `#!` comments, as the splitter does.
- Tests for aliases and columns named `format`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The SELECT check of the last commit also rejected
`INSERT ... SELECT ... FROM input(...) FORMAT TSV`, which has inline
data. Instead, read the word after FORMAT: a SQL keyword (FROM, WHERE,
AS, ...) shows an identifier named `format`; any other word is a format
name. hasWord is not used anymore.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread tests/clickhouse-test-runner/split.go Resolved

@polyglotAI-bot polyglotAI-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested in the review of b71abce: see the summary comment and the inline threads.

- A query: a clause keyword after FORMAT (ASC, DESC, ...) shows an
  identifier, so `SELECT 1 AS format ORDER BY format DESC` is not a
  FORMAT DESC clause.
- An INSERT ... SELECT: the FORMAT clause comes after the top-level FROM
  of the SELECT. Before it, `format` is in the select list
  (`SELECT format x FROM s`, `SELECT format DIV 2 FROM s`). GLOBAL, DIV,
  MOD and other operator words are clause keywords too.
- Unit tests for these cases, and live tests that compare the queries
  with the server over both protocols.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@security-ai-gate-reviewer

security-ai-gate-reviewer Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

AI review gate — auto-approval withheld

The AI review gate is green, but the approver did not approve 7402883a: it changes paths that always require a human reviewer: .github/workflows/upstream-sql-tests.yml (commit 77a0c5f has no verified signature; commit 77a0c5f has no verified signature).

What to do: ask someone with write access who is not the PR author to approve this pull request.

History (latest 5 heads, newest first)

  • 7402883a2187 withheld — it changes paths that always require a human reviewer: .github/workflows/upstream-sql-tests.yml (commit 77a0c5f has no verified signature; commit 77a0c5f has no verified signature)
  • 30b9eb86a2de withheld — it changes paths that always require a human reviewer: .github/workflows/upstream-sql-tests.yml (commit 77a0c5f has no verified signature; commit 77a0c5f has no verified signature)
  • 54b10c0700db withheld — it changes paths that always require a human reviewer: .github/workflows/upstream-sql-tests.yml (commit 77a0c5f has no verified signature; commit 77a0c5f has no verified signature)
  • 56fdaf2c8273 withheld — it changes paths that always require a human reviewer: .github/workflows/upstream-sql-tests.yml (commit 77a0c5f has no verified signature; commit 77a0c5f has no verified signature)
  • 01e701f46b0c withheld — it changes paths that always require a human reviewer: .github/workflows/upstream-sql-tests.yml (commit 77a0c5f has no verified signature; commit 77a0c5f has no verified signature)

ccoltx and others added 3 commits November 18, 2026 15:57
A known client bug (the pool race, [native-pool-race-new-connection] in
allowlist.txt) makes random tests with SET fail now and then over the
native protocol: about 40 of 10,000 tests per CI run, a different set
each time. With RETRY_FAILED_ONCE=1, the run script runs the failed
tests once more. A test that passes on the second run is a warning; a
test that fails twice fails the job. CI sets it; local runs that grow or
check the allowlist do not.

Move five tests that fail on almost every CI run into their
known-failure groups: three hit the pool race at the same point each
time, and two hit the HTTP server-hello bug in a LogsQL session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sixteen tests failed in two or more of five CI runs, or twice in one run
with the retry. They hit the pool race ([native-pool-race-new-connection])
too often for a retry to keep CI stable, so they move into that group.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With about 40 race victims per native run, one retry was not enough:
now and then one victim is hit twice (05218_join_use_nulls_lc_key_full_sorting_merge
failed at two different points, both with join_use_nulls lost). The run
script now takes RETRY_FAILED=<n> (CI: 2). A test that passes on a retry
is a warning; a test that fails on every attempt fails the job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@polyglotAI-bot polyglotAI-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested in the review of f0c36b2: see the summary comment and the inline threads.

`SELECT 1 AS format, format x` was read as the clause `FORMAT x`. A
FORMAT clause follows a complete query; when the text before the word
ends with a comma, an opening parenthesis, an operator or a keyword such
as SELECT, the word starts an expression, so it is an identifier. The
same check applies to INSERT inline data.

Over all 190,722 statements of the allowlisted tests, the old and the new
parser give the same result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread tests/clickhouse-test-runner/split.go Resolved
…t-star

startsExpression looked back through whitespace only, so a FORMAT clause
after a block comment (`*/`) or after a select-star (`SELECT * FORMAT
TSV`, `FROM t SELECT * FORMAT TSV`) was read as an identifier. It now
takes the last significant token before the word, outside comments and
quotes. A `*` after SELECT, DISTINCT, a comma, `(` or `.` is a
select-star, which can end a query; another `*` multiplies.

Over all 190,722 statements of the allowlisted tests, the parser gives
the same result as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@polyglotAI-bot polyglotAI-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested in the review of 9381219: see the summary comment and the inline threads.

The invocation log (CLICKHOUSE_CLIENT_CLI_LOG) recorded the raw command
line, so the value of --password went into the file in clear. CI uploads
that file as an artifact when a shard fails, and GitHub does not mask
secrets in artifacts.

redactPassword walks the arguments by the rules of parseArgs and
replaces the value that parseArgs takes as the password by `***`. The
other log records hold no password; the new live test checks the whole
log over both protocols.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@polyglotAI-bot polyglotAI-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested in the review of ad93c4c: see the summary comment and the inline threads.

ccoltx and others added 2 commits November 21, 2026 00:22
If the runner printed a failure for test A and then exited abnormally
(for example killed with 137) before test B ran, the retry of A alone
could make the shard pass, and B never ran. Now an attempt counts only
when every one of its tests has a result line and the runner exited with
0 or 1; otherwise the run fails at once, and the error names the tests
without a result.

The retry logic moves to scripts/retry.sh. retry_test.go runs it with
fake runners: all pass, a flaky test, a broken test, a runner killed
after one failure, an attempt with a missing result, a killed retry, and
a success with a missing result. Without the completeness check, the
three cases with a missing result fail.

Checked with the real runner on the pinned server: a full native shard
(1,251 tests, 1,251 results) and a forced retry of a failing test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- 04056_ast_json_limits fails over HTTP on every attempt in CI: after its
  expected TOO_BIG_AST error, clickhouse-go opens a new HTTP connection,
  and its server-info query runs in the session, where the test set
  max_ast_elements = 15 ("failed to query server hello ... AST is too
  big"). Group [http-server-hello-in-session].
- 05046_trino_dialect_review_regressions failed on all three attempts of
  one native run, at different statements, with the Trino dialect lost.
  Group [native-pool-race-new-connection].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread tests/clickhouse-test-runner/split.go Resolved
Comment thread tests/clickhouse-test-runner/runner.go Resolved

@polyglotAI-bot polyglotAI-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested in the review of a865fd8: see the summary comment and the inline threads.

- The splitter trimmed the end of every statement, also INSERT inline
  data, where a trailing tab or space is data (in TSV, a trailing tab is
  an empty last column). Now inline data loses only the line breaks before
  the empty line that ends it.
- parseSetAssignments read `SET /* c */ max_threads = 3` as the setting
  `/* c */ max_threads`, so the command-line value of max_threads stayed
  on later queries and overrode the SET. Comments outside quotes are now
  dropped before the names and values are read.

Unit tests for both; live tests, over both protocols, that fail without
the fixes: a TSV row with a trailing tab, and `--max_threads=5` with
`SET /* c */ max_threads = 3`. The live tests now take the server ports
from CLICKHOUSE_PORT_TCP and CLICKHOUSE_PORT_HTTP, as the runner does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ccoltx

ccoltx commented Nov 21, 2026

Copy link
Copy Markdown
Contributor Author

@polyglotAI-bot both Majors of your review of a865fd8 are fixed in 01e701f4:

  • split.go: inline INSERT data keeps its trailing spaces and tabs; only the line breaks before the empty line that ends the data go.
  • runner.go: parseSetAssignments drops comments outside quotes before it reads names and values, so SET /* c */ max_threads = 3 replaces the command-line value.

Unit tests for both, and live tests over native and HTTP that fail without the fixes (a TSV row with a trailing tab; --max_threads=5 with SET /* c */ max_threads = 3). Please review again.

Comment thread tests/clickhouse-test-runner/runner.go Resolved

@polyglotAI-bot polyglotAI-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested in the review of 01e701f: see the summary comment and the inline threads.

writeRows kept the rendered rows in a local buffer and wrote them only at
the end, so an exception in a later block (for example an expected
serverError) dropped the rows already received. clickhouse-client prints
them, and the .reference files hold them. Now the rows rendered so far are
written on every return path. A row is rendered apart first, so a row that
fails to render is not written in part.

A live test, over both protocols, checks
`SELECT throwIf(number = 3) FROM numbers(6) SETTINGS max_block_size = 1`
with its serverError hint: 0, 0, 0, then the next statement, as the real
client prints.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ccoltx

ccoltx commented Nov 21, 2026

Copy link
Copy Markdown
Contributor Author

@polyglotAI-bot the Major of your review of 01e701f (rows lost before a later stream error) is fixed in 56fdaf2c: writeRows writes the rows rendered so far on every return path, as clickhouse-client prints the blocks it received before an exception; a row that fails to render is not written in part. A live test over native and HTTP checks SELECT throwIf(number = 3) FROM numbers(6) SETTINGS max_block_size = 1 with its serverError hint: 0, 0, 0, then the next statement, the same as the real client. Please review again.

@polyglotAI-bot polyglotAI-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested in the review of 56fdaf2: see the summary comment and the inline threads.

The live test expected exactly three rows before the exception, but the
server decides how many blocks it sends first (it can squash small
blocks, and over HTTP it buffers the response), so the test could fail
intermittently with a correct shim. The behavior to check is the shim's:
it writes every row that it received before the error. writerows_test.go
checks that with a fake driver.Rows, with no server and no timing. With
the old writeRows, the test gets an empty output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ccoltx

ccoltx commented Nov 21, 2026

Copy link
Copy Markdown
Contributor Author

@polyglotAI-bot the Major of your review of 56fdaf2 (the late-error live test depends on scheduling) is fixed in 54b10c07: the live test is gone. The server decides how many blocks it sends before the exception, so the shim's own behavior (it writes every row it received before the error) is now checked by a unit test with a fake driver.Rows, in writerows_test.go, with no server and no timing. With the old writeRows, that test gets an empty output. Please review again.

Comment thread tests/clickhouse-test-runner/runner.go Resolved
func TestWriteRowsBeforeStreamError(t *testing.T) {
streamErr := errors.New("exception in a later block")
r := &runner{}
err := r.writeRows(&fakeRows{vals: []uint64{0, 1, 2}, err: streamErr}, outputFormat{Name: "TabSeparated", Supported: true, Names: true})

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Major: Stream-error coverage uses mocked driver rows

The stream-error regression now bypasses both protocols and the expected-error output path. It cannot catch failures in their interaction with writeRows.

  • Per .claude/skills/review-pr/review-core.md, tests under /tests/ must not mock driver.Rows; use real ClickHouse coverage.
  • Inferred from the current tests: fakeRows supplies fixed values directly, while TestLiveErrors covers only immediate exceptions.

Replace the doubles with stable live coverage of completed-row preservation and a late exception over TCP and HTTP.

A reply that holds up drops this finding at the next review.

@polyglotAI-bot polyglotAI-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested in the review of 54b10c0: see the summary comment and the inline threads.

- parseUseDatabase could not read `USE /* c */ db`, so the shim stopped
  after the server had run the statement. Comments outside quotes are now
  dropped before the name is read, as for SET.
- The rows-before-error check runs over both protocols again. The server
  decides how many blocks it sends before the exception, and over native
  clickhouse-go can drop some of them (see below), so the live test needs
  some rows, all of them the values of the query, then the next
  statement. writerows_test.go still checks the exact output.
- New known-failure group [native-rows-dropped-before-error]: rows.Next()
  reads the block and error channels in one select, so an error can win
  over blocks that were already received. With the public API, 4 of 20
  runs of a query that fails after 1000 rows gave 970 or 990 rows. No
  allowlisted test is known to fail because of it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ccoltx

ccoltx commented Nov 21, 2026

Copy link
Copy Markdown
Contributor Author

@polyglotAI-bot both Majors of your review of 54b10c0 are addressed in 30b9eb86:

  • USE /* c */ db: parseUseDatabase drops comments outside quotes before it reads the name, as parseSetAssignments does. Unit tests, and a live test over native and HTTP (USE /* c */ system; SELECT currentDatabase();).
  • Rows before a stream error: the check runs over both protocols again (TestLiveRowsBeforeStreamError), and writerows_test.go keeps the exact check. The live test needs some rows, all of them the values of the query, then the next statement. It cannot need an exact count: the server decides how many blocks it sends before the exception, and over native clickhouse-go itself can drop received blocks. rows.Next() reads the block and error channels in one select, so an error can win over blocks already in the channel; with the public API, 4 of 20 runs of a query that fails after 1000 rows gave 970 or 990 rows. That is now the known-failure group [native-rows-dropped-before-error] in allowlist.txt.

Please review again.

newline += pos
}
if newline <= skipWhitespaceAndComments(text, pos) {
pos = newline

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Major: Multiline trailing comments become SQL fragments

A multiline comment after a semicolon stops an otherwise valid script on both protocols.

Inferred from code: SELECT 1; /* note continued */ SELECT 2; advances pos to the newline inside the comment. The next iteration sends continued */ SELECT 2; as SQL, causing a syntax error instead of printing the second result.

Keep block comments intact when consuming the trailing suffix, with parser and live regressions over TCP and HTTP.

A reply that holds up drops this finding at the next review.

@polyglotAI-bot polyglotAI-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested in the review of 30b9eb8: see the summary comment and the inline threads.

Over HTTP, the server can buffer the response and send only the
exception, so zero rows before the error is a correct result there. The
live test now needs at least one row over native only; over both
protocols, every row must be a value of the query, followed by the next
statement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ccoltx

ccoltx commented Nov 21, 2026

Copy link
Copy Markdown
Contributor Author

@polyglotAI-bot about the three Majors of your review of 30b9eb8:

  1. split.go:61, multiline trailing comments: no change, because the shim does what clickhouse-client does. The client also cuts at the first line break after the ;, inside the comment. Checked against clickhouse-client 26.10.4.207 with SELECT 1; /* a\nb */ SELECT 2;: the client prints 1, then fails with Syntax error: failed at position 1 (b): b */ SELECT 2;. The shim gives the same output and the same error (also for /* a\nb */\nSELECT 2; and /* x */ /* y\nz */ SELECT 2;). The .reference files come from that client, so a shim that splits differently would not match them.
  2. live_test.go:243, HTTP rows before a buffered exception: fixed in 7402883a. Over HTTP, zero rows before the error is accepted; over native, at least one row is still needed. Every row must be a value of the query, followed by the next statement.
  3. writerows_test.go, driver mocks: no change. The rule in .claude/CLAUDE.md is "Do not introduce an interface solely to enable test mocking". This test introduces no interface: it implements the public driver.Rows to check the shim's own output. The live test covers both protocols, and it cannot check an exact count: the server decides the blocks before the exception, and over native the driver can drop received blocks ([native-rows-dropped-before-error]). Your review of 56fdaf2 rejected a live test with an exact count for that reason, so the exact check stays deterministic, in the unit test.

Please review again.

@polyglotAI-bot polyglotAI-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested in the review of 7402883: see the summary comment and the inline threads.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants