Skip to content

panic: makeslice: len out of range when reading JSON column with deprecated v0 wire format #1854

Open

Description

@amalakar

Observed

  1. Create a JSON column and insert 4 rows with heterogeneous value types across the same JSON
    paths (e.g. latency_ms is an array in one row, a string in another).
  2. SELECT the column back using the Native protocol with clickhouse-go v2.
  3. The driver panics or returns a wire decode error:
    • panic: runtime error: makeslice: len out of range (in production with large datasets)
    • read data: failed to decode block: unexpected value N for boolean
    • clickhouse [process]: unexpected packet N

Expected behaviour

The SELECT should return all rows without error.

Code example

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/ClickHouse/clickhouse-go/v2"
)

// rows contains heterogeneous JSON documents. The same key appears with
// different value types across rows (Int64, String, Array, Bool, Float64,
// nested object, timestamp) so that each JSON path acquires multiple
// inferred CH types — the condition that triggers the discriminator
// index shift in the v0 deprecated decoder.
var rows = []string{
	`{"latency_ms":[]}`,
	`{"cpu_pct":1,"region":"x","request_id":1,"response_times":[],"service":"x","tags":[]}`,
	`{"counts":[],"created_at":"x","ended_at":"x","errors":[],"labels":[],"span_ids":[],"started_at":"x","trace_id":"x","user_id":"x"}`,
	`{"attachments":[],"config":{"host":"host-149","nested":{"extra":null,"name":"svc-147","parent":null},"value":null},"context":{"name":"ctx-273","parent":null,"root":false},"durations":[],"logged_at":"x","metadata":{"active":true,"ids":[null,null],"ts":"2026-05-04T15:18:01Z"},"metrics":{"primary":{"count":542265,"enabled":true,"parent":null},"samples":[18485,18893],"secondary":{"extra":null,"label":"label-166","parent":null}},"org_id":"x","stack_frames":[],"updated_at":"x"}`,
}

func main() {
	conn, err := clickhouse.Open(&clickhouse.Options{
		Addr:     []string{"127.0.0.1:9002"},
		Auth:     clickhouse.Auth{Database: "default", Username: "default", Password: "password"},
		Protocol: clickhouse.Native,
	})
	if err != nil {
		log.Fatal(err)
	}
	defer conn.Close()

	ctx := context.Background()
	if err := conn.Ping(ctx); err != nil {
		log.Fatalf("ping: %v", err)
	}

	exec := func(q string) {
		if err := conn.Exec(ctx, q); err != nil {
			log.Fatalf("%s\n  %v", q, err)
		}
	}
	exec("DROP TABLE IF EXISTS test_json_table")
	exec("CREATE TABLE test_json_table (raw JSON(max_dynamic_paths=32)) ENGINE = MergeTree ORDER BY tuple()")

	batch, err := conn.PrepareBatch(ctx, "INSERT INTO test_json_table (raw)")
	if err != nil {
		log.Fatal(err)
	}
	for _, row := range rows {
		if err := batch.Append(row); err != nil {
			log.Fatal(err)
		}
	}
	if err := batch.Send(); err != nil {
		log.Fatal(err)
	}

	result, err := conn.Query(ctx, "SELECT raw FROM test_json_table SETTINGS max_memory_usage=0")
	if err != nil {
		log.Fatal(err)
	}
	defer result.Close()

	var n int
	for result.Next() {
		var raw clickhouse.JSON
		if err := result.Scan(&raw); err != nil {
			log.Fatal(err)
		}
		n++
	}
	if err := result.Err(); err != nil {
		log.Fatal(err)
	}
	fmt.Printf("read %d rows OK\n", n)
}

Run with:

docker run --rm -p 9002:9000 -p 8123:8123 \
  -e CLICKHOUSE_PASSWORD=password \
  clickhouse/clickhouse-server:26.2

go run .

Error log

read data: failed to decode block from 127.0.0.1:9002 (conn_id=1, compression=none): unexpected value 97 for boolean

In production with larger datasets the panic form is observed:

panic: runtime error: makeslice: len out of range

goroutine 120050 [running]:
  github.com/ClickHouse/clickhouse-go/v2/lib/column.(*JSON).decodeObjectData_v1
    lib/column/json_deprecated.go:111
  github.com/ClickHouse/clickhouse-go/v2/lib/column.(*JSON).Decode
    lib/column/json.go:768
  github.com/ClickHouse/clickhouse-go/v2/lib/column.(*Dynamic).decodeData_v1
    lib/column/dynamic_deprecated.go:174
  github.com/ClickHouse/clickhouse-go/v2/lib/column.(*Dynamic).Decode
  github.com/ClickHouse/clickhouse-go/v2/lib/column.(*Array).Decode
    lib/column/array.go:227
  github.com/ClickHouse/clickhouse-go/v2/lib/proto.(*Block).Decode
    lib/proto/block.go:232
  github.com/ClickHouse/clickhouse-go/v2.(*connect).readData
    conn_process.go:107

Root cause (hypothesis)

ClickHouse 26.2 sends JSON column data in the deprecated v0 wire format
(output_format_native_use_flattened_dynamic_and_json_serialization=0 by default). In this format
each JSON path with multiple inferred types is encoded as a Dynamic column with N variant types.

Dynamic.decodeHeader_v1 reads the N type names from the wire, then inserts "SharedVariant" and
re-sorts the list alphabetically. This shifts the driver's discriminator→column index mapping
relative to what the server wrote. For example, if a path has variants ["Array(Int64)", "String"],
the server uses discriminator 1 = String, but the driver maps discriminator 1 → SharedVariant
(which sorts between "Array..." and "String..."). Every subsequent column read is then offset,
causing the decoder to consume the wrong bytes — producing a makeslice panic, an unexpected-boolean
error, or an unexpected-packet error depending on what those bytes look like to the next decoder.

Details

Environment

  • clickhouse-go version: v2.45.0
  • Interface: ClickHouse API (clickhouse.Open)
  • Go version: 1.26.0
  • Operating system: macOS / darwin arm64
  • ClickHouse version: 26.2.15.4
  • Is it a ClickHouse Cloud? No (also reproduces on self-hosted)
  • ClickHouse Server non-default settings: none
    (output_format_native_use_flattened_dynamic_and_json_serialization=0 is the default)
  • CREATE TABLE statement:
CREATE TABLE test_json_table (raw JSON(max_dynamic_paths=32))
ENGINE = MergeTree ORDER BY tuple()

Activity

  1. abonander commented on May 5, 2026

    @abonander
    panic: runtime error: makeslice: len out of range
    
    goroutine 120050 [running]:
      github.com/ClickHouse/clickhouse-go/v2/lib/column.(*JSON).decodeObjectData_v1
        lib/column/json_deprecated.go:111
    

    Are you certain this is the correct stack trace for this panic? There's no make() call at that line to produce this panic:

    err := col.Decode(reader, rows)

  2. amalakar commented on May 5, 2026

    @amalakar
    Author

    @abonander there is a possibility the stack trace is not accurate for the issue I am describing. I tried to piece together what I saw in production logs, vs what I can reproduce.

  3. abonander commented on May 5, 2026

    @abonander

    When running in a debugger, I found the boolean error is in decodeBlockInfo() here:

    if _, err := reader.Bool(); err != nil {
    return err
    }

    Not really sure what that means, yet. It seems like a prior packet may not have been fully consumed and we're reading junk data.

  4. ShamsRKhan commented on Aug 13, 2026

    @ShamsRKhan

    We hit this in production-shaped testing, root-caused it in the driver source, and have deterministic minimal repros for the two routes that trigger it. Posting here rather than opening a new issue because the symptoms in this thread — makeslice: len out of range, unexpected value N for boolean, unexpected packet N, and the "junk data" a commenter mentions — all fall out of the same two gaps.

    Environment: server 26.3.17, driver v2.48.0, native protocol.

    Root cause

    (a) lib/column/json_deprecated.go — shared data is skipped, desyncing the stream.

    After decoding typed and dynamic paths, Decode reads only the outer offsets and never consumes the shared-data payload:

    // SharedData per row, ignored for now. May cause stream offset issues if present
    _, err := reader.ReadRaw(8 * rows) // one UInt64 per row

    That comment is accurate. One UInt64 per row is the offsets array; the values it points at are left in the stream. Every column decoded after the JSON column in that block then reads from the wrong position. That is the mechanism behind unexpected packet N and the makeslice panic in this thread — a length field parsed out of misaligned bytes is arbitrary, so where you land depends on what follows the JSON column in the block.

    (b) lib/column/sharedvariant.go — SharedVariant is a String passthrough.

    // SharedVariant deprecated. Use Dynamic/JSON serialization version 3.
    type SharedVariant struct {
    	name       string
    	stringData String
    }

    Every method delegates to stringData. So any value stored as SharedVariant — which happens when a path exceeds max_dynamic_types — is handed to Go as its raw type-tagged binary inside a string, with no error from Query, Scan, or rows.Err().

    This is the nastier of the two. No panic, no failed query, just wrong values. We observed "\x0e\x00\x00\x00\x00\x00\x00&@" — a 0x0E type tag followed by little-endian float64 11.0 — being served as a value on a live dashboard.

    Reproduction

    Plain Go, no deps beyond the driver. R2 runs before R1 because R1's stream desync can leave the pooled connection broken.

    package main
    
    import (
    	"context"
    	"fmt"
    	"log"
    	"strings"
    
    	"github.com/ClickHouse/clickhouse-go/v2"
    	"github.com/ClickHouse/clickhouse-go/v2/lib/chcol"
    )
    
    func main() {
    	ctx := context.Background()
    	conn, err := clickhouse.Open(&clickhouse.Options{
    		Addr: []string{"localhost:9000"},
    		Auth: clickhouse.Auth{Database: "default", Username: "default"},
    	})
    	if err != nil {
    		log.Fatal(err)
    	}
    	defer conn.Close()
    
    	exec := func(q string) {
    		if err := conn.Exec(ctx, q); err != nil {
    			log.Fatalf("setup: %s: %v", q, err)
    		}
    	}
    
    	// ---- R2: max_dynamic_types exhaustion -> SharedVariant --------------
    	// Deterministic: one part, one INSERT, no merges involved. 'satellites'
    	// holds Int64 and Float64 against a type budget of 1, so one of them is
    	// stored as SharedVariant.
    	fmt.Println("=== R2: SharedVariant (silent raw bytes) ===")
    	exec(`DROP TABLE IF EXISTS r2`)
    	exec(`CREATE TABLE r2 (id UInt64, doc JSON(max_dynamic_paths = 8, max_dynamic_types = 1))
    	      ENGINE = MergeTree ORDER BY id`)
    	exec(`INSERT INTO r2 VALUES (1, '{"satellites": 11}'), (2, '{"satellites": 12.5}')`)
    
    	// Control: the server has the values, and knows both types.
    	ctl, err := conn.Query(ctx, `SELECT toString(getSubcolumn(doc,'satellites')),
    	                                    dynamicType(getSubcolumn(doc,'satellites'))
    	                             FROM r2 ORDER BY id`)
    	if err != nil {
    		log.Fatal(err)
    	}
    	for ctl.Next() {
    		var v, t string
    		if err := ctl.Scan(&v, &t); err != nil {
    			log.Fatal(err)
    		}
    		fmt.Printf("  server: %-6s %s\n", v, t)
    	}
    	ctl.Close()
    
    	// (i) subcolumn read
    	sub, err := conn.Query(ctx, `SELECT getSubcolumn(doc,'satellites') FROM r2 ORDER BY id`)
    	if err != nil {
    		log.Fatal(err)
    	}
    	for i := 0; sub.Next(); i++ {
    		var v chcol.Dynamic
    		if err := sub.Scan(&v); err != nil { // never fires
    			fmt.Printf("  subcolumn row %d: scan error %v\n", i, err)
    			break
    		}
    		fmt.Printf("  subcolumn row %d: %T %q\n", i, v.Any(), fmt.Sprint(v.Any()))
    	}
    	if err := sub.Err(); err != nil { // never fires
    		fmt.Printf("  subcolumn rows.Err: %v\n", err)
    	}
    	sub.Close()
    
    	// (ii) whole-column read
    	whole, err := conn.Query(ctx, `SELECT doc FROM r2 ORDER BY id`)
    	if err != nil {
    		fmt.Printf("  whole-column: query error %v\n", err)
    	} else {
    		for i := 0; whole.Next(); i++ {
    			var d chcol.JSON
    			if err := whole.Scan(&d); err != nil {
    				fmt.Printf("  whole-column row %d: scan error %v\n", i, err)
    				break
    			}
    			v := d.ValuesByPath()["satellites"]
    			if dyn, ok := v.(chcol.Dynamic); ok {
    				v = dyn.Any()
    			}
    			fmt.Printf("  whole-column row %d: %T %q\n", i, v, fmt.Sprint(v))
    		}
    		if err := whole.Err(); err != nil {
    			fmt.Printf("  whole-column rows.Err: %v\n", err)
    		}
    		whole.Close()
    	}
    	fmt.Println("  EXPECTED: int64(11) and float64(12.5) from both reads")
    
    	// ---- R1: max_dynamic_paths overflow -> shared data ------------------
    	// 12 distinct paths into an 8-path budget. Runs last: its stream desync
    	// can leave the pooled connection broken.
    	fmt.Println("\n=== R1: shared data (stream desync) ===")
    	var b strings.Builder
    	b.WriteString("{")
    	for i := 0; i < 11; i++ {
    		fmt.Fprintf(&b, `"k%02d":%d,`, i, i)
    	}
    	b.WriteString(`"satellites":11}`)
    
    	exec(`DROP TABLE IF EXISTS r1`)
    	exec(`CREATE TABLE r1 (id UInt64, doc JSON(max_dynamic_paths = 8))
    	      ENGINE = MergeTree ORDER BY id`)
    	exec(fmt.Sprintf(`INSERT INTO r1 (id, doc) VALUES (1, '%s')`, b.String()))
    
    	rows, err := conn.Query(ctx, `SELECT doc FROM r1`)
    	if err != nil {
    		fmt.Printf("  OBSERVED: %v\n", err)
    	} else {
    		for rows.Next() {
    			var d chcol.JSON
    			if err := rows.Scan(&d); err != nil {
    				fmt.Printf("  OBSERVED: scan error %v\n", err)
    				break
    			}
    			fmt.Printf("  decoded: %v\n", d.ValuesByPath())
    		}
    		if err := rows.Err(); err != nil {
    			fmt.Printf("  OBSERVED: rows.Err %v\n", err)
    		}
    		rows.Close()
    	}
    	fmt.Println("  EXPECTED: the row decodes with all 12 paths")
    
    	// In some runs the connection is left broken: an unrelated query fails too.
    	if err := conn.Exec(ctx, `SELECT 1`); err != nil {
    		fmt.Printf("  OBSERVED: unrelated query on same conn now fails: %v\n", err)
    	}
    }

    Observed

    Verbatim output of the program above (server 26.3.17, driver v2.48.0):

    R2, no error anywhere on the path:

    server: 11     Int64
    server: 12.5   Float64
    
    subcolumn row 0: int64 "11"                                <- correct
    subcolumn row 1: string "\x0e\x00\x00\x00\x00\x00\x00)@"   <- 0x0E tag + LE float64 12.5
    
    whole-column row 0: int64 "11"
    whole-column row 1: string "\x0e\x00\x00\x00\x00\x00\x00)@"
    

    The value that lost the dynamic-type slot arrives as raw bytes; its neighbor decodes correctly — same path, same column, same result set — so a single result set mixes correct values and garbage, and nothing distinguishes them at the call site.

    R1:

    decoded: map[k00:{0 Int64} k01:{1 Int64} ... k07:{7 Int64}]
    OBSERVED: rows.Err clickhouse [process]: unexpected packet 57
    

    Four of the twelve paths (k08–k10, satellites) are silently ABSENT from the decoded row — the shared-data overflow — and the stream then desyncs at rows.Err. Which face R1 shows varies with block layout: across variations of this shape we have also seen the query fail outright with unexpected packet 4 and the pooled connection stay broken for subsequent unrelated queries. Single part, single row, no FINAL, no type mixing needed for any of them.

    One more data point on severity: in production-shaped testing we watched a live dashboard serve the raw-bytes values for a while, and the symptom later vanished on its own after background merges reorganised the parts. Readability tracks per-part storage layout, so the same query over the same logical data can return correct values or garbage depending on when it runs. That is a large part of why this is hard to catch in testing.

    Master

    Both files are byte-identical on master — checked against pseudo-version v2.48.1-0.20260812104646-3a8dafcff5e2. Neither gap is fixed there.

    The wire format is decodable

    clickhouse-connect (Python) fully decodes both encodings — see datatypes/binary_value.py. So this is a driver gap rather than anything missing server-side, and there is a working reference implementation to check against.

    Two asks

    1. Default output_format_native_use_flattened_dynamic_and_json_serialization on when the server is >= 25.6. We verified end-to-end that it avoids both routes. That closes the practical problem for most users without waiting on a decoder.
    2. Until SharedVariant decoding is implemented, make it error loudly rather than pass raw bytes through as a string. Root cause (a) at least announces itself. Root cause (b) does not, and silent corruption is strictly worse than a failed query — we shipped wrong numbers to a dashboard for a while without a single error in the logs.

    Happy to test patches against our reproduction, or to open a separate issue for either root cause if you would rather track them apart.

  5. polyglotAI-bot commented on Nov 14, 2026

    @polyglotAI-bot
    Collaborator

    Recommendation: Keep open — fix needed
    The V1 JSON decoder has two defects. Open PR #2004 fixes the first one (the skipped shared data that breaks the stream), but not the second one (SharedVariant values come back as raw bytes, with no error); the review on #2004 requests changes. Next step: finish #2004 and fix the SharedVariant decoding too (in #2004 or in a separate PR). #2114 can be closed as a duplicate of this issue.

    Analysis: This client sends protocol revision 54460 (lib/proto/const.go:23), so when output_format_native_use_flattened_dynamic_and_json_serialization is 0 (the default), the server sends JSON with the V1 object serialization. Defect 1: decodeObjectData_v1 reads only the shared-data offsets and leaves the paths and values in the stream (lib/column/json_deprecated.go:117-118), so all later data is read from the wrong position (cases A and C). The reporter's rows have 33 paths for max_dynamic_paths=32, so 1 path goes to the shared data. Defect 2: SharedVariant is a String passthrough (lib/column/sharedvariant.go:10-13), so a value that is over max_dynamic_types comes back as its type-tagged binary encoding, with no error (case B: 0x0e is the Float64 tag, then 12.5 as little-endian bytes). The discriminator-shift hypothesis in the report does not agree with the run: case D has one path with four types and no shared data, and it decodes correctly, so the variant order after the sort in lib/column/dynamic_deprecated.go:108-109 agrees with the server. A workaround: set output_format_native_use_flattened_dynamic_and_json_serialization=1 in Options.Settings; then all cases decode correctly.

    Reproduction: complete program and output
    package main
    
    import (
    	"context"
    	"encoding/json"
    	"fmt"
    	"os"
    	"strings"
    
    	"github.com/ClickHouse/clickhouse-go/v2"
    	"github.com/ClickHouse/clickhouse-go/v2/lib/chcol"
    )
    
    // The reporter's four rows: together they have more distinct paths than max_dynamic_paths=32.
    var reporterRows = []string{
    	`{"latency_ms":[]}`,
    	`{"cpu_pct":1,"region":"x","request_id":1,"response_times":[],"service":"x","tags":[]}`,
    	`{"counts":[],"created_at":"x","ended_at":"x","errors":[],"labels":[],"span_ids":[],"started_at":"x","trace_id":"x","user_id":"x"}`,
    	`{"attachments":[],"config":{"host":"host-149","nested":{"extra":null,"name":"svc-147","parent":null},"value":null},"context":{"name":"ctx-273","parent":null,"root":false},"durations":[],"logged_at":"x","metadata":{"active":true,"ids":[null,null],"ts":"2026-05-04T15:18:01Z"},"metrics":{"primary":{"count":542265,"enabled":true,"parent":null},"samples":[18485,18893],"secondary":{"extra":null,"label":"label-166","parent":null}},"org_id":"x","stack_frames":[],"updated_at":"x"}`,
    }
    
    const flat = "output_format_native_use_flattened_dynamic_and_json_serialization"
    
    func open(flattened bool) clickhouse.Conn {
    	addr := os.Getenv("CLICKHOUSE_ADDR")
    	if addr == "" {
    		addr = "localhost:9000"
    	}
    	opts := &clickhouse.Options{Addr: []string{addr}, Protocol: clickhouse.Native}
    	if flattened {
    		opts.Settings = clickhouse.Settings{flat: 1}
    	}
    	conn, err := clickhouse.Open(opts)
    	if err != nil {
    		panic(err)
    	}
    	return conn
    }
    
    func exec(conn clickhouse.Conn, q string) {
    	if err := conn.Exec(context.Background(), q); err != nil {
    		panic(fmt.Sprintf("%s: %v", q, err))
    	}
    }
    
    func setup(conn clickhouse.Conn) {
    	ctx := context.Background()
    	exec(conn, "DROP TABLE IF EXISTS issue1854_a")
    	exec(conn, "CREATE TABLE issue1854_a (raw JSON(max_dynamic_paths=32)) ENGINE = MergeTree ORDER BY tuple()")
    	batch, err := conn.PrepareBatch(ctx, "INSERT INTO issue1854_a (raw)")
    	if err != nil {
    		panic(err)
    	}
    	for _, r := range reporterRows {
    		if err := batch.Append(r); err != nil {
    			panic(err)
    		}
    	}
    	if err := batch.Send(); err != nil {
    		panic(err)
    	}
    
    	// SharedVariant: one type slot per path, two types for path "satellites".
    	exec(conn, "DROP TABLE IF EXISTS issue1854_b")
    	exec(conn, "CREATE TABLE issue1854_b (id UInt64, doc JSON(max_dynamic_paths=8, max_dynamic_types=1)) ENGINE = MergeTree ORDER BY id")
    	exec(conn, `INSERT INTO issue1854_b VALUES (1, '{"satellites": 11}'), (2, '{"satellites": 12.5}')`)
    
    	// Shared data: 12 paths, 8-path budget, and a column after the JSON column.
    	var b strings.Builder
    	b.WriteString("{")
    	for i := 0; i < 11; i++ {
    		fmt.Fprintf(&b, `"k%02d":%d,`, i, i)
    	}
    	b.WriteString(`"satellites":11}`)
    	exec(conn, "DROP TABLE IF EXISTS issue1854_c")
    	exec(conn, "CREATE TABLE issue1854_c (id UInt64, doc JSON(max_dynamic_paths=8), tail String) ENGINE = MergeTree ORDER BY id")
    	exec(conn, fmt.Sprintf(`INSERT INTO issue1854_c VALUES (1, '%s', 'tail-1')`, b.String()))
    
    	// No shared data: default path budget, one path with three types (Array, String, Int64).
    	exec(conn, "DROP TABLE IF EXISTS issue1854_d")
    	exec(conn, "CREATE TABLE issue1854_d (raw JSON) ENGINE = MergeTree ORDER BY tuple()")
    	for _, r := range append([]string{`{"latency_ms":[1,2]}`, `{"latency_ms":"x"}`, `{"latency_ms":7}`}, reporterRows...) {
    		exec(conn, fmt.Sprintf("INSERT INTO issue1854_d VALUES ('%s')", r))
    	}
    	exec(conn, "OPTIMIZE TABLE issue1854_d FINAL")
    }
    
    func caseA(conn clickhouse.Conn) string {
    	rows, err := conn.Query(context.Background(), "SELECT raw FROM issue1854_a")
    	if err != nil {
    		return "query error: " + err.Error()
    	}
    	defer rows.Close()
    	n, paths := 0, 0
    	for rows.Next() {
    		var raw clickhouse.JSON
    		if err := rows.Scan(&raw); err != nil {
    			return "scan error: " + err.Error()
    		}
    		n++
    		paths += len(raw.ValuesByPath())
    	}
    	if err := rows.Err(); err != nil {
    		return fmt.Sprintf("rows=%d rows.Err: %v", n, err)
    	}
    	return fmt.Sprintf("rows=%d paths=%d OK", n, paths)
    }
    
    func caseD(conn clickhouse.Conn) string {
    	rows, err := conn.Query(context.Background(), "SELECT raw.latency_ms FROM issue1854_d WHERE raw.latency_ms IS NOT NULL ORDER BY toString(raw.latency_ms)")
    	if err != nil {
    		return "query error: " + err.Error()
    	}
    	defer rows.Close()
    	var out []string
    	for rows.Next() {
    		var v chcol.Dynamic
    		if err := rows.Scan(&v); err != nil {
    			return "scan error: " + err.Error()
    		}
    		js, err := json.Marshal(v.Any())
    		if err != nil {
    			return "marshal error: " + err.Error()
    		}
    		out = append(out, fmt.Sprintf("%s %s", v.Type(), js))
    	}
    	if err := rows.Err(); err != nil {
    		return "rows.Err: " + err.Error()
    	}
    	return strings.Join(out, ", ")
    }
    
    func caseB(conn clickhouse.Conn) string {
    	rows, err := conn.Query(context.Background(), "SELECT getSubcolumn(doc, 'satellites') FROM issue1854_b ORDER BY id")
    	if err != nil {
    		return "query error: " + err.Error()
    	}
    	defer rows.Close()
    	var out []string
    	for rows.Next() {
    		var v chcol.Dynamic
    		if err := rows.Scan(&v); err != nil {
    			return "scan error: " + err.Error()
    		}
    		out = append(out, fmt.Sprintf("%T %q", v.Any(), fmt.Sprint(v.Any())))
    	}
    	if err := rows.Err(); err != nil {
    		return "rows.Err: " + err.Error()
    	}
    	return strings.Join(out, ", ")
    }
    
    func caseC(conn clickhouse.Conn) string {
    	rows, err := conn.Query(context.Background(), "SELECT doc, tail FROM issue1854_c")
    	if err != nil {
    		return "query error: " + err.Error()
    	}
    	defer rows.Close()
    	var out []string
    	for rows.Next() {
    		var d chcol.JSON
    		var tail string
    		if err := rows.Scan(&d, &tail); err != nil {
    			return "scan error: " + err.Error()
    		}
    		out = append(out, fmt.Sprintf("paths=%d tail=%q", len(d.ValuesByPath()), tail))
    	}
    	if err := rows.Err(); err != nil {
    		return strings.Join(out, ", ") + " rows.Err: " + err.Error()
    	}
    	return strings.Join(out, ", ")
    }
    
    func run(name string, flattened bool, f func(clickhouse.Conn) string) {
    	conn := open(flattened)
    	defer conn.Close()
    	defer func() {
    		if r := recover(); r != nil {
    			fmt.Printf("%-44s flattened=%d -> PANIC: %v\n", name, map[bool]int{false: 0, true: 1}[flattened], r)
    		}
    	}()
    	fmt.Printf("%-44s flattened=%d -> %s\n", name, map[bool]int{false: 0, true: 1}[flattened], f(conn))
    }
    
    func main() {
    	conn := open(false)
    	var version string
    	if err := conn.QueryRow(context.Background(), "SELECT version()").Scan(&version); err != nil {
    		panic(err)
    	}
    	fmt.Println("server", version)
    	setup(conn)
    	conn.Close()
    	for _, flattened := range []bool{false, true} {
    		run("A reporter rows: SELECT raw", flattened, caseA)
    		run("B max_dynamic_types=1: SELECT satellites", flattened, caseB)
    		run("C 12 paths into 8: SELECT doc, tail", flattened, caseC)
    		run("D no shared data: SELECT raw.latency_ms", flattened, caseD)
    	}
    }

    Output on main (d124d2b0), ClickHouse 26.9.1.1629:

    server 26.9.1.1629
    A reporter rows: SELECT raw                  flattened=0 -> rows=4 rows.Err: read data: failed to decode block from … (conn_id=1, compression=none): unexpected value 51 for boolean
    B max_dynamic_types=1: SELECT satellites     flattened=0 -> int64 "11", string "\x0e\x00\x00\x00\x00\x00\x00)@"
    C 12 paths into 8: SELECT doc, tail          flattened=0 ->  rows.Err: read data: failed to decode block from … (conn_id=1, compression=none): clickhouse: unsupported column type "k09"
    D no shared data: SELECT raw.latency_ms      flattened=0 -> Int64 7, Array(Nullable(Int64)) [1,2], Array(Nullable(String)) [], String "x"
    A reporter rows: SELECT raw                  flattened=1 -> rows=4 paths=132 OK
    B max_dynamic_types=1: SELECT satellites     flattened=1 -> int64 "11", float64 "12.5"
    C 12 paths into 8: SELECT doc, tail          flattened=1 -> paths=12 tail="tail-1"
    D no shared data: SELECT raw.latency_ms      flattened=1 -> Int64 7, Array(Nullable(Int64)) [1,2], Array(Nullable(String)) [], String "x"
    

    Output on main with the diff of PR #2004 applied:

    server 26.9.1.1629
    A reporter rows: SELECT raw                  flattened=0 -> rows=4 paths=129 OK
    B max_dynamic_types=1: SELECT satellites     flattened=0 -> int64 "11", string "\x0e\x00\x00\x00\x00\x00\x00)@"
    C 12 paths into 8: SELECT doc, tail          flattened=0 -> paths=12 tail="tail-1"
    D no shared data: SELECT raw.latency_ms      flattened=0 -> Int64 7, Array(Nullable(Int64)) [1,2], Array(Nullable(String)) [], String "x"
    A reporter rows: SELECT raw                  flattened=1 -> rows=4 paths=132 OK
    B max_dynamic_types=1: SELECT satellites     flattened=1 -> int64 "11", float64 "12.5"
    C 12 paths into 8: SELECT doc, tail          flattened=1 -> paths=12 tail="tail-1"
    D no shared data: SELECT raw.latency_ms      flattened=1 -> Int64 7, Array(Nullable(Int64)) [1,2], Array(Nullable(String)) [], String "x"
    

    Automated triage by Polyglot AI. A maintainer makes the final decision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions