Skip to content

ci: Run CodSpeed with GCC 15 on Ubuntu 26.04 - #1798

Merged
chfast merged 1 commit into
masterfrom
ci/codspeed-gcc15
Nov 3, 2026
Merged

chfast merged 1 commit into
masterfrom
ci/codspeed-gcc15

Conversation

@chfast

@chfast chfast commented Nov 1, 2026

Copy link
Copy Markdown
Member

The CodSpeed benchmarks were built with GCC 14, the newest GCC on the ubuntu-24.04 image, while performance changes are evaluated with GCC 15 and 16. The ubuntu-26.04 image is generally available, CodSpeed ships its instrumented Valgrind for it, and its default GCC is 15.2. GCC 16 is only a pre-release snapshot there.

@codecov

codecov Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.24%. Comparing base (07f5d5a) to head (36de88c).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1798      +/-   ##
==========================================
+ Coverage   97.77%   98.24%   +0.46%     
==========================================
  Files         186      186              
  Lines       21613    17172    -4441     
  Branches     3922     3922              
==========================================
- Hits        21132    16870    -4262     
+ Misses        412      233     -179     
  Partials       69       69              
Flag Coverage Δ
eest-develop-gmp 26.96% <ø> (ø)
eest-legacy 17.23% <ø> (ø)
eest-libsecp256k1 29.10% <ø> (ø)
eest-stable 84.23% <ø> (ø)
evmone-unittests 94.11% <ø> (-0.39%) ⬇️
riscv64 94.82% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
core 96.62% <ø> (+0.86%) ⬆️
tooling 94.87% <ø> (+0.83%) ⬆️
tests 99.82% <ø> (-0.01%) ⬇️
see 178 files with indirect coverage changes
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@chfast
chfast force-pushed the ci/codspeed-gcc15 branch from 7917ca8 to 7cb1aba Compare November 1, 2026 18:49
@codspeed

codspeed Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will regress 57 benchmarks

2 warnings

⚠️ 537 benchmarks with a significant change ran on different hardware

Their base and head results were measured on machines with a different CPU, core count or memory, so the changes might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

⚠️ 720 benchmarks with a significant change ran in a different software environment

Their base and head results were measured with a different OS or toolchain, such as compiler, runtime or library versions, so the changes might come from this difference rather than from the code. Open an affected benchmark to see what changed.

See affected benchmarks

⚡ 663 improved benchmarks
❌ 57 regressed benchmarks
✅ 179 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
❌ precompile<ecrecover, ecrecover_execute_libsecp256k1> 3 ms 3.3 ms -9.31%
❌ test_ecrecover[fork_Osaka-blockchain_test-ecrecover-benchmark-gas-value_1M] 90.2 ms 98.9 ms -8.76%
❌ precompile<ecpairing, ecpairing_execute> 256.5 ms 276.5 ms -7.23%
❌ test_alt_bn128_benchmark[fork_Osaka-blockchain_test-num_pairs_1-benchmark-gas-value_1M] 97.4 ms 104.9 ms -7.11%
❌ test_alt_bn128[fork_Osaka-blockchain_test-bn128_one_pairing-benchmark-gas-value_1M] 97.5 ms 104.9 ms -7.1%
❌ test_alt_bn128[fork_Osaka-blockchain_test-bn128_two_pairings-benchmark-gas-value_1M] 89 ms 95.6 ms -6.91%
❌ test_alt_bn128[fork_Osaka-blockchain_test-ec_pairing_2_pair-benchmark-gas-value_1M] 89 ms 95.6 ms -6.91%
❌ test_alt_bn128_benchmark[fork_Osaka-blockchain_test-num_pairs_3-benchmark-gas-value_1M] 84.8 ms 91 ms -6.79%
❌ test_alt_bn128[fork_Osaka-blockchain_test-ec_pairing_4_pair-benchmark-gas-value_1M] 85.5 ms 91.7 ms -6.76%
❌ test_alt_bn128[fork_Osaka-blockchain_test-ec_pairing_5_pair-benchmark-gas-value_1M] 80.5 ms 86.3 ms -6.68%
❌ test_alt_bn128_benchmark[fork_Osaka-blockchain_test-num_pairs_12-benchmark-gas-value_1M] 82.2 ms 87.9 ms -6.58%
❌ test_alt_bn128_benchmark[fork_Osaka-blockchain_test-num_pairs_6-benchmark-gas-value_1M] 69.8 ms 74.7 ms -6.56%
❌ test_alt_bn128_benchmark[fork_Osaka-blockchain_test-num_pairs_24-benchmark-gas-value_1M] 77.3 ms 82.7 ms -6.47%
❌ precompile<ecmul, ecmul_execute> 4.5 ms 4.8 ms -5.64%
❌ test_arithmetic[fork_Osaka-blockchain_test-opcode_MUL--benchmark-gas-value_1M] 12.4 ms 13.1 ms -5.61%
❌ test_alt_bn128[fork_Osaka-blockchain_test-bn128_mul_1_2_32_byte_scalar-benchmark-gas-value_1M] 75.5 ms 79.8 ms -5.35%
❌ test_alt_bn128[fork_Osaka-blockchain_test-bn128_mul_32_byte_coord_and_scalar-benchmark-gas-value_1M] 75.7 ms 80 ms -5.34%
❌ test_alt_bn128[fork_Osaka-blockchain_test-bn128_mul-benchmark-gas-value_1M] 74.6 ms 78.8 ms -5.33%
❌ modexp<expmod_execute_evmone>[mod_len:32/mod_tz:127/exp_bits:256] 99.1 µs 102.8 µs -3.64%
❌ test_calldatacopy_from_origin[fork_Osaka-blockchain_test-calldata_size_32-fixed_src_dst_True-mem_size_1024-benchmark-gas-value_1M] 11.5 ms 11.9 ms -3.15%
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ci/codspeed-gcc15 (36de88c) with master (07f5d5a)

Open in CodSpeed

The CodSpeed benchmarks were built with GCC 14, the newest GCC on the
ubuntu-24.04 image, while performance changes are evaluated with GCC
15 and 16. The ubuntu-26.04 image is generally available, CodSpeed
ships its instrumented Valgrind for it, and its default GCC is 15.2.
GCC 16 is only a pre-release snapshot there.
@chfast
chfast force-pushed the ci/codspeed-gcc15 branch from e3ac62e to 36de88c Compare November 3, 2026 09:49
@chfast
chfast merged commit 7ca3b5a into master Nov 3, 2026
31 of 32 checks passed
@chfast
chfast deleted the ci/codspeed-gcc15 branch November 3, 2026 11:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant