Repository navigation
Narrow remembered count()/sizeof() of an array when the array expression is narrowed - #6953
phpstan-bot wants to merge 5 commits into
Conversation
VincentLanglet
left a comment
There was a problem hiding this comment.
Is it something really specific to count ?
Also, why do we remember count($expr) given the fact int<0, max> doesn't give any type improvement
| * @param array<string, true> $differingExpressionKeys | ||
| * @return array<string, true> | ||
| */ | ||
| private function findUninformativeRememberedCalls(array $differingExpressionKeys, bool $afterExtractCall): array |
There was a problem hiding this comment.
What's the perf impact of checking this every time ?
There was a problem hiding this comment.
I didn't change any code. The measurements below show the check costs nothing measurable, so there was nothing to commit or push, and the working tree is clean.
What's the perf impact of checking this every time?
It's negligible. The expensive part only runs when it has something to do:
- The cheap path: most merges never get past the first loop. That loop only looks at keys whose types differ between the two branches, keeps those where the merged entry is certain, and checks with an
instanceofthat it is a function, method or static call. Only if such an entry exists does it build a scope without them and resolve the calls again. - How often it runs (PHPStan analysing its own source,
make phpstan):findUninformativeRememberedCalls()was called 79,480 times. Only 8,692 of those (≈11%) had any candidate calls and built the extra scope. They checked 11,737 candidate calls in total, and 6,104 entries were dropped. - Time spent inside it: 1.27 s of CPU summed over all 4 workers, out of about 307 s of worker CPU. That's about 0.4%.
End-to-end comparison (with a cleared result cache, this PR against 9d55d96a8, the code before the PR):
| run | wall | user CPU |
|---|---|---|
| base | 80.98 s | 307.8 s |
| head | 80.91 s | 308.1 s |
| base | 80.48 s | 305.0 s |
| head | 80.41 s | 306.4 s |
tests/bench (phpbench, all 86 cases, one iteration each): 36.4 s total before the PR, 36.7 s with it (+0.8%, within noise). The largest single difference was bug-14996.php at +7.8%. Running it by itself three times per side gave 2.83–2.97 s for base and 2.85–2.92 s for head, so that was noise too.
Part of the cost pays for itself. When a remembered call is dropped, the scope gets smaller, and each later merge, scope comparison and conditional-expression pass has fewer entries to handle.
I didn't re-measure with the turbo extension loaded (it isn't installed here). The C++ mirror follows the same logic, but its numbers are unconfirmed.
There was a problem hiding this comment.
why do we remember a type if it does not give us more information? can't we detect this on the writing side, instead of filtering things out after the fact?
There was a problem hiding this comment.
The full runs are in progress; I'll wait for the completion notice.
I changed the code so uninformative merged calls are never written into the merged scope. Before, the full merged scope was built and the entries were filtered out afterwards. This is pushed as bf0738afc.
Why is the type remembered at all? Neither branch writes anything useless. if (count($arr) > 2) stores count($arr) as int<3, max> in the truthy branch and int<0, 2> in the falsy branch. Both are more precise than what count($arr) resolves to by itself. The entry only becomes useless when the two branches are merged into their union, int<0, max>. So the merge is the only point where this can be detected, and it is also where the merged scope gets built.
What changed in mergeWithVariableState():
- Before: it built three scopes. First the full merged scope, then a copy without the candidate calls (to resolve them), then a third copy without the uninformative ones.
- Now: it picks the candidates first, straight from the merged tables. Candidates are certain function, method or static calls whose type differs between the branches; that selection is now a static helper,
findMergedCallCandidates(). - The merged scope is built without the candidates. Each candidate is resolved on that scope by
findUninformativeMergedCalls(), which compares both the PHPDoc and the native type usingequals(). - Only if a candidate still carries information is a second scope built with those entries put back.
- Result: when there are no candidates, or all of them are uninformative, only one scope is built. These are the common cases.
The same restructuring is ported to the C++ mirror in turbo-ext/src/MutatingScope.cpp. Inference doesn't change: the existing bug-13750.php assertions pass unchanged.
Verification:
make testspasses both without and with the rebuilt turbo extension loaded (23404 tests each).make phpstanreports no errors.- The turbo smoke test prints
ALL OK, and signature parity passes (18145 members compared). - I didn't re-run the timing benchmark. Every case now builds the same number of scopes as before or fewer, so it can't be slower than the earlier measured 0.4% of worker CPU.
The PR title and description still describe only the original count-only narrowing. They should also cover the merge-time change.
|
Waiting for the test run to finish. The suites and I replaced the count-specific narrowing with a general fix. The stale Is it specific to Why was The change:
Verification:
Trade-off: a partly informative merged count, like Not done: A background watcher I'd started to wait on the test log was stopped when it hit its time limit. It probably never exited because its process check matched itself. It doesn't affect anything: I had already read that log myself, and it showed both test runs and |
VincentLanglet
left a comment
There was a problem hiding this comment.
a partly informative merged count, like int<0, 2>|int<5, max>, is still kept as is. A later $arr !== [] therefore won't trim it to int<1, 2>|int<5, max>. The earlier count-only narrowing did handle that case.
What are the different solution to handle this case ?
Is is better to have a count-only narrowing ? Is there other function/method impacted ?
Can't we fix this by adding an extra inference non-empty-array => count !== 0 ?
|
I kept both fixes. The general merge-time pruning stays, and I added back the count-only narrowing to handle the partly informative case. It's pushed as What are the options for the partly informative case?
Should it be count-only? Not on its own. The pruning when scopes merge is still needed, because it fixes the fully uninformative case for any call: Other functions affected: any remembered call whose result depends on its argument's type has the same partly informative gap. For example, Separate bug I found: after Test: I added Verification:
The PR title and description still describe the original count-only approach and should be updated to cover both changes. |
|
I would be interested about your opinion on this/those fix @staabm |
staabm
left a comment
There was a problem hiding this comment.
my feeling is that its not worth having a hack for a single function like that.
maybe we should look into why count() has this problem and maybe we need to handle count() itself differently?
|
Waiting on the build and test runs. I removed the count-only narrowing, as you suggested. The PR now has only the general fix in how branches are merged, which works for every function and method call. Pushed as Why
What's left in the PR:
What was removed:
Trade-off: a merged count that still carries some information, such as Verification:
The PR title and description still describe the count-only approach that's now gone. They need rewriting to cover only the merge-time change. |
9f750f5 to
2dfb122
Compare
VincentLanglet
left a comment
There was a problem hiding this comment.
I feel like the general fix is ok for now, and it avoids keeping in memory useless expression types.
…ression is narrowed
- MutatingScope::specifyExpressionTypeInPlace() now calls the new
narrowRememberedCountsInPlace(): when the narrowed expression is an
array and the scope remembers `count($expr)` / `sizeof($expr)` (Name or
FullyQualified spelling), the remembered type is intersected with the
new type's getArraySize(), for both PHPDoc and native tables. Intersections
that would be never are skipped.
- Previously a `count($arr)` entry left behind by merging branches
(e.g. int<0, max> after `if (count($arr) > 2) {}`) shadowed the size
implied by later narrowing of `$arr`. The same staleness affected every
way of narrowing the array, not only `count($arr, COUNT_RECURSIVE)`:
`$arr !== []`, `$arr === []`, truthiness, `!empty()`, `isset($arr[0])`,
`array_key_exists()`, recursive `sizeof()`, and property fetches like
`$this->prop`. All of these are covered by the regression test.
- Ported to the native mirror in turbo-ext/src/MutatingScope.cpp, including
the new private COUNT_FUNCTION_KEY_NAMES constant.
… itself
Instead of narrowing a remembered count()/sizeof() whenever its array is
narrowed, do not keep such an entry after merging scopes in the first
place: merging count($a) of int<3, max> and int<0, 2> gives int<0, max>,
which is exactly what count($a) resolves to on its own. The stale entry
then shadowed what later narrowing of $a implied for the call.
The merge now drops a remembered FuncCall/MethodCall/StaticCall whose
merged type (PHPDoc and native) equals the type the call resolves to
without the entry. This is not specific to count() - strlen($s) after
`if (strlen($s) > 2) {}` had the same problem with `$s !== ''`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The merge-time pruning only drops remembered calls that are no more precise than the call itself. A partly informative count, like int<0, 2>|int<5, max> after returning on 3-4 elements, is kept, so a later `$arr !== []` left it untrimmed. Narrowing an array expression now intersects any remembered count($expr)/sizeof($expr) (Name and FullyQualified spellings) with the new type's getArraySize(), for the PHPDoc and native tables, skipping results that would be never. Ported to turbo-ext/src/MutatingScope.cpp. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…runing it afterwards mergeWithVariableState() used to build the full merged scope, then a second scope without the candidate calls to resolve them, and a third scope without the uninformative ones. Now the candidates are picked from the merged tables before any scope exists, the merged scope is built without them, and the calls are resolved on that scope. Only when some candidate is still informative is a scope with those entries written; in the common cases (no candidates, or all of them uninformative) a single scope is built. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The general merge-time pruning of uninformative remembered calls already fixes the reported case for every function. The count()/sizeof()-specific intersection on narrowing only covered partly informative merged counts and is not worth a special case for a single function. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2dfb122 to
61aeb79
Compare
|
this PR has no CPP components. so lets wait until 2.3.x is green before merging |
Summary
After
if (count($arr) > 2) {}, merging the two branches leaves a rememberedcount($arr)ofint<0, max>in the scope. Later narrowing of$arr(the issue usedcount($arr, COUNT_RECURSIVE) > 2) correctly turned$arrintonon-empty-array. Butcount($arr)was read from the stale remembered entry and stayedint<0, max>when it should have beenint<1, max>.With this change, narrowing an array expression also narrows any remembered
count()/sizeof()of that expression.Changes
src/Analyser/MutatingScope.php:specifyExpressionTypeInPlace()calls the new privatenarrowRememberedCountsInPlace(). For each of the keyscount(...),sizeof(...),\count(...)and\sizeof(...)built from the expression, if an entry exists and the new type is an array, it is intersected withgetArraySize(). This applies to both the PHPDoc and the native expression tables. Certainty is kept, and a result ofneveris skipped. The key names live in the new private constantCOUNT_FUNCTION_KEY_NAMES.turbo-ext/src/MutatingScope.cpp: the same logic in the native mirror (narrowRememberedCountsInPlace()/narrowRememberedCountInTable()), plus the native declaration ofCOUNT_FUNCTION_KEY_NAMES.Analogous cases probed: all of these were broken the same way, and all are fixed by this change and covered by the test:
$arr !== []and$arr === []if ($arr)and!empty($arr)isset($arr[0])andarray_key_exists()sizeof()and recursivesizeof()count($this->prop))Root cause
Narrowing an expression's type only rewrote that expression's own entry. Remembered expressions derived from it, here
count($expr), were neither invalidated nor refined, so their stale type took precedence over whatCountFunctionReturnTypeExtensionwould have computed from the narrowed array. Assignments were already safe becauseassignExpression()invalidates dependent entries, andunset()explicitly invalidatescount/sizeof. Only narrowing had the gap, and this change closes it.Test
tests/PHPStan/Analyser/nsrt/bug-13750.phpcontains the issue's reproducer verbatim, plus the analogous narrowing forms listed above. One extra case checks that an already narrower remembered count (int<3, max>) is kept. The test fails without the fix (9 failing assertions), both with the PHP implementation and with the turbo extension loaded.The full test suite passes with and without the turbo extension.
make phpstan, the turbo smoke test and signature parity also pass.Fixes phpstan/phpstan#13750