Repository navigation
docs: add examples for walking and rewriting logical plans - #25954
mattsu2020 wants to merge 3 commits into
Conversation
kosiew
left a comment
There was a problem hiding this comment.
Thanks for adding the plan traversal examples and documentation. The approach is appropriate and clearly demonstrates how to inspect and rewrite logical plans.
The existing traversal APIs provide the right abstraction. Ordinary traversal handles plan inputs, while subquery-aware traversal reaches expression-embedded subqueries. The examples correctly distinguish removed Filters from pushed-down scan predicates and acknowledge the resulting semantic changes. No production behavior or public API changes are needed.
I'm happy to approve. The implementation and documentation are sound, with no blocking concerns. I've left two optional suggestions to make the examples more instructive, but neither is required for merging.
| println!("Filter predicate: {predicate}"); | ||
| } | ||
|
|
||
| let mut rewriter = RemoveFilters::default(); |
There was a problem hiding this comment.
Optional: Could we compare this with ordinary rewrite using a fresh RemoveFilters? Asserting removed == 1 and checking that summarize still finds the subquery Filter and pushed-down scan predicate would help illustrate why rewrite_with_subqueries is needed.
| impl TreeNodeRewriter for RemoveFilters { | ||
| type Node = LogicalPlan; | ||
|
|
||
| // f_up runs after the inputs (and, with rewrite_with_subqueries, embedded |
There was a problem hiding this comment.
Optional: A small Filter -> Filter -> TableScan example asserting removed == 2 and leaving only the scan would make the nested-Filter behavior clearer. The current Filters are on separate branches, so they don't demonstrate consecutive Filter removal.
Which issue does this PR close?
TreeNodeAPIs to walk and manipulate LogicalPlans #10628.Rationale for this change
Users need concrete examples of inspecting and rewriting
LogicalPlantrees with theTreeNodeAPIs, including subqueries embedded in expressions.What changes are included in this PR?
LogicalPlanrustdoc example to collect table names withTreeNode::apply.TreeNode::rewriteexample that removes Filter nodes, explaining traversal order and the change in query semantics.query_planning -- plan_walkexample demonstrating plan inspection, stateful rewriting, subquery traversal, and collecting predicates from both Filter nodes and TableScan filters.What is the testing strategy for this PR?
The rustdoc examples and assertions in the executable example verify the collected table names, rewritten plan, subquery traversal, and filter predicates.
cargo test --profile ci -p datafusion-expr --doc logical_plan::plan::LogicalPlan(8 passed)cargo run --profile ci -p datafusion-examples --example query_planning -- plan_walkcargo fmt --allcargo clippy --all-targets --all-features -- -D warnings./ci/scripts/doc_prettier_check.sh --write --allow-dirty./dev/rust_lint.sh(run viauv run --package datafusion-dev --with PyYAML)Are there any user-facing changes?
The API documentation now includes additional plan walking and rewriting examples and links to an executable walkthrough. There are no changes to query execution or public APIs.