Skip to content

docs: add examples for walking and rewriting logical plans - #25954

Open
mattsu2020 wants to merge 3 commits into
apache:mainfrom
mattsu2020:10628
Open

mattsu2020 wants to merge 3 commits into
apache:mainfrom
mattsu2020:10628

Conversation

@mattsu2020

Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

Users need concrete examples of inspecting and rewriting LogicalPlan trees with the TreeNode APIs, including subqueries embedded in expressions.

What changes are included in this PR?

  • Extend the LogicalPlan rustdoc example to collect table names with TreeNode::apply.
  • Add a tested TreeNode::rewrite example that removes Filter nodes, explaining traversal order and the change in query semantics.
  • Add the query_planning -- plan_walk example demonstrating plan inspection, stateful rewriting, subquery traversal, and collecting predicates from both Filter nodes and TableScan filters.
  • Register the example in the query planning runner and update the generated examples README.

What is the testing strategy for this PR?

The rustdoc examples and assertions in the executable example verify the collected table names, rewritten plan, subquery traversal, and filter predicates.

  • cargo test --profile ci -p datafusion-expr --doc logical_plan::plan::LogicalPlan (8 passed)
  • cargo run --profile ci -p datafusion-examples --example query_planning -- plan_walk
  • cargo fmt --all
  • cargo clippy --all-targets --all-features -- -D warnings
  • ./ci/scripts/doc_prettier_check.sh --write --allow-dirty
  • ./dev/rust_lint.sh (run via uv run --package datafusion-dev --with PyYAML)

Are there any user-facing changes?

The API documentation now includes additional plan walking and rewriting examples and links to an executable walkthrough. There are no changes to query execution or public APIs.

@github-actions github-actions Bot added the logical-expr Logical plan and expressions label Oct 1, 2026

@kosiew kosiew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattsu2020,

Thanks for adding the plan traversal examples and documentation. The approach is appropriate and clearly demonstrates how to inspect and rewrite logical plans.

The existing traversal APIs provide the right abstraction. Ordinary traversal handles plan inputs, while subquery-aware traversal reaches expression-embedded subqueries. The examples correctly distinguish removed Filters from pushed-down scan predicates and acknowledge the resulting semantic changes. No production behavior or public API changes are needed.

I'm happy to approve. The implementation and documentation are sound, with no blocking concerns. I've left two optional suggestions to make the examples more instructive, but neither is required for merging.

println!("Filter predicate: {predicate}");
}

let mut rewriter = RemoveFilters::default();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: Could we compare this with ordinary rewrite using a fresh RemoveFilters? Asserting removed == 1 and checking that summarize still finds the subquery Filter and pushed-down scan predicate would help illustrate why rewrite_with_subqueries is needed.

impl TreeNodeRewriter for RemoveFilters {
type Node = LogicalPlan;

// f_up runs after the inputs (and, with rewrite_with_subqueries, embedded

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: A small Filter -> Filter -> TableScan example asserting removed == 2 and leaving only the scan would make the nested-Filter behavior clearer. The current Filters are on separate branches, so they don't demonstrate consecutive Filter removal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

logical-expr Logical plan and expressions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Examples of using TreeNode APIs to walk and manipulate LogicalPlans

2 participants