Skip to content

Conversation

gongxun0928
Copy link
Contributor

this pr is base on #1324

feat: pushdown bloom filter to pax table am

This optimization pushes down Bloom Filter conditions for runtime filters
to the Pax Table AM layer.

By applying the filter earlier than the SeqNext() function, it eliminates
the overhead of converting data from columnar format to TableTupleSlot,
resulting in faster query execution

CREATE TABLE t2(c1 int, c2 int, c3 int, c4 int, c5 int) with (appendonly=true, orientation=column) distributed REPLICATED;
INSERT INTO t2 VALUES (1,1,1,1,1), (2,2,2,2,2), (3,3,3,3,3), (4,4,4,4,4);
INSERT INTO t2 select * FROM t2;
INSERT INTO t2 select * FROM t2;
INSERT INTO t2 select * FROM t2;
CREATE TABLE t3(c1 int, c2 int, c3 int, c4 int, c5 int) using pax;


gpadmin=# insert into t3 select b,i,i,i,i from generate_series(1,5) b,generate_series(1,2000000) i;

INSERT 0 10000000
gpadmin=#
gpadmin=# analyze t3;
ANALYZE
gpadmin=# EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF)
SELECT t3.c3 FROM t3, t2 WHERE t3.c2 = t2.c4;
                                        QUERY PLAN
-------------------------------------------------------------------------------------------
 Gather Motion 3:1  (slice1; segments: 3) (actual time=355.000..1482.000 rows=160 loops=1)
   ->  Hash Join (actual time=0.000..1482.000 rows=96 loops=1)
         Hash Cond: (t3.c2 = t2.c4)
         Extra Text: (seg0)   Hash chain length 8.0 avg, 8 max, using 4 of 524288 buckets.
         ->  Seq Scan on t3 (actual time=0.000..691.000 rows=6000000 loops=1)
         ->  Hash (actual time=0.000..0.000 rows=32 loops=1)
               Buckets: 524288  Batches: 1  Memory Usage: 4098kB
               ->  Seq Scan on t2 (actual time=0.000..0.000 rows=32 loops=1)
 Optimizer: GPORCA
(9 rows)

gpadmin=# set gp_enable_runtime_filter_pushdown to on;
SET
gpadmin=# EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF)
SELECT t3.c3 FROM t3, t2 WHERE t3.c2 = t2.c4;
                                        QUERY PLAN
-------------------------------------------------------------------------------------------
 Gather Motion 3:1  (slice1; segments: 3) (actual time=99.000..303.000 rows=160 loops=1)
   ->  Hash Join (actual time=1.000..303.000 rows=96 loops=1)
         Hash Cond: (t3.c2 = t2.c4)
         Extra Text: (seg0)   Hash chain length 8.0 avg, 8 max, using 4 of 524288 buckets.
         ->  Seq Scan on t3 (actual time=1.000..301.000 rows=12 loops=1)
               Rows Removed by Pushdown Runtime Filter: 5999988
         ->  Hash (actual time=0.000..0.000 rows=32 loops=1)
               Buckets: 524288  Batches: 1  Memory Usage: 4098kB
               ->  Seq Scan on t2 (actual time=0.000..0.000 rows=32 loops=1)
 Optimizer: GPORCA
(10 rows)

gpadmin=# set pax_enable_row_filter to on;
SET
gpadmin=# EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF)
SELECT t3.c3 FROM t3, t2 WHERE t3.c2 = t2.c4;
                                        QUERY PLAN
-------------------------------------------------------------------------------------------
 Gather Motion 3:1  (slice1; segments: 3) (actual time=48.000..138.000 rows=160 loops=1)
   ->  Hash Join (actual time=0.000..138.000 rows=96 loops=1)
         Hash Cond: (t3.c2 = t2.c4)
         Extra Text: (seg0)   Hash chain length 8.0 avg, 8 max, using 4 of 524288 buckets.
         ->  Seq Scan on t3 (actual time=0.000..137.000 rows=12 loops=1)
               Rows Removed by Pushdown Runtime Filter: 5999988
         ->  Hash (actual time=0.000..0.000 rows=32 loops=1)
               Buckets: 524288  Batches: 1  Memory Usage: 4098kB
               ->  Seq Scan on t2 (actual time=0.000..0.000 rows=32 loops=1)
 Optimizer: GPORCA
(10 rows)

What does this PR do?

Type of Change

  • Bug fix (non-breaking change)
  • New feature (non-breaking change)
  • Breaking change (fix or feature with breaking changes)
  • Documentation update

Breaking Changes

Test Plan

  • Unit tests added/updated
  • Integration tests added/updated
  • Passed make installcheck
  • Passed make -C src/test installcheck-cbdb-parallel

Impact

Performance:

User-facing changes:

Dependencies:

Checklist

Additional Context

CI Skip Instructions


when seq scan begins, check whether the scanflags of table am is set to
determine whether the runtime filter is pushed down.

When the runtime filter is pushed down to pax am, pax am converts the min/max
scankey in the runtime filter into PFTNode and performs min/max filtering.
This optimization pushes down Bloom Filter conditions for runtime filters
to the Pax Table AM layer.

By applying the filter earlier than the SeqNext() function, it eliminates
the overhead of converting data from columnar format to TableTupleSlot,
resulting in faster query execution
@gongxun0928 gongxun0928 force-pushed the pax/support-bloom-filter-pushdown branch from a02e2c3 to 1fb74ea Compare August 27, 2025 10:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant