feat: hilbert clustering

[zhyass]

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

This PR refines Hilbert clustering in Databend by adopting a range-based partitioning approach. It samples cluster keys, assigns range partition IDs, and calculates Hilbert indexes for efficient pruning and clustering. Key changes include:

1. Removing the old Hilbert clustering logic.

2. Stable segments are excluded from reclustering to preserve optimal clustering results.

The alter table t recluster is equivalent to

WITH _keys_bound AS (
  SELECT 
    range_bound(1024, 1000)(a) AS a_bound, 
    range_bound(1024, 1000)(b) AS b_bound 
  FROM 
    default.t
), 
_source_data AS (
  SELECT 
    t.*, 
    hilbert_index(
      [hilbert_key(cast(ifnull(range_partition_id(t.a, _keys_bound.a_bound), 1023) as uint16)), hilbert_key(cast(ifnull(range_partition_id(t.b, _keys_bound.b_bound), 1023) as uint16))], 
      2
    ) AS _hilbert_index 
  FROM 
    default.t, 
    _keys_bound
) 
SELECT 
  * EXCLUDE(_hilbert_index) 
FROM 
  _source_data 
ORDER BY 
  _hilbert_index

Tests

• Unit Test
• Logic Test
• Benchmark Test
• No Test - Explain why

Type of change

• Bug Fix (non-breaking change which fixes an issue)
• New Feature (non-breaking change which adds functionality)
• Breaking Change (fix or feature that could cause existing functionality not to work as expected)
• Documentation Update
• Refactoring
• Performance Improvement
• Other (please describe):

---

This change is 

[zhang2014]

Maybe should add performance test with hilbert clustering?

[github-actions]

Docker Image for PR

• tag: pr-17045-5895f17-1735794560

note: this image tag is only available for internal use,
please check the internal doc for more details.