Table maintenance Support in Polaris

[flyrain]

Describe the solution you'd like

Iceberg table maintenance allows users to focus on their business needs without worrying about table management. It ensures tables stay optimized, perform efficiently, and comply with governance policies like data retention. Common maintenance tasks include data compaction, metadata compaction, snapshot expiration, partition stats collection, and orphan file removal.
The Table Maintenance System (TMS) can operate independently from a catalog like Polaris, providing maintenance services even without direct integration. However, it functions more effectively when paired with a catalog. Meanwhile, Polaris should allow pluggable TMS options so users can choose the one that fits their needs. This document outlines how Polaris supports Iceberg table maintenance—defining its role and clarifying what it should handle and what it should leave to TMS.

Here is the proposal, https://docs.google.com/document/d/1Pd_mzZcfvnUvcH98IbwsIYf4eryet1lQDfclKYx-t-M/edit?usp=sharing.

Describe alternatives you've considered

No response

Additional context

No response

[jbonofre]

I guess we should provide an update about policy management (document) there.

[flyrain]

Here is a related doc for policy management: https://docs.google.com/document/d/1kIiVkFFg9tPa5SH70b9WwzbmclrzH3qWHKfCKXw5lbs/edit?tab=t.0#heading=h.nly223xz13km

[benhuds]

Hi @flyrain , should we make a separate GitHub issue to track the Policy Management proposal you've linked above? Wondering if it makes sense to track progress of that proposal, independent of TMS