You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I noticed that the current implementation of dropTableWithPurge doesn’t delete snapshots. I took a look into the TaskExecution, TableCleanupTaskHandler, ManifestFileCleanupTaskHandler, and it appears that only the metadata JSON files, manifest files, and data files are removed.
To Reproduce
Launch Polaris locally
Create a Polaris catalog: polaris_catalog
Configure Spark to connect to the local Polaris instance
Create a namespace: create namespace ns;
Create an Iceberg table: create table ns.t1 (id INT, name STRING);
Insert some data into it: insert into ns.t1 values(1, 'XJKDC')
Do drop table with purge: drop table ns.t1 purge;
Check the s3 bucket to see if there are some manifest lists file remaining.
Actual Behavior
The snapshots files (manifest list files) are not deleted.
Expected Behavior
The table location should not have any files in it after we drop the table with purge.
Describe the bug
I noticed that the current implementation of
dropTableWithPurge
doesn’t delete snapshots. I took a look into theTaskExecution
,TableCleanupTaskHandler
,ManifestFileCleanupTaskHandler
, and it appears that only the metadata JSON files, manifest files, and data files are removed.To Reproduce
create namespace ns;
create table ns.t1 (id INT, name STRING);
insert into ns.t1 values(1, 'XJKDC')
drop table ns.t1 purge;
Actual Behavior
The snapshots files (manifest list files) are not deleted.
Expected Behavior
The table location should not have any files in it after we drop the table with purge.
Additional context
Introduced in this PR: #312
System information
MacOS
Lastest Polaris
The text was updated successfully, but these errors were encountered: