Background
Currently, when Hive schema changes the table is dropped and re-created with new schema.
But this requires re-scanning of all partitions which is slow, especially on S3.
A better solution could be based on:
ALTER TABLE my_ext_table REPLACE COLUMNS (
col1 INT,
col2 STRING
);
Feature
When replacing Hive schema use 'ALTER TABLE' to avoid 'MSCK REPAIR TABLE'.
Example [Optional]
--
Proposed Solution [Optional]
--
Background
Currently, when Hive schema changes the table is dropped and re-created with new schema.
But this requires re-scanning of all partitions which is slow, especially on S3.
A better solution could be based on:
Feature
When replacing Hive schema use 'ALTER TABLE' to avoid 'MSCK REPAIR TABLE'.
Example [Optional]
--
Proposed Solution [Optional]
--