Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better support for COPY FROM statements #1095

Merged
merged 45 commits into from
Jan 7, 2025
Merged

Better support for COPY FROM statements #1095

merged 45 commits into from
Jan 7, 2025

Conversation

zachmu
Copy link
Member

@zachmu zachmu commented Dec 30, 2024

This change makes it possible to load data into tables without the full set of columns specified. pg_dump will not include generated columns in its output.

Also fixes a bug for the load from file use case, which only supported CSV files before (now supports the same set of formats).

Copy link
Contributor

github-actions bot commented Dec 30, 2024

Main PR
covering_index_scan_postgres 375.89/s 370.09/s -1.6%
index_join_postgres 154.07/s 154.19/s 0.0%
index_join_scan_postgres 184.59/s 183.85/s -0.5%
index_scan_postgres 12.48/s 12.59/s +0.8%
oltp_point_select 2772.58/s 2781.32/s +0.3%
oltp_read_only 1877.70/s 1878.50/s 0.0%
select_random_points 111.98/s 113.01/s +0.9%
select_random_ranges 133.03/s 132.59/s -0.4%
table_scan_postgres 11.84/s 11.78/s -0.6%
types_table_scan_postgres 5.49/s 5.52/s +0.5%

Copy link
Contributor

github-actions bot commented Dec 30, 2024

Main PR
Total 42090 42090
Successful 15351 15358
Failures 26739 26732
Partial Successes1 5234 5238
Main PR
Successful 36.4718% 36.4885%
Failures 63.5282% 63.5115%

${\color{red}Regressions (1)}$

alter_table

QUERY:          copy attest from stdin;
EXPECTED ERROR: extra data after last expected column
RECEIVED ERROR: extra data after last expected column (errno 1105) (sqlstate HY000)

${\color{lightgreen}Progressions (6)}$

constraints

QUERY: COPY COPY_TBL FROM '/home/runner/work/doltgresql/doltgresql/testing/go/regression/data/constrf.data';

identity

QUERY: COPY itest9 (b, c) FROM stdin;

numeric

QUERY: COPY width_bucket_test (operand_num) FROM stdin;

rowsecurity

QUERY: COPY t3(id, a,b,c) FROM stdin;

triggers

QUERY: insert into trig_table values
  (1, 'one a'),
  (1, 'one b'),
  (2, 'two a'),
  (2, 'two b'),
  (3, 'three a'),
  (3, 'three b');
QUERY: drop table refd_table, trig_table;

Footnotes

  1. These are tests that we're marking as Successful, however they do not match the expected output in some way. This is due to small differences, such as different wording on the error messages, or the column names being incorrect while the data itself is correct.

@zachmu zachmu marked this pull request as ready for review January 3, 2025 01:04
@zachmu zachmu requested a review from fulghum January 3, 2025 01:05
@zachmu zachmu changed the title Zachmu/copy Better support for COPY FROM statements Jan 3, 2025
Copy link
Contributor

@fulghum fulghum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Nice work expanding COPY FROM support!

server/analyzer/type_sanitizer.go Outdated Show resolved Hide resolved
server/logrepl/replication.go Outdated Show resolved Hide resolved
core/dataloader/dataloader.go Outdated Show resolved Hide resolved
@zachmu zachmu enabled auto-merge January 6, 2025 23:45
@zachmu zachmu merged commit a8d996d into main Jan 7, 2025
14 checks passed
@zachmu zachmu deleted the zachmu/copy branch January 7, 2025 00:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants