Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Ion Scan #544

Merged
merged 3 commits into from
Feb 4, 2025
Merged

Add Ion Scan #544

merged 3 commits into from
Feb 4, 2025

Conversation

jpschorr
Copy link
Contributor

This PR builds on #540, #542, & #543 and adds an ion_scan that reads boxed ion variants.

There are still todos and some additional test failures that will be addressed by future PRs that add functionality.

To see the ultimate end-point of this integration, refer to #536 and note the test coverage and conformance test results.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@jpschorr jpschorr requested a review from alancai98 January 31, 2025 21:54
Copy link

github-actions bot commented Jan 31, 2025

Conformance comparison report

Base (b970752) a93e62f +/-
% Passing 87.14% 87.15% 0.02%
✅ Passing 5602 5603 1
❌ Failing 827 826 -1
🔶 Ignored 0 0 0
Total Tests 6429 6429 0

Number passing in both: 5602

Number failing in both: 826

Number passing in Base (b970752) but now fail: 0

Number failing in Base (b970752) but now pass: 1

The following test(s) were previously failing but now pass. Before merging, confirm they are intended to pass:

Click here to see
  • partiql_tests::eval::primitives::path::path::path::permissive_path_index_bag_literal

@jpschorr jpschorr force-pushed the feat-boxed-variant-comparisons branch from 95a28d7 to 68f01d4 Compare February 3, 2025 19:51
Base automatically changed from feat-boxed-variant-comparisons to main February 3, 2025 21:47
@jpschorr jpschorr force-pushed the feat-boxed-ion-scan branch from 8aaecbf to 40f679b Compare February 3, 2025 23:03
@jpschorr jpschorr changed the base branch from main to feat-ion-null_nan February 3, 2025 23:03
Copy link

codecov bot commented Feb 3, 2025

Codecov Report

Attention: Patch coverage is 61.53846% with 120 lines in your changes missing coverage. Please review.

Project coverage is 80.01%. Comparing base (b970752) to head (5deb50e).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
extension/partiql-extension-ion/src/boxed_ion.rs 50.57% 39 Missing and 4 partials ⚠️
...on/partiql-extension-ion-functions/src/read_ion.rs 69.35% 14 Missing and 5 partials ⚠️
partiql-value/src/datum.rs 13.63% 19 Missing ⚠️
...on/partiql-extension-ion-functions/src/scan_ion.rs 71.42% 14 Missing and 4 partials ⚠️
...tension/partiql-extension-ion-functions/src/lib.rs 52.38% 8 Missing and 2 partials ⚠️
partiql-eval/src/eval/expr/base_table.rs 73.33% 4 Missing ⚠️
partiql-value/src/value/iter.rs 55.55% 4 Missing ⚠️
...sion/partiql-extension-ion-functions/src/buffer.rs 93.33% 1 Missing ⚠️
...nsion/partiql-extension-value-functions/src/lib.rs 50.00% 1 Missing ⚠️
partiql-logical-planner/src/typer.rs 0.00% 1 Missing ⚠️

❌ Your patch status has failed because the patch coverage (61.53%) is below the target coverage (85.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #544      +/-   ##
==========================================
- Coverage   80.12%   80.01%   -0.11%     
==========================================
  Files          95       98       +3     
  Lines       20581    20630      +49     
  Branches    20581    20630      +49     
==========================================
+ Hits        16490    16508      +18     
- Misses       3648     3670      +22     
- Partials      443      452       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Comment on lines +333 to +338
.map(|elt| Cow::Owned(self.child_value(elt.clone()))), // TODO find a way to remove clone
BoxedIonValue::Value(elt) => match elt.expect_sequence() {
Ok(seq) => seq
.iter()
.nth(k as usize)
.map(|elt| Cow::Owned(self.child_value(elt.clone()))), // TODO remove clone
Err(_) => todo!(),
.map(|elt| Cow::Owned(self.child_value(elt.clone()))), // TODO find a way to remove clone
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there an easy way to remove this clone in this PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No

Comment on lines 99 to 103
if header.starts_with(&[0x1f, 0x8b]) {
let decoder = flate2::read::GzDecoder::new(reader);
let buffered = BufReader::new(decoder);
parse_ion_buff(buffered)
} else if header.starts_with(&[0x28, 0xB5, 0x2F, 0xFD]) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious where do these header vars come from? Could include some comments explaining where they come from. Similarly in read_ion.rs

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add comments to the code.

Cf. https://datatracker.ietf.org/doc/html/rfc1952#page-6 section 2.3.1
Cf. https://datatracker.ietf.org/doc/rfc8478/ section 3.1.1

#[error("Ion Stream Error: `{}`", .0)]
IonStreamError(IonDecodeError),

/// Ion Stream Error
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Ion Stream Error
/// Ion Read Error

Base automatically changed from feat-ion-null_nan to main February 4, 2025 02:02
@jpschorr jpschorr force-pushed the feat-boxed-ion-scan branch from 40f679b to c459c1b Compare February 4, 2025 20:14
@jpschorr jpschorr force-pushed the feat-boxed-ion-scan branch from c459c1b to 5deb50e Compare February 4, 2025 20:20
@jpschorr jpschorr requested a review from alancai98 February 4, 2025 20:22
@jpschorr jpschorr merged commit 50b55e3 into main Feb 4, 2025
18 of 19 checks passed
@jpschorr jpschorr deleted the feat-boxed-ion-scan branch February 4, 2025 21:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants