Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Cargo.lock #14483

Merged
merged 27 commits into from
Feb 8, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
65ce278
Add `Cargo.lock`
mbrobbel Feb 4, 2025
4b6eb70
Style fix
mbrobbel Feb 4, 2025
c41b1bc
Update `Cargo.lock`
mbrobbel Feb 4, 2025
b76ad41
Remove unused path
mbrobbel Feb 4, 2025
4b2e5e3
Remove cli-specific ci job and dependabot config
mbrobbel Feb 4, 2025
f847184
Remove home pin
mbrobbel Feb 4, 2025
2e16216
Make cli test work with backtrace feature
mbrobbel Feb 4, 2025
586334c
More changes resulting from moving the cli crate in the workspace
mbrobbel Feb 4, 2025
25a05f2
Merge branch 'main' into cargo-lock
mbrobbel Feb 4, 2025
6643145
Merge branch 'main' into cargo-lock
mbrobbel Feb 4, 2025
37078eb
Exclude `depcheck` `Cargo.lock`
mbrobbel Feb 4, 2025
3ee82a5
Remove `--locked` from `depcheck` run
mbrobbel Feb 4, 2025
0c9d9cb
Merge branch 'main' into cargo-lock
mbrobbel Feb 5, 2025
32b2178
Refer to guidance instead of updated guidance
mbrobbel Feb 5, 2025
684c7df
Remove `--locked` from benchmark script
mbrobbel Feb 5, 2025
9d0d5cb
Only run with `--locked` in `linux-build-lib` job of `Rust` workflow
mbrobbel Feb 5, 2025
29d8521
Remove unrelated formatting changes
mbrobbel Feb 5, 2025
51d6bf9
Merge branch 'main' into cargo-lock
mbrobbel Feb 5, 2025
152f1cf
Add a note about using `--locked`
mbrobbel Feb 5, 2025
a911120
Merge branch 'main' into cargo-lock
mbrobbel Feb 5, 2025
9291eae
Merge branch 'main' into cargo-lock
mbrobbel Feb 5, 2025
f1cdd72
Merge remote-tracking branch 'apache/main' into cargo-lock
alamb Feb 8, 2025
ce4a4ef
Update cargo.lock
alamb Feb 8, 2025
5d30308
Update .github/workflows/rust.yml
alamb Feb 8, 2025
eb3046f
Update Cargo.lock
alamb Feb 8, 2025
8fe3cb9
Merge branch 'cargo-lock' of github.com:mbrobbel/datafusion into carg…
alamb Feb 8, 2025
347cc0a
fix yaml
alamb Feb 8, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 0 additions & 15 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,24 +28,9 @@ updates:
# arrow is bumped manually
- dependency-name: "arrow*"
update-types: ["version-update:semver-major"]
- package-ecosystem: cargo
directory: "datafusion-cli/"
schedule:
interval: daily
open-pull-requests-limit: 10
target-branch: main
labels: [auto-dependencies]
ignore:
# arrow is bumped manually
- dependency-name: "arrow*"
update-types: ["version-update:semver-major"]
# datafusion is bumped manually
- dependency-name: "datafusion*"
update-types: ["version-update:semver-major"]
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "daily"
open-pull-requests-limit: 10
labels: [auto-dependencies]

4 changes: 3 additions & 1 deletion .github/workflows/dependencies.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,11 @@ on:
push:
paths:
- "**/Cargo.toml"
- "**/Cargo.lock"
pull_request:
paths:
- "**/Cargo.toml"
- "**/Cargo.lock"
# manual trigger
# https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
workflow_dispatch:
Expand All @@ -50,4 +52,4 @@ jobs:
- name: Check dependencies
run: |
cd dev/depcheck
cargo run
cargo run
130 changes: 46 additions & 84 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,11 @@ jobs:
with:
rust-version: stable
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a nice idea to only use --locked in one test 👍

- name: Prepare cargo build
run: cargo check --profile ci --all-targets --features integration-tests
run: |
# Adding `--locked` here to assert that the `Cargo.lock` file is up to
# date with the manifest. When this fails, please make sure to commit
# the changes to `Cargo.lock` after building with the updated manifest.
cargo check --profile ci --workspace --all-targets --features integration-tests --locked

# cargo check common, functions and substrait with no default features
linux-cargo-check-no-default-features:
Expand Down Expand Up @@ -95,12 +99,6 @@ jobs:
- name: Check workspace with additional features
run: cargo check --profile ci --workspace --benches --features avro,json,integration-tests

- name: Check Cargo.lock for datafusion-cli
run: |
# If this test fails, try running `cargo update` in the `datafusion-cli` directory
# and check in the updated Cargo.lock file.
cargo check --profile ci --manifest-path datafusion-cli/Cargo.toml --locked

# cargo check datafusion to ensure that the datafusion crate can be built with only a
# subset of the function packages enabled.
linux-cargo-check-datafusion:
Expand Down Expand Up @@ -189,28 +187,6 @@ jobs:
- name: Verify Working Directory Clean
run: git diff --exit-code

linux-test-datafusion-cli:
name: cargo test datafusion-cli (amd64)
needs: linux-build-lib
runs-on: ubuntu-latest
container:
image: amd64/rust
steps:
- uses: actions/checkout@v4
with:
submodules: true
fetch-depth: 1
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
rust-version: stable
- name: Run tests (excluding doctests)
run: |
cd datafusion-cli
cargo test --profile ci --lib --tests --bins --all-features
- name: Verify Working Directory Clean
run: git diff --exit-code

linux-test-example:
name: cargo examples (amd64)
needs: linux-build-lib
Expand Down Expand Up @@ -252,10 +228,7 @@ jobs:
with:
rust-version: stable
- name: Run doctests
run: |
cargo test --profile ci --doc --features avro,json
cd datafusion-cli
cargo test --profile ci --doc --all-features
run: cargo test --profile ci --doc --features avro,json
- name: Verify Working Directory Clean
run: git diff --exit-code

Expand Down Expand Up @@ -364,45 +337,40 @@ jobs:
POSTGRES_HOST: postgres
POSTGRES_PORT: ${{ job.services.postgres.ports[5432] }}

# Temporarily commenting out the Windows flow, the reason is enormously slow running build
# Waiting for new Windows 2025 github runner
# Details: https://github.com/apache/datafusion/issues/13726
#
# windows:
# name: cargo test (win64)
# runs-on: windows-latest
# steps:
# - uses: actions/checkout@v4
# with:
# submodules: true
# - name: Setup Rust toolchain
# uses: ./.github/actions/setup-windows-builder
# - name: Run tests (excluding doctests)
# shell: bash
# run: |
# export PATH=$PATH:$HOME/d/protoc/bin
# cargo test --lib --tests --bins --features avro,json,backtrace
# cd datafusion-cli
# cargo test --lib --tests --bins --all-features

# Commenting out intel mac build as so few users would ever use it
# Details: https://github.com/apache/datafusion/issues/13846
# macos:
# name: cargo test (macos)
# runs-on: macos-latest
# steps:
# - uses: actions/checkout@v4
# with:
# submodules: true
# fetch-depth: 1
# - name: Setup Rust toolchain
# uses: ./.github/actions/setup-macos-builder
# - name: Run tests (excluding doctests)
# shell: bash
# run: |
# cargo test run --profile ci --exclude datafusion-examples --exclude datafusion-benchmarks --workspace --lib --tests --bins --features avro,json,backtrace
# cd datafusion-cli
# cargo test run --profile ci --lib --tests --bins --all-features
# Temporarily commenting out the Windows flow, the reason is enormously slow running build
# Waiting for new Windows 2025 github runner
# Details: https://github.com/apache/datafusion/issues/13726
#
# windows:
# name: cargo test (win64)
# runs-on: windows-latest
# steps:
# - uses: actions/checkout@v4
# with:
# submodules: true
# - name: Setup Rust toolchain
# uses: ./.github/actions/setup-windows-builder
# - name: Run tests (excluding doctests)
# shell: bash
# run: |
# export PATH=$PATH:$HOME/d/protoc/bin
# cargo test --lib --tests --bins --features avro,json,backtrace

# Commenting out intel mac build as so few users would ever use it
# Details: https://github.com/apache/datafusion/issues/13846
# macos:
# name: cargo test (macos)
# runs-on: macos-latest
# steps:
# - uses: actions/checkout@v4
# with:
# submodules: true
# fetch-depth: 1
# - name: Setup Rust toolchain
# uses: ./.github/actions/setup-macos-builder
# - name: Run tests (excluding doctests)
# shell: bash
# run: cargo test run --profile ci --exclude datafusion-examples --exclude datafusion-benchmarks --workspace --lib --tests --bins --features avro,json,backtrace

macos-aarch64:
name: cargo test (macos-aarch64)
Expand All @@ -416,10 +384,7 @@ jobs:
uses: ./.github/actions/setup-macos-aarch64-builder
- name: Run tests (excluding doctests)
shell: bash
run: |
cargo test --profile ci --lib --tests --bins --features avro,json,backtrace,integration-tests
cd datafusion-cli
cargo test --profile ci --lib --tests --bins --all-features
run: cargo test --profile ci --lib --tests --bins --features avro,json,backtrace,integration-tests

test-datafusion-pyarrow:
name: cargo test pyarrow (amd64)
Expand Down Expand Up @@ -615,19 +580,19 @@ jobs:
# (Min Supported Rust Version) than the one specified in the
# `rust-version` key of `Cargo.toml`.
#
# To reproduce:
# 1. Install the version of Rust that is failing. Example:
# To reproduce:
# 1. Install the version of Rust that is failing. Example:
# rustup install 1.80.1
# 2. Run the command that failed with that version. Example:
# cargo +1.80.1 check -p datafusion
#
#
# To resolve, either:
# 1. Change your code to use older Rust features,
# 1. Change your code to use older Rust features,
# 2. Revert dependency update
# 3. Update the MSRV version in `Cargo.toml`
#
# Please see the DataFusion Rust Version Compatibility Policy before
# updating Cargo.toml. You may have to update the code instead.
# updating Cargo.toml. You may have to update the code instead.
# https://github.com/apache/datafusion/blob/main/README.md#rust-version-compatibility-policy
cargo msrv --output-format json --log-target stdout verify
- name: Check datafusion-substrait
Expand All @@ -636,6 +601,3 @@ jobs:
- name: Check datafusion-proto
working-directory: datafusion/proto
run: cargo msrv --output-format json --log-target stdout verify
- name: Check datafusion-cli
working-directory: datafusion-cli
run: cargo msrv --output-format json --log-target stdout verify
2 changes: 0 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,6 @@ venv/*

# Rust
target
Cargo.lock
!datafusion-cli/Cargo.lock

rusty-tags.vi
.history
Expand Down
Loading