Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HPCC-33405: Add support for ubuntu arm #19571

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

GordonSmith
Copy link
Member

@GordonSmith GordonSmith commented Feb 27, 2025

Move antlr3c into vcpkg
DC: Fix for arm64 register naming
Rework dockerfiles/vcpkg/build.sh to support arm

Signed-off-by Gordon Smith [email protected]

Type of change:

  • This change is a bug fix (non-breaking change which fixes an issue).
  • This change is a new feature (non-breaking change which adds functionality).
  • This change improves the code (refactor or other change that does not change the functionality)
  • This change fixes warnings (the fix does not alter the functionality or the generated code)
  • This change is a breaking change (fix or feature that will cause existing behavior to change).
  • This change alters the query API (existing queries will have to be recompiled)

Checklist:

  • My code follows the code style of this project.
    • My code does not create any new warnings from compiler, build system, or lint.
  • The commit message is properly formatted and free of typos.
    • The commit message title makes sense in a changelog, by itself.
    • The commit is signed.
  • My change requires a change to the documentation.
    • I have updated the documentation accordingly, or...
    • I have created a JIRA ticket to update the documentation.
    • Any new interfaces or exported functions are appropriately commented.
  • I have read the CONTRIBUTORS document.
  • The change has been fully tested:
    • I have added tests to cover my changes.
    • All new and existing tests passed.
    • I have checked that this change does not introduce memory leaks.
    • I have used Valgrind or similar tools to check for potential issues.
  • I have given due consideration to all of the following potential concerns:
    • Scalability
    • Performance
    • Security
    • Thread-safety
    • Cloud-compatibility
    • Premature optimization
    • Existing deployed queries will not be broken
    • This change fixes the problem, not just the symptom
    • The target branch of this pull request is appropriate for such a change.
  • There are no similar instances of the same problem that should be addressed
    • I have addressed them here
    • I have raised JIRA issues to address them separately
  • This is a user interface / front-end modification
    • I have tested my changes in multiple modern browsers
    • The component(s) render as expected

Smoketest:

  • Send notifications about my Pull Request position in Smoketest queue.
  • Test my draft Pull Request.

Testing:

@GordonSmith GordonSmith force-pushed the HPCC-33405-ARM64_LINUX branch from 0670a96 to 0901101 Compare March 3, 2025 13:18
@GordonSmith GordonSmith requested a review from Copilot March 3, 2025 15:05

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Overview

This PR bumps vcpkg to version 2025.02.13 and extends support for ARM architectures by updating workflows and build scripts. Key changes include:

  • Adding new GitHub workflow jobs for ARM-based builds on Ubuntu and macOS.
  • Modifying docker build scripts to incorporate an arm-specific tag postfix.
  • Adjusting cache key generation to account for the tag postfix in ARM builds.

Reviewed Changes

File Description
.github/workflows/build-vcpkg.yml Added ARM build jobs for Ubuntu on different versions.
.github/workflows/build-docker.yml Updated inputs and commands to support ARM tagging and builds.

Copilot reviewed 28 out of 28 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (2)

.github/workflows/build-docker.yml:73

  • Appending tag_postfix directly to the runner label may cause a mismatch with available GitHub runner labels. Consider using a separate input or matrix variable to set a valid runner label.
runs-on: ubuntu-22.04${{ inputs.tag_postfix }}

.github/workflows/build-docker.yml:160

  • The expression '-D$plugin=ON' appears to contain an undefined or incorrectly referenced variable. Verify the intended variable substitution and update it to correctly reference the desired plugin setting.
cmake -G Ninja -S /hpcc-dev/${{ inputs.ln == true && 'LN' || 'HPCC-Platform' }} -B /hpcc-dev/build -DHPCC_SOURCE_DIR=/hpcc-dev/HPCC-Platform -DCMAKE_BUILD_TYPE=${{ inputs.build-type }} -DCONTAINERIZED=${{ inputs.containerized == true && 'ON' || 'OFF' }} -DCPACK_STRIP_FILES=${{ inputs.strip-files == true && 'ON' || 'OFF' }} ${{ inputs.single-package == true && '-DINCLUDE_PLUGINS=ON' || '-D$plugin=ON' }} ${{ inputs.cmake-configuration }} ${{ inputs.cmake-configuration-ex }} && \
@GordonSmith GordonSmith force-pushed the HPCC-33405-ARM64_LINUX branch 2 times, most recently from 3884826 to 02ce562 Compare March 3, 2025 16:17
@GordonSmith GordonSmith changed the title feat: Bump vcpkg to 2025.02.13 with arm support HPCC-33405: Add support for ubuntu arm Mar 3, 2025
@GordonSmith GordonSmith marked this pull request as ready for review March 3, 2025 16:18
@GordonSmith GordonSmith requested review from dcamper and jackdelv March 3, 2025 16:19
@GordonSmith
Copy link
Member Author

@jackdelv Can you look at the mongodb changes?

@GordonSmith GordonSmith requested a review from ghalliday March 3, 2025 16:24
Copy link
Contributor

@dcamper dcamper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. One random comment.

@@ -19,8 +19,15 @@ elseif(APPLE)
set(VCPKG_TARGET_TRIPLET "x64-osx" CACHE STRING "target triplet")
endif()
elseif(UNIX)
execute_process(COMMAND uname -m OUTPUT_VARIABLE ARCHITECTURE)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aside: If we ever want to do cross-compiling, this won't be correct.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree - I had a mental note to see if there was a better way than this (like the OSX flag).

Copy link
Contributor

@jackdelv jackdelv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GordonSmith MongoDB changes look good.

Copy link
Member

@ghalliday ghalliday left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GordonSmith the unit test failure looks significant.

Copy link
Member

@ghalliday ghalliday left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment above

@dcamper
Copy link
Contributor

dcamper commented Mar 4, 2025

It looks like there was some merging issues. Parquet tests were failing due primarily to duplicated lines in the ECL code.

testing/regress/ecl/parquetTypes.ecl:65:

integerDatasetIn := ParquetIO.Read(integerRecord, dropzoneDirectory + 'IntegerTest.parquet');
integerDatasetIn := ParquetIO.Read(integerRecord, dropzoneDirectory + '/IntegerTest.parquet');

Actually, there are many more problems with that file. For this one file, it would be better to take the previous commit from Jack than actually merge.

@GordonSmith GordonSmith force-pushed the HPCC-33405-ARM64_LINUX branch 2 times, most recently from 4ac9a44 to 3767da5 Compare March 6, 2025 17:01
Bump vcpkg to 2025.02.13 with arm support
Move antlr3c into vcpkg
DC: Fix for arm64 register naming
DC: Fix arm64 build issue in _cpyrevn
DC: Move large zero array outside of functions (jlib file unittest)
Rework dockerfiles/vcpkg/build.sh to support arm

Signed-off-by Gordon Smith <[email protected]>

Move large zero array outside of functions
@GordonSmith GordonSmith force-pushed the HPCC-33405-ARM64_LINUX branch from 215138d to 87dd939 Compare March 10, 2025 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants