Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[INFRA] libc++ CI #236

Merged
merged 8 commits into from
Dec 5, 2023
+260 −55
Merged

[INFRA] libc++ CI #236

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
40715e9
[INFRA] Update HIBF
eseiler Nov 29, 2023
b92c2e2
[INFRA] Update seqan3
eseiler Nov 29, 2023
3b9dba2
[INFRA] Update sharg
eseiler Nov 29, 2023
ec3f7c1
[INFRA] Add libc++ CI
eseiler Nov 29, 2023
00aa07c
[INFRA] Remove redundant checks
eseiler Nov 29, 2023
6a8365d
[INFRA] Ignore unused function warning in header tests
eseiler Nov 29, 2023
207cc23
[FIX] Add missing include
eseiler Nov 29, 2023
485576e
[FIX] tests
eseiler Nov 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/ci_coverage.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,8 +68,7 @@ jobs:
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=${{ matrix.build_type }} \
-DCMAKE_CXX_FLAGS="-Werror -Wno-interference-size" \
-DCHOPPER_NATIVE_BUILD=OFF
-DCMAKE_CXX_FLAGS="-Werror -Wno-interference-size"
make -j2 gtest_build
- name: Build tests
Expand Down
20 changes: 11 additions & 9 deletions .github/workflows/ci_linux.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,21 +35,27 @@ jobs:
compiler: "clang-17"
build: unit
build_type: Release
cxx_flags: "-stdlib=libc++"

- name: "gcc13"
compiler: "gcc-13"
build_type: Release
cxx_flags: -Wno-interference-size
cxx_flags: "-Wno-interference-size"

- name: "gcc12"
compiler: "gcc-12"
build_type: Release
cxx_flags: -Wno-interference-size
cxx_flags: "-Wno-interference-size"

- name: "gcc11"
compiler: "gcc-11"
build_type: Release
cxx_flags: -Wno-interference-size
cxx_flags: "-Wno-interference-size"

- name: "IntelLLVM"
compiler: "intel"
build_type: Release
cxx_flags: "-fp-model=strict"

steps:
- name: Checkout
Expand All @@ -64,11 +70,8 @@ jobs:
compiler: ${{ matrix.compiler }}
ccache_size: 125M

- name: Install OpenMP
if: contains(matrix.name, 'clang')
run: install libomp-17-dev

- name: Install CMake
if: contains(matrix.compiler, 'intel') == false
uses: seqan/actions/setup-cmake@main
with:
cmake: 3.18.6
Expand All @@ -78,8 +81,7 @@ jobs:
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=${{ matrix.build_type }} \
-DCMAKE_CXX_FLAGS="-Werror ${{ matrix.cxx_flags }}" \
-DCHOPPER_NATIVE_BUILD=OFF
-DCMAKE_CXX_FLAGS="-Werror ${{ matrix.cxx_flags }}"
make -j2 gtest_build

- name: Build tests
Expand Down
11 changes: 9 additions & 2 deletions .github/workflows/ci_macos.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,17 +31,25 @@ jobs:
fail-fast: true
matrix:
include:
- name: "clang17"
compiler: "clang-17"
build_type: Release
cxx_flags: "-stdlib=libc++" # For verbosity. This is already the default.

- name: "gcc13"
compiler: "gcc-13"
build_type: Release
cxx_flags: "-Wno-interference-size"

- name: "gcc12"
compiler: "gcc-12"
build_type: Release
cxx_flags: "-Wno-interference-size"

- name: "gcc11"
compiler: "gcc-11"
build_type: Release
cxx_flags: "-Wno-interference-size"

steps:
- name: Checkout
Expand All @@ -66,8 +74,7 @@ jobs:
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=${{ matrix.build_type }} \
-DCMAKE_CXX_FLAGS="-Werror -Wno-interference-size" \
-DCHOPPER_NATIVE_BUILD=OFF
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is -DCHOPPER_NATIVE_BUILD=OFF removed completely?

Copy link
Member Author

@eseiler eseiler Dec 5, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is there because the ci sometimes crashed because of illegal instruction.
But this doesn't seem to happen anymore :)

-DCMAKE_CXX_FLAGS="-Werror ${{ matrix.cxx_flags }}"
make -j3 gtest_build

- name: Build tests
Expand Down
13 changes: 10 additions & 3 deletions .github/workflows/ci_misc.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,17 +29,25 @@ jobs:
fail-fast: false
matrix:
include:
- name: "Header clang17"
compiler: "clang-17"
build_type: Release
cxx_flags: "-stdlib=libc++"

- name: "Header gcc13"
compiler: "gcc-13"
build_type: Release
cxx_flags: "-Wno-interference-size"

- name: "Header gcc12"
compiler: "gcc-12"
build_type: Release
cxx_flags: "-Wno-interference-size"

- name: "Header gcc11"
compiler: "gcc-11"
build_type: Release
cxx_flags: "-Wno-interference-size"

steps:
- name: Checkout
Expand All @@ -64,9 +72,8 @@ jobs:
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=${{ matrix.build_type }} \
-DCMAKE_CXX_FLAGS="-Werror -Wno-interference-size" \
-DCHOPPER_HEADER_TEST_ONLY=ON \
-DCHOPPER_NATIVE_BUILD=OFF
-DCMAKE_CXX_FLAGS="-Werror ${{ matrix.cxx_flags }}" \
-DCHOPPER_HEADER_TEST_ONLY=ON

- name: Build tests
env:
Expand Down
31 changes: 1 addition & 30 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ cmake_minimum_required (VERSION 3.18)

# Define the application name and version.
project (chopper
LANGUAGES CXX C
LANGUAGES CXX
VERSION 1.0.0
)

Expand All @@ -28,40 +28,11 @@ if (NOT CMAKE_BUILD_TYPE)
)
endif ()

# Check CXX and C compiler versions.
string (REGEX MATCH "^[0-9]+" CHOPPER_CXX_MAJOR_VERSION "${CMAKE_CXX_COMPILER_VERSION}")
string (REGEX MATCH "^[0-9]+" CHOPPER_C_MAJOR_VERSION "${CMAKE_C_COMPILER_VERSION}")
if (NOT "${CHOPPER_CXX_MAJOR_VERSION}" STREQUAL "${CHOPPER_C_MAJOR_VERSION}")
message (FATAL_ERROR "CXX and C compiler major versions differ (${CHOPPER_CXX_MAJOR_VERSION} and\
${CHOPPER_C_MAJOR_VERSION})! This will likely result in linker errors. Please set both\
-DCMAKE_CXX_COMPILER and -DCMAKE_C_COMPILER to use the same major version, and/or set the\
environment variables CXX and CC accordingly."
)
endif ()

set (CHOPPER_SUBMODULES_DIR
"${CMAKE_CURRENT_LIST_DIR}/lib"
CACHE STRING "Directory containing submodules."
)

# Flag checks
include (CheckCXXCompilerFlag)

check_cxx_compiler_flag ("-fopenmp" CHOPPER_HAS_OPENMP)
if (CHOPPER_HAS_OPENMP)
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fopenmp")
endif ()

check_cxx_compiler_flag ("-fopenmp-simd" CHOPPER_HAS_OPENMP_SIMD)
if (CHOPPER_HAS_OPENMP_SIMD)
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fopenmp-simd -DSIMDE_ENABLE_OPENMP")
endif ()

check_cxx_compiler_flag ("-fopenmp-simd" CHOPPER_SUPPRESS_GCC4_ABI)
if (CHOPPER_SUPPRESS_GCC4_ABI)
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-psabi")
endif ()

# Specify the directories where to store the built archives, libraries and executables
set (CMAKE_ARCHIVE_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/lib")
set (CMAKE_LIBRARY_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/lib")
Expand Down
2 changes: 1 addition & 1 deletion lib/hibf
Open in desktop
Submodule hibf updated 25 files
+0 −1 .github/workflows/ci_coverage.yml
+2 −6 .github/workflows/ci_linux.yml
+5 −1 .github/workflows/ci_macos.yml
+22 −1 .github/workflows/ci_misc.yml
+110 −0 .github/workflows/ci_sanitizer.yml
+1 −2 .github/workflows/ci_util.yml
+37 −17 cmake/configuration.cmake
+1 −0 cmake/hibf-config.cmake.in
+2 −0 include/hibf/layout/layout.hpp
+46 −5 include/hibf/misc/bit_vector.hpp
+1 −0 include/hibf/sketch/hyperloglog.hpp
+1 −1 src/CMakeLists.txt
+7 −4 src/layout/graph.cpp
+7 −0 src/layout/layout.cpp
+54 −18 src/sketch/hyperloglog.cpp
+12 −0 test/all/CMakeLists.txt
+6 −3 test/header/CMakeLists.txt
+1 −0 test/hibf-test.cmake
+5 −0 test/performance/sketch/CMakeLists.txt
+65 −0 test/performance/sketch/hyperloglog_benchmark.cpp
+1 −1 test/snippet/CMakeLists.txt
+1 −0 test/snippet/compare_snippet_output.cmake
+31 −20 test/unit/hibf/layout/layout_test.cpp
+4 −0 test/unit/hibf/sketch/toolbox_test.cpp
+8 −0 test/unit/test/expect_same_type_test.cpp
2 changes: 1 addition & 1 deletion lib/seqan3
Open in desktop
Submodule seqan3 updated 62 files
+6 −0 .github/workflows/ci_linux.yml
+6 −0 .github/workflows/ci_macos.yml
+21 −0 .github/workflows/ci_misc.yml
+1 −1 doc/cookbook/simd_dna4.cpp
+0 −17 doc/tutorial/04_alphabet/alphabet_main.cpp
+1 −1 doc/tutorial/06_minimisers/minimisers_snippets.cpp
+0 −8 doc/tutorial/08_pairwise_alignment/configurations.cpp
+4 −2 doc/tutorial/11_read_mapper/read_mapper_step2.cpp
+2 −1 doc/tutorial/11_read_mapper/read_mapper_step3.cpp
+2 −2 include/seqan3/alignment/aligned_sequence/aligned_sequence_concept.hpp
+53 −25 include/seqan3/alignment/pairwise/alignment_result.hpp
+3 −0 include/seqan3/core/debug_stream.hpp
+6 −1 include/seqan3/core/debug_stream/debug_stream_type.hpp
+2 −2 include/seqan3/io/sam_file/format_bam.hpp
+8 −8 include/seqan3/io/sequence_file/format_embl.hpp
+4 −2 include/seqan3/io/sequence_file/format_fasta.hpp
+5 −1 include/seqan3/io/sequence_file/format_fastq.hpp
+10 −7 include/seqan3/io/sequence_file/format_genbank.hpp
+1 −0 include/seqan3/io/stream/detail/fast_istreambuf_iterator.hpp
+1 −1 include/seqan3/io/views/detail/take_until_view.hpp
+2 −2 include/seqan3/search/configuration/parallel.hpp
+14 −17 include/seqan3/search/detail/search_configurator.hpp
+24 −0 include/seqan3/std/new
+16 −1 include/seqan3/utility/char_operations/predicate_detail.hpp
+1 −1 include/seqan3/utility/simd/detail/builtin_simd.hpp
+12 −3 include/seqan3/utility/tuple/pod_tuple.hpp
+31 −0 test/api_stability/3.3.0/0001-FIX-take_until_view_test.patch
+25 −0 test/api_stability/3.3.0/0002-FIX-Change-constexpr-test.patch
+2 −2 test/api_stability/CMakeLists.txt
+4 −4 test/documentation/DoxygenLayout.xml
+1 −0 test/documentation/doc_dev/CMakeLists.txt
+2 −1 test/documentation/doc_usr/CMakeLists.txt
+2 −2 test/documentation/seqan3_doxygen_cfg.in
+5 −2 test/header/CMakeLists.txt
+3 −1 test/include/seqan3/test/performance/units.hpp
+1 −0 test/include/seqan3/test/pretty_printing.hpp
+3 −2 test/performance/alignment/global_affine_alignment_parallel_benchmark.cpp
+5 −5 test/performance/io/format_vienna_benchmark.cpp
+12 −1 test/performance/io/lowlevel_stream_input_benchmark.cpp
+3 −3 test/performance/range/container_assignment_benchmark.cpp
+2 −2 test/performance/range/container_push_back_benchmark.cpp
+8 −5 test/performance/range/container_seq_read_benchmark.cpp
+3 −3 test/performance/range/container_seq_write_benchmark.cpp
+3 −3 test/performance/search/views/view_minimiser_hash_benchmark.cpp
+1 −1 test/performance/utility/simd/views/iota_simd_benchmark.cpp
+4 −0 test/seqan3-test.cmake
+2 −1 test/snippet/CMakeLists.txt
+1 −1 test/snippet/alphabet/nucleotide/@target_alphabet@_implicit_conversion_from_@source_alphabet@_views.cpp.in
+1 −1 test/snippet/io/views/detail/take_line_view_adaptor_def.cpp
+14 −14 test/snippet/utility/char_operations/char_predicate.cpp
+1 −1 test/snippet/utility/type_pack/detail/type_pack_algorithm_all_of.cpp
+2 −1 test/unit/alignment/matrix/detail/debug_matrix_test.cpp
+9 −3 test/unit/alignment/pairwise/edit_distance/proxy_reference_test.cpp
+1 −1 test/unit/alphabet/views/char_strictly_to_test.cpp
+1 −1 test/unit/alphabet/views/validate_char_for_test.cpp
+11 −0 test/unit/core/debug_stream_test.cpp
+1 −2 test/unit/io/sequence_file/sequence_file_format_genbank_test.cpp
+8 −1 test/unit/test/expect_same_type_test.cpp
+21 −0 test/unit/test/sequence_generator_test.cpp
+4 −0 test/unit/utility/container/aligned_allocator_test.cpp
+4 −0 test/unit/utility/detail/type_name_as_string_test.cpp
+55 −50 test/unit/utility/tuple/pod_tuple_test.cpp
2 changes: 1 addition & 1 deletion lib/sharg-parser
Open in desktop
Submodule sharg-parser updated 7 files
+8 −0 .github/workflows/ci_linux.yml
+6 −0 .github/workflows/ci_macos.yml
+15 −0 .github/workflows/ci_misc.yml
+1 −0 include/sharg/version.hpp
+5 −2 test/header/CMakeLists.txt
+1 −0 test/sharg-test.cmake
+4 −0 test/unit/detail/type_name_as_string_test.cpp
1 change: 1 addition & 0 deletions src/layout/hibf_statistics.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
#endif // CHOPPER_WORKAROUND_GCC_BOGUS_MEMCPY
#include <map>
#include <numeric>
#include <sstream>
#include <string>
#include <utility>
#include <vector>
Expand Down
99 changes: 99 additions & 0 deletions test/api/layout/execute_layout_test.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -321,6 +321,104 @@ TEST(execute_test, many_ubs)
"#LOWER_LEVEL_IBF_26 fullest_technical_bin_idx:0\n"
"#LOWER_LEVEL_IBF_27 fullest_technical_bin_idx:0\n"
"#USER_BIN_IDX\tTECHNICAL_BIN_INDICES\tNUMBER_OF_TECHNICAL_BINS\n"
#ifdef _LIBCPP_VERSION // seqan::hibf::sketch::toolbox::sort_by_cardinalities is not stable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but it is deterministic within the same compiler?

what a code bloat 😓

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's deterministic/constitent within each stl (libc++/libstdc++).

The alternative would be to use:

  • std::ranges::stable_sort in the HIBF method, but this has a potentially higher complexity, and also needs additional memory
  • make each size distinct. E. G. 200,201,202,..

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nah. it's fine.

"5\t0;0\t1;14\n"
"4\t0;14\t1;10\n"
"3\t0;24\t1;10\n"
"2\t0;34\t1;10\n"
"1\t0;44\t1;10\n"
"0\t0;54\t1;10\n"
"11\t1;0\t1;14\n"
"10\t1;14\t1;10\n"
"9\t1;24\t1;10\n"
"8\t1;34\t1;10\n"
"7\t1;44\t1;10\n"
"6\t1;54\t1;10\n"
"17\t2;0\t1;14\n"
"16\t2;14\t1;10\n"
"15\t2;24\t1;10\n"
"14\t2;34\t1;10\n"
"13\t2;44\t1;10\n"
"12\t2;54\t1;10\n"
"21\t3;0\t1;26\n"
"20\t3;26\t1;26\n"
"19\t3;52\t1;6\n"
"18\t3;58\t1;6\n"
"24\t4;0\t1;22\n"
"23\t4;22\t1;21\n"
"22\t4;43\t1;21\n"
"27\t5;0\t1;22\n"
"26\t5;22\t1;21\n"
"25\t5;43\t1;21\n"
"30\t6;0\t1;22\n"
"29\t6;22\t1;21\n"
"28\t6;43\t1;21\n"
"33\t7;0\t1;22\n"
"32\t7;22\t1;21\n"
"31\t7;43\t1;21\n"
"38\t8;0\t1;22\n"
"35\t8;22\t1;21\n"
"34\t8;43\t1;21\n"
"37\t9;0\t1;22\n"
"36\t9;22\t1;21\n"
"39\t9;43\t1;21\n"
"55\t10\t1\n"
"54\t11\t1\n"
"53\t12\t1\n"
"52\t13\t1\n"
"51\t14\t1\n"
"50\t15\t1\n"
"49\t16\t1\n"
"57\t17\t1\n"
"47\t18\t1\n"
"46\t19\t1\n"
"45\t20\t1\n"
"44\t21\t1\n"
"43\t22\t1\n"
"42\t23\t1\n"
"41\t24\t1\n"
"40\t25\t1\n"
"59\t26;0\t1;32\n"
"58\t26;32\t1;32\n"
"48\t27;0\t1;32\n"
"56\t27;32\t1;32\n"
"60\t28\t1\n"
"61\t29\t1\n"
"62\t30\t1\n"
"63\t31\t1\n"
"64\t32\t1\n"
"65\t33\t1\n"
"66\t34\t1\n"
"67\t35\t1\n"
"68\t36\t1\n"
"69\t37\t1\n"
"70\t38\t1\n"
"71\t39\t1\n"
"72\t40\t1\n"
"73\t41\t1\n"
"74\t42\t1\n"
"75\t43\t1\n"
"76\t44\t1\n"
"79\t45\t1\n"
"78\t46\t1\n"
"77\t47\t1\n"
"80\t48\t1\n"
"81\t49\t1\n"
"82\t50\t1\n"
"83\t51\t1\n"
"84\t52\t1\n"
"85\t53\t1\n"
"86\t54\t1\n"
"87\t55\t1\n"
"88\t56\t1\n"
"89\t57\t1\n"
"90\t58\t1\n"
"91\t59\t1\n"
"92\t60\t1\n"
"93\t61\t1\n"
"94\t62\t1\n"
"95\t63\t1\n"};
#else
"15\t0;0\t1;14\n"
"16\t0;14\t1;10\n"
"17\t0;24\t1;10\n"
Expand Down Expand Up @@ -417,6 +515,7 @@ TEST(execute_test, many_ubs)
"81\t61\t1\n"
"80\t62\t1\n"
"95\t63\t1\n"};
#endif
std::string const actual_file{string_from_file(layout_file)};

EXPECT_EQ(actual_file, expected_file) << actual_file << std::endl;
Expand Down
Loading