Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support c10s nodejs 22 #443

Merged
merged 12 commits into from
Sep 9, 2024
Merged

Support c10s nodejs 22 #443

merged 12 commits into from
Sep 9, 2024

Conversation

phracek
Copy link
Member

@phracek phracek commented Aug 8, 2024

This pull request introduces NodeJS 22 for CentOS Stream 10.

The pull request is separated into several commits.

  1. Move version 20->22, and 20-minimal -> 22-minimal
  2. Copy version 22->20, and 22-minimal -> 20-minimal back again
  3. Replace 20/test test with symlink
  4. Fix Dockerfiles so that are a bit similar.
  5. Dockerfile.c10s updates for version 22 and 22-minimal
  6. Update documentation

phracek added 8 commits August 8, 2024 11:04
Signed-off-by: Petr "Stone" Hracek <[email protected]>
Signed-off-by: Petr "Stone" Hracek <[email protected]>
Signed-off-by: Petr "Stone" Hracek <[email protected]>
Signed-off-by: Petr "Stone" Hracek <[email protected]>
Signed-off-by: Petr "Stone" Hracek <[email protected]>
@phracek

This comment was marked as outdated.

1 similar comment
@phracek

This comment was marked as outdated.

@phracek phracek force-pushed the support_c10s_nodejs_22 branch from c413ec5 to 7af11f7 Compare August 8, 2024 14:26
@phracek phracek force-pushed the support_c10s_nodejs_22 branch from 7af11f7 to 53172ce Compare August 8, 2024 14:35
@phracek

This comment was marked as outdated.

1 similar comment
@phracek

This comment was marked as outdated.

@phracek phracek force-pushed the support_c10s_nodejs_22 branch from e013ba0 to 0e3cd0a Compare August 13, 2024 12:13
@phracek phracek force-pushed the support_c10s_nodejs_22 branch from 0e3cd0a to 5f1200a Compare August 13, 2024 13:02
@phracek
Copy link
Member Author

phracek commented Aug 13, 2024

[test]

1 similar comment
@phracek
Copy link
Member Author

phracek commented Aug 21, 2024

[test]

@phracek
Copy link
Member Author

phracek commented Aug 21, 2024

@mhdawson @lholmquist NodeJS 22 test is failing:

Running test test_run_binary_application (starting at 2024-08-21 10:23:39+00:00) ... 
-----------------------------------------------
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: 	git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: 	git branch -m <name>
Initialized empty Git repository in /root/sclorg/s2i-nodejs-container/test/test-binary/.git/
[master (root-commit) fb077df] Sample commit
 2 files changed, 28 insertions(+)
 create mode 100644 hw.js
 create mode 100644 package.json
STEP 1/8: FROM quay.io/sclorg/nodejs-22-c10s:1
STEP 2/8: LABEL "io.openshift.s2i.build.image"="quay.io/sclorg/nodejs-22-c10s:1"       "io.openshift.s2i.build.source-location"="file:///root/sclorg/s2i-nodejs-container/test/test-binary"
--> 247e9c8e0e5e
STEP 3/8: USER root
--> 48cf75fb337a
STEP 4/8: COPY upload/src/ /tmp/src
--> a509f29dd1ec

[snipped]

(by pkg-config)... failed
npm error checking for liblz4 (by compile)... failed (disable)
npm error checking for syslog (by compile)... ok
npm error checking for rapidjson (by compile)... failed (disable)
npm error checking for crc32chw (by compile)... ok
npm error checking for regex (by compile)... ok
npm error checking for rand_r (by compile)... ok
npm error checking for strndup (by compile)... ok
npm error checking for strlcpy (by compile)... ok
npm error checking for strerror_r (by compile)... ok
npm error checking for strcasestr (by compile)... ok
npm error checking for pthread_setname_gnu (by compile)... ok
npm error checking for nm (by env NM)... ok (cached)
npm error checking for python3 (by command)... ok
npm error checking for getrusage (by compile)... ok
npm error Generated Makefile.config
npm error Generated config.h
npm error
npm error Configuration summary:
npm error   prefix                   /opt/app-root/src/node_modules/node-rdkafka/build/deps
npm error   MKL_DISTRO               centos
npm error   SOLIB_EXT                .so
npm error   ARCH                     x86_64
npm error   CPU                      generic
npm error   GEN_PKG_CONFIG           y
npm error   libdir                   /opt/app-root/src/node_modules/node-rdkafka/build/deps
npm error   MKL_APP_NAME             librdkafka
npm error   MKL_APP_DESC_ONELINE     The Apache Kafka C/C++ library
npm error   LDFLAGS                  -L/opt/app-root/src/node_modules/node-rdkafka/build/deps
npm error   CC                       gcc
npm error   CXX                      g++
npm error   LD                       ld
npm error   NM                       nm
npm error   OBJDUMP                  objdump
npm error   STRIP                    strip
npm error   RANLIB                   ranlib
npm error   CPPFLAGS                 -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wcast-align
npm error   PKG_CONFIG               pkg-config
npm error   INSTALL                  /usr/bin/install
npm error   HAS_GNU_AR               y
npm error   LIB_LDFLAGS              -shared -Wl,-soname,$(LIBFILENAME)
npm error   LDFLAG_LINKERSCRIPT      -Wl,--version-script=
npm error   RDKAFKA_VERSION_STR      2.5.0
npm error   MKL_APP_VERSION          2.5.0
npm error   LIBS                     -lm -lssl -lcrypto -ldl -lpthread -lrt -lpthread -lrt
npm error   MKL_PKGCONFIG_LIBS_PRIVATE -lm -ldl -lpthread -lrt -lpthread -lrt
npm error   MKL_PKGCONFIG_REQUIRES_PRIVATE libcrypto libssl
npm error   CFLAGS                   
npm error   MKL_PKGCONFIG_REQUIRES   libcrypto libssl
npm error   CXXFLAGS                 -Wno-non-virtual-dtor
npm error   SYMDUMPER                $(NM) -D
npm error   MKL_DYNAMIC_LIBS         -lm -lssl -lcrypto -ldl -lpthread -lrt -lpthread -lrt
npm error   exec_prefix              /opt/app-root/src/node_modules/node-rdkafka/build/deps
npm error   bindir                   /opt/app-root/src/node_modules/node-rdkafka/build/deps/bin
npm error   sbindir                  /opt/app-root/src/node_modules/node-rdkafka/build/deps/sbin
npm error   libexecdir               /opt/app-root/src/node_modules/node-rdkafka/build/deps/libexec
npm error   datadir                  /opt/app-root/src/node_modules/node-rdkafka/build/deps/share
npm error   sysconfdir               /opt/app-root/src/node_modules/node-rdkafka/build/deps/etc
npm error   sharedstatedir           /opt/app-root/src/node_modules/node-rdkafka/build/deps/com
npm error   localstatedir            /opt/app-root/src/node_modules/node-rdkafka/build/deps/var
npm error   runstatedir              /opt/app-root/src/node_modules/node-rdkafka/build/deps/var/run
npm error   includedir               /opt/app-root/src/node_modules/node-rdkafka/build/deps/include
npm error   infodir                  /opt/app-root/src/node_modules/node-rdkafka/build/deps/info
npm error   mandir                   /opt/app-root/src/node_modules/node-rdkafka/build/deps/man
npm error   BUILT_WITH               GCC GXX PKGCONFIG INSTALL GNULD LDS C11THREADS LIBDL PLUGINS SSL HDRHISTOGRAM SYSLOG SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER CRC32C_HW
npm error Generated config.cache
npm error
npm error Now type 'make' to build
npm error   TOUCH 11a9e3388a67e1ca5c31c1d8da49cb6d2714eb41.intermediate
npm error   ACTION deps_librdkafka_gyp_librdkafka_target_build_dependencies 11a9e3388a67e1ca5c31c1d8da49cb6d2714eb41.intermediate
npm error make[1]: Entering directory '/opt/app-root/src/node_modules/node-rdkafka/deps/librdkafka'
npm error make[2]: Entering directory '/opt/app-root/src/node_modules/node-rdkafka/deps/librdkafka/src'
npm error gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wcast-align -I.  -c rdkafka.c -o rdkafka.o
npm error make[2]: Leaving directory '/opt/app-root/src/node_modules/node-rdkafka/deps/librdkafka/src'
npm error make[1]: Leaving directory '/opt/app-root/src/node_modules/node-rdkafka/deps/librdkafka'
npm error rm 11a9e3388a67e1ca5c31c1d8da49cb6d2714eb41.intermediate
npm error make: Leaving directory '/opt/app-root/src/node_modules/node-rdkafka/build'
npm error gyp info it worked if it ends with ok
npm error gyp info using [email protected]
npm error gyp info using [email protected] | linux | x64
npm error gyp info find Python using Python version 3.12.5 found at "/usr/bin/python3"
npm error gyp http GET https://nodejs.org/download/release/v22.4.1/node-v22.4.1-headers.tar.gz
npm error gyp http 200 https://nodejs.org/download/release/v22.4.1/node-v22.4.1-headers.tar.gz
npm error gyp http GET https://nodejs.org/download/release/v22.4.1/SHASUMS256.txt
npm error gyp http 200 https://nodejs.org/download/release/v22.4.1/SHASUMS256.txt
npm error gyp info spawn /usr/bin/python3
npm error gyp info spawn args [
npm error gyp info spawn args '/usr/lib/node_modules_22/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm error gyp info spawn args 'binding.gyp',
npm error gyp info spawn args '-f',
npm error gyp info spawn args 'make',
npm error gyp info spawn args '-I',
npm error gyp info spawn args '/opt/app-root/src/node_modules/node-rdkafka/build/config.gypi',
npm error gyp info spawn args '-I',
npm error gyp info spawn args '/usr/lib/node_modules_22/npm/node_modules/node-gyp/addon.gypi',
npm error gyp info spawn args '-I',
npm error gyp info spawn args '/opt/app-root/src/.cache/node-gyp/22.4.1/include/node/common.gypi',
npm error gyp info spawn args '-Dlibrary=shared_library',
npm error gyp info spawn args '-Dvisibility=default',
npm error gyp info spawn args '-Dnode_root_dir=/opt/app-root/src/.cache/node-gyp/22.4.1',
npm error gyp info spawn args '-Dnode_gyp_dir=/usr/lib/node_modules_22/npm/node_modules/node-gyp',
npm error gyp info spawn args '-Dnode_lib_file=/opt/app-root/src/.cache/node-gyp/22.4.1/<(target_arch)/node.lib',
npm error gyp info spawn args '-Dmodule_root_dir=/opt/app-root/src/node_modules/node-rdkafka',
npm error gyp info spawn args '-Dnode_engine=v8',
npm error gyp info spawn args '--depth=.',
npm error gyp info spawn args '--no-parallel',
npm error gyp info spawn args '--generator-output',
npm error gyp info spawn args 'build',
npm error gyp info spawn args '-Goutput_dir=.'
npm error gyp info spawn args ]
npm error gyp info spawn make
npm error gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
npm error In file included from rdkafka_int.h:116,
npm error                  from rdkafka.c:43:
npm error rdkafka_conf.h:40:10: fatal error: openssl/engine.h: No such file or directory
npm error    40 | #include <openssl/engine.h>
npm error       |          ^~~~~~~~~~~~~~~~~~
npm error compilation terminated.
npm error make[2]: *** [../mklove/Makefile.base:97: rdkafka.o] Error 1
npm error make[1]: *** [Makefile:27: libs] Error 2
npm error make: *** [deps/librdkafka.target.mk:30: 11a9e3388a67e1ca5c31c1d8da49cb6d2714eb41.intermediate] Error 2
npm error gyp ERR! build error 
npm error gyp ERR! stack Error: `make` failed with exit code: 2
npm error gyp ERR! stack at ChildProcess.<anonymous> (/usr/lib/node_modules_22/npm/node_modules/node-gyp/lib/build.js:209:23)
npm error gyp ERR! System Linux 6.11.0-0.rc3.19.el10.x86_64
npm error gyp ERR! command "/usr/bin/node-22" "/usr/lib/node_modules_22/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm error gyp ERR! cwd /opt/app-root/src/node_modules/node-rdkafka
npm error gyp ERR! node -v v22.4.1
npm error gyp ERR! node-gyp -v v10.1.0
npm error gyp ERR! not ok
npm notice
npm notice New patch version of npm available! 10.8.1 -> 10.8.2
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.2
npm notice To update run: npm install -g [email protected]
npm notice
npm error A complete log of this run can be found in: /opt/app-root/src/.npm/_logs/2024-08-21T10_23_42_981Z-debug-0.log
Error: building at STEP "RUN /usr/libexec/s2i/assemble": while running runtime: exit status 1
  ERROR: Failed to to build /tmp/tmp.u4GzdGMgWr/Dockerfile.rfa9
Test for image 'quay.io/sclorg/nodejs-22-c10s:1' FAILED (exit code: 1)
Trying to pull quay.io/sclorg/nodejs-22-c10s:1-testbinary...
Error: initializing source docker://quay.io/sclorg/nodejs-22-c10s:1-testbinary: reading manifest 1-testbinary in quay.io/sclorg/nodejs-22-c10s: unauthorized: access to the requested resource is not authorized
Waiting for container start...
Waiting for container start...

@phracek phracek requested review from lholmquist and mhdawson August 21, 2024 14:09
@hhorak hhorak force-pushed the support_c10s_nodejs_22 branch from 06efaf1 to 84ef93c Compare September 2, 2024 10:28
@hhorak
Copy link
Member

hhorak commented Sep 2, 2024

[test]

1 similar comment
@phracek
Copy link
Member Author

phracek commented Sep 3, 2024

[test]

@phracek
Copy link
Member Author

phracek commented Sep 4, 2024

@hhorak @mhdawson @lholmquist @pkubatrh What do you think about merging this pull request? Fedora 22 is GREEN, CEntOS Stream 10 22-minimal is GREEN but CentOS Stream 22 is failing on reason mentioned in comment #443 (comment). Thank you for your response.

@mhdawson
Copy link
Member

mhdawson commented Sep 5, 2024

@pacostas and I looked a bit at #443 (comment) today.

I'll let Costas add more details (he'll do that tomorrow as it was his end of day when we looked at it) but the short answer is that we don't believe it is related to Node.js, instead the base library used by all language clients that re-use the common librdkafka library will be broken.

In that context I think it should not hold up landing this PR.

My guess is that CEntOS Stream 10 22-minimal is GREEN because we don't test packages which need compilers on it.

@pacostas
Copy link
Contributor

pacostas commented Sep 6, 2024

@pacostas and I looked a bit at #443 (comment) today.

I'll let Costas add more details (he'll do that tomorrow as it was his end of day when we looked at it) but the short answer is that we don't believe it is related to Node.js, instead the base library used by all language clients that re-use the common librdkafka library will be broken.

In that context I think it should not hold up landing this PR.

My guess is that CEntOS Stream 10 22-minimal is GREEN because we don't test packages which need compilers on it.

Yes, the error is happening on the librdkafka level on this line https://github.com/confluentinc/librdkafka/blob/9416dd80fb0dba71ff73a8cb4d2b919f54651006/src/rdkafka_conf.h#L40 Practically, the node-rdkafka npm package is a wrapper of the librdkafka library. This error is not specific to nodejs as a result affects all the packages that wrap this functionality e.g. python https://docs.confluent.io/kafka-clients/python/current/overview.html#python-installation Go https://github.com/confluentinc/confluent-kafka-go etc.

@phracek phracek merged commit 60426f6 into master Sep 9, 2024
17 of 18 checks passed
@phracek phracek deleted the support_c10s_nodejs_22 branch September 9, 2024 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants