Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ansible: replace Ubuntu 18.04 Jenkins workspace with 22.04 #3722

Merged
merged 3 commits into from
May 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions ansible/MANUAL_STEPS.md
Original file line number Diff line number Diff line change
Expand Up @@ -658,15 +658,16 @@ Test the connection to the target machine with `ansible HOST -m win_ping -vvvv`.

## jenkins-workspace

The hosts labelled jenkins-workspace are used to "execute" the coordination of Jenkins jobs. Jenkins uses them to do the initial Git work to figure out what needs to be done before farming off to the actual test machines. These machines are lower powered but have large disks so they can waste space with the numerous Git repositories Jenkins will create in this process. The use of these hosts takes a load off the Jenkins master and prevents the Jenkins master from filling up its disk with Git repositories.
The hosts labelled [jenkins-workspace] are used to "execute" the coordination of Jenkins jobs. Jenkins uses them to do the initial Git work to figure out what needs to be done before farming off to the actual test machines. These machines are lower powered but have large disks so they can waste space with the numerous Git repositories Jenkins will create in this process. The use of these hosts takes a load off the Jenkins master and prevents the Jenkins master from filling up its disk with Git repositories.

Note that not all jobs can use jenkins-workspace servers for execution, some are tied to other hosts.
Note that not all jobs can use [jenkins-workspace] servers for execution, some are tied to other hosts.

The jenkins-workspace hosts are setup as standard Node.js nodes but are only given the `jenkins-workspace` label. After setup, they require the following manual steps:
The [jenkins-workspace] hosts are setup as standard Node.js nodes but are only given the [jenkins-workspace] label.

* Download the Coverity Build Tool for Linux x64 at <https://scan.coverity.com/download> (requires a Coverity login)
* Extract to `/var`, e.g. so the resulting directory looks like `/var/cov-analysis-linux64-2017.07/` or similar
* Ensure that the [node-coverity-daily](https://ci.nodejs.org/job/node-daily-coverity/configure) job matches the path used in its explicit `PATH` setting
The playbook should download and install the Coverity build tool needed for static analysis into `/var/`. The extracted build tool should end up in a directory similar to `/var/cov-analysis-linux64-2023.6.2`. This directory must match the `PATH` setting in the [node-daily-coverity][] job. According to Synopsis the tool is usually updated twice yearly -- if it is updated the directory will change and the following steps should be done:

* Run the playbook on all [jenkins-workspace][] machines so that they have the same version of the Coverity build tool installed.
* Update the [node-daily-coverity][] job so that the set `PATH` contains the new directory name.

## Docker hosts

Expand Down Expand Up @@ -777,5 +778,7 @@ pax -rf /u/unix1/SDK8_64bit_SR6_FP10.PAX.Z -ppx


[Setting up a Windows Host]: https://docs.ansible.com/ansible/latest/user_guide/windows_setup.html
[jenkins-workspace]: https://ci.nodejs.org/label/jenkins-workspace/
[newer Ansible configuration]: https://github.com/nodejs/build/tree/main/ansible
[node-daily-coverity]: https://ci.nodejs.org/job/node-daily-coverity/configure
[stand-alone]: https://github.com/nodejs/build/tree/main/setup/windows
2 changes: 1 addition & 1 deletion ansible/inventory.yml
Original file line number Diff line number Diff line change
Expand Up @@ -165,9 +165,9 @@ hosts:
rhel8-x64-2: {ip: 169.61.75.58, build_test_v8: yes}
rhel8-x64-3: {ip: 52.117.26.13, build_test_v8: yes}
rhel9-x64-1: {ip: 169.60.150.92, swap_file_size_mb: 2048}
ubuntu1804-x64-1: {ip: 52.117.26.14, alias: jenkins-workspace-6}
ubuntu2204-x64-1: {ip: 169.60.150.82}
ubuntu2204-x64-2: {ip: 169.44.168.2}
ubuntu2204-x64-3: {ip: 52.117.26.14, alias: jenkins-workspace-6}
ubuntu2204_docker-x64-1: {ip: 52.117.26.9}

- equinix_mnx:
Expand Down
4 changes: 2 additions & 2 deletions ansible/playbooks/jenkins/worker/create.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@
- hosts:
- test-equinix-ubuntu2204-x64-1
- test-equinix-ubuntu2204-x64-2
- test-ibm-ubuntu1804-x64-1
- test-ibm-ubuntu2204-x64-3

roles:
- jenkins-workspace
Expand All @@ -83,7 +83,7 @@
- release
- "!test-equinix-ubuntu2204-x64-1"
- "!test-equinix-ubuntu2204-x64-2"
- "!test-ibm-ubuntu1804-x64-1"
- "!test-ibm-ubuntu2204-x64-3"
tasks:
- name: remove node and npm packages
when:
Expand Down
55 changes: 45 additions & 10 deletions ansible/roles/jenkins-workspace/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
#

- name: Run which git-shell
changed_when: false
command: "which git-shell"
register: which_git_shell

Expand Down Expand Up @@ -63,6 +64,7 @@
state: link
owner: "binary_tmp"
group: "binary_tmp"
follow: false
mode: 0755

- name: Initialize Git repository
Expand Down Expand Up @@ -128,19 +130,14 @@
check_mode: no
register: release_codename

- name: Add nodesource repo
- name: Add nodesource 20 repo
apt_repository:
repo: deb https://deb.nodesource.com/node_16.x {{ release_codename.stdout }} main
repo: deb https://deb.nodesource.com/node_20.x {{ release_codename.stdout }} main
state: present

- name: Remove nodesource 14 repos
apt_repository:
repo: deb https://deb.nodesource.com/node_14.x {{ release_codename.stdout }} main
state: absent

- name: Remove nodesource 12 repos

- name: Remove nodesource 16 repo
apt_repository:
repo: deb https://deb.nodesource.com/node_12.x {{ release_codename.stdout }} main
repo: deb https://deb.nodesource.com/node_16.x {{ release_codename.stdout }} main
state: absent

- name: Install node
Expand All @@ -161,3 +158,41 @@
name: jq
state: latest
update_cache: yes

# Coverity build tool. See MANUAL_STEPS.md.
- name: Get md5sum for Coverity build tool
ansible.builtin.uri:
body:
md5: 1
token: "{{ secrets.coverity_token }}"
project: Node.js
body_format: form-urlencoded
method: POST
url: https://scan.coverity.com/download/linux64
return_content: true
register: coverity_build_tool_meta

- name: Download Coverity build tool
ansible.builtin.uri:
body:
token: "{{ secrets.coverity_token }}"
project: Node.js
body_format: form-urlencoded
dest: /tmp/
method: POST
url: https://scan.coverity.com/download/linux64
register: coverity_build_tool

- name: Validate checksum of downloaded Coverity build tool
ansible.builtin.stat:
checksum_algorithm: md5
path: "{{ coverity_build_tool.path }}"
failed_when: coverity_build_tool_file.stat.checksum != coverity_build_tool_meta.content
register: coverity_build_tool_file

- name: Unpack Coverity build tool
ansible.builtin.unarchive:
creates: "/var/{{ coverity_build_tool.path|regex_search('/tmp/(.+)\\.tar\\.gz', '\\1')|first }}"
dest: /var/
remote_src: true
src: "{{ coverity_build_tool.path }}"