Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Clear up 'offline' agents in jenkins #2879

Closed
74 tasks
sxa opened this issue Jan 11, 2023 · 5 comments
Closed
74 tasks

Clear up 'offline' agents in jenkins #2879

sxa opened this issue Jan 11, 2023 · 5 comments
Assignees
@steelhead31
Labels
critical currency
Milestone
2023-01 (January)

Comments

@sxa
Copy link
Member

sxa commented Jan 11, 2023 •
edited by steelhead31
Loading

We have over 70 systems which are defined in jenkins but are not currently online. We should go through them and see if they are still required, available, and needed and if not remove them. I suspect that if jenkins is expecting them to be online then they are using resources on the server as it tries to reconnect to them, which will likely have contributed to the recent issues reported in #2875:

I suspect the test-docker ones can be easily identified and resolved so I'm listing them first - the others may require a bit more investigation:

(This list was generated by @steelhead31):

  • test-docker-alma8-aarch64-1
  • test-docker-alpine311-aarch64-1
  • test-docker-alpine312-aarch64-1
  • test-docker-alpine313-aarch64-1
  • test-docker-alpine314-aarch64-1
  • test-docker-alpine314-aarch64-2
  • test-docker-alpine315-aarch64-1
  • test-docker-fedora33-ppc64le-1
  • test-docker-fedora35-aarch64-1
  • test-docker-fedora35-armv8l-1
  • test-docker-fedora36-aarch64-1
  • test-docker-rocky9-aarch64-1
  • test-docker-ubi9-aarch64-1
  • test-docker-ubuntu1804-aarch64-1
  • test-docker-ubuntu1804-ppc64le-1
  • test-docker-ubuntu2004-aarch64-1
  • test-docker-ubuntu2004-armv8l-2
  • test-docker-ubuntu2004-armv8l-3
  • test-docker-ubuntu2004-ppc64le-1
  • test-docker-ubuntu2010-armv8l-2
  • test-docker-ubuntu2010-ppc64le-1
  • test-docker-ubuntu2204-armv8l-3

And the ones which aren't test-docker:

  • build-alibaba-ubuntu1804-armv8-1
  • build-alibaba-ubuntu1804-armv8-2
  • build-azure-win2012r2-x64-3-sxa
  • build-docker-ubuntu2004-armv7l-1
  • build-equinix-centos8-armv8-1
  • build-equinix-ubuntu1604-armv7l-2
  • build-equinix_esxi-solaris10-x64-1-NEW
  • build-marist-rhel79-s390x-1
  • build-osuosl-ubuntu2204-aarch64-1
  • build-scaleway-ubuntu1604-x64-1
  • build-spearhead-freebsd12-x64-1
  • build-sxa-centos7-aarch64-1
  • build-sxa-ubuntu1804-armv7l
  • docker-aws-ubuntu1604-x64-0
  • docker-aws-ubuntu1604-x64-1
  • docker-aws-ubuntu1604-x64-2
  • docker-equinix-ubuntu2004-armv8-1
  • docker-packet-ubuntu2004-armv8-1f33
  • docker-packet-ubuntu2004-armv8-1u1804
  • docker-packet-ubuntu2004-armv8-1u2004
  • docker-packet-ubuntu2004-armv8-1u2010
  • infra-aws-ubuntu1804-x64-1
  • test-aahlenst-armv7-raspbian-pi4-1
  • test-aahlenst-armv7-raspbian-pi4-2
  • test-alibaba-debian-riscv64-1
  • test-alibaba-ubuntu1804-armv8-1
  • test-alibaba-ubuntu1804-armv8-2
  • test-aws-ubuntu1804-armv8-1
  • test-aws-ubuntu2004-x64-1
  • test-azure-win11-aarch64-1
  • test-equinix-ubuntu1604-x64-1
  • test-equinix-win2012r2-x64-1
  • test-equinix_esxi-solaris10-x64-1-NEW
  • test-gdams-debian10-riscv64-1
  • test-gdams-debian10-riscv64-2
  • test-ibmcloud-rhel6-x64-1
  • test-macstadium-macos1014-x64-4
  • test-marist-ubuntu1804-s390x-1
  • test-nine-macos1015-x64-1
  • test-nine-macos1015-x64-2
  • test-osuosl-ubuntu1604-ppc64le-3
  • test-osuosl-ubuntu1604-ppc64le-4
  • test-packet-ubuntu1604-armv8-2-OFF
  • test-plctlab-ubuntu2104-riscv64-3
  • test-plctlab-ubuntu2104-riscv64-5
  • test-sxa-armv7-raspbian-pi4
  • test-sxa-armv7l-ubuntu2004-odroid-2
  • test-sxa-fedora-riscv-1
  • test-sxa-ubuntu-aarch64-jetson
  • test-sxa-ubuntu2004-aarch64-odroid
  • test-sxamb-armv7l-raspbian-pi4-1
  • test-will-debian-riscv-1
@sxa sxa added this to the 2023-01 (January) milestone Jan 11, 2023
@steelhead31 steelhead31 self-assigned this Jan 11, 2023
@sxa sxa added the currency label Jan 25, 2023
@steelhead31
Copy link
Contributor

steelhead31 commented Jan 30, 2023 •
edited
Loading

Here are the current offline machines...
Name Status IP Port Java Version
test-alibaba-ubuntu1804-armv8-2 Offline 114.119.175.125 22 Empty
build-alibaba-ubuntu1804-armv8-1 Offline 119.8.164.219 22 Empty
test-alibaba-ubuntu1804-armv8-1 Offline 119.8.166.104 22 Empty
docker-packet-ubuntu2004-armv8-1f33 Offline 139.178.82.146 2227 /usr/lib/jvm/jdk8/bin/java
build-equinix-centos8-armv8-1 Offline 139.178.86.243 22 Empty
test-docker-debian11-armv8-1 Online 139.178.86.243 2211 /usr/lib/jvm/jdk17/bin/java
test-docker-alpine314-armv8-3 Online 139.178.86.243 2214 /usr/lib/jvm/jdk17/bin/java
test-docker-alpine314-armv8-3 Online 139.178.86.243 2214 /usr/lib/jvm/jdk17/bin/java
test-docker-ubuntu1804-aarch64-1 Offline 139.178.86.243 2223 /usr/lib/jvm/jdk17/bin/java
test-docker-centos8-armv8-1 Online 139.178.86.243 2228 /usr/lib/jvm/jdk17/bin/java
test-docker-centos8-armv8-1 Online 139.178.86.243 2228 /usr/lib/jvm/jdk17/bin/java
test-docker-ubi9-aarch64-1 Offline 139.178.86.243 2229 /usr/lib/jvm/jdk17/bin/java
test-docker-alpine314-aarch64-2 Online 139.178.86.243 2231 /usr/lib/jvm/jdk17/bin/java
test-docker-alpine314-aarch64-2 Online 139.178.86.243 2231 /usr/lib/jvm/jdk17/bin/java
test-docker-ubuntu2204-armv8-2 Online 139.178.86.243 2232 /usr/lib/jvm/jdk17/bin/java
test-docker-ubuntu2204-armv8-2 Online 139.178.86.243 2232 /usr/lib/jvm/jdk17/bin/java
test-docker-fedora35-armv8-1 Online 139.178.86.243 2235 /usr/lib/jvm/jdk17/bin/java
test-docker-fedora35-armv8-1 Online 139.178.86.243 2235 /usr/lib/jvm/jdk17/bin/java
test-docker-fedora36-armv8-1 Online 139.178.86.243 2236 /usr/lib/jvm/jdk17/bin/java
test-docker-fedora36-armv8-1 Online 139.178.86.243 2236 /usr/lib/jvm/jdk17/bin/java
test-docker-ubi8-armv8-1 Offline 139.178.86.243 2247 /usr/lib/jvm/jdk17/bin/java
test-docker-ubi8-armv8-1 Offline 139.178.86.243 2247 /usr/lib/jvm/jdk17/bin/java
test-docker-alma8-armv8-2 Online 139.178.86.243 2248 /usr/lib/jvm/jdk17/bin/java
test-docker-alma8-armv8-2 Online 139.178.86.243 2248 /usr/lib/jvm/jdk17/bin/java
build-osuosl-centos74-ppc64le-2 Online 140.211.168.117 22 Empty
dockerhost-osuosl-ubuntu2004-ppc64le-1 Online 140.211.168.214 22 /usr/lib/jvm/jdk-11/bin/java
test-docker-ubuntu1804-ppc64le-1 Offline 140.211.168.214 2223 /usr/lib/jvm/jdk8/bin/java
test-osuosl-ubuntu1804-ppc64le-1 Online 140.211.168.5 22 Empty
build-osuosl-ubuntu2204-aarch64-1 Offline 140.211.169.57 22 Empty
build-osuosl-aix71-ppc64-1 Online 140.211.9.10 22 Empty
test-osuosl-aix715-ppc64-3 Online 140.211.9.168 22 /usr/java8_64/bin/java
test-osuosl-aix72-ppc64-1 Online 140.211.9.28 22 /usr/java8_64/bin/java
test-osuosl-aix72-ppc64-2 Online 140.211.9.36 22 /usr/java8_64/bin/java
test-docker-fedora35-x64-2 Online 145.40.113.173 2225 /usr/lib/jvm/jdk17/bin/java
test-docker-ubi8-x64-2 Offline 145.40.113.173 2226 /usr/lib/jvm/jdk17/bin/java
test-docker-ubuntu2004-x64-2 Online 145.40.113.173 2227 /usr/lib/jvm/jdk17/bin/java
test-docker-debian11-x64-2 Online 145.40.113.173 2228 /usr/lib/jvm/jdk17/bin/java
test-docker-debian11-x64-2 Online 145.40.113.173 2228 /usr/lib/jvm/jdk17/bin/java
test-docker-centos8-x64-2 Offline 145.40.113.173 2229 /usr/lib/jvm/jdk17/bin/java
test-docker-alpine317-x64-2 Online 145.40.113.173 2230 /usr/lib/jvm/jdk17/bin/java
test-docker-alpine314-x64-1 Online 145.40.114.58 2223 /usr/lib/jvm/jdk17/bin/java
test-docker-alpine314-x64-1 Online 145.40.114.58 2223 /usr/lib/jvm/jdk17/bin/java
test-docker-fedora35-x64-1 Offline 145.40.114.58 2225 /usr/lib/jvm/jdk17/bin/java
test-docker-fedora35-x64-1 Offline 145.40.114.58 2225 /usr/lib/jvm/jdk17/bin/java
test-docker-ubuntu2004-x64-1 Online 145.40.114.58 2227 /usr/lib/jvm/jdk17/bin/java
test-docker-debian11-x64-1 Online 145.40.114.58 2228 /usr/lib/jvm/jdk17/bin/java
test-docker-debian11-x64-1 Online 145.40.114.58 2228 /usr/lib/jvm/jdk17/bin/java
test-docker-alpine317-x64-1 Online 145.40.114.58 2230 /usr/lib/jvm/jdk17/bin/java
test-docker-alpine317-x64-1 Online 145.40.114.58 2230 /usr/lib/jvm/jdk17/bin/java
test-equinix_esxi-ubuntu2204-x64-1 Online 145.40.115.44 22 /usr/lib/jvm/jdk-17/bin/java
test-equinix_esxi-solaris10-x64-1-NEW Online 145.40.115.45 22 /usr/lib/jvm/jdk-11.0.17/bin/java
test-equinix_esxi-ubuntu2204-x64-2 Online 145.40.115.46 22 /usr/lib/jvm/jdk-17/bin/java
dockerhost-equinix-ubuntu2004-armv8-1 Online 147.75.35.203 22 /home/jenkins/jdk-11.0.10+9/bin/java
test-docker-ubuntu2004-armv8l-2 Offline 147.75.35.203 2226 /usr/lib/jvm/jdk8/bin/java
test-docker-ubuntu1804-armv8l-4 Online 147.75.35.203 2230 /usr/lib/jvm/jdk8/bin/java
test-docker-ubuntu2004-armv8l-3 Offline 147.75.35.203 2232 /usr/lib/jvm/jdk8/bin/java
test-docker-ubuntu2004-armv8l-3 Offline 147.75.35.203 2232 /usr/lib/jvm/jdk8/bin/java
test-docker-ubuntu2110-armv8l-1 Online 147.75.35.203 2235 /usr/lib/jvm/jdk8/bin/java
test-docker-ubuntu2110-armv8l-1 Online 147.75.35.203 2235 /usr/lib/jvm/jdk8/bin/java
test-docker-ubuntu2004-armv7l-3 Online 147.75.35.203 2324 /usr/lib/jvm/jdk17/bin/java
test-marist-sles15-s390x-2 Offline 148.100.74.154 22 /usr/lib/jvm/jdk-11/bin/java
test-marist-sles12-s390x-2 Online 148.100.74.193 22 /usr/lib/jvm/jdk-11/bin/java
test-docker-sles15-s390x-1 Online 148.100.74.237 2211 /usr/bin/java
build-marist-rhel79-s390x-2 Online 148.100.74.54 22 /usr/lib/jvm/jdk-17/bin/java
build-marist-rhel79-s390x-1 Offline 148.100.75.212 22 /usr/lib/jvm/jdk-17/bin/java
test-ibmcloud-rhel7-x64-1 Online 169.48.4.136 22 Empty
test-aws-rhel76-armv8-1 Online 18.202.36.216 22 Empty
build-spearhead-freebsd12-x64-1 Offline 185.131.222.224 22 /usr/local/openjdk8/bin/java
test-macstadium-macos11-arm64-1 Online 199.7.163.51 22 /Library/Java/JavaVirtualMachines/jdk-17/Contents/Home/bin/java
test-macstadium-macos11-arm64-2 Online 199.7.163.52 22 /Library/Java/JavaVirtualMachines/jdk-17/Contents/Home/bin/java
test-skytap-ubuntu2004-ppc64le-1 Online 20.61.136.210 22 Empty
dockerhost-skytap-ubuntu2004-ppc64le-1 Online 20.61.136.212 22 /usr/lib/jvm/jdk-11/bin/java
test-docker-debian11-ppc64le-1 Offline 20.61.136.212 2211 /usr/lib/jvm/jdk17/bin/java
test-docker-debian11-ppc64le-1 Offline 20.61.136.212 2211 /usr/lib/jvm/jdk17/bin/java
test-docker-debian11-ppc64le-2 Offline 20.61.136.212 2212 /usr/lib/jvm/jdk17/bin/java
test-docker-debian11-ppc64le-2 Offline 20.61.136.212 2212 /usr/lib/jvm/jdk17/bin/java
test-docker-ubuntu2204-ppc64le-1 Offline 20.61.136.212 2213 /usr/lib/jvm/jdk17/bin/java
test-docker-ubuntu2204-ppc64le-2 Offline 20.61.136.212 2214 /usr/lib/jvm/jdk17/bin/java
test-docker-ubuntu2204-ppc64le-2 Offline 20.61.136.212 2214 /usr/lib/jvm/jdk17/bin/java
test-docker-debian11-ppc64le-3 Online 20.61.136.212 2215 /usr/lib/jvm/jdk17/bin/java
test-docker-debian11-ppc64le-3 Online 20.61.136.212 2215 /usr/lib/jvm/jdk17/bin/java
test-macstadium-macos1015-x64-1 Online 207.254.28.171 22 /Library/Java/JavaVirtualMachines/jdk-17/Contents/Home/bin/java
test-nine-macos1015-x64-1 Offline 2a02:418:3001:40::10 22 Empty
test-nine-macos1015-x64-2 Offline 2a02:418:3001:40::11 22 Empty
build-alibaba-win2012r2-x64-1 Online Empty Empty Empty
build-azure-win2012r2-x64-1 Online Empty Empty Empty
build-azure-win2012r2-x64-2 Online Empty Empty Empty
build-equinix-ubuntu1604-armv7l-2 Offline Empty Empty Empty
build-ibmcloud-win2012r2-x64-2 Online Empty Empty Empty
build-sxa-centos7-aarch64-1 Offline Empty Empty Empty
test-azure-win11-aarch64-1 Offline Empty Empty Empty
test-azure-win2016-x64-1 Offline Empty Empty Empty
test-docker-ubuntu2204-armv8l-3 Offline Empty Empty Empty
test-docker-ubuntu2204-armv8l-3 Offline Empty Empty Empty
test-equinix-win2012r2-x64-1 Offline Empty Empty Empty
test-ibmcloud-win2012r2-x64-1 Offline Empty Empty Empty
test-plctlab-ubuntu2104-riscv64-0 Online Empty Empty Empty
test-plctlab-ubuntu2104-riscv64-5 Offline Empty Empty Empty
test-sxa-armv7-raspbian-pi4 Offline Empty Empty Empty
test-sxa-debian11-riscv64-1 Offline Empty Empty Empty
test-sxa-fedora-riscv-1 Offline Empty Empty Empty
test-gdams-debian10-riscv64-1 Offline gdams.ddns.net 22 /home/jenkins/jdk-11.0.10+9/bin/java
test-gdams-debian10-riscv64-2 Offline gdams.ddns.net 2222 /home/jenkins/jdk-11.0.8+10/bin/java
test-osuosl-aix715-ppc64-1 Online p9-aix1-adopt05.osuosl.org 22 /usr/java8_64/bin/java

@sxa sxa added the critical label Jan 30, 2023
@steelhead31
Copy link
Contributor

steelhead31 commented Jan 30, 2023 •
edited
Loading

The following have been removed..
Removed - test-docker-ubuntu2004-aarch64-1 - Duplicate Of test-docker-ubuntu2004-armv8-1
Removed - test-docker-fedora36-aarch64-1 - Duplicate Of test-docker-fedora36-armv8-1
Removed - test-docker-alma8-aarch64-1 - Incorrect Port Setup - Now on 2215
Removed - test-aahlenst-armv7-raspbian-pi4-1 - Not Online In > 3 years
Removed - test-aahlenst-armv7-raspbian-pi4-2 - Not Online In > 3 years
Removed - build-scaleway-ubuntu1604-x64-1 - Not Online In > 3 years
Removed - test-will-debian-riscv-1 - Not Online In > 3 years
Removed - test-docker-ubi9-aarch64-1 - Container removed may be replaced in time, but until then..
Removed - test-docker-rocky9-aarch64-1 - Container removed may be replaced in time, but until then..
Removed - test-docker-ubuntu1804-aarch64-1 - Container removed may be replaced in time, but until then..
Removed - test-docker-ubuntu2204-armv8l-3 - Container removed may be replaced in time, but until then..

@steelhead31
Copy link
Contributor

Removed 11 defunct nodes, most other nodes are either deliberately offline, or are development ones, marked as temporarily offline. Nodes marked as temporarily offline, do not constantly reconnect and consume resources.

@steelhead31
Copy link
Contributor

Also removed..

docker-equinix-ubuntu2004-armv8-1Offline139.178.82.146
docker-packet-ubuntu2004-armv8-1u1804Offline139.178.82.146
docker-packet-ubuntu2004-armv8-1u2004Offline139.178.82.146
docker-packet-ubuntu2004-armv8-1u2010Offline139.178.82.146
docker-packet-ubuntu2004-armv8-1f33Offline139.178.82.146

@sxa sxa mentioned this issue Feb 2, 2023
Closed
@steelhead31
Copy link
Contributor

Also removed
test-equinix-win2012r2-x64-1

@Haroon-Khel Haroon-Khel mentioned this issue Jul 6, 2023
Merged
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@steelhead31 steelhead31

Labels
critical currency
Projects
No open projects
Adoptium 1Q 2023 Plan
Status: Done
Milestone
2023-01 (January)
Development

No branches or pull requests
2 participants
@sxa @steelhead31