Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MDBF-960: Document environment variables in master-libvirt #706

Open
wants to merge 3 commits into
base: dev
Choose a base branch
from

Conversation

cvicentiu
Copy link
Member

@cvicentiu cvicentiu commented Feb 10, 2025

This pull request cleans up and documents a lot of variables in master-libvirt.

Additionally it refactors bash_lib to handle URL archive & logic instead of having it coded into the master.cfg.

@cvicentiu cvicentiu force-pushed the refactor_master_libvirt branch 3 times, most recently from 7ae4362 to 74c34d8 Compare February 12, 2025 07:03
@cvicentiu cvicentiu changed the title WIP: Refactor master libvirt Cleanup master libvirt Feb 12, 2025
@cvicentiu cvicentiu requested a review from fauust February 12, 2025 07:06
@cvicentiu cvicentiu changed the title Cleanup master libvirt MDBF-960: Document environment variables in master-libvirt Feb 12, 2025
@cvicentiu cvicentiu force-pushed the refactor_master_libvirt branch from 74c34d8 to 86d1a4c Compare February 12, 2025 07:22
@cvicentiu cvicentiu marked this pull request as ready for review February 12, 2025 07:26
@cvicentiu
Copy link
Member Author

@fauust I need your eyes on the bash_lib.sh changes. I'm no expert in bash writing, but I have tested that the URLs are generated properly on almalinux8.

@cvicentiu
Copy link
Member Author

@RazvanLiviuVarzaru I need your review on the other code changes. Again, all these were tested locally and I don't expect surprises, but I may have missed something.

Copy link
Collaborator

@fauust fauust left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a lot of changes so this is quite complex to review without testing.
Your key is not deployed on VMs but I will deploy them today and explain you how you can manually test your changes on the DEV platform.

For now, error is:

INFO: setup MariaDB repository for 11.7 branch
+ command -v wget
+ local baseurl
+ wget -q --spider https://rpm.mariadb.org/11.7/almalinux9-almalinux9-amd64
+ wget -q --spider https://archive.mariadb.org/mariadb-11.7/yum//almalinux9-almalinux9-amd64
./bash_lib.sh: line 372: baseurl: unbound variable

Almalinux 9 AMD64 running :
./rpm-upgrade.sh https://buildbot.mariadb.org/#/builders/612/builds/1817.

scripts/bash_lib.sh Show resolved Hide resolved
@fauust
Copy link
Collaborator

fauust commented Feb 12, 2025

Ok, your keys are deployed (or currently being deployed on DEV), here is how to test those scripts:

1/ start a VM on the host, example with AMD64 and almalinux 9:

ssh ro-apexis-bbw03-x64 
faust@ro-apexis-bbw03-x64:~$ sudo virsh list --all
 Id    Name                                                  State
-----------------------------------------------------------------------
 1     bb-rhel9-docker                                       running
 260   bb-ro-apexis-bbw03-x64-openeuler-2403-amd64_install   running
 -     bb-ro-apexis-bbw03-x64-almalinux-8-amd64              shut off
 -     bb-ro-apexis-bbw03-x64-almalinux-9-amd64              shut off
 -     bb-ro-apexis-bbw03-x64-centos-7-amd64                 shut off
 -     bb-ro-apexis-bbw03-x64-centos-stream9-amd64           shut off
 -     bb-ro-apexis-bbw03-x64-debian-10-amd64                shut off
 -     bb-ro-apexis-bbw03-x64-debian-11-amd64                shut off
 -     bb-ro-apexis-bbw03-x64-debian-12-amd64                shut off
 -     bb-ro-apexis-bbw03-x64-debian-sid-amd64               shut off
 -     bb-ro-apexis-bbw03-x64-fedora-38-amd64                shut off
 -     bb-ro-apexis-bbw03-x64-fedora-39-amd64                shut off
 -     bb-ro-apexis-bbw03-x64-fedora-40-amd64                shut off
 -     bb-ro-apexis-bbw03-x64-fedora-41-amd64                shut off
 -     bb-ro-apexis-bbw03-x64-opensuse-15-amd64              shut off
 -     bb-ro-apexis-bbw03-x64-opensuse-1505-amd64            shut off
 -     bb-ro-apexis-bbw03-x64-opensuse-1506-amd64            shut off
 -     bb-ro-apexis-bbw03-x64-rhel-8-amd64                   shut off
 -     bb-ro-apexis-bbw03-x64-rhel-9-amd64                   shut off
 -     bb-ro-apexis-bbw03-x64-rockylinux-8-amd64             shut off
 -     bb-ro-apexis-bbw03-x64-rockylinux-9-amd64             shut off
 -     bb-ro-apexis-bbw03-x64-sles-1506-amd64                shut off
 -     bb-ro-apexis-bbw03-x64-ubuntu-1804-amd64              shut off
 -     bb-ro-apexis-bbw03-x64-ubuntu-2004-amd64              shut off
 -     bb-ro-apexis-bbw03-x64-ubuntu-2204-amd64              shut off
 -     bb-ro-apexis-bbw03-x64-ubuntu-2404-amd64              shut off
 -     bb-ro-apexis-bbw03-x64-ubuntu-2410-amd64              shut off

faust@ro-apexis-bbw03-x64:~$ sudo virsh start bb-ro-apexis-bbw03-x64-almalinux-9-amd64

2/ find the VM IP:

faust@ro-apexis-bbw03-x64:~$ sudo virsh net-dhcp-leases default 
 Expiry Time           MAC address         Protocol   IP address           Hostname                                      Client ID or DUID
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 2025-02-12 15:57:57   52:54:00:6f:7f:d8   ipv4       192.168.122.244/24   bb-ro-apexis-bbw03-x64-almalinux-9-amd64      01:52:54:00:6f:7f:d8

3/ make sure your local ssh config contain something similar to:

host bb-vm
hostname 192.168.122.244
User buildbot
ProxyJump ro-apexis-bbw03-x64

4/ scp the scripts to the VM and run a BB build:

~/buildbot refactor_master_libvirt
.venv ❯ scp -r scripts/ bb-vm:
deb-install.sh                                                                       100% 5705    52.1KB/s   00:00    
docker-library-build.sh                                                              100% 5319    48.8KB/s   00:00    
pam-test.sh                                                                          100% 2289    21.5KB/s   00:00    
aix-build-and-test.sh                                                                100% 4896    35.6KB/s   00:00    
rpm-install.sh                                                                       100% 3206    30.1KB/s   00:00    
rpm-upgrade.sh                                                                       100% 8763    80.4KB/s   00:00    
deb-upgrade.sh                                                                       100% 7851    73.5KB/s   00:00    
docker-library-test.sh                                                               100%  637     6.0KB/s   00:00    
service-test.sh                                                                      100% 7331    69.4KB/s   00:00    
bb_package_rotation.sh                                                               100% 1488    14.1KB/s   00:00    
docker-library-manifest.sh                                                           100% 6322    46.4KB/s   00:00    
bash_lib.sh                                                                          100%   25KB 157.0KB/s   00:00    

~/buildbot refactor_master_libvirt 9s
.venv ❯ ssh bb-vm
[buildbot@bb-ro-apexis-bbw03-x64-almalinux-9-amd64 ~]$ cd scripts/
[buildbot@bb-ro-apexis-bbw03-x64-almalinux-9-amd64 scripts]$ ./rpm-upgrade.sh https://buildbot.mariadb.org/#/builders/612/builds/1817

5/ power off the VM (sudo poweroff) when happy. The VM will be replaced by the original clean image.

…scripts

The arch environment variable was overused to generate proper repository
paths, when the whole information was already available as part of
dist_name and version_name.

This patch moves the URL transformation logic within bash_lib.sh,
allowing master_libvirt to pass in fewer environment variables.
@cvicentiu cvicentiu force-pushed the refactor_master_libvirt branch from 86d1a4c to 0042bfc Compare February 13, 2025 05:30
@cvicentiu
Copy link
Member Author

There are a lot of changes so this is quite complex to review without testing. Your key is not deployed on VMs but I will deploy them today and explain you how you can manually test your changes on the DEV platform.

For now, error is:

INFO: setup MariaDB repository for 11.7 branch
+ command -v wget
+ local baseurl
+ wget -q --spider https://rpm.mariadb.org/11.7/almalinux9-almalinux9-amd64
+ wget -q --spider https://archive.mariadb.org/mariadb-11.7/yum//almalinux9-almalinux9-amd64
./bash_lib.sh: line 372: baseurl: unbound variable

Fixed this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants