Skip to content

Commit c8cb1d6

Browse files
ObihoernchenObihoernchen
authored
Merge pull request #7257 from Obihoernchen/aarch64
Initial aarch64 support, cross arch stateless images with qemu-user-static, grub2 for x86
2 parents c41a2d4 + 597fea5 commit c8cb1d6

File tree

26 files changed

+169
-85
lines changed

26 files changed

+169
-85
lines changed

‎buildcore.sh‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -371,6 +371,9 @@ if [ "$OSNAME" != "AIX" ]; then
371371
if [ $? -ne 0 ]; then FAILEDRPMS="$FAILEDRPMS xCAT-genesis-scripts-x86_64"; fi
372372
./makerpm xCAT-genesis-scripts ppc64 "$EMBED"
373373
if [ $? -ne 0 ]; then FAILEDRPMS="$FAILEDRPMS xCAT-genesis-scripts-ppc64"; fi
374+
# Do not build xCAT-genesis-scripts-aarch64 yet
375+
#./makerpm xCAT-genesis-scripts aarch64 "$EMBED"
376+
#if [ $? -ne 0 ]; then FAILEDRPMS="$FAILEDRPMS xCAT-genesis-scripts-aarch64"; fi
374377
if [ "$FAILEDRPMS" = "$ORIGFAILEDRPMS" ]; then # all succeeded
375378
rm -f $DESTDIR/xCAT-genesis-scripts*rpm
376379
rm -f $SRCDIR/xCAT-genesis-scripts*rpm
@@ -392,7 +395,7 @@ for rpmname in xCAT xCATsn; do
392395
./makerpm $rpmname "$EMBED"
393396
if [ $? -ne 0 ]; then FAILEDRPMS="$FAILEDRPMS $rpmname"; fi
394397
else
395-
for arch in x86_64 ppc64 ppc64le s390x; do
398+
for arch in x86_64 ppc64 ppc64le s390x aarch64; do
396399
if [ "$rpmname" = "xCAT-OpenStack" -a "$arch" != "x86_64" ] || [ "$rpmname" = "xCAT-OpenStack-baremetal" -a "$arch" != "x86_64" ] ; then continue; fi # only bld openstack for x86_64 for now
397400
./makerpm $rpmname $arch "$EMBED"
398401
if [ $? -ne 0 ]; then FAILEDRPMS="$FAILEDRPMS $rpmname-$arch"; fi

‎buildlocal.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -123,7 +123,7 @@ echo "This is an $OSNAME system"
123123
$CURDIR/makerpm $rpmname
124124
if [ $? -ne 0 ]; then FAILEDRPMS="$FAILEDRPMS $rpmname"; fi
125125
else
126-
for arch in x86_64 ppc64 s390x; do
126+
for arch in x86_64 ppc64 s390x aarch64; do
127127
$CURDIR/makerpm $rpmname $arch
128128
if [ $? -ne 0 ]; then FAILEDRPMS="$FAILEDRPMS $rpmname-$arch"; fi
129129
done

‎docs/source/advanced/mixed_cluster/building_stateless_images.rst‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,28 @@ In a homogeneous cluster, the management node is the same hardware architecture
99

1010
The issues arises in a heterogeneous cluster, where the management node is running a different level operating system *or* hardware architecture as the compute nodes in which to deploy the image. The ``genimage`` command that builds stateless images depends on various utilities provided by the base operating system and needs to be run on a node with the same hardware architecture and *major* Operating System release as the nodes that will be booted from the image.
1111

12+
When running xCAT >= 2.17 on EL >= 8 based management node with x86_64 architecture, qemu-user-static can be used to cross-build ppc64* and aarch64 osimages. Therefore, you don't need to build images on systems with the target architecture anymore.
13+
14+
Cross-build ppc64*/aarch64 stateless/statelite image on x86_64 management node
15+
------------------------------------------------------------------------------
16+
17+
#. Download qemu-user-static binaries for ppc64le and/or aarch64: ::
18+
19+
wget https://github.com/multiarch/qemu-user-static/releases/latest/download/qemu-ppc64le-static -P /usr/bin
20+
wget https://github.com/multiarch/qemu-user-static/releases/latest/download/qemu-aarch64-static -P /usr/bin
21+
chmod 755 /usr/bin/qemu-*-static
22+
23+
#. Configure systemd-binfmt accordingly: ::
24+
25+
wget https://raw.githubusercontent.com/qemu/qemu/master/scripts/qemu-binfmt-conf.sh
26+
bash qemu-binfmt-conf.sh --systemd 'ALL' --qemu-path '/usr/bin' --qemu-suffix '-static' --persistent 'yes'
27+
systemctl restart systemd-binfmt.service
28+
29+
#. Make sure ``osimage.osarch`` of your image is set to your target architecture eg. ``ppc64le`` or ``aarch64``.
30+
31+
Now, ``genimage`` will use systemd-binfmt and the qemu-user-static binary to build the image for the specified architecture.
32+
33+
1234
Same Operating System, Different Architecture
1335
---------------------------------------------
1436

‎docs/source/advanced/mixed_cluster/index.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
Mixed Cluster
22
=============
33

4-
The concept of mixed cluster support in xCAT is managing cluster which contain different hardware architectures. Most commonly, this is usually a cluster consisting of both Power and x86 systems.
4+
The concept of mixed cluster support in xCAT is managing cluster which contain different hardware architectures. Most commonly, this is usually a cluster consisting of both Power, ARM and x86 systems.
55

66
.. toctree::
77
:maxdepth: 2

‎docs/source/advanced/mixed_cluster/support_matrix.rst‎

Lines changed: 39 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -1,49 +1,50 @@
11
Support Matrix
22
==============
33

4-
+---------+---------+---------+---------+---------+---------+---------+---------+---------+
5-
| | RHEL | SLES | RHEL | SLES | Ubuntu | RHEL | SLES | Ubuntu |
6-
| | ppc64 | ppc64 | x86_64 | x86_64 | x86_64 | ppc64le | ppc64le | ppc64el |
7-
| | CN | CN | CN | CN | CN | CN | CN | CN |
8-
+=========+=========+=========+=========+=========+=========+=========+=========+=========+
9-
| RHEL | | | | | | | | |
10-
| ppc64 | yes | yes | yes | yes | yes | yes | yes | yes |
11-
| MN/SN | | | [1]_ | [1]_ | [1]_ | | | |
12-
+---------+---------+---------+---------+---------+---------+---------+---------+---------+
13-
| SLES | | | | | | | | |
14-
| ppc64 | yes | yes | yes | yes | yes | yes | yes | yes |
15-
| MN/SN | | | [1]_ | [1]_ | [1]_ | | | |
16-
+---------+---------+---------+---------+---------+---------+---------+---------+---------+
17-
| RHEL | | | | | | | | |
18-
| x86_64 | yes | yes | yes | yes | yes | yes | yes | yes |
19-
| MN/SN | [4]_ | [4]_ | | | | | | |
20-
+---------+---------+---------+---------+---------+---------+---------+---------+---------+
21-
| SLES | | | | | | | | |
22-
| x86_64 | yes | yes | yes | yes | yes | yes | yes | yes |
23-
| MN/SN | [4]_ | [4]_ | | | | | | |
24-
+---------+---------+---------+---------+---------+---------+---------+---------+---------+
25-
| Ubuntu | | | | | | | | |
26-
| x86_64 | yes | yes | yes | yes | yes | yes | yes | yes |
27-
| MN/SN | [5]_ | [5]_ | | | | | | |
28-
+---------+---------+---------+---------+---------+---------+---------+---------+---------+
29-
| RHEL | | | | | | | | |
30-
| ppc64le | yes | yes | yes | yes | yes | yes | yes | yes |
31-
| MN/SN | [2]_ | [2]_ | | | | | | |
32-
+---------+---------+---------+---------+---------+---------+---------+---------+---------+
33-
| SLES | | | | | | | | |
34-
| ppc64le | no | no | yes | yes | yes | yes | yes | yes |
35-
| MN/SN | | | | | | | | |
36-
+---------+---------+---------+---------+---------+---------+---------+---------+---------+
37-
| Ubuntu | | | | | | | | |
38-
| ppc64el | yes | yes | yes | yes | yes | yes | yes | yes |
39-
| MN/SN | [3]_ | [3]_ | | | | | | |
40-
+---------+---------+---------+---------+---------+---------+---------+---------+---------+
4+
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+
5+
| | RHEL | SLES | RHEL | SLES | Ubuntu | RHEL | SLES | Ubuntu | RHEL | SLES | Ubuntu |
6+
| | ppc64 | ppc64 | x86_64 | x86_64 | x86_64 | ppc64le | ppc64le | ppc64el | aarch64 | aarch64 | aarch64 |
7+
| | CN | CN | CN | CN | CN | CN | CN | CN | CN | CN | CN |
8+
+=========+=========+=========+=========+=========+=========+=========+=========+=========+=========+=========+=========+
9+
| RHEL | | | | | | | | | | | |
10+
| ppc64 | yes | yes | yes | yes | yes | yes | yes | yes | no | no | no |
11+
| MN/SN | | | [1]_ | [1]_ | [1]_ | | | | | | |
12+
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+
13+
| SLES | | | | | | | | | | | |
14+
| ppc64 | yes | yes | yes | yes | yes | yes | yes | yes | no | no | no |
15+
| MN/SN | | | [1]_ | [1]_ | [1]_ | | | | | | |
16+
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+
17+
| RHEL | | | | | | | | | | | |
18+
| x86_64 | yes | yes | yes | yes | yes | yes | yes | yes | yes | no | no |
19+
| MN/SN | [4]_ | [4]_ | | | | | | | | | |
20+
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+
21+
| SLES | | | | | | | | | | | |
22+
| x86_64 | yes | yes | yes | yes | yes | yes | yes | yes | yes | no | no |
23+
| MN/SN | [4]_ | [4]_ | | | | | | | | | |
24+
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+
25+
| Ubuntu | | | | | | | | | | | |
26+
| x86_64 | yes | yes | yes | yes | yes | yes | yes | yes | yes | no | no |
27+
| MN/SN | [5]_ | [5]_ | | | | | | | | | |
28+
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+
29+
| RHEL | | | | | | | | | | | |
30+
| ppc64le | yes | yes | yes | yes | yes | yes | yes | yes | no | no | no |
31+
| MN/SN | [2]_ | [2]_ | | | | | | | | | |
32+
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+
33+
| SLES | | | | | | | | | | | |
34+
| ppc64le | no | no | yes | yes | yes | yes | yes | yes | no | no | no |
35+
| MN/SN | | | | | | | | | | | |
36+
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+
37+
| Ubuntu | | | | | | | | | | | |
38+
| ppc64el | yes | yes | yes | yes | yes | yes | yes | yes | no | no | no |
39+
| MN/SN | [3]_ | [3]_ | | | | | | | | | |
40+
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+
41+
4142

4243
Notes:
4344

4445
* The support statements refers to hardware control, operating system (os) provisioning, and general purpose system management where we do not see any obvious problems with the indicated combination.
4546

46-
* For diskless mixed cluster support, the initial diskless image must be created on a node running the target operating system version and architecture. see :doc:`/advanced/mixed_cluster/building_stateless_images` for more details.
47+
* For xCAT < 2.17 diskless mixed cluster support, the initial diskless image must be created on a node running the target operating system version and architecture. On xCAT >= 2.17 with x86 based management nodes QEMU emulation can be used for different architectures (eg. ppc64 or aarch64) if qemu-user-static is installed and configured via systemd-binfmt. See :doc:`/advanced/mixed_cluster/building_stateless_images` for more details.
4748

4849
.. rubric:: Footnotes
4950

‎docs/source/guides/admin-guides/basic_concepts/xcat_object/node.rst‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ Key Attributes
1313
The operating system deployed on this node. Valid values: AIX, rhels*, rhelc*, rhas*, centos*, SL*, fedora*, sles* (where * is the version #)
1414

1515
* arch:
16-
The hardware architecture of this node. Valid values: x86_64, ppc64, x86, ia64.
16+
The hardware architecture of this node. Valid values: x86_64, ppc64, x86, ia64, aarch64.
1717

1818
* groups:
1919
Usually, there are a set of nodes with some attributes in common, xCAT admin can define a node group containing these nodes, so that the management task can be issued against the group instead of individual nodes. A node can be a member of different groups, so the value of this attributes is a comma-delimited list of groups. At least one group is required to create a node. The new created group names should not be prefixed with "__" as this token has been preserved as the internal group name.
@@ -33,7 +33,7 @@ Key Attributes
3333
+--------------------------+----------------------+-----------------------------------+
3434
| Arch and Machine Type | OS | valid netboot options |
3535
+==========================+======================+===================================+
36-
| x86, x86_64 | ALL | pxe, xnba |
36+
| x86, x86_64 | ALL | pxe, xnba, grub2 |
3737
+--------------------------+----------------------+-----------------------------------+
3838
| ppc64 | <=rhel6, <=sles11.3 | yaboot |
3939
+--------------------------+----------------------+-----------------------------------+
@@ -42,6 +42,8 @@ Key Attributes
4242
| ppc64le NonVirtualize | ALL | petitboot |
4343
+--------------------------+----------------------+-----------------------------------+
4444
| ppc64le PowerKVM Guest | ALL | grub2,grub2-http,grub2-tftp |
45+
+--------------------------+----------------------+-----------------------------------+
46+
| aarch64 | >=el8 | grub2 |
4547
+-------------------------------------------------+-----------------------------------+
4648

4749
* postscripts:

‎docs/source/guides/admin-guides/references/man1/genimage.1.rst‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,9 @@ Besides prompting for some parameter values, the \ **genimage**\ command takes
4949

5050
If \ **-**\ **-onlyinitrd**\ is specified, genimage only regenerates the initrd for a stateless image to be used for a diskless install.
5151

52-
The \ **genimage**\ command must be run on a system that is the same architecture and same distro with same major release version as the nodes it will be used on. If the management node is not the same architecture or same distro level, copy the contents of
52+
The \ **genimage**\ command must be run on a system that is the same architecture and same distro with same major release version as the nodes it will be used on.
53+
For different architectures, \ **genimage**\ will try to use QEMU emulation if qemu-user-static is installed and configured via systemd-binfmt.
54+
If this does not work or the management node is not the same distro level, copy the contents of
5355
/opt/xcat/share/xcat/netboot/<os> to a system that is the proper architecture, and mount /install from
5456
the management node to that system. Then change directory to /opt/xcat/share/xcat/netboot/<os> and run ./genimage.
5557

@@ -70,7 +72,7 @@ OPTIONS
7072

7173
\ **-a**\ \ *arch*\
7274

73-
The hardware architecture of this node: ppc64le, x86_64, ppc64, x86, ia64, etc. If omitted, the current hardware architecture will be used.
75+
The hardware architecture of this node: ppc64le, x86_64, ppc64, x86, ia64, aarch64 etc. If omitted, the current hardware architecture will be used.
7476

7577

7678

‎docs/source/guides/admin-guides/references/man5/noderes.5.rst‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,11 +56,12 @@ noderes Attributes:
5656
.. code-block:: perl
5757
5858
Arch OS valid netboot options
59-
x86, x86_64 ALL pxe, xnba
59+
x86, x86_64 ALL pxe, xnba, grub2
6060
ppc64 <=rhel6, <=sles11.3 yaboot
6161
ppc64 >=rhels7, >=sles11.4 grub2,grub2-http,grub2-tftp
6262
ppc64le NonVirtualize ALL petitboot
6363
ppc64le PowerKVM Guest ALL grub2,grub2-http,grub2-tftp
64+
aarch64 >=el8 grub2
6465
6566
6667

‎docs/source/guides/admin-guides/references/man5/nodetype.5.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ nodetype Attributes:
5050

5151
\ **arch**\
5252

53-
The hardware architecture of this node. Valid values: x86_64, ppc64, x86, ia64.
53+
The hardware architecture of this node. Valid values: x86_64, ppc64, x86, ia64, aarch64.
5454

5555

5656

‎docs/source/guides/admin-guides/references/man5/osimage.5.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,7 @@ osimage Attributes:
110110

111111
\ **osarch**\
112112

113-
The hardware architecture of this node. Valid values: x86_64, ppc64, x86, ia64.
113+
The hardware architecture of this node. For netboot/statelite images, QEMU emulation for non-native architectures is used if qemu-user-static is installed and configured via systemd-binfmt. Valid values: x86_64, ppc64, x86, ia64, aarch64.
114114

115115

116116

0 commit comments

Comments
 (0)