netperf_stress: Support running netperf clients on systems with more than 256 CPUs

[bgartzi]

Netperf client couldn't start on systems with more than 256 cpus. This impacts the ability to run netperf_stress tests and possible some other more in those systems.

This patch tries to alleviate the issue by modifying the number of maximum cpus supported by netperf prior to the binary compilation.

To do so, the patch is divided in two commits:

1. To compile netperf, first the source code needs to be decompressed. Then, the code is compiled into a binary. Both commands were run through the same remote session. The first commit in this patch splits the pre-compilation and compilation steps into two different steps. That way, we could modify netperf's source code on-the-go if needed.
2. The second commit adds a mechanism to detect whether the system running netperf has a number of CPUS bigger than the value defined in netperf's MAXCPUS or not. First, it finds the number of CPUs the system is running on. Then, it finds the source code file in which MAXCPUS is defined and its value. If the value found is smaller than the number of CPUs in the system, it updates it's value by issuing a sed command. If it is bigger, the value is left as-is. This happens prior to compilation.

ID: 3365

[bgartzi]

@rh-jugraham could you have a look these patches and confirm that it would help fix your issue?

[rh-jugraham]

@bgartzi This does address the issue!!

To test it out, since there are currently very few machines with a CPU count > 256, I tested to ensure this change didn't negatively affect the tests when the machine already had fewer than 256 CPUs - all tests passed. Then, I made a slight modification to have the code set MAXCPUS to a smaller value, which in turn caused the tests to fail as expected - this showed that MAXCPUS was successfully modified and it had the expected effect on the tests.

max_cpus_file_path, current_max_cpus = self._get_current_max_cpus()
        if current_max_cpus >= n_cpus:
            LOG.debug("Bypassing netperf's MAXCPUS value modification")
            # return
        n_cpus = 32     # test that changing MAXCPUS affects the test
        LOG.info("Increasing netperf's MAXCPUS to %d" % n_cpus)
        sed_cmd = 'sed -i "s/^\(#define *MAXCPUS *\)[0-9]* /\\1%d/g" %s' % (
            n_cpus,
            max_cpus_file_path,
        )

[bgartzi]

Thanks for confirming @rh-jugraham!
@yiqianwei, @yanglei-rh could you share your thoughts about this patch? It shouldn't add any regression, but it could affect test cases based on netperf, so I think you might be interested too.
I'm concerned about windows guests, although I couldn't find any similar issue reported for that scenario.

• This issue is related to number of CPUs, rather than the system being ARM or x86.
• If I understand correctly, windows netperf testing is based on an .exe that is precompiled (see https://github.com/autotest/tp-qemu/blob/master/deps/netperf/netperf.exe). That is, it's not compiled right before the test as it happens with linux guests, at least on tp-qemu.
• So this patch wouldn't fix the issue for windows guests with more than 256 CPUs.