Description
CPANtester Carlos Guevara informed me yesterday that tests of WWW::Mechanize were hanging on various smoke rigs he runs. I confirmed his hunch that the failing tests are run only when HTTP::Daemon is already installed against the underlying perl. The failure will look like this:
$ bleadprove -vb t/local/referer.t
t/local/referer.t ..
1..13
ok 1 - use WWW::Mechanize;
ok 2 - An object of class 'WWW::Mechanize' isa 'WWW::Mechanize'
Error GETing http://127.0.0.1:65263/: Can't connect to 127.0.0.1:65263 (Connection refused) at t/local/referer.t line 39.
At this point the test process hangs indefinitely; it does not die. If this installation is being attempted as part of use of a CPAN installer like cpanm
or if module installation is being attempted in dependency order, then the human running the process has to intervene with a Ctrl-C
.
We first hypthosized that this was a Blead Breaks CPAN problem. After considerable work -- because of the need to manually kill hanging processes -- we found that this was occurring as far back as perl-5.30.0. But Carlos's smokers only began to hang in the past few days.
Our attention turned to HTTP-Daemon, which is required for WWW-Mechanize's tests and for the failing test in particular. My analysis suggests that there is a problem with HTTP-Daemon-6.05 that is not being detected by the distro's own test suite but which does appear in that of WWW-Mechanize.
To reproduce:
- Install perl 5 blead (though perl-5.31.2 would probably suffice); install
cpanm
against it. - Get the previous version of HTTP-Daemon from CPAN:
$ wget ftp://ftp.cpan.org/pub/CPAN/modules/by-module/HTTP/HTTP-Daemon-6.04.tar.gz
- Install it against that perl.
$ ./bin/cpanm HTTP-Daemon-6.04.tar.gz
- Confirm that we still have older version of Daemon
$ ./bin/perl -Ilib -MHTTP::Daemon -E 'say $HTTP::Daemon::VERSION;'
6.04
- Install other prerequisites for WWW-Mechanize ... but not Daemon and not Mechanize
$ ./bin/cpanm Test::Deep HTTP::Request HTML::HeadParser LWP::Simple URI::Escape \
HTML::TreeBuilder HTTP::Request::Common URI HTTP::Response URI::URL LWP CGI \
URI::file HTML::TokeParser Test::Fatal Test::Warnings LWP::UserAgent HTML::Form \
Test::Output HTTP::Server::Simple::CGI HTTP::Cookies URI::URL
- Re-confirm that we still have older version of Daemon
./bin/perl -Ilib -MHTTP::Daemon -E 'say $HTTP::Daemon::VERSION;'
- Test WWW::Mechanize; no need to install it yet
$ ./bin/cpanm --test-only WWW::Mechanize
--> Working on WWW::Mechanize
Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/WWW-Mechanize-1.91.tar.gz ... OK
Configuring WWW-Mechanize-1.91 ... OK
Building and testing WWW-Mechanize-1.91 ... OK
Successfully tested WWW-Mechanize-1.91
- Re-confirm that we still have older version of Daemon
./bin/perl -Ilib -MHTTP::Daemon -E 'say $HTTP::Daemon::VERSION;'
- Install newer version of HTTP::Daemon
$ ./bin/cpanm HTTP::Daemon
- Confirm that we have newer version of Daemon
./bin/perl -Ilib -MHTTP::Daemon -E 'say $HTTP::Daemon::VERSION;'
6.05
- Re-test WWW::Mechanize now that we have newer Daemon
$ ./bin/cpanm --verbose --test-only WWW::Mechanize
...
t/local/back.t ........................... ok
t/local/click.t .......................... ok
t/local/click_button.t ................... ok
t/local/content.t ........................ 1/10 # Running tests against http://127.0.0.1:35460/?xml=1
t/local/content.t ........................ ok
t/local/encoding.t ....................... ok
t/local/failure.t ........................ ok
t/local/follow.t ......................... ok
t/local/form.t ........................... ok
t/local/get.t ............................ ok
t/local/nonascii.t ....................... ok
t/local/overload.t ....................... skipped: Mysteriously stopped passing, and I don't know why.
t/local/page_stack.t ..................... ok
t/local/post.t ........................... ok
t/local/referer.t ........................ 1/13 Error GETing http://127.0.0.1:33348/: Can't connect to 127.0.0.1:33348 (Connection refused) at t/local/referer.t line 39.
HANG
- Reproduce the hang
$ bleadprove -vb t/local/referer.t
t/local/referer.t ..
1..13
ok 1 - use WWW::Mechanize;
ok 2 - An object of class 'WWW::Mechanize' isa 'WWW::Mechanize'
Error GETing http://127.0.0.1:65263/: Can't connect to 127.0.0.1:65263 (Connection refused) at t/local/referer.t line 39.
- Manually run the WWW::Mechanize tests blocked by the hang
$ cd ~/.cpanm/latest-build/WWW-Mechanize-1.91
$ bleadprove -vb t/local/reload.t t/local/submit.t
t/local/reload.t ..
1..15
ok 1 - use WWW::Mechanize;
ok 2 - An object of class 'LocalServer' isa 'LocalServer'
ok 3 - 'Created object' isa 'WWW::Mechanize'
ok 4 - Initial reload should fail
ok 5 - An object of class 'HTTP::Response' isa 'HTTP::Response'
ok 6 - Get google webpage
ok 7 - Valid HTML
ok 8
ok 9
ok 10 - Not HTML
ok 11 - An object of class 'HTTP::Response' isa 'HTTP::Response'
ok 12 - Valid HTML
ok 13 - WWW::Mechanize test page
ok 14 - cookies are not multiplied
ok 15 # skip Test::Memory::Cycle not installed
ok
t/local/submit.t ..
1..13
ok 1 - use WWW::Mechanize;
ok 2 - An object of class 'LocalServer' isa 'LocalServer'
ok 3 - 'Created the object' isa 'WWW::Mechanize'
ok 4 - 'Got back a response' isa 'HTTP::Response'
ok 5 - Got the correct page
ok 6 - Got local page
ok 7 - is HTML
ok 8 - Hopefully no upload happens
ok 9 - 'Got back a response' isa 'HTTP::Response'
ok 10 - Can click "submit" ("submit" button)
ok 11 - Found "Foo"
ok 12 - No upload happens
ok 13 # skip Test::Memory::Cycle not installed
ok
All tests successful.
Files=2, Tests=28, 8 wallclock secs ( 0.04 usr 0.01 sys + 0.59 cusr 0.14 csys = 0.78 CPU)
Result: PASS
$ perl -V
Summary of my perl5 (revision 5 version 31 subversion 3) configuration:
Commit id: 82007f754ed1e129a53fc7c964d84cddba7ca0de
Platform:
osname=freebsd
osvers=11.2-stable
archname=amd64-freebsd-thread-multi
uname='freebsd perlmonger.nycbug.org 11.2-stable freebsd 11.2-stable #0 r339445: sat oct 20 00:08:11 utc 2018 [email protected]:usrobjusrsrcsysgeneric amd64 '
config_args='-des -Dusedevel -Uversiononly -Dprefix=/home/jkeenan/testing/blead -Dman1dir=none -Dman3dir=none -Duseithreads -Doptimize=-O2 -pipe -fstack-protector -fno-strict-aliasing'
hint=recommended
useposix=true
d_sigaction=define
useithreads=define
usemultiplicity=define
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler:
cc='cc'
ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_FORTIFY_SOURCE=2'
optimize='-O2 -pipe -fstack-protector -fno-strict-aliasing'
cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion=''
gccversion='4.2.1 Compatible FreeBSD Clang 6.0.1 (tags/RELEASE_601/final 335540)'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries:
ld='cc'
ldflags ='-pthread -Wl,-E -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/lib /usr/local/lib /usr/lib/clang/6.0.1/lib /usr/lib
libs=-lpthread -lgdbm -ldl -lm -lcrypt -lutil
perllibs=-lpthread -ldl -lm -lcrypt -lutil
libc=
so=so
useshrplib=false
libperl=libperl.a
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags=' '
cccdlflags='-DPIC -fPIC'
lddlflags='-shared -L/usr/local/lib -fstack-protector-strong'
Characteristics of this binary (from libperl):
Compile-time options:
HAS_TIMES
MULTIPLICITY
PERLIO_LAYERS
PERL_COPY_ON_WRITE
PERL_DONT_CREATE_GVSV
PERL_IMPLICIT_CONTEXT
PERL_MALLOC_WRAP
PERL_OP_PARENT
PERL_PRESERVE_IVUV
PERL_USE_DEVEL
USE_64_BIT_ALL
USE_64_BIT_INT
USE_ITHREADS
USE_LARGE_FILES
USE_LOCALE
USE_LOCALE_COLLATE
USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC
USE_LOCALE_TIME
USE_PERLIO
USE_PERL_ATOF
USE_REENTRANT_API
Built under freebsd
Compiled at Jul 30 2019 23:20:52
Can you investigate?
Thank you very much.
Jim Keenan