Skip to content

HTTP-Daemon-6.05 breaks WWW::Mechanize #33

Closed
@jkeenan

Description

@jkeenan

CPANtester Carlos Guevara informed me yesterday that tests of WWW::Mechanize were hanging on various smoke rigs he runs. I confirmed his hunch that the failing tests are run only when HTTP::Daemon is already installed against the underlying perl. The failure will look like this:

$ bleadprove -vb t/local/referer.t 
t/local/referer.t .. 
1..13
ok 1 - use WWW::Mechanize;
ok 2 - An object of class 'WWW::Mechanize' isa 'WWW::Mechanize'
Error GETing http://127.0.0.1:65263/: Can't connect to 127.0.0.1:65263 (Connection refused) at t/local/referer.t line 39.

At this point the test process hangs indefinitely; it does not die. If this installation is being attempted as part of use of a CPAN installer like cpanm or if module installation is being attempted in dependency order, then the human running the process has to intervene with a Ctrl-C.

We first hypthosized that this was a Blead Breaks CPAN problem. After considerable work -- because of the need to manually kill hanging processes -- we found that this was occurring as far back as perl-5.30.0. But Carlos's smokers only began to hang in the past few days.

Our attention turned to HTTP-Daemon, which is required for WWW-Mechanize's tests and for the failing test in particular. My analysis suggests that there is a problem with HTTP-Daemon-6.05 that is not being detected by the distro's own test suite but which does appear in that of WWW-Mechanize.

To reproduce:

  1. Install perl 5 blead (though perl-5.31.2 would probably suffice); install cpanm against it.
  2. Get the previous version of HTTP-Daemon from CPAN:
$ wget ftp://ftp.cpan.org/pub/CPAN/modules/by-module/HTTP/HTTP-Daemon-6.04.tar.gz
  1. Install it against that perl.
$ ./bin/cpanm HTTP-Daemon-6.04.tar.gz
  1. Confirm that we still have older version of Daemon
$ ./bin/perl -Ilib -MHTTP::Daemon -E 'say $HTTP::Daemon::VERSION;'
6.04
  1. Install other prerequisites for WWW-Mechanize ... but not Daemon and not Mechanize
$ ./bin/cpanm Test::Deep HTTP::Request HTML::HeadParser LWP::Simple URI::Escape \
  HTML::TreeBuilder HTTP::Request::Common URI HTTP::Response URI::URL LWP CGI \
  URI::file HTML::TokeParser Test::Fatal Test::Warnings LWP::UserAgent HTML::Form \
  Test::Output HTTP::Server::Simple::CGI HTTP::Cookies URI::URL
  1. Re-confirm that we still have older version of Daemon
./bin/perl -Ilib -MHTTP::Daemon -E 'say $HTTP::Daemon::VERSION;'
  1. Test WWW::Mechanize; no need to install it yet
$ ./bin/cpanm --test-only WWW::Mechanize
--> Working on WWW::Mechanize
Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/WWW-Mechanize-1.91.tar.gz ... OK
Configuring WWW-Mechanize-1.91 ... OK
Building and testing WWW-Mechanize-1.91 ... OK
Successfully tested WWW-Mechanize-1.91
  1. Re-confirm that we still have older version of Daemon
./bin/perl -Ilib -MHTTP::Daemon -E 'say $HTTP::Daemon::VERSION;'
  1. Install newer version of HTTP::Daemon
$ ./bin/cpanm HTTP::Daemon
  1. Confirm that we have newer version of Daemon
./bin/perl -Ilib -MHTTP::Daemon -E 'say $HTTP::Daemon::VERSION;'
6.05
  1. Re-test WWW::Mechanize now that we have newer Daemon
$ ./bin/cpanm --verbose --test-only WWW::Mechanize
...
t/local/back.t ........................... ok     
t/local/click.t .......................... ok   
t/local/click_button.t ................... ok    
t/local/content.t ........................ 1/10 # Running tests against http://127.0.0.1:35460/?xml=1
t/local/content.t ........................ ok     
t/local/encoding.t ....................... ok   
t/local/failure.t ........................ ok     
t/local/follow.t ......................... ok     
t/local/form.t ........................... ok     
t/local/get.t ............................ ok     
t/local/nonascii.t ....................... ok   
t/local/overload.t ....................... skipped: Mysteriously stopped passing, and I don't know why.
t/local/page_stack.t ..................... ok    
t/local/post.t ........................... ok   
t/local/referer.t ........................ 1/13 Error GETing http://127.0.0.1:33348/: Can't connect to 127.0.0.1:33348 (Connection refused) at t/local/referer.t line 39.

HANG

  1. Reproduce the hang
$ bleadprove -vb t/local/referer.t 
t/local/referer.t .. 
1..13
ok 1 - use WWW::Mechanize;
ok 2 - An object of class 'WWW::Mechanize' isa 'WWW::Mechanize'
Error GETing http://127.0.0.1:65263/: Can't connect to 127.0.0.1:65263 (Connection refused) at t/local/referer.t line 39.

  1. Manually run the WWW::Mechanize tests blocked by the hang
$ cd ~/.cpanm/latest-build/WWW-Mechanize-1.91
$ bleadprove -vb t/local/reload.t t/local/submit.t
t/local/reload.t .. 
1..15
ok 1 - use WWW::Mechanize;
ok 2 - An object of class 'LocalServer' isa 'LocalServer'
ok 3 - 'Created object' isa 'WWW::Mechanize'
ok 4 - Initial reload should fail
ok 5 - An object of class 'HTTP::Response' isa 'HTTP::Response'
ok 6 - Get google webpage
ok 7 - Valid HTML
ok 8
ok 9
ok 10 - Not HTML
ok 11 - An object of class 'HTTP::Response' isa 'HTTP::Response'
ok 12 - Valid HTML
ok 13 - WWW::Mechanize test page
ok 14 - cookies are not multiplied
ok 15 # skip Test::Memory::Cycle not installed
ok
t/local/submit.t .. 
1..13
ok 1 - use WWW::Mechanize;
ok 2 - An object of class 'LocalServer' isa 'LocalServer'
ok 3 - 'Created the object' isa 'WWW::Mechanize'
ok 4 - 'Got back a response' isa 'HTTP::Response'
ok 5 - Got the correct page
ok 6 - Got local page
ok 7 - is HTML
ok 8 - Hopefully no upload happens
ok 9 - 'Got back a response' isa 'HTTP::Response'
ok 10 - Can click "submit" ("submit" button)
ok 11 - Found "Foo"
ok 12 - No upload happens
ok 13 # skip Test::Memory::Cycle not installed
ok
All tests successful.
Files=2, Tests=28,  8 wallclock secs ( 0.04 usr  0.01 sys +  0.59 cusr  0.14 csys =  0.78 CPU)
Result: PASS
$ perl -V
Summary of my perl5 (revision 5 version 31 subversion 3) configuration:
  Commit id: 82007f754ed1e129a53fc7c964d84cddba7ca0de
  Platform:
    osname=freebsd
    osvers=11.2-stable
    archname=amd64-freebsd-thread-multi
    uname='freebsd perlmonger.nycbug.org 11.2-stable freebsd 11.2-stable #0 r339445: sat oct 20 00:08:11 utc 2018 [email protected]:usrobjusrsrcsysgeneric amd64 '
    config_args='-des -Dusedevel -Uversiononly -Dprefix=/home/jkeenan/testing/blead -Dman1dir=none -Dman3dir=none -Duseithreads -Doptimize=-O2 -pipe -fstack-protector -fno-strict-aliasing'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_FORTIFY_SOURCE=2'
    optimize='-O2 -pipe -fstack-protector -fno-strict-aliasing'
    cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 Compatible FreeBSD Clang 6.0.1 (tags/RELEASE_601/final 335540)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags ='-pthread -Wl,-E  -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/lib /usr/local/lib /usr/lib/clang/6.0.1/lib /usr/lib
    libs=-lpthread -lgdbm -ldl -lm -lcrypt -lutil
    perllibs=-lpthread -ldl -lm -lcrypt -lutil
    libc=
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags='-DPIC -fPIC'
    lddlflags='-shared  -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_TIMES
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_IMPLICIT_CONTEXT
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_DEVEL
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
    USE_REENTRANT_API
  Built under freebsd
  Compiled at Jul 30 2019 23:20:52

Can you investigate?

Thank you very much.
Jim Keenan

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions