HTTP-Daemon-6.05 breaks WWW::Mechanize

[jkeenan]

CPANtester Carlos Guevara informed me yesterday that tests of WWW::Mechanize were hanging on various smoke rigs he runs. I confirmed his hunch that the failing tests are run only when HTTP::Daemon is already installed against the underlying perl. The failure will look like this:

$ bleadprove -vb t/local/referer.t 
t/local/referer.t .. 
1..13
ok 1 - use WWW::Mechanize;
ok 2 - An object of class 'WWW::Mechanize' isa 'WWW::Mechanize'
Error GETing http://127.0.0.1:65263/: Can't connect to 127.0.0.1:65263 (Connection refused) at t/local/referer.t line 39.

At this point the test process hangs indefinitely; it does not die. If this installation is being attempted as part of use of a CPAN installer like cpanm or if module installation is being attempted in dependency order, then the human running the process has to intervene with a Ctrl-C.

We first hypthosized that this was a Blead Breaks CPAN problem. After considerable work -- because of the need to manually kill hanging processes -- we found that this was occurring as far back as perl-5.30.0. But Carlos's smokers only began to hang in the past few days.

Our attention turned to HTTP-Daemon, which is required for WWW-Mechanize's tests and for the failing test in particular. My analysis suggests that there is a problem with HTTP-Daemon-6.05 that is not being detected by the distro's own test suite but which does appear in that of WWW-Mechanize.

To reproduce:

1. Install perl 5 blead (though perl-5.31.2 would probably suffice); install cpanm against it.
2. Get the previous version of HTTP-Daemon from CPAN:

$ wget ftp://ftp.cpan.org/pub/CPAN/modules/by-module/HTTP/HTTP-Daemon-6.04.tar.gz

3. Install it against that perl.

$ ./bin/cpanm HTTP-Daemon-6.04.tar.gz

4. Confirm that we still have older version of Daemon

$ ./bin/perl -Ilib -MHTTP::Daemon -E 'say $HTTP::Daemon::VERSION;'
6.04

5. Install other prerequisites for WWW-Mechanize ... but not Daemon and not Mechanize

$ ./bin/cpanm Test::Deep HTTP::Request HTML::HeadParser LWP::Simple URI::Escape \
  HTML::TreeBuilder HTTP::Request::Common URI HTTP::Response URI::URL LWP CGI \
  URI::file HTML::TokeParser Test::Fatal Test::Warnings LWP::UserAgent HTML::Form \
  Test::Output HTTP::Server::Simple::CGI HTTP::Cookies URI::URL

6. Re-confirm that we still have older version of Daemon

./bin/perl -Ilib -MHTTP::Daemon -E 'say $HTTP::Daemon::VERSION;'

7. Test WWW::Mechanize; no need to install it yet

$ ./bin/cpanm --test-only WWW::Mechanize
--> Working on WWW::Mechanize
Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/WWW-Mechanize-1.91.tar.gz ... OK
Configuring WWW-Mechanize-1.91 ... OK
Building and testing WWW-Mechanize-1.91 ... OK
Successfully tested WWW-Mechanize-1.91

8. Re-confirm that we still have older version of Daemon

./bin/perl -Ilib -MHTTP::Daemon -E 'say $HTTP::Daemon::VERSION;'

9. Install newer version of HTTP::Daemon

$ ./bin/cpanm HTTP::Daemon

10. Confirm that we have newer version of Daemon

./bin/perl -Ilib -MHTTP::Daemon -E 'say $HTTP::Daemon::VERSION;'
6.05

11. Re-test WWW::Mechanize now that we have newer Daemon

$ ./bin/cpanm --verbose --test-only WWW::Mechanize
...
t/local/back.t ........................... ok     
t/local/click.t .......................... ok   
t/local/click_button.t ................... ok    
t/local/content.t ........................ 1/10 # Running tests against http://127.0.0.1:35460/?xml=1
t/local/content.t ........................ ok     
t/local/encoding.t ....................... ok   
t/local/failure.t ........................ ok     
t/local/follow.t ......................... ok     
t/local/form.t ........................... ok     
t/local/get.t ............................ ok     
t/local/nonascii.t ....................... ok   
t/local/overload.t ....................... skipped: Mysteriously stopped passing, and I don't know why.
t/local/page_stack.t ..................... ok    
t/local/post.t ........................... ok   
t/local/referer.t ........................ 1/13 Error GETing http://127.0.0.1:33348/: Can't connect to 127.0.0.1:33348 (Connection refused) at t/local/referer.t line 39.

HANG

12. Reproduce the hang

$ bleadprove -vb t/local/referer.t 
t/local/referer.t .. 
1..13
ok 1 - use WWW::Mechanize;
ok 2 - An object of class 'WWW::Mechanize' isa 'WWW::Mechanize'
Error GETing http://127.0.0.1:65263/: Can't connect to 127.0.0.1:65263 (Connection refused) at t/local/referer.t line 39.

13. Manually run the WWW::Mechanize tests blocked by the hang

$ cd ~/.cpanm/latest-build/WWW-Mechanize-1.91
$ bleadprove -vb t/local/reload.t t/local/submit.t
t/local/reload.t .. 
1..15
ok 1 - use WWW::Mechanize;
ok 2 - An object of class 'LocalServer' isa 'LocalServer'
ok 3 - 'Created object' isa 'WWW::Mechanize'
ok 4 - Initial reload should fail
ok 5 - An object of class 'HTTP::Response' isa 'HTTP::Response'
ok 6 - Get google webpage
ok 7 - Valid HTML
ok 8
ok 9
ok 10 - Not HTML
ok 11 - An object of class 'HTTP::Response' isa 'HTTP::Response'
ok 12 - Valid HTML
ok 13 - WWW::Mechanize test page
ok 14 - cookies are not multiplied
ok 15 # skip Test::Memory::Cycle not installed
ok
t/local/submit.t .. 
1..13
ok 1 - use WWW::Mechanize;
ok 2 - An object of class 'LocalServer' isa 'LocalServer'
ok 3 - 'Created the object' isa 'WWW::Mechanize'
ok 4 - 'Got back a response' isa 'HTTP::Response'
ok 5 - Got the correct page
ok 6 - Got local page
ok 7 - is HTML
ok 8 - Hopefully no upload happens
ok 9 - 'Got back a response' isa 'HTTP::Response'
ok 10 - Can click "submit" ("submit" button)
ok 11 - Found "Foo"
ok 12 - No upload happens
ok 13 # skip Test::Memory::Cycle not installed
ok
All tests successful.
Files=2, Tests=28,  8 wallclock secs ( 0.04 usr  0.01 sys +  0.59 cusr  0.14 csys =  0.78 CPU)
Result: PASS

$ perl -V
Summary of my perl5 (revision 5 version 31 subversion 3) configuration:
  Commit id: 82007f754ed1e129a53fc7c964d84cddba7ca0de
  Platform:
    osname=freebsd
    osvers=11.2-stable
    archname=amd64-freebsd-thread-multi
    uname='freebsd perlmonger.nycbug.org 11.2-stable freebsd 11.2-stable #0 r339445: sat oct 20 00:08:11 utc 2018 root@perlmonger.nycbug.org:usrobjusrsrcsysgeneric amd64 '
    config_args='-des -Dusedevel -Uversiononly -Dprefix=/home/jkeenan/testing/blead -Dman1dir=none -Dman3dir=none -Duseithreads -Doptimize=-O2 -pipe -fstack-protector -fno-strict-aliasing'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_FORTIFY_SOURCE=2'
    optimize='-O2 -pipe -fstack-protector -fno-strict-aliasing'
    cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 Compatible FreeBSD Clang 6.0.1 (tags/RELEASE_601/final 335540)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags ='-pthread -Wl,-E  -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/lib /usr/local/lib /usr/lib/clang/6.0.1/lib /usr/lib
    libs=-lpthread -lgdbm -ldl -lm -lcrypt -lutil
    perllibs=-lpthread -ldl -lm -lcrypt -lutil
    libc=
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags='-DPIC -fPIC'
    lddlflags='-shared  -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_TIMES
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_IMPLICIT_CONTEXT
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_DEVEL
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
    USE_REENTRANT_API
  Built under freebsd
  Compiled at Jul 30 2019 23:20:52

Can you investigate?

Thank you very much.
Jim Keenan

[genio]

@jkeenan Thank you for all of your effort here. Our most recent change added IPv6 support by switching to IO::Socket::IP. I'll investigate with the others to see what we can come up with for a resolution.

[jkeenan]

On IRC @genio pointed me to this related ticket in WWW-Mechanize: libwww-perl/WWW-Mechanize#272

[ppisar]

This is a foreseen fallout of enabling HTTP::Daemon to listen on IPv6 sockets. There is bunch of distribution on CPAN that suffer from that. Whenever we hit them, we reported it and provided a fix. That their maintainers did not apply them is another matter. This WWW-Mechanize one is the old libwww-perl/WWW-Mechanize#101. I recommend closing HTTP-Daemon issue as not-a-bug.

[genio]

This, as pointed out, isn't really a bug with this distribution, but in the test suites of other dists. I'm closing this out here and we will pursue the issue in those distributions.

[skaji]

@jkeenan Can you try libwww-perl/WWW-Mechanize#280 ?

[jkeenan]

On 8/3/19 8:24 PM, Shoichi Kaji wrote: @jkeenan <https://github.com/jkeenan> Can you try libwww-perl/WWW-Mechanize#280 <libwww-perl/WWW-Mechanize#280> ?

For consistency with my earlier testing I would like a tarball (.gz) that holds this code (presumably with the pull request applied). I would like that so that I can call bleadcpanm WWW-Mechanize-N.NN.tar.gz Can you provide me such a tarball? Or is there some easy way to get one? Thank you very much. Jim Keenan

[karenetheridge]

You can pull down pull request branches by adding fetch = +refs/pull/*/head:refs/remotes/$REMOTENAME/pr/* to your local .git/config in the section for the new remote and then doing git fetch $REMOTENAME; git checkout $REMOTENAME/pr/280. Then dzil build --tar will give you a tarball. (You may need to git remote add $REMOTENAME git@github.com:libwww-perl/WWW-Mechanize.git first if you have not done so.)

But I have built one for you here: https://www.dropbox.com/s/p2txqi8ct394ltl/WWW-Mechanize-1.92.tar.gz?dl=0

[skaji]

@jkeenan https://skaji.github.io/WWW-Mechanize-1.92.tar.gz

[jkeenan]

Thank you for providing the tarballs. Unfortunately, neither enabled t/local/referer.t to PASS.

[blead] $ ./bin/cpanm --verbose ~/learn/perl/p5p/WWW-Mechanize-1.92.tar.gz
cpanm (App::cpanminus) 1.7044 on perl 5.031003 built for amd64-freebsd-thread-multi
Work directory is /home/jkeenan/.cpanm/work/1564922799.75723
You have make /usr/bin/make
You have LWP 6.39
You have /usr/bin/tar: bsdtar 3.3.3 - libarchive 3.3.3 zlib/1.2.11 liblzma/5.2.4 bz2lib/1.0.6 
You have /usr/bin/unzip
--> Working on /home/jkeenan/learn/perl/p5p/WWW-Mechanize-1.92.tar.gz
Fetching file:///usr/home/jkeenan/learn/perl/p5p/WWW-Mechanize-1.92.tar.gz ... OK
Unpacking WWW-Mechanize-1.92.tar.gz
x WWW-Mechanize-1.92/
x WWW-Mechanize-1.92/bin/
...
t/local/page_stack.t ..................... ok    
t/local/post.t ........................... ok   
t/local/referer.t ........................ 1/13 Error GETing http://localhost:47774/: Can't connect to localhost:47774 (Connection refused) at t/local/referer.t line 39.
^C*** Signal 2

Try t/local/referer.t by itself:

[blead] $ cd ~/.cpanm/latest-build/WWW-Mechanize-1.92
[WWW-Mechanize-1.92] $ bleadprove -vb t/local/referer.t
t/local/referer.t .. 
1..13
ok 1 - use WWW::Mechanize;
ok 2 - An object of class 'WWW::Mechanize' isa 'WWW::Mechanize'
Error GETing http://localhost:45395/: Can't connect to localhost:45395 (Connection refused) at t/local/referer.t line 39.

[skaji]

@jkeenan Thanks.
Could you also run this script in your freebsd?
https://gist.github.com/skaji/d0a71d9f02576a2f529798e1d8f6a98d

On my linux:

❯ perl test.pl
HTTP::Daemon 6.05, IO::Socket::IP 0.39, LWP::UserAgent 6.39, Net::HTTP 6.19, Socket 2.029

server
* V6Only (N/A, you can set via $ENV{V6Only})
* $server->sockhost "::"
* $server->sockport 38679
* $server->sockhostname "::"
* $server->sockservice 38679
* $server->sockdomain 10
* $server->sockaddr "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
client
* http://localhost:38679 200 OK
* http://localhost6:38679 200 OK
* http://127.0.0.1:38679 200 OK
* http://[::1]:38679 200 OK
* http://0.0.0.0:38679 200 OK
* http://[::]:38679 200 OK

[jkeenan]

$ bleadperl skaji-www-mechanize.pl
HTTP::Daemon 6.05, IO::Socket::IP 0.39, LWP::UserAgent 6.39, Net::HTTP 6.19, Socket 2.029

server
* V6Only (N/A, you can set via $ENV{V6Only})
* $server->sockhost "::"
* $server->sockport 35898
* $server->sockhostname "::"
* $server->sockservice 35898
* $server->sockdomain 28
* $server->sockaddr "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
client
* http://localhost:35898 500 Can't connect to localhost:35898 (Connection refused)
* http://localhost6:35898 500 Can't connect to localhost6:35898 (hostname nor servname provided, or not known)
* http://127.0.0.1:35898 500 Can't connect to 127.0.0.1:35898 (Connection refused)
* http://[::1]:35898 200 OK
* http://0.0.0.0:35898 500 Can't connect to 0.0.0.0:35898 (Connection refused)
* http://[::]:35898 200 OK

$ bleadperl -V
Summary of my perl5 (revision 5 version 31 subversion 3) configuration:
  Commit id: 82007f754ed1e129a53fc7c964d84cddba7ca0de
  Platform:
    osname=freebsd
    osvers=11.2-stable
    archname=amd64-freebsd-thread-multi
    uname='freebsd perlmonger.nycbug.org 11.2-stable freebsd 11.2-stable #0 r339445: 
sat oct 20 00:08:11 utc 2018 root@perlmonger.nycbug.org:usrobjusrsrcsysgeneric amd64 '
    config_args='-des -Dusedevel -Uversiononly -Dprefix=/home/jkeenan/testing/blead 
-Dman1dir=none -Dman3dir=none -Duseithreads 
-Doptimize=-O2 -pipe -fstack-protector -fno-strict-aliasing'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H 
-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include 
-D_FORTIFY_SOURCE=2'
    optimize='-O2 -pipe -fstack-protector -fno-strict-aliasing'
    cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H 
-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 Compatible FreeBSD Clang 6.0.1 
(tags/RELEASE_601/final 335540)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags ='-pthread -Wl,-E  -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/lib /usr/local/lib /usr/lib/clang/6.0.1/lib /usr/lib
    libs=-lpthread -lgdbm -ldl -lm -lcrypt -lutil
    perllibs=-lpthread -ldl -lm -lcrypt -lutil
    libc=
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags='-DPIC -fPIC'
    lddlflags='-shared  -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_TIMES
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_IMPLICIT_CONTEXT
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_DEVEL
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
    USE_REENTRANT_API
  Built under freebsd
  Compiled at Jul 30 2019 23:20:52

[ppisar]

On Sun, Aug 04, 2019 at 03:06:45PM -0700, James E Keenan wrote: ``` $ bleadperl skaji-www-mechanize.pl HTTP::Daemon 6.05, IO::Socket::IP 0.39, LWP::UserAgent 6.39, Net::HTTP 6.19, Socket 2.029 server * V6Only (N/A, you can set via $ENV{V6Only}) * $server->sockhost "::" * $server->sockport 35898 * $server->sockhostname "::" * $server->sockservice 35898 * $server->sockdomain 28 * $server->sockaddr "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0" client * http://localhost:35898 500 Can't connect to localhost:35898 (Connection refused) * http://localhost6:35898 500 Can't connect to localhost6:35898 (hostname nor servname provided, or not known) * http://127.0.0.1:35898 500 Can't connect to 127.0.0.1:35898 (Connection refused) * http://[::1]:35898 200 OK * http://0.0.0.0:35898 500 Can't connect to 0.0.0.0:35898 (Connection refused) * http://[::]:35898 200 OK ``` ```

Your host does not resolve "localhost" name to "::1" address. That violates RFC 6761: 6.3. Domain Name Reservation Considerations for "localhost." The domain "localhost." and any names falling within ".localhost." are special in the following ways: 1. Users are free to use localhost names as they would any other domain names. Users may assume that IPv4 and IPv6 address queries for localhost names will always resolve to the respective IP loopback address.

…

-- Petr