Get IO::Socket::IP into core for seamless v4/v6 support

[p5pRT]

Migrated from rt.perl.org#116433 (status was 'resolved')

Searchable as RT116433$

[p5pRT]

From @wolfsage

Created by wolfsage@gmail.com

There've been efforts to get IO​::Socket​::IP into the core distribution to
replace IO​::Socket​::INET, but they've stalled a bit.

Doing this would allow IPv4 and IPv6 support to be shipped with Perl.

I believe that this is a good idea, and in #p5p we came up with the
following plan​:

1. Encourage people to switch to IO​::Socket​::IP now
2. Branch Blead, test core Perl with an IO​::Socket​::IP drop-in replacement
of IP​::Socket​::INET
3. Once 2 is complete, Smoke CPAN against the changes
4. Evaluate results. Do most (all) things work? Will patches fix a few
modules if broken?
5. If results are good, get IO​::Socket​::IP into the core distribution,
along side IO​::Socket​::INET
6. Encourage people to switch to IO​::Socket​::IP, this will carry more wait
since it's in core, maybe.
7. Eventually, deprecate and evict IO​::Socket​::INET from the core
distribution.

Perl Info

Flags:
    category=library
    severity=low

Site configuration information for perl 5.10.1:

Configured by Debian Project at Fri Apr 22 18:29:38 UTC 2011.

Summary of my perl5 (revision 5 version 10 subversion 1) configuration:

  Platform:
    osname=linux, osvers=2.6.24-28-server,
archname=i486-linux-gnu-thread-multi
    uname='linux roseapple 2.6.24-28-server #1 smp wed aug 18 21:17:51 utc
2010 i686 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
-Dcccdlflags=-fPIC -Darchname=i486-linux-gnu -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.10 -Darchlib=/usr/lib/perl/5.10
-Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl/5.10.1
-Dsitearch=/usr/local/lib/perl/5.10.1 -Dman1dir=/usr/share/man/man1
-Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1
-Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl
-Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm
-DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.10.1
-Dd_dosuid -des'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.4.3', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib /usr/lib64
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=/lib/libc-2.11.1.so, so=so, useshrplib=true,
libperl=libperl.so.5.10.1
    gnulibc_version='2.11.1'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib
-fstack-protector'

Locally applied patches:



@INC for perl 5.10.1:
    /home/mhorsfall/perl5/lib/perl5/i686-linux
    /home/mhorsfall/perl5/lib/perl5/i486-linux-gnu-thread-multi
    /home/mhorsfall/perl5/lib/perl5
    /etc/perl
    /usr/local/lib/perl/5.10.1
    /usr/local/share/perl/5.10.1
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.10
    /usr/share/perl/5.10
    /usr/local/lib/site_perl
    .


Environment for perl 5.10.1:
    HOME=/home/mhorsfall
    LANG=C
    LANGUAGE=en_US:en
    LC_MESSAGES=en_US.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)

PATH=/home/mhorsfall/bin:/home/mhorsfall/perl5/perlbrew/bin:/home/mhorsfall/perl5/bin:/home/mhorsfall/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

PERL5LIB=/home/mhorsfall/perl5/lib/perl5/i686-linux:/home/mhorsfall/perl5/lib/perl5:
    PERLBREW_BASHRC_VERSION=0.33
    PERLBREW_HOME=/home/mhorsfall/.perlbrew
    PERLBREW_PATH=/home/mhorsfall/perl5/perlbrew/bin
    PERLBREW_ROOT=/home/mhorsfall/perl5/perlbrew
    PERL_BADLANG (unset)
    PERL_LOCAL_LIB_ROOT=/home/mhorsfall/perl5
    PERL_MB_OPT=--install_base /home/mhorsfall/perl5
    PERL_MM_OPT=INSTALL_BASE=/home/mhorsfall/perl5
    SHELL=/bin/bash

[p5pRT]

From @wolfsage

I've done some preliminary setup and testing, but I need to do more
before I move on. However it's quite late and I need sleep.

I hope to get to more soon (tomorrow, this weekend).

Cheers,

-- Matthew Horsfall (alh)

[p5pRT]

From [Unknown Contact. See original ticket]

I've done some preliminary setup and testing, but I need to do more
before I move on. However it's quite late and I need sleep.

I hope to get to more soon (tomorrow, this weekend).

Cheers,

-- Matthew Horsfall (alh)

[p5pRT]

@wolfsage - Status changed from 'new' to 'open'

[p5pRT]

From @leonerd

On Wed, Jan 16, 2013 at 07​:17​:57PM -0800, Matthew Horsfall via RT wrote​:

I've done some preliminary setup and testing, but I need to do more
before I move on. However it's quite late and I need sleep.

Hi there,

Thanks for taking a start at this one.

How is progress? What's working currently...? Is there anything that
needs doing at the moment?

--
Paul "LeoNerd" Evans

leonerd@​leonerd.org.uk
ICQ# 4135350 | Registered Linux# 179460
http​://www.leonerd.org.uk/

[p5pRT]

From @wolfsage

On Mon, Feb 4, 2013 at 7​:27 AM, Paul LeoNerd Evans
<leonerd@​leonerd.org.uk>wrote​:

Hi there,

Thanks for taking a start at this one.

How is progress? What's working currently...? Is there anything that
needs doing at the moment?

Paul,

I hadn't got much done on this, but as discussed earlier in chat, I've
created a release that's up-to-date with blead with IO​::Socket​::IP in-place
of IO​::Socket​::INET.

There are 3 tests that fail that may or may not be important (at least one
of them relies on the internals of IO​::Socket​::INET which are no longer
there)​:

Failed 3 tests out of 2252, 99.87% okay.
  ../dist/IO/t/io_multihomed.t
  ../dist/IO/t/io_sock.t
  ../dist/IO/t/io_udp.t

The full test output is attached.

If you'd like to download and hack on the branch to try to fix these, you
can clone my repo at​:

git​://github.com/wolfsage/perl-1.git

Something like​:

  git clone git​://github.com/wolfsage/perl-1.git
  cd perl-1
  git checkout rt-116433
  ./Configure -d -Dusedevel
  make
  make test

If you clone this through github, feel free to push patches with pull
requests, otherwise please email me and I can incorporate them in.

Thanks,

And again, I apologize for taking so long to get back to this.

-- Matthew Horsfall (alh)

[p5pRT]

From @wolfsage

test3.txt

[p5pRT]

From @leonerd

On Mon, Feb 04, 2013 at 07​:52​:34PM -0500, Matthew Horsfall (alh) wrote​:

Paul,

I hadn't got much done on this, but as discussed earlier in chat, I've
created a release that's up-to-date with blead with IO​::Socket​::IP in-place
of IO​::Socket​::INET.

There are 3 tests that fail that may or may not be important (at least one
of them relies on the internals of IO​::Socket​::INET which are no longer
there)​:

Failed 3 tests out of 2252, 99.87% okay.
../dist/IO/t/io_multihomed.t
../dist/IO/t/io_sock.t
../dist/IO/t/io_udp.t

The full test output is attached.

Will take a look...

If you'd like to download and hack on the branch to try to fix these, you
can clone my repo at​:

git​://github.com/wolfsage/perl-1.git

Something like​:

git clone git​://github.com/wolfsage/perl-1.git
cd perl-1
git checkout rt-116433
./Configure -d -Dusedevel
make
make test

If you clone this through github, feel free to push patches with pull
requests, otherwise please email me and I can incorporate them in.

Thanks,

And again, I apologize for taking so long to get back to this.

dist/IO/t/io_multihomed ....................................... FAILED--expected test 2, saw test 5

This unit test employs Unwarranted Chumminess With Code Under Test.
Namely, it tries to subclass IO​::Socket​::INET then relies on lots of
internals on how it works, to ensure that the methods defined in the
package on lines 30-69 get called. With :​:IP having different internals,
these methods don't get called, so the output never prints

  ok 2
  ok 3
  ok 4

I can't see a way to change INET.pm to handle this case; the entire unit
test is just too fragile.

dist/IO/t/io_sock ............................................. IO​::Socket​: Unsupported socket domain at t/io_sock.t line 155.
IO​::Socket​: Unsupported socket domain at t/io_sock.t line 173.
FAILED--expected 26 tests, saw 10

Oops; this I expect is due to a lack of

  use IO​::Socket​::IP -register;

And a lack of

  Family => AF_INET in the @​_-passing case

And that IO​::Socket​::IP didn't construct a new unbound, unconnected
socket handle when presented with only a Family argument.

dist/IO/t/io_udp .............................................. # Failed test 7 - at t/io_udp.t line 72
FAILED at test 7

Fails on the ->connected test, because of IO​::Socket​::IP's subtly
different overridden behaviour. $self->peername caches, so it never
notices the missing value. getpeername($self) is required.

Now fixed upstream in IO​::Socket​::IP.

Find attached a combined patch with newer IO​::Socket​::IP and fixes to
the INET.pm wrapper.

--
Paul "LeoNerd" Evans

leonerd@​leonerd.org.uk
ICQ# 4135350 | Registered Linux# 179460
http​://www.leonerd.org.uk/

[p5pRT]

From @leonerd

0002-Newer-IO-Socket-IP-minor-bugfixes-to-fake-INET.pm-wr.patch

From 7974d32e0d5c18a40d55b34201a0a3c011d96ca8 Mon Sep 17 00:00:00 2001
From: "Paul \"LeoNerd\" Evans" <leonerd@leonerd.org.uk>
Date: Mon, 11 Mar 2013 23:14:00 +0000
Subject: [PATCH 2/2] Newer IO::Socket::IP; minor bugfixes to fake INET.pm
 wrapper to make more of IO's unit tests pass

---
 cpan/IO-Socket-IP/lib/IO/Socket/IP.pm |   15 ++++++++++++---
 dist/IO/lib/IO/Socket/INET.pm         |    3 ++-
 2 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/cpan/IO-Socket-IP/lib/IO/Socket/IP.pm b/cpan/IO-Socket-IP/lib/IO/Socket/IP.pm
index c40bc67..26dc3de 100644
--- a/cpan/IO-Socket-IP/lib/IO/Socket/IP.pm
+++ b/cpan/IO-Socket-IP/lib/IO/Socket/IP.pm
@@ -1,7 +1,7 @@
 #  You may distribute under the terms of either the GNU General Public License
 #  or the Artistic License (the same terms as Perl itself)
 #
-#  (C) Paul Evans, 2010-2012 -- leonerd@leonerd.org.uk
+#  (C) Paul Evans, 2010-2013 -- leonerd@leonerd.org.uk
 
 package IO::Socket::IP;
 
@@ -9,7 +9,7 @@ use strict;
 use warnings;
 use base qw( IO::Socket );
 
-our $VERSION = '0.18';
+our $VERSION = '0.18_001';
 
 use Carp;
 
@@ -512,6 +512,15 @@ sub _configure
       }
    }
 
+   if( !@infos and defined $hints{family} ) {
+      # If there was a Family hint then create a plain unbound, unconnected socket
+      @infos = ( {
+         family   => $hints{family},
+         socktype => $hints{socktype},
+         protocol => $hints{protocol},
+      } );
+   }
+
    # In the nonblocking case, caller will be calling ->setup multiple times.
    # Store configuration in the object for the ->setup method
    # Yes, these are messy. Sorry, I can't help that...
@@ -633,7 +642,7 @@ sub connected
    my $self = shift;
    return defined $self->fileno &&
           !${*$self}{io_socket_ip_connect_in_progress} &&
-          defined $self->peername;
+          defined getpeername( $self ); # ->peername caches, we need to detect disconnection
 }
 
 =head1 METHODS
diff --git a/dist/IO/lib/IO/Socket/INET.pm b/dist/IO/lib/IO/Socket/INET.pm
index 3c04b57..e98461d 100644
--- a/dist/IO/lib/IO/Socket/INET.pm
+++ b/dist/IO/lib/IO/Socket/INET.pm
@@ -1,5 +1,6 @@
 package IO::Socket::INET;
 
+use IO::Socket::IP -register;
 use base qw( IO::Socket::IP );
 use Socket qw( PF_INET );
 
@@ -7,7 +8,7 @@ sub new
 {
   my $class = shift;
   return $class->SUPER::new(Family => PF_INET, PeerAddr => shift) if @_ == 1;
-  return $class->SUPER::new(@_);
+  return $class->SUPER::new(Family => PF_INET, @_);
 }
 
 1;
-- 
1.7.10.4

[p5pRT]

From @wolfsage

Alright, I've updated https://github.com/wolfsage/perl-1/tree/rt-116433
to include the new IO​::Socket​::IP from CPAN, as well as applied your
patch to IO​::Socket​::INET.pm.

I've removed multihomed.t since it's testing internal state of a module
we're replacing for this test.

All tests pass on my system, though META.JSON and META.YAML aren't up to
date since I'm having trouble rebuilding them...

rjbs / porters - can we get this branch smoked now? What do I need to do?

Thanks,

-- Matthew Horsfall (alh)

[p5pRT]

From @rjbs

* Matthew Horsfall via RT <perlbug-followup@​perl.org> [2013-04-05T15​:39​:33]

All tests pass on my system, though META.JSON and META.YAML aren't up to
date since I'm having trouble rebuilding them...

Here, the manifest test is broken, too.

  porting/manifest.t (Wstat​: 0 Tests​: 10765 Failed​: 3)
  porting/regen.t (Wstat​: 0 Tests​: 33 Failed​: 2)
  ../dist/IO/t/io_sock.t (Wstat​: 1024 Tests​: 11 Failed​: 0)

I fixed the two porting tests and pushed the branch to
smoke-me/rjbs/io-socket-ip, but that failing IO test may be "everywhere."

The result is not immediately useful to me​:

  1..26
  ok 1
  ok 2
  ok 3
  ok 4
  ok 5
  ok 6
  ok 7
  ok 8
  ok 9
  ok 10
  ok 11
  Died at ../dist/IO/t/io_sock.t line 33, <GEN5> line 2.

There is a long pause between ok 11 and death.

--
rjbs

[p5pRT]

From @wolfsage

On Sat, Apr 6, 2013 at 10​:59 AM, Ricardo Signes via RT <
perlbug-followup@​perl.org> wrote​:

I fixed the two porting tests and pushed the branch to
smoke-me/rjbs/io-socket-ip, but that failing IO test may be "everywhere."

The result is not immediately useful to me​:

1..26
ok 1
ok 2
ok 3
ok 4
ok 5
ok 6
ok 7
ok 8
ok 9
ok 10
ok 11
Died at ../dist/IO/t/io_sock.t line 33, <GEN5> line 2.

There is a long pause between ok 11 and death.

I finally reproduced this and tracked it down.

Here in io_sock.t line 172 on​:

# Then test UDP sockets
$server = IO​::Socket->new(Domain => AF_INET,
  Proto => 'udp',
  LocalAddr => 'localhost')
  || IO​::Socket->new(Domain => AF_INET,
  Proto => 'udp',
  LocalAddr => '127.0.0.1');
$port = $server->sockport;

The call to IO​::Socket->new() is creating a handle and then blessing it to
an IO​::Socket​::IP object. This skips the extra arguments in
IO​::Socket​::INET->new() which ensure that the IO​::Socket​::IP object does
IPv4 and causes the server to listen on IPv6 when it shouldn't.

Currently discussing options with LeoNerd to get this moving along.

-- Matthew Horsfall (alh)

[p5pRT]

From @wolfsage

Whoops, forgot cc...

On Tue, Apr 23, 2013 at 12​:34 PM, Matthew Horsfall (alh) <wolfsage@​gmail.com

wrote​:

On Sat, Apr 6, 2013 at 10​:59 AM, Ricardo Signes via RT <
perlbug-followup@​perl.org> wrote​:

I fixed the two porting tests and pushed the branch to
smoke-me/rjbs/io-socket-ip, but that failing IO test may be "everywhere."

The result is not immediately useful to me​:

1..26
ok 1
ok 2
ok 3
ok 4
ok 5
ok 6
ok 7
ok 8
ok 9
ok 10
ok 11
Died at ../dist/IO/t/io_sock.t line 33, <GEN5> line 2.

There is a long pause between ok 11 and death.

I finally reproduced this and tracked it down.

Here in io_sock.t line 172 on​:

# Then test UDP sockets
$server = IO​::Socket->new(Domain => AF_INET,
Proto => 'udp',
LocalAddr => 'localhost')
|| IO​::Socket->new(Domain => AF_INET,
Proto => 'udp',
LocalAddr => '127.0.0.1');
$port = $server->sockport;

The call to IO​::Socket->new() is creating a handle and then blessing it to
an IO​::Socket​::IP object. This skips the extra arguments in
IO​::Socket​::INET->new() which ensure that the IO​::Socket​::IP object does
IPv4 and causes the server to listen on IPv6 when it shouldn't.

Currently discussing options with LeoNerd to get this moving along.

-- Matthew Horsfall (alh)

[p5pRT]

From @leonerd

On Tue, 23 Apr 2013 12​:34​:49 -0400
"Matthew Horsfall (alh)" <wolfsage@​gmail.com> wrote​:

Currently discussing options with LeoNerd to get this moving along.

This I'm classing as a bug in

  use IO​::Socket​::IP -register;

Because IO​::Socket​::new deletes the Domain argument, there is nothing
to tell the chosen subclass what domain the user wanted. This
previously hasn't been a problem because so far there was a 1​:1 mapping
from Domain to subclass. Since :​:IP wants both AF_INET and AF_INET6,
this presents a problem.

--
Paul "LeoNerd" Evans

leonerd@​leonerd.org.uk
ICQ# 4135350 | Registered Linux# 179460
http​://www.leonerd.org.uk/

[p5pRT]

From @wolfsage

On Tue, Apr 23, 2013 at 12​:34 PM, Matthew Horsfall (alh) <
wolfsage@​gmail.com> wrote​:

Currently discussing options with LeoNerd to get this moving along.

-- Matthew Horsfall (alh)

LeoNerd pushed a dev branch to CPAN of IO​::Socket​::IP containing a fix for
this, I've updated and tested it locally and it appears to solve the issue.

rjbs has pulled it into the smoke me branch and now we're waiting testing
results.

-- Matthew Horsfall (alh)