socket-0.8.3.0 test failure

[alexeyzab]

.Socket
    accept
      Inet/Stream/TCP
        cancel operation:                                        OK (1.00s)
    connect
      Inet/Stream/TCP
        connect to closed port on inetLoopback:                  OK
        connect to closed port on inetNone:                      OK
        connect to open socket on localhost:                     OK
        connect closed socket:                                   OK
        connect already connected socket:                        OK
    listen
      Inet/Datagram/UDP
        listen on bound socket:                                  OK
      Inet/Stream/TCP
        listen on bound socket:                                  OK
    send/receive
      Inet/Stream/TCP
        send and receive a chunk:                                OK
        recv empty bytestring when peer disconnected gracefully: OK
        trigger ePipe exception:                                 OK (2.01s)
      Inet/Datagram/UDP
        send and receive a datagram:                             OK (1.01s)
    sendAll/receiveAll
      Inet/Stream/TCP
        sendAll and receiveAll a 128MB chunk:                    OK (0.79s)
        sendAllLazy and receiveAll a 128MB chunk:                OK (0.53s)
        sendAllBuilder and receiveAll a 128MB chunk:             OK (0.67s)
    setSocketOption
      V6Only
        present:                                                 OK (2.01s)
        absent:                                                  OK (1.01s)
    getAddress
      getAddress after bind:                                     OK
    getAddrInfo
      getAddrInfo "127.0.0.1" "80":                              OK
      getAddrInfo "" "":                                         OK
  System.Socket.Inet
    System.Socket.Family.Inet
      inetAddressFromTuple (127,0,0,1) == inetLoopback:          OK
      inetAddressToTuple (inetAddressFromTuple x) == x:          OK
        +++ OK, passed 100 tests.
    System.Socket.Family.Inet6
      inet6AddressFromTuple (0,0,0,0,0,0,0,1) == inet6Loopback:  OK
      inet6AddressToTuple (inet6AddressFromTuple x) == x:        OK
        +++ OK, passed 100 tests.

All 24 tests passed (9.03s)
.Socket
    accept
      Inet/Stream/TCP
        cancel operation:                                        FAIL
          Exception: eAddressInUse
          
          HasCallStack backtrace:
            throwIO, called at src/System/Socket.hsc:269:58 in socket-0.8.3.0-4HRqFv0QZsQ2AY3gZSdkwp:System.Socket
          
          Use -p '/cancel operation/' to rerun this test only.
    connect
      Inet/Stream/TCP
        connect to closed port on inetLoopback:                  FAIL
          test/test.hs:91:
          connection should have failed
          Use -p '/connect to closed port on inetLoopback/' to rerun this test only.
        connect to closed port on inetNone:                      OK
        connect to open socket on localhost:                     FAIL
          Exception: eAddressInUse
          
          HasCallStack backtrace:
            throwIO, called at src/System/Socket.hsc:269:58 in socket-0.8.3.0-4HRqFv0QZsQ2AY3gZSdkwp:System.Socket
          
          Use -p '/connect to open socket on localhost/' to rerun this test only.
        connect closed socket:                                   FAIL
          test/test.hs:139:
          expected eBadFileDescriptor
          Use -p '/connect closed socket/' to rerun this test only.
        connect already connected socket:                        FAIL
          test/test.hs:159:
          expected eIsConnected
          Use -p '/connect already connected socket/' to rerun this test only.
    listen
      Inet/Datagram/UDP
        listen on bound socket:                                  OK
      Inet/Stream/TCP
        listen on bound socket:                                  FAIL
          Exception: eAddressInUse
          
          HasCallStack backtrace:
            throwIO, called at src/System/Socket.hsc:269:58 in socket-0.8.3.0-4HRqFv0QZsQ2AY3gZSdkwp:System.Socket
          
          Use -p '/Inet\/Stream\/TCP.listen on bound socket/' to rerun this test only.
    send/receive
      Inet/Stream/TCP
        send and receive a chunk:                                FAIL
          Exception: eAddressInUse
          
          HasCallStack backtrace:
            throwIO, called at src/System/Socket.hsc:269:58 in socket-0.8.3.0-4HRqFv0QZsQ2AY3gZSdkwp:System.Socket
          
          Use -p '/send and receive a chunk/' to rerun this test only.
        recv empty bytestring when peer disconnected gracefully: FAIL
          Exception: eAddressInUse
          
          HasCallStack backtrace:
            throwIO, called at src/System/Socket.hsc:269:58 in socket-0.8.3.0-4HRqFv0QZsQ2AY3gZSdkwp:System.Socket
          
          Use -p '/recv empty bytestring when peer disconnected gracefully/' to rerun this test only.
        trigger ePipe exception:                                 FAIL
          Exception: eAddressInUse
          
          HasCallStack backtrace:
            throwIO, called at src/System/Socket.hsc:269:58 in socket-0.8.3.0-4HRqFv0QZsQ2AY3gZSdkwp:System.Socket
          
          Use -p '/trigger ePipe exception/' to rerun this test only.
      Inet/Datagram/UDP
        send and receive a datagram:                             FAIL
          Exception: eAddressInUse
          
          HasCallStack backtrace:
            throwIO, called at src/System/Socket.hsc:269:58 in socket-0.8.3.0-4HRqFv0QZsQ2AY3gZSdkwp:System.Socket
          
          Use -p '/send and receive a datagram/' to rerun this test only.
    sendAll/receiveAll
      Inet/Stream/TCP
        sendAll and receiveAll a 128MB chunk:                    FAIL
          Exception: eAddressInUse
          
          HasCallStack backtrace:
            throwIO, called at src/System/Socket.hsc:269:58 in socket-0.8.3.0-4HRqFv0QZsQ2AY3gZSdkwp:System.Socket
          
          Use -p '/sendAll and receiveAll a 128MB chunk/' to rerun this test only.
        sendAllLazy and receiveAll a 128MB chunk:                FAIL
          Exception: eAddressInUse
          
          HasCallStack backtrace:
            throwIO, called at src/System/Socket.hsc:269:58 in socket-0.8.3.0-4HRqFv0QZsQ2AY3gZSdkwp:System.Socket
          
          Use -p '/sendAllLazy and receiveAll a 128MB chunk/' to rerun this test only.
        sendAllBuilder and receiveAll a 128MB chunk:             threaded: Uncaught exception tasty-1.5.4-Kfl7PU027dS8hQ3pGH08Ih:Test.Tasty.Runners.Utils.SignalException:

SignalException 15

cc @sternenseemann

[sternenseemann]

Can I view the snapshot update / version changes that led to this failure? I can reproduce it and this may be useful for purposes of debugging. The same version of socket works fine against Stackage LTS.

[Bodigrim]

This is caused by update from tasty-1.5.3 to tasty-1.5.4, which enabled mutithreaded execution of tests by default. The simplest fix is to remove ghc-options: -threaded from the test suite, if it actually does not support concurrent execution. Or one can wrap the entire test suite into localOption (NumThreads 1). Or perhaps use inOrderTestGroup to sequentialize only affected tests.

[juhp]

Can I view the snapshot update / version changes that led to this failure? I can reproduce it and this may be useful for purposes of debugging. The same version of socket works fine against Stackage LTS.

You know about the diff view in the web ui, right?

https://www.stackage.org/nightly-2026-03-27 > View Changes >
https://www.stackage.org/diff/nightly-2026-03-25/nightly-2026-03-27

That's the best we have at the moment.
(Sadly our tooling doesn't make our build results public - I wish we could make it so.)
Of course often these issues can get opened before the snapshot with the change is built of course...

[sternenseemann]

@Bodigrim Thanks! That was it. I proposed lpeterse/haskell-socket#73. The threaded test suite as currently published on Hackage won't work with tasty-1.5.4 because the change makes it bind to the same port in parallel multiple times.

You know about the diff view in the web ui, right?

Yeah, I just wasn't quite sure how to correlate the issues with the bumps, especially since they may be opened before a new nightly is published, right?

[sternenseemann]

We should maybe make a tracking issue for tasty-1.5.4 since there are more test suites affected. I've diagnosed criterion already (haskell/criterion#290) and I suspect pandoc is also affected (at least 3.7).

[juhp]

Well we could sure, but I think individual issues need to be filed against each affected upstream project really for them to get fixed. ((Personally I tend file testsuite issues found in Stackage directly in upstream issue trackers, though this has the disadvantage that other Stackage curators will likely miss any fixes - though we are anyway not systematically re-enabling testsuites alas, though I try when I can; in the end it is really up to the maintainer to re-enable but generally does not happen. We do now have logs over time inside the build server so we can in principle observe repeated "unexpected" passing testsuites, though even that is not 100% reliable: some failures are transient and we don't have a way to mark them as such, to be handled differently, alas.))

I suppose a tracking ticket would help to expose the number of affected packages 👍
Perhaps it really belongs upstream though.

[sternenseemann]

@Bodigrim btw, localOption does not seem to be an option for this. The documentation states:

Note that ingredient options (number of threads, hide successes etc.) set in this way will not have any effect. This is for modifying per-test options, such as timeout, number of generated tests etc.

This also matches up with my testing.

[sternenseemann]

@juhp I suppose it's more useful for those looking to patch around this downstream, but that's really just Nixpkgs and we'll be opening issues upstream anyways so…

[Bodigrim]

@sternenseemann this feature is undocumented indeed, but localOption for NumThreads should work if it's top-level, just under defaultMain.

[sternenseemann]

(I tried it for a single TestTree among others and at least explicitly passing -j64 did overwrite the NumThreads 1 for the single tree, so I guess it's somewhat limited.)