proxy: Figure out the plan for Hyper 1.0 migration

[olix0r]

Hyper is planning a major 1.0 milestone that will impact many of their public APIs and, therefore, the proxy. We should get a better understanding of the planned changes so that we can begin to scope and plan the required proxy changes (and so that we can provide meaningful feedback before the APIs are finalized).

---

"here's some links!" -kate 💐 🧢

issues and pull requests related to upgrading linkerd2 to hyper 1.0:

• refactor(proxy/http): create linkerd-http-version crate linkerd2-proxy#3379
• refactor(proxy/http): create linkerd-http-insert crate linkerd2-proxy#3380
• refactor(proxy/http): extricate Body middleware types linkerd2-proxy#3382
• chore(hyper): enable deprecated feature flag  linkerd2-proxy#3405
• chore(server/conn): backport max_pending_accept_reset_streams() hyperium/hyper#3796
• refactor(app): update deprecated hyper body calls linkerd2-proxy#3411
• refactor(app/test): remove unused test Server interfaces linkerd2-proxy#3421
• chore(proxy/http): address hyper::client::conn::Builder deprecations linkerd2-proxy#3427
• refactor(app/test): minor tweaks to linkerd-app-test linkerd2-proxy#3428
• chore(app/inbound): address server::conn::Http deprecations linkerd2-proxy#3432
• refactor(app/test): address hyper deprecations in test helpers linkerd2-proxy#3433
• chore(hyper): address miscellaneous deprecations linkerd2-proxy#3444
• chore(app/inbound): address hyper deprecations in http/2 tests linkerd2-proxy#3445
• chore(app/inbound): address hyper deprecations in http/1 tests linkerd2-proxy#3454
• chore(fuzz): address hyper deprecations in fuzz tests linkerd2-proxy#3455
• chore(hyper): define hyper as a workspace dependency linkerd2-proxy#3456
• chore(hyper): upgrade hyper to include hyperium/hyper#3796 linkerd2-proxy#3457
• chore(ci): remove duplicate connect_and_accept_http1(..) function linkerd2-proxy#3461
• chore(proxy/http): address hyper deprecations in ServeHttp<N> linkerd2-proxy#3459
• docs(app/test): fix unresolved SendRequest links linkerd2-proxy#3465
• chore(cargo.toml): define http workspace dependencies linkerd2-proxy#3466
• refactor: move away from legacy hyper::body::HttpBody linkerd2-proxy#3467
• feat(http/box): BoxBody::empty() creates an empty body linkerd2-proxy#3468
• chore(deps): update to http v1 versions linkerd2-proxy-api#421
• chore(deps): update from hyper 0.14.28 to 0.14.32 #13492
• chore(deps): address hyper deprecations in policy controller #13493
• fix(http2): gate server Builder keep-alive interfaces hyperium/hyper#3816
• chore(hyper): upgrade to hyper 0.14.32 linkerd2-proxy#3472
• chore(cargo.toml): define more workspace dependencies linkerd2-proxy#3473
• chore(http/upgrade): replace hyper::Body with BoxBody linkerd2-proxy#3479
• sync: add Receiver::poll_recv(..) method tokio-rs/tokio#7059
• refactor(http/retry): PeekTrailersBody<B> only peeks empty bodies linkerd2-proxy#3509
• util: add a channel body hyperium/http-body#140
• chore(http/retry): use boxed bodies instead of hyper::Body linkerd2-proxy#3515
• chore(http-body): update msrv to 1.61 hyperium/http-body#141
• refactor(http/upgrade): internal interfaces are private linkerd2-proxy#3530
• docs(http/upgrade): document linkerd-http-upgrade linkerd2-proxy#3531
• refactor(http/upgrade): Http11Upgrade is Clone linkerd2-proxy#3540
• feat(http/retry): add a unit test suite to PeekTrailersBody<B> linkerd2-proxy#3556
• fix(http/retry): is_end_stream() is true for empty bodies linkerd2-proxy#3558
• feat(http/retry): model PeekTrailersBody<B> with Frame<T> linkerd2-proxy#3559
• refactor(http/retry): port some ReplayBody tests to Frame<T> linkerd2-proxy#3564
• refactor(http/retry): port remaining ReplayBody tests to Frame<T> linkerd2-proxy#3567
• refactor(http/retry): update more Body::data() calls linkerd2-proxy#3573
• feat(http/retry): add more replay body test coverage linkerd2-proxy#3583
• docs: notes on poll_frame return values hyperium/http-body#144
• refactor(http/retry): replayed body is not optional linkerd2-proxy#3585
• refactor(http/retry): use Poll::{map_ok, map_err} linkerd2-proxy#3586
• refactor(http/retry): outline bounded replay buffer linkerd2-proxy#3597
• feat(http/retry): model ReplayBody<B> with Frame<T> linkerd2-proxy#3598
• refactor(mock/http-body): outline MockBody test body linkerd2-proxy#3611
• refactor(http/retry): outline ForwardCompatibleBody<B> linkerd2-proxy#3614
• refactor(app/core): a unit test suite for rescue body linkerd2-proxy#3615
• refactor(app/core): prepare rescue body for http-body upgrade linkerd2-proxy#3616
• chore(tls): remove redundant i/o bounds linkerd2-proxy#3647
• chore(app/trace-collector): remove Default bounds linkerd2-proxy#3651
• chore(proxy/identity-client): remove Default bounds linkerd2-proxy#3653
• chore(app/inbound): remove Default bounds linkerd2-proxy#3654
• chore(app/outbound): remove Default bounds linkerd2-proxy#3655
• chore(proxy/api-resolve): remove Default bounds linkerd2-proxy#3656
• chore(service-profiles): remove Default bounds linkerd2-proxy#3657
• chore(app/trace-collector): remove Default bound linkerd2-proxy#3660
• refactor(app/inbound): forward-compatible test code linkerd2-proxy#3671
• refactor(app/outbound): forward-compatible test code linkerd2-proxy#3672
• refactor(app/integration): forward-compatible test code linkerd2-proxy#3673
• refactor(app/outbound): forward-compatible test code linkerd2-proxy#3676
• feat(util): introduce Sender::{capacity, max_capacity} hyperium/http-body#147
• refactor(app/integration): remove Request, Response aliases linkerd2-proxy#3692
• refactor(app/integration): forward-compatible test code linkerd2-proxy#3699
• refactor(app/integration): remove artificial Sync bounds linkerd2-proxy#3700
• chore(deps): prost-build is a workspace dependency linkerd2-proxy#3701
• chore(deps): remove unused opencensus, opentelemetry dependencies linkerd2-proxy#3708
• chore(deps): update tower dependency to v0.5.2 drain-rs#36
• chore(deps): tower crates are workspace dependencies linkerd2-proxy#3715
• chore(deps): drain is a workspace dependency linkerd2-proxy#3717
• chore(deps): update rustix to 1.0.1 eminence/procfs#340
• http-body-util: v0.1.3 hyperium/http-body#149
• refactor(stack/switch_ready): use Either<A, B> future linkerd2-proxy#3739
• chore: replace tower with tower-layer and tower-service drain-rs#41
• refactor(http/upgrade): remove HttpConnect extension linkerd2-proxy#3779

in particular, this PR:

• chore(deps)!: upgrade to hyper 1.x linkerd2-proxy#3504
• chore(deps): upgrade to tower 0.5 linkerd2-proxy#3744

[hawkw]

Some of the things that are definitely worth keeping an eye on:

• The future of hyper's use of the Service trait: whether hyper will use tower::Service in 1.0 is currently up in the air. If tower-service isn't 1.0 in time, it definitely won't. The question then becomes whether hyper will provide its own Service trait, or whether the interface will be changed significantly, potentially to one where user code calls into hyper.

  If hyper ends up moving away from tower-service, there will definitely need to be some kind of glue layer (which could be a generic tower-hyper crate or similar); the complexity of this adapter will depend on how different the interfaces end up being.

• I/O traits: It's unclear whether hyper 1.0 is going to use Tokio's AsyncRead/AsyncWrite traits (as it does currently), or something else.

  • It seems like the use of the Tokio traits is more likely than the version from futures, but not set in stone, especially if there's an eventual plan for AsyncRead/AsyncWrite in std.
  • There are also questions about whether the style of async I/O traits currently in tokio and in futures-io can ever efficiently support io_uring-style async IO, where ownership of the buffer is transferred to the I/O resource (and then to the OS). This is a broader ecosystem-level question, but it's worth keeping an eye on.

• A bunch of stuff currently in hyper will move to hyper-util. This one is not a huge deal as that should generally be a pretty mechanical transformation.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[risingspiral]

https://docs.rs/hyper/1.0.0-rc.4/hyper/

[cratelyn]

i've been driving this work forward recently, and was kindly pointed at this issue by @olix0r. i thought i'd lay down some thoughts on the work done thus far:

first, i drove a few small exploratory spikes into upgrading the proxy here, here, and here.

most of the friction related to this upgrade will be found in the linkerd-proxy-http crate, which provides a number of core facilities wrapping hyper's client and server interfaces used throughout the rest of the proxy.

i've landed a few preparatory changes, in linkerd/linkerd2-proxy#3379, linkerd/linkerd2-proxy#3380, and linkerd/linkerd2-proxy#3382. these pulled assorted standalone bits of http/hyper infrastructure that define http_body::Body middleware out of the linkerd-proxy-http library, and into their own distinct crates.

• refactor(proxy/http): create linkerd-http-version crate linkerd2-proxy#3379
• refactor(proxy/http): create linkerd-http-insert crate linkerd2-proxy#3380
• refactor(proxy/http): extricate Body middleware types linkerd2-proxy#3382

this will let us bump linkerd-http-classify, linkerd-http-stream-timeouts, linkerd-http-retain, linkerd-http-override-authority, [..] and other linkerd-http-* crates up to hyper 1.0 in separate steps, which should both facilitate review and mitigate the risk of bugs slipping in while addressing the changes to the Body trait in particular.

🥚 🔜 🐣 next steps

next, we can use the deprecated and backports feature flags provided in the 0.14 minor release to further prepare for the upgrade.

linkerd/linkerd2-proxy#3405 adds the deprecated flag to the hyper dependencies in the cargo workspace, and #[allow(deprecated)] attributes to code using deprecated hyper interfaces.

next, we'll address those deprecations, making use of the backports feature. i've landed hyperium/hyper#3796 upstream to backport a 1.0 interface that we'll need before we can use the connection builder backported to 0.14.