Pinterest is now on HTTP/3
Liang Ma | Software Engineer, Core Eng; Scott Beardsley | Engineering Manager, Traffic; Haowei Yuan | Software Engineer, Traffic
Now Pinterest operates on HTTP/3. We have enabled HTTP/3 for major Pinterest production domains on our multi-CDN edge network, and we’ve upgraded client apps’ network stack to support the new protocol. This allows us to catch up with industry trends. Most importantly, faster and more reliable networking improves Pinners’ experience and business metrics.
Context
Network performance (such as latency and throughput) is critical to Pinners’ experience.
In 2021, a group of client networking enthusiasts at Pinterest started thinking about adopting HTTP/3 (fka QUIC) for Pinterest, from traffic/CDN to client apps. We worked on it throughout 2022, and we’ve achieved our initial goal (with continued work ongoing in 2023 and beyond).
Terms:
- HTTP/3: the next generation of the HTTP protocol. It has been stabilized and now finalized by the IETF working group.
- QUIC: created by Chromium/Google for HTTP over UDP; later submitted to IEFT for standardization (HTTP/3).
How Does HTTP/3 Help
HTTP/3 is a modern HTTP protocol that has many advantages compared to HTTP/2, including but not limited to:
- No TCP head of line blocking problem in comparison to HTTP/2
- Connection migration across IP addresses, which is good for mobile use cases
- Able to change/tune Lost detection and Congestion control
- Reduced connection time (0-RTT, while HTTP/2 still requires TCP 3-way handshakes)
- More efficient for large payload use-cases, such as images downloading, video streaming, etc.
These advancements fit well with Pinterest use cases — enabling faster connection establishment (time to first byte of first request), improved congestion control (large media as we have), multiplexing without TCP head-of-line blocking (multiple downloads at the same time), and continued in-flight requests when pinners’ device network/ip changes. So Pinners will have a much faster and more reliable experience when they’re working their inspiration on the Pinterest platform.
Adopting HTTP/3 at Pinterest
Strategy
Safety and metrics came first. Though Pinterest is focused on executing with velocity, it was critical that we took a thoughtful approach to adopting HTTP/3. First, we upgraded the client network stacks and created an end to end A/B test for each traffic type (e.g., image, video). Then, we ran extensive experimentations before enabling HTTP/3 on both CDNs and clients.
Challenges:
The course of turning on HTTP/3 on our CDNs and client apps is not simple for a few main reasons:
- For the web app, some browsers already have HTTP/3 or QUIC support. Though these browsers utilize HTTP/3, there may be compatibility issues, which could break the Pinterest web app.
- New iOS versions (starting with iOS 15) have early QUIC support, and we don’t have a way to control that by code unless we disable QUIC on the server side.
- Our CDN vendors are at different phases of HTTP/3 support. As we gradually enable HTTP/3 by CDN, our multi-CDN edge network only partially supports HTTP/3 for a fairly long period of time, making it an interesting problem to ensure reliability & performance when traffic switches CDNs (link).
Solutions:
- First, we created a A/B domain-level (CDN) test where we cloned one domain to enable HTTP/3 and validated clients (including Web) thoroughly. For example: in the images HTTP/3 validation plan, we used i2.pinimg.com to validate image traffic on HTTP/3.
- For the multi-CDN issue, we opted for a relatively short Alt-Svc TTL to be close to DNS record TTL, and we tried to configure the same protocol set on those CDNs.
- Extensive tests were performed to validate cross-CDN HTTP/3 behaviors using the most commonly used browsers.
- After A/B testing passed, we enabled HTTP/3 on production (CDN by CDN), then had client apps use a feature flag to control HTTP/3 with capable network clients, which was safer and allowed for metrics collection and comparison.
Current State
We have enabled HTTP/3 on critical traffic types and upgraded/leveraged mobile clients’ network stacks to utilize HTTP/3.
Traffic: HTTP/3 is enabled for major Pinterest production domains on our multi-CDN edge network
Clients:
- Web will get it for free on eligible browser and traffic
- iOS — Images/API traffic are served by Cronet + HTTP/3. Today, 70% of iOS image traffic is over HTTP/3.
- iOS’s native networking stack can utilize HTTP/3 after we enabled it on the traffic side. Apple’s HTTP/3 adoption rate continues to improve year over year. We’re seeing the benefits of this in our Video metrics, which has significantly improved with HTTP/3 (via AVPlayer).
- Android Video has utilized HTTP3 through Exoplayer+Cronet.
Showcase
Our analysis indicates that HTTP/3 (and Cronet) has improved core network metrics (round-trip latency and reliability). Improved latency/throughput are critical to large media features (such as video, images). A faster and more reliable network is also able to move user engagement metrics.
Video metrics
Video closeup GVV (iOS: Apple networking + HTTP/3):
Video closeup GVV (Android: Exoplayer + Cronet + HTTP/3):
Engagement metrics(iOS): HTTP/3’s direct impact
Network metrics
- Round-trip latency (ms):
Note: 1) measured from the client side, from request sent to response received; 2) based on one-week of network logs that were collected in Q3, 2022 with Apple networking (HTTP/2), and Q1, 2023 with Cronet (when HTTP/3 is enabled).
- Reliability improved as well.
Future Work
We will continue to invest in HTTP/3 for sustained impact, including:
- Increasing HTTP/3 coverage; explore other network stacks on Android.
- Furthering improvement of HTTP/3 adoption rate; set to higher max-age values.
- Experimenting with various congestion control algorithms.
- Exploring 0-RTT connection establishments.
To learn more about engineering at Pinterest, check out the rest of our Engineering Blog and visit our Pinterest Labs site. To explore life at Pinterest, visit our Careers page.
