Improving the Player on Android
Grey Skold | (former Android Video Engineer) ; Lin Wang | Android Performance Engineer; Sheng Liu | Android Performance Engineer
Pinterest Android App offers a rare experience with a mix of images and videos on a two-column grid. In order to maintain a performant video experience on Android devices, we focused on:
- Warming up
- Configurations
- Pooling players
Warming Up
In order to reduce the startup latency, we establish a video network connection by sending a dummy HTTP HEAD request during the early application startup time. The same connection can be used to play future videos. This is done even before any video urls are returned from our server.
The same strategy also applies to UI rendering. We found Exoplayer tends to do lots of work after parsing the information from the video url. It:
- Calculates the aspect ratio
- Invokes on onContentAspectRatioChanged()
- Informs the AspectRatioFrameLayout of the aspect ratio
Since most of our videos’ aspect ratios are pre-determined, we can prevent the above work by:
- Setting the video aspect ratio via AspectRatioFrameLayout.setAspectRatio()
- Overriding the PlayerView.onContentAspectRatioChanged() method with an empty body
The latter prevents the player from trying to recalculate the aspect ratio.
Configurations
DefaultLoadControl
ExoPlayer provides us the setBufferDurationsMs() to customize various buffering durations used to delay playback until we have sufficient data. Since a majority of Pinterest’s media content is in short-form, we can use much shorter buffering durations which will result in less time waiting for data to load.
By doing this, we saw a significant reduction in video startup latency. Although the rebuffer rate increases a bit, the overall video UX is still improved.
DefaultTrackSelector
Using the following two parameters, we could effectively ensure videos loaded in-feed are limited by their viewport size (i.e. to avoid loading a large 1080p video in a small 360 pixel viewport).
The Pinterest app plays multiple muted videos in the grid at the same time. Using the following parameters, we could disable the audio rendering to save network bandwidth to download the audio and memory consumption to process them.
SimpleCache
Exoplayer provides a cache interface to keep downloaded media data on disk. However, in the cases when we run into fatal errors caused by backend bugs, the bad contents can also stick in the cache. This can cause the application to continue encountering the same playback errors even though the backend is fixed.
We use SimpleCache.removeResource() to purge the dirty cache when the following fatal IO errors are returned from the Player.Listener.onPlayerError():
- ERROR_CODE_IO_UNSPECIFIED 2000
- ERROR_CODE_IO_INVALID_HTTP_CONTENT_TYPE 2003
- ERROR_CODE_IO_BAD_HTTP_STATUS 2004
- ERROR_CODE_IO_FILE_NOT_FOUND 2005
- ERROR_CODE_IO_READ_POSITION_OUT_OF_RANGE 2008
Pooling Players
Lastly, we built our own cache to pool players as they’re needed. Historically, we instantiated new player instances on the fly, which caused significant overhead on both the memory and bandwidth. Here are some high level learnings:
Separate By Encoding
Player instances are effectively holding onto a reference to an underlying decoder based on the last rendered media’s encoding type. It takes a measurable amount of work for players to switch context between different decoders. Therefore we pooled our players based on the initial decoding format. By ensuring the recycled players with decoders matching the media’s encoding, we removed any latency overhead caused when switching encoding formats.
Smart Sizing
The pool size itself ran through multiple iterations to find the ideal space needed to house multiple video plays while avoiding OutOfMemory (OOM) and ANR. The following two APIs are utilized to not over-burdening the system:
This was our key callback used to inform us that we should clear our player cache pool as we start to get dangerously close to sending OOMs.
Setting this flag will let the Exoplayer to retain a direct reference to the video decoder and keep it in memory even in the idle state. However, it takes a significant toll on memory and device stability. We had to build logic to use this method conservatively, based on both the current application lifecycle and current available memory on the device.
Playtime is Never Over
Improving the performance of video playback is a never-ending investigation into the inner-workings of the ExoPlayer library as well as our own product’s distinct use-cases, but what’s the most challenging is building an architecture that works for the uniquely Pinterest experience of the home feed. Our long-term goal is to share this work to other developers looking to build a seamless video UX with minimal setup required. In the meantime, we hope everyone builds a more performant video experience.
To learn more about engineering at Pinterest, check out the rest of our Engineering Blog and visit our Pinterest Labs site. To explore life at Pinterest, visit our Careers page.
