iOS HLS Cache｜Optimize AVPlayer Streaming with Real-Time Caching Techniques

点击这里查看本文章简体中文版本。

點擊這裡查看本文章正體中文版本。

This post was translated with AI assistance — let me know if anything sounds off!

---

Exploring Practical Methods for iOS HLS Cache Implementation

How to Enable Simultaneous Playback and Caching When Using AVPlayer to Play m3u8 Streaming Media Files

photo by Mihis Alex

[2023/03/12] Update

• Next article: “Complete Guide to Implementing Local Cache with AVPlayer” teaches you how to implement AVPlayer caching.

I have open-sourced my previous implementation. Feel free to use it if you need.

• Custom cache strategies can use PINCache or others…

• Externally, just call the AVAsset factory with the URL, and the AVAsset will support caching.

• Using Combine to implement data flow strategy

• Wrote some tests

About

HTTP Live Streaming (HLS) is an Apple-proposed HTTP-based streaming media network transmission protocol.

For playing music, in non-streaming cases, we use mp3 files as music files. The playback can only start after the entire file is downloaded, which takes time depending on the file size. HLS, on the other hand, splits a file into multiple small segments and plays as it reads. So, playback can start as soon as the first segment is received, without downloading the whole file!

.m3u8 files record the bitrate, playback order, duration of the segmented .ts files, and overall audio information. They can also support encryption, decryption, low-latency streaming, and more.

.m3u8 file example (aviciiwakemeup.m3u8):

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:9.900411,
aviciiwakemeup–00001.ts
#EXTINF:9.900400,
aviciiwakemeup–00002.ts
#EXTINF:9.900411,
aviciiwakemeup–00003.ts
#EXTINF:9.900411,
.
.
.
#EXTINF:6.269389,
aviciiwakemeup-00028.ts
#EXT-X-ENDLIST

EXT-X-ALLOW-CACHE has been deprecated in iOS≥ 8/Protocol Ver.7, so including it has no effect.

Goal

For a streaming service, Cache is extremely important; each audio file can range from a few MB to several GB. If the file has to be fetched from the server every time it is replayed, it puts a heavy load on the server and consumes a lot of bandwidth, which is costly \(\). Having a cache layer can save the service a lot of money and also prevent users from wasting network data and time on re-downloading. It’s a win-win mechanism (but remember to set limits and clear the cache periodically to avoid filling up the user’s device).

Question

In the past, before streaming, there was not much to handle with mp3/mp4 files. The files were downloaded to the device before playback, and playback would start only after the download was complete. Since the entire file had to be downloaded anyway, we could simply use URLSession to download the file first, then feed the local file path with a file:// URL to AVPlayer for playback. Alternatively, the proper way is to use AVAssetResourceLoaderDelegate to cache the downloaded data within the delegate methods.

The idea behind streaming is actually straightforward: first read the .m3u8 file, then parse the information inside, and cache each .ts file; but in practice, it turned out to be more complicated than I expected, which is why I wrote this article!

For playback, we use iOS AVFoundation’s AVPlayer directly. There is no difference in handling streaming or non-streaming files.

Example:

let url:URL = URL(string:"https://zhgchg.li/aviciiwakemeup.m3u8")
var player: AVPlayer = AVPlayer(url: url)
player.play() // Play the media

2021–01–05 Update:

As a fallback, we switch back to using mp3 files, which allows direct implementation with AVAssetResourceLoaderDelegate. For detailed implementation, refer to “AVPlayer Streaming Cache Practical Guide”.

Implementation Plan

Regarding several solutions to achieve our goals and the challenges encountered during implementation.

Option 1. AVAssetResourceLoaderDelegate ❌

The first idea is to follow the same approach as with mp3/mp4! Use AVAssetResourceLoaderDelegate to cache .ts files within the delegate methods.

However, sorry, this approach won’t work because the download request information for .ts files cannot be intercepted in the Delegate. You can refer to this Q&A and the official documentation for details on this issue.

The implementation of AVAssetResourceLoaderDelegate can refer to “AVPlayer Streaming and Caching in Practice”.

Option 2.1 URLProtocol Intercept Requests ❌

URLProtocol is a method I recently learned. All requests based on the URL Loading System (URLSession, API calls, image downloads, etc.) can be intercepted by us to modify the Request and Response, then returned as if nothing happened—secretly. For more on URLProtocol, refer to this article.

Using this method, we plan to intercept AVFoundation AVPlayer requests for .m3u8 and .ts files. If there is a local cache, we return the cached data directly; if not, we send the actual request. This approach helps us achieve our goal.

Similarly, sorry, this path also doesn’t work; because AVFoundation AVPlayer requests are not made through the URL Loading System, we cannot intercept them.
There is a saying that it works on the simulator but not on a real device

Plan 2.2 Forcibly Allow URLProtocol ❌

According to Plan 2.1, the wildly creative brute force method, if I change the request URL to a custom scheme (e.g., streetVoiceCache://), since AVFoundation cannot handle this request, it will throw it out. This way, our URLProtocol can intercept it and do what we want.

let url:URL = URL(string:"streetVoiceCache://zhgchg.li/aviciiwakemeup.m3u8?originSchme=https")
var player: AVPlayer = AVPlayer(url: url)
player.play()

URLProtocol intercepts streetVoiceCache://zhgchg.li/aviciiwakemeup.m3u8?originSchme=https. At this point, we just need to restore it to the original URL, then use URLSession to request the data, allowing us to handle caching here. Requests for .ts files in the m3u8 will also be intercepted by URLProtocol, enabling us to cache them as well.

Everything seemed perfect, but when I excitedly Build-Run the app, Apple slapped me hard:

Error: 12881 “CoreMediaErrorDomain custom url not redirect”

He does not accept the .ts file Request Response Data I provide. I can only use the urlProtocol:wasRedirectedTo method to redirect to the original HTTPS request to play properly. Even if I download the .ts file locally and redirect to that file:// file, it is not accepted. Checking the official forum reveals the answer: this cannot be done. The .m3u8 must come from HTTP/HTTPS (so even if you put the entire .m3u8 and all segmented .ts files locally, AVPlayer cannot play them using file://). Also, .ts files cannot use URLProtocol to provide Data by yourself.

fxxk…

Plan 2.2–2 – Same as Plan 2.2 but implemented with Plan 1 AVAssetResourceLoaderDelegate ❌

The implementation follows method 2.2, feeding AVPlayer a custom scheme to trigger AVAssetResourceLoaderDelegate; then we handle it ourselves.

Same as 2.2 results:

Error: 12881 “CoreMediaErrorDomain custom url not redirect”

Official Forum same answer.

Can be used for decryption processing (refer to this article or this example), but caching functionality still cannot be achieved.

Option 3. Reverse Proxy Server ⍻ (Feasible but Imperfect)

This method is the most common answer when looking for how to handle HLS Cache; it involves running an HTTP Server on the app to serve as a Reverse Proxy Server.

The principle is simple: the app runs an HTTP server, for example on port 8080, so the URL will be http://127.0.0.1:8080/; then we can handle incoming requests and provide responses.

Applied to our case, replace the request URL with:
http://127.0.0.1:8080/aviciiwakemeup.m3u8?origin=http://zhgchg.li/

In the HTTP Server’s Handler, intercept requests for *.m3u8. When a request comes in, it will enter our Handler, allowing us to do whatever we want. We have full control over the response data. .ts files will also come through here; this is where we can implement our desired caching mechanism.

For AVPlayer, it is a standard http://.m3u8 streaming audio file, so there will be no issues.

A complete implementation example can be found at:

Because I also referred to this example, I used GCDWebServer for the Local HTTP Server part. Additionally, there is an updated Telegraph available. (I do not recommend CocoaHttpServer as it has not been updated for a long time.)

Looks good! But there’s a problem:

Our service is music streaming, not a video playback platform. In music streaming, users often switch tracks while running the app in the background; would the Local HTTP Server still be running then??

The description of GCDWebServer states that it will automatically disconnect when entering the background and automatically resume when returning to the foreground. However, you can disable this behavior by setting the parameter GCDWebServerOption_AutomaticallySuspendInBackground:false.

However, in actual tests, if no requests are sent for a while, the server still disconnects (and the status is incorrect, still showing as isRunning). It feels like the system is killing it. After digging into the HTTP Server approach, I found that the underlying implementation is based on sockets. After checking the official documentation on socket services, this issue cannot be resolved. When there are no new connections in the background, the system will suspend it.

*There are complicated methods found online… such as sending a long request or continuously sending empty requests to ensure the server is not paused by the system in the background.

The above points all refer to the app running in the background. When in the foreground, the server is stable and won’t be paused due to inactivity, so this issue does not occur!

Since it relies on other services, even if testing in the development environment works fine, it is recommended to implement a rollback handling (AVPlayer.AVPlayerItemFailedToPlayToEndTimeErrorKey notification) in actual use; otherwise, if the service fails, the user will be stuck.

So it's not perfect…

Option 4. Using the HTTP Client’s own caching mechanism ❌

Our .m3u8/.ts files’ Response Headers all include Cache-Control, Age, eTag, and other HTTP Client Cache information. Our website’s cache mechanism works perfectly on Chrome. Additionally, the official new preliminary specification for the Protocol Extension for Low-Latency HLS also mentions that cache-control headers can be set for caching.

However, in reality, AVFoundation AVPlayer does not have any HTTP Client Caching effect, so this approach is also ineffective! It’s just wishful thinking.

Solution 5. Playing Audio Files Without Using AVFoundation AVPlayer ✔

Implement audio file parsing, caching, encoding, and playback functions by yourself.

Too hardcore, requires deep technical skills and a lot of time; no research done.

Here is an open-source online player for reference: FreeStreamer. If you really want to choose this solution, it’s better to stand on the shoulders of giants and use a third-party library directly.

Plan 5–1. Without Using HLS

Same as Plan 5, too hardcore, requires deep technical skills and a lot of time; no research done.

Solution 6. Convert .ts Split Files to .mp3/.mp4 Files ✔

No research done, but it is indeed feasible; however, it feels complicated. You need to handle the downloaded .ts files, convert each one into .mp3 or .mp4 files, then play them in order, or compress them into a single file, which sounds difficult to do.

If interested, you can refer to this article.

Option 7. Download the complete file before playing ⍻

This method cannot be accurately called streaming with caching; it actually downloads the entire audio file before starting playback. For .m3u8 files, as mentioned in solution 2.2, you cannot directly download and play them locally.

To implement this, you need to use the iOS ≥ 10 API AVAssetDownloadTask.makeAssetDownloadTask. It actually packages the .m3u8 into a .movpkg stored locally for user playback.

This is more like offline playback rather than a caching feature.

Users can also view and manage downloaded audio files from “Settings” -> “General” -> “iPhone Storage” -> APP.

下方 已下載的影片 部分

Below is the Downloaded Videos section

For detailed implementation, refer to this example:

Conclusion

The above exploration took nearly a whole week, going around in circles and almost driving me crazy; currently, there is no reliable and easy-to-deploy method.

If there are new ideas, I will update again!

References

• iOS Audio Playback (Part 9): Play While Caching

• StyleShare/HLSCachingReverseProxyServer

If you have any questions or feedback, feel free to contact me.

---

This post was originally published on Medium (View original post), and automatically converted and synced by ZMediumToMarkdown.

Improve this page on Github.

Follow Me on Medium 1K+ Followers