App Lifecycle Management｜Mitmproxy & Apple Configurator for Preserving Removed Apps
Developers and users facing app discontinuation can use mitmproxy combined with Apple Configurator to keep apps permanently accessible in their last available state, preventing loss of functionality after removal.
点击这里查看本文章简体中文版本。
點擊這裡查看本文章正體中文版本。
This post was translated with AI assistance — let me know if anything sounds off!
What can be done to commemorate when an app product reaches its end?
Using mitmproxy + Apple Configurator to Keep an App Permanently in the Pre-Removal State
Preface
Jujutsu Kaisen
Having worked for a long time and handled many products, I have started to encounter products I once participated in reaching their end (discontinuation). Developing a product from scratch is like nurturing a new life; the team worked hard for 3–4 months to bring it to birth. Although later stages were handed over to other caretakers (engineers) for further development, hearing that it is about to reach the end of its product life cycle still brings some regret.
Life is the same; we never know whether the sun will rise first or an accident will happen tomorrow. The only thing we can do is cherish the present and do things well.
Remembrance
Wherever we go, traces are left behind. We hope to do something before the product reaches its end, giving everyone a chance to reminisce and at least leave proof that it once existed. The following methods all require the app to still be online; if it has been removed, only memories remain.
Non-technical Method — Recording
Besides using the iPhone’s built-in screen recording feature, we can also connect the phone to a Mac and use QuickTime Player to record and export videos on the computer.
- Open QuickTime Player App on Mac
- Select “File” -> “New Movie Recording” from the top left toolbar.
- After exiting the recording interface, click the “v” next to the 🔴, and select your connected phone for the screen and speaker.
- The recording interface will then display the phone screen.
Click “🔴” to start recording, then return to your phone to perform the actions you want to record.
During recording, the current video size will be displayed. Press the “🔴” button again to stop recording.
You can easily trim videos using the QuickTime Player toolbar. Finally, press “Command” + “s” to export and save the video to a specified location, completing the recording.
The advantage of recording videos is that future memories are easier to connect than with pictures. The longer you record, the more detailed the documentation. If you want to turn specific scenes into pictures, you can simply take screenshots, which is very convenient.
Technical Approach
App technical backup can be divided into two directions: the “bone,” where the app itself is just a framework, and the “flesh,” which is composed of API response data and forms the core content of the app.
The app will disappear once it is removed from the App Store.
Meat will disappear when the API host or server is shut down.
Therefore, we also divided the backup into two technical methods: skeleton and flesh.
Disclaimer
This article is for technical research sharing only and does not encourage the use of any technology for illegal or infringing activities.
[Core] Backup .ipa App Installation Files
Once an app is removed from the store, it will remain on any device that has already downloaded it, as long as the user does not delete it. If you switch phones using a transfer method, the app will also be transferred.
But if we accidentally delete the app or change phones without transferring it, then it’s truly goodbye forever; at this point, having a manual backup of the store’s .ipa file can bring it back to life.
A long time ago, the reverse engineering article mentioned this, but this time we only need to back up the .ipa file without cracking; everything is done using Apple’s official tools.
1. Install Apple Configurator 2
First, go to the Mac App Store to download and install Apple Configurator 2
2. Connect your iPhone to your Mac and tap Trust This Computer
After a successful connection, the iPhone home screen will appear.
3. Confirm that your phone has the app installed for the .ipa file you want to back up
We need to create a replacement screen in Apple Configurator 2 to obtain the .ipa file downloaded to the cache. Therefore, we must first ensure the target app is installed on the phone.
4. Back to Apple Configurator 2 on Mac
Double-tap the iPhone home screen displayed above to enter the information page.
Switch to “App” -> Top right corner “+ Add” -> “App”
After logging into your App Store account, you can access the list of apps you have previously purchased.
Search for the target app you want to back up, then select it and click “Add”.
A waiting window will appear, showing “Adding App on XXX” and “Downloading ‘XXX’.”
5. Extract .ipa File
Wait for the download to complete, then a prompt will appear asking if you want to replace the existing installed app.
Do not press any button at this time. Do not press any button at this time. Do not press any button at this time.
Let’s open a Finder:
In the top-left toolbar, select “Go” -> “Go to Folder”
Paste the following path:
1
~/Library/Group Containers/K36BKF7T3D.group.com.apple.configurator/Library/Caches/Assets/TemporaryItems/MobileApps
You can find the target App .ipa file that has been downloaded and is ready to be installed:
Simply copying it completes the App .ipa file backup.
After completing the file copy, return to Apple Configurator 2 and click Stop to terminate the operation.
[Core] Restore .ipa App Installation File
Connect the iPhone with the app to be restored to the Mac and open Apple Configurator 2, then go to the Add Apps interface.
To restore, select “Choose from my Mac…” at the bottom left corner.
Select the backup App .ipa file and click “Add”.
Wait for the transfer and installation to complete, then return to your phone to reopen the app and successfully revive it!
[Meat] Backup the Final API Response Data
Here, we will use the method and the open-source project created previously in the article App End-to-End Testing Local Snapshot API Mock Server (refer to it for detailed principles):
Similar to using recorded API Request & Response for running E2E Testing, we can also use it to capture the final API Request & Response data before an app is taken down or shut off.
1. Install mitmproxy
1
brew install mitmproxy
mitmproxy is an open-source man-in-the-middle attack and network request sniffing tool.
If you are not familiar with how man-in-the-middle attacks work in Mitmproxy, you can first refer to my previous article: “APP uses HTTPS transmission, but data was still stolen.” or the Mitmproxy official documentation.
If you are using it for pure network request sniffing and are not comfortable with the mitmproxy interface, you can switch to Proxyman. Refer to the usage in the previous article.
2. Complete mitmproxy Certificate Setup
For HTTPS encrypted connections, root certificate replacement is required to perform a man-in-the-middle attack. Therefore, the first use requires downloading and enabling the root certificate on the mobile device.
If your App & API Server implement SSL Pinning, you also need to add the pinning certificate to mitmproxy.
First, make sure your iPhone and Mac are connected to the same network.
If there is no WiFi and your computer is on a wired network, you can also enable Mac’s WiFi sharing feature to let your phone connect to the Mac’s network.
Start mitmproxy or mitmweb (Web GUI version) in the Terminal.
1
mitmproxy
Seeing this screen means the mitmproxy service has started. There is no traffic currently, so it is empty. Stay on this screen and do not close the Terminal.
- Check the Mac’s IP Address in Mac Network Settings
Go back to the phone’s WiFi settings and tap the “i” to enter detailed settings. Find the “Configure Proxy” option at the bottom:
Enter the Mac computer’s IP address on the server
Enter port 8080
Save
Open Safari on your phone and enter: http://mitm.it/
If it appears:
1
// If you can see this, traffic is not passing through mitmproxy.
It means the mobile device’s network proxy server is not set up correctly or mitmproxy is not running on the Mac.
Normally, the following will appear:
At this point, only HTTP traffic can be sniffed; HTTPS traffic will cause errors. We will continue with the configuration.
Indicates a successful connection. We found the iOS section and clicked “Get mimproxy-ca-cert.pem.”
- Click “Allow”
After the download is complete, go to your phone’s settings and you will see “Profile Downloaded.” Tap to enter.
- Click to enter, tap “Install” at the top right, and enter your phone password to complete the installation.
Go to Settings -> “General” -> “About” -> scroll to the bottom to “Certificate Trust Settings” -> enable “mitmproxy”.
- “Continue” to complete activation.
At this point, we have completed all the preparatory work for the man-in-the-middle attack.
Remember that all your phone’s current data traffic will go through the proxy on your Mac. After finishing, be sure to turn off the proxy server setting in your phone’s network settings, otherwise your phone’s WiFi will not connect to the internet.
Back in the Terminal mitmproxy, you can see all captured API request logs while operating the app on your phone.
Each request allows you to view detailed Request & Response content:
The above covers the basic setup and practical use of mitmproxy.
3. Sniffing and Understanding API Architecture
Next, use mitmproxy’s mitmdump service combined with the mitmproxy-rodo addons I developed earlier to record and replay requests.
My implementation principle is to convert the request parameters into a hash value. During replay, the request is hashed again, and if a matching hash value backup response is found locally, it is returned. If multiple requests share the same hash value, they are stored and replayed in sequence.
We can first sniff the app’s API using the above methods (or use Proxyman) to observe which fields might affect Hash Mapping. Record these fields to exclude them in later settings. For example, some APIs always include the ?ts parameter, which does not affect the returned content but impacts the Hash calculation, causing local backups to be unrecognized. We need to identify and exclude such parameters in the subsequent settings.
4. Set up mitmproxy-rodo :
Use the open-source script I wrote for recording and playback.
For detailed parameter settings, please refer to the documentation of the open-source project.
1
2
git clone git@github.com:ZhgChgLi/mitmproxy-rodo.git
cd mitmproxy-rodo
Fill the parameters selected in step 3 into the config.json configuration file:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
{
"ignored": {
"*": {
"*": {
"*": {
"iterable": false,
"enables": [
"query",
"formData"
],
"rules": {
"query": {
"parameters": [
"ts",
"connect_id",
"device_id",
"device_name"
]
},
"formData": {
"parameters": [
"aidck",
"device_id",
"ver_name"
]
}
}
}
}
}
}
}
The above parameters are excluded when calculating the hash value, and specific exclusion rules can be set for individual endpoint paths.
5. Enable recording, then run the following in Terminal:
1
mitmdump -s rodo.py --set dumper_folder=zhgchgli --set config_file=config.json --set record=true "~d zhgchg.li"
The ending
"~d zhgchg.li"means to exclude traffic only from *.zhgchg.li.dumper_folder: Output destination directory name
6. Return to the phone to operate the target app and perform the process path you want to record
It is recommended to restart or reinstall the app to begin with the cleanest setup.
It is recommended to record a video to help remember and reproduce the steps.
While operating, you can see the output directory contains many captured API Response Data, stored according to Domain -> API Path -> HTTP Method -> Hash Value -> Header-X / Content-X (if the same Hash is requested twice, they will be saved sequentially).
Re-recording allows you to directly delete the output directory and let it re-extract.
If the returned data contains personal information, please remember to adjust the extracted content to anonymize it.
[Meat] Playback Captured API Response Data
After recording, be sure to replay once to test if the data is correct. If the Hash Hit is very low (almost no matching Response found during replay), you can repeat the sniffing steps to identify the variable that changes each time the App runs and affects the Hash value, then exclude it.
Execution Replay:
1
mitmdump -s rodo.py --set dumper_folder=zhgchgli --set config_file=config.json
dumper_folder: Output destination directory nameBy default, if there is no Hash Mapping for the Response Data locally, a 404 will be returned directly, causing the app to display a blank screen. This way, you can know whether the fetched data is valid.
Pages visited during recording are correctly displayed again during playback: OK!
Recording captures pages not visited during the process, playback shows network error: OK!
Remembrance
At this point, we can recreate the final moments before the app reached its end by restoring the framework and the last bits of content, cherishing the time when everyone worked together in unity to produce it.
This article commemorates the team from my first job and my transition from web backend development to iOS app development through learning on the job. In just 3–4 months, I independently built the product from scratch and successfully launched it together with Android developers, designers, PM supervisors, and backend colleagues. Although the product is now nearing the end of its lifecycle, I will always remember the bittersweet experiences and the excitement of seeing it go live and being used for the first time.
“Thank you”
Welcome to Contribute
If you share the same regret, I hope this article can help you. The tool mitmproxy-rodo was originally developed as a POC (proof of concept). Contributions, bug reports, or PRs to fix bugs are welcome.
If you have any questions or feedback, feel free to contact me.
This post was originally published on Medium (View original post), and automatically converted and synced by ZMediumToMarkdown.