Exploring the Long-Standing Issues of XCode Virtual Directories and My Open Source Tool Solution
The occupational hazards of Apple developers using virtual directories in Xcode early on, leading to chaotic directory structures and difficulties integrating modern tools like XcodeGen and Tuist.
ℹ️ℹ️ℹ️ The following content is translated by OpenAI.
Click here to view the original Chinese version. | 點此查看本文中文版
Exploring the Long-Standing Issues of XCode Virtual Directories and My Open Source Tool Solution
The occupational hazards of Apple developers using virtual directories in Xcode early on, leading to chaotic directory structures and difficulties integrating modern tools like XcodeGen and Tuist.
{:target="_blank"}](/assets/fd719053b376/1*fYk27y-BjMBjBFnwDfhxlw.jpeg)
Photo by Saad Salim
Background
As the team size and project scale grow, the size of the XCode project file ( .xcodeproj) gradually increases. Depending on the complexity of the project, it can reach hundreds of thousands or even millions of lines. With multiple developers working in parallel, conflicts are inevitable; once a conflict occurs in the XcodeProj file, it is as difficult to resolve as conflicts in Storyboard / .xib files. Since it is also a purely descriptive file, it is easy to accidentally remove files added by others or have references to files removed by others reappear during conflict resolution.
Another issue is that with the advancement of modularization, the process of creating and managing modules in the XCode project file ( .xcodeproj) is very unfriendly. When modules change, one can only view the changes through the Diff of the XCode project file, which is not conducive to the team’s move towards modularization.
If the goal is simply to prevent conflicts, you can easily implement File Sorting in pre-commit. There are many existing scripts on GitHub that you can refer to for setup.
XCFolder
In short, I developed a tool that can help you convert the early virtual directories of XCode into physical directories according to the directory structure in XCode.
Scroll down to continue the story…
Modernizing Xcode Project File Management
The specific thinking is similar to our discouragement of using Storyboard or .xib during multi-developer projects; we need a well-maintained, iterative, and code-reviewable interface to manage the “XCode project file”. Currently, there are two mainstream free tools available in the market:
- XCodeGen: An established tool that uses YAML to define the contents of the XCode project and then converts it into an XCode project file ( .xcodeproj). It directly uses YAML to define the structure, has a lower difficulty and learning cost for introduction, but is weaker in modularization and dependency management functions, and has limited dynamic YAML settings.
- Tuist: A newer tool that uses Swift DSL to define the contents of the XCode project and then converts it into an XCode project file ( .xcodeproj). It is more stable and versatile, with built-in modularization and dependency management features, but has a higher learning and introduction threshold.
Regardless of which set, our core workflow will be:
- Create XCode project configuration file (XCodeGen
project.yamlor TuistProject.swift) - Add XCodeGen or Tuist to the developer and CI/CD Server environment
- Use XCodeGen or Tuist to generate the
.xcodeprojXCode project file from the configuration file - Add
/*.xcodeprojdirectory files to.gitignore - Adjust the developer workflow, requiring XCodeGen or Tuist to generate the
.xcodeprojXCode project file from the configuration file when switching branches - Adjust the CI/CD workflow, requiring XCodeGen or Tuist to generate the
.xcodeprojXCode project file from the configuration file - Complete
The .xcodeproj XCode project file is generated by XCodeGen or Tuist based on the YAML or Swift DSL configuration file. The same configuration file and tool version will produce the same result; therefore, we do not need to upload the .xcodeproj XCode project file to Git, which ensures that there will be no more .xcodeproj file conflicts in the future. Changes in project builds and modifications to modules will be adjusted back in the defined configuration file. Since it is written in YAML or Swift, we can easily iterate and conduct code reviews.
Tuist Swift Example: Project.swift
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import ProjectDescription
let project = Project(
name: "MyApp",
targets: [
Target(
name: "MyApp",
platform: .iOS,
product: .app,
bundleId: "com.example.myapp",
deploymentTarget: .iOS(targetVersion: "15.0", devices: [.iphone, .ipad]),
infoPlist: .default,
sources: ["Sources/**"],
resources: ["Resources/**"],
dependencies: []
),
Target(
name: "MyAppTests",
platform: .iOS,
product: .unitTests,
bundleId: "com.example.myapp.tests",
deploymentTarget: .iOS(targetVersion: "15.0", devices: [.iphone, .ipad]),
infoPlist: .default,
sources: ["Tests/**"],
dependencies: [.target(name: "MyApp")]
)
]
)
XCodeGen YAML Example: project.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
name: MyApp
options:
bundleIdPrefix: com.example
deploymentTarget:
iOS: '15.0'
targets:
MyApp:
type: application
platform: iOS
sources: [Sources]
resources: [Resources]
info:
path: Info.plist
properties:
UILaunchScreen: {}
dependencies:
- framework: Vendor/SomeFramework.framework
- sdk: UIKit.framework
- package: Alamofire
MyAppTests:
type: bundle.unit-test
platform: iOS
sources: [Tests]
dependencies:
- target: MyApp
File Directory Structure
XCodeGen or Tuist will generate the directory structure of the XCode project file ( .xcodeproj) based on the actual directory and location of the files, where the actual directory is the directory of the XCode project file.
Therefore, the actual directory location of the files is very important, and we will directly use it as the directory for the XCode project file.
In modern XCode / XCode projects, it is quite common for these two directory locations to be equal, but this is the topic that this article aims to explore.
Early Xcode Used Virtual Directories
In early versions of XCode, right-clicking on the file directory and selecting “New Group” would not create an actual directory; the files would be placed in the project root directory and referenced in the .xcodeproj XCode project file. Thus, the directory was only visible in the XCode project file and did not actually exist.
As time evolved, Apple gradually phased out this peculiar design in XCode. Later versions of XCode introduced “New Group with Folder,” which by default would create a physical directory. To avoid creating one, you would need to select “New Group without Folder.” Now, in the current version (XCode 16), there is only “New Group” plus automatic generation of the XCode project file directory based on the actual directory.
Issues with Virtual Directories
- Cannot use XCodeGen or Tuist, as both require the physical directory location to generate the XCode project file ( .xcodeproj).
- Code Review difficulties: The directory structure cannot be seen on the Git Web GUI; all files are flattened.
- DevOps and third-party tool integration difficulties: For example, Sentry and GitHub can issue alerts or Auto-Assigners based on directory structure; without directories, only files cannot be configured.
- The project directory structure is extremely complex, with a large number of files flattened in the root directory.
For a project that is quite old and did not realize the issue of virtual directories early on, there are over 3,000 virtual directory files. If one were to manually map and move them, they would probably quit to sell rice cakes afterward; this can truly be called a “ Apple Developer Occupational Hazard 😔.”
Converting Xcode Project Virtual Directories to Physical Directories
For the reasons mentioned above, we urgently need to convert the virtual directories of the XCode project into physical directories; otherwise, subsequent project modernization and more efficient development processes cannot be advanced.
❌ Xcode 16 “Convert to folder” Option
XCode 16
When XCode 16 was first released last year, I noticed this new option in the menu. The initial expectation was that it could automatically convert virtual directory files into physical directories.
However, this is not the case. It requires you to first create the directories and place the files in the corresponding physical locations. After clicking “Convert to folder,” it will change to the new XCode Project directory setting method “ PBXFileSystemSynchronizedRootGroup.” To be honest, this is not useful for conversion; it is more about upgrading to a new directory setting method after conversion.
If the directories are not created and the files are not placed correctly, clicking “Convert to folder” will result in the following error:
1
2
3
4
Missing Associated Folder
Each group must have an associated folder. The following groups are not associated with a folder:
• xxxx
Use the file inspector to associate a folder with each group, or delete the groups after moving their content to another group.
🫥 Open Source Projects venmo / synx
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
After searching on GitHub for a long time, I only found this open-source tool written in Ruby for converting virtual directories to physical ones. It works effectively when run, but due to being outdated (approximately 10 years without updates), many files still need to be manually mapped and moved, making complete conversion impossible. Therefore, I gave up.
> _However, I am still very grateful for the inspiration from this open-source project, which led me to think about developing my own conversion tool._
#### ✅ My Open Source Projects [ZhgChgLi](https://github.com/ZhgChgLi){:target="_blank"} / [XCFolder](https://github.com/ZhgChgLi/XCFolder){:target="_blank"}
[](https://github.com/ZhgChgLi/XCFolder){:target="_blank"}
Developed using pure Swift, this Command Line Tool is based on [XcodeProj](https://github.com/tuist/XcodeProj){:target="_blank"} to parse `.xcodeproj` Xcode project files, read all files to obtain their directories, parse the directories to convert virtual directories into physical ones, move the files to the correct locations, and finally adjust the directory settings in the `.xcodeproj` Xcode project file to complete the conversion.
**Usage**
```bash
git clone https://github.com/ZhgChgLi/XCFolder.git
cd ./XCFolder
swift run XCFolder YOUR_XCODEPROJ_FILE.xcodeproj ./Configuration.yaml
For Example:
1
swift run XCFolder ./TestProject/DCDeviceTest.xcodeproj ./Configuration.yaml
CI/CD Mode ( Non Interactive Mode )：
1
swift run XCFolder YOUR_XCODEPROJ_FILE.xcodeproj ./Configuration.yaml --is-non-interactive-mode
Configuration.yaml can be used to set the desired execution parameters:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# Directories to ignore, will not be parsed or converted
ignorePaths:
- "Pods"
- "Frameworks"
- "Products"
# File types to ignore, will not be converted or moved
ignoreFileTypes:
- "wrapper.framework" # Frameworks
- "wrapper.pb-project" # Xcode project files
#- "wrapper.application" # Applications
#- "wrapper.cfbundle" # Bundles
#- "wrapper.plug-in" # Plug-ins
#- "wrapper.xpc-service" # XPC services
#- "wrapper.xctest" # XCTest bundles
#- "wrapper.app-extension" # App extensions
# Only create directories and move files, do not adjust .xcodeproj Xcode project file directory settings
moveFileOnly: false
# Prefer using git mv command to move files
gitMove: true
⚠️ Please note before executing:
- Ensure that Git has no uncommitted changes, as the script may corrupt your project directory if there are issues. (The script will check, and if there are uncommitted changes, it will throw an error
❌ Error: There are uncommitted changes in the repository.) - By default, it prefers to use the
git mvcommand to move files to ensure complete git file log records. If the move fails or if it is not a Git project, it will use FileSystem Move to move the files.
Just wait for the execution to complete:
```
⚠️ Please note after execution:
- Please check if there are any missing (red) files in the project directory. If there are few, you can manually fix them; if there are many, please confirm whether the settings for
ignorePathsandignoreFileTypesin Configuration.yaml are correct, or create an Issue to let me know. - Check if the relevant paths in Build Settings, e.g.,
LIBRARY_SEARCH_PATHS, need to be manually changed. - It is recommended to Clean & Build to see the results.
- If you don’t want to deal with the current
.xcodeprojXCode project file, you can also directly start using XCodeGen or Tuist to regenerate the directory files.
Modify Script:
You can directly click ./Package.swift to open the project and adjust the script content.
Other Development Notes
- Thanks to XcodeProj, we can easily access the contents of the
.xcodeprojXCode project file using Swift objects. - Developed using the Clean Architecture structure.
- If there is no path in the PBXGroup settings and only a name, it is a virtual directory; otherwise, it is a physical directory.
- The new directory setting
PBXFileSystemSynchronizedRootGroupin XCode 16 only requires declaring the main directory, and it will automatically resolve from the physical directory without needing to declare every directory and file in the.xcodeprojXCode project file. - Developing command line tools directly using SPM (Package.swift) is really convenient!
If you have any questions or suggestions, feel free to contact me.
This article was first published on Medium ➡️ Click Here
Automatically converted and synchronized using ZMediumToMarkdown and Medium-to-jekyll-starter.
