CI/CD Practical Guide (Part 1): What is CI/CD? How to Build a Stable and Efficient Development Team through CI/CD? Tool Selection?
Using the App (iOS) Team as an example, this guide will introduce you to CI/CD from scratch and the tangible value it can bring after implementation.
ℹ️ℹ️ℹ️ The following content is translated by OpenAI.
Click here to view the original Chinese version. | 點此查看本文中文版
CI/CD Practical Guide (Part 1): What is CI/CD? How to Build a Stable and Efficient Development Team through CI/CD? Tool Selection?
Using the App (iOS) Team as an example, this guide will introduce you to CI/CD from scratch and the tangible value it can bring after implementation.
{:target="_blank"}](/assets/c008a9e8ceca/1*vokpvb4dyWHOnVnF3WGbfw.jpeg)
Photo by Leif Christoph Gottwald
Introduction
After going through the experience of building App CI/CD in two different development teams, I finally found time to organize my thoughts from “why do it” to “how to do it.” I can’t guarantee that this is the most standard CI/CD workflow, but it is definitely a worthwhile starting point to help your team begin implementation, improve product stability, and enhance overall development efficiency.
Chapters
This series of articles will start with “What is CI/CD and what value does it bring?” Then, we will walk through the hands-on implementation of “How to set up a CI/CD environment using GitHub Actions + self-hosted Runner” and “Using App development as an example, practically implementing CI and CD.” Finally, we will introduce how to “Use AI Vibe Coding Google Apps Script Web App combined with GitHub Actions to create a convenient app packaging platform for cross-team use.” I hope this series will be helpful to you.
- CI/CD Practical Guide (Part 1): What is CI/CD? How to Build a Stable and Efficient Development Team through CI/CD? Tool Selection?
- CI/CD Practical Guide (Part 2): Comprehensive Guide to Using and Building GitHub Actions and Self-hosted Runner
- CI/CD Practical Guide (Part 3): Implementing CI and CD Workflows for App Projects using GitHub Actions
- CI/CD Practical Guide (Part 4): Using Google Apps Script Web App (AI Vibe Coding) combined with GitHub Actions to Create a Free and Easy-to-Use Cross-Team Packaging Tool
Final Results
Without further ado, here are the final results.
CI/CD — All developed using GitHub Actions, easy to maintain and expand.
CI:
- Automatically trigger unit tests upon PR submission
- Execute corresponding tests based on the scope of changed files
- Only merge PRs after tests pass
CD:
- Google Apps Script Web App (CD packaging interface) allows engineers, QA, and PMs to package the app on their computers or mobile devices
- GitHub Actions Self-hosted Runner uses your own machine for unlimited CI/CD usage
- Integrate with Firebase App Distribution API to directly obtain download links for packaged test versions
Automation:
- Automatically assign self upon PR submission
- Randomly assign reviewers upon PR submission
- Label PR size
What is CI/CD?
Story — Development Process Without CI/CD
Before discussing what CI/CD is, let’s set aside the term “CI/CD” and recall how a startup development team without any established workflow operates. The process can be roughly summarized in the following diagram:
- A product has a bug, Developer T creates a branch fix/bug-c from the main branch to fix it, and after completing the fix, merges it back to the main branch.
- Immediately after, Developer Z creates a branch feature/a from the main branch for Requirement A. Halfway through, they notice something is off with the feature and upon checking, realize that the current functionality has been broken and the tests are also failing, so they notify Developer T to make the fix.
- Once all development is completed, Developer Z packages the version for QA testing on their computer, going back and forth to make corrections and repack. Finally, after confirming everything is fine, they merge the feature back to the main branch.
- As the sprint comes to an end, they need to package a release for users; Developer Z puts aside their current work to assist in packaging from the main branch for QA to conduct regression testing, similarly going back and forth to fix issues and repackage before submitting the app for review.
- After Apple/Google reviews, the app is released to users.
Problems
From the above story, we can identify two major issues.
Question 1: There is no unified check mechanism for current functional changes.
- Code that does not conform to coding style can still be merged
- I can merge even if the build fails
- Basic unit tests and important checks can be bypassed even if they fail
- My environment may work correctly, but others may not
- It affects others who are still developing
Question 2: A significant amount of manpower is wasted on packaging tasks.
- Packaging requires engineers to manually package, interrupting current development work
- The cost of switching between packaging and development is extremely high
- Waiting time for packaging prevents other development work from being done
- The time cost of engineers translates to money
- Manual operations may lead to errors
- QA needs to ask engineers to package (back and forth communication)
CI — Continuous Integration
In response to Question 1, “Continuous Integration” aims to ensure that all changes automatically undergo a unified environment build and test to ensure that modifications pass all test cases and comply with team standards before entering the production environment — “continuously and automatically ensuring correct code integration into the production environment.”
Additionally, it can include nightly builds and more automated testing phases to ensure stability.
CD — Continuous Delivery / Deployment
In response to Question 2, “Continuous Deployment” aims to ensure that after the CI phase shows no abnormalities, the results of changes are automatically packaged and deployed through tedious processes for internal testing (QA, Debug, Staging, Beta…) or external release (Production, Release…).
- Continuous Deployment: Fully automated direct deployment to the Production environment
- Continuous Delivery: Automatically deploys to Staging/Debug environments, requiring manual verification before deploying to the Production environment
In the context of app development, it leans more towards Continuous Delivery, as we want to ensure that the app is fully verified by humans before release, ensuring the accuracy of release timing and functionality.
Story — Building a Stable and Efficient Development Team through CI/CD
Looking back at our story, after implementing CI/CD:
- CI All adjustments require passing automated testing verification before entering the main branch, with the addition of nightly builds for regular automated testing to enhance stability.
- CD Unified CD packaging allows Developer T and Developer Z to focus entirely on business development, reducing manual communication and operational errors.
Team efficiency and product stability 🚀🚀🚀🚀🚀
The Value of CI/CD
Combining the core concept of agile development “small steps, fast iterations,” CI/CD provides the foundation for stability and work efficiency during “frequent continuous iterations of features.”
Automated Unified Verification of Iteration Results
- Ensures all adjustments meet the expected results, do not affect other functionalities, and do not impact other team members
Automated Execution of Tedious Deployment Processes
- Allows team members to focus on primary business development, reducing manual operational errors
The Effectiveness of CI/CD
Reflecting on the 2021 presentation at Pinkoi “2021 Pinkoi Tech Career Talk — Secrets of High-Efficiency Engineering Teams,” the content is essentially the same, revolving around “automation, reducing dependency on individuals, and focusing on primary business.” Implementing CI/CD aligns perfectly with these three directions, allowing us to estimate effectiveness in the same way.
Another point that needs to be highlighted is the cost of switching mental states:
When we continuously engage in work for a period, we enter a “flow” state where our thoughts and productivity peak, allowing for the most effective output. However, if interrupted, it takes time to return to that flow, which can be estimated at around 30 minutes.
In a scenario without CI/CD, it might look like this: Spending a lot of time only to discover that something was broken and then going back to communicate adjustments (CI), and having QA/PM ask engineers to help package the test version of the app (CD).
Estimating CI/CD Effectiveness
Team size of 6 people / Monthly calculation
Here, we use a monthly basis, assuming that without a CI/CD process, there would be 4 instances of unexpected issues breaking the main branch each month, leading to subsequent correction and communication costs, approximately taking 720 minutes; plus the time spent packaging test versions, official versions, and the potential errors caused by manual operations, totaling around 1,010 minutes; calculating with an engineer’s monthly salary of 80,000, this results in approximately 13,000 in wasted costs each month.
CI/CD Implementation Costs
- Labor Costs: Based on the construction in this series of articles, it is estimated that 1 person will need to invest 10 days = 4,800 minutes to complete it. (~= NT$36,384 )
- Equipment and Execution Costs: Using GitHub Actions self-hosted Runner only requires the procurement of 1–2 Mac Mini or directly using existing replaced MacBook Pro to serve as CI/CD Runner. Taking the procurement of a brand new Mac Mini with 32G RAM as an example: (= NT$40,900 )
The total cost of approximately NT$80,000 can complete the setup, with benefits starting to be enjoyed after about six months.
Tool Selection for CI/CD
Cloud Services Bitrise / XCode Cloud
- Bitrise: The earliest cloud service focused on providing App CI/CD. My first experience with CI/CD was also using Bitrise, which offers a user-friendly step editing tool to quickly set up the App CI/CD process. Cons: Initially, it was $99 for unlimited usage, but when the Apple M series processors were newly released, they switched to usage-based billing ( bait and switch ), estimating that the team would need to spend at least $500 monthly, so we migrated to GitHub Actions. However, I recently checked their website, and they now offer 1 App / 1 Concurrent / unlimited usage for $89 per month.
- XCode Cloud: 100 hours / 1 month / $50, with the advantage of being highly integrated with XCode and app development; however, it also has the drawback of not supporting Android and some custom steps being somewhat difficult; but if it’s a small iOS app, I would reconsider using it directly.
I am really worried about cloud services being a bait and switch, hoping to keep control in my hands, so I am considering on-premises services.
On-Premises Services Jenkins / GitHub Actions / Gitlab CI/CD
- Gitlab CI/CD: Launched earlier than GitHub Actions and has more complete features, but since our project is hosted on GitHub, we won’t consider using Gitlab CI/CD; however, the functionalities are similar, and this series of articles will use GitHub Actions as an example.
- GitHub Actions Launched in 2018, GitHub’s CI/CD service is directly tied to GitHub projects, continuously updating and improving features over the years, with many pre-packaged steps available in the ( Marketplace ) for direct use; it supports self-hosted runners that allow unlimited use of your own machines. (equivalent to hybrid cloud)
- Jenkins: An open-source free tool specifically for handling CI/CD, old but powerful; from application layer task design, permission management to underlying service dispatch execution, Jenkins covers it all; it also has ( Plugins ) that can be used directly, making it an essential tool for early DevOps CI/CD.
Jenkins vs. GitHub Actions
TL;DR
For an App Team without dedicated DevOps, having app developers set up and maintain a Jenkins environment from scratch is too high a barrier, and there are not many who know how to do it, which can also lead to network security issues; choosing to use GitHub Actions directly allows app developers to focus solely on CI/CD process design, and with a quick glance at the official documentation on how to write and start the Runner, they can quickly set up a free, stable, and secure CI/CD service.
The following comparison is based solely on setting up App CI/CD and may not apply to all technical scenarios.
Setup and Maintenance Difficulty Jenkins »> GitHub Actions
Here, I will use a not-so-professional structural hierarchy diagram to explain the differences between the two. As mentioned, Jenkins encompasses all functionalities from top to bottom, making self-hosting much more complex; whereas GitHub Actions only requires writing YAML workflows on GitHub, and the local machine only needs to register the GitHub self-hosted Runner (5 commands to complete), and GitHub will automatically dispatch tasks to the local machine for execution. Other aspects, including GitHub Actions/Runner version upgrades or task dispatch issues, are maintained by GitHub, and we do not need to handle them.
Another more troublesome point is that Jenkins operates independently of Git, requiring communication through APIs (e.g., GitHub API/WebHook), which complicates the setup further.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
---
title: CI/CD Practical Guide: What is CI/CD and How to Build an Efficient Development Team
---
Previously, I conducted a survey among about 30 iOS developers I could reach, and only a handful (2) understood Jenkins, while more than 10 were using GitHub Actions. After all, you can complete CI/CD tasks just by writing some YAML.
#### Learning Difficulty: Jenkins >>> GitHub Actions
As mentioned, you only need to refer to the official documentation to learn how to use YAML commands with GitHub Actions and how to set up your own Runner locally.
#### Stability: GitHub Actions > Jenkins
In this regard, I believe GitHub Actions slightly outperforms Jenkins.
Jenkins can crash due to system upgrades or conflicts with installed plugins (however, if it’s running smoothly and you don’t touch it, there’s basically no problem).
GitHub Actions is limited by the [GitHub service status](https://www.githubstatus.com/){:target="_blank"} (if GitHub is down, it will also be down), but this happens infrequently, with an average uptime of 99.9%; if issues do arise, you don’t need to handle them, just wait for a fix.
#### Security: GitHub Actions > Jenkins
Considering that the GitHub Actions/Runner service is maintained and automatically updated by GitHub, it may be more secure than Jenkins, which requires manual updates.
Additionally, Jenkins needs to open API/WebHook ports for communication with GitHub, which is relatively risky. In contrast, GitHub integrates seamlessly with GitHub Actions, and the self-hosted Runner operates in an observer mode, requesting tasks from GitHub, so the Runner itself does not need to open external interfaces.
However, in a fully closed network environment, Jenkins would be more secure than GitHub Actions.
#### Permission Control: Jenkins >>> GitHub Actions
This point needs to be highlighted for comparison. Jenkins allows for separate account login permissions for control; GitHub Actions is directly tied to the GitHub Repo, meaning only those with Repo permissions can use it.
> _\*This is why the following articles will use GAS Web App to build a cross-team operational platform._
#### Usage Scope: Jenkins >>> GitHub Actions
In teams with a complete DevOps team, it is undoubtedly preferable to choose Jenkins, as it has been running the longest in other fields (such as Web, backend, Java, etc.), has the most plugins, and can unify a CI/CD service for all teams for easier management, or handle complex CI/CD scenarios like automatically deploying the frontend after backend deployment.
> _\*GitHub Actions later also supported cross-repo Actions/Runners._
#### Richness of Third-Party Plugins: Jenkins > GitHub Actions
In terms of quantity, GitHub Actions has more than Jenkins, but Jenkins' CI/CD capabilities are deeper and more powerful, while many GitHub Actions features are just automation functions.
#### Depth of Functionality: Jenkins >>> GitHub Actions
This is not comparable; Jenkins has been around for nearly 20 years, while GitHub Actions still has many features to add, such as: permission management, secret management (currently limited to plain text; key files must be converted to plain text), **no Auto Scaling Runner** (needs to be implemented via API), Runner weight settings, etc.
#### Customization Depth: Jenkins >>> GitHub Actions
Jenkins is fully under your control, allowing for greater customization that can affect the entire system. GitHub Actions can only customize different steps at the application layer.
For example, the built-in Artifacts in GitHub Actions do not support self-hosted, so you can only change it to `sh copy` to another directory in the steps, without the ability to customize the implementation of Artifacts.
> _\*App CI/CD scenarios do not require very deep functionality._
#### Usability: GitHub Actions >>> Jenkins
In terms of interface, GitHub Actions is a newer tool that is easier to use than Jenkins; in terms of script settings, Jenkins uses Pipeline Script stored on Jenkins, while GitHub Actions uses YAML files managed with Git, making it easier to set up than Jenkins.
#### Cost Risk: Jenkins > GitHub Actions
Jenkins is fully open-source and free, with control in your hands. GitHub Actions is partially open-source, but task distribution and execution are a closed SaaS service by GitHub; currently, the policy is that GitHub Actions is completely free, but using GitHub Runner incurs costs (for Private Repos), while using self-hosted runners is free.
### Purpose of Google Apps Script Web App and Why It Was Chosen
Another tool choice is Google Apps Script Web App. The reason for needing this is that the form functionality provided by GitHub Actions is too basic (the interface is too engineering-focused and only static), and the execution permissions are tied to the GitHub Repo, making it very cumbersome if we need to provide it for other functional partners to use.
**As follows:**
For CD packaging, we hope to allow operators to fill in some information, such as Release Notes.
Therefore, we need a "interface" tool to provide for other partners or even for our engineers to use more conveniently.
**Required Scenario:**
> _Fill in the required information on this more user-friendly "interface," connect to project management tools (e.g., Jira, Asana) to obtain tasks or directly retrieve the PR list from GitHub, select from a dropdown menu, and submit, triggering GitHub Actions for packaging via the GitHub API._
#### Slack
When we first introduced CI/CD, we chose to integrate with the Slack API to achieve effects similar to the following:
{:target="_blank"}](/assets/c008a9e8ceca/1*m85bTTlrwAmCxoVqVXo2fg.png)
[https://slack\.com/intl/zh\-tw/blog/productivity/workflow\-builder\-tools\-automation\-examples](https://slack.com/intl/zh-tw/blog/productivity/workflow-builder-tools-automation-examples){:target="_blank"}
- Partners can directly use forms in Slack to fill in information, trigger CD packaging, and receive Slack notifications.
The operation is smooth and unified within daily office tools (SSOT), requiring no relearning; however, the underlying issue is that **the development and maintenance costs are very high**. One reason is that the Slack Outgoing-Webhook API has very high response requirements (needs to respond within 3 seconds), which basically rules out using FAAS services for simple integration (e.g., Cloud Functions, GAS, Lambda…).
**Previously, one partner who was very interested in automation and backend developed a complete backend service using Kotlin+ktor, and then set up a server on GCP for Slack integration.**
> **_The development and maintenance costs are extremely high and very difficult to hand over._**
#### Google Apps Script — Web App
I previously shared "[Using Google Apps Script Web App Forms to Integrate GitHub Action CI/CD Work](../4cb4437818f2/)":
{:target="_blank"}](/assets/c008a9e8ceca/1*NJRcY2ULVylZlsKnBtM27A.png)
[Demo Web App Form URL](https://script.google.com/macros/s/AKfycbw8SuK7lLLMdY86y3jxMJyzXqa5tdxJryRnteOnNi-lK--j6CmKYXj7UuU58DiS0NSVvA/exec){:target="_blank"}
**The advantages of using Google Apps Script — Web App are:**
- Web-based
- Permission management with Google Workspace enterprise accounts, allowing access only to Google accounts within the organization
- **Completely free**
- **Function as a Service without needing to set up and maintain servers**
- Easier to maintain and hand over
- Usable on mobile
- **AI Can Help!**
**Whether it's ChatGPT or other AI tools, they are very familiar with GAS and can directly help us create packaging forms and integrate with the GitHub API.**
- Can also integrate with Jira, Asana, Slack notification APIs.
For the second promotion, I switched to using GAS Web App for partners, and received very positive feedback. The only difference from Slack is that you need to bookmark one more URL; when packaging is needed, just open the URL and operate the packaging through the web form.
### Complete Tool Workflow for App CI/CD
Here, I will provide the complete workflow. The next article will gradually introduce how to use and integrate each tool.
#### Tool Roles:
- **GitHub Actions**: CI/CD logic script code
- **GitHub Actions — Self-hosted Runner**: The actual execution location for CI/CD, using a self-hosted Runner to execute tasks without limits on usage, only incurring the cost of purchasing the machine.
- **Google Apps Script Web App**: Since packaging may not always be the responsibility of engineers, a platform is needed for cross-functional partners to use; GAS Web App can quickly create a web tool and share the URL for others to operate.
- **Asana/Jira**: Project management tools that can integrate with GAS Web App, allowing QA/PM to directly select tasks to be packaged.
- **Slack**: Responsible for receiving execution result notifications.
#### Scenarios:
- End-User (QA/PM/PD/Developer): Uses GAS Web App to submit the packaging form (fetching the corresponding branch for Jira or Asana tasks) -> GAS calls GitHub API -> triggers CD packaging in GitHub Actions <- GitHub self-hosted runner listens for tasks and pulls them back to execute -> upon completion, Slack notification, updates GAS Web App packaging status.
- End-User (Developer): Opens PR, pushes new commits to PR -> triggers CI testing process <- GitHub self-hosted runner listens for tasks and pulls them back to execute -> upon completion, comments on test results, updates Checks.
### Conclusion
This article primarily aims to give everyone a preliminary understanding of what CI/CD is and the benefits it brings. The next article will delve into the technical aspects, guiding you step-by-step to understand and implement GitHub Actions CI/CD to achieve the final results mentioned earlier.
#### Next Article: [**CI/CD Practical Guide (2): Using and Building GitHub Actions and Self-hosted Runners**](../404bd5c70040/)
If you have any questions or suggestions, feel free to [contact me](https://www.zhgchg.li/contact){:target="_blank"}.
This article was first published on Medium ➡️ Click Here
Automatically converted and synchronized using ZMediumToMarkdown and Medium-to-jekyll-starter.
