What is CI/CD?
CI/CD is an umbrella term for three related but distinct practices that together form the backbone of modern software delivery. Continuous Integration (CI) means every developer merges their code changes into a shared repository frequently — at least once a day — and each merge triggers an automated build and test suite. The goal is to detect integration problems early, when they are small and cheap to fix, rather than discovering them weeks later during a painful "integration phase" that delays the entire release.
Continuous Delivery (CD) extends CI by automatically preparing every successful build for release. After code passes all tests, it is packaged, versioned, and staged in a production-like environment — ready to be deployed at the push of a button. The key distinction is that deployment still requires a human decision. This gives teams confidence that they can release at any time while retaining control over when changes go live.
Continuous Deployment goes one step further by removing the manual gate entirely. Every change that passes the full pipeline — build, test, security scan, staging validation — is automatically deployed to production without human intervention. This requires exceptional test coverage and monitoring, but teams that achieve it can ship hundreds of times per day with lower risk than teams deploying manually once a month.
The 4 Pipeline Stages
Every CI/CD pipeline, regardless of the tools or cloud provider used, follows the same four fundamental stages. Understanding these stages is essential before writing your first workflow file, because the structure of your pipeline should mirror this progression from code commit to production deployment. Each stage acts as a quality gate — if any stage fails, the pipeline stops and the team is notified immediately.
The philosophy behind this staged approach is fail fast. The cheapest bugs to fix are the ones caught earliest. A syntax error caught during the Build stage costs minutes to resolve. The same error discovered in production costs hours of debugging, customer impact, and potentially lost revenue. By structuring your pipeline as a series of increasingly rigorous checks, you ensure that only thoroughly validated code reaches your users.
CI/CD Pipeline Stages
| Stage | What Happens | Tools Used | Success Criteria |
|---|---|---|---|
| Source | A code commit or pull request triggers the pipeline. The source code is checked out from the repository and made available to subsequent stages. | Git, GitHub, GitLab, Bitbucket. Webhook triggers or polling. | Repository is cloned successfully. Correct branch and commit SHA are checked out. |
| Build | The application is compiled, dependencies are installed, and build artefacts are created. This stage verifies that the code can be assembled into a working application. | npm, Maven, Gradle, Docker, Webpack, esbuild, Go build. | Build completes without errors. Artefacts are generated and stored for later stages. |
| Test | Automated tests are executed against the build artefacts. This includes unit tests, integration tests, linting, code formatting checks, and security vulnerability scans. | Jest, pytest, JUnit, ESLint, SonarQube, Snyk, Trivy. | All tests pass. Code coverage meets minimum threshold. No critical security vulnerabilities detected. |
| Deploy | The validated artefact is deployed to a staging environment for final verification, then promoted to production. Smoke tests confirm the deployment is healthy. | AWS CLI, Terraform, kubectl, Helm, Vercel CLI, rsync. | Deployment completes without errors. Smoke tests pass. Health checks return 200 OK. |
Fail Fast, Fix Fast
The most effective pipelines are designed to catch failures as early as possible. Run your fastest checks first — linting and unit tests typically complete in seconds — before moving on to slower integration tests and deployments. If a linting error can be caught in 10 seconds, there is no reason to wait 5 minutes for the full test suite to fail on the same issue. Structure your pipeline stages from fastest to slowest, and your team will spend less time waiting and more time shipping.
GitHub Actions Fundamentals
GitHub Actions is a CI/CD platform built directly into GitHub. It allows you to automate your build, test, and deployment pipelines using YAML configuration files stored in your repository. Because it is tightly integrated with GitHub's pull request and issue workflows, Actions has become the most popular CI/CD tool for open-source and commercial projects hosted on the platform.
The power of GitHub Actions lies in its composability. You define workflows as YAML files, each workflow contains one or more jobs, each job contains a series of steps, and each step can run a shell command or invoke a reusable action from the GitHub Marketplace. With over 20,000 community-maintained actions available, you can assemble sophisticated pipelines without writing boilerplate scripts from scratch.
Key GitHub Actions Concepts
| Concept | Description | YAML Example |
|---|---|---|
| Workflow | A YAML configuration file stored in .github/workflows/. Each workflow defines a complete automation pipeline that runs in response to events. | name: CI Pipeline |
| Event / Trigger | The event that starts a workflow. Common triggers include push, pull_request, schedule (cron), workflow_dispatch (manual), and release. | on: [push, pull_request] |
| Job | A set of steps that execute on the same runner. Jobs run in parallel by default, but can be configured to run sequentially using the needs keyword. | jobs: build: runs-on: ubuntu-latest |
| Step | An individual task within a job. Each step either runs a shell command (run) or invokes a reusable action (uses). | - run: npm test |
| Action | A reusable, versioned unit of automation. Actions are referenced by owner/repo@version. The GitHub Marketplace hosts over 20,000 community actions. | - uses: actions/checkout@v4 |
| Runner | The virtual machine or container that executes a job. GitHub provides hosted runners for Linux, macOS, and Windows. Self-hosted runners can be used for custom environments. | runs-on: ubuntu-latest |
| Secret | An encrypted environment variable stored in repository or organisation settings. Secrets are masked in logs and never exposed in workflow files. | $${{ secrets.DEPLOY_KEY }} |
| Artefact | A build output (compiled binary, test report, Docker image) that can be shared between jobs or downloaded after a workflow completes. | - uses: actions/upload-artifact@v4 |
GitHub Actions Free Tier
GitHub provides 2,000 minutes per month of free GitHub Actions usage for public and private repositories on the Free plan. Public repositories get unlimited minutes. This is more than enough for most small-to-medium teams to run comprehensive CI/CD pipelines without any cost. Linux runners are the cheapest (1x multiplier), while macOS runners cost 10x — so use Linux runners wherever possible and reserve macOS for builds that genuinely require it, such as iOS applications.
Building Your First CI Workflow
The best way to learn GitHub Actions is to build a real workflow step by step. The following example creates a complete CI pipeline for a Node.js project that triggers on every push and pull request, runs linting, executes the test suite, and produces a build artefact. Every section of the YAML is explained so you understand not just what to write, but why each piece exists.
Create a file at .github/workflows/ci.yml in your repository. GitHub automatically detects and runs any YAML file in this directory. The file name can be anything, but using descriptive names like ci.yml, deploy.yml, or release.yml makes it obvious what each workflow does when your repository grows to contain multiple pipelines.
CI Workflow YAML Breakdown
YAML Section Purpose Example name A human-readable name for the workflow. This appears in the GitHub Actions tab and in status checks on pull requests. name: CI Pipelineon (triggers) Defines which events trigger the workflow. You can specify branches, paths, and event types to control when the pipeline runs. on: push: branches: [main] pull_request: branches: [main]jobs The top-level key that contains all jobs. Each job has a unique identifier and runs on its own runner. Jobs run in parallel unless dependencies are declared. jobs: build: ...runs-on Specifies the runner environment for the job. ubuntu-latest is the most common choice — it is fast, cheap, and compatible with most tools. runs-on: ubuntu-latestCheckout The first step in almost every job. Uses the official actions/checkout action to clone your repository code onto the runner. - uses: actions/checkout@v4Setup Node Installs the specified Node.js version on the runner. The node-version input supports exact versions, ranges, and aliases like lts/*. - uses: actions/setup-node@v4 with: node-version: 20Install dependencies Installs project dependencies using the package manager's clean install command. npm ci is preferred over npm install because it uses the lockfile exactly. - run: npm ciLint Runs the linting step to enforce code style and catch common errors before tests run. This is typically the fastest check in the pipeline. - run: npm run lintTest Executes the test suite. If any test fails, the entire job fails and the pipeline stops. Test results can be uploaded as artefacts for inspection. - run: npm testBuild Compiles the application and produces build artefacts. This verifies that the code can be packaged for deployment even if no deployment step follows. - run: npm run build
For projects that need to support multiple runtime versions, GitHub Actions provides the matrix strategy. A matrix lets you run the same job across different combinations of operating systems, language versions, or any other variable. For example, testing against Node.js 18, 20, and 22 simultaneously ensures your code works across all supported versions. The matrix runs each combination as a separate job in parallel, so the total pipeline time remains roughly the same as a single-version run.
Matrix Strategy Configuration
Parameter Purpose Example strategy.matrix Defines the variables and their values. GitHub Actions creates a job for each combination of values in the matrix. strategy: matrix: node-version: [18, 20, 22]fail-fast When set to false, all matrix jobs run to completion even if one fails. Useful for understanding the full scope of a failure across versions. strategy: fail-fast: falseMatrix reference Use the matrix context to reference the current value in steps. This dynamically sets the Node version for each parallel job. $${{ matrix.node-version }}
Cache Dependencies to Speed Up Builds
Installing dependencies from scratch on every pipeline run wastes time and bandwidth. GitHub Actions provides built-in caching via actions/cache or the cache option in setup actions like actions/setup-node. Caching node_modules or the npm/Yarn cache directory can reduce install times from minutes to seconds. A typical Node.js project sees a 60-80% reduction in pipeline time after enabling dependency caching. Always use a cache key that includes your lockfile hash so the cache is invalidated automatically when dependencies change.
Adding Continuous Deployment
Once your CI pipeline is running reliably, the next step is to automate deployment. Continuous Deployment eliminates the manual steps between a successful build and a live release, reducing the risk of human error and enabling your team to ship changes faster. The deployment target you choose depends on your application type, infrastructure requirements, and budget.
GitHub Actions supports deployment to virtually any platform through official and community actions. Static sites can be deployed to GitHub Pages in seconds. Containerised applications can be pushed to AWS ECS, Google Cloud Run, or Azure Container Apps. Serverless functions deploy to AWS Lambda or Azure Functions. The following table compares the most common deployment targets and their trade-offs.
Deployment Target Comparison
Target Deployment Action Complexity Cost GitHub Pages actions/deploy-pages@v4 — deploys static assets directly from the workflow. Ideal for documentation sites, SPAs, and static marketing pages.Low — minimal configuration required. Built into GitHub with no external services needed. Free for public repositories. Included with GitHub Pro and Team plans for private repositories. AWS (S3 + CloudFront, ECS, Lambda) aws-actions/configure-aws-credentials@v4 combined with AWS CLI commands or dedicated actions for ECS, Lambda, and S3 sync.Medium to High — requires IAM roles, credential management, and familiarity with AWS services and networking. Pay-as-you-go. S3 hosting is pennies per month. ECS and Lambda costs scale with traffic and compute time. Azure (App Service, Static Web Apps) azure/webapps-deploy@v3 for App Service. azure/static-web-apps-deploy@v1 for Static Web Apps with built-in staging environments.Medium — Azure Portal setup required. Service principal credentials stored as GitHub Secrets. Free tier available for Static Web Apps. App Service starts at approximately $13/month for Basic tier. Vercel / Netlify Vercel and Netlify provide built-in Git integration that auto-deploys on push. Alternatively, use vercel --prod or netlify deploy --prod via CLI in a workflow step. Low — both platforms detect framework and build settings automatically. Preview deployments on every PR. Generous free tiers. Vercel: 100GB bandwidth. Netlify: 100GB bandwidth, 300 build minutes per month.
Production deployments require safeguards. GitHub provides environment protection rules that add approval gates, wait timers, and branch restrictions to your deployment workflows. These rules ensure that code cannot reach production without the appropriate reviews and validation, even if the pipeline itself is fully automated.
Environment Protection Rules
Rule What It Does When to Use Required reviewers The deployment pauses and waits for one or more designated reviewers to approve before proceeding. Reviewers receive a notification in GitHub. Production deployments where a human must verify readiness. Compliance-regulated environments requiring sign-off before release. Wait timer Adds a configurable delay (in minutes) before the deployment begins. The deployment can be cancelled during the wait period. Staggered rollouts where you want a grace period to monitor staging before production. Deployments during business hours only. Deployment branches Restricts which branches can deploy to the environment. Only workflows running on the specified branches are allowed to proceed. Ensuring only main or release/* branches can deploy to production. Preventing feature branches from accidentally reaching live environments.
Never Store Secrets in Code
API keys, database passwords, deployment credentials, and access tokens must never be committed to your repository — not even in private repositories. Use GitHub Secrets to store sensitive values and reference them in workflows with $${{ secrets.SECRET_NAME }}. Secrets are encrypted at rest, masked in workflow logs, and only available to workflows running in the repository where they are defined. For additional security, use OpenID Connect (OIDC) to authenticate with cloud providers like AWS and Azure without storing long-lived credentials at all.
Pipeline Best Practices
A working pipeline is a good start, but a well-optimised pipeline is what separates productive teams from frustrated ones. Slow pipelines drain developer productivity — every minute a developer waits for CI is a minute of lost focus and momentum. Flaky pipelines erode trust, leading teams to ignore failures or merge without waiting for results. The following best practices, drawn from the experience of high-performing engineering teams, will help you build pipelines that are fast, reliable, and maintainable.
These practices align with the DORA research programme (DevOps Research and Assessment), which has studied thousands of organisations to identify the capabilities that drive software delivery performance. Teams that excel at these practices consistently achieve higher deployment frequency, shorter lead times, lower change failure rates, and faster recovery from incidents.
Pipeline Optimisation Practices
Practice Why It Matters How to Implement Keep pipelines fast (under 10 minutes) Developers context-switch when pipelines take too long. Research shows that pipeline times above 10 minutes cause developers to start other work and lose focus on the change being validated. Profile your pipeline to identify bottlenecks. Parallelise independent steps. Remove unnecessary stages. Set a team target and monitor pipeline duration as a metric. Run tests in parallel Running tests sequentially wastes runner time. Most test suites can be split into independent shards that run simultaneously, dramatically reducing total wall-clock time. Use matrix strategies to split tests by module or test file. Tools like Jest support --shard for automatic splitting. Integration tests can run in parallel with unit tests. Cache dependencies Downloading and installing dependencies from scratch on every run is the single biggest source of pipeline slowness for most projects. Caching eliminates this redundant work. Use actions/cache@v4 with a key based on your lockfile hash. Most setup-* actions support a cache input. Verify cache hit rates in your workflow logs. Use branch protection rules Without branch protection, developers can push directly to main, bypassing the pipeline entirely. Protection rules enforce that all changes go through CI before merging. Require status checks to pass before merging. Require pull request reviews. Disable force-pushes to protected branches. Enable in GitHub repository settings. Monitor pipeline metrics You cannot improve what you do not measure. Tracking pipeline success rates, duration, and flakiness over time reveals trends that inform optimisation decisions. Use GitHub Actions built-in analytics or third-party tools like Datadog CI Visibility. Track mean pipeline duration, failure rate, and flaky test frequency weekly. Version your workflows Pinning action versions to specific tags (e.g., @v4) prevents breaking changes from upstream actions from unexpectedly breaking your pipeline overnight. Always use versioned tags like actions/checkout@v4 rather than @main. Use Dependabot to automate action version updates with pull requests you can review. Use reusable workflows Duplicating workflow logic across multiple repositories creates maintenance burden. When a best practice changes, you must update every copy individually. Define shared workflows in a central repository using workflow_call triggers. Reference them from other repositories with uses: org/repo/.github/workflows/ci.yml@v1. Implement rollback strategy Deployments will occasionally fail in production despite passing all pipeline stages. Without a rollback strategy, recovery requires a new commit, a full pipeline run, and redeployment — adding 10-30 minutes to an incident. Keep previous deployment artefacts available. Use blue-green or canary deployments. Create a manual workflow dispatch that redeploys the last known good version within minutes.
Measure What Matters: The DORA Metrics
The four DORA metrics are the industry standard for measuring software delivery performance. Deployment frequency measures how often your team ships to production. Lead time for changes measures the time from code commit to production deployment. Change failure rate measures the percentage of deployments that cause a failure in production. Mean time to recovery (MTTR) measures how quickly your team restores service after an incident. Elite teams deploy on demand (multiple times per day), have lead times under one hour, change failure rates below 5%, and recover from incidents in under one hour. Track these metrics monthly and use them to guide your CI/CD improvement efforts.
Master DevOps Engineering
Ready to build production-grade CI/CD pipelines and automate your entire software delivery lifecycle? Our accredited DevOps course covers GitHub Actions, infrastructure as code, containerisation, monitoring, and the practices that power high-performing engineering teams. Turn theory into hands-on skills employers are actively hiring for.
Explore Our DevOps Course