GitOps Deployment Workflow Guide

Overview

This guide explains our GitOps-based deployment workflow using Docker Swarm, Portainer, and GitHub Actions. The workflow automatically builds, pushes, and deploys applications across multiple environments (dev, prod, infra, docs, obsv) using infrastructure as code principles.

Architecture

Our deployment system consists of:

• Application Repositories: Contain source code and CI/CD workflows
• GitOps Repository: Contains deployment configurations organized by profiles
• Docker Swarm: Container orchestration platform
• Portainer: Management UI for Docker Swarm
• Ansible: Configuration management for applying deployments

Deployment Profiles

We maintain several deployment profiles:

• dev: Development applications
• prod: Production applications
• infra: Infrastructure components
• docs: Documentation services
• obsv: Monitoring and observability tools

Adding a New Application

1. Create Application Structure

Create a new folder in the appropriate profile:

dev/[project_name]/
├── docker-compose.yml
└── vars.yml

2. Configure Docker Compose

Convert your Docker Compose file to Docker Swarm format with these common networks:

networks:
  swarm_traefik_network:
    external: true
  pg_cluster_network:
    external: true
  redis_cluster_network:
    external: true

# Common deploy configuration
x-common-deploy: &common-deploy
  replicas: 1
  restart_policy:
    condition: on-failure
    delay: 5s
    max_attempts: 3

# Common networks
x-common-networks: &common-networks
  - swarm_traefik_network
  - pg_cluster_network
  - redis_cluster_network

services:
  your-service:
    image: ${YOUR_IMAGE}
    deploy: *common-deploy
    networks: *common-networks
    labels:
      # Use traefik.swarm labels (not traefik.docker)
      - "traefik.enable=true"
      - "traefik.http.routers.your-service.rule=Host(`your-service.example.com`)"

3. Configure Variables

Extract environment variables to vars.yml:

# vars.yml
SERVICE_IMAGE: "{{ lookup('env', 'PROJECT_NAME_IMAGE') }}"
SERVICE_PORT: "{{ lookup('env', 'PROJECT_NAME_PORT') }}"

Use placeholders in docker-compose.yml:

services:
  your-service:
    image: ${SERVICE_IMAGE}
    environment:
      - PORT=${SERVICE_PORT}

4. Update Environment Files

Add variables to .env.dev.example:

# ============================
# Project Name
# ============================
PROJECT_NAME_IMAGE=
PROJECT_NAME_PORT=

Set actual values in .env.dev.

5. Update Deployment Playbook

Add your service to deploy-dev.yml with appropriate tags.

CI/CD Workflow

GitHub Actions Template

Use this template in your application repository as .github/workflows/build-and-deploy.yml:

name: Image Build and Push

on:
  workflow_dispatch:
  push:
    branches:
      - "main"
      - "dev"
      - "v*" # version tags
      - "feature/*"
      - "feat/*"

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: skyconnect-inc/skypesa-api
  BUILDKIT: 1
  GITOPS_REPO: Skyconnect-Inc/portainer-gitops
  GITOPS_BRANCH: main
  GITOPS_PATH: dev/skypesa/docker-compose.yml

jobs:

  release-docker-image:
    name: build and push oci image
    runs-on: ubuntu-latest
    permissions:
      contents: write
      packages: write
      actions: write

    steps:

      - name: checkout
        uses: actions/checkout@v3

      - name: Set up Docker Builds
        uses: docker/setup-buildx-action@v2

      - name: Login to Container registry
        uses: docker/login-action@v2
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Extract metadata (tags, labels) for Docker
        id: meta
        uses: docker/metadata-action@69f6fc9d46f2f8bf0d5491e4aabe0bb8c6a4678a
        with:
          images: |
            ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
          tags: |
            type=ref,event=branch
            type=ref,event=tag
            type=ref,event=pr
            type=sha
          flavor: |
            latest=auto
            prefix=
            suffix=

      - name: Build and push hash tagged image
        id: build
        uses: docker/build-push-action@v2
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:cache
          cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:cache

  update-gitops:
    needs: release-docker-image
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main' && github.event_name == 'push'

    steps:
    - name: Extract short SHA
      id: sha
      run: echo "short_sha=sha-${GITHUB_SHA::7}" >> $GITHUB_OUTPUT

    - name: Checkout GitOps repository
      uses: actions/checkout@v4
      with:
        repository: ${{ env.GITOPS_REPO }}
        token: ${{ secrets.GITOPS_TOKEN }}
        path: gitops

    - name: Update docker-compose.yml
      run: |
        cd gitops

        # Update the image tag in docker-compose.yml
        sed -i "s|${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:.*|${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.sha.outputs.short_sha }}|g" ${{ env.GITOPS_PATH }}

        # Verify the change was made
        echo "Updated ${{ env.GITOPS_PATH }} content:"
        cat ${{ env.GITOPS_PATH }}

    - name: Commit and push changes
      run: |
        cd gitops
        git config --local user.email "action@github.com"
        git config --local user.name "GitHub Action"

        git add ${{ env.GITOPS_PATH }}

        # Check if there are changes to commit
        if git diff --staged --quiet; then
          echo "No changes to commit"
        else
          git commit -m "Update ${{ env.IMAGE_NAME }} image to ${{ steps.sha.outputs.short_sha }}"
          git push
          echo "Changes committed and pushed successfully"
        fi

Required Secrets

Add these secrets to your application repository:

• GITOPS_TOKEN: Personal access token with write access to the GitOps repository

Environment Variables to Customize

Update these variables in the workflow file:

• IMAGE_NAME: Your container image name (e.g., your-org/your-app)
• GITOPS_REPO: GitOps repository (e.g., your-org/portainer-gitops)
• GITOPS_PATH: Path to your docker-compose.yml in GitOps repo

Workflow Process

1. Code Push: Developer pushes code to application repository
2. Build: GitHub Actions builds and pushes Docker image with SHA tag
3. GitOps Update: Workflow updates the GitOps repository with new image tag
4. Deployment: Ansible playbooks or Portainer detect changes and redeploy services

Benefits

• GitOps Principles: Infrastructure and deployments are version controlled
• Automatic Deployments: No manual intervention required for deployments
• Rollback Capability: Easy rollbacks using Git history
• Environment Consistency: Same deployment process across all environments
• Audit Trail: Full visibility of what was deployed when and by whom

Best Practices

1. Tagging Strategy: Use short SHA tags for traceability
2. Environment Isolation: Separate profiles prevent cross-environment issues
3. Security: Use least-privilege tokens and secrets management
4. Monitoring: Implement health checks and monitoring for deployed services
5. Testing: Test deployment configurations in dev before promoting to prod

Troubleshooting

• Check GitHub Actions logs for build failures
• Verify GitOps repository permissions
• Ensure Docker Swarm networks exist before deployment
• Validate environment variables in .env files
• Monitor Portainer for service deployment status

This workflow provides a robust, scalable deployment pipeline that follows GitOps best practices while maintaining simplicity for developers.