[转]图文详解k8s自动化持续集成之GitLab CI/CD

Windows里面使用Debian命令行工具完成

和Docker网络相关的命令

查看某一个容器的网络

docker inspect 容器ID

查看docker当前网络

docker network ls

首先创建一个网络（下面通过docker-compose的yml文件创建，这里只做命令的了解）

docker network create gitlab-network

将现有的容器连到创建的网络中（每个容器都要连到这个网络里）

docker network connect gitlab-network gitlab-server   #容器名

docker network connect gitlab-network gitlab-runner

查看网络内的容器信息

docker network inspect gitlab-network

也可以在运行时指定镜像

docker run --network gitlab-network 镜像名称

移除指定的网络

docker network rm gitlab-network

先删掉下面要用的bridge名称

使用Docker Compose安装GitLab-ce和GitLab-runner

Docker Compose で GitLab + GitLab Runner の環境を整える(Window10) 重要参考

Win10下使用Docker搭建Gitlab CI自动构建平台

docker rm & docker rmi & docker prune 的差异

docker rm ：删除一个或多个容器
docker rmi : 删除一个或多个镜像
docker prune：用来删除不再使用的 docker 对象

docker+gitlab+gitlab-runner部署(CentOs7)重要参考

Docker Compose yml for Gitlab and Gitlab Runner

SMTP settings(备用参考)

#vim /etc/gitlab/gitlab.rb

gitlab_rails['smtp_enable'] = true

gitlab_rails['smtp_address'] = "smtp.exmail.qq.com"

gitlab_rails['smtp_port'] = 465

gitlab_rails['smtp_user_name'] = "developers@aaa.net"

gitlab_rails['smtp_password'] = "Del43@\#$@1"

gitlab_rails['smtp_domain'] = "exmail.qq.com"

gitlab_rails['smtp_authentication'] = "login"

gitlab_rails['smtp_enable_starttls_auto'] = true

gitlab_rails['smtp_tls'] = true

gitlab_rails['gitlab_email_from'] = "developers@aaa.net"

测试邮件

#gitlab-ctl reconfigure

#gitlab-rails console

#Notify.test_email('315360007@qq.com', 'Message Subject', 'Message Body').deliver_now

配置本地windows的host文件：

127.0.0.1 gitlab-server
127.0.0.1 nexus3

浏览器访问：http://gitlab-server:9080/ 或者 http://ip:9080/
用户名：root，密码：自己设置

用root用户登录

创建测试项目：mvcdockerrunner1

添加SSH密钥

本地windows需要在当前登录用户的目录里面找到.ssh目录下面的****.pub文件添加公钥，同时设置config文件

Host gitlab-server

HostName gitlab-server

User git

PreferredAuthentications publickey

IdentityFile ~/.ssh/315360007

Port 23

.ssh/config

克隆代码

手动设置激活的Runner(很重要)

1、注册gitlab-runner（按照提示一步一步做），熟悉了直接用下面第二种

docker exec -it gitlab-runner gitlab-runner register

2. 我们会输入 http://gitlab.local.net:9080 或 http://ip:port 也就是我们安装在本地的GitLab

3. Please enter the gitlab-ci token for this runner 要求输入 gitlab-ci token

在项目的管理区域->runners中可以找到(这里注册的是share类型runner)

4. 输入描述,如：gitlab-runner

5. 输入tag（留空也可以，之后可以进行编辑）

6. 选择当遇到没有打标签的提交时是否会执行，我们选 true

7. 是否锁定此runner 到当前项目，我们选 false

8. 选一个执行者 executor

这一步比较重要 (ssh, docker+machine, docker-ssh+machine, kubernetes, docker, parallels, virtualbox, docker-ssh, shell)

我们选docker

9. 选择默认使用的镜像： docker:stable

2、注册gitlab-runner（直接传参数一次完成）

# 手动设置激活的Runner

docker exec -it gitlab-runner gitlab-runner register \

  --non-interactive \

  --executor "docker" \

  --docker-image docker:stable \

  --url "http://gitlab-server:9080/" \

  --registration-token "K6BiurzDBuzx23a-mV-f" \

  --description "gitlab-runner" \

  --tag-list "docker,company" \

  --run-untagged="true" \

  --locked="false" \

  --docker-privileged="true" \

  --docker-extra-hosts "gitlab-server:172.20.0.1" \

  --docker-extra-hosts "nexus3:172.20.0.1"

#解释很重要

#"docker:stable"这里之所以这样是因为基于这个镜像它包含了docker等工具，可以在gitlab-runner执行的.gitlab-ci.yml中有docker指令时

#而无需再安装docker 参考地址：https://docs.gitlab.com/ee/ci/docker/using_docker_build.html

#privileged=true #使用docker-in-docker时通常为true

#extra-hosts     #设置gitlab-server的网关地址

进入 gitlab-runner的容器里

 docker exec -it runner-container-id /bin/bash

查看/etc/gitlab-runner/config.toml文件

cat /etc/gitlab-runner/config.toml

concurrent = 1

check_interval = 0

[session_server]

  session_timeout = 1800

[[runners]]

  name = "gitlab-runner"

  url = "http://gitlab-server:9080/"

  token = "mWpRt6ry7HimmAyt86T1"

  executor = "docker"

  [runners.custom_build_dir]

  [runners.cache]

    [runners.cache.s3]

    [runners.cache.gcs]

    [runners.cache.azure]

  [runners.docker]

    tls_verify = false

    image = "docker:stable"

    privileged = true

    disable_entrypoint_overwrite = false

    oom_kill_disable = false

    disable_cache = false

    volumes = ["/cache"]

    extra_hosts = ["gitlab-server:172.20.0.1"]

    shm_size = 0

/etc/gitlab-runner/config.toml

五、提交项目代码完成CI/CD

Gitlab CI/CD 打包过程报 “ERROR: Uploading artifacts to coordinator... too large” 如何解决

Uploading artifacts for successful job

00:13

Uploading artifacts...

mevdockerrunner1.tar: found 1 matching files and directories

ERROR: Uploading artifacts as "archive" to coordinator... too large archive  id=34 responseStatus=413 Request Entity Too Large status=413 token=fhcnwRyE

FATAL: too large

GitLab管理中心里设置CI/CD持续集成和部署里最大产物大小

设置和使用Gitlab-ci变量连接Kubernetes：

 1 .kube_deploy:

 2   image: boxboat/kubectl:1.17.4

 3   variables:

 4     HELM_TOKEN: ${HELM_USER_TOKEN}

 5     K8S_API_SERVER: ${KUBE_API_ADDRESS}

 6     K8S_API_SERVER_CERT: ${KUBE_API_CA}

 7   before_script:

 8     - mkdir -p /home/alpine/.kube

 9     - cp .ci/kube-config-template /home/alpine/.kube/config

10     - sed -i "s/{{ HELM_USER_TOKEN }}/${HELM_TOKEN}/g" /home/alpine/.kube/config

11     - sed -i "s/{{ KUBE_API_ADDRESS }}/${K8S_API_SERVER}/g" /home/alpine/.kube/config

12     - sed -i "s/{{ KUBE_API_CA }}/${K8S_API_SERVER_CERT}/g" /home/alpine/.kube/config

.gitlab-ci.yml

设置Kubernetes的config文件，kube-config-template文件内容：

 1 apiVersion: v1

 2 clusters:

 3 - cluster:

 4     certificate-authority-data: {{ KUBE_API_CA }}

 5     server: https://{{ KUBE_API_ADDRESS }}

 6   name: helm

 7 contexts:

 8 - context:

 9     cluster: helm

10     namespace: meshop-dev

11     user: helm

12   name: helm

13 current-context: "helm"

14 kind: Config

15 preferences: {}

16 users:

17 - name: helm

18   user:

19     token: {{ HELM_USER_TOKEN }}

.ci/kube-config-template

获取KUBE_API_CA和HELM_USER_TOKEN的值

#kubectl get sa -A   #查看所有security account

#kubectl get sa/helm -n kube-system -o yaml #查看在某一个命名空间下的某一个security account=helm的yaml，得到secrets的name

#kubectl get secrets/helm-token-wstgs  -o yaml -n kube-system #查看在某一个命名空间下的某一个secrets的name的yaml，得到token值和ca.crt值

使用变量

设置默认分支：

设置仓库不允许推送代码，只能自己创建分支提交，在服务器上合并分支

git流水线克隆代码

流水线触发器的设置，添加一个token

设置sourceTree自定义操作，设置token=上面的值

1 curl -X POST -F token=f21cbb2d6f8c10cad915a380c49b99 -F 'ref=dev' -F 'variables[MESHOP_BUILD_ONLY]=sso' https://git.runshopstore.com/api/v4/projects/2/trigger/pipeline

sso_restore.sh

设置ci里面使用触发器

1 workflow:

2   rules:

3     - if: $MESHOP_BUILD_ONLY != null

4       when: always

5     - if: $CI_PIPELINE_SOURCE =~ /^trigger|pipeline|web|api$/

6       when: always

.gitlab-ci.yml

合并请求必须保证编译通过：同时删除合并请求的分支

设置关闭仓库，测试可以使用的项目：

设置标记：

规范化git commit信息

团队敏捷实践 —— 使用 semantic-release 自动管理发布版本

单个项目最简单的自动化脚本（Config项目）

stages:

  - restore

  - compile

  - version

  - build

  - deploy

.load_environment_variables: &load_environment_variables

  - echo "init environment variables."

  - for environment in $(ls -d -1 .ci/environments/*.sh);

    do source $environment;

    done

default:

  image: docker.tidebuy.net/dotnet/sdk:5.0

  tags:

    - docker

    - company

  before_script:

    - *load_environment_variables

variables:

  # docker in docker

  DOCKER_DRIVER: "overlay2"

  DOCKER_HOST: tcp://localhost:2375

  # cache

  NUGET_PACKAGES_DIRECTORY: ".nuget"

  OBJECTS_DIRECTORY: "obj"

  SOURCE_CODE_PATH: "*/*/"

.nuget_cache:

  cache:

    key: "${CI_PROJECT_NAME}-${CI_COMMIT_REF_SLUG}"

    paths:

      - "$SOURCE_CODE_PATH$OBJECTS_DIRECTORY/project.assets.json"

      - "$SOURCE_CODE_PATH$OBJECTS_DIRECTORY/*.csproj.nuget.*"

      - "$NUGET_PACKAGES_DIRECTORY"

restore:

  stage: restore

  extends: .nuget_cache

  cache:

    policy: pull-push

  script:

    - echo "dotnet restore..."

    - dotnet restore

      $CI_PROJECT_DIR/MeShop.Config/MeShop.Config.sln

      --packages $NUGET_PACKAGES_DIRECTORY

      --runtime linux-x64

      --configfile $CI_PROJECT_DIR/.ci/nuget.config

  only:

    refs:

      - branches

      - merge_requests

compile:

  stage: compile

  before_script:

    - *load_environment_variables

  extends:

    - .nuget_cache

  cache:

    policy: pull

  script:

    - echo "compile project."

    - dotnet publish MeShop.Config/MeShop.View.Config/MeShop.View.Config.csproj --no-restore

      --runtime linux-x64 -c Release -o ${CI_PROJECT_DIR}/publish

  artifacts:

    name: "${CI_JOB_NAME}-${CI_COMMIT_REF_NAME}"

    paths:

      - publish/

    expire_in: 1 days

  only:

    refs:

      - branches

      - merge_requests

version:

  stage: version

  image: meshop/semantic-release:17.1.1

  script:

    - if [ $CI_COMMIT_REF_NAME == 'alpha' -o $CI_COMMIT_REF_NAME  == 'master' ];

      then

      semantic-release;

      fi

    - if [ ! -f ./.version ];

      then

      echo "VERSION=${MESHOP_BUILD_VERSION:-$(date +%Y%m%d%H%M%S)}">.version;

      fi

    - cat .version

  artifacts:

    paths:

      - .version

    expire_in: 1 days

  needs:

    - job: compile

      artifacts: false

  rules:

    - if: "$CI_PIPELINE_SOURCE == 'merge_request_event'"

      when: never

    - if: "$CI_COMMIT_REF_NAME == 'dev'"

      when: on_success

    - if: "$CI_COMMIT_REF_NAME == 'alpha'"

      when: on_success

    - if: "$CI_COMMIT_REF_NAME  == 'master'"

      when: on_success

    - if: "$CI_COMMIT_REF_NAME  =~ /.*-bug-fix/"

      when: on_success

build:

  stage: build

  image: "docker:stable"

  services:

    - docker:stable-dind

  variables:

    DOCKER_TLS_CERTDIR: ""

  script:

    - echo "Logging to GitLab Container Registry with CI credentials..."

    - echo "$MESHOP_BUILD_REGISTRY_PASSWORD" | docker login -u "$MESHOP_BUILD_REGISTRY_USER" --password-stdin "$MESHOP_BUILD_REGISTRY"

    - source ./.version

    - image_version="${VERSION}"

    - image_name="${MESHOP_BUILD_REGISTRY}/meshop/shop/config"

    - docker build -f .ci/Dockerfile

      --tag "$image_name:$image_version"

      .

    - docker push "$image_name:$image_version"

    - if [ $MESHOP_BUILD_LATEST = 'true' ];

      then

      docker tag $image_name:$image_version $image_name:latest &&

      docker push "$image_name:latest" ;

      fi

  needs:

    - job: version

      artifacts: true

    - job: compile

      artifacts: true

  rules:

    - if: "$CI_PIPELINE_SOURCE  == 'merge_request_event'"

      when: never

    - if: "$CI_COMMIT_REF_NAME == 'dev'"

      when: on_success

    - if: "$CI_COMMIT_REF_NAME == 'alpha'"

      when: on_success

    - if: "$CI_COMMIT_REF_NAME  == 'master'"

      when: on_success

    - if: "$CI_COMMIT_REF_NAME  =~ /.*-bug-fix/"

      when: on_success

deploy:

  stage: deploy

  image: boxboat/kubectl:1.17.4

  script:

    - mkdir -p /home/alpine/.kube

    - cp .ci/kube-config-template /home/alpine/.kube/config

    - sed -i "s/{{ HELM_USER_TOKEN }}/${HELM_USER_TOKEN}/g" /home/alpine/.kube/config

    - sed -i "s/{{ KUBE_API_ADDRESS }}/${KUBE_API_ADDRESS}/g" /home/alpine/.kube/config

    - sed -i "s/{{ KUBE_API_CA }}/${KUBE_API_CA}/g" /home/alpine/.kube/config

    - kubectl rollout restart deployment/meshop-config

  rules:

    - if: "$CI_PIPELINE_SOURCE  == 'merge_request_event'"

      when: never

    - if: "$CI_COMMIT_REF_NAME == 'dev'"

      when: on_success

.gitlab-ci.yml

Docker安装GitLab与Runner（网关），常规设置，自动化用到k8s+token

[转]图文详解k8s自动化持续集成之GitLab CI/CD

Docker Compose で GitLab + GitLab Runner の環境を整える(Window10) 重要参考

Win10下使用Docker搭建Gitlab CI自动构建平台

docker rm & docker rmi & docker prune 的差异

Docker Compose yml for Gitlab and Gitlab Runner

五、提交项目代码完成CI/CD

Gitlab CI/CD 打包过程报 “ERROR: Uploading artifacts to coordinator... too large” 如何解决

规范化git commit信息

团队敏捷实践 —— 使用 semantic-release 自动管理发布版本

Docker安装GitLab与Runner（网关），常规设置，自动化用到k8s+token的相关教程结束。

相关推荐

《最新出炉》系列初窥篇-Python+Playwright自动化测试-2-playwright的API及其他知识

使用openresty替换线上nginx网关之openresty安装细节

【数据库】时区及JDBC的时区设置

20191012——POI设置单元格自动行高（思路）

openpyxl 设置单元格颜色

Aspose.Cells设置单元格格式

unidbgrid 设置单元格颜色

NPOI设置单元格背景色

Docker安装GitLab与Runner（网关），常规设置，自动化用到k8s+token

[转]图文详解k8s自动化持续集成之GitLab CI/CD

Docker Compose で GitLab + GitLab Runner の環境を整える(Window10) 重要参考

Win10下使用Docker搭建Gitlab CI自动构建平台

docker rm & docker rmi & docker prune 的差异

Docker Compose yml for Gitlab and Gitlab Runner

五、提交项目代码完成CI/CD

Gitlab CI/CD 打包过程报 “ERROR: Uploading artifacts to coordinator... too large” 如何解决

规范化git commit信息

团队敏捷实践 —— 使用 semantic-release 自动管理发布版本

Docker安装GitLab与Runner（网关），常规设置，自动化用到k8s+token的相关教程结束。

相关推荐

《最新出炉》系列初窥篇-Python+Playwright自动化测试-2-playwright的API及其他知识

使用openresty替换线上nginx网关之openresty安装细节

【数据库】时区及JDBC的时区设置

20191012——POI设置单元格自动行高（思路）

openpyxl 设置单元格颜色

Aspose.Cells设置单元格格式

unidbgrid 设置 单元格颜色

NPOI设置单元格背景色

unidbgrid 设置单元格颜色