paul/gilde-ci-cd
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Merge branch 'feat/tbe_improvments' into 'master'

Browse Source 
Feat/tbe improvments

See merge request gilden/ci/gitlab-ci-templates!95
This commit is contained in:
Marcel Feix 2022-07-29 15:22:49 +00:00
commit 239118ec2e
2 changed files with 60 additions and 36 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

56
documentation/chapters/templates/IaC/docker-compose.adoc
View File

@ -3,38 +3,38 @@
.Provided Templates: .Provided Templates:
|=== |===
|Name |Description |Name |Description
|.docker-compose-deploy | Executes Docker Compose against a remote maschine. |.docker-compose-deploy | Executes docker compose against a remote machine.
|.docker-compose-validate | Validates the syntactic correctness of your compose files. Still can have execution Problems. |.docker-compose-validate | Validates the syntactic correctness of your compose files. Still can have execution problems.
|=== |===
Copy files with scp onto from your Repo to the Remote system. Copy files with scp from your repository onto the remote system.
Then executes Docker-compose over a SSH connection. Then executes `docker compose` over an SSH connection.
For this SSH connection you have to Provide a SSH Key under a CI-CD-Variable with the name $SSH_PRIVATE_KEY. For this SSH connection you have to provide an SSH key under a CI-CD-Variable with the name $SSH_PRIVATE_KEY.
If you already have a different $SSH_PRIVATE_KEY you can overwrite the Variable SSH_PRIVATE_KEY_DEPLOY. If you already have a different $SSH_PRIVATE_KEY you can overwrite the variable SSH_PRIVATE_KEY_DEPLOY.
To Parametrise the Docker-Compose File you can use a .env File. To parametrise the docker-compose file you can use a .env file.
The .env File should not contain any Secrets. The .env file should not contain any secrets.
For this reason you can create a .env file dynamically. For this reason you can create a .env file dynamically.
To generate a .env file you can provide a script under the path ENV_SETUP_FILE. To generate a .env file you can provide a script under the path ENV_SETUP_FILE.
.Beispiel für ein setup_env.sh .Example for a setup_env.sh
---- ----
#!/bin/sh #!/bin/sh
#Prints all Enviroments into the .env File. # Prints all enviroments into the .env file.
printenv >> .env printenv >> .env
#to set a diffrent Project Prefix use this ENV # to set a different project prefix use this ENV
echo COMPOSE_PROJECT_NAME=project-prefix >>.env echo COMPOSE_PROJECT_NAME=project-prefix >>.env
# Secrets # Secrets
#Some Secret is a Variable within the gitlab CI Variable # $SOME_PASSWORD is a variable within the gitlab CI variable
echo SOME_PASSWORD="$SOME_PASSWORD" >>.env echo SOME_PASSWORD="$SOME_PASSWORD" >>.env
---- ----
== SSH Problem workaround == SSH Problem workaround
Sometimes there is a problem with some SSH servers and Docker. Sometimes there is a problem with some SSH servers and docker.
We have a workaround for this, please override your script with the following: We have a workaround for this, please override your script with the following:
---- ----
@ -59,43 +59,51 @@ We have a workaround for this, please override your script with the following:
| 22 | 22
| SSH_USER | SSH_USER
| The user that is used to login into the remote maschine. | The user that is used to log into the remote maschine.
| - | -
| DOCKER_REGISTRY | DOCKER_REGISTRY
| The Container Registry that will be used to pull images. | The container registry that will be used to pull images.
| "$CI_REGISTRY" | $CI_REGISTRY
| DOCKER_REGISTRY_PASS | DOCKER_REGISTRY_PASS
| The passwort to the docker registry that should be used. | The passwort to the docker registry that should be used.
| "$CI_REGISTRY_PASSWORD" | $CI_REGISTRY_PASSWORD
| DOCKER_REGISTRY_USER | DOCKER_REGISTRY_USER
| The user to the docker registry that should be used. | The user to the docker registry that should be used.
| "$CI_REGISTRY_USER" | $CI_REGISTRY_USER
| ENV_SETUP_FILE | ENV_SETUP_FILE
| If you want to use a .env file, this skript can be used to create one. | If you want to use a .env file, this script can be used to create one.
Secrets should not be stored within your repository. Use the CI-CD Variables in the setting page. Secrets should not be stored within your repository. Use the CI-CD variables in the gitlab setting page.
| "setup_env.sh" | setup_env.sh
| SCP_DESTINATION | SCP_DESTINATION
| Target path for scp to push files to. | Target path for scp to push files to.
| - | -
| SCP_SOURCE | SCP_SOURCE
| Path you want to copy to the remote System. | Path you want to copy to the remote system.
| $CI_PROJECT_DIR | $CI_PROJECT_DIR
| COMPOSE_EXTRA_ARGS | COMPOSE_EXTRA_ARGS
| Additional arguments you want to send to Docker Compose. | Additional arguments you want to send to docker compose.
For example multiple `-f compose.file.extra` definitions. For example multiple `-f compose.file.extra` definitions.
| - | -
| COMPOSE_UP_EXTRA_ARGS | COMPOSE_UP_EXTRA_ARGS
| Additional arguments you want to send to Docker Compose up. | Additional arguments you want to send to docker compose up.
For example multiple `--no-deps --no-build` definitions. For example multiple `--no-deps --no-build` definitions.
| - | -
| DOCKER_COMPOSE_VERSION
| Version of `docker compose`.
| v2.7.0
| DOCKER_COMPOSE
| Local path to `docker compose`. If this file does not exist, the script will download it from github.
| $CI_PROJECT_DIR/docker-compose
|=== |===
:template: IaC/docker-compose.gitlab-ci.yml :template: IaC/docker-compose.gitlab-ci.yml

38
templates/IaC/docker-compose.gitlab-ci.yml
View File

@ -3,6 +3,7 @@
stage: deploy stage: deploy
variables: variables:
DOCKER_COMPOSE_VERSION: "v2.7.0" DOCKER_COMPOSE_VERSION: "v2.7.0"
DOCKER_COMPOSE: "$CI_PROJECT_DIR/docker-compose"
SCP_DESTINATION: "" SCP_DESTINATION: ""
SCP_SOURCE: $CI_PROJECT_DIR SCP_SOURCE: $CI_PROJECT_DIR
COMPOSE_EXTRA_ARGS: "" COMPOSE_EXTRA_ARGS: ""
@ -17,6 +18,10 @@
SSH_PRIVATE_KEY_DEPLOY: $SSH_PRIVATE_KEY SSH_PRIVATE_KEY_DEPLOY: $SSH_PRIVATE_KEY
rules: rules:
- when: manual - when: manual
cache:
key: "docker-compose-$DOCKER_COMPOSE_VERSION"
paths:
- $CI_PROJECT_DIR/docker-compose
# https://docs.gitlab.com/ee/ci/ssh_keys/ # https://docs.gitlab.com/ee/ci/ssh_keys/
before_script: before_script:
- if [ -f "$ENV_SETUP_FILE" ]; then chmod +x ./$ENV_SETUP_FILE; ./$ENV_SETUP_FILE; fi - if [ -f "$ENV_SETUP_FILE" ]; then chmod +x ./$ENV_SETUP_FILE; ./$ENV_SETUP_FILE; fi
@ -27,35 +32,46 @@
- chmod 700 ~/.ssh - chmod 700 ~/.ssh
# Add keys to known hosts for Docker Compose # Add keys to known hosts for Docker Compose
- ssh-keyscan $SSH_URL >> ~/.ssh/known_hosts - ssh-keyscan $SSH_URL >> ~/.ssh/known_hosts
#Install docker Compose, from release directly # If missing, install docker-compose from release directly
- wget https://github.com/docker/compose/releases/download/$DOCKER_COMPOSE_VERSION/docker-compose-linux-x86_64 -O /usr/local/bin/docker-compose - >
if [[ ! -f "$DOCKER_COMPOSE" ]]; then
wget https://github.com/docker/compose/releases/download/$DOCKER_COMPOSE_VERSION/docker-compose-linux-x86_64 -O $DOCKER_COMPOSE
fi
- chmod +x /usr/local/bin/docker-compose - chmod +x /usr/local/bin/docker-compose
script: script:
- export DOCKER_HOST=ssh://$SSH_USER@$SSH_URL:$SSH_PORT - export DOCKER_HOST=ssh://$SSH_USER@$SSH_URL:$SSH_PORT
- if [[ -n "$SCP_SOURCE" && -n "$SCP_DESTINATION" ]]; then scp -rp $SCP_SOURCE $SSH_USER@$SSH_URL:$SCP_DESTINATION; fi - if [[ -n "$SCP_SOURCE" && -n "$SCP_DESTINATION" ]]; then scp -rp $SCP_SOURCE $SSH_USER@$SSH_URL:$SCP_DESTINATION; fi
- if [[ -n "$DOCKER_REGISTRY" ]]; then echo "$DOCKER_REGISTRY_PASS" | docker login $DOCKER_REGISTRY --username $DOCKER_REGISTRY_USER --password-stdin; fi - if [[ -n "$DOCKER_REGISTRY" ]]; then echo "$DOCKER_REGISTRY_PASS" | docker login $DOCKER_REGISTRY --username $DOCKER_REGISTRY_USER --password-stdin; fi
- docker-compose $COMPOSE_EXTRA_ARGS pull - $DOCKER_COMPOSE $COMPOSE_EXTRA_ARGS pull
- docker-compose $COMPOSE_EXTRA_ARGS up -d --remove-orphans $COMPOSE_UP_EXTRA_ARGS - $DOCKER_COMPOSE $COMPOSE_EXTRA_ARGS up -d --remove-orphans $COMPOSE_UP_EXTRA_ARGS
.broken-ssh-connection-workaround: .broken-ssh-connection-workaround:
script: script:
- export DOCKER_HOST=unix:///tmp/docker.sock - export DOCKER_HOST=unix:///tmp/docker.sock
- if [[ -n "$SCP_SOURCE" && -n "$SCP_DESTINATION" ]]; then scp -rp $SCP_SOURCE $SSH_USER@$SSH_URL:$SCP_DESTINATION; fi - if [[ -n "$SCP_SOURCE" && -n "$SCP_DESTINATION" ]]; then scp -rp $SCP_SOURCE $SSH_USER@$SSH_URL:$SCP_DESTINATION; fi
# Tunnel Docker socket from remote Maschine to local Maschine. This should be more stable than the current SSH Implementation of Docker # Tunnel Docker socket from remote machine to local machine. This should be more stable than the current SSH implementation of Docker
- rm -f /tmp/docker.sock - rm -f /tmp/docker.sock
- ssh -M -S docker-ctrl-socket -fnNT -L /tmp/docker.sock:/var/run/docker.sock "$SSH_USER@$SSH_URL" -p $SSH_PORT -o ControlPersist=no - ssh -M -S docker-ctrl-socket -fnNT -L /tmp/docker.sock:/var/run/docker.sock "$SSH_USER@$SSH_URL" -p $SSH_PORT -o ControlPersist=no
- if [[ -n "$DOCKER_REGISTRY" ]]; then echo "$DOCKER_REGISTRY_PASS" | docker login $DOCKER_REGISTRY --username $DOCKER_REGISTRY_USER --password-stdin; fi - if [[ -n "$DOCKER_REGISTRY" ]]; then echo "$DOCKER_REGISTRY_PASS" | docker login $DOCKER_REGISTRY --username $DOCKER_REGISTRY_USER --password-stdin; fi
- docker-compose $COMPOSE_EXTRA_ARGS pull - $DOCKER_COMPOSE $COMPOSE_EXTRA_ARGS pull
- docker-compose $COMPOSE_EXTRA_ARGS up -d --remove-orphans $COMPOSE_UP_EXTRA_ARGS - $DOCKER_COMPOSE $COMPOSE_EXTRA_ARGS up -d --remove-orphans $COMPOSE_UP_EXTRA_ARGS
.docker-compose-validate: .docker-compose-validate:
image: docker:20.10.16-alpine3.15 image: docker:20.10.16-alpine3.15
stage: verify stage: verify
variables: variables:
DOCKER_COMPOSE_VERSION: 2.7.0 DOCKER_COMPOSE_VERSION: "v2.7.0"
DOCKER_COMPOSE: "$CI_PROJECT_DIR/docker-compose"
cache:
key: "docker-compose-$DOCKER_COMPOSE_VERSION"
paths:
- $CI_PROJECT_DIR/docker-compose
before_script: before_script:
#Install docker Compose, from release directly # If missing, install docker-compose from release directly
- wget https://github.com/docker/compose/releases/download/$DOCKER_COMPOSE_VERSION/docker-compose-linux-x86_64 -O /usr/local/bin/docker-compose - >
if [[ ! -f "$DOCKER_COMPOSE" ]]; then
wget https://github.com/docker/compose/releases/download/$DOCKER_COMPOSE_VERSION/docker-compose-linux-x86_64 -O $DOCKER_COMPOSE
fi
- chmod +x /usr/local/bin/docker-compose - chmod +x /usr/local/bin/docker-compose
script: script:
- docker-compose $COMPOSE_EXTRA_ARGS config - $DOCKER_COMPOSE $COMPOSE_EXTRA_ARGS config --quiet