Merge branch 'feat/tbe_gradle_anpassungen' into 'master'

refactor(cache): Löse globalen-Cache aus common-ci-variablen heraus

See merge request gilden/ci/gitlab-ci-templates!90
This commit is contained in:
Marcel Feix 2022-07-22 07:13:26 +00:00
commit 5ddc47a2ed
23 changed files with 55 additions and 62 deletions

View File

@ -16,11 +16,11 @@ The main-pipeline provides infrastructure to create Gitlab-releases and the trig
The child pipeline is meant to create a new git tag with an increased Version. The child pipeline is meant to create a new git tag with an increased Version.
When executed on it also creates a new Release Branch. When executed on it also creates a new Release Branch.
You can easily extend the child pipeline to publish Artefacts. You can easily extend the child pipeline to publish artifacts.
Although this is possible we recommend using tagged pipelines to publish Artefacts. Although this is possible we recommend using tagged pipelines to publish artifacts.
Tagged pipelines have the benefit that your pipelines are linked to a tag. Tagged pipelines have the benefit that your pipelines are linked to a tag.
This makes publishing out of order or hotfixes simpler. This makes publishing out of order or hotfixes simpler.
Since you do not relly on prior CI artefacts. Since you do not relly on prior CI artifacts.
== Overview diagram == Overview diagram

View File

@ -4,15 +4,15 @@
.Provided Templates: .Provided Templates:
|=== |===
|Name |Description |Name |Description
| .container-build-jib-maven | Baut Container mit dem Jib Plugin für Maven | .container-build-jib-maven | Builds the container with the Jib plugin for maven
| .container-build-jib-gradle | Baut Container mit dem Jib Plugin für Gradle | .container-build-jib-gradle | Builds the container with the Jib plugin for gradle
| .container-build-kaniko | Baut Container mit dem Kaniko | .container-build-kaniko | Builds the container with Kaniko
|=== |===
Allows the simple building of Container Images. Allows the simple building of Container Images.
We use Kaniko because it is faster than Docker builds and has less security problems. We use Kaniko because it is faster than Docker builds and has less security problems.
The generated image will be passed as a Tar-Artefact between stages. The generated image will be passed as a tar-artifact between stages.
This allows us to make forder manipulations and scan jobs on one container Image. This allows us to make further manipulations and scan jobs on the container image.
The template <<_container_publish>> allows you to publish the build image to a registry. The template <<_container_publish>> allows you to publish the build image to a registry.
@ -21,7 +21,7 @@ The template <<_container_publish>> allows you to publish the build image to a r
|Name |Description | Default Value |Name |Description | Default Value
|CONTEXT_LOCATION |CONTEXT_LOCATION
| Defines the context of the container build. In case of jib this Path should point to the location of your Gradle/Maven projects. | Defines the context of the container build. In case of jib, this path should point to the location of your Gradle/Maven projects.
| $CI_PROJECT_DIR | $CI_PROJECT_DIR
| EXTRA_ARGS | EXTRA_ARGS
@ -35,7 +35,7 @@ The template <<_container_publish>> allows you to publish the build image to a r
| $CI_PROJECT_DIR/Dockerfile | $CI_PROJECT_DIR/Dockerfile
| REGISTRY_MIRROR | REGISTRY_MIRROR
| Define the container Mirror that should be used | Define the container mirror that should be used
| exxcellent Nexus | exxcellent Nexus
|=== |===

View File

@ -3,7 +3,7 @@
This template resolves the gradle dependencies of your project and builds the project. This template resolves the gradle dependencies of your project and builds the project.
The result will be cached when you include the common CI variables. The result will be cached when you include the common CI variables.
The resulting build files will be published as artefacts. The resulting build files will be published as artifacts.
.Variables .Variables
|=== |===

View File

@ -15,7 +15,7 @@ Use the alias field in the service definition to define the address for the for
|=== |===
|Name |Description | Default Value |Name |Description | Default Value
|GRADLE_CLI_OPTS | Additional options send to gradle | "" |GRADLE_CLI_OPTS | Additional options send to gradle | ""
| PATH_TO_DIFF | Path to the resulting diff is defined in gradle config, this will also be exposed as the artefact of the Job. | "build/db-diff.xml" | PATH_TO_DIFF | Path to the resulting diff is defined in gradle config, this will also be exposed as the artifact of the Job. | "build/db-diff.xml"
|=== |===
:template: java/gradle/gradle-database-diff.gitlab-ci.yml :template: java/gradle/gradle-database-diff.gitlab-ci.yml

View File

@ -4,9 +4,9 @@ endif::[]
= Gradle Database Diff = Gradle Database Diff
These templates starts `gradle test` and saves the results as Artifacts. These templates starts `gradle test` and saves the results as artifacts.
When Junit Reports are generated they will be published as Artefacts within the Pipelines. When Junit Reports are generated they will be published as artifacts within the Pipelines.
You can get a detailed view when clicking on Tests. You can get a detailed view when clicking on Tests.
For more information look into the official https://docs.gitlab.com/ee/ci/unit_test_reports.html[Documentation]. For more information look into the official https://docs.gitlab.com/ee/ci/unit_test_reports.html[Documentation].

View File

@ -9,7 +9,7 @@ Please look at the resulting changelog since it can have errors.
|Name |Description | Default Value |Name |Description | Default Value
| PATH_TO_DIFF | PATH_TO_DIFF
| Path to the resulting diff is defined in gradle config, this will also be exposed as the artefact of the Job. | Path to the resulting diff is defined in gradle config, this will also be exposed as the artifact of the Job.
| "target/db-diff.xml" | "target/db-diff.xml"
|MAVEN_CLI_OPTS | Additional Options that will be passed to Maven | "" |MAVEN_CLI_OPTS | Additional Options that will be passed to Maven | ""

View File

@ -4,9 +4,9 @@ endif::[]
= Maven verify template = Maven verify template
This templates starts `mvn verify` and saves the results as Artifacts. This templates starts `mvn verify` and saves the results as artifacts.
When Junit Reports are generated they will be published as Artefacts within the Pipelines. When Junit Reports are generated they will be published as artifacts within the Pipelines.
You can get a detailed view when clicking on Tests. You can get a detailed view when clicking on Tests.
For more information look into the official https://docs.gitlab.com/ee/ci/unit_test_reports.html[Documentation]. For more information look into the official https://docs.gitlab.com/ee/ci/unit_test_reports.html[Documentation].

View File

@ -52,18 +52,18 @@ include::chapters/templates/IaC/ssh-transfer-data.adoc[leveloffset=2]
== Container Templates == Container Templates
We have optimised building container within our templates a lot. We have optimised building container within our templates a lot.
We found divide building and releasing containers into different jobs to be the fastest way to build container artefacts. We found divide building and releasing containers into different jobs to be the fastest way to build container artifacts.
Because you can build your container while executing other verify stages. Because you can build your container while executing other verify stages.
Publishing containers is a very cheap operation. Publishing containers is a very cheap operation.
The following graph charts shows thru which stages containers will go and which artefakts will be passed between stages. The following graph charts shows through which stages containers will go and which artifacts will be passed between stages.
[mermaid] [mermaid]
.... ....
flowchart LR flowchart LR
subgraph Build Stage subgraph Build Stage
build_src(Source Code Build) -- Needs Source artefacts --> build_c build_src(Source Code Build) -- Needs Source artifacts --> build_c
build_c(Container-Build) build_c(Container-Build)
end end
@ -84,9 +84,9 @@ include::chapters/templates/container/docker-in-docker.adoc[leveloffset=2]
== Releasing Templates == Releasing Templates
The following Templates provide an simpler integration into the https://gitlab.exxcellent.de/gilden/ci/exxcellent-increase-version-tool[Increase-Version-Tool] of the CI-Gilde. The following templates provide a simpler integration into the https://gitlab.exxcellent.de/gilden/ci/exxcellent-increase-version-tool[Increase-Version-Tool] of the CI-Gilde.
We also provide finished Pipelines in our <<_pipelines>> section. We also provide finished pipelines in our <<_pipelines>> section.
We highly recommend using those Pipelines since this is our reference Implementation. We highly recommend using those pipelines since this is our reference implementation.
include::chapters/templates/releasing/push-tag-and-version-template.adoc[leveloffset=3] include::chapters/templates/releasing/push-tag-and-version-template.adoc[leveloffset=3]
include::chapters/templates/releasing/set-version-template.adoc[leveloffset=3] include::chapters/templates/releasing/set-version-template.adoc[leveloffset=3]
@ -115,7 +115,7 @@ include::chapters/templates/java/maven/maven-db-diff-template.adoc[leveloffset=2
== Gradle Templates == Gradle Templates
Our collection of Gradle Templates. Our collection of Gradle Templates.
Currently, we start our Templates from the Gradle Wrapper within your Projekt. Currently, we start our templates from the gradle wrapper within your project.
If you do not have a wrapper you can create on with the command `gradle wrapper`. If you do not have a wrapper you can create on with the command `gradle wrapper`.
Please keep in mind updating the wrapper from time to time. Please keep in mind updating the wrapper from time to time.
We recommend using https://www.exxcellent.de/confluence/x/S4WCC[Renovate] for this task. We recommend using https://www.exxcellent.de/confluence/x/S4WCC[Renovate] for this task.
@ -126,7 +126,7 @@ include::chapters/templates/java/gradle/gradle-db-diff-template.adoc[leveloffset
== Static Analysis == Static Analysis
With the following collection we provide Static code and Artefact analysis tools. With the following collection we provide static code and artifact analysis tools.
include::chapters/templates/static-analysis/repo-cve-scan.adoc[leveloffset=2] include::chapters/templates/static-analysis/repo-cve-scan.adoc[leveloffset=2]
include::chapters/templates/static-analysis/sonar-7.adoc[leveloffset=2] include::chapters/templates/static-analysis/sonar-7.adoc[leveloffset=2]
@ -134,13 +134,13 @@ include::chapters/templates/static-analysis/sonar-8.adoc[leveloffset=2]
== Documentation == Documentation
This collection provides templates to simplify your dokumentation prozess. This collection provides templates to simplify your documentation process.
include::chapters/templates/documentation/documentation.adoc[leveloffset=2] include::chapters/templates/documentation/documentation.adoc[leveloffset=2]
== Licensing == Licensing
This collection provides templates to simplify your licencing prozess. This collection provides templates to simplify your licencing process.
include::chapters/templates/license/license.gitlab-ci.adoc[leveloffset=2] include::chapters/templates/license/license.gitlab-ci.adoc[leveloffset=2]
include::chapters/templates/java/gradle/gradle-license.gitlab-ci.adoc[leveloffset=2] include::chapters/templates/java/gradle/gradle-license.gitlab-ci.adoc[leveloffset=2]

View File

@ -19,8 +19,8 @@ flowchart LR
Jobs within this stage are loading all needed Dependencies. This is done for better cache management of Dependencies. Jobs within this stage are loading all needed Dependencies. This is done for better cache management of Dependencies.
=== Build - build === Build - build
Jobs within this stage, build all needed artefacts that you need in later stages and for releasing. Jobs within this stage, build all needed artifacts that you need in later stages and for releasing.
In case you need to build multiple artefacts execute following jobs as soon as needed artefacts are build. In case you need to build multiple artifacts execute following jobs as soon as needed artifacts are build.
Look into Gitlabs https://docs.gitlab.com/ee/ci/yaml/#needs[Needs Documentation]. Look into Gitlabs https://docs.gitlab.com/ee/ci/yaml/#needs[Needs Documentation].
=== Verify - verify === Verify - verify
@ -32,7 +32,7 @@ Verify and Test as much as you can in these Jobs.
Jobs within this stage build and publish your documentation. Jobs within this stage build and publish your documentation.
=== Release - release === Release - release
Jobs within this stage publish your artefacts or automate creating Tags. Jobs within this stage publish your artifacts or automate creating Tags.
For more information look into <<_pipeline_release>> For more information look into <<_pipeline_release>>
== About Templates and Pipelines == About Templates and Pipelines

View File

@ -0,0 +1,12 @@
cache:
key: "$CI_JOB_NAME"
paths:
# Cache Maven
- .m2/repository
# Cache NPM
#- $FRONTEND_DIR/dist
# Cache Gradle
- .gradle
- '**/build'
- "**/target"
#- "**/node_modules"

View File

@ -1,17 +1,3 @@
cache:
key: "$CI_JOB_NAME"
paths:
# Cache Maven
- .m2/repository
# Cache NPM
#- $FRONTEND_DIR/dist
# Cache Gradle
- .gradle
- '**/build'
- "**/target"
#- "**/node_modules"
variables: variables:
##NPM / Node Settings ##NPM / Node Settings

View File

@ -27,7 +27,7 @@
if [ "$CONTEXT_LOCATION" != "" ]; then if [ "$CONTEXT_LOCATION" != "" ]; then
cd $CONTEXT_LOCATION cd $CONTEXT_LOCATION
fi fi
- ./gradlew $GRADLE_CLI_OPTS jibBuildTar -DskipTests -Djib.outputPaths.tar=$CI_PROJECT_DIR/image.tar $EXTRA_ARGS - ./gradlew --build-cache $GRADLE_OPTS $GRADLE_CLI_OPTS jibBuildTar -DskipTests -Djib.outputPaths.tar=$CI_PROJECT_DIR/image.tar $EXTRA_ARGS
.container-build-kaniko: .container-build-kaniko:
extends: .container-build-common extends: .container-build-common

View File

@ -13,7 +13,7 @@
$(sed -i /^version/s/$GRADLE_VERSION/\'$CI_COMMIT_TAG\'/ $PATH_TO_VERSION_FILE) $(sed -i /^version/s/$GRADLE_VERSION/\'$CI_COMMIT_TAG\'/ $PATH_TO_VERSION_FILE)
fi fi
script: script:
- "./gradlew --build-cache assemble $GRADLE_CLI_BUILD_OPTS $GRADLE_CLI_OPTS $GRADLE_EXTRA_OPTS" - ./gradlew --build-cache $GRADLE_OPTS assemble $GRADLE_CLI_BUILD_OPTS $GRADLE_CLI_OPTS $GRADLE_EXTRA_OPTS
artifacts: artifacts:
expire_in: 2 weeks expire_in: 2 weeks
when: on_success when: on_success

View File

@ -13,7 +13,7 @@ include:
if [ "$BACKEND_DIR" != "" ]; then if [ "$BACKEND_DIR" != "" ]; then
cd $BACKEND_DIR cd $BACKEND_DIR
fi fi
- ./gradlew liquibase:update liquibase:diff $GRADLE_CLI_OPTS $GRADLE_EXTRA_OPTS - ./gradlew --build-cache $GRADLE_OPTS liquibase:update liquibase:diff $GRADLE_CLI_OPTS $GRADLE_EXTRA_OPTS
artifacts: artifacts:
when: on_failure when: on_failure
expire_in: 2 weeks expire_in: 2 weeks

View File

@ -6,7 +6,7 @@
if [ "$BACKEND_DIR" != "" ]; then if [ "$BACKEND_DIR" != "" ]; then
cd $BACKEND_DIR cd $BACKEND_DIR
fi fi
- "./gradlew --build-cache cyclonedxBom $GRADLE_CLI_OPTS $GRADLE_EXTRA_OPTS" - ./gradlew --build-cache $GRADLE_OPTS cyclonedxBom $GRADLE_CLI_OPTS $GRADLE_EXTRA_OPTS
needs: [] needs: []
artifacts: artifacts:
expire_in: 2 weeks expire_in: 2 weeks

View File

@ -6,7 +6,7 @@
if [ "$BACKEND_DIR" != "" ]; then if [ "$BACKEND_DIR" != "" ]; then
cd $BACKEND_DIR cd $BACKEND_DIR
fi fi
- "./gradlew --build-cache test $GRADLE_CLI_OPTS $GRADLE_EXTRA_OPTS" - ./gradlew --build-cache $GRADLE_OPTS test $GRADLE_CLI_OPTS $GRADLE_EXTRA_OPTS
artifacts: artifacts:
expire_in: 2 weeks expire_in: 2 weeks
reports: reports:

View File

@ -3,12 +3,11 @@
stage: build stage: build
variables: variables:
FRONTEND_BUILD_DIR: '**/dist' FRONTEND_BUILD_DIR: '**/dist'
before_script: script:
- > - >
if [ "$FRONTEND_DIR" != "" ]; then if [ "$FRONTEND_DIR" != "" ]; then
cd $FRONTEND_DIR cd $FRONTEND_DIR
fi fi
script:
- npm run ci:build - npm run ci:build
artifacts: artifacts:
expire_in: 2 weeks expire_in: 2 weeks

View File

@ -1,12 +1,11 @@
.npm-dependencies: .npm-dependencies:
image: $NODE_IMAGE image: $NODE_IMAGE
stage: dependencies stage: dependencies
before_script: script:
- > - >
if [ "$FRONTEND_DIR" != "" ]; then if [ "$FRONTEND_DIR" != "" ]; then
cd $FRONTEND_DIR cd $FRONTEND_DIR
fi fi
script:
- npm ci --no-audit - npm ci --no-audit
artifacts: artifacts:
expire_in: 1 days expire_in: 1 days

View File

@ -1,12 +1,11 @@
.npm-e2e: .npm-e2e:
image: $NODE_E2E_IMAGE image: $NODE_E2E_IMAGE
stage: verify stage: verify
before_script: script:
- > - >
if [ "$FRONTEND_DIR" != "" ]; then if [ "$FRONTEND_DIR" != "" ]; then
cd $FRONTEND_DIR cd $FRONTEND_DIR
fi fi
script:
- npm run ci:e2e - npm run ci:e2e
artifacts: artifacts:
expire_in: '7d' expire_in: '7d'

View File

@ -1,10 +1,9 @@
.npm-lint: .npm-lint:
image: $NODE_IMAGE image: $NODE_IMAGE
stage: verify stage: verify
before_script: script:
- > - >
if [ "$FRONTEND_DIR" != "" ]; then if [ "$FRONTEND_DIR" != "" ]; then
cd $FRONTEND_DIR cd $FRONTEND_DIR
fi fi
script:
- npm run ci:lint - npm run ci:lint

View File

@ -1,12 +1,11 @@
.npm-verify: .npm-verify:
image: $NODE_IMAGE image: $NODE_IMAGE
stage: verify stage: verify
before_script: script:
- > - >
if [ "$FRONTEND_DIR" != "" ]; then if [ "$FRONTEND_DIR" != "" ]; then
cd $FRONTEND_DIR cd $FRONTEND_DIR
fi fi
script:
- npm run ci:test - npm run ci:test
artifacts: artifacts:
expire_in: '7d' expire_in: '7d'

View File

@ -35,7 +35,7 @@
if [ "$BACKEND_DIR" != "" ]; then if [ "$BACKEND_DIR" != "" ]; then
cd $BACKEND_DIR cd $BACKEND_DIR
fi fi
- ./gradlew $GRADLE_OPTS sonarqube $GRADLE_CLI_OPTS - ./gradlew --build-cache $GRADLE_OPTS sonarqube $GRADLE_CLI_OPTS $GRADLE_EXTRA_OPTS
"-Dsonar.host.url=$SONAR_HOST_URL" "-Dsonar.host.url=$SONAR_HOST_URL"
"-Dsonar.login=$SONAR_TOKEN" "-Dsonar.login=$SONAR_TOKEN"
"-Dsonar.projectKey=$SONAR_PROJECT_KEY" "-Dsonar.projectKey=$SONAR_PROJECT_KEY"

View File

@ -4,7 +4,7 @@
SONAR_HOST_URL: https://sonar.exxcellent.de/sonar8 SONAR_HOST_URL: https://sonar.exxcellent.de/sonar8
SONAR_PROJECT_KEY: "" SONAR_PROJECT_KEY: ""
SONAR_PROJECT_NAME: "" SONAR_PROJECT_NAME: ""
SONAR_EXCLUSIONS: "**/target/**,**/src/test/**" SONAR_EXCLUSIONS: "**/target/**,**/build/**,**/src/test/**"
SONAR_QUALITYGATE_WAIT: "true" SONAR_QUALITYGATE_WAIT: "true"
SONAR_QUALITYGATE_TIMEOUT: "10000" SONAR_QUALITYGATE_TIMEOUT: "10000"
@ -36,7 +36,7 @@
if [ "$BACKEND_DIR" != "" ]; then if [ "$BACKEND_DIR" != "" ]; then
cd $BACKEND_DIR cd $BACKEND_DIR
fi fi
- ./gradlew $GRADLE_OPTS sonarqube $GRADLE_CLI_OPTS - ./gradlew --build-cache $GRADLE_OPTS sonarqube $GRADLE_CLI_OPTS $GRADLE_EXTRA_OPTS
"-Dsonar.host.url=$SONAR_HOST_URL" "-Dsonar.host.url=$SONAR_HOST_URL"
"-Dsonar.login=$SONAR_TOKEN" "-Dsonar.login=$SONAR_TOKEN"
"-Dsonar.projectKey=$SONAR_PROJECT_KEY" "-Dsonar.projectKey=$SONAR_PROJECT_KEY"