Concourse CI 3.3.x has introduced the ability to cache paths between task runs. This feature helps speed up tasks which cache content in specific folders – here I will demonstrate how this feature can be used for speeding up maven and gradle based java builds.
The code and the pipeline that I am using for this post is available at my github repo here – https://github.com/bijukunjummen/ci-concourse-caching-sample
Let me start with the gradle build, if I were to build the project using a gradle wrapper using the following command:
./gradlew clean build
then gradle would download the dependent libraries into a “.gradle” folder in the users home folder by default. This location of this folder can be changed using a “GRADLE_USER_HOME” environment variable, which is what I will be using in a concourse task to control the location of a cached path.
A concourse task which builds my project looks like this:
---
platform: linux
image_resource:
type: docker-image
source:
repository: openjdk
tag: 8-jdk
inputs:
- name: repo
outputs:
- name: out
run:
path: /bin/bash
args:
- repo/ci/tasks/build.sh
caches:
- path: .gradle/
- path: .m2/
params:
PROJECT_TYPE:See the caches parameter is specified as “.gradle” above. So all I have to do now is to ensure that Gradle uses this location as its home folder, which I would do in my build script:
export ROOT_FOLDER=$( pwd )
export GRADLE_USER_HOME="${ROOT_FOLDER}/.gradle"The process to cache maven resources for a maven build is along the same lines, maven caches the dependent jars in a location that can be specified in a variety of ways, the one I have used is to specify this location via a dynamically generated settings.xml file the following way:
M2_HOME=${HOME}/.m2
mkdir -p ${M2_HOME}
M2_LOCAL_REPO="${ROOT_FOLDER}/.m2"
mkdir -p "${M2_LOCAL_REPO}/repository"
cat > ${M2_HOME}/settings.xml <<EOF
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
https://maven.apache.org/xsd/settings-1.0.0.xsd">
<localRepository>${M2_LOCAL_REPO}/repository</localRepository>
</settings>
EOFwhich is quite a bit of bash scripting, all it is doing is generating a settings.xml with a localRepository tag set to “.m2/repository” folder which is relative to the temporary folder created by concourse for the build and thus can be cached.
With these changes in place, the behavior is that the downloads happen for the first run of the task but then get cached for subsequent runs. In my local concourse set-up a gradle build taking about 2 mins for a first time build takes about 20 seconds for a subsequent build !
You can try out this feature in my demo project here – https://github.com/bijukunjummen/ci-concourse-caching-sample
| Reference: | Concourse caching for Java Maven and Gradle builds from our JCG partner Biju Kunjummen at the all and sundry blog. |
