我们有一个play2/scala
用gitlab ci构建的应用程序.
我们.gitlab-ci.yml
(至少重要的部分)看起来如下:
image: hseeberger/scala-sbt variables: SBT_GLOBAL_BASE_DIR: "$CI_PROJECT_DIR/cache/.sbt" IVY2_CACHE_DIR: "$CI_PROJECT_DIR/cache/.ivy2" SBT_BOOT_DIR: "$CI_PROJECT_DIR/cache/.sbt/boot" M2_HOME_DIR: "$CI_PROJECT_DIR/cache/.m2" before_script: # Log the sbt version - sbt sbt-version build: stage: build script: - ./build.sh
用build.sh
:
sbt -Dsbt.global.base=$SBT_GLOBAL_BASE_DIR \ -Dsbt.ivy.home=$IVY2_CACHE_DIR \ -Dsbt.boot.directory=$SBT_BOOT_DIR \ compile
不幸的是,我们的管道总是运行大约30-40分钟,包括所有步骤(构建,验证,部署).大部分时间sbt
通过一次又一次地下载真正烦人的事情来花费.
我可能不太了解,gitlab ci runners
但根据我的理解,通过使用hseeberger/scala-sbt作为图像,sbt
应该是全局可用的,不应该下载它.
然后gitlab的这个解决方案也没有必要.
无论如何,sbt
每当服务器运行任何sbt
命令时,如果在每次部署期间不会完全下载6次,我会很高兴.
有人可以解释我如何使用正确image
或image
正确的方式,或者我如何缓存这些sbt
东西?
在过去的几天里,我docker
和他一起战斗了很多gitlab ci
.我发现这个问题与不下载互联网时描述的几乎相同 .看起来这是一项艰巨的任务,拥有所有的依赖关系,最好通过安装它们来完成.遗憾的是,在共享的gitlab ci运行器上不可能这样.
我继续发现了sbt-docker,它允许你从build.sbt
文件构建docker容器.使用包基本方法,我尝试将项目的所有本地可用依赖项作为全局sbt插件包含到容器中.但这也无济于事.
我的最后一个发现是关于maven
解决方案的答案,并试图将其转化为我们的sbt
项目:
image: hseeberger/scala-sbt variables: MAVEN_OPTS: -Dmaven.repo.local=/cache/maven.repository stages: - build - test - staging - deploy build: stage: build script: - sbt compile -Dsbt.ivy.home=/cache/.ivy2 -Dsbt.global.base=/cache/.sbt/0.13 -Dsbt.boot.directory=/cache/.sbt/boot -Dsbt.repository.config=/cache/.sbt/repositories
我可以gitlab ci
再次访问日志.他们看起来基本如下:
[info] Loading project definition from /builds/kwiqjobs/backend/project [info] Updating {file:/builds/kwiqjobs/backend/project/}backend-build... [info] Resolving com.typesafe.play#sbt-plugin;2.5.4 ... [info] Resolving com.typesafe.play#sbt-plugin;2.5.4 ... [info] Resolving com.typesafe.play#sbt-routes-compiler_2.10;2.5.4 ... [info] Resolving com.typesafe.play#sbt-routes-compiler_2.10;2.5.4 ... [info] Resolving org.scala-lang#scala-library;2.10.6 ... [info] Resolving com.typesafe.play#twirl-api_2.10;1.1.1 ... [info] Resolving com.typesafe.play#twirl-api_2.10;1.1.1 ... ... a **lot** more [info] [SUCCESSFUL ] com.typesafe.sbt#sbt-twirl;1.1.1!sbt-twirl.jar (1033ms) [info] downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-native-packager/scala_2.10/sbt_0.13/1.0.3/jars/sbt-native-packager.jar ... [info] [SUCCESSFUL ] com.typesafe.sbt#sbt-native-packager;1.0.3!sbt-native-packager.jar (954ms) [info] downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-web/scala_2.10/sbt_0.13/1.3.0/jars/sbt-web.jar ... [info] [SUCCESSFUL ] com.typesafe.sbt#sbt-web;1.3.0!sbt-web.jar (1010ms) [info] downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-js-engine/scala_2.10/sbt_0.13/1.1.3/jars/sbt-js-engine.jar ... [info] [SUCCESSFUL ] com.typesafe.sbt#sbt-js-engine;1.1.3!sbt-js-engine.jar (1147ms) [info] downloading https://repo1.maven.org/maven2/com/typesafe/play/twirl-api_2.10/1.1.1/twirl-api_2.10-1.1.1.jar ... [info] [SUCCESSFUL ] com.typesafe.play#twirl-api_2.10;1.1.1!twirl-api_2.10.jar (89ms) [info] downloading https://repo1.maven.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.jar ... [info] [SUCCESSFUL ] commons-io#commons-io;2.4!commons-io.jar (48ms) a **lot** more [info] Done updating. [info] Compiling 228 Scala sources and 4 Java sources to /builds/kwiqjobs/backend/target/scala-2.11/classes... [info] 'compiler-interface' not yet compiled for Scala 2.11.8. Compiling... [info] Compilation completed in 17.735 s [success] Total time: 149 s, completed Jan 20, 2017 2:22:52 PM Build succeeded
我想摆脱所有的下载.
如果您不想使用自定义图像,最好的解决方案是使用Gitlab CI的缓存机制.
要做到这一点有点困难,但是这篇博文描述了如何为SBT做到这一点.
例.gitlab-ci.yml
引自博客文章,我自己纠正了一些小错误:
# some parts originally from https://github.com/randm-ch/units-of-information/blob/master/.gitlab-ci.yml image: "hseeberger/scala-sbt" variables: SBT_VERSION: "0.13.9" SBT_OPTS: "-Dsbt.global.base=sbt-cache/.sbtboot -Dsbt.boot.directory=sbt-cache/.boot -Dsbt.ivy.home=sbt-cache/.ivy" cache: key: "$CI_BUILD_REF_NAME" # contains either the branch or the tag, so it's caching per branch untracked: true paths: - "sbt-cache/.ivy/cache" - "sbt-cache/.boot" - "sbt-cache/.sbtboot" - "sbt-cache/target" stages: - test test: script: - sbt test第二个例子,还包括
apt-get
缓存
这是我用于我的项目,可用于更一般的用例和Docker镜像:
image: java:8 stages: - test variables: SBT_VERSION: "0.13.9" SBT_OPTS: "-Dsbt.global.base=sbt-cache/.sbtboot -Dsbt.boot.directory=sbt-cache/.boot -Dsbt.ivy.home=sbt-cache/.ivy" SBT_CACHE_DIR: "sbt-cache/.ivy/cache" cache: key: "$CI_BUILD_REF_NAME" # contains either the branch or the tag, so it's caching per branch untracked: true paths: - "apt-cache/" - "sbt-cache/.ivy/cache" - "sbt-cache/.boot" - "sbt-cache/.sbtboot" - "sbt-cache/target" before_script: - export APT_CACHE_DIR=`pwd`/apt-cache - mkdir -pv $APT_CACHE_DIR - ls $APT_CACHE_DIR || echo "no apt-cache dir found" - apt-get -o dir::cache::archives=$APT_CACHE_DIR update -y - apt-get -o dir::cache::archives=$APT_CACHE_DIR install apt-transport-https -y # Install SBT - mkdir -pv $SBT_CACHE_DIR - ls $SBT_CACHE_DIR || echo "no ivy2 cache fir found" - echo "deb http://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list - apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 642AC823 - apt-get -o dir::cache::archives=$APT_CACHE_DIR update -y - apt-get -o dir::cache::archives=$APT_CACHE_DIR install sbt -y - sbt -v sbtVersion test: stage: test script: - sbt -v sbtVersion