Comment fonctionne le cache gitlab-ci sur docker runner? qu'est-ce que le répertoire / cache? qu'est-ce que cache_dir?

Question

Comment fonctionne le cache gitlab-ci sur docker runner?
Qu'est-ce que le répertoire/cache?
Qu'est-ce que cache_dir?
Où et comment les fichiers correspondant à "chemins" dans "cache" gitlab-ci.yml sont stockés?

srghma · Accepted Answer

Volume monté sur /cache le répertoire est créé automatiquement lors de l'installation de gitlab-runner et géré par cache_dir réglage

plus à propos cache_dir:

If you modify the /cache storage path, you also need to make sure to mark this directory as persistent by defining it in volumes = ["/my/cache/"] under the [runners.docker] section in config.toml.

TLDR

/cache dir est différent de cache config dans gitlab-ci.yml

/cache dir dans le conteneur de jobs est l'endroit où les fichiers mis en cache sont stockés
les fichiers correspondant à cache config dans gitlab-ci.yml sont copiés dans /cache/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/<cache-key>-<cache-number> en fin d'emploi
Le bouton "Clear Runner Caches" de la page "Pipelines" de votre projet planifie de NE PAS extraire /cache/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/<cache-key>-<cache-number>/cache.Zip vers dir spécifié dans cache config dans gitlab-ci.yml (au lieu de "supprimer le contenu du dossier/cache comme je le pensais au début)

P.S.

Il existe un conteneur nommé gitlab-runner-cache créé sur la machine avec gitlab-runner ( https://gitlab.com/gitlab-org/gitlab-runner/blob/af343971874198a1923352107409583b78e8aa80/executors/docker/executor_docker.go#L382 )

(On dirait) Ce conteneur est utilisé pour créer un volume anonyme où les données/cache sont stockées. Une fois le volume anonyme créé, ce conteneur est arrêté.

Les conteneurs de travaux (c'est-à-dire que les tests étaient généralement exécutés par les conteneurs) montent ce volume anonyme

Preuves

AYANT gitlab-ci.yml

image: srghma/docker-nixos-with-git-crypt cache: key: "test00000" # to reset cache - change this key OR clear cache in project settings page paths: - .mycache # gitlab allows only cache dirs that are relative to project root OR /cache (created automatically) testtest: script: - nix-env -i tree - tree --dirsfirst -L 4 /cache - ls -al ./.mycache || true - echo "test" > /cache/test - mkdir -p ./.mycache - echo "test" > ./.mycache/test - tree --dirsfirst -L 4 /cache - ls -al ./.mycache || true

Production:

sur la première manche

Running with gitlab-runner 11.6.0 (f100a208) on srghma_gitlab_runner 9b3980da Using Docker executor with image srghma/docker-nixos-with-git-crypt ... Pulling docker image srghma/docker-nixos-with-git-crypt ... Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ... Running on runner-9b3980da-project-222-concurrent-0 via myrunner.com... Fetching changes... Removing .mycache/ HEAD is now at 675caa7 feat: cache update From https://gitlab.com/srghma/myproject 675caa7..3d1e223 nix -> Origin/nix Checking out 3d1e2237 as nix... Skipping Git submodules setup Checking cache for test00000-11... No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted. Successfully extracted cache $ nix-env -i tree installing 'tree-1.8.0' these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked): /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0 copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'... building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'... created 233 symlinks in user environment $ tree --dirsfirst -L 4 /cache /cache 0 directories, 0 files $ ls -al ./.mycache || true $ echo "test" > /cache/test ls: ./.mycache: No such file or directory $ mkdir -p ./.mycache $ echo "test" > ./.mycache/test $ tree --dirsfirst -L 4 /cache /cache `-- test 0 directories, 1 file $ ls -al ./.mycache || true total 12 drwxr-xr-x 2 root root 4096 Feb 24 11:44 . drwxrwxrwx 20 root root 4096 Feb 24 11:44 .. -rw-r--r-- 1 root root 5 Feb 24 11:44 test Creating cache test00000-11... .mycache: found 2 matching files No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally. Created cache Job succeeded

sur la deuxième manche

Running with gitlab-runner 11.6.0 (f100a208) on srghma_gitlab_runner 9b3980da Using Docker executor with image srghma/docker-nixos-with-git-crypt ... Pulling docker image srghma/docker-nixos-with-git-crypt ... Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ... Running on runner-9b3980da-project-222-concurrent-0 via myrunner.com... Fetching changes... Removing .mycache/ HEAD is now at 3d1e223 feat: cache update Checking out 3d1e2237 as nix... Skipping Git submodules setup Checking cache for test00000-11... No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted. Successfully extracted cache $ nix-env -i tree installing 'tree-1.8.0' these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked): /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0 copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'... building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'... created 233 symlinks in user environment $ tree --dirsfirst -L 4 /cache /cache |-- srghma | `-- myproject | `-- test00000-11 | `-- cache.Zip `-- test 3 directories, 2 files $ ls -al ./.mycache || true total 12 drwxr-xr-x 2 root root 4096 Feb 24 11:44 . drwxrwxrwx 20 root root 4096 Feb 24 11:44 .. -rw-r--r-- 1 root root 5 Feb 24 11:44 test $ echo "test" > /cache/test $ mkdir -p ./.mycache $ echo "test" > ./.mycache/test $ tree --dirsfirst -L 4 /cache /cache |-- srghma | `-- myproject | `-- test00000-11 | `-- cache.Zip `-- test 3 directories, 2 files $ ls -al ./.mycache || true total 12 drwxr-xr-x 2 root root 4096 Feb 24 11:44 . drwxrwxrwx 20 root root 4096 Feb 24 11:44 .. -rw-r--r-- 1 root root 5 Feb 24 11:44 test Creating cache test00000-11... .mycache: found 2 matching files No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally. Created cache Job succeeded

après avoir effacé le cache en cliquant sur "Effacer les caches des coureurs" dans la page "Pipelines" de votre projet

Running with gitlab-runner 11.6.0 (f100a208) on srghma_gitlab_runner 9b3980da Using Docker executor with image srghma/docker-nixos-with-git-crypt ... Pulling docker image srghma/docker-nixos-with-git-crypt ... Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ... Running on runner-9b3980da-project-222-concurrent-0 via myrunner.com... Fetching changes... Removing .mycache/ HEAD is now at 3d1e223 feat: cache update Checking out 3d1e2237 as nix... Skipping Git submodules setup Checking cache for test00000-12... No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted. Successfully extracted cache $ nix-env -i tree installing 'tree-1.8.0' these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked): /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0 copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'... building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'... created 233 symlinks in user environment $ tree --dirsfirst -L 4 /cache /cache |-- srghma | `-- myproject | `-- test00000-11 | `-- cache.Zip `-- test 3 directories, 2 files $ ls -al ./.mycache || true ls: ./.mycache: No such file or directory $ echo "test" > /cache/test $ mkdir -p ./.mycache $ echo "test" > ./.mycache/test $ tree --dirsfirst -L 4 /cache /cache |-- srghma | `-- myproject | `-- test00000-11 | `-- cache.Zip `-- test 3 directories, 2 files $ ls -al ./.mycache || true total 12 drwxr-xr-x 2 root root 4096 Feb 24 11:45 . drwxrwxrwx 20 root root 4096 Feb 24 11:45 .. -rw-r--r-- 1 root root 5 Feb 24 11:45 test Creating cache test00000-12... .mycache: found 2 matching files No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally. Created cache Job succeeded