npm or yarn cache to speed up the deployment

[mohdmusthafa]

I have a project as monorepo (turbo repo) + yarn. I have 8 packages in the repo and I filter out the installations using yarn workspace api install, yarn workspace admin install etc. the issue is that yarn install is taking too much time for each deployment as it needs to download +700mb of packages, is there any way that I can cache the dependencies to reuse in the deployment

Note:

1. docker cleanup enabled with frequency (*/10 * * * *)
2. packages tend to use same packages
3. I am looking for something like we have on the github actions (actions/cache@v4)
4. If attaching a volume to yarn cache directory helps, please elaborate on how to achieve that (I'm new to this..:)

[namdien177]

I was trying to use the persistence storage feature to achieve what you said as well, but that feature seems to mount the directory AFTER the docker image finishes the run, so nothing much can be done there.

If we can have persistence storage with a "checkbox" to mount before the build, then it's perfect, IMO.

(my Raspberry Pi 4 takes like ~1500 seconds just for bun install. I would love to have .bun and node_modules cached somehow.)

[darklight9811]

Would love to have this too, bun install is taking around 2 minutes

[hades200082]

With nextjs + payload + mastra deployments take nearly 15 minutes. This is an essential feature request.

[djsisson]

run

docker buildx create --name test --driver docker-container --use

this will create a build container that uses a volume for caching, so it will not get effected by coolify docker cleanup, please note this volume defaults to reserving 30% of diskspace, you can configure it in options or with config

https://docs.docker.com/reference/cli/docker/buildx/create/#buildkitd-config

So now your build use this buildx container, you just need to mount your cache in your docker files

a rough example

# syntax=docker/dockerfile:1

#### Build Stage ####
FROM node:23-alpine AS builder

# Install necessary build dependencies
RUN apk add --no-cache libc6-compat

# Set working directory
WORKDIR /app

# Copy package manifests
COPY package.json package-lock.json ./

# Node Env (prodution to skip dev dependencies)
ENV NODE_ENV=production

# Install dependencies with caching
RUN --mount=type=cache,target=/root/.npm \
    npm ci

# Copy remaining source code
COPY . .

# Build Args for static generation
ARG NEXT_PUBLIC_XXXX

# Build in standalone mode
RUN npm run build

#### Production Stage ####
FROM node:23-alpine AS runner

# Node Env
ENV NODE_ENV=production

# Create non-root user and group
RUN addgroup -g 1001 node \
 && adduser -S -u 1001 -G node nextjs

# Set working directory
WORKDIR /app

COPY --from=builder /app/public ./public

# Set the correct permission for prerender cache
RUN mkdir .next
RUN chown nextjs:nodejs .next

COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

USER nextjs

EXPOSE 3000
ENV HOSTNAME="0.0.0.0"
CMD ["node", "server.js"]

you can cache apt(bookworm etc) turbo and packages using cache mount, you can specify id if you build multiple projects and don't want them to share cache, as it defaults to id = path
ps. this assumes you're running npm and standalone in nextjs

each framework has different caching strategies, so you would need to change cache mounts as required

e.g for go its

RUN --mount=type=cache,target=/go/pkg/mod/ \
    --mount=type=bind,source=go.mod,target=go.mod \
    --mount=type=bind,source=go.sum,target=go.sum \
    go mod download -x

ARG TARGETARCH
ARG TARGETOS

RUN --mount=type=cache,target=/go/pkg/mod/ \
    --mount=type=bind,target=. \
    --mount=type=cache,target=/root/.cache/go-build \
    CGO_ENABLED=0 GOARCH=$TARGETARCH GOOS=$TARGETOS go build -ldflags="-s -w" -o /usr/bin/server ./cmd/main

[theDanielJLewis]

That seems interesting, but how would we do this when we're running applications without custom docker or compose files?

[ntgussoni]

eg: on nixpacks

[djsisson]

nixpacks already uses the same mount caches in the generated Dockerfile
but it uses an id so that they are scoped only to that id (to keep caches isolated to each repo), which is some kind of hash which includes the build path

but i believe each deployment is given its own deployment uuid by coolify, therefore every build path is different, so the id will be different each time so there will be no cache to reuse and the build cache just gets bigger overtime

coolify/app/Jobs/ApplicationDeploymentJob.php

Lines 214 to 215 in 96b3e26

	$this->basedir = $this->application->generateBaseDir($this->deployment_uuid);
	$this->workdir = "{$this->basedir}".rtrim($this->application->base_directory, '/');

coolify/app/Models/Application.php

Lines 970 to 973 in 96b3e26

	public function generateBaseDir(string $uuid)
	{
	return "/artifacts/{$uuid}";
	}

where uuid is the deployment uuid

when running commands inside the coolify helper there is no need to change the base dir everytime since the container is isolated so this is something that should absolutely be changed

[ntgussoni]

@andrasbacsai is there any workaround we can do or way to help make a change ?