Building a parallel publish job for a pnpm monorepo

How to turn a slow, hand-maintained, sequential publish loop into a parallel one — and the traps that only surface once you try it. Most of them are silent: the job stays green while doing the wrong thing.

Audience: release automation for a pnpm workspace publishing to a GitLab Package Registry.

1. The starting point

A typical first-generation publish job looks like this:

# Dependent must come after its dependency: publish rewrites workspace:*
for pkg in core ui platform-apis ... cli sdk; do
  PKG_DIR="$CI_PROJECT_DIR/packages/@scope/$pkg"
  [ -d "$PKG_DIR" ] || continue
  cd "$PKG_DIR"
  cp $CI_PROJECT_DIR/.npmrc .
  VERSION=$(node -p "require('./package.json').version")
  if ! npm view @scope/$pkg@$VERSION 2>/dev/null; then
    pnpm publish --no-git-checks
  fi
done

Three problems:

1. It is sequential. Each publish is mostly network wait. Many packages serialize into a minute or more of dead time.
2. The list is hand-maintained. Adding a package means remembering to edit CI. Forgetting is silent: the package never publishes, and the job stays green.
3. The ordering comment is a misconception (see below). It constrains the design for a reason that does not hold.

2. Mental model

Four moving parts — get all four straight before writing any shell:

The invariant worth designing around: never publish a version that is not recorded in git. Push the version-bump commit before publishing. If the push fails, publish nothing.

3. Discover packages instead of listing them

for d in "$CI_PROJECT_DIR"/packages/@scope/*/; do
  [ "$(node -p "require('${d}package.json').private || false" 2>/dev/null)" = "true" ] && continue
  pkg=$(basename "$d")
  ...
done

Two notes on the require() calls, which use different path forms deliberately:

• require('${d}package.json') — $d comes from the glob, so it is absolute with a trailing slash. require() accepts absolute paths.
• require('./package.json') — used later, after cd "$d", resolving against the working directory.

A relative path that does not start with ./ or / would be treated as a module name and fail.

The private filter is not optional. The old hardcoded list excluded non-publishable packages by simply not naming them. A glob names everything, and pnpm publish on a private: true package fails the job.

Trap — directories that are not packages. A glob also picks up any directory without a package.json. The private check returns empty, VERSION comes out empty, and then pnpm publish walks up to the workspace root and tries to publish that instead. Guard explicitly:

[ -f "${d}package.json" ] || continue

4. Ordering: what actually matters

The common belief is that a dependent must publish after its dependency so the workspace:* rewrite picks up the right version. That is wrong. pnpm resolves workspace:* from the local workspace, never the registry, so order is irrelevant to rewrite correctness.

Trap — the rewrite needs an install. pnpm resolves that version through the installed workspace link, not by reading the sibling package.json. With no node_modules, it fails outright:

ERR_PNPM_CANNOT_RESOLVE_WORKSPACE_PROTOCOL
Cannot resolve workspace protocol of dependency "@scope/cli"
because this dependency is not installed.

So an install must precede the publish loop. Where ordering does matter is registry self-consistency. Publishing in parallel means a dependent can appear on the registry seconds before its dependency exists. Anyone installing in that window gets an unresolvable dependency.

In practice the window is a few seconds and consumers install minutes-to-days later. A hand-ordered sequential list is also easy to get wrong — so the same window often already existed, unnoticed.

If you need to close it properly, publish in topological levels — parallel within each level, barrier between levels.

5. Parallelizing in POSIX sh

Trap — there is no bash. node:18-alpine ships busybox as /bin/sh and has no bash at all. xargs -P combined with export -f does not work. Neither do arrays, [[ ]], or <<<.

The portable pattern is background subshells plus explicit wait per PID:

fail=0
pids=""
for d in "$CI_PROJECT_DIR"/packages/@scope/*/; do
  [ "$(node -p "require('${d}package.json').private || false" 2>/dev/null)" = "true" ] && continue
  pkg=$(basename "$d")
  (
    cd "$d" || exit 1
    cp "$CI_PROJECT_DIR/.npmrc" .
    VERSION=$(node -p "require('./package.json').version")
    if npm view "@scope/$pkg@$VERSION" >/dev/null 2>&1; then
      echo "SKIPPED @scope/$pkg@$VERSION — already on the registry."
    elif pnpm publish --no-git-checks >"/tmp/publish-$pkg.log" 2>&1; then
      echo "published @scope/$pkg@$VERSION"
    else
      echo "FAILED to publish @scope/$pkg@$VERSION:"; sed 's/^/  /' "/tmp/publish-$pkg.log"
      exit 1
    fi
  ) &
  pids="$pids $!"
done
for pid in $pids; do wait "$pid" || fail=1; done
[ "$fail" -eq 0 ] || { echo "ERROR: one or more packages failed to publish"; exit 1; }

• Capture output per package. Interleaved parallel output is unreadable. Redirect each publish to its own log and print it only on failure.
• wait "$pid" returns the subshell’s exit status. Without collecting it, a failed publish is invisible and the job goes green.
• The final [ ... ] || { ...; exit 1; } is what actually fails the job.

Note on set -e. GitLab wraps job scripts with it. The line [ test ] && continue returns non-zero whenever the test is false — but because it is an AND-OR list inside a loop, and not the last command, set -e does not abort. Worth verifying rather than assuming.

How solid is the npm view guard?

Good enough for the job’s purpose, but it is not a lock. GitLab’s package list and packument lag a publish by one to two seconds. Within that window, a query for a version that was just published can still report it missing — so a fast re-run can miss its own just-published version and publish it again. A duplicate publish can be quiet: the registry records an error-status package row while the client still exits 0.

In practice this is unreachable in a normal pipeline, because resource_group serializes publish jobs and versions only change via a bump.

6. The concurrency problem nobody expects

Parallelism inside one job is the easy half. The harder half is two publish jobs racing each other when several MRs merge close together.

1. Job A resets to the master tip and computes version bumps.
2. Another MR merges. Master moves.
3. Job A pushes its bump commit — rejected: cannot lock ref 'refs/heads/master'.
4. Job A’s versions are now stale: they did not account for the newly merged changesets.

resource_group does not prevent this. It serializes jobs, not merges — master can always move under a running job.

Aborting here is correct but leaves a red pipeline. Better: discard the stale attempt, resync, and recompute.

MAX_ATTEMPTS=5
attempt=0
give_up_reason="push was rejected repeatedly"
while :; do
  attempt=$((attempt + 1))
  [ "$attempt" -gt "$MAX_ATTEMPTS" ] && { echo "ERROR: gave up — $give_up_reason"; exit 1; }
  [ "$attempt" -gt 1 ] && git reset --hard origin/master

  [ -z "$(ls -A .changeset/*.md 2>/dev/null | grep -v README)" ] && { echo "No changesets"; break; }

  pnpm changeset version
  pnpm install --no-frozen-lockfile
  git add ... && git commit -m "chore: bump versions [skip ci]" || break

  git push --no-verify origin HEAD:master && break

  git fetch origin master:refs/remotes/origin/master
  if git merge-base --is-ancestor origin/master HEAD; then
    give_up_reason="push kept failing while master stayed put (transient)"
  else
    give_up_reason="master advanced on every attempt (concurrent merges)"
  fi
done

• The cap bounds consecutive races, not merges. An attempt is only lost if a merge lands inside its ~30–60s window.
• Giving up stays safe because a rejected push means a newer merge exists, and that merge’s own serialized job will drain the changesets.
• Classify the failure for the message only. Both cases recompute, but reporting “master advanced” for what was really an auth failure sends the next reader hunting for a race that never happened.
• Queued jobs become no-ops: they resync, find .changeset/ empty, skip the bump, and hit SKIPPED on every package — green rather than red.

Result: N MRs merged back to back produce N green jobs and one version bump, with multiple changesets for one package collapsing into a single version.

7. Caching

Keep dependency cache and build cache separate, or a lockfile change discards both:

cache:
  - key: "deps-$CI_COMMIT_REF_SLUG"
    fallback_keys: ["deps-master"]
    paths: [".pnpm-store/"]
  - key: "turbo-$CI_COMMIT_REF_SLUG"
    fallback_keys: ["turbo-master"]
    paths: [".turbo/cache"]

Cache .turbo/cache rather than .turbo/ — the latter also holds logs and daemon state.

GitLab suffixes cache keys by ref protection status and applies that suffix to fallback_keys too. An unprotected branch may resolve the same fallback to a non-protected variant that master never wrote, and miss every time. Verify by reading the “Restoring cache” section on a brand-new branch’s first job.

8. Testing it locally

Shell correctness

Extract each job’s script from the YAML and syntax-check it in the real image:

docker run --rm -v "$PWD:/s:ro" node:18-alpine sh -c 'sh -n /s/publish-loop.sh'

Prepend set -e when you execute it, so you match how GitLab runs job scripts. A macOS host shell is bash and will happily run bash-isms that die in CI.

The publish loop — Verdaccio

Useful fixtures:

• "private": true → must be excluded
• Package pre-published at the same version → must hit SKIP
• Dependent with "workspace:*" → published metadata shows a concrete version
• "prepack": "exit 1" → must fail the job
• N packages with "prepack": "sleep 3" → wall-clock proves parallelism

Two things that waste an afternoon:

• Don’t simulate a publish failure with publishConfig.registry. A scoped registry line in .npmrc overrides it. Use a failing prepack script instead.
• npm view caches negative responses. Use --prefer-online, or assert against the registry’s own API.

Version-bump races — three local git repos

A bare repo as origin, a “CI” clone, and a second clone that pushes a competing merge from inside a stubbed changeset version. That reproduces the race deterministically. A pre-receive hook that exits 1 simulates a push rejection where master does not move — how you test the transient-vs-race distinction.

Confirming resource_group

It is server-side, so it cannot be reproduced locally — but it can be observed from job metadata. Compare a publish job’s started_at against its dependency job’s finished_at and the previous publish job’s finished_at. If it starts seconds after the previous holder released, long after its own dependencies were ready, that gap is the resource group.