How I got pnpm in a serverless monorepo to deploy

For some context, we’re running a microservice architecture on both K8s and Lambda stored across a dozen monorepos (turbo) and use either yarn or npm for package management. We’re unifying our approach to make DX better, and settled on PNPM as our choice for package management.

For the most part the migration was straightforward, but we hit an issue when deploying using serverless with pnpm. After a successful deploy the node_modules folder on the lambda was empty!

The reason for this is that Serverless+Lambda do not support symlinks, but pnpm relies on them for version management. I attempted to find solutions and found old plugins, a bunch of github issues, and a lot of frustration

Press enter or click to view image in full size

After an embarrassing number of build attempts I found a functional path forward by following these steps.

1. Since we’re using Turbo, first we needed to run turbo prune --scope=<package-name> --docker . This creates a docker-ready set of files that we need for the scoped package.
2. We’ll copy /out/full (the output from prune) into the Docker container using COPY . . in the Dockerfile and add a root-level .npmrc file. This .npmrc file needs to contain node-linker=hoisted and symlinks=false .
3. In the Dockerfile we can then get serverless globally installed RUN pnpm install --global serverless@3 — I use v3 because v4 requires a license that I don’t currently have.
4. Install all of your dependencies. This will only be for the packages you included in your prune. RUN pnpm install
5. Package all production dependencies to be included in the artifact using RUN pnpm --filter data-syndication-lambda deploy output --prod
6. For serverless to work we also need to install the dev dependencies for this artifact, so first move into the output directory created in step 5 using WORKDIR /usr/src/app/output
7. Then we’ll do this one again RUN pnpm install — The only thing this ends up doing is adding the dev dependencies for the final step
8. Finally, we can deploy to Lambda! ENTRYPOINT pnpm run deploy

We now have a docker image that, when run, will deploy the code to lambda. This final process does end up installing the code multiple times so it could probably be optimized further, but it works and I’m not sure if any of these installs can be skipped. Plus, with PNPM caching it should actually be fairly performant.

Here are our CI config and Dockerfile

# CI Steps
    steps:
      - install_code
      - attach_workspace:
          at: .
      - get_package_version:
          packageName: <package-name>
      - run: pnpm build --scope=<package-name>
      - run: pnpm turbo prune --scope=<package-name> --docker
      - run:
          command: |
            echo -e "//registry.npmjs.org/:_authToken=${NPM_AUTH_TOKEN}\nnode-linker=hoisted\nsymlink=false" > .npmrc
      - run: cp .npmrc out/full/.npmrc
      - docker-build:
          image: luxurypresence/<package-name>
          packageName: <package-name>
          tag: $CIRCLE_BRANCH-v$VERSION

# Dockerfile
FROM node:18-alpine

ARG NODE_ENV
ENV NODE_ENV ${NODE_ENV}
RUN corepack enable
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"

ARG NPM_AUTH_TOKEN
ENV NPM_AUTH_TOKEN=$NPM_AUTH_TOKEN

WORKDIR /usr/src/app

COPY . .

RUN pnpm install --global serverless@3
RUN pnpm install

# Package all production dependencies to be included in the artifact
RUN pnpm --filter <package-name> deploy output --prod

# Switch to deploy directory
WORKDIR /usr/src/app/output

# Installs package-level dev dependencies
RUN pnpm install 

# Deploy to lambda
ENTRYPOINT pnpm run deploy

It took a lot of trial and error to get this working. The documentation for each of these tools is good, but combining them leaves gaps. After getting things wrong enough times I finally found a path forward that worked.

Hopefully my mistakes will help save you a headache. As always, thanks for reading.