Configuring CI Using Circle CI and Nx
There are two general approaches to setting up CI with Nx - using a single job or distributing tasks across multiple jobs. For smaller repositories, a single job is faster and cheaper, but once a full CI run starts taking 10 to 15 minutes, using multiple jobs becomes the better option. Nx Cloud's distributed task execution allows you to keep the CI pipeline fast as you scale. As the repository grows, all you need to do is add more agents.
Process Only Affected Projects With One Job on Circle CI
Below is an example of an Circle CI setup that runs on a single job, building and testing only what is affected. This uses the nx affected command to run the tasks only for the projects that were affected by that PR.
1version: 2.1
2orbs:
3 nx: nrwl/nx@1.5.1
4jobs:
5 main:
6 docker:
7 - image: cimg/node:lts-browsers
8 steps:
9 - checkout
10 - run: npm ci
11 - nx/set-shas
12
13 - run: npx nx format:check
14 - run: npx nx affected --base=$NX_BASE --head=$NX_HEAD -t lint,test,build --parallel=3 --configuration=ci
15workflows:
16 build:
17 jobs:
18 - main
19Get the Commit of the Last Successful Build
CircleCI can track the last successful run on the main branch and use this as a reference point for the BASE. The Nx Orb provides a convenient implementation of this functionality which you can drop into your existing CI config. Specifically, nx/set-shas populates the $NX_BASE environment variable with the commit SHA of the last successful run.
To understand why knowing the last successful build is important for the affected command, check out the in-depth explanation in Orb's docs.
Using CircleCI in a private repository
To use the Nx Orb with a private repository on your main branch, you need to grant the orb access to your CircleCI API. You can do this by creating an environment variable called CIRCLE_API_TOKEN in the context or the project.
It should be a user token, not the project token.
Distribute Tasks Across Agents on Circle CI
To set up Distributed Task Execution (DTE), you can run this generator:
❯
npx nx g ci-workflow --ci=circleci
Or you can copy and paste the workflow below:
1version: 2.1
2orbs:
3 nx: nrwl/nx@1.5.1
4jobs:
5 main:
6 docker:
7 - image: cimg/node:lts-browsers
8 steps:
9 - checkout
10 - run: npm ci
11 - nx/set-shas
12
13 # Tell Nx Cloud to use DTE and stop agents when the build tasks are done
14 - run: npx nx-cloud start-ci-run --stop-agents-after=build
15 # Send logs to Nx Cloud for any CLI command
16 - run: npx nx-cloud record -- npx nx format:check
17 # Lint, test and build on agent jobs everything affected by a change
18 - run: npx nx affected --base=$NX_BASE --head=$NX_HEAD -t lint,test,build --parallel=2 --configuration=ci
19 agent:
20 docker:
21 - image: cimg/node:lts-browsers
22 parameters:
23 ordinal:
24 type: integer
25 steps:
26 - checkout
27 - run: npm ci
28 # Wait for instructions from Nx Cloud
29 - run:
30 command: npx nx-cloud start-agent
31 no_output_timeout: 60m
32workflows:
33 build:
34 jobs:
35 - agent:
36 matrix:
37 parameters:
38 ordinal: [1, 2, 3]
39 - main
40This configuration is setting up two types of jobs - a main job and three agent jobs.
The main job tells Nx Cloud to use DTE and then runs normal Nx commands as if this were a single pipeline set up. Once the commands are done, it notifies Nx Cloud to stop the agent jobs.
The agent jobs set up the repo and then wait for Nx Cloud to assign them tasks.
Two Types of ParallelizationThe ordinal: [1, 2, 3] line and the --parallel flag both parallelize tasks, but in different ways. The way this workflow is written, there will be 3 agents running tasks and each agent will try to run 2 tasks at once. If a particular CI run only has 2 tasks, only one agent will be used.