The three-layer architecture

The DevOps code in a DIA repository is not one flat pile of scripts. It is organised into three layers, each with a distinct job and a distinct caller. Understanding the layers is the single most useful mental model for working with the toolchain, because it tells you where any given piece of behaviour lives.

graph TD
    YML["bitbucket-pipelines.yml"] -->|npm run …| NPM["package.json scripts
    (stable names)"]
    NPM --> CLI["dia-scripts CLI
    (node index.mjs …)"]
    CLI --> Tasks["Task scripts
    (deploy/, org-pool/, …)"]
    CLI --> Shared["Shared library
    (env, log, sf-utils, sf-cli)"]
    Tasks --> Shared

Layer 1 — pipeline entry (bitbucket-pipelines.yml + npm scripts)

The pipeline calls nothing directly. Every step runs a named npm script — npm run ci:auth:deploy, npm run deploy, npm run validate — and each of those scripts in package.json is a thin line that invokes the CLI (node ./scripts/dia-scripts/index.mjs <command>). A handful of scripts instead call a tool straight (build → webpack, lint:yaml → yamllint) or a task script directly (data:import → deploy/import-data.mjs).

This indirection is the point: the npm names are the stable contract. bitbucket-pipelines.yml references npm run <name>, so the pipeline never has to change when a command's implementation does — only the script's right-hand side moves. The CLI is the entry point; npm is the glue.

Layer 2 — the dia-scripts CLI

The heart of the toolchain is an internal yargs command-line interface. Everything a human or a pipeline does — deploy, validate, create a scratch org, maintain a pool, generate a test suite — is a command on this CLI.

graph LR
    index["index.mjs
    (Node version guard)"] --> run["cli/run-script.mjs
    (yargs wrapper)"]
    run --> cmd["commands/*.mjs
    (command modules)"]

The entry chain is deliberate:

• index.mjs checks the Node version and refuses to run on an unsupported one, then hands off.
• cli/run-script.mjs builds the yargs instance, registers every command with an explicit static import, and installs a custom failure handler that hides stack traces unless DEBUG is set.
• Each command module exports the yargs quartet — command, desc, builder, handler — so registration is uniform. The ci, org, apex and docs groups register their children in their own builder and fail when no child matched; lint and test instead dispatch to their children in series; deploy has a default that runs the orchestrator.

The registration is explicit rather than directory-scanned because the project is native ESM — see the codebase and the CLI.

Commands are grouped: apex, ci, deploy, docs, org, plus a few top-level ones. A group is itself a command whose subcommands are the real work (org scratch-create, ci auth-deploy, and so on). The full list, the flags and the exit codes are in the CLI reference.

Layer 3 — task scripts

Below the commands sit the task scripts that do one concrete thing: a metadata workaround, a step of scratch-org creation, a piece of the pooling machinery. They live under deploy/, org-pool/, and alongside the commands. Commands orchestrate them; they rarely run on their own.

The deployment orchestrator is the busiest consumer — it weaves a chain of deploy/*.mjs task scripts around every deploy to patch metadata that the platform will not accept as-is.

Workarounds vs. custom steps

Two kinds of task script sit in that chain and are easy to conflate, because both let a project run code around a deploy. The contract each follows is what actually distinguishes them:

• A workaround patches the tracked source tree before a deploy to get metadata past a platform limitation the toolchain owns, not the project, and is undone afterwards. Patching has two sides: the files themselves, and — for anything removed — the delta manifest, which is computed from git and would otherwise still demand the removed component (see workaround removals and the delta manifest). One registration per workaround, in one list shared by the deploy orchestrator and org scratch-create, is the point: a delta validation and a full push must patch the same metadata the same way, and a single registration cannot be honoured on one path and forgotten on the other. The revert exists for local runs, where a developer's working tree has to come back clean; a CI workspace is disposable. The contract itself is in the workaround reference; the steps are in add a deployment workaround.
• A custom step is a project-specific custom-steps-*.mjs module exporting run(). It has no revert contract at all. The orchestrator passes it the target org, so it can act against the org the deploy just ran against; a hook that ignores the argument still works. It fits logic that acts on the running environment or the target org itself, not on the tracked source tree. See add a custom pre/post deployment step.

Three decision rules follow from that distinction:

• A value that has to be computed at deploy time is a workaround, not a custom step. Value replacements (replaceWhenEnv in sfdx-project.json) cover values known per stage — a committed file or an environment variable's string. Anything derived from the org or the checkout at run time has to be written into the source tree before the push, and that is what the workaround contract is for.
• A workaround's tags come from what it fixes. Components a persistent org already has and must not receive again belong to ORG_DEPLOY; production-only users, addresses, URLs, or references the scratchinit context excludes belong to SCRATCH; settings a fresh org rejects on its initial push belong to SCRATCH_PUSH. A workaround may carry several tags.
• A workaround registers in the chain unless it has to sit between two org actions. Package removal + package install and queue removal + queue deploy interleave a metadata patch with an org call, so their position differs per consumer; those stay explicit in the callers, as does the .forceignore context swap. Everything else goes into the one list.

The deploy hook sequence

Around the main deployment the orchestrator runs two kinds of project hook: hook folders — root-level pre/ and post/ directories whose metadata is deployed as-is when they exist and are non-empty — and the two custom-step modules. The order is fixed, and the same order applies to org scratch-create's initial push, minus the main deployment step itself:

graph LR
    Pre["deploy pre/ folder"] --> CSPre["custom-steps-pre-deployment.mjs"]
    CSPre --> Main["main deployment
    (quick, delta or full)"]
    Main --> Post["deploy post/ folder"]
    Post --> CSPost["custom-steps-post-deployment.mjs"]

The post step runs last on purpose: by the time it executes, every metadata change — including the post/ folder — has landed, so it can rely on the org being in its final shape. deploy validate runs none of the hooks: a check-only deploy commits nothing to the org, so there is nothing for a post step to act on. The flags that skip hooks or workarounds are in the CLI reference.

Per-context .forceignore profiles

One of those task scripts, prepare-forceignore, exists because the committed .forceignore mixes two kinds of ignore rule that need different lifetimes. The file carries a marker line, KEEP_UNTIL_HERE_FOR_INITIAL_PUSH. Entries above it are permanent exclusions — generated LWC test scaffolding, TypeScript sources, and the like — that stay ignored no matter how the push happens. Entries below it are meant only for a developer's own local iteration against an already-created scratch org: they are skipped when someone calls sf project deploy start or sf project source push directly, but must not be skipped when the tooling itself pushes, because the DIA scripts' own first push into a fresh scratch org (org scratch-create) and every stage deploy the orchestrator runs still need that metadata to go out.

apply resolves that by being stateless and destructive: it backs the working file up to .forceignore.bac, then truncates it to everything up to and including the token, dropping the below-token entries entirely for the duration of the run. This runs on every DIA-scripts-driven push — the orchestrator's stage deploys and org scratch-create's own initial metadata push alike — not only on some notion of "the first push"; there is no first-push/later-push state anywhere in the code. The distinction the token actually draws is CLI-tooling push versus raw, manual sf invocation. restore puts the untruncated file back from the backup once the run finishes, so a developer's subsequent raw pushes see the below-token entries again.

On top of that truncated base, the script layers a context-specific extra file for the run — deployment mode appends .forceignoreDeployment, scratchinit mode appends .forceignoreScratchInit — so the orchestrator's stage deploys and scratch-create's scratch-org push can each carry permanent exclusions of their own without leaking into the other context. The deploy orchestrator defaults to deployment; org scratch-create applies scratchinit on its own push and on the pooled delta validation it runs through the orchestrator, so neither consumer needs the context passed by hand. Unknown context values are rejected hard, because a silently-ignored profile deploys with the wrong exclusions rather than failing loud. See exclude metadata from a deployment or scratch push for the steps.

The shared library — the common vocabulary

Cutting across all three layers is a small shared library that everything imports: the environment reader, the logging helpers with their stream convention (payload to stdout, diagnostics to stderr), the single path for JSON-returning sf calls, and a few higher-level sf helpers. The module list is in the CLI reference.

Where does my change go?

• Changing what the pipeline runs or in what order → bitbucket-pipelines.yml and the npm run names.
• Changing how a command behaves → the command module under commands/.
• Changing a single metadata fix-up → the relevant deploy/*.mjs task script.
• Reading a new environment variable → add it to utils/env.mjs and export a semantic value.