mirrors/zizmor
1
0
Fork 0
mirror of https://github.com/zizmorcore/zizmor.git synced 2025-12-23 08:47:33 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

cli: add a "GitHub" output format (#634)

Browse source 
* cli: add a "GitHub" output format

Closes #633.

Signed-off-by: William Woodruff <william@yossarian.net>

* try using SARIF path

Signed-off-by: William Woodruff <william@yossarian.net>

* fix lines

Signed-off-by: William Woodruff <william@yossarian.net>

* fmt

Signed-off-by: William Woodruff <william@yossarian.net>

* add --no-exit-codes

Signed-off-by: William Woodruff <william@yossarian.net>

* bump help snippet

Signed-off-by: William Woodruff <william@yossarian.net>

* bump snippet

Signed-off-by: William Woodruff <william@yossarian.net>

* integration test for github output

Signed-off-by: William Woodruff <william@yossarian.net>

* github: output tweaks

* update snapshot

* test-output: test GitHub output on just one file

* remove columns

* bump snapshot

* try something else

Signed-off-by: William Woodruff <william@yossarian.net>

* fixup snapshot

Signed-off-by: William Woodruff <william@yossarian.net>

* one last hack

Signed-off-by: William Woodruff <william@yossarian.net>

* add primary annotation to message

Signed-off-by: William Woodruff <william@yossarian.net>

* usage: document --format=github, add integration docs

Signed-off-by: William Woodruff <william@yossarian.net>

* docs: update release notes

---------

Signed-off-by: William Woodruff <william@yossarian.net>
This commit is contained in:
William Woodruff 2025-04-07 19:51:19 -04:00 • committed by GitHub
parent 2f0227dde0
commit 4d5c79a582
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
13 changed files with 408 additions and 94 deletions
Download patch file Download diff file Expand all files Collapse all files

25
.github/workflows/test-sarif.yml → .github/workflows/test-output.yml vendored
View file

@ -1,4 +1,4 @@
name: Test SARIF Presentation
name: Test output formats
on:
pull_request:
@ -46,3 +46,26 @@ jobs:
repo: context.repo.repo,
body: `:robot: Presentation results: <${url}>`
})
test-github-presentation:
runs-on: ubuntu-latest
if: contains(github.event.pull_request.labels.*.name, 'test-github-presentation')
permissions: {}
steps:
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
with:
persist-credentials: false
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2
- name: Run zizmor
run: |
# Normally we'd want a workflow to fail if the audit fails,
# but we're only testing presentation here.
cargo run \
-- \
--no-exit-codes \
--format github \
tests/integration/test-data/several-vulnerabilities.yml

8
docs/release-notes.md
View file

@ -9,6 +9,14 @@ of `zizmor`.
## Next (UNRELEASED)
### New Features 🌈
* `zizmor` now supports `--format=github` as an output format.
This format produces check annotations via GitHub workflow commands,
e.g. `::warning` and `::error`. See the
[Output formats](./usage.md#output-formats) documentation for more information
on annotations, including key limitations (#634)
### Improvements 🌱
* The SARIF output format now marks each rule as a "security" rule,

2
docs/snippets/help.txt
View file

@ -25,7 +25,7 @@ Options:
--no-progress
Don't show progress bars, even if the terminal supports them
--format <FORMAT>
The output format to emit. By default, plain text will be emitted [default: plain] [possible values: plain, json, sarif]
The output format to emit. By default, cargo-style diagnostics will be emitted [default: plain] [possible values: plain, json, sarif, github]
--color <MODE>
Control the use of color in output [possible values: auto, always, never]
-c, --config <CONFIG>

325
docs/usage.md
View file

@ -105,28 +105,143 @@ zizmor --no-online-audits --gh-token ghp-... example/example
`zizmor` always produces output on `stdout`.
By default, `zizmor` produces `cargo`-style diagnostic output. This output
will be colorized by default when sent to a supporting terminal and
See [Integration](#integration) for suggestions on when to use each format.
### Cargo-style output ("plain")
By default, `zizmor` produces `cargo`-style diagnostic output.
```console
error[template-injection]: code injection via template expansion
--> ./tests/integration/test-data/template-injection/pr-425-backstop/action.yml:28:7
|
28 | - name: case4
| ^^^^^^^^^^^ this step
29 | uses: azure/powershell
30 | with:
31 | inlineScript: Get-AzVM -ResourceGroupName "${{ inputs.expandme }}"
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ inputs.expandme may expand into attacker-controllable code
|
= note: audit confidence → Low
```
This output will be colorized by default when sent to a supporting terminal and
uncolorized by default when piped to another program. Users can also explicitly
disable output colorization by setting `NO_COLOR=1` in their environment.
Apart from the default, `zizmor` supports JSON and [SARIF] as machine-readable
output modes. These can be selected via the `--format` option:
This format can also be explicitly selected with `--format=plain`:
Output formats can be controlled explicitly via the `--format` option:
### JSON
!!! important
The JSON format is currently a flat array of findings, and is not
currently versioned.
Future versions of `zizmor` may change the top-level structure of the
JSON output,
With `--format=json`, `zizmor` will produce a flat array of findings in
JSON format:
```bash
# use the default diagnostic output explicitly
zizmor --format plain
# emit zizmor's own JSON format
zizmor --format json
# emit SARIF JSON instead of normal JSON
zizmor --format sarif
zizmor --format=json . | jq .[0]
```
See [Integration](#integration) for suggestions on when to use each format.
??? Example "Example output"
```json
{
"ident": "github-env",
"desc": "dangerous use of environment file",
"url": "https://woodruffw.github.io/zizmor/audits/#github-env",
"determinations": {
"confidence": "Low",
"severity": "High",
"persona": "Regular"
},
"locations": [
{
"symbolic": {
"key": {
"Local": {
"prefix": ".",
"given_path": "./tests/integration/test-data/github-env/action.yml"
}
},
"annotation": "write to GITHUB_ENV may allow code execution",
"route": {
"components": [
{
"Key": "runs"
},
{
"Key": "steps"
},
{
"Index": 0
},
{
"Key": "run"
}
]
},
"kind": "Primary"
},
"concrete": {
"location": {
"start_point": {
"row": 9,
"column": 6
},
"end_point": {
"row": 10,
"column": 40
},
"offset_span": {
"start": 202,
"end": 249
}
},
"feature": " run: |\n echo \"foo=$(bar)\" >> $GITHUB_ENV",
"comments": []
}
}
],
"ignored": false
}
```
### SARIF
`zizmor` supports [SARIF] via `--format=sarif`.
SARIF is a JSON-based standard for representing static analysis results.
See [Use in GitHub Actions](#use-in-github-actions) for
information on using `zizmor` with GitHub's Advanced Security
functionality via GitHub Actions.
### GitHub Annotations
`zizmor` supports GitHub annotations via `--format=github`.
See [Workflow Commands for GitHub Actions] for additional information about
annotations.
!!! warning
GitHub annotations come with significant limitations: a single CI step
can only render 10 annotations at a time.
If your `zizmor` run produces more than 10 findings, only the first 10 will
be rendered; all subsequent findings will be logged in the actions log but
**will not be rendered** as annotations.
See orgs/community?26680 and orgs/community?68471 for additional
information.
## Exit codes
@ -392,89 +507,147 @@ zizmor --cache-dir /tmp/zizmor ...
### Use in GitHub Actions
`zizmor` is designed to integrate with GitHub Actions. In particular,
`zizmor --format sarif` specifies [SARIF] as the output format, which GitHub's
code scanning feature uses.
`zizmor` is designed to integrate with GitHub Actions. There are
two primary ways to use `zizmor` in GitHub Actions:
You can integrate `zizmor` into your CI/CD however you please, but one
easy way to do it is with a workflow that connects to
[GitHub's code scanning functionality].
1. With `--format=sarif` via Advanced Security (recommended)
2. With `--format=github` via GitHub Annotations
!!! important
=== "With Advanced Security (recommended)"
The workflow below performs a [SARIF] upload, which is available for public
repositories and for GitHub Enterprise Cloud organizations that have
[Advanced Security]. If neither of these apply to you, then you can
adapt the workflow to emit JSON or diagnostic output via `--format json`
or `--format plain` respectively.
GitHub's Advanced Security and [code scanning functionality] supports
[SARIF], which `zizmor` can produce via `--format=sarif`.
```yaml title="zizmor.yml"
name: GitHub Actions Security Analysis with zizmor 🌈
!!! important
on:
push:
branches: ["main"]
pull_request:
branches: ["**"]
The workflow below performs a [SARIF] upload, which is available for public
repositories and for GitHub Enterprise Cloud organizations that have
[Advanced Security]. If neither of these apply to you, then you can
use `--format=github` or adapt the `--format=json` or `--format=plain`
output formats to your needs.
jobs:
zizmor:
name: zizmor latest via PyPI
runs-on: ubuntu-latest
permissions:
security-events: write
contents: read # only needed for private repos
actions: read # only needed for private repos
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
persist-credentials: false
```yaml title="zizmor.yml"
name: GitHub Actions Security Analysis with zizmor 🌈
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
on:
push:
branches: ["main"]
pull_request:
branches: ["**"]
- name: Run zizmor 🌈
run: uvx zizmor --format sarif . > results.sarif # (2)!
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} # (1)!
jobs:
zizmor:
name: zizmor latest via PyPI
runs-on: ubuntu-latest
permissions:
security-events: write
contents: read # only needed for private repos
actions: read # only needed for private repos
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Upload SARIF file
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: results.sarif
category: zizmor
```
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
1. Optional: Remove the `env:` block to only run `zizmor`'s offline audits.
- name: Run zizmor 🌈
run: uvx zizmor --format=sarif . > results.sarif # (2)!
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} # (1)!
2. This installs the [zizmor package from PyPI], since it's pre-compiled
and therefore completes much faster. You could instead compile `zizmor`
within CI/CD with `cargo install zizmor`.
- name: Upload SARIF file
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: results.sarif
category: zizmor
```
For more inspiration, see `zizmor`'s own [repository workflow scan], as well
as GitHub's example of [running ESLint] as a security workflow.
1. Optional: Remove the `env:` block to only run `zizmor`'s offline audits.
!!! important
2. This installs the [zizmor package from PyPI], since it's pre-compiled
and therefore completes much faster. You could instead compile `zizmor`
within CI/CD with `cargo install zizmor`.
When using `--format sarif`, `zizmor` does not use its
[exit codes](#exit-codes) to signal the presence of findings. As a result,
`zizmor` will always exit with code `0` even if findings are present,
**unless** an internal error occurs during the audit.
For more inspiration, see `zizmor`'s own [repository workflow scan], as well
as GitHub's example of [running ESLint] as a security workflow.
As a result of this, the `zizmor.yml` workflow itself will always
succeed, resulting in a green checkmark in GitHub Actions.
This should **not** be confused with a lack of findings.
!!! important
To prevent a branch from being merged with findings present, you can
use GitHub's rulesets feature. For more information, see
[About code scanning alerts - Pull request check failures for code scanning alerts].
When using `--format=sarif`, `zizmor` does not use its
[exit codes](#exit-codes) to signal the presence of findings. As a result,
`zizmor` will always exit with code `0` even if findings are present,
**unless** an internal error occurs during the audit.
As a result of this, the `zizmor.yml` workflow itself will always
succeed, resulting in a green checkmark in GitHub Actions.
This should **not** be confused with a lack of findings.
To prevent a branch from being merged with findings present, you can
use GitHub's rulesets feature. For more information, see
[About code scanning alerts - Pull request check failures for code scanning alerts].
=== "With annotations"
A simpler (but more limited) way to use `zizmor` in GitHub Actions is
with annotations, which `zizmor` can produce via `--format=github`.
This is a good option if:
1. You don't have Advanced Security (or you don't want to use it)
1. You don't want to run `zizmor` with `security-events: write`
```yaml title="zizmor.yml"
name: GitHub Actions Security Analysis with zizmor 🌈
on:
push:
branches: ["main"]
pull_request:
branches: ["**"]
jobs:
zizmor:
name: zizmor latest via PyPI
runs-on: ubuntu-latest
permissions:
contents: read # only needed for private repos
actions: read # only needed for private repos
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
- name: Run zizmor 🌈
run: uvx zizmor --format=github . # (2)!
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} # (1)!
```
1. Optional: Remove the `env:` block to only run `zizmor`'s offline audits.
2. This installs the [zizmor package from PyPI], since it's pre-compiled
and therefore completes much faster. You could instead compile `zizmor`
within CI/CD with `cargo install zizmor`.
!!! warning
GitHub Actions has a limit of 10 annotations per step.
If your `zizmor` run produces more than 10 findings, only the first 10 will
be rendered; all subsequent findings will be logged in the actions log but
**will not be rendered** as annotations.
[zizmor package from PyPI]: https://pypi.org/p/zizmor
[SARIF]: https://sarifweb.azurewebsites.net/
[GitHub's code scanning functionality]: https://docs.github.com/en/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github
[Workflow Commands for GitHub Actions]: https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/workflow-commands-for-github-actions
[code scanning functionality]: https://docs.github.com/en/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github
[repository workflow scan]: https://github.com/woodruffw/zizmor/blob/main/.github/workflows/zizmor.yml

15
src/main.rs
View file

@ -31,9 +31,8 @@ mod expr;
mod finding;
mod github_api;
mod models;
mod output;
mod registry;
mod render;
mod sarif;
mod state;
mod utils;
@ -81,7 +80,7 @@ struct App {
#[arg(long)]
no_progress: bool,
/// The output format to emit. By default, plain text will be emitted
/// The output format to emit. By default, cargo-style diagnostics will be emitted.
#[arg(long, value_enum, default_value_t)]
format: OutputFormat,
@ -137,10 +136,15 @@ struct App {
#[derive(Debug, Default, Copy, Clone, ValueEnum)]
pub(crate) enum OutputFormat {
/// cargo-style output.
#[default]
Plain,
/// JSON-formatted output.
Json,
/// SARIF-formatted output.
Sarif,
/// GitHub Actions workflow command-formatted output.
Github,
}
#[derive(Debug, Copy, Clone, ValueEnum)]
@ -533,11 +537,12 @@ fn run() -> Result<ExitCode> {
}
match app.format {
OutputFormat::Plain => render::render_findings(&app, &registry, &results),
OutputFormat::Plain => output::plain::render_findings(&app, &registry, &results),
OutputFormat::Json => serde_json::to_writer_pretty(stdout(), &results.findings())?,
OutputFormat::Sarif => {
serde_json::to_writer_pretty(stdout(), &sarif::build(results.findings()))?
serde_json::to_writer_pretty(stdout(), &output::sarif::build(results.findings()))?
}
OutputFormat::Github => output::github::output(stdout(), results.findings())?,
};
if app.no_exit_codes || matches!(app.format, OutputFormat::Sarif) {

64
src/output/github.rs Normal file
View file

@ -0,0 +1,64 @@
//! GitHub workflow command-formatted output.
//!
//! See: <https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions>
use std::io;
use anyhow::Result;
use crate::{Severity, finding::Finding};
impl Severity {
/// Converts a `Severity` to a GitHub Actions command command.
fn as_github_command(&self) -> &str {
// TODO: Does this mapping make sense?
match self {
Severity::Unknown => "notice",
Severity::Informational => "notice",
Severity::Low => "warning",
Severity::Medium => "warning",
Severity::High => "error",
}
}
}
impl Finding<'_> {
fn format_command(&self, sink: &mut impl io::Write) -> Result<()> {
let primary = self
.visible_locations()
.find(|l| l.symbolic.is_primary())
.unwrap();
// NOTE: We intentionally only use the start line, since our spans
// sometimes end at EOF and GitHub's annotations don't handle that
// gracefully.
let filepath = primary.symbolic.key.sarif_path();
let start_line = primary.concrete.location.start_point.row + 1;
let title = self.ident;
let message = format!(
"{filename}:{start_line}: {desc}: {annotation}",
filename = primary.symbolic.key.filename(),
desc = self.desc,
annotation = primary.symbolic.annotation,
);
writeln!(
sink,
"::{} file={filepath},line={start_line},title={title}::{message}",
self.determinations.severity.as_github_command()
)?;
Ok(())
}
}
pub(crate) fn output(sink: impl io::Write, findings: &[Finding]) -> Result<()> {
let mut sink = sink;
for finding in findings {
finding.format_command(&mut sink)?;
}
Ok(())
}

3
src/output/mod.rs Normal file
View file

@ -0,0 +1,3 @@
pub(crate) mod github;
pub(crate) mod plain;
pub(crate) mod sarif;

2
src/render.rs → src/output/plain.rs
View file

@ -1,4 +1,4 @@
//! APIs for rendering zizmor's "plain" (i.e. terminal) output format.
//! "plain" (i.e. cargo-style) output.
use std::collections::{HashMap, hash_map::Entry};

2
src/sarif.rs → src/output/sarif.rs
View file

@ -1,4 +1,4 @@
//! APIs for rendering SARIF outputs.
//! SARIF output.
use std::collections::HashSet;

19
src/utils.rs
View file

@ -234,15 +234,16 @@ mod tests {
fn test_extract_expressions() {
let multiple = r#"echo "OSSL_PATH=${{ github.workspace }}/osslcache/${{ matrix.PYTHON.OPENSSL.TYPE }}-${{ matrix.PYTHON.OPENSSL.VERSION }}-${OPENSSL_HASH}" >> $GITHUB_ENV"#;
for (raw, expected) in &[(
multiple,
[
"${{ github.workspace }}",
"${{ matrix.PYTHON.OPENSSL.TYPE }}",
"${{ matrix.PYTHON.OPENSSL.VERSION }}",
]
.as_slice(),
)] {
{
let (raw, expected) = &(
multiple,
[
"${{ github.workspace }}",
"${{ matrix.PYTHON.OPENSSL.TYPE }}",
"${{ matrix.PYTHON.OPENSSL.VERSION }}",
]
.as_slice(),
);
let exprs = extract_expressions(raw)
.into_iter()
.map(|(e, _)| e.as_curly().to_string())

13
tests/integration/snapshot.rs
View file

@ -30,6 +30,19 @@ fn test_invalid_inputs() -> Result<()> {
Ok(())
}
#[test]
fn test_github_output() -> Result<()> {
insta::assert_snapshot!(
zizmor()
.offline(true)
.input(input_under_test("several-vulnerabilities.yml"))
.args(["--persona=auditor", "--format=github"])
.run()?
);
Ok(())
}
#[test]
fn artipacked() -> Result<()> {
insta::assert_snapshot!(

8
tests/integration/snapshots/integration__snapshot__github_output.snap Normal file
View file

@ -0,0 +1,8 @@
---
source: tests/integration/snapshot.rs
expression: "zizmor().offline(true).input(input_under_test(\"several-vulnerabilities.yml\")).args([\"--persona=auditor\",\n\"--format=github\"]).run()?"
---
::error file=@@INPUT@@,line=5,title=excessive-permissions::several-vulnerabilities.yml:5: overly broad permissions: uses write-all permissions
::error file=@@INPUT@@,line=11,title=excessive-permissions::several-vulnerabilities.yml:11: overly broad permissions: uses write-all permissions
::error file=@@INPUT@@,line=2,title=dangerous-triggers::several-vulnerabilities.yml:2: use of fundamentally insecure workflow trigger: pull_request_target is almost always used insecurely
::error file=@@INPUT@@,line=15,title=template-injection::several-vulnerabilities.yml:15: code injection via template expansion: github.event.pull_request.title may expand into attacker-controllable code

16
tests/integration/test-data/several-vulnerabilities.yml Normal file
View file

@ -0,0 +1,16 @@
name: several vulnerabilities
on:
pull_request_target:
permissions: write-all
jobs:
hackme:
name: hackme
runs-on: ubuntu-latest
permissions: write-all
steps:
- name: hackme
run: |
echo "${{ github.event.pull_request.title }}"