Commit 96f3442ce1
Verified · cmc ci/build: success ci/quality: success ci/test: success
Layout: unified · split
.gitbay/wiki/Admin.org +14
| @@ -477,6 +477,20 @@ the findings replace the repository's last set; =gitbay-bot= keeps a | |||
| 477 | the dependency issues are. An older runner runs the steps and uploads | 477 | the dependency issues are. An older runner runs the steps and uploads |
| 478 | nothing, so the issue stays as it was. | 478 | nothing, so the issue stays as it was. |
| 479 | 479 | ||
| 480 | Not every finding is listed. A rule from one of opengrep-rules' library | ||
| 481 | directories (=python/django=, =javascript/jquery=, | ||
| 482 | =typescript/react/portability/i18next=, …) counts only where a manifest | ||
| 483 | at the scanned commit names that library: =go.mod=, =requirements.txt=, | ||
| 484 | =pyproject.toml=, =package.json= or =Cargo.toml=, in any directory | ||
| 485 | outside =node_modules=, =vendor= and =third_party=. The =yaml/semgrep= | ||
| 486 | lints count only in a repository with a =.semgrep= config. Paths | ||
| 487 | =.gitattributes= marks =linguist-vendored= or =linguist-generated= are | ||
| 488 | set aside, as are the repository's =security-ignore= paths and | ||
| 489 | =security-exclude-rule= rules. The issue says how many were set aside | ||
| 490 | and why. A finding a writer dismisses (=repo security dismiss=, by the | ||
| 491 | fingerprint the issue prints) stays out of the issue across scans until | ||
| 492 | it is reopened; the issue closes when nothing is left open. | ||
| 493 | |||
| 480 | The bundled rules are =opengrep/opengrep-rules=, LGPL-2.1 with the | 494 | The bundled rules are =opengrep/opengrep-rules=, LGPL-2.1 with the |
| 481 | Commons Clause: an instance that sells hosting should supply its own. | 495 | Commons Clause: an instance that sells hosting should supply its own. |
| 482 | 496 | ||
.gitbay/wiki/Architecture/02-Components.org +1 −1
| @@ -38,7 +38,7 @@ the hidden =hook= used by git (=cmd/gitbayd/main.go=, | |||
| 38 | | =internal/mailin=, =internal/imapc=, =internal/mailreply= | Reply by mail: the IMAP client, the reply token, and the processor that posts a reply through =issue comment= / =mr comment=. | | 38 | | =internal/mailin=, =internal/imapc=, =internal/mailreply= | Reply by mail: the IMAP client, the reply token, and the processor that posts a reply through =issue comment= / =mr comment=. | |
| 39 | | =internal/deps= | Dependency manifest and lockfile parsing; registry and OSV checks (opt-in per repository). | | 39 | | =internal/deps= | Dependency manifest and lockfile parsing; registry and OSV checks (opt-in per repository). | |
| 40 | | =internal/leaks= | Credential rules and the blob scan pre-receive and =repo secret-scan= run. | | 40 | | =internal/leaks= | Credential rules and the blob scan pre-receive and =repo secret-scan= run. | |
| 41 | | =internal/codescan= | SARIF from the code scan, and the findings issue's body. | | 41 | | =internal/codescan= | SARIF from the code scan, what a repository sets aside, the findings issue. | |
| 42 | | =internal/botissue= | The issues gitbay-bot keeps: open on first report, rewrite on change, close when empty. | | 42 | | =internal/botissue= | The issues gitbay-bot keeps: open on first report, rewrite on change, close when empty. | |
| 43 | | =internal/config= | Configuration load and validation. | | 43 | | =internal/config= | Configuration load and validation. | |
| 44 | | =internal/web= | Embedded templates, stylesheet and fonts. | | 44 | | =internal/web= | Embedded templates, stylesheet and fonts. | |
.gitbay/wiki/Architecture/06-Data-and-Cryptography.org +1 −1
| @@ -20,7 +20,7 @@ content (as confidential as the repository), *O* operational. | |||
| 20 | | Signatures | =commit_signatures=, =settings.key_epoch= | O | verification cache | | 20 | | Signatures | =commit_signatures=, =settings.key_epoch= | O | verification cache | |
| 21 | | Audit and feed | =audit_log=, =events= | O, P | actor ids, pruned argv, fingerprints and IPs in some audit rows, a hash chain (=prev_hash=, =hash=) | | 21 | | Audit and feed | =audit_log=, =events= | O, P | actor ids, pruned argv, fingerprints and IPs in some audit rows, a hash chain (=prev_hash=, =hash=) | |
| 22 | | Dependencies | =dep_checks=, =dep_reports=, =dep_advisories= | O | | 22 | | Dependencies | =dep_checks=, =dep_reports=, =dep_advisories= | O | |
| 23 | | Code scanning | =security_scans=, =security_findings= | O | | | 23 | | Code scanning | =security_scans=, =security_findings=, =security_dismissals= | O | | |
| 24 | 24 | ||
| 25 | Outside the database: | 25 | Outside the database: |
| 26 | 26 | ||
.gitbay/wiki/Parity.org +2
| @@ -249,6 +249,8 @@ there shows the app's rendering rather than the one the web page shows. | |||
| 249 | | code scanning on/off | yes | yes | yes | | 249 | | code scanning on/off | yes | yes | yes | |
| 250 | | code scanning status | yes | yes | yes | | 250 | | code scanning status | yes | yes | yes | |
| 251 | | code scan now | yes | no | yes | | 251 | | code scan now | yes | no | yes | |
| 252 | | code scan dismiss, reopen | yes | yes | no | | ||
| 253 | | code scan ignores, rules | yes | yes | no | | ||
| 252 | | dependency checks on/off | yes | yes | yes | | 254 | | dependency checks on/off | yes | yes | yes | |
| 253 | | dependency status | yes | yes | yes | | 255 | | dependency status | yes | yes | yes | |
| 254 | | delete, transfer | yes | yes | yes | | 256 | | delete, transfer | yes | yes | yes | |
.gitbay/wiki/Users.org +4
| @@ -243,6 +243,10 @@ gitbay repo settings secret-scan-allow you/project 'testdata/**' # paths whose | |||
| 243 | gitbay repo secret-scan you/project # scan the whole history | 243 | gitbay repo secret-scan you/project # scan the whole history |
| 244 | gitbay repo security enable you/project # scheduled code scan, where the instance offers one | 244 | gitbay repo security enable you/project # scheduled code scan, where the instance offers one |
| 245 | gitbay repo security status you/project # its findings, also kept in an issue | 245 | gitbay repo security status you/project # its findings, also kept in an issue |
| 246 | gitbay repo security dismiss you/project 3f9a0c1d2e --reason false-positive # or wont-fix, test | ||
| 247 | gitbay repo security reopen you/project 3f9a0c1d2e | ||
| 248 | gitbay repo settings security-ignore you/project 'testdata/**' # paths set aside from the scan | ||
| 249 | gitbay repo settings security-exclude-rule you/project swift.lang.crypto.insecure-random | ||
| 246 | gitbay repo settings git-daemon you/project on # expose over git:// | 250 | gitbay repo settings git-daemon you/project on # expose over git:// |
| 247 | gitbay repo topics add you/project cli forge # free-form tags, shown on the web | 251 | gitbay repo topics add you/project cli forge # free-form tags, shown on the web |
| 248 | gitbay repo search forge # find repos by name/description/topic | 252 | gitbay repo search forge # find repos by name/description/topic |
cmd/gitbay/main.go +4
| @@ -602,6 +602,8 @@ func repoCmd() *cobra.Command { | |||
| 602 | pass("disable", passOpts{server: []string{"repo", "security", "disable"}, needsRepo: true}), | 602 | pass("disable", passOpts{server: []string{"repo", "security", "disable"}, needsRepo: true}), |
| 603 | pass("run", passOpts{server: []string{"repo", "security", "run"}, needsRepo: true}), | 603 | pass("run", passOpts{server: []string{"repo", "security", "run"}, needsRepo: true}), |
| 604 | pass("status", passOpts{server: []string{"repo", "security", "status"}, needsRepo: true}), | 604 | pass("status", passOpts{server: []string{"repo", "security", "status"}, needsRepo: true}), |
| 605 | pass("dismiss", passOpts{server: []string{"repo", "security", "dismiss"}, needsRepo: true}), | ||
| 606 | pass("reopen", passOpts{server: []string{"repo", "security", "reopen"}, needsRepo: true}), | ||
| 605 | ), | 607 | ), |
| 606 | group("deps", "check dependencies against upstream registries and OSV", | 608 | group("deps", "check dependencies against upstream registries and OSV", |
| 607 | pass("enable", passOpts{server: []string{"repo", "deps", "enable"}, needsRepo: true}), | 609 | pass("enable", passOpts{server: []string{"repo", "deps", "enable"}, needsRepo: true}), |
| @@ -645,6 +647,8 @@ func repoCmd() *cobra.Command { | |||
| 645 | pass("require-signed", passOpts{server: []string{"repo", "settings", "require-signed"}, needsRepo: true}), | 647 | pass("require-signed", passOpts{server: []string{"repo", "settings", "require-signed"}, needsRepo: true}), |
| 646 | pass("secret-scan", passOpts{server: []string{"repo", "settings", "secret-scan"}, needsRepo: true}), | 648 | pass("secret-scan", passOpts{server: []string{"repo", "settings", "secret-scan"}, needsRepo: true}), |
| 647 | pass("secret-scan-allow", passOpts{server: []string{"repo", "settings", "secret-scan-allow"}, needsRepo: true}), | 649 | pass("secret-scan-allow", passOpts{server: []string{"repo", "settings", "secret-scan-allow"}, needsRepo: true}), |
| 650 | pass("security-ignore", passOpts{server: []string{"repo", "settings", "security-ignore"}, needsRepo: true}), | ||
| 651 | pass("security-exclude-rule", passOpts{server: []string{"repo", "settings", "security-exclude-rule"}, needsRepo: true}), | ||
| 648 | pass("require-mr", passOpts{server: []string{"repo", "settings", "require-mr"}, needsRepo: true}), | 652 | pass("require-mr", passOpts{server: []string{"repo", "settings", "require-mr"}, needsRepo: true}), |
| 649 | pass("description", passOpts{server: []string{"repo", "settings", "description"}, needsRepo: true}), | 653 | pass("description", passOpts{server: []string{"repo", "settings", "description"}, needsRepo: true}), |
| 650 | pass("website", passOpts{server: []string{"repo", "settings", "website"}, needsRepo: true}), | 654 | pass("website", passOpts{server: []string{"repo", "settings", "website"}, needsRepo: true}), |
cmd/gitbay/summaries_gen.go +266 −262
| @@ -3,266 +3,270 @@ | |||
| 3 | package main | 3 | package main |
| 4 | 4 | ||
| 5 | var summaries = map[string]string{ | 5 | var summaries = map[string]string{ |
| 6 | "account delete": "delete your account: mails a link, then purges seven days after it is opened", | 6 | "account delete": "delete your account: mails a link, then purges seven days after it is opened", |
| 7 | "account export": "write your account bundle (profile, repos, issues, MRs) as JSON", | 7 | "account export": "write your account bundle (profile, repos, issues, MRs) as JSON", |
| 8 | "account import-bundle": "replay an account bundle (see gitbay migrate)", | 8 | "account import-bundle": "replay an account bundle (see gitbay migrate)", |
| 9 | "admin email verify": "mark an address verified by admin assertion", | 9 | "admin email verify": "mark an address verified by admin assertion", |
| 10 | "admin invite": "issue a registration invite and mail its code", | 10 | "admin invite": "issue a registration invite and mail its code", |
| 11 | "admin mail inbound check": "connect to the reply mailbox read-only and report what is waiting", | 11 | "admin mail inbound check": "connect to the reply mailbox read-only and report what is waiting", |
| 12 | "admin mr prune": "drop merged or closed MRs' head refs and the objects only they kept, e.g. after a history rewrite (instance admins; audited)", | 12 | "admin mr prune": "drop merged or closed MRs' head refs and the objects only they kept, e.g. after a history rewrite (instance admins; audited)", |
| 13 | "admin org limits": "show or set an organization's repository and storage caps (instance admins)", | 13 | "admin org limits": "show or set an organization's repository and storage caps (instance admins)", |
| 14 | "admin repo archive": "archive any repository (instance admins; audited)", | 14 | "admin repo archive": "archive any repository (instance admins; audited)", |
| 15 | "admin repo delete": "delete any repository (instance admins; audited)", | 15 | "admin repo delete": "delete any repository (instance admins; audited)", |
| 16 | "admin repo list": "list every repository with size and last push (instance admins)", | 16 | "admin repo list": "list every repository with size and last push (instance admins)", |
| 17 | "admin repo unarchive": "unarchive any repository (instance admins; audited)", | 17 | "admin repo unarchive": "unarchive any repository (instance admins; audited)", |
| 18 | "admin repo visibility": "set any repository's visibility (instance admins; audited)", | 18 | "admin repo visibility": "set any repository's visibility (instance admins; audited)", |
| 19 | "admin runners forget": "alias of admin runners remove", | 19 | "admin runners forget": "alias of admin runners remove", |
| 20 | "admin runners remove": "drop a key's runner heartbeat row, e.g. one that polled once by mistake (instance admins)", | 20 | "admin runners remove": "drop a key's runner heartbeat row, e.g. one that polled once by mistake (instance admins)", |
| 21 | "admin runners": "the build queue and runner accounts: last poll, scope, the build each holds (instance admins)", | 21 | "admin runners": "the build queue and runner accounts: last poll, scope, the build each holds (instance admins)", |
| 22 | "admin stats": "instance statistics: counts and per-repository disk usage", | 22 | "admin stats": "instance statistics: counts and per-repository disk usage", |
| 23 | "admin symbols reindex": "rebuild a repository's symbol index, even when its tree is already indexed (instance admins)", | 23 | "admin symbols reindex": "rebuild a repository's symbol index, even when its tree is already indexed (instance admins)", |
| 24 | "admin user create": "create an account, optionally with a key and a verified address (instance admins)", | 24 | "admin user create": "create an account, optionally with a key and a verified address (instance admins)", |
| 25 | "admin user delete": "delete an account that anchors nothing (keys, emails and sessions go with it)", | 25 | "admin user delete": "delete an account that anchors nothing (keys, emails and sessions go with it)", |
| 26 | "admin user demote": "remove instance admin from an account (never the last one)", | 26 | "admin user demote": "remove instance admin from an account (never the last one)", |
| 27 | "admin user disable": "suspend an account: SSH, web sessions and API tokens refused until re-enabled", | 27 | "admin user disable": "suspend an account: SSH, web sessions and API tokens refused until re-enabled", |
| 28 | "admin user enable": "restore a suspended account", | 28 | "admin user enable": "restore a suspended account", |
| 29 | "admin user limits": "show or set an account's repository, storage and organization caps (instance admins)", | 29 | "admin user limits": "show or set an account's repository, storage and organization caps (instance admins)", |
| 30 | "admin user list": "list accounts (instance admins)", | 30 | "admin user list": "list accounts (instance admins)", |
| 31 | "admin user promote": "make an account an instance admin", | 31 | "admin user promote": "make an account an instance admin", |
| 32 | "admin user show": "show an account: keys, emails, orgs, tokens, sessions (instance admins)", | 32 | "admin user show": "show an account: keys, emails, orgs, tokens, sessions (instance admins)", |
| 33 | "audit": "instance audit log (admins)", | 33 | "audit": "instance audit log (admins)", |
| 34 | "build cancel": "withdraw a queued build before a runner claims it", | 34 | "build cancel": "withdraw a queued build before a runner claims it", |
| 35 | "build jobs": "list the jobs a trigger can name", | 35 | "build jobs": "list the jobs a trigger can name", |
| 36 | "build list": "list recent builds", | 36 | "build list": "list recent builds", |
| 37 | "build log": "print a build's log, or follow it until the build ends", | 37 | "build log": "print a build's log, or follow it until the build ends", |
| 38 | "build show": "show one build", | 38 | "build show": "show one build", |
| 39 | "build trigger": "queue a job now (scheduled or not)", | 39 | "build trigger": "queue a job now (scheduled or not)", |
| 40 | "dashboard": "one read for the account dashboard: review queue, assigned and open work, pins, activity, builds", | 40 | "dashboard": "one read for the account dashboard: review queue, assigned and open work, pins, activity, builds", |
| 41 | "email add": "add an address and mail a verification code", | 41 | "email add": "add an address and mail a verification code", |
| 42 | "email list": "list the addresses on your account", | 42 | "email list": "list the addresses on your account", |
| 43 | "email primary": "make a verified address the primary", | 43 | "email primary": "make a verified address the primary", |
| 44 | "email remove": "remove an address; not the primary, nor the last verified one", | 44 | "email remove": "remove an address; not the primary, nor the last verified one", |
| 45 | "email verify": "confirm a verification code", | 45 | "email verify": "confirm a verification code", |
| 46 | "explore": "list public repositories", | 46 | "explore": "list public repositories", |
| 47 | "feed": "activity on repositories you can reach", | 47 | "feed": "activity on repositories you can reach", |
| 48 | "help": "list available commands", | 48 | "help": "list available commands", |
| 49 | "issue assign": "add or remove assignees", | 49 | "issue assign": "add or remove assignees", |
| 50 | "issue close": "close an issue", | 50 | "issue close": "close an issue", |
| 51 | "issue comment": "add a comment", | 51 | "issue comment": "add a comment", |
| 52 | "issue create": "open an issue", | 52 | "issue create": "open an issue", |
| 53 | "issue edit": "edit title or body", | 53 | "issue edit": "edit title or body", |
| 54 | "issue label": "add or remove labels", | 54 | "issue label": "add or remove labels", |
| 55 | "issue list": "list issues", | 55 | "issue list": "list issues", |
| 56 | "issue milestone": "set or clear an issue's milestone", | 56 | "issue milestone": "set or clear an issue's milestone", |
| 57 | "issue react": "react to an issue or one of its comments", | 57 | "issue react": "react to an issue or one of its comments", |
| 58 | "issue reopen": "reopen an issue", | 58 | "issue reopen": "reopen an issue", |
| 59 | "issue show": "show an issue with comments", | 59 | "issue show": "show an issue with comments", |
| 60 | "issue templates": "list issue templates (.gitbay/issue-template*.md)", | 60 | "issue templates": "list issue templates (.gitbay/issue-template*.md)", |
| 61 | "keys add": "register an SSH public key (authorized_keys format)", | 61 | "keys add": "register an SSH public key (authorized_keys format)", |
| 62 | "keys label": "name a key; an empty label clears it", | 62 | "keys label": "name a key; an empty label clears it", |
| 63 | "keys list": "list registered SSH keys", | 63 | "keys list": "list registered SSH keys", |
| 64 | "keys remove": "remove an SSH key by fingerprint", | 64 | "keys remove": "remove an SSH key by fingerprint", |
| 65 | "label list": "list a repository's labels with colour and use", | 65 | "label list": "list a repository's labels with colour and use", |
| 66 | "label remove": "remove a label from the repository and from every issue and merge request", | 66 | "label remove": "remove a label from the repository and from every issue and merge request", |
| 67 | "label set": "create a label or set its colour", | 67 | "label set": "create a label or set its colour", |
| 68 | "milestone close": "close a milestone", | 68 | "milestone close": "close a milestone", |
| 69 | "milestone create": "create a milestone", | 69 | "milestone create": "create a milestone", |
| 70 | "milestone list": "list milestones with progress", | 70 | "milestone list": "list milestones with progress", |
| 71 | "milestone reopen": "reopen a milestone", | 71 | "milestone reopen": "reopen a milestone", |
| 72 | "mr apply-suggestion": "commit a review thread's suggestion to the source branch", | 72 | "mr apply-suggestion": "commit a review thread's suggestion to the source branch", |
| 73 | "mr close": "close without merging", | 73 | "mr close": "close without merging", |
| 74 | "mr comment": "add a comment", | 74 | "mr comment": "add a comment", |
| 75 | "mr create": "open a merge request", | 75 | "mr create": "open a merge request", |
| 76 | "mr diff": "show the diff", | 76 | "mr diff": "show the diff", |
| 77 | "mr diff-comment": "comment on a diff line", | 77 | "mr diff-comment": "comment on a diff line", |
| 78 | "mr draft": "mark a merge request as work in progress", | 78 | "mr draft": "mark a merge request as work in progress", |
| 79 | "mr edit": "edit title or body", | 79 | "mr edit": "edit title or body", |
| 80 | "mr label": "add or remove labels", | 80 | "mr label": "add or remove labels", |
| 81 | "mr list": "list merge requests", | 81 | "mr list": "list merge requests", |
| 82 | "mr merge": "merge", | 82 | "mr merge": "merge", |
| 83 | "mr milestone": "set or clear an MR's milestone", | 83 | "mr milestone": "set or clear an MR's milestone", |
| 84 | "mr range-diff": "what changed between two revisions of a merge request", | 84 | "mr range-diff": "what changed between two revisions of a merge request", |
| 85 | "mr react": "react to a merge request or one of its comments", | 85 | "mr react": "react to a merge request or one of its comments", |
| 86 | "mr ready": "take the draft mark off, so it can merge", | 86 | "mr ready": "take the draft mark off, so it can merge", |
| 87 | "mr resolve": "resolve a review thread", | 87 | "mr resolve": "resolve a review thread", |
| 88 | "mr retarget": "retarget onto another branch", | 88 | "mr retarget": "retarget onto another branch", |
| 89 | "mr review request": "ask specific people for a review", | 89 | "mr review request": "ask specific people for a review", |
| 90 | "mr review": "record a review verdict", | 90 | "mr review": "record a review verdict", |
| 91 | "mr revisions": "the heads a merge request has had", | 91 | "mr revisions": "the heads a merge request has had", |
| 92 | "mr show": "show a merge request", | 92 | "mr show": "show a merge request", |
| 93 | "mr threads": "review threads on an MR", | 93 | "mr threads": "review threads on an MR", |
| 94 | "mr unresolve": "reopen a review thread", | 94 | "mr unresolve": "reopen a review thread", |
| 95 | "notifications device add": "register an Apple device for push, token on stdin", | 95 | "notifications device add": "register an Apple device for push, token on stdin", |
| 96 | "notifications device list": "your registered devices", | 96 | "notifications device list": "your registered devices", |
| 97 | "notifications device remove": "deregister a device", | 97 | "notifications device remove": "deregister a device", |
| 98 | "notifications list": "your notification inbox, newest first", | 98 | "notifications list": "your notification inbox, newest first", |
| 99 | "notifications read": "mark notifications read", | 99 | "notifications read": "mark notifications read", |
| 100 | "notifications settings mail": "activity by mail as well as the inbox (login links are unaffected)", | 100 | "notifications settings mail": "activity by mail as well as the inbox (login links are unaffected)", |
| 101 | "notifications settings push": "activity on your registered devices as well as the inbox", | 101 | "notifications settings push": "activity on your registered devices as well as the inbox", |
| 102 | "notifications settings reply": "reply to issue and merge request mail to comment", | 102 | "notifications settings reply": "reply to issue and merge request mail to comment", |
| 103 | "notifications settings show": "your notification preferences", | 103 | "notifications settings show": "your notification preferences", |
| 104 | "notifications settings watch": "every issue and merge request on repositories you can write to", | 104 | "notifications settings watch": "every issue and merge request on repositories you can write to", |
| 105 | "org create": "create an organization (you become its first admin)", | 105 | "org create": "create an organization (you become its first admin)", |
| 106 | "org delete": "delete an empty organization", | 106 | "org delete": "delete an empty organization", |
| 107 | "org label list": "list an org's labels with use across the repositories you can read", | 107 | "org label list": "list an org's labels with use across the repositories you can read", |
| 108 | "org label remove": "remove an org label from the org and from every issue under it", | 108 | "org label remove": "remove an org label from the org and from every issue under it", |
| 109 | "org label set": "create an org label every org repository sees, or set its colour; folds in same-named repo labels", | 109 | "org label set": "create an org label every org repository sees, or set its colour; folds in same-named repo labels", |
| 110 | "org list": "list organizations you belong to", | 110 | "org list": "list organizations you belong to", |
| 111 | "org members add": "add or update a member", | 111 | "org members add": "add or update a member", |
| 112 | "org members list": "list members", | 112 | "org members list": "list members", |
| 113 | "org members remove": "remove a member", | 113 | "org members remove": "remove a member", |
| 114 | "org milestone close": "close an org milestone", | 114 | "org milestone close": "close an org milestone", |
| 115 | "org milestone create": "create an org milestone spanning every org repository; folds in same-titled repo milestones", | 115 | "org milestone create": "create an org milestone spanning every org repository; folds in same-titled repo milestones", |
| 116 | "org milestone list": "list an org's milestones with progress across the repositories you can read", | 116 | "org milestone list": "list an org's milestones with progress across the repositories you can read", |
| 117 | "org milestone reopen": "reopen an org milestone", | 117 | "org milestone reopen": "reopen an org milestone", |
| 118 | "org profile": "show or set an org's profile", | 118 | "org profile": "show or set an org's profile", |
| 119 | "org rename": "rename an organization", | 119 | "org rename": "rename an organization", |
| 120 | "org settings members-role": "role plain membership implies on every org repo", | 120 | "org settings members-role": "role plain membership implies on every org repo", |
| 121 | "org show": "show an organization and its members", | 121 | "org show": "show an organization and its members", |
| 122 | "org team add": "add org members to a team", | 122 | "org team add": "add org members to a team", |
| 123 | "org team create": "create a team", | 123 | "org team create": "create a team", |
| 124 | "org team delete": "delete a team (its grants with it)", | 124 | "org team delete": "delete a team (its grants with it)", |
| 125 | "org team grant": "grant a team a role on an org repo", | 125 | "org team grant": "grant a team a role on an org repo", |
| 126 | "org team list": "list an org's teams", | 126 | "org team list": "list an org's teams", |
| 127 | "org team remove": "remove members from a team", | 127 | "org team remove": "remove members from a team", |
| 128 | "org team revoke": "revoke a team's grant", | 128 | "org team revoke": "revoke a team's grant", |
| 129 | "org team show": "show a team's members and grants", | 129 | "org team show": "show a team's members and grants", |
| 130 | "pgp add": "register an OpenPGP public key (armored)", | 130 | "pgp add": "register an OpenPGP public key (armored)", |
| 131 | "pgp list": "list registered OpenPGP keys", | 131 | "pgp list": "list registered OpenPGP keys", |
| 132 | "pgp remove": "remove an OpenPGP key by fingerprint", | 132 | "pgp remove": "remove an OpenPGP key by fingerprint", |
| 133 | "profile set": "set your profile", | 133 | "profile set": "set your profile", |
| 134 | "profile show": "show a user's or org's profile", | 134 | "profile show": "show a user's or org's profile", |
| 135 | "query list": "list your saved queries", | 135 | "query list": "list your saved queries", |
| 136 | "query pin": "show a saved query on your dashboard", | 136 | "query pin": "show a saved query on your dashboard", |
| 137 | "query remove": "delete a saved query", | 137 | "query remove": "delete a saved query", |
| 138 | "query run": "list the issues and merge requests a saved query matches", | 138 | "query run": "list the issues and merge requests a saved query matches", |
| 139 | "query save": "save an issue and merge request query across repositories under a name", | 139 | "query save": "save an issue and merge request query across repositories under a name", |
| 140 | "query show": "show a saved query and how many rows it matches", | 140 | "query show": "show a saved query and how many rows it matches", |
| 141 | "query unpin": "take a saved query off your dashboard", | 141 | "query unpin": "take a saved query off your dashboard", |
| 142 | "register": "create an account (only meaningful for unregistered keys)", | 142 | "register": "create an account (only meaningful for unregistered keys)", |
| 143 | "release asset add": "upload an asset from stdin", | 143 | "release asset add": "upload an asset from stdin", |
| 144 | "release asset get": "write an asset to stdout", | 144 | "release asset get": "write an asset to stdout", |
| 145 | "release asset remove": "remove an asset", | 145 | "release asset remove": "remove an asset", |
| 146 | "release create": "create a release on a tag", | 146 | "release create": "create a release on a tag", |
| 147 | "release delete": "delete a release and its assets", | 147 | "release delete": "delete a release and its assets", |
| 148 | "release edit": "update a release's title and notes", | 148 | "release edit": "update a release's title and notes", |
| 149 | "release list": "list releases", | 149 | "release list": "list releases", |
| 150 | "release notes": "draft notes: one line per merge request since the previous release", | 150 | "release notes": "draft notes: one line per merge request since the previous release", |
| 151 | "release show": "show a release with assets", | 151 | "release show": "show a release with assets", |
| 152 | "repo access grant": "grant access", | 152 | "repo access grant": "grant access", |
| 153 | "repo access list": "list who can reach the repository, with the role and where it comes from", | 153 | "repo access list": "list who can reach the repository, with the role and where it comes from", |
| 154 | "repo access revoke": "revoke access", | 154 | "repo access revoke": "revoke access", |
| 155 | "repo archive": "archive a repository (read-only: pushes and issue/MR writes refused)", | 155 | "repo archive": "archive a repository (read-only: pushes and issue/MR writes refused)", |
| 156 | "repo blame": "attribute lines to commits", | 156 | "repo blame": "attribute lines to commits", |
| 157 | "repo bookmark": "bookmark a repository to come back to", | 157 | "repo bookmark": "bookmark a repository to come back to", |
| 158 | "repo bookmarks": "list the repositories you have bookmarked", | 158 | "repo bookmarks": "list the repositories you have bookmarked", |
| 159 | "repo cat": "read a file", | 159 | "repo cat": "read a file", |
| 160 | "repo commit": "show one commit with its patch", | 160 | "repo commit": "show one commit with its patch", |
| 161 | "repo commit-file": "write a file and commit it", | 161 | "repo commit-file": "write a file and commit it", |
| 162 | "repo create": "create a repository", | 162 | "repo create": "create a repository", |
| 163 | "repo definitions": "where each name a file uses is defined, from the symbol index", | 163 | "repo definitions": "where each name a file uses is defined, from the symbol index", |
| 164 | "repo delete": "delete a repository", | 164 | "repo delete": "delete a repository", |
| 165 | "repo deploy-key add": "bind a read-only (or --rw) key to one repository", | 165 | "repo deploy-key add": "bind a read-only (or --rw) key to one repository", |
| 166 | "repo deploy-key list": "list deploy keys", | 166 | "repo deploy-key list": "list deploy keys", |
| 167 | "repo deploy-key remove": "remove a deploy key", | 167 | "repo deploy-key remove": "remove a deploy key", |
| 168 | "repo deps disable": "stop checking dependencies", | 168 | "repo deps disable": "stop checking dependencies", |
| 169 | "repo deps enable": "check dependencies for updates and advisories", | 169 | "repo deps enable": "check dependencies for updates and advisories", |
| 170 | "repo deps status": "show dependency check state", | 170 | "repo deps status": "show dependency check state", |
| 171 | "repo diff": "the patch between two refs, from their merge base", | 171 | "repo diff": "the patch between two refs, from their merge base", |
| 172 | "repo domain add": "claim a custom pages domain (verify with a DNS TXT record)", | 172 | "repo domain add": "claim a custom pages domain (verify with a DNS TXT record)", |
| 173 | "repo domain list": "list custom pages domains", | 173 | "repo domain list": "list custom pages domains", |
| 174 | "repo domain remove": "remove a custom pages domain", | 174 | "repo domain remove": "remove a custom pages domain", |
| 175 | "repo domain verify": "check the DNS challenge and activate a claim", | 175 | "repo domain verify": "check the DNS challenge and activate a claim", |
| 176 | "repo download": "write a tar.gz of a ref to stdout", | 176 | "repo download": "write a tar.gz of a ref to stdout", |
| 177 | "repo fork": "fork a repository under your account", | 177 | "repo fork": "fork a repository under your account", |
| 178 | "repo grep": "search file contents", | 178 | "repo grep": "search file contents", |
| 179 | "repo import": "server-side mirror of a foreign repository", | 179 | "repo import": "server-side mirror of a foreign repository", |
| 180 | "repo import-issues": "import issue and PR history from GitHub or Forgejo", | 180 | "repo import-issues": "import issue and PR history from GitHub or Forgejo", |
| 181 | "repo list": "list repositories you own or can access", | 181 | "repo list": "list repositories you own or can access", |
| 182 | "repo log": "commit log with signature states", | 182 | "repo log": "commit log with signature states", |
| 183 | "repo mirror add": "mirror to or from a remote", | 183 | "repo mirror add": "mirror to or from a remote", |
| 184 | "repo mirror list": "list mirrors with sync status", | 184 | "repo mirror list": "list mirrors with sync status", |
| 185 | "repo mirror remove": "remove a mirror", | 185 | "repo mirror remove": "remove a mirror", |
| 186 | "repo mirror sync": "schedule an immediate sync", | 186 | "repo mirror sync": "schedule an immediate sync", |
| 187 | "repo mute": "mute a repository, including work you are part of", | 187 | "repo mute": "mute a repository, including work you are part of", |
| 188 | "repo pin": "pin a repository to your dashboard", | 188 | "repo pin": "pin a repository to your dashboard", |
| 189 | "repo readme": "print a repository's README", | 189 | "repo readme": "print a repository's README", |
| 190 | "repo refs": "list branches and tags", | 190 | "repo refs": "list branches and tags", |
| 191 | "repo rename": "rename a repository", | 191 | "repo rename": "rename a repository", |
| 192 | "repo runner add": "attach a runner's public key to a repository", | 192 | "repo runner add": "attach a runner's public key to a repository", |
| 193 | "repo runner list": "list the runners attached to a repository", | 193 | "repo runner list": "list the runners attached to a repository", |
| 194 | "repo runner remove": "detach a runner from a repository", | 194 | "repo runner remove": "detach a runner from a repository", |
| 195 | "repo search": "find repositories by name, description, or topic", | 195 | "repo search": "find repositories by name, description, or topic", |
| 196 | "repo secret list": "list build secret names", | 196 | "repo secret list": "list build secret names", |
| 197 | "repo secret remove": "remove a build secret", | 197 | "repo secret remove": "remove a build secret", |
| 198 | "repo secret set": "set a build secret", | 198 | "repo secret set": "set a build secret", |
| 199 | "repo secret-scan": "scan the whole history for credentials", | 199 | "repo secret-scan": "scan the whole history for credentials", |
| 200 | "repo security disable": "stop code scanning", | 200 | "repo security disable": "stop code scanning", |
| 201 | "repo security enable": "scan the default branch on the instance's code scanning schedule", | 201 | "repo security dismiss": "dismiss code scanning findings, which then stay out of the issue", |
| 202 | "repo security run": "queue a code scan of the default branch now", | 202 | "repo security enable": "scan the default branch on the instance's code scanning schedule", |
| 203 | "repo security status": "show code scanning state and the last scan's findings", | 203 | "repo security reopen": "take back dismissals of code scanning findings", |
| 204 | "repo settings default-branch": "set the default branch", | 204 | "repo security run": "queue a code scan of the default branch now", |
| 205 | "repo settings description": "set the repository description", | 205 | "repo security status": "show code scanning state and the last scan's findings", |
| 206 | "repo settings git-daemon": "expose over git://", | 206 | "repo settings default-branch": "set the default branch", |
| 207 | "repo settings protect": "protect a branch", | 207 | "repo settings description": "set the repository description", |
| 208 | "repo settings protect-tag": "protect tags matching a glob (created once, never moved or deleted)", | 208 | "repo settings git-daemon": "expose over git://", |
| 209 | "repo settings require-approvals": "require N fresh approvals to merge", | 209 | "repo settings protect": "protect a branch", |
| 210 | "repo settings require-checks": "gate merges on green statuses", | 210 | "repo settings protect-tag": "protect tags matching a glob (created once, never moved or deleted)", |
| 211 | "repo settings require-codeowners": "require an owner's approval for every file CODEOWNERS covers", | 211 | "repo settings require-approvals": "require N fresh approvals to merge", |
| 212 | "repo settings require-contexts": "name the statuses the checks gate waits for, and turn the gate on", | 212 | "repo settings require-checks": "gate merges on green statuses", |
| 213 | "repo settings require-mr": "protected branches take changes through merge requests only", | 213 | "repo settings require-codeowners": "require an owner's approval for every file CODEOWNERS covers", |
| 214 | "repo settings require-resolved": "require all review threads resolved to merge", | 214 | "repo settings require-contexts": "name the statuses the checks gate waits for, and turn the gate on", |
| 215 | "repo settings require-signed": "require verified commit signatures", | 215 | "repo settings require-mr": "protected branches take changes through merge requests only", |
| 216 | "repo settings secret-scan": "scan pushes for credentials: off, warn the pusher, or block the push", | 216 | "repo settings require-resolved": "require all review threads resolved to merge", |
| 217 | "repo settings secret-scan-allow": "paths whose secret scan findings are ignored", | 217 | "repo settings require-signed": "require verified commit signatures", |
| 218 | "repo settings show": "show settings", | 218 | "repo settings secret-scan": "scan pushes for credentials: off, warn the pusher, or block the push", |
| 219 | "repo settings unprotect": "unprotect a branch", | 219 | "repo settings secret-scan-allow": "paths whose secret scan findings are ignored", |
| 220 | "repo settings unprotect-tag": "drop a protected-tag glob", | 220 | "repo settings security-exclude-rule": "code scanning rules, or families of them, whose findings are set aside", |
| 221 | "repo settings visibility": "set repository visibility", | 221 | "repo settings security-ignore": "paths whose code scanning findings are set aside", |
| 222 | "repo settings website": "set the repository website", | 222 | "repo settings show": "show settings", |
| 223 | "repo show": "show repository details", | 223 | "repo settings unprotect": "unprotect a branch", |
| 224 | "repo symbols": "find where a name is defined, from the default branch's symbol index", | 224 | "repo settings unprotect-tag": "drop a protected-tag glob", |
| 225 | "repo topics add": "add topics", | 225 | "repo settings visibility": "set repository visibility", |
| 226 | "repo topics remove": "remove topics", | 226 | "repo settings website": "set the repository website", |
| 227 | "repo topics": "list topics", | 227 | "repo show": "show repository details", |
| 228 | "repo transfer": "move a repository to another owner", | 228 | "repo symbols": "find where a name is defined, from the default branch's symbol index", |
| 229 | "repo tree": "list a directory", | 229 | "repo topics add": "add topics", |
| 230 | "repo unarchive": "unarchive a repository", | 230 | "repo topics remove": "remove topics", |
| 231 | "repo unbookmark": "remove a bookmark", | 231 | "repo topics": "list topics", |
| 232 | "repo unpin": "unpin a repository", | 232 | "repo transfer": "move a repository to another owner", |
| 233 | "repo unwatch": "back to the default: only work you are part of", | 233 | "repo tree": "list a directory", |
| 234 | "repo watch": "hear about all activity on a repository", | 234 | "repo unarchive": "unarchive a repository", |
| 235 | "runner done": "finish a build", | 235 | "repo unbookmark": "remove a bookmark", |
| 236 | "runner log": "append a build's log from stdin", | 236 | "repo unpin": "unpin a repository", |
| 237 | "runner next": "claim the oldest pending build this key may run (runner protocol)", | 237 | "repo unwatch": "back to the default: only work you are part of", |
| 238 | "runner results": "upload a code scan's SARIF from stdin", | 238 | "repo watch": "hear about all activity on a repository", |
| 239 | "search": "find repositories, issues and merge requests across the instance", | 239 | "runner done": "finish a build", |
| 240 | "snippet create": "create a snippet from one file on stdin", | 240 | "runner log": "append a build's log from stdin", |
| 241 | "snippet delete": "delete a snippet and its files", | 241 | "runner next": "claim the oldest pending build this key may run (runner protocol)", |
| 242 | "snippet edit": "change a snippet's description or visibility", | 242 | "runner results": "upload a code scan's SARIF from stdin", |
| 243 | "snippet file get": "write a snippet file to stdout", | 243 | "search": "find repositories, issues and merge requests across the instance", |
| 244 | "snippet file remove": "remove a file from a snippet", | 244 | "snippet create": "create a snippet from one file on stdin", |
| 245 | "snippet file set": "add a file to a snippet, or replace one, from stdin", | 245 | "snippet delete": "delete a snippet and its files", |
| 246 | "snippet list": "list your snippets, or an owner's public ones", | 246 | "snippet edit": "change a snippet's description or visibility", |
| 247 | "snippet show": "show a snippet's metadata and files", | 247 | "snippet file get": "write a snippet file to stdout", |
| 248 | "status list": "statuses on a commit", | 248 | "snippet file remove": "remove a file from a snippet", |
| 249 | "status set": "report a commit status (CI)", | 249 | "snippet file set": "add a file to a snippet, or replace one, from stdin", |
| 250 | "token create": "mint an API token (shown once)", | 250 | "snippet list": "list your snippets, or an owner's public ones", |
| 251 | "token list": "list API tokens", | 251 | "snippet show": "show a snippet's metadata and files", |
| 252 | "token revoke": "revoke an API token by name", | 252 | "status list": "statuses on a commit", |
| 253 | "web diff set": "draw web diffs unified or side by side", | 253 | "status set": "report a commit status (CI)", |
| 254 | "web diff show": "the diff layout the web UI uses for you", | 254 | "token create": "mint an API token (shown once)", |
| 255 | "web login": "mint a one-time browser login URL", | 255 | "token list": "list API tokens", |
| 256 | "web sessions list": "list your browser sessions", | 256 | "token revoke": "revoke an API token by name", |
| 257 | "web sessions revoke": "end a browser session, or all of them", | 257 | "web diff set": "draw web diffs unified or side by side", |
| 258 | "web theme set": "follow the browser's scheme, or force light or dark", | 258 | "web diff show": "the diff layout the web UI uses for you", |
| 259 | "web theme show": "the colour scheme the web UI uses for you", | 259 | "web login": "mint a one-time browser login URL", |
| 260 | "webhook add": "add a webhook", | 260 | "web sessions list": "list your browser sessions", |
| 261 | "webhook deliveries": "recent deliveries", | 261 | "web sessions revoke": "end a browser session, or all of them", |
| 262 | "webhook list": "list webhooks", | 262 | "web theme set": "follow the browser's scheme, or force light or dark", |
| 263 | "webhook redeliver": "queue a delivery again", | 263 | "web theme show": "the colour scheme the web UI uses for you", |
| 264 | "webhook remove": "remove a webhook", | 264 | "webhook add": "add a webhook", |
| 265 | "whoami": "show the authenticated account", | 265 | "webhook deliveries": "recent deliveries", |
| 266 | "wiki list": "list a repository's wiki pages", | 266 | "webhook list": "list webhooks", |
| 267 | "wiki show": "print a wiki page", | 267 | "webhook redeliver": "queue a delivery again", |
| 268 | "webhook remove": "remove a webhook", | ||
| 269 | "whoami": "show the authenticated account", | ||
| 270 | "wiki list": "list a repository's wiki pages", | ||
| 271 | "wiki show": "print a wiki page", | ||
| 268 | } | 272 | } |
internal/codescan/codescan.go +87 −10
| @@ -21,9 +21,11 @@ import ( | |||
| 21 | // IssueTitle is fixed so the issue is recognizable across scans. | 21 | // IssueTitle is fixed so the issue is recognizable across scans. |
| 22 | const IssueTitle = "Code scanning findings" | 22 | const IssueTitle = "Code scanning findings" |
| 23 | 23 | ||
| 24 | // MaxUpload bounds one upload. MaxFindings bounds what is kept from it. | 24 | // MaxUpload bounds one upload, maxResults what is read from it, and |
| 25 | // MaxFindings what is reported of that once a Scope has set some aside. | ||
| 25 | const ( | 26 | const ( |
| 26 | MaxUpload = 8 << 20 | 27 | MaxUpload = 8 << 20 |
| 28 | maxResults = 10 * MaxFindings | ||
| 27 | MaxFindings = 500 | 29 | MaxFindings = 500 |
| 28 | ) | 30 | ) |
| 29 | 31 | ||
| @@ -61,7 +63,7 @@ type sarif struct { | |||
| 61 | 63 | ||
| 62 | // Parse returns the findings in a SARIF document, deduplicated by | 64 | // Parse returns the findings in a SARIF document, deduplicated by |
| 63 | // fingerprint and sorted by path and line. It reports whether results past | 65 | // fingerprint and sorted by path and line. It reports whether results past |
| 64 | // MaxFindings were dropped. | 66 | // maxResults were dropped. |
| 65 | func Parse(raw []byte) ([]store.SecurityFinding, bool, error) { | 67 | func Parse(raw []byte) ([]store.SecurityFinding, bool, error) { |
| 66 | var doc sarif | 68 | var doc sarif |
| 67 | if err := json.Unmarshal(raw, &doc); err != nil { | 69 | if err := json.Unmarshal(raw, &doc); err != nil { |
| @@ -110,7 +112,7 @@ func Parse(raw []byte) ([]store.SecurityFinding, bool, error) { | |||
| 110 | continue | 112 | continue |
| 111 | } | 113 | } |
| 112 | seen[h.place()] = true | 114 | seen[h.place()] = true |
| 113 | if len(out) >= MaxFindings { | 115 | if len(out) >= maxResults { |
| 114 | truncated = true | 116 | truncated = true |
| 115 | break | 117 | break |
| 116 | } | 118 | } |
| @@ -248,6 +250,18 @@ func Same(a, b []store.SecurityFinding) bool { | |||
| 248 | return true | 250 | return true |
| 249 | } | 251 | } |
| 250 | 252 | ||
| 253 | // ByPlace sorts a set by path and line, the way a scan lists it. | ||
| 254 | func ByPlace(fs []store.SecurityFinding) []store.SecurityFinding { | ||
| 255 | out := append([]store.SecurityFinding(nil), fs...) | ||
| 256 | sort.SliceStable(out, func(i, j int) bool { | ||
| 257 | if out[i].Path != out[j].Path { | ||
| 258 | return out[i].Path < out[j].Path | ||
| 259 | } | ||
| 260 | return out[i].Line < out[j].Line | ||
| 261 | }) | ||
| 262 | return out | ||
| 263 | } | ||
| 264 | |||
| 251 | // ByFingerprint sorts a set the way the store returns it, for Same. | 265 | // ByFingerprint sorts a set the way the store returns it, for Same. |
| 252 | func ByFingerprint(fs []store.SecurityFinding) []store.SecurityFinding { | 266 | func ByFingerprint(fs []store.SecurityFinding) []store.SecurityFinding { |
| 253 | out := append([]store.SecurityFinding(nil), fs...) | 267 | out := append([]store.SecurityFinding(nil), fs...) |
| @@ -255,14 +269,25 @@ func ByFingerprint(fs []store.SecurityFinding) []store.SecurityFinding { | |||
| 255 | return out | 269 | return out |
| 256 | } | 270 | } |
| 257 | 271 | ||
| 258 | // Body renders the issue: one list per tool, a line per finding. | 272 | // Report is what the issue says: the findings still open, and how many |
| 259 | func Body(branch string, findings []store.SecurityFinding, truncated bool) string { | 273 | // others were dismissed or set aside. |
| 274 | type Report struct { | ||
| 275 | Repo, Branch string | ||
| 276 | Open []store.SecurityFinding | ||
| 277 | Dismissed int | ||
| 278 | Aside Aside | ||
| 279 | Truncated bool | ||
| 280 | } | ||
| 281 | |||
| 282 | // Body renders the issue: one list per tool, a line per finding with its | ||
| 283 | // fingerprint, then what is not listed. | ||
| 284 | func Body(r Report) string { | ||
| 260 | var b strings.Builder | 285 | var b strings.Builder |
| 261 | fmt.Fprintf(&b, "The scheduled code scan of `%s` reported these findings. "+ | 286 | fmt.Fprintf(&b, "The scheduled code scan of `%s` reported these findings. "+ |
| 262 | "This issue is kept up to date as that changes, and closed once a scan reports none.\n", branch) | 287 | "This issue is kept up to date as that changes, and closed once none are open.\n", r.Branch) |
| 263 | byTool := map[string][]store.SecurityFinding{} | 288 | byTool := map[string][]store.SecurityFinding{} |
| 264 | var tools []string | 289 | var tools []string |
| 265 | for _, f := range findings { | 290 | for _, f := range r.Open { |
| 266 | if _, ok := byTool[f.Tool]; !ok { | 291 | if _, ok := byTool[f.Tool]; !ok { |
| 267 | tools = append(tools, f.Tool) | 292 | tools = append(tools, f.Tool) |
| 268 | } | 293 | } |
| @@ -276,11 +301,63 @@ func Body(branch string, findings []store.SecurityFinding, truncated bool) strin | |||
| 276 | } | 301 | } |
| 277 | fmt.Fprintf(&b, "\n### %s\n\n", name) | 302 | fmt.Fprintf(&b, "\n### %s\n\n", name) |
| 278 | for _, f := range byTool[tool] { | 303 | for _, f := range byTool[tool] { |
| 279 | fmt.Fprintf(&b, "- `%s:%d` %s (%s): %s\n", f.Path, f.Line, f.Rule, f.Level, f.Message) | 304 | fmt.Fprintf(&b, "- `%s:%d` %s (%s): %s `%s`\n", f.Path, f.Line, f.Rule, f.Level, f.Message, Short(f.Fingerprint)) |
| 280 | } | 305 | } |
| 281 | } | 306 | } |
| 282 | if truncated { | 307 | if r.Truncated { |
| 283 | fmt.Fprintf(&b, "\nOnly the first %d results are listed.\n", MaxFindings) | 308 | fmt.Fprintf(&b, "\nOnly the first %d findings are listed.\n", MaxFindings) |
| 309 | } | ||
| 310 | var notListed []string | ||
| 311 | if r.Dismissed > 0 { | ||
| 312 | notListed = append(notListed, fmt.Sprintf("%d dismissed", r.Dismissed)) | ||
| 313 | } | ||
| 314 | if r.Aside.Ignored > 0 { | ||
| 315 | notListed = append(notListed, fmt.Sprintf("%d under paths or rules the repository ignores", r.Aside.Ignored)) | ||
| 316 | } | ||
| 317 | if len(r.Aside.Unused) > 0 { | ||
| 318 | notListed = append(notListed, fmt.Sprintf("%d from rules for libraries it does not use (%s)", total(r.Aside.Unused), counts(r.Aside.Unused))) | ||
| 319 | } | ||
| 320 | if len(notListed) > 0 { | ||
| 321 | fmt.Fprintf(&b, "\nNot listed: %s. `repo security status %s` shows them.\n", strings.Join(notListed, "; "), r.Repo) | ||
| 322 | } | ||
| 323 | if len(r.Open) > 0 { | ||
| 324 | fmt.Fprintf(&b, "\nTo dismiss one: `repo security dismiss %s <fingerprint> --reason false-positive|wont-fix|test`.\n", r.Repo) | ||
| 284 | } | 325 | } |
| 285 | return b.String() | 326 | return b.String() |
| 286 | } | 327 | } |
| 328 | |||
| 329 | // Short is the fingerprint as the issue prints it; Dismiss takes it back. | ||
| 330 | func Short(fingerprint string) string { | ||
| 331 | return fingerprint[:min(len(fingerprint), ShortLen)] | ||
| 332 | } | ||
| 333 | |||
| 334 | // ShortLen is how much of a fingerprint is printed and the least that | ||
| 335 | // names one. | ||
| 336 | const ShortLen = 10 | ||
| 337 | |||
| 338 | func total(m map[string]int) int { | ||
| 339 | n := 0 | ||
| 340 | for _, v := range m { | ||
| 341 | n += v | ||
| 342 | } | ||
| 343 | return n | ||
| 344 | } | ||
| 345 | |||
| 346 | // counts lists a map's keys with their counts, most first. | ||
| 347 | func counts(m map[string]int) string { | ||
| 348 | keys := make([]string, 0, len(m)) | ||
| 349 | for k := range m { | ||
| 350 | keys = append(keys, k) | ||
| 351 | } | ||
| 352 | sort.Slice(keys, func(i, j int) bool { | ||
| 353 | if m[keys[i]] != m[keys[j]] { | ||
| 354 | return m[keys[i]] > m[keys[j]] | ||
| 355 | } | ||
| 356 | return keys[i] < keys[j] | ||
| 357 | }) | ||
| 358 | parts := make([]string, len(keys)) | ||
| 359 | for i, k := range keys { | ||
| 360 | parts[i] = fmt.Sprintf("%s %d", k, m[k]) | ||
| 361 | } | ||
| 362 | return strings.Join(parts, ", ") | ||
| 363 | } | ||
internal/codescan/codescan_test.go +2 −2
| @@ -45,7 +45,7 @@ func TestParse(t *testing.T) { | |||
| 45 | func TestParseTruncates(t *testing.T) { | 45 | func TestParseTruncates(t *testing.T) { |
| 46 | var b strings.Builder | 46 | var b strings.Builder |
| 47 | b.WriteString(`{"runs":[{"tool":{"driver":{"name":"t"}},"results":[`) | 47 | b.WriteString(`{"runs":[{"tool":{"driver":{"name":"t"}},"results":[`) |
| 48 | for i := 0; i < MaxFindings+5; i++ { | 48 | for i := 0; i < maxResults+5; i++ { |
| 49 | if i > 0 { | 49 | if i > 0 { |
| 50 | b.WriteString(",") | 50 | b.WriteString(",") |
| 51 | } | 51 | } |
| @@ -53,7 +53,7 @@ func TestParseTruncates(t *testing.T) { | |||
| 53 | } | 53 | } |
| 54 | b.WriteString(`]}]}`) | 54 | b.WriteString(`]}]}`) |
| 55 | got, truncated, err := Parse([]byte(b.String())) | 55 | got, truncated, err := Parse([]byte(b.String())) |
| 56 | if err != nil || !truncated || len(got) != MaxFindings { | 56 | if err != nil || !truncated || len(got) != maxResults { |
| 57 | t.Fatalf("len %d truncated %v err %v", len(got), truncated, err) | 57 | t.Fatalf("len %d truncated %v err %v", len(got), truncated, err) |
| 58 | } | 58 | } |
| 59 | } | 59 | } |
internal/codescan/libraries.go added +115
| @@ -0,0 +1,115 @@ | |||
| 1 | package codescan | ||
| 2 | |||
| 3 | import "strings" | ||
| 4 | |||
| 5 | // library is a family of rules that is about one library: opengrep-rules | ||
| 6 | // keeps those under <language>/<library>/, which a rule id carries as | ||
| 7 | // language.library. Its findings apply only to a repository whose manifests | ||
| 8 | // name one of packages, written ecosystem:name; a go: name ending in / is a | ||
| 9 | // module path prefix, and root: is an entry at the top of the tree. | ||
| 10 | // Directories that are about a language, a runtime or the standard library | ||
| 11 | // (lang, browser, audit, correctness, template, node-crypto, …) are not | ||
| 12 | // here, so their rules apply everywhere. Neither are languages whose | ||
| 13 | // manifests gitbay does not read. | ||
| 14 | type library struct { | ||
| 15 | rules string | ||
| 16 | packages []string | ||
| 17 | } | ||
| 18 | |||
| 19 | var libraries = []library{ | ||
| 20 | {"go.aws-lambda", []string{"go:github.com/aws/aws-lambda-go"}}, | ||
| 21 | {"go.gorilla", []string{"go:github.com/gorilla/"}}, | ||
| 22 | {"go.gorm", []string{"go:gorm.io/gorm", "go:github.com/jinzhu/gorm"}}, | ||
| 23 | {"go.grpc", []string{"go:google.golang.org/grpc"}}, | ||
| 24 | {"go.jwt-go", []string{"go:github.com/dgrijalva/jwt-go", "go:github.com/golang-jwt/jwt/", "go:github.com/form3tech-oss/jwt-go"}}, | ||
| 25 | {"go.otto", []string{"go:github.com/robertkrimen/otto"}}, | ||
| 26 | |||
| 27 | {"javascript.ajv", []string{"npm:ajv"}}, | ||
| 28 | {"javascript.angular", []string{"npm:angular", "npm:@angular/core"}}, | ||
| 29 | {"javascript.apollo", []string{"npm:apollo-server", "npm:apollo-server-express", "npm:@apollo/server"}}, | ||
| 30 | {"javascript.argon2", []string{"npm:argon2"}}, | ||
| 31 | {"javascript.bluebird", []string{"npm:bluebird"}}, | ||
| 32 | {"javascript.chrome-remote-interface", []string{"npm:chrome-remote-interface"}}, | ||
| 33 | {"javascript.express", []string{"npm:express"}}, | ||
| 34 | {"javascript.fbjs", []string{"npm:fbjs"}}, | ||
| 35 | {"javascript.grpc", []string{"npm:grpc", "npm:@grpc/grpc-js"}}, | ||
| 36 | {"javascript.jose", []string{"npm:jose", "npm:node-jose"}}, | ||
| 37 | {"javascript.jquery", []string{"npm:jquery"}}, | ||
| 38 | {"javascript.jsonwebtoken", []string{"npm:jsonwebtoken"}}, | ||
| 39 | {"javascript.jwt-simple", []string{"npm:jwt-simple"}}, | ||
| 40 | {"javascript.monaco-editor", []string{"npm:monaco-editor"}}, | ||
| 41 | {"javascript.node-expat", []string{"npm:node-expat"}}, | ||
| 42 | {"javascript.passport-jwt", []string{"npm:passport-jwt"}}, | ||
| 43 | {"javascript.phantom", []string{"npm:phantom"}}, | ||
| 44 | {"javascript.playwright", []string{"npm:playwright", "npm:@playwright/test"}}, | ||
| 45 | {"javascript.puppeteer", []string{"npm:puppeteer", "npm:puppeteer-core"}}, | ||
| 46 | {"javascript.react", []string{"npm:react"}}, | ||
| 47 | {"javascript.sandbox", []string{"npm:sandbox"}}, | ||
| 48 | {"javascript.sax", []string{"npm:sax"}}, | ||
| 49 | {"javascript.sequelize", []string{"npm:sequelize"}}, | ||
| 50 | {"javascript.serialize-javascript", []string{"npm:serialize-javascript"}}, | ||
| 51 | {"javascript.shelljs", []string{"npm:shelljs"}}, | ||
| 52 | {"javascript.thenify", []string{"npm:thenify"}}, | ||
| 53 | {"javascript.vm2", []string{"npm:vm2"}}, | ||
| 54 | {"javascript.vue", []string{"npm:vue"}}, | ||
| 55 | {"javascript.wkhtmltoimage", []string{"npm:wkhtmltoimage"}}, | ||
| 56 | {"javascript.wkhtmltopdf", []string{"npm:wkhtmltopdf"}}, | ||
| 57 | {"javascript.xml2json", []string{"npm:xml2json"}}, | ||
| 58 | |||
| 59 | {"typescript.angular", []string{"npm:@angular/core"}}, | ||
| 60 | {"typescript.aws-cdk", []string{"npm:aws-cdk-lib", "npm:@aws-cdk/core"}}, | ||
| 61 | {"typescript.nestjs", []string{"npm:@nestjs/core"}}, | ||
| 62 | {"typescript.react", []string{"npm:react"}}, | ||
| 63 | {"typescript.react.portability.i18next", []string{"npm:i18next", "npm:react-i18next"}}, | ||
| 64 | |||
| 65 | {"python.airflow", []string{"pypi:apache-airflow"}}, | ||
| 66 | {"python.attr", []string{"pypi:attrs"}}, | ||
| 67 | {"python.bokeh", []string{"pypi:bokeh"}}, | ||
| 68 | {"python.boto3", []string{"pypi:boto3"}}, | ||
| 69 | {"python.click", []string{"pypi:click"}}, | ||
| 70 | {"python.cryptography", []string{"pypi:cryptography"}}, | ||
| 71 | {"python.distributed", []string{"pypi:distributed", "pypi:dask"}}, | ||
| 72 | {"python.django", []string{"pypi:django"}}, | ||
| 73 | {"python.docker", []string{"pypi:docker"}}, | ||
| 74 | {"python.fastapi", []string{"pypi:fastapi"}}, | ||
| 75 | {"python.flask", []string{"pypi:flask"}}, | ||
| 76 | {"python.jinja2", []string{"pypi:jinja2", "pypi:flask"}}, | ||
| 77 | {"python.jwt", []string{"pypi:pyjwt"}}, | ||
| 78 | {"python.pycryptodome", []string{"pypi:pycryptodome", "pypi:pycryptodomex"}}, | ||
| 79 | {"python.pymongo", []string{"pypi:pymongo"}}, | ||
| 80 | {"python.pyramid", []string{"pypi:pyramid"}}, | ||
| 81 | {"python.requests", []string{"pypi:requests"}}, | ||
| 82 | {"python.sh", []string{"pypi:sh"}}, | ||
| 83 | {"python.sqlalchemy", []string{"pypi:sqlalchemy", "pypi:flask-sqlalchemy"}}, | ||
| 84 | {"python.twilio", []string{"pypi:twilio"}}, | ||
| 85 | |||
| 86 | // Lints for writing Semgrep rules, which match any YAML with a list | ||
| 87 | // of rules that have ids. | ||
| 88 | {"yaml.semgrep", []string{"root:.semgrep", "root:.semgrep.yml", "root:.semgrep.yaml", "pypi:semgrep"}}, | ||
| 89 | } | ||
| 90 | |||
| 91 | // libraryFor returns the most specific library a rule id belongs to. The | ||
| 92 | // id may carry a prefix from where the rules were mounted | ||
| 93 | // (opt.opengrep-rules.python.django…). | ||
| 94 | func libraryFor(rule string) (library, bool) { | ||
| 95 | var best library | ||
| 96 | found := false | ||
| 97 | for _, l := range libraries { | ||
| 98 | if ruleIn(rule, l.rules) && len(l.rules) > len(best.rules) { | ||
| 99 | best, found = l, true | ||
| 100 | } | ||
| 101 | } | ||
| 102 | return best, found | ||
| 103 | } | ||
| 104 | |||
| 105 | // ruleIn reports whether a rule id is family or within it: family is a | ||
| 106 | // whole id, a run of its dot-separated segments, or a suffix. | ||
| 107 | func ruleIn(rule, family string) bool { | ||
| 108 | return rule == family || strings.HasPrefix(rule, family+".") || | ||
| 109 | strings.HasSuffix(rule, "."+family) || strings.Contains(rule, "."+family+".") | ||
| 110 | } | ||
| 111 | |||
| 112 | // name is how the issue and status refer to a library. | ||
| 113 | func (l library) name() string { | ||
| 114 | return l.rules[strings.LastIndex(l.rules, ".")+1:] | ||
| 115 | } | ||
internal/codescan/scope.go added +110
| @@ -0,0 +1,110 @@ | |||
| 1 | package codescan | ||
| 2 | |||
| 3 | import ( | ||
| 4 | "bufio" | ||
| 5 | "bytes" | ||
| 6 | "path" | ||
| 7 | "strings" | ||
| 8 | |||
| 9 | "gitbay.org/gitbay/internal/ci" | ||
| 10 | "gitbay.org/gitbay/internal/store" | ||
| 11 | ) | ||
| 12 | |||
| 13 | // Scope is what a repository sets aside from what its scan reports. | ||
| 14 | type Scope struct { | ||
| 15 | // Ignore are path globs, as CI paths take them (security-ignore). | ||
| 16 | Ignore []string | ||
| 17 | // Vendored are the .gitattributes patterns Vendored returns. | ||
| 18 | Vendored []string | ||
| 19 | // ExcludeRules are rule ids or families (security-exclude-rule). | ||
| 20 | ExcludeRules []string | ||
| 21 | // Uses reports whether the scanned commit depends on any of packages, | ||
| 22 | // written as library lists them. Nil uses everything. | ||
| 23 | Uses func(packages []string) bool | ||
| 24 | } | ||
| 25 | |||
| 26 | // Aside counts the findings a scope set aside: Ignored by path or rule, | ||
| 27 | // Unused by the library a rule is about. | ||
| 28 | type Aside struct { | ||
| 29 | Ignored int `json:"ignored,omitempty"` | ||
| 30 | Unused map[string]int `json:"unused,omitempty"` | ||
| 31 | } | ||
| 32 | |||
| 33 | // Apply splits findings into those reported, at most MaxFindings of them, | ||
| 34 | // and counts of those set aside. It reports whether reported ones were | ||
| 35 | // dropped for the cap. | ||
| 36 | func (s Scope) Apply(findings []store.SecurityFinding) ([]store.SecurityFinding, Aside, bool) { | ||
| 37 | var kept []store.SecurityFinding | ||
| 38 | var aside Aside | ||
| 39 | truncated := false | ||
| 40 | for _, f := range findings { | ||
| 41 | if s.ignored(f) { | ||
| 42 | aside.Ignored++ | ||
| 43 | continue | ||
| 44 | } | ||
| 45 | if l, ok := libraryFor(f.Rule); ok && s.Uses != nil && !s.Uses(l.packages) { | ||
| 46 | if aside.Unused == nil { | ||
| 47 | aside.Unused = map[string]int{} | ||
| 48 | } | ||
| 49 | aside.Unused[l.name()]++ | ||
| 50 | continue | ||
| 51 | } | ||
| 52 | if len(kept) == MaxFindings { | ||
| 53 | truncated = true | ||
| 54 | continue | ||
| 55 | } | ||
| 56 | kept = append(kept, f) | ||
| 57 | } | ||
| 58 | return kept, aside, truncated | ||
| 59 | } | ||
| 60 | |||
| 61 | func (s Scope) ignored(f store.SecurityFinding) bool { | ||
| 62 | for _, g := range s.Ignore { | ||
| 63 | if ci.Match(g, f.Path) { | ||
| 64 | return true | ||
| 65 | } | ||
| 66 | } | ||
| 67 | for _, p := range s.Vendored { | ||
| 68 | if attrMatch(p, f.Path) { | ||
| 69 | return true | ||
| 70 | } | ||
| 71 | } | ||
| 72 | for _, r := range s.ExcludeRules { | ||
| 73 | if ruleIn(f.Rule, r) { | ||
| 74 | return true | ||
| 75 | } | ||
| 76 | } | ||
| 77 | return false | ||
| 78 | } | ||
| 79 | |||
| 80 | // Vendored returns the patterns a .gitattributes file marks | ||
| 81 | // linguist-vendored or linguist-generated. Unset and false values are left | ||
| 82 | // out. | ||
| 83 | func Vendored(gitattributes []byte) []string { | ||
| 84 | var out []string | ||
| 85 | sc := bufio.NewScanner(bytes.NewReader(gitattributes)) | ||
| 86 | for sc.Scan() { | ||
| 87 | fields := strings.Fields(sc.Text()) | ||
| 88 | if len(fields) < 2 || strings.HasPrefix(fields[0], "#") { | ||
| 89 | continue | ||
| 90 | } | ||
| 91 | for _, a := range fields[1:] { | ||
| 92 | if a == "linguist-vendored" || a == "linguist-generated" || a == "linguist-vendored=true" || a == "linguist-generated=true" { | ||
| 93 | out = append(out, fields[0]) | ||
| 94 | break | ||
| 95 | } | ||
| 96 | } | ||
| 97 | } | ||
| 98 | return out | ||
| 99 | } | ||
| 100 | |||
| 101 | // attrMatch matches a path the way git matches a .gitattributes pattern: one | ||
| 102 | // without a slash names files by base name at any depth, one with a slash is | ||
| 103 | // relative to the top. | ||
| 104 | func attrMatch(pattern, file string) bool { | ||
| 105 | if !strings.Contains(pattern, "/") { | ||
| 106 | ok, err := path.Match(pattern, path.Base(file)) | ||
| 107 | return err == nil && ok | ||
| 108 | } | ||
| 109 | return ci.Match(strings.TrimPrefix(pattern, "/"), file) | ||
| 110 | } | ||
internal/control/repo.go +4
| @@ -844,6 +844,8 @@ func runSettingsShow(c *Ctx, args []string) int { | |||
| 844 | "require signed commits", strconv.FormatBool(repo.Settings.RequireSignedCommits), | 844 | "require signed commits", strconv.FormatBool(repo.Settings.RequireSignedCommits), |
| 845 | "secret scan", leaks.Mode(repo.Settings.SecretScan), | 845 | "secret scan", leaks.Mode(repo.Settings.SecretScan), |
| 846 | "secret scan allow", strings.Join(repo.Settings.SecretScanAllow, ", "), | 846 | "secret scan allow", strings.Join(repo.Settings.SecretScanAllow, ", "), |
| 847 | "security ignore", strings.Join(repo.Settings.SecurityIgnore, ", "), | ||
| 848 | "security exclude rules", strings.Join(repo.Settings.SecurityExcludeRules, ", "), | ||
| 847 | "git daemon", strconv.FormatBool(repo.Settings.GitDaemon), | 849 | "git daemon", strconv.FormatBool(repo.Settings.GitDaemon), |
| 848 | "archived", strconv.FormatBool(repo.Settings.Archived), | 850 | "archived", strconv.FormatBool(repo.Settings.Archived), |
| 849 | ) | 851 | ) |
| @@ -882,6 +884,8 @@ func runSettingsShow(c *Ctx, args []string) int { | |||
| 882 | {"Signed commits", []cell{onOff(set.RequireSignedCommits)}}, | 884 | {"Signed commits", []cell{onOff(set.RequireSignedCommits)}}, |
| 883 | {"Secret scan", []cell{cText(leaks.Mode(set.SecretScan))}}, | 885 | {"Secret scan", []cell{cText(leaks.Mode(set.SecretScan))}}, |
| 884 | {"Scan allows", []cell{list(set.SecretScanAllow)}}, | 886 | {"Scan allows", []cell{list(set.SecretScanAllow)}}, |
| 887 | {"Scan ignores", []cell{list(set.SecurityIgnore)}}, | ||
| 888 | {"Rules off", []cell{list(set.SecurityExcludeRules)}}, | ||
| 885 | {"Git daemon", []cell{onOff(set.GitDaemon)}}, | 889 | {"Git daemon", []cell{onOff(set.GitDaemon)}}, |
| 886 | {"Archived", []cell{onOff(set.Archived)}}, | 890 | {"Archived", []cell{onOff(set.Archived)}}, |
| 887 | }, actions: []action{ | 891 | }, actions: []action{ |
internal/control/security.go +360 −23
| @@ -1,15 +1,22 @@ | |||
| 1 | package control | 1 | package control |
| 2 | 2 | ||
| 3 | import ( | 3 | import ( |
| 4 | "encoding/json" | ||
| 4 | "errors" | 5 | "errors" |
| 5 | "fmt" | 6 | "fmt" |
| 6 | "io" | 7 | "io" |
| 8 | "path" | ||
| 9 | "slices" | ||
| 10 | "sort" | ||
| 7 | "strconv" | 11 | "strconv" |
| 12 | "strings" | ||
| 8 | "time" | 13 | "time" |
| 9 | 14 | ||
| 10 | "gitbay.org/gitbay/internal/botissue" | 15 | "gitbay.org/gitbay/internal/botissue" |
| 11 | "gitbay.org/gitbay/internal/ci" | 16 | "gitbay.org/gitbay/internal/ci" |
| 12 | "gitbay.org/gitbay/internal/codescan" | 17 | "gitbay.org/gitbay/internal/codescan" |
| 18 | "gitbay.org/gitbay/internal/deps" | ||
| 19 | "gitbay.org/gitbay/internal/gitutil" | ||
| 13 | "gitbay.org/gitbay/internal/policy" | 20 | "gitbay.org/gitbay/internal/policy" |
| 14 | "gitbay.org/gitbay/internal/protocol" | 21 | "gitbay.org/gitbay/internal/protocol" |
| 15 | "gitbay.org/gitbay/internal/store" | 22 | "gitbay.org/gitbay/internal/store" |
| @@ -40,6 +47,29 @@ func init() { | |||
| 40 | Usage: "repo security status <owner/name>", | 47 | Usage: "repo security status <owner/name>", |
| 41 | Examples: []string{"repo security status krz/gitbay"}, | 48 | Examples: []string{"repo security status krz/gitbay"}, |
| 42 | ReadOnly: true, Run: runSecurityStatus}) | 49 | ReadOnly: true, Run: runSecurityStatus}) |
| 50 | register(Command{Path: []string{"repo", "security", "dismiss"}, | ||
| 51 | Summary: "dismiss code scanning findings, which then stay out of the issue", | ||
| 52 | Usage: "repo security dismiss <owner/name> <fingerprint>... --reason false-positive|wont-fix|test", | ||
| 53 | Flags: []Flag{ | ||
| 54 | {"--reason", "false-positive|wont-fix|test", "why: not a problem, accepted, or only in tests", ""}, | ||
| 55 | }, | ||
| 56 | Examples: []string{"repo security dismiss krz/gitbay 3f9a0c1d2e --reason false-positive"}, | ||
| 57 | Run: runSecurityDismiss}) | ||
| 58 | register(Command{Path: []string{"repo", "security", "reopen"}, | ||
| 59 | Summary: "take back dismissals of code scanning findings", | ||
| 60 | Usage: "repo security reopen <owner/name> <fingerprint>...", | ||
| 61 | Examples: []string{"repo security reopen krz/gitbay 3f9a0c1d2e"}, | ||
| 62 | Run: runSecurityReopen}) | ||
| 63 | register(Command{Path: []string{"repo", "settings", "security-ignore"}, | ||
| 64 | Summary: "paths whose code scanning findings are set aside", | ||
| 65 | Usage: "repo settings security-ignore <owner/name> [<glob>...] (none clears the list)", | ||
| 66 | Examples: []string{"repo settings security-ignore krz/gitbay 'testdata/**' 'web/templates/*.html'"}, | ||
| 67 | Run: runSecurityIgnore}) | ||
| 68 | register(Command{Path: []string{"repo", "settings", "security-exclude-rule"}, | ||
| 69 | Summary: "code scanning rules, or families of them, whose findings are set aside", | ||
| 70 | Usage: "repo settings security-exclude-rule <owner/name> [<rule>...] (none clears the list)", | ||
| 71 | Examples: []string{"repo settings security-exclude-rule krz/gitbay swift.lang.crypto.insecure-random"}, | ||
| 72 | Run: runSecurityExcludeRule}) | ||
| 43 | register(Command{Path: []string{"runner", "results"}, | 73 | register(Command{Path: []string{"runner", "results"}, |
| 44 | Summary: "upload a code scan's SARIF from stdin", | 74 | Summary: "upload a code scan's SARIF from stdin", |
| 45 | Usage: "runner results <build-id>", | 75 | Usage: "runner results <build-id>", |
| @@ -47,25 +77,34 @@ func init() { | |||
| 47 | ReadsStdin: true, Run: runRunnerResults}) | 77 | ReadsStdin: true, Run: runRunnerResults}) |
| 48 | } | 78 | } |
| 49 | 79 | ||
| 50 | // SecurityOut is what `repo security status` emits. | 80 | // SecurityOut is what `repo security status` emits: the last scan's open |
| 81 | // findings, the ones dismissed, and how many it set aside. | ||
| 51 | type SecurityOut struct { | 82 | type SecurityOut struct { |
| 52 | Enabled bool `json:"enabled"` | 83 | Enabled bool `json:"enabled"` |
| 53 | NextRun string `json:"next_run,omitempty"` | 84 | NextRun string `json:"next_run,omitempty"` |
| 54 | LastResults string `json:"last_results,omitempty"` | 85 | LastResults string `json:"last_results,omitempty"` |
| 55 | IssueNumber int64 `json:"issue_number,omitempty"` | 86 | IssueNumber int64 `json:"issue_number,omitempty"` |
| 56 | Findings []SecurityFinding `json:"findings"` | 87 | Findings []SecurityFinding `json:"findings"` |
| 88 | Dismissed []SecurityFinding `json:"dismissed"` | ||
| 89 | Aside codescan.Aside `json:"aside"` | ||
| 57 | } | 90 | } |
| 58 | 91 | ||
| 59 | // SecurityFinding is one result of the last scan. | 92 | // SecurityFinding is one result of the last scan. |
| 60 | type SecurityFinding struct { | 93 | type SecurityFinding struct { |
| 61 | Tool string `json:"tool"` | 94 | Fingerprint string `json:"fingerprint"` |
| 62 | Rule string `json:"rule"` | 95 | Tool string `json:"tool"` |
| 63 | Level string `json:"level"` | 96 | Rule string `json:"rule"` |
| 64 | Path string `json:"path"` | 97 | Level string `json:"level"` |
| 65 | Line int `json:"line"` | 98 | Path string `json:"path"` |
| 66 | Message string `json:"message"` | 99 | Line int `json:"line"` |
| 100 | Message string `json:"message"` | ||
| 101 | Reason string `json:"reason,omitempty"` | ||
| 102 | DismissedBy string `json:"dismissed_by,omitempty"` | ||
| 67 | } | 103 | } |
| 68 | 104 | ||
| 105 | // Reasons a finding is dismissed for. | ||
| 106 | var dismissReasons = []string{"false-positive", "wont-fix", "test"} | ||
| 107 | |||
| 69 | func runSecurityEnable(c *Ctx, args []string) int { | 108 | func runSecurityEnable(c *Ctx, args []string) int { |
| 70 | if len(args) != 1 { | 109 | if len(args) != 1 { |
| 71 | return c.usage() | 110 | return c.usage() |
| @@ -153,10 +192,22 @@ func runSecurityStatus(c *Ctx, args []string) int { | |||
| 153 | if err != nil { | 192 | if err != nil { |
| 154 | return c.fail(protocol.ExitFailure, "%v", err) | 193 | return c.fail(protocol.ExitFailure, "%v", err) |
| 155 | } | 194 | } |
| 195 | dismissed, err := c.Store.SecurityDismissals(repo.ID) | ||
| 196 | if err != nil { | ||
| 197 | return c.fail(protocol.ExitFailure, "%v", err) | ||
| 198 | } | ||
| 156 | out := SecurityOut{Enabled: true, NextRun: scan.NextRun, LastResults: scan.LastResults, | 199 | out := SecurityOut{Enabled: true, NextRun: scan.NextRun, LastResults: scan.LastResults, |
| 157 | IssueNumber: scan.IssueNumber, Findings: []SecurityFinding{}} | 200 | IssueNumber: scan.IssueNumber, Findings: []SecurityFinding{}, Dismissed: []SecurityFinding{}} |
| 158 | for _, f := range reported { | 201 | json.Unmarshal([]byte(scan.Aside), &out.Aside) |
| 159 | out.Findings = append(out.Findings, SecurityFinding{f.Tool, f.Rule, f.Level, f.Path, f.Line, f.Message}) | 202 | for _, f := range codescan.ByPlace(reported) { |
| 203 | o := SecurityFinding{Fingerprint: f.Fingerprint, Tool: f.Tool, Rule: f.Rule, Level: f.Level, | ||
| 204 | Path: f.Path, Line: f.Line, Message: f.Message} | ||
| 205 | if d, ok := dismissed[f.Fingerprint]; ok { | ||
| 206 | o.Reason, o.DismissedBy = d.Reason, d.By | ||
| 207 | out.Dismissed = append(out.Dismissed, o) | ||
| 208 | continue | ||
| 209 | } | ||
| 210 | out.Findings = append(out.Findings, o) | ||
| 160 | } | 211 | } |
| 161 | return c.emitView(out, func(w io.Writer) { | 212 | return c.emitView(out, func(w io.Writer) { |
| 162 | tracked := "" | 213 | tracked := "" |
| @@ -169,12 +220,21 @@ func runSecurityStatus(c *Ctx, args []string) int { | |||
| 169 | "next scan", scan.NextRun, | 220 | "next scan", scan.NextRun, |
| 170 | "last results", c.when(scan.LastResults), | 221 | "last results", c.when(scan.LastResults), |
| 171 | "tracked in", tracked, | 222 | "tracked in", tracked, |
| 223 | "set aside", asideText(out.Aside), | ||
| 172 | ) | 224 | ) |
| 173 | if len(out.Findings) > 0 { | 225 | if len(out.Findings) > 0 { |
| 174 | v.section("findings") | 226 | v.section("findings") |
| 175 | tb := c.table(w, "TOOL", "RULE", "LEVEL", "PATH", "LINE") | 227 | tb := c.table(w, "FINGERPRINT", "TOOL", "RULE", "LEVEL", "PATH", "LINE") |
| 176 | for _, f := range out.Findings { | 228 | for _, f := range out.Findings { |
| 177 | tb.row(cText(f.Tool), cText(f.Rule), cText(f.Level), cRef(f.Path), cNum(int64(f.Line))) | 229 | tb.row(cRef(codescan.Short(f.Fingerprint)), cText(f.Tool), cText(f.Rule), cText(f.Level), cRef(f.Path), cNum(int64(f.Line))) |
| 230 | } | ||
| 231 | tb.flush() | ||
| 232 | } | ||
| 233 | if len(out.Dismissed) > 0 { | ||
| 234 | v.section("dismissed") | ||
| 235 | tb := c.table(w, "FINGERPRINT", "REASON", "BY", "RULE", "PATH", "LINE") | ||
| 236 | for _, f := range out.Dismissed { | ||
| 237 | tb.row(cRef(codescan.Short(f.Fingerprint)), cText(f.Reason), cText(f.DismissedBy), cText(f.Rule), cRef(f.Path), cNum(int64(f.Line))) | ||
| 178 | } | 238 | } |
| 179 | tb.flush() | 239 | tb.flush() |
| 180 | } | 240 | } |
| @@ -188,12 +248,25 @@ func runSecurityStatus(c *Ctx, args []string) int { | |||
| 188 | n := strconv.FormatInt(scan.IssueNumber, 10) | 248 | n := strconv.FormatInt(scan.IssueNumber, 10) |
| 189 | s.fields = append(s.fields, field{"Tracked in", []cell{cLink("#"+n, c.siteURL(repo.Path(), "issues", n))}}) | 249 | s.fields = append(s.fields, field{"Tracked in", []cell{cLink("#"+n, c.siteURL(repo.Path(), "issues", n))}}) |
| 190 | } | 250 | } |
| 251 | if a := asideText(out.Aside); a != "" { | ||
| 252 | s.fields = append(s.fields, field{"Set aside", []cell{cMeta(a)}}) | ||
| 253 | } | ||
| 191 | found := section{title: "Findings", n: len(out.Findings)} | 254 | found := section{title: "Findings", n: len(out.Findings)} |
| 192 | for _, f := range out.Findings { | 255 | for _, f := range out.Findings { |
| 193 | found.rows = append(found.rows, rowOf(cRef(f.Path+":"+strconv.Itoa(f.Line)), cText(f.Rule), cMeta(f.Level))) | 256 | found.rows = append(found.rows, rowOf(cRef(f.Path+":"+strconv.Itoa(f.Line)), cText(f.Rule), cFlexRef(codescan.Short(f.Fingerprint)))) |
| 194 | } | 257 | } |
| 195 | s.sections = []section{found} | 258 | s.sections = []section{found} |
| 259 | if len(out.Dismissed) > 0 { | ||
| 260 | gone := section{title: "Dismissed", n: len(out.Dismissed)} | ||
| 261 | for _, f := range out.Dismissed { | ||
| 262 | gone.rows = append(gone.rows, rowOf(cRef(f.Path+":"+strconv.Itoa(f.Line)), cState(f.Reason), cFlexRef(codescan.Short(f.Fingerprint)))) | ||
| 263 | } | ||
| 264 | s.sections = append(s.sections, gone) | ||
| 265 | } | ||
| 196 | s.actions = []action{{"Security", []string{"repo", "security", "run", repo.Path()}}} | 266 | s.actions = []action{{"Security", []string{"repo", "security", "run", repo.Path()}}} |
| 267 | if len(out.Findings) > 0 { | ||
| 268 | s.actions = append(s.actions, action{"Security", []string{"repo", "security", "dismiss", repo.Path(), "<fp>", "--reason", "<why>"}}) | ||
| 269 | } | ||
| 197 | return s | 270 | return s |
| 198 | }) | 271 | }) |
| 199 | } | 272 | } |
| @@ -244,7 +317,7 @@ func runRunnerResults(c *Ctx, args []string) int { | |||
| 244 | if len(raw) > codescan.MaxUpload { | 317 | if len(raw) > codescan.MaxUpload { |
| 245 | return c.fail(protocol.ExitFailure, "results are over %d bytes", codescan.MaxUpload) | 318 | return c.fail(protocol.ExitFailure, "results are over %d bytes", codescan.MaxUpload) |
| 246 | } | 319 | } |
| 247 | findings, truncated, err := codescan.Parse(raw) | 320 | all, overflow, err := codescan.Parse(raw) |
| 248 | if err != nil { | 321 | if err != nil { |
| 249 | return c.fail(protocol.ExitFailure, "%v", err) | 322 | return c.fail(protocol.ExitFailure, "%v", err) |
| 250 | } | 323 | } |
| @@ -252,19 +325,19 @@ func runRunnerResults(c *Ctx, args []string) int { | |||
| 252 | if err != nil { | 325 | if err != nil { |
| 253 | return c.fail(protocol.ExitFailure, "%v", err) | 326 | return c.fail(protocol.ExitFailure, "%v", err) |
| 254 | } | 327 | } |
| 328 | findings, aside, truncated := scanScope(RepoDir(c.Cfg.Server.Root, repo.OwnerName, repo.Name), repo, b.SHA).Apply(all) | ||
| 255 | previous, err := c.Store.ReportedFindings(repo.ID) | 329 | previous, err := c.Store.ReportedFindings(repo.ID) |
| 256 | if err != nil { | 330 | if err != nil { |
| 257 | return c.fail(protocol.ExitFailure, "%v", err) | 331 | return c.fail(protocol.ExitFailure, "%v", err) |
| 258 | } | 332 | } |
| 259 | err = botissue.Maintain(c.Store, c.Cfg, repo, botissue.Tracked{ | 333 | asideJSON := "" |
| 260 | Title: codescan.IssueTitle, | 334 | if aside.Ignored > 0 || len(aside.Unused) > 0 { |
| 261 | Number: scan.IssueNumber, | 335 | j, _ := json.Marshal(aside) |
| 262 | Record: func(n int64) error { return c.Store.SetSecurityIssue(repo.ID, n) }, | 336 | asideJSON = string(j) |
| 263 | Empty: len(findings) == 0, | 337 | } |
| 264 | Changed: !codescan.Same(previous, codescan.ByFingerprint(findings)), | 338 | changed := !codescan.Same(previous, codescan.ByFingerprint(findings)) || asideJSON != scan.Aside |
| 265 | Save: func() error { return c.Store.ReplaceFindings(repo.ID, findings) }, | 339 | err = maintainFindings(c, repo, scan, findings, aside, truncated || overflow, changed, |
| 266 | Body: func() string { return codescan.Body(repo.DefaultBranch, findings, truncated) }, | 340 | func() error { return c.Store.ReplaceFindings(repo.ID, findings, asideJSON) }) |
| 267 | }) | ||
| 268 | if err != nil { | 341 | if err != nil { |
| 269 | return c.fail(protocol.ExitFailure, "%v", err) | 342 | return c.fail(protocol.ExitFailure, "%v", err) |
| 270 | } | 343 | } |
| @@ -277,3 +350,267 @@ func runRunnerResults(c *Ctx, args []string) int { | |||
| 277 | fmt.Fprintf(w, "%d findings recorded for build %d\n", len(findings), b.Number) | 350 | fmt.Fprintf(w, "%d findings recorded for build %d\n", len(findings), b.Number) |
| 278 | }) | 351 | }) |
| 279 | } | 352 | } |
| 353 | |||
| 354 | // maintainFindings brings the findings issue in line with a set of | ||
| 355 | // findings: the open ones listed, the dismissed and set-aside ones | ||
| 356 | // counted, closed when none are open. | ||
| 357 | func maintainFindings(c *Ctx, repo store.Repo, scan store.SecurityScan, findings []store.SecurityFinding, | ||
| 358 | aside codescan.Aside, truncated, changed bool, save func() error) error { | ||
| 359 | dismissed, err := c.Store.SecurityDismissals(repo.ID) | ||
| 360 | if err != nil { | ||
| 361 | return err | ||
| 362 | } | ||
| 363 | var open []store.SecurityFinding | ||
| 364 | for _, f := range findings { | ||
| 365 | if _, ok := dismissed[f.Fingerprint]; !ok { | ||
| 366 | open = append(open, f) | ||
| 367 | } | ||
| 368 | } | ||
| 369 | return botissue.Maintain(c.Store, c.Cfg, repo, botissue.Tracked{ | ||
| 370 | Title: codescan.IssueTitle, | ||
| 371 | Number: scan.IssueNumber, | ||
| 372 | Record: func(n int64) error { return c.Store.SetSecurityIssue(repo.ID, n) }, | ||
| 373 | Empty: len(open) == 0, | ||
| 374 | Changed: changed, | ||
| 375 | Save: save, | ||
| 376 | Body: func() string { | ||
| 377 | return codescan.Body(codescan.Report{Repo: repo.Path(), Branch: repo.DefaultBranch, Open: codescan.ByPlace(open), | ||
| 378 | Dismissed: len(findings) - len(open), Aside: aside, Truncated: truncated}) | ||
| 379 | }, | ||
| 380 | }) | ||
| 381 | } | ||
| 382 | |||
| 383 | // manifestLimit bounds a manifest or .gitattributes read for a scan's scope. | ||
| 384 | const manifestLimit = 1 << 20 | ||
| 385 | |||
| 386 | // maxManifestDirs bounds how many directories' manifests a scope reads. | ||
| 387 | const maxManifestDirs = 50 | ||
| 388 | |||
| 389 | // scanScope is what a repository sets aside from a scan of commit sha: its | ||
| 390 | // settings, what .gitattributes marks vendored or generated, and findings | ||
| 391 | // from rules about libraries no manifest at that commit names. Manifests are | ||
| 392 | // read in every directory that has one, outside dependency directories, so | ||
| 393 | // a monorepo's web/package.json counts. | ||
| 394 | func scanScope(dir string, repo store.Repo, sha string) codescan.Scope { | ||
| 395 | sc := codescan.Scope{Ignore: repo.Settings.SecurityIgnore, ExcludeRules: repo.Settings.SecurityExcludeRules} | ||
| 396 | if raw, err := gitutil.ReadBlob(dir, sha, ".gitattributes", manifestLimit); err == nil { | ||
| 397 | sc.Vendored = codescan.Vendored(raw) | ||
| 398 | } | ||
| 399 | root := map[string]bool{} | ||
| 400 | if entries, err := gitutil.ListTree(dir, sha, ""); err == nil { | ||
| 401 | for _, e := range entries { | ||
| 402 | root[e.Name] = true | ||
| 403 | } | ||
| 404 | } | ||
| 405 | dirs := []string{""} | ||
| 406 | if entries, err := gitutil.ListTreeRecursive(dir, sha, ""); err == nil { | ||
| 407 | seen := map[string]bool{"": true} | ||
| 408 | for _, e := range entries { | ||
| 409 | d := path.Dir(e.Name) | ||
| 410 | if seen[d] || len(dirs) == maxManifestDirs || !slices.Contains(deps.ManifestFiles, path.Base(e.Name)) || dependencyDir(d) { | ||
| 411 | continue | ||
| 412 | } | ||
| 413 | seen[d] = true | ||
| 414 | dirs = append(dirs, d) | ||
| 415 | } | ||
| 416 | } | ||
| 417 | uses := map[string]bool{} | ||
| 418 | for _, d := range dirs { | ||
| 419 | read := func(p string) ([]byte, error) { | ||
| 420 | if d != "" { | ||
| 421 | p = d + "/" + p | ||
| 422 | } | ||
| 423 | return gitutil.ReadBlob(dir, sha, p, manifestLimit) | ||
| 424 | } | ||
| 425 | for _, dep := range deps.Names(read) { | ||
| 426 | uses[dep.Ecosystem+":"+dep.Name] = true | ||
| 427 | } | ||
| 428 | } | ||
| 429 | sc.Uses = func(packages []string) bool { | ||
| 430 | for _, p := range packages { | ||
| 431 | if name, ok := strings.CutPrefix(p, "root:"); ok { | ||
| 432 | if root[name] { | ||
| 433 | return true | ||
| 434 | } | ||
| 435 | continue | ||
| 436 | } | ||
| 437 | if strings.HasPrefix(p, "go:") && strings.HasSuffix(p, "/") { | ||
| 438 | for u := range uses { | ||
| 439 | if strings.HasPrefix(u, p) { | ||
| 440 | return true | ||
| 441 | } | ||
| 442 | } | ||
| 443 | continue | ||
| 444 | } | ||
| 445 | if uses[p] { | ||
| 446 | return true | ||
| 447 | } | ||
| 448 | } | ||
| 449 | return false | ||
| 450 | } | ||
| 451 | return sc | ||
| 452 | } | ||
| 453 | |||
| 454 | // dependencyDir reports whether a directory holds other projects' code | ||
| 455 | // rather than the repository's own. | ||
| 456 | func dependencyDir(d string) bool { | ||
| 457 | for _, part := range strings.Split(d, "/") { | ||
| 458 | if part == "node_modules" || part == "vendor" || part == "third_party" { | ||
| 459 | return true | ||
| 460 | } | ||
| 461 | } | ||
| 462 | return false | ||
| 463 | } | ||
| 464 | |||
| 465 | // asideText is what the last scan set aside, in a line. | ||
| 466 | func asideText(a codescan.Aside) string { | ||
| 467 | var parts []string | ||
| 468 | if a.Ignored > 0 { | ||
| 469 | parts = append(parts, fmt.Sprintf("%d ignored", a.Ignored)) | ||
| 470 | } | ||
| 471 | libs := make([]string, 0, len(a.Unused)) | ||
| 472 | for l := range a.Unused { | ||
| 473 | libs = append(libs, l) | ||
| 474 | } | ||
| 475 | sort.Strings(libs) | ||
| 476 | for _, l := range libs { | ||
| 477 | parts = append(parts, fmt.Sprintf("%d %s (not used)", a.Unused[l], l)) | ||
| 478 | } | ||
| 479 | return strings.Join(parts, ", ") | ||
| 480 | } | ||
| 481 | |||
| 482 | func runSecurityDismiss(c *Ctx, args []string) int { | ||
| 483 | f, err := c.parseArgs(args, flagSpec{Values: []string{"--reason"}, MaxPos: -1, Usage: c.Cmd.Usage}) | ||
| 484 | if err != nil { | ||
| 485 | return c.fail(protocol.ExitUsage, "%v", err) | ||
| 486 | } | ||
| 487 | if len(f.Pos) < 2 { | ||
| 488 | return c.usage() | ||
| 489 | } | ||
| 490 | reason := f.Value("--reason") | ||
| 491 | if !slices.Contains(dismissReasons, reason) { | ||
| 492 | return c.fail(protocol.ExitUsage, "--reason is one of %s", strings.Join(dismissReasons, ", ")) | ||
| 493 | } | ||
| 494 | return changeDismissals(c, f.Pos[0], f.Pos[1:], func(repoID int64, fps []string) error { | ||
| 495 | return c.Store.DismissFindings(repoID, c.User.ID, fps, reason) | ||
| 496 | }, "dismissed") | ||
| 497 | } | ||
| 498 | |||
| 499 | func runSecurityReopen(c *Ctx, args []string) int { | ||
| 500 | if len(args) < 2 { | ||
| 501 | return c.usage() | ||
| 502 | } | ||
| 503 | return changeDismissals(c, args[0], args[1:], c.Store.ReopenFindings, "reopened") | ||
| 504 | } | ||
| 505 | |||
| 506 | // changeDismissals resolves fingerprints, given in full or as the issue | ||
| 507 | // prints them, against the last scan's findings, applies change, and | ||
| 508 | // rewrites the issue to match. Writers may do it: it decides what the | ||
| 509 | // issue lists, as editing the issue would. | ||
| 510 | func changeDismissals(c *Ctx, name string, given []string, change func(int64, []string) error, verb string) int { | ||
| 511 | repo, code := resolveRepo(c, name, policy.CanWrite) | ||
| 512 | if code >= 0 { | ||
| 513 | return code | ||
| 514 | } | ||
| 515 | scan, err := c.Store.SecurityScanFor(repo.ID) | ||
| 516 | if errors.Is(err, store.ErrNotFound) { | ||
| 517 | return c.fail(protocol.ExitFailure, "code scanning is off on %s (repo security enable %s)", repo.Path(), repo.Path()) | ||
| 518 | } else if err != nil { | ||
| 519 | return c.fail(protocol.ExitFailure, "%v", err) | ||
| 520 | } | ||
| 521 | findings, err := c.Store.ReportedFindings(repo.ID) | ||
| 522 | if err != nil { | ||
| 523 | return c.fail(protocol.ExitFailure, "%v", err) | ||
| 524 | } | ||
| 525 | var fps []string | ||
| 526 | for _, g := range given { | ||
| 527 | fp, msg := resolveFingerprint(findings, g) | ||
| 528 | if msg != "" { | ||
| 529 | return c.fail(protocol.ExitNotFound, "%s", msg) | ||
| 530 | } | ||
| 531 | if !slices.Contains(fps, fp) { | ||
| 532 | fps = append(fps, fp) | ||
| 533 | } | ||
| 534 | } | ||
| 535 | if err := change(repo.ID, fps); err != nil { | ||
| 536 | return c.fail(protocol.ExitFailure, "%v", err) | ||
| 537 | } | ||
| 538 | var aside codescan.Aside | ||
| 539 | json.Unmarshal([]byte(scan.Aside), &aside) | ||
| 540 | err = maintainFindings(c, repo, scan, findings, aside, len(findings) >= codescan.MaxFindings, true, | ||
| 541 | func() error { return nil }) | ||
| 542 | if err != nil { | ||
| 543 | return c.fail(protocol.ExitFailure, "%v", err) | ||
| 544 | } | ||
| 545 | return c.emit(map[string]any{verb: fps}, func(w io.Writer) { | ||
| 546 | fmt.Fprintf(w, "%s %d on %s\n", verb, len(fps), repo.Path()) | ||
| 547 | }) | ||
| 548 | } | ||
| 549 | |||
| 550 | // resolveFingerprint finds the one finding a fingerprint or a prefix of at | ||
| 551 | // least codescan.ShortLen names, or says why not. | ||
| 552 | func resolveFingerprint(findings []store.SecurityFinding, given string) (string, string) { | ||
| 553 | if len(given) < codescan.ShortLen { | ||
| 554 | return "", fmt.Sprintf("fingerprint %q is too short; give at least %d characters", given, codescan.ShortLen) | ||
| 555 | } | ||
| 556 | match := "" | ||
| 557 | for _, f := range findings { | ||
| 558 | if strings.HasPrefix(f.Fingerprint, given) { | ||
| 559 | if match != "" { | ||
| 560 | return "", fmt.Sprintf("fingerprint %q names more than one finding", given) | ||
| 561 | } | ||
| 562 | match = f.Fingerprint | ||
| 563 | } | ||
| 564 | } | ||
| 565 | if match == "" { | ||
| 566 | return "", fmt.Sprintf("no finding %q in the last scan", given) | ||
| 567 | } | ||
| 568 | return match, "" | ||
| 569 | } | ||
| 570 | |||
| 571 | // maxSecurityList bounds the ignore and exclude-rule settings. | ||
| 572 | const maxSecurityList = 50 | ||
| 573 | |||
| 574 | func runSecurityIgnore(c *Ctx, args []string) int { | ||
| 575 | return setSecurityList(c, args, "a glob", func(s *store.RepoSettings, xs []string) { s.SecurityIgnore = xs }, "ignores") | ||
| 576 | } | ||
| 577 | |||
| 578 | func runSecurityExcludeRule(c *Ctx, args []string) int { | ||
| 579 | return setSecurityList(c, args, "a rule", func(s *store.RepoSettings, xs []string) { s.SecurityExcludeRules = xs }, "excludes") | ||
| 580 | } | ||
| 581 | |||
| 582 | // setSecurityList replaces one of the code scanning lists, which apply from | ||
| 583 | // the next scan. | ||
| 584 | func setSecurityList(c *Ctx, args []string, what string, set func(*store.RepoSettings, []string), verb string) int { | ||
| 585 | if len(args) < 1 { | ||
| 586 | return c.usage() | ||
| 587 | } | ||
| 588 | var xs []string | ||
| 589 | for _, x := range args[1:] { | ||
| 590 | if x == "" || len(x) > 200 || strings.ContainsAny(x, " \t\r\n") { | ||
| 591 | return c.fail(protocol.ExitUsage, "%s is 1 to 200 characters with no spaces: %q", what, x) | ||
| 592 | } | ||
| 593 | if !slices.Contains(xs, x) { | ||
| 594 | xs = append(xs, x) | ||
| 595 | } | ||
| 596 | } | ||
| 597 | if len(xs) > maxSecurityList { | ||
| 598 | return c.fail(protocol.ExitUsage, "at most %d entries", maxSecurityList) | ||
| 599 | } | ||
| 600 | repo, code := resolveRepo(c, args[0], policy.CanAdmin) | ||
| 601 | if code >= 0 { | ||
| 602 | return code | ||
| 603 | } | ||
| 604 | s, err := c.Store.UpdateRepoSettings(repo.ID, func(s *store.RepoSettings) { set(s, xs) }) | ||
| 605 | if err != nil { | ||
| 606 | return c.fail(protocol.ExitFailure, "%v", err) | ||
| 607 | } | ||
| 608 | return c.emit(s, func(w io.Writer) { | ||
| 609 | if len(xs) == 0 { | ||
| 610 | fmt.Fprintf(w, "code scanning %s cleared on %s\n", verb, repo.Path()) | ||
| 611 | return | ||
| 612 | } | ||
| 613 | fmt.Fprintf(w, "code scanning %s on %s: %s; from the next scan (repo security run %s)\n", | ||
| 614 | verb, repo.Path(), strings.Join(xs, ", "), repo.Path()) | ||
| 615 | }) | ||
| 616 | } | ||
internal/control/security_test.go +182
| @@ -2,6 +2,9 @@ package control | |||
| 2 | 2 | ||
| 3 | import ( | 3 | import ( |
| 4 | "encoding/json" | 4 | "encoding/json" |
| 5 | "os" | ||
| 6 | "os/exec" | ||
| 7 | "path/filepath" | ||
| 5 | "strings" | 8 | "strings" |
| 6 | "testing" | 9 | "testing" |
| 7 | 10 | ||
| @@ -9,6 +12,7 @@ import ( | |||
| 9 | "gitbay.org/gitbay/internal/codescan" | 12 | "gitbay.org/gitbay/internal/codescan" |
| 10 | "gitbay.org/gitbay/internal/config" | 13 | "gitbay.org/gitbay/internal/config" |
| 11 | "gitbay.org/gitbay/internal/protocol" | 14 | "gitbay.org/gitbay/internal/protocol" |
| 15 | "gitbay.org/gitbay/internal/store" | ||
| 12 | ) | 16 | ) |
| 13 | 17 | ||
| 14 | const testSARIF = `{"version":"2.1.0","runs":[{"tool":{"driver":{"name":"opengrep"}},"results":[ | 18 | const testSARIF = `{"version":"2.1.0","runs":[{"tool":{"driver":{"name":"opengrep"}},"results":[ |
| @@ -113,3 +117,181 @@ func TestSecurityScanRoundTrip(t *testing.T) { | |||
| 113 | t.Errorf("results from another job: %d %s", code, out) | 117 | t.Errorf("results from another job: %d %s", code, out) |
| 114 | } | 118 | } |
| 115 | } | 119 | } |
| 120 | |||
| 121 | // scopeRepo commits files to a bare repository and returns its directory | ||
| 122 | // and the commit. | ||
| 123 | func scopeRepo(t *testing.T, files map[string]string) (string, string) { | ||
| 124 | t.Helper() | ||
| 125 | root := t.TempDir() | ||
| 126 | src, dir := filepath.Join(root, "src"), filepath.Join(root, "bare.git") | ||
| 127 | git := func(cwd string, args ...string) string { | ||
| 128 | t.Helper() | ||
| 129 | cmd := exec.Command("git", args...) | ||
| 130 | cmd.Dir, cmd.Env = cwd, gitTestEnv() | ||
| 131 | out, err := cmd.CombinedOutput() | ||
| 132 | if err != nil { | ||
| 133 | t.Fatalf("git %v: %v\n%s", args, err, out) | ||
| 134 | } | ||
| 135 | return strings.TrimSpace(string(out)) | ||
| 136 | } | ||
| 137 | for name, body := range files { | ||
| 138 | os.MkdirAll(filepath.Join(src, filepath.Dir(name)), 0o755) | ||
| 139 | os.WriteFile(filepath.Join(src, name), []byte(body), 0o644) | ||
| 140 | } | ||
| 141 | git(root, "init", "-q", "-b", "main", "src") | ||
| 142 | git(src, "add", ".") | ||
| 143 | git(src, "commit", "-q", "-m", "first") | ||
| 144 | git(root, "clone", "-q", "--bare", src, dir) | ||
| 145 | return dir, git(dir, "rev-parse", "HEAD") | ||
| 146 | } | ||
| 147 | |||
| 148 | func TestScanScope(t *testing.T) { | ||
| 149 | dir, sha := scopeRepo(t, map[string]string{ | ||
| 150 | "requirements.txt": "flask>=3\n", | ||
| 151 | "web/package.json": `{"dependencies":{"react":"^19"}}`, | ||
| 152 | "node_modules/x/package.json": `{"dependencies":{"jquery":"3"}}`, | ||
| 153 | ".gitattributes": "Sources/Vendored/** linguist-vendored\n*.min.js linguist-generated\ndocs/** -linguist-vendored\n", | ||
| 154 | "app.py": "", | ||
| 155 | }) | ||
| 156 | repo := store.Repo{Settings: store.RepoSettings{SecurityIgnore: []string{"testdata/**"}, | ||
| 157 | SecurityExcludeRules: []string{"swift.lang.crypto.insecure-random"}}} | ||
| 158 | f := func(rule, path string) store.SecurityFinding { | ||
| 159 | return store.SecurityFinding{Rule: "opt.opengrep-rules." + rule, Path: path} | ||
| 160 | } | ||
| 161 | in := []store.SecurityFinding{ | ||
| 162 | f("python.flask.security.xss.x", "app.py"), // flask is used | ||
| 163 | f("python.django.security.django-no-csrf-token", "t.html"), // django is not | ||
| 164 | f("javascript.jquery.security.audit.prohibit-jquery-html", "a.ts"), // only node_modules has it | ||
| 165 | f("typescript.react.security.audit.x", "web/a.tsx"), // web/package.json | ||
| 166 | f("typescript.react.portability.i18next.jsx-not-internationalized", "web/a.tsx"), | ||
| 167 | f("yaml.semgrep.missing-language-field", "rules.yaml"), | ||
| 168 | f("python.lang.security.audit.x", "testdata/a.py"), | ||
| 169 | f("swift.lang.crypto.insecure-random", "A.swift"), | ||
| 170 | f("c.lang.security.insecure-use-memset", "Sources/Vendored/a.c"), | ||
| 171 | f("javascript.browser.security.insecure-innerhtml", "lib/x.min.js"), | ||
| 172 | f("javascript.browser.security.insecure-innerhtml", "docs/x.js"), | ||
| 173 | } | ||
| 174 | kept, aside, truncated := scanScope(dir, repo, sha).Apply(in) | ||
| 175 | var got []string | ||
| 176 | for _, k := range kept { | ||
| 177 | got = append(got, k.Path) | ||
| 178 | } | ||
| 179 | if strings.Join(got, " ") != "app.py web/a.tsx docs/x.js" || truncated { | ||
| 180 | t.Errorf("kept %v truncated %v", got, truncated) | ||
| 181 | } | ||
| 182 | if aside.Ignored != 4 || aside.Unused["django"] != 1 || aside.Unused["jquery"] != 1 || | ||
| 183 | aside.Unused["i18next"] != 1 || aside.Unused["semgrep"] != 1 || len(aside.Unused) != 4 { | ||
| 184 | t.Errorf("aside = %+v", aside) | ||
| 185 | } | ||
| 186 | |||
| 187 | // A repository that authors rules keeps the rule lints. | ||
| 188 | dir, sha = scopeRepo(t, map[string]string{".semgrep/rules.yaml": "rules: []\n"}) | ||
| 189 | if kept, _, _ := scanScope(dir, store.Repo{}, sha).Apply(in[5:6]); len(kept) != 1 { | ||
| 190 | t.Errorf("rule lints set aside in a repository with .semgrep") | ||
| 191 | } | ||
| 192 | } | ||
| 193 | |||
| 194 | const twoFindings = `{"version":"2.1.0","runs":[{"tool":{"driver":{"name":"opengrep"}},"results":[ | ||
| 195 | {"ruleId":"go.lang.security.audit.sqli","message":{"text":"SQL built from input"}, | ||
| 196 | "locations":[{"physicalLocation":{"artifactLocation":{"uri":"store/q.go"},"region":{"startLine":12}}}]}, | ||
| 197 | {"ruleId":"go.lang.security.audit.sqli","message":{"text":"SQL built from input"}, | ||
| 198 | "locations":[{"physicalLocation":{"artifactLocation":{"uri":"store/q.go"},"region":{"startLine":40}}}]}]}]}` | ||
| 199 | |||
| 200 | // Dismissing takes a finding out of the issue and closes it once none are | ||
| 201 | // open; reopening brings it back. | ||
| 202 | func TestSecurityDismiss(t *testing.T) { | ||
| 203 | st, repo, u, root := browseFixture(t) | ||
| 204 | p := repo.Path() | ||
| 205 | sec := config.Security{Image: "localhost/gitbay-security:1", Schedule: "0 4 * * *", Steps: config.DefaultSecuritySteps} | ||
| 206 | run := func(f func(*Ctx, []string) int, stdin string, args ...string) (int, string) { | ||
| 207 | t.Helper() | ||
| 208 | c, out := runnerCtx(st, u.ID, root) | ||
| 209 | c.Cfg.Security = sec | ||
| 210 | c.Stdin = strings.NewReader(stdin) | ||
| 211 | c.JSON = true | ||
| 212 | return f(c, args), out.String() | ||
| 213 | } | ||
| 214 | run(runSecurityEnable, "", p) | ||
| 215 | run(runSecurityRun, "", p) | ||
| 216 | _, claim := run(runRunnerNext, "") | ||
| 217 | var claimed struct { | ||
| 218 | Data struct { | ||
| 219 | ID int64 `json:"id"` | ||
| 220 | } `json:"data"` | ||
| 221 | } | ||
| 222 | json.Unmarshal([]byte(claim), &claimed) | ||
| 223 | if code, out := run(runRunnerResults, twoFindings, itoa(claimed.Data.ID)); code != protocol.ExitOK { | ||
| 224 | t.Fatalf("results: %d %s", code, out) | ||
| 225 | } | ||
| 226 | scan, _ := st.SecurityScanFor(repo.ID) | ||
| 227 | findings, _ := st.ReportedFindings(repo.ID) | ||
| 228 | if len(findings) != 2 { | ||
| 229 | t.Fatalf("two hits of one rule in one file stored as %d", len(findings)) | ||
| 230 | } | ||
| 231 | issue, _ := st.IssueByNumber(repo.ID, scan.IssueNumber) | ||
| 232 | first, second := codescan.ByPlace(findings)[0].Fingerprint, codescan.ByPlace(findings)[1].Fingerprint | ||
| 233 | if !strings.Contains(issue.Body, codescan.Short(first)) || !strings.Contains(issue.Body, "repo security dismiss "+p) { | ||
| 234 | t.Fatalf("issue:\n%s", issue.Body) | ||
| 235 | } | ||
| 236 | |||
| 237 | dismiss := func(args ...string) (int, string) { | ||
| 238 | t.Helper() | ||
| 239 | var out strings.Builder | ||
| 240 | c, buf := runnerCtx(st, u.ID, root) | ||
| 241 | code := runSecurityDismiss(c, args) | ||
| 242 | out.WriteString(buf.String()) | ||
| 243 | return code, out.String() | ||
| 244 | } | ||
| 245 | if code, out := dismiss(p, codescan.Short(first), "--reason", "nope"); code != protocol.ExitUsage { | ||
| 246 | t.Errorf("bad reason: %d %s", code, out) | ||
| 247 | } | ||
| 248 | if code, out := dismiss(p, first[:4], "--reason", "test"); code != protocol.ExitNotFound || !strings.Contains(out, "too short") { | ||
| 249 | t.Errorf("short prefix: %d %s", code, out) | ||
| 250 | } | ||
| 251 | if code, out := dismiss(p, codescan.Short(first), "--reason", "false-positive"); code != protocol.ExitOK { | ||
| 252 | t.Fatalf("dismiss: %d %s", code, out) | ||
| 253 | } | ||
| 254 | issue, _ = st.IssueByNumber(repo.ID, scan.IssueNumber) | ||
| 255 | if strings.Contains(issue.Body, codescan.Short(first)) || !strings.Contains(issue.Body, codescan.Short(second)) || | ||
| 256 | !strings.Contains(issue.Body, "1 dismissed") || issue.State != "open" { | ||
| 257 | t.Errorf("after one dismissal (%s):\n%s", issue.State, issue.Body) | ||
| 258 | } | ||
| 259 | |||
| 260 | var status struct { | ||
| 261 | Data SecurityOut `json:"data"` | ||
| 262 | } | ||
| 263 | json.Unmarshal([]byte(dispatchIn(t, st, u, root, "", "repo", "security", "status", p, "--json")), &status) | ||
| 264 | if len(status.Data.Findings) != 1 || len(status.Data.Dismissed) != 1 || | ||
| 265 | status.Data.Dismissed[0].Reason != "false-positive" || status.Data.Dismissed[0].DismissedBy != u.Username { | ||
| 266 | t.Errorf("status = %+v", status.Data) | ||
| 267 | } | ||
| 268 | screen := atTerminalIn(t, st, u, root, "repo", "security", "status", p) | ||
| 269 | if !strings.Contains(screen, "Dismissed (1)") { | ||
| 270 | t.Errorf("screen:\n%s", screen) | ||
| 271 | } | ||
| 272 | checkLegend(t, screen) | ||
| 273 | |||
| 274 | // The same set from the next scan leaves the issue alone. | ||
| 275 | run(runSecurityRun, "", p) | ||
| 276 | _, claim = run(runRunnerNext, "") | ||
| 277 | json.Unmarshal([]byte(claim), &claimed) | ||
| 278 | run(runRunnerResults, twoFindings, itoa(claimed.Data.ID)) | ||
| 279 | if again, _ := st.IssueByNumber(repo.ID, scan.IssueNumber); again.Body != issue.Body || again.State != "open" { | ||
| 280 | t.Errorf("an unchanged scan rewrote the issue:\n%s", again.Body) | ||
| 281 | } | ||
| 282 | |||
| 283 | if code, out := dismiss(p, second, "--reason", "wont-fix"); code != protocol.ExitOK { | ||
| 284 | t.Fatalf("dismiss second: %d %s", code, out) | ||
| 285 | } | ||
| 286 | if closed, _ := st.IssueByNumber(repo.ID, scan.IssueNumber); closed.State != "closed" { | ||
| 287 | t.Errorf("issue with nothing open is %s", closed.State) | ||
| 288 | } | ||
| 289 | c, out := runnerCtx(st, u.ID, root) | ||
| 290 | if code := runSecurityReopen(c, []string{p, codescan.Short(second)}); code != protocol.ExitOK { | ||
| 291 | t.Fatalf("reopen: %d %s", code, out) | ||
| 292 | } | ||
| 293 | reopened, _ := st.SecurityScanFor(repo.ID) | ||
| 294 | if reopened.IssueNumber == scan.IssueNumber || reopened.IssueNumber == 0 { | ||
| 295 | t.Errorf("reopening a finding should open a fresh issue, got #%d", reopened.IssueNumber) | ||
| 296 | } | ||
| 297 | } | ||
internal/deps/manifest_test.go +15
| @@ -174,3 +174,18 @@ func TestScanEmptyTree(t *testing.T) { | |||
| 174 | t.Errorf("Scan of empty tree = %v", got) | 174 | t.Errorf("Scan of empty tree = %v", got) |
| 175 | } | 175 | } |
| 176 | } | 176 | } |
| 177 | |||
| 178 | // Names keeps what Scan leaves out for having no single version. | ||
| 179 | func TestNames(t *testing.T) { | ||
| 180 | got := found(t, Names(tree(map[string]string{ | ||
| 181 | "go.mod": "module x\n\nrequire github.com/gorilla/mux v1.8.1\n", | ||
| 182 | "requirements.txt": "Django>=4,<6\nrequests\n", | ||
| 183 | "package.json": `{"dependencies":{"jquery":"^3"},"devDependencies":{"vite":"git+https://x/y"},"peerDependencies":{"react":"*"}}`, | ||
| 184 | "Cargo.toml": "[dependencies]\nlocal = { path = \"../local\" }\n", | ||
| 185 | }))) | ||
| 186 | for _, want := range []string{"go:github.com/gorilla/mux", "pypi:django", "pypi:requests", "npm:jquery", "npm:vite", "npm:react", "cargo:local"} { | ||
| 187 | if v, ok := got[want]; !ok || v != "" { | ||
| 188 | t.Errorf("%s: %q %v in %v", want, v, ok, got) | ||
| 189 | } | ||
| 190 | } | ||
| 191 | } | ||
internal/deps/names.go added +54
| @@ -0,0 +1,54 @@ | |||
| 1 | package deps | ||
| 2 | |||
| 3 | import ( | ||
| 4 | "encoding/json" | ||
| 5 | |||
| 6 | "github.com/BurntSushi/toml" | ||
| 7 | ) | ||
| 8 | |||
| 9 | // Names returns the direct dependencies the manifests in read name, whatever | ||
| 10 | // version each asks for, with Current left empty. Code scanning uses it to | ||
| 11 | // tell which libraries a repository uses; a range or a git source still | ||
| 12 | // means the library is there. | ||
| 13 | func Names(read ReadFile) []Dep { | ||
| 14 | var out []Dep | ||
| 15 | for _, d := range parseGoMod(read) { | ||
| 16 | out = append(out, Dep{Ecosystem: EcoGo, Name: d.Name}) | ||
| 17 | } | ||
| 18 | for _, d := range pythonDeps(read, func(string) string { return "any" }) { | ||
| 19 | out = append(out, Dep{Ecosystem: EcoPyPI, Name: d.Name}) | ||
| 20 | } | ||
| 21 | if raw, err := read("package.json"); err == nil { | ||
| 22 | var pkg struct { | ||
| 23 | Dependencies map[string]any `json:"dependencies"` | ||
| 24 | DevDependencies map[string]any `json:"devDependencies"` | ||
| 25 | PeerDependencies map[string]any `json:"peerDependencies"` | ||
| 26 | } | ||
| 27 | if json.Unmarshal(raw, &pkg) == nil { | ||
| 28 | for _, set := range []map[string]any{pkg.Dependencies, pkg.DevDependencies, pkg.PeerDependencies} { | ||
| 29 | for name := range set { | ||
| 30 | out = append(out, Dep{Ecosystem: EcoNPM, Name: name}) | ||
| 31 | } | ||
| 32 | } | ||
| 33 | } | ||
| 34 | } | ||
| 35 | if raw, err := read("Cargo.toml"); err == nil { | ||
| 36 | var manifest struct { | ||
| 37 | Dependencies map[string]any `toml:"dependencies"` | ||
| 38 | DevDependencies map[string]any `toml:"dev-dependencies"` | ||
| 39 | BuildDependencies map[string]any `toml:"build-dependencies"` | ||
| 40 | } | ||
| 41 | if toml.Unmarshal(raw, &manifest) == nil { | ||
| 42 | for _, set := range []map[string]any{manifest.Dependencies, manifest.DevDependencies, manifest.BuildDependencies} { | ||
| 43 | for name := range set { | ||
| 44 | out = append(out, Dep{Ecosystem: EcoCargo, Name: name}) | ||
| 45 | } | ||
| 46 | } | ||
| 47 | } | ||
| 48 | } | ||
| 49 | return out | ||
| 50 | } | ||
| 51 | |||
| 52 | // ManifestFiles are the files Names reads, for finding the directories of a | ||
| 53 | // repository worth reading it in. | ||
| 54 | var ManifestFiles = []string{"go.mod", "requirements.txt", "pyproject.toml", "package.json", "Cargo.toml"} | ||
internal/httpd/depspage_test.go +28
| @@ -4,6 +4,7 @@ import ( | |||
| 4 | "strings" | 4 | "strings" |
| 5 | "testing" | 5 | "testing" |
| 6 | 6 | ||
| 7 | "gitbay.org/gitbay/internal/codescan" | ||
| 7 | "gitbay.org/gitbay/internal/control" | 8 | "gitbay.org/gitbay/internal/control" |
| 8 | "gitbay.org/gitbay/internal/web" | 9 | "gitbay.org/gitbay/internal/web" |
| 9 | ) | 10 | ) |
| @@ -46,3 +47,30 @@ func TestSettingsPageHidesDepsStatusWhenOff(t *testing.T) { | |||
| 46 | t.Error("the report shows with checks off") | 47 | t.Error("the report shows with checks off") |
| 47 | } | 48 | } |
| 48 | } | 49 | } |
| 50 | |||
| 51 | // The code scanning section offers each open finding for dismissal, each | ||
| 52 | // dismissed one for reopening, and says what the scan set aside. | ||
| 53 | func TestSettingsPageRendersSecurityTriage(t *testing.T) { | ||
| 54 | var sb strings.Builder | ||
| 55 | err := web.Render(&sb, "settings.html", settingsPage{ | ||
| 56 | repoPage: testRepoPage(), | ||
| 57 | Security: control.SecurityOut{ | ||
| 58 | Enabled: true, LastResults: "2026-11-01T04:02:00Z", | ||
| 59 | Findings: []control.SecurityFinding{{Fingerprint: "aaaa1111bbbb", Rule: "go.lang.x", Path: "a.go", Line: 3}}, | ||
| 60 | Dismissed: []control.SecurityFinding{{Fingerprint: "cccc2222dddd", Rule: "go.lang.y", Path: "b.go", Line: 9, Reason: "test", DismissedBy: "alice"}}, | ||
| 61 | Aside: codescan.Aside{Unused: map[string]int{"django": 26}}, | ||
| 62 | }, | ||
| 63 | }) | ||
| 64 | if err != nil { | ||
| 65 | t.Fatal(err) | ||
| 66 | } | ||
| 67 | page := sb.String() | ||
| 68 | for _, want := range []string{ | ||
| 69 | `value="security-dismiss"`, `value="aaaa1111bbbb"`, `value="security-reopen"`, `value="cccc2222dddd"`, | ||
| 70 | "by alice", "26 from django rules", `name="security-ignore"`, `name="security-exclude-rule"`, | ||
| 71 | } { | ||
| 72 | if !strings.Contains(page, want) { | ||
| 73 | t.Errorf("settings page does not show %q", want) | ||
| 74 | } | ||
| 75 | } | ||
| 76 | } | ||
internal/httpd/settings.go +8
| @@ -169,6 +169,14 @@ func (s *Server) settingsSubmit(w http.ResponseWriter, r *http.Request, u store. | |||
| 169 | verb = "enable" | 169 | verb = "enable" |
| 170 | } | 170 | } |
| 171 | argv = []string{"repo", "security", verb, repo} | 171 | argv = []string{"repo", "security", verb, repo} |
| 172 | case "security-dismiss": | ||
| 173 | argv = []string{"repo", "security", "dismiss", repo, v("fingerprint"), "--reason", v("reason")} | ||
| 174 | case "security-reopen": | ||
| 175 | argv = []string{"repo", "security", "reopen", repo, v("fingerprint")} | ||
| 176 | case "security-ignore": | ||
| 177 | argv = append([]string{"repo", "settings", "security-ignore", repo}, strings.Fields(v("security-ignore"))...) | ||
| 178 | case "security-exclude-rule": | ||
| 179 | argv = append([]string{"repo", "settings", "security-exclude-rule", repo}, strings.Fields(v("security-exclude-rule"))...) | ||
| 172 | case "secret-scan": | 180 | case "secret-scan": |
| 173 | argv = []string{"repo", "settings", "secret-scan", repo, v("secret-scan")} | 181 | argv = []string{"repo", "settings", "secret-scan", repo, v("secret-scan")} |
| 174 | case "secret-scan-allow": | 182 | case "secret-scan-allow": |
internal/store/migrations/0083_security_dismissals.down.sql added +2
| @@ -0,0 +1,2 @@ | |||
| 1 | ALTER TABLE security_scans DROP COLUMN aside; | ||
| 2 | DROP TABLE security_dismissals; | ||
internal/store/migrations/0083_security_dismissals.up.sql added +16
| @@ -0,0 +1,16 @@ | |||
| 1 | -- A finding a repository dismissed, by fingerprint, so it stays dismissed | ||
| 2 | -- across scans and line moves. Kept while scanning is off: turning it back | ||
| 3 | -- on does not bring back what was decided. by_id is NULL once that | ||
| 4 | -- account is gone. | ||
| 5 | CREATE TABLE security_dismissals ( | ||
| 6 | repo_id INTEGER NOT NULL REFERENCES repos(id) ON DELETE CASCADE, | ||
| 7 | fingerprint TEXT NOT NULL, | ||
| 8 | reason TEXT NOT NULL, | ||
| 9 | by_id INTEGER REFERENCES users(id) ON DELETE SET NULL, | ||
| 10 | created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')), | ||
| 11 | PRIMARY KEY (repo_id, fingerprint) | ||
| 12 | ); | ||
| 13 | |||
| 14 | -- What the last scan set aside rather than reported (codescan.Aside, as | ||
| 15 | -- JSON), so a change in it rewrites the issue. | ||
| 16 | ALTER TABLE security_scans ADD COLUMN aside TEXT NOT NULL DEFAULT ''; | ||
internal/store/repos.go +4
| @@ -42,6 +42,10 @@ type RepoSettings struct { | |||
| 42 | SecretScan string `json:"secret_scan,omitempty"` | 42 | SecretScan string `json:"secret_scan,omitempty"` |
| 43 | // SecretScanAllow are CI path globs whose findings are ignored. | 43 | // SecretScanAllow are CI path globs whose findings are ignored. |
| 44 | SecretScanAllow []string `json:"secret_scan_allow,omitempty"` | 44 | SecretScanAllow []string `json:"secret_scan_allow,omitempty"` |
| 45 | // SecurityIgnore are CI path globs, and SecurityExcludeRules rule ids | ||
| 46 | // or families, whose code scanning findings are set aside. | ||
| 47 | SecurityIgnore []string `json:"security_ignore,omitempty"` | ||
| 48 | SecurityExcludeRules []string `json:"security_exclude_rules,omitempty"` | ||
| 45 | } | 49 | } |
| 46 | 50 | ||
| 47 | // Path returns the canonical owner/name form. | 51 | // Path returns the canonical owner/name form. |
internal/store/security.go +73 −4
| @@ -12,6 +12,7 @@ type SecurityScan struct { | |||
| 12 | NextRun string | 12 | NextRun string |
| 13 | LastResults string // when results were last uploaded, '' for never | 13 | LastResults string // when results were last uploaded, '' for never |
| 14 | IssueNumber int64 | 14 | IssueNumber int64 |
| 15 | Aside string // what the last scan set aside, as JSON | ||
| 15 | } | 16 | } |
| 16 | 17 | ||
| 17 | // SecurityFinding is one result from a scan, as last reported. | 18 | // SecurityFinding is one result from a scan, as last reported. |
| @@ -49,8 +50,8 @@ func (s *Store) DisableSecurityScan(repoID int64) error { | |||
| 49 | func (s *Store) SecurityScanFor(repoID int64) (SecurityScan, error) { | 50 | func (s *Store) SecurityScanFor(repoID int64) (SecurityScan, error) { |
| 50 | var sc SecurityScan | 51 | var sc SecurityScan |
| 51 | err := s.DB.QueryRow( | 52 | err := s.DB.QueryRow( |
| 52 | "SELECT repo_id, next_run, last_results, issue_number FROM security_scans WHERE repo_id = ?", repoID). | 53 | "SELECT repo_id, next_run, last_results, issue_number, aside FROM security_scans WHERE repo_id = ?", repoID). |
| 53 | Scan(&sc.RepoID, &sc.NextRun, &sc.LastResults, &sc.IssueNumber) | 54 | Scan(&sc.RepoID, &sc.NextRun, &sc.LastResults, &sc.IssueNumber, &sc.Aside) |
| 54 | if errors.Is(err, sql.ErrNoRows) { | 55 | if errors.Is(err, sql.ErrNoRows) { |
| 55 | return sc, ErrNotFound | 56 | return sc, ErrNotFound |
| 56 | } | 57 | } |
| @@ -111,13 +112,17 @@ func (s *Store) ReportedFindings(repoID int64) ([]SecurityFinding, error) { | |||
| 111 | return out, rows.Err() | 112 | return out, rows.Err() |
| 112 | } | 113 | } |
| 113 | 114 | ||
| 114 | // ReplaceFindings swaps in an upload's findings wholesale. | 115 | // ReplaceFindings swaps in an upload's findings wholesale, with what it set |
| 115 | func (s *Store) ReplaceFindings(repoID int64, findings []SecurityFinding) error { | 116 | // aside. |
| 117 | func (s *Store) ReplaceFindings(repoID int64, findings []SecurityFinding, aside string) error { | ||
| 116 | tx, err := s.DB.Begin() | 118 | tx, err := s.DB.Begin() |
| 117 | if err != nil { | 119 | if err != nil { |
| 118 | return err | 120 | return err |
| 119 | } | 121 | } |
| 120 | defer tx.Rollback() | 122 | defer tx.Rollback() |
| 123 | if _, err := tx.Exec("UPDATE security_scans SET aside = ? WHERE repo_id = ?", aside, repoID); err != nil { | ||
| 124 | return err | ||
| 125 | } | ||
| 121 | if _, err := tx.Exec("DELETE FROM security_findings WHERE repo_id = ?", repoID); err != nil { | 126 | if _, err := tx.Exec("DELETE FROM security_findings WHERE repo_id = ?", repoID); err != nil { |
| 122 | return err | 127 | return err |
| 123 | } | 128 | } |
| @@ -138,3 +143,67 @@ func (s *Store) StampSecurityResults(repoID int64) error { | |||
| 138 | WHERE repo_id = ?`, repoID) | 143 | WHERE repo_id = ?`, repoID) |
| 139 | return err | 144 | return err |
| 140 | } | 145 | } |
| 146 | |||
| 147 | // SecurityDismissal is a finding a repository dismissed. | ||
| 148 | type SecurityDismissal struct { | ||
| 149 | Fingerprint string | ||
| 150 | Reason string | ||
| 151 | By string // username, '' once the account is gone | ||
| 152 | CreatedAt string | ||
| 153 | } | ||
| 154 | |||
| 155 | // DismissFindings dismisses findings by fingerprint, replacing the reason | ||
| 156 | // of any already dismissed. | ||
| 157 | func (s *Store) DismissFindings(repoID, byID int64, fingerprints []string, reason string) error { | ||
| 158 | tx, err := s.DB.Begin() | ||
| 159 | if err != nil { | ||
| 160 | return err | ||
| 161 | } | ||
| 162 | defer tx.Rollback() | ||
| 163 | for _, fp := range fingerprints { | ||
| 164 | if _, err := tx.Exec(` | ||
| 165 | INSERT INTO security_dismissals (repo_id, fingerprint, reason, by_id) VALUES (?, ?, ?, ?) | ||
| 166 | ON CONFLICT (repo_id, fingerprint) DO UPDATE SET reason = excluded.reason, by_id = excluded.by_id, | ||
| 167 | created_at = strftime('%Y-%m-%dT%H:%M:%fZ','now')`, | ||
| 168 | repoID, fp, reason, byID); err != nil { | ||
| 169 | return err | ||
| 170 | } | ||
| 171 | } | ||
| 172 | return tx.Commit() | ||
| 173 | } | ||
| 174 | |||
| 175 | // ReopenFindings takes dismissals back. | ||
| 176 | func (s *Store) ReopenFindings(repoID int64, fingerprints []string) error { | ||
| 177 | tx, err := s.DB.Begin() | ||
| 178 | if err != nil { | ||
| 179 | return err | ||
| 180 | } | ||
| 181 | defer tx.Rollback() | ||
| 182 | for _, fp := range fingerprints { | ||
| 183 | if _, err := tx.Exec("DELETE FROM security_dismissals WHERE repo_id = ? AND fingerprint = ?", repoID, fp); err != nil { | ||
| 184 | return err | ||
| 185 | } | ||
| 186 | } | ||
| 187 | return tx.Commit() | ||
| 188 | } | ||
| 189 | |||
| 190 | // SecurityDismissals returns a repo's dismissals by fingerprint. | ||
| 191 | func (s *Store) SecurityDismissals(repoID int64) (map[string]SecurityDismissal, error) { | ||
| 192 | rows, err := s.DB.Query(` | ||
| 193 | SELECT d.fingerprint, d.reason, COALESCE(u.username, ''), d.created_at | ||
| 194 | FROM security_dismissals d LEFT JOIN users u ON u.id = d.by_id | ||
| 195 | WHERE d.repo_id = ?`, repoID) | ||
| 196 | if err != nil { | ||
| 197 | return nil, err | ||
| 198 | } | ||
| 199 | defer rows.Close() | ||
| 200 | out := map[string]SecurityDismissal{} | ||
| 201 | for rows.Next() { | ||
| 202 | var d SecurityDismissal | ||
| 203 | if err := rows.Scan(&d.Fingerprint, &d.Reason, &d.By, &d.CreatedAt); err != nil { | ||
| 204 | return nil, err | ||
| 205 | } | ||
| 206 | out[d.Fingerprint] = d | ||
| 207 | } | ||
| 208 | return out, rows.Err() | ||
| 209 | } | ||
internal/web/templates/settings.html +34 −2
| @@ -182,16 +182,48 @@ | |||
| 182 | </form> | 182 | </form> |
| 183 | {{if .Security.Enabled}} | 183 | {{if .Security.Enabled}} |
| 184 | <p class="meta">Next scan {{.Security.NextRun}} · last results {{if .Security.LastResults}}{{when .Security.LastResults}}{{else}}never{{end}}{{if .Security.IssueNumber}} · <a href="/{{.Repo.OwnerName}}/{{.Repo.Name}}/issues/{{.Security.IssueNumber}}">#{{.Security.IssueNumber}}</a>{{end}}</p> | 184 | <p class="meta">Next scan {{.Security.NextRun}} · last results {{if .Security.LastResults}}{{when .Security.LastResults}}{{else}}never{{end}}{{if .Security.IssueNumber}} · <a href="/{{.Repo.OwnerName}}/{{.Repo.Name}}/issues/{{.Security.IssueNumber}}">#{{.Security.IssueNumber}}</a>{{end}}</p> |
| 185 | {{with .Security.Aside}}{{if or .Ignored .Unused}}<p class="meta">Set aside by the last scan:{{if .Ignored}} {{.Ignored}} under the paths or rules below, or marked vendored or generated in <code>.gitattributes</code>.{{end}}{{range $lib, $n := .Unused}} {{$n}} from {{$lib}} rules, which no manifest names.{{end}}</p>{{end}}{{end}} | ||
| 185 | {{if .Security.Findings}}<div class="tablewrap"><table class="keys"> | 186 | {{if .Security.Findings}}<div class="tablewrap"><table class="keys"> |
| 186 | <tr class="cols"><th scope="col">location</th><th scope="col">rule</th><th scope="col">level</th></tr> | 187 | <tr class="cols"><th scope="col">location</th><th scope="col">rule</th><th scope="col">level</th><th scope="col"><span class="vh">dismiss</span></th></tr> |
| 187 | {{range .Security.Findings}}<tr> | 188 | {{range .Security.Findings}}<tr> |
| 188 | <td class="mono">{{.Path}}:{{.Line}}</td> | 189 | <td class="mono">{{.Path}}:{{.Line}}</td> |
| 189 | <td class="mono">{{.Rule}}</td> | 190 | <td class="mono">{{.Rule}}</td> |
| 190 | <td>{{.Level}}</td> | 191 | <td>{{.Level}}</td> |
| 192 | <td><form method="post" action="{{$base}}" class="inline"> | ||
| 193 | <input type="hidden" name="field" value="security-dismiss"> | ||
| 194 | <input type="hidden" name="fingerprint" value="{{.Fingerprint}}"> | ||
| 195 | <select name="reason" aria-label="Reason"><option value="false-positive">False positive</option><option value="wont-fix">Won't fix</option><option value="test">Used in tests</option></select> | ||
| 196 | <button type="submit">Dismiss</button> | ||
| 197 | </form></td> | ||
| 191 | </tr> | 198 | </tr> |
| 192 | {{end}}</table></div> | 199 | {{end}}</table></div> |
| 193 | {{else if .Security.LastResults}}<p class="none">The last scan found nothing.</p>{{end}} | 200 | {{else if .Security.LastResults}}<p class="none">The last scan found nothing open.</p>{{end}} |
| 201 | {{if .Security.Dismissed}}<h3>Dismissed</h3><div class="tablewrap"><table class="keys"> | ||
| 202 | <tr class="cols"><th scope="col">location</th><th scope="col">rule</th><th scope="col">reason</th><th scope="col"><span class="vh">reopen</span></th></tr> | ||
| 203 | {{range .Security.Dismissed}}<tr> | ||
| 204 | <td class="mono">{{.Path}}:{{.Line}}</td> | ||
| 205 | <td class="mono">{{.Rule}}</td> | ||
| 206 | <td>{{.Reason}}{{if .DismissedBy}} <span class="meta">by {{.DismissedBy}}</span>{{end}}</td> | ||
| 207 | <td><form method="post" action="{{$base}}" class="inline"> | ||
| 208 | <input type="hidden" name="field" value="security-reopen"> | ||
| 209 | <input type="hidden" name="fingerprint" value="{{.Fingerprint}}"> | ||
| 210 | <button type="submit">Reopen</button> | ||
| 211 | </form></td> | ||
| 212 | </tr> | ||
| 213 | {{end}}</table></div>{{end}} | ||
| 194 | {{end}} | 214 | {{end}} |
| 215 | <form method="post" action="{{$base}}" class="setform"> | ||
| 216 | <input type="hidden" name="field" value="security-ignore"> | ||
| 217 | <div><label for="security-ignore">Ignored paths</label><p class="hint">Findings under these paths are set aside from the next scan. Path globs separated by spaces: <code>testdata/**</code>, <code>web/templates/*.html</code>. Paths <code>.gitattributes</code> marks <code>linguist-vendored</code> or <code>linguist-generated</code> are set aside without being listed.</p></div> | ||
| 218 | <div><input type="text" id="security-ignore" name="security-ignore" value="{{join .Repo.Settings.SecurityIgnore " "}}"></div> | ||
| 219 | <div><button type="submit" class="btn">Save</button></div> | ||
| 220 | </form> | ||
| 221 | <form method="post" action="{{$base}}" class="setform"> | ||
| 222 | <input type="hidden" name="field" value="security-exclude-rule"> | ||
| 223 | <div><label for="security-exclude-rule">Excluded rules</label><p class="hint">Findings from these rules are set aside from the next scan. A rule id or a family of them, separated by spaces: <code>swift.lang.crypto.insecure-random</code>, <code>javascript.browser</code>.</p></div> | ||
| 224 | <div><input type="text" id="security-exclude-rule" name="security-exclude-rule" value="{{join .Repo.Settings.SecurityExcludeRules " "}}"></div> | ||
| 225 | <div><button type="submit" class="btn">Save</button></div> | ||
| 226 | </form> | ||
| 195 | </section> | 227 | </section> |
| 196 | {{end}} | 228 | {{end}} |
| 197 | 229 | ||