Commit 96f3442ce1

96f3442ce1f25c4871bfb9db23d9ec19c902e7d8

parent: 359b4a6fb4

Verified · cmc ci/build: success ci/quality: success ci/test: success

cmc <hello@cleberg.net> · 2026-11-01 06:07 UTC

Code scanning sets aside what does not apply; dismissals

- Findings from a rule about a library (opengrep-rules' <lang>/<lib>/)
  count only where a manifest at the scanned commit names the library,
  in any directory outside node_modules, vendor and third_party;
  yaml/semgrep lints only where the repository has a .semgrep config.
- Paths .gitattributes marks linguist-vendored or linguist-generated
  are set aside, as are repo settings security-ignore (path globs) and
  security-exclude-rule (rule ids or families).
- repo security dismiss <repo> <fingerprint>... --reason
  false-positive|wont-fix|test, and repo security reopen. Dismissals are
  kept by fingerprint (migration 0083) and rewrite the issue at once.
- The issue lists open findings with a short fingerprint, counts what
  was dismissed or set aside, and closes when nothing is open. Status
  and the settings page show the same, and the page can dismiss,
  reopen, and edit both lists.

Closes #344

Layout: unified · split

.gitbay/wiki/Admin.org +14
@@ -477,6 +477,20 @@ the findings replace the repository's last set; =gitbay-bot= keeps a
477the dependency issues are. An older runner runs the steps and uploads 477the dependency issues are. An older runner runs the steps and uploads
478nothing, so the issue stays as it was. 478nothing, so the issue stays as it was.
479 479
480Not every finding is listed. A rule from one of opengrep-rules' library
481directories (=python/django=, =javascript/jquery=,
482=typescript/react/portability/i18next=, …) counts only where a manifest
483at the scanned commit names that library: =go.mod=, =requirements.txt=,
484=pyproject.toml=, =package.json= or =Cargo.toml=, in any directory
485outside =node_modules=, =vendor= and =third_party=. The =yaml/semgrep=
486lints count only in a repository with a =.semgrep= config. Paths
487=.gitattributes= marks =linguist-vendored= or =linguist-generated= are
488set aside, as are the repository's =security-ignore= paths and
489=security-exclude-rule= rules. The issue says how many were set aside
490and why. A finding a writer dismisses (=repo security dismiss=, by the
491fingerprint the issue prints) stays out of the issue across scans until
492it is reopened; the issue closes when nothing is left open.
493
480The bundled rules are =opengrep/opengrep-rules=, LGPL-2.1 with the 494The bundled rules are =opengrep/opengrep-rules=, LGPL-2.1 with the
481Commons Clause: an instance that sells hosting should supply its own. 495Commons Clause: an instance that sells hosting should supply its own.
482 496
.gitbay/wiki/Architecture/02-Components.org +1 −1
@@ -38,7 +38,7 @@ the hidden =hook= used by git (=cmd/gitbayd/main.go=,
38| =internal/mailin=, =internal/imapc=, =internal/mailreply= | Reply by mail: the IMAP client, the reply token, and the processor that posts a reply through =issue comment= / =mr comment=. | 38| =internal/mailin=, =internal/imapc=, =internal/mailreply= | Reply by mail: the IMAP client, the reply token, and the processor that posts a reply through =issue comment= / =mr comment=. |
39| =internal/deps= | Dependency manifest and lockfile parsing; registry and OSV checks (opt-in per repository). | 39| =internal/deps= | Dependency manifest and lockfile parsing; registry and OSV checks (opt-in per repository). |
40| =internal/leaks= | Credential rules and the blob scan pre-receive and =repo secret-scan= run. | 40| =internal/leaks= | Credential rules and the blob scan pre-receive and =repo secret-scan= run. |
41| =internal/codescan= | SARIF from the code scan, and the findings issue's body. | 41| =internal/codescan= | SARIF from the code scan, what a repository sets aside, the findings issue. |
42| =internal/botissue= | The issues gitbay-bot keeps: open on first report, rewrite on change, close when empty. | 42| =internal/botissue= | The issues gitbay-bot keeps: open on first report, rewrite on change, close when empty. |
43| =internal/config= | Configuration load and validation. | 43| =internal/config= | Configuration load and validation. |
44| =internal/web= | Embedded templates, stylesheet and fonts. | 44| =internal/web= | Embedded templates, stylesheet and fonts. |
.gitbay/wiki/Architecture/06-Data-and-Cryptography.org +1 −1
@@ -20,7 +20,7 @@ content (as confidential as the repository), *O* operational.
20| Signatures | =commit_signatures=, =settings.key_epoch= | O | verification cache | 20| Signatures | =commit_signatures=, =settings.key_epoch= | O | verification cache |
21| Audit and feed | =audit_log=, =events= | O, P | actor ids, pruned argv, fingerprints and IPs in some audit rows, a hash chain (=prev_hash=, =hash=) | 21| Audit and feed | =audit_log=, =events= | O, P | actor ids, pruned argv, fingerprints and IPs in some audit rows, a hash chain (=prev_hash=, =hash=) |
22| Dependencies | =dep_checks=, =dep_reports=, =dep_advisories= | O | 22| Dependencies | =dep_checks=, =dep_reports=, =dep_advisories= | O |
23| Code scanning | =security_scans=, =security_findings= | O | | 23| Code scanning | =security_scans=, =security_findings=, =security_dismissals= | O | |
24 24
25Outside the database: 25Outside the database:
26 26
.gitbay/wiki/Parity.org +2
@@ -249,6 +249,8 @@ there shows the app's rendering rather than the one the web page shows.
249| code scanning on/off | yes | yes | yes | 249| code scanning on/off | yes | yes | yes |
250| code scanning status | yes | yes | yes | 250| code scanning status | yes | yes | yes |
251| code scan now | yes | no | yes | 251| code scan now | yes | no | yes |
252| code scan dismiss, reopen | yes | yes | no |
253| code scan ignores, rules | yes | yes | no |
252| dependency checks on/off | yes | yes | yes | 254| dependency checks on/off | yes | yes | yes |
253| dependency status | yes | yes | yes | 255| dependency status | yes | yes | yes |
254| delete, transfer | yes | yes | yes | 256| delete, transfer | yes | yes | yes |
.gitbay/wiki/Users.org +4
@@ -243,6 +243,10 @@ gitbay repo settings secret-scan-allow you/project 'testdata/**' # paths whose
243gitbay repo secret-scan you/project # scan the whole history 243gitbay repo secret-scan you/project # scan the whole history
244gitbay repo security enable you/project # scheduled code scan, where the instance offers one 244gitbay repo security enable you/project # scheduled code scan, where the instance offers one
245gitbay repo security status you/project # its findings, also kept in an issue 245gitbay repo security status you/project # its findings, also kept in an issue
246gitbay repo security dismiss you/project 3f9a0c1d2e --reason false-positive # or wont-fix, test
247gitbay repo security reopen you/project 3f9a0c1d2e
248gitbay repo settings security-ignore you/project 'testdata/**' # paths set aside from the scan
249gitbay repo settings security-exclude-rule you/project swift.lang.crypto.insecure-random
246gitbay repo settings git-daemon you/project on # expose over git:// 250gitbay repo settings git-daemon you/project on # expose over git://
247gitbay repo topics add you/project cli forge # free-form tags, shown on the web 251gitbay repo topics add you/project cli forge # free-form tags, shown on the web
248gitbay repo search forge # find repos by name/description/topic 252gitbay repo search forge # find repos by name/description/topic
cmd/gitbay/main.go +4
@@ -602,6 +602,8 @@ func repoCmd() *cobra.Command {
602 pass("disable", passOpts{server: []string{"repo", "security", "disable"}, needsRepo: true}), 602 pass("disable", passOpts{server: []string{"repo", "security", "disable"}, needsRepo: true}),
603 pass("run", passOpts{server: []string{"repo", "security", "run"}, needsRepo: true}), 603 pass("run", passOpts{server: []string{"repo", "security", "run"}, needsRepo: true}),
604 pass("status", passOpts{server: []string{"repo", "security", "status"}, needsRepo: true}), 604 pass("status", passOpts{server: []string{"repo", "security", "status"}, needsRepo: true}),
605 pass("dismiss", passOpts{server: []string{"repo", "security", "dismiss"}, needsRepo: true}),
606 pass("reopen", passOpts{server: []string{"repo", "security", "reopen"}, needsRepo: true}),
605 ), 607 ),
606 group("deps", "check dependencies against upstream registries and OSV", 608 group("deps", "check dependencies against upstream registries and OSV",
607 pass("enable", passOpts{server: []string{"repo", "deps", "enable"}, needsRepo: true}), 609 pass("enable", passOpts{server: []string{"repo", "deps", "enable"}, needsRepo: true}),
@@ -645,6 +647,8 @@ func repoCmd() *cobra.Command {
645 pass("require-signed", passOpts{server: []string{"repo", "settings", "require-signed"}, needsRepo: true}), 647 pass("require-signed", passOpts{server: []string{"repo", "settings", "require-signed"}, needsRepo: true}),
646 pass("secret-scan", passOpts{server: []string{"repo", "settings", "secret-scan"}, needsRepo: true}), 648 pass("secret-scan", passOpts{server: []string{"repo", "settings", "secret-scan"}, needsRepo: true}),
647 pass("secret-scan-allow", passOpts{server: []string{"repo", "settings", "secret-scan-allow"}, needsRepo: true}), 649 pass("secret-scan-allow", passOpts{server: []string{"repo", "settings", "secret-scan-allow"}, needsRepo: true}),
650 pass("security-ignore", passOpts{server: []string{"repo", "settings", "security-ignore"}, needsRepo: true}),
651 pass("security-exclude-rule", passOpts{server: []string{"repo", "settings", "security-exclude-rule"}, needsRepo: true}),
648 pass("require-mr", passOpts{server: []string{"repo", "settings", "require-mr"}, needsRepo: true}), 652 pass("require-mr", passOpts{server: []string{"repo", "settings", "require-mr"}, needsRepo: true}),
649 pass("description", passOpts{server: []string{"repo", "settings", "description"}, needsRepo: true}), 653 pass("description", passOpts{server: []string{"repo", "settings", "description"}, needsRepo: true}),
650 pass("website", passOpts{server: []string{"repo", "settings", "website"}, needsRepo: true}), 654 pass("website", passOpts{server: []string{"repo", "settings", "website"}, needsRepo: true}),
cmd/gitbay/summaries_gen.go +266 −262
@@ -3,266 +3,270 @@
3package main 3package main
4 4
5var summaries = map[string]string{ 5var summaries = map[string]string{
6 "account delete": "delete your account: mails a link, then purges seven days after it is opened", 6 "account delete": "delete your account: mails a link, then purges seven days after it is opened",
7 "account export": "write your account bundle (profile, repos, issues, MRs) as JSON", 7 "account export": "write your account bundle (profile, repos, issues, MRs) as JSON",
8 "account import-bundle": "replay an account bundle (see gitbay migrate)", 8 "account import-bundle": "replay an account bundle (see gitbay migrate)",
9 "admin email verify": "mark an address verified by admin assertion", 9 "admin email verify": "mark an address verified by admin assertion",
10 "admin invite": "issue a registration invite and mail its code", 10 "admin invite": "issue a registration invite and mail its code",
11 "admin mail inbound check": "connect to the reply mailbox read-only and report what is waiting", 11 "admin mail inbound check": "connect to the reply mailbox read-only and report what is waiting",
12 "admin mr prune": "drop merged or closed MRs' head refs and the objects only they kept, e.g. after a history rewrite (instance admins; audited)", 12 "admin mr prune": "drop merged or closed MRs' head refs and the objects only they kept, e.g. after a history rewrite (instance admins; audited)",
13 "admin org limits": "show or set an organization's repository and storage caps (instance admins)", 13 "admin org limits": "show or set an organization's repository and storage caps (instance admins)",
14 "admin repo archive": "archive any repository (instance admins; audited)", 14 "admin repo archive": "archive any repository (instance admins; audited)",
15 "admin repo delete": "delete any repository (instance admins; audited)", 15 "admin repo delete": "delete any repository (instance admins; audited)",
16 "admin repo list": "list every repository with size and last push (instance admins)", 16 "admin repo list": "list every repository with size and last push (instance admins)",
17 "admin repo unarchive": "unarchive any repository (instance admins; audited)", 17 "admin repo unarchive": "unarchive any repository (instance admins; audited)",
18 "admin repo visibility": "set any repository's visibility (instance admins; audited)", 18 "admin repo visibility": "set any repository's visibility (instance admins; audited)",
19 "admin runners forget": "alias of admin runners remove", 19 "admin runners forget": "alias of admin runners remove",
20 "admin runners remove": "drop a key's runner heartbeat row, e.g. one that polled once by mistake (instance admins)", 20 "admin runners remove": "drop a key's runner heartbeat row, e.g. one that polled once by mistake (instance admins)",
21 "admin runners": "the build queue and runner accounts: last poll, scope, the build each holds (instance admins)", 21 "admin runners": "the build queue and runner accounts: last poll, scope, the build each holds (instance admins)",
22 "admin stats": "instance statistics: counts and per-repository disk usage", 22 "admin stats": "instance statistics: counts and per-repository disk usage",
23 "admin symbols reindex": "rebuild a repository's symbol index, even when its tree is already indexed (instance admins)", 23 "admin symbols reindex": "rebuild a repository's symbol index, even when its tree is already indexed (instance admins)",
24 "admin user create": "create an account, optionally with a key and a verified address (instance admins)", 24 "admin user create": "create an account, optionally with a key and a verified address (instance admins)",
25 "admin user delete": "delete an account that anchors nothing (keys, emails and sessions go with it)", 25 "admin user delete": "delete an account that anchors nothing (keys, emails and sessions go with it)",
26 "admin user demote": "remove instance admin from an account (never the last one)", 26 "admin user demote": "remove instance admin from an account (never the last one)",
27 "admin user disable": "suspend an account: SSH, web sessions and API tokens refused until re-enabled", 27 "admin user disable": "suspend an account: SSH, web sessions and API tokens refused until re-enabled",
28 "admin user enable": "restore a suspended account", 28 "admin user enable": "restore a suspended account",
29 "admin user limits": "show or set an account's repository, storage and organization caps (instance admins)", 29 "admin user limits": "show or set an account's repository, storage and organization caps (instance admins)",
30 "admin user list": "list accounts (instance admins)", 30 "admin user list": "list accounts (instance admins)",
31 "admin user promote": "make an account an instance admin", 31 "admin user promote": "make an account an instance admin",
32 "admin user show": "show an account: keys, emails, orgs, tokens, sessions (instance admins)", 32 "admin user show": "show an account: keys, emails, orgs, tokens, sessions (instance admins)",
33 "audit": "instance audit log (admins)", 33 "audit": "instance audit log (admins)",
34 "build cancel": "withdraw a queued build before a runner claims it", 34 "build cancel": "withdraw a queued build before a runner claims it",
35 "build jobs": "list the jobs a trigger can name", 35 "build jobs": "list the jobs a trigger can name",
36 "build list": "list recent builds", 36 "build list": "list recent builds",
37 "build log": "print a build's log, or follow it until the build ends", 37 "build log": "print a build's log, or follow it until the build ends",
38 "build show": "show one build", 38 "build show": "show one build",
39 "build trigger": "queue a job now (scheduled or not)", 39 "build trigger": "queue a job now (scheduled or not)",
40 "dashboard": "one read for the account dashboard: review queue, assigned and open work, pins, activity, builds", 40 "dashboard": "one read for the account dashboard: review queue, assigned and open work, pins, activity, builds",
41 "email add": "add an address and mail a verification code", 41 "email add": "add an address and mail a verification code",
42 "email list": "list the addresses on your account", 42 "email list": "list the addresses on your account",
43 "email primary": "make a verified address the primary", 43 "email primary": "make a verified address the primary",
44 "email remove": "remove an address; not the primary, nor the last verified one", 44 "email remove": "remove an address; not the primary, nor the last verified one",
45 "email verify": "confirm a verification code", 45 "email verify": "confirm a verification code",
46 "explore": "list public repositories", 46 "explore": "list public repositories",
47 "feed": "activity on repositories you can reach", 47 "feed": "activity on repositories you can reach",
48 "help": "list available commands", 48 "help": "list available commands",
49 "issue assign": "add or remove assignees", 49 "issue assign": "add or remove assignees",
50 "issue close": "close an issue", 50 "issue close": "close an issue",
51 "issue comment": "add a comment", 51 "issue comment": "add a comment",
52 "issue create": "open an issue", 52 "issue create": "open an issue",
53 "issue edit": "edit title or body", 53 "issue edit": "edit title or body",
54 "issue label": "add or remove labels", 54 "issue label": "add or remove labels",
55 "issue list": "list issues", 55 "issue list": "list issues",
56 "issue milestone": "set or clear an issue's milestone", 56 "issue milestone": "set or clear an issue's milestone",
57 "issue react": "react to an issue or one of its comments", 57 "issue react": "react to an issue or one of its comments",
58 "issue reopen": "reopen an issue", 58 "issue reopen": "reopen an issue",
59 "issue show": "show an issue with comments", 59 "issue show": "show an issue with comments",
60 "issue templates": "list issue templates (.gitbay/issue-template*.md)", 60 "issue templates": "list issue templates (.gitbay/issue-template*.md)",
61 "keys add": "register an SSH public key (authorized_keys format)", 61 "keys add": "register an SSH public key (authorized_keys format)",
62 "keys label": "name a key; an empty label clears it", 62 "keys label": "name a key; an empty label clears it",
63 "keys list": "list registered SSH keys", 63 "keys list": "list registered SSH keys",
64 "keys remove": "remove an SSH key by fingerprint", 64 "keys remove": "remove an SSH key by fingerprint",
65 "label list": "list a repository's labels with colour and use", 65 "label list": "list a repository's labels with colour and use",
66 "label remove": "remove a label from the repository and from every issue and merge request", 66 "label remove": "remove a label from the repository and from every issue and merge request",
67 "label set": "create a label or set its colour", 67 "label set": "create a label or set its colour",
68 "milestone close": "close a milestone", 68 "milestone close": "close a milestone",
69 "milestone create": "create a milestone", 69 "milestone create": "create a milestone",
70 "milestone list": "list milestones with progress", 70 "milestone list": "list milestones with progress",
71 "milestone reopen": "reopen a milestone", 71 "milestone reopen": "reopen a milestone",
72 "mr apply-suggestion": "commit a review thread's suggestion to the source branch", 72 "mr apply-suggestion": "commit a review thread's suggestion to the source branch",
73 "mr close": "close without merging", 73 "mr close": "close without merging",
74 "mr comment": "add a comment", 74 "mr comment": "add a comment",
75 "mr create": "open a merge request", 75 "mr create": "open a merge request",
76 "mr diff": "show the diff", 76 "mr diff": "show the diff",
77 "mr diff-comment": "comment on a diff line", 77 "mr diff-comment": "comment on a diff line",
78 "mr draft": "mark a merge request as work in progress", 78 "mr draft": "mark a merge request as work in progress",
79 "mr edit": "edit title or body", 79 "mr edit": "edit title or body",
80 "mr label": "add or remove labels", 80 "mr label": "add or remove labels",
81 "mr list": "list merge requests", 81 "mr list": "list merge requests",
82 "mr merge": "merge", 82 "mr merge": "merge",
83 "mr milestone": "set or clear an MR's milestone", 83 "mr milestone": "set or clear an MR's milestone",
84 "mr range-diff": "what changed between two revisions of a merge request", 84 "mr range-diff": "what changed between two revisions of a merge request",
85 "mr react": "react to a merge request or one of its comments", 85 "mr react": "react to a merge request or one of its comments",
86 "mr ready": "take the draft mark off, so it can merge", 86 "mr ready": "take the draft mark off, so it can merge",
87 "mr resolve": "resolve a review thread", 87 "mr resolve": "resolve a review thread",
88 "mr retarget": "retarget onto another branch", 88 "mr retarget": "retarget onto another branch",
89 "mr review request": "ask specific people for a review", 89 "mr review request": "ask specific people for a review",
90 "mr review": "record a review verdict", 90 "mr review": "record a review verdict",
91 "mr revisions": "the heads a merge request has had", 91 "mr revisions": "the heads a merge request has had",
92 "mr show": "show a merge request", 92 "mr show": "show a merge request",
93 "mr threads": "review threads on an MR", 93 "mr threads": "review threads on an MR",
94 "mr unresolve": "reopen a review thread", 94 "mr unresolve": "reopen a review thread",
95 "notifications device add": "register an Apple device for push, token on stdin", 95 "notifications device add": "register an Apple device for push, token on stdin",
96 "notifications device list": "your registered devices", 96 "notifications device list": "your registered devices",
97 "notifications device remove": "deregister a device", 97 "notifications device remove": "deregister a device",
98 "notifications list": "your notification inbox, newest first", 98 "notifications list": "your notification inbox, newest first",
99 "notifications read": "mark notifications read", 99 "notifications read": "mark notifications read",
100 "notifications settings mail": "activity by mail as well as the inbox (login links are unaffected)", 100 "notifications settings mail": "activity by mail as well as the inbox (login links are unaffected)",
101 "notifications settings push": "activity on your registered devices as well as the inbox", 101 "notifications settings push": "activity on your registered devices as well as the inbox",
102 "notifications settings reply": "reply to issue and merge request mail to comment", 102 "notifications settings reply": "reply to issue and merge request mail to comment",
103 "notifications settings show": "your notification preferences", 103 "notifications settings show": "your notification preferences",
104 "notifications settings watch": "every issue and merge request on repositories you can write to", 104 "notifications settings watch": "every issue and merge request on repositories you can write to",
105 "org create": "create an organization (you become its first admin)", 105 "org create": "create an organization (you become its first admin)",
106 "org delete": "delete an empty organization", 106 "org delete": "delete an empty organization",
107 "org label list": "list an org's labels with use across the repositories you can read", 107 "org label list": "list an org's labels with use across the repositories you can read",
108 "org label remove": "remove an org label from the org and from every issue under it", 108 "org label remove": "remove an org label from the org and from every issue under it",
109 "org label set": "create an org label every org repository sees, or set its colour; folds in same-named repo labels", 109 "org label set": "create an org label every org repository sees, or set its colour; folds in same-named repo labels",
110 "org list": "list organizations you belong to", 110 "org list": "list organizations you belong to",
111 "org members add": "add or update a member", 111 "org members add": "add or update a member",
112 "org members list": "list members", 112 "org members list": "list members",
113 "org members remove": "remove a member", 113 "org members remove": "remove a member",
114 "org milestone close": "close an org milestone", 114 "org milestone close": "close an org milestone",
115 "org milestone create": "create an org milestone spanning every org repository; folds in same-titled repo milestones", 115 "org milestone create": "create an org milestone spanning every org repository; folds in same-titled repo milestones",
116 "org milestone list": "list an org's milestones with progress across the repositories you can read", 116 "org milestone list": "list an org's milestones with progress across the repositories you can read",
117 "org milestone reopen": "reopen an org milestone", 117 "org milestone reopen": "reopen an org milestone",
118 "org profile": "show or set an org's profile", 118 "org profile": "show or set an org's profile",
119 "org rename": "rename an organization", 119 "org rename": "rename an organization",
120 "org settings members-role": "role plain membership implies on every org repo", 120 "org settings members-role": "role plain membership implies on every org repo",
121 "org show": "show an organization and its members", 121 "org show": "show an organization and its members",
122 "org team add": "add org members to a team", 122 "org team add": "add org members to a team",
123 "org team create": "create a team", 123 "org team create": "create a team",
124 "org team delete": "delete a team (its grants with it)", 124 "org team delete": "delete a team (its grants with it)",
125 "org team grant": "grant a team a role on an org repo", 125 "org team grant": "grant a team a role on an org repo",
126 "org team list": "list an org's teams", 126 "org team list": "list an org's teams",
127 "org team remove": "remove members from a team", 127 "org team remove": "remove members from a team",
128 "org team revoke": "revoke a team's grant", 128 "org team revoke": "revoke a team's grant",
129 "org team show": "show a team's members and grants", 129 "org team show": "show a team's members and grants",
130 "pgp add": "register an OpenPGP public key (armored)", 130 "pgp add": "register an OpenPGP public key (armored)",
131 "pgp list": "list registered OpenPGP keys", 131 "pgp list": "list registered OpenPGP keys",
132 "pgp remove": "remove an OpenPGP key by fingerprint", 132 "pgp remove": "remove an OpenPGP key by fingerprint",
133 "profile set": "set your profile", 133 "profile set": "set your profile",
134 "profile show": "show a user's or org's profile", 134 "profile show": "show a user's or org's profile",
135 "query list": "list your saved queries", 135 "query list": "list your saved queries",
136 "query pin": "show a saved query on your dashboard", 136 "query pin": "show a saved query on your dashboard",
137 "query remove": "delete a saved query", 137 "query remove": "delete a saved query",
138 "query run": "list the issues and merge requests a saved query matches", 138 "query run": "list the issues and merge requests a saved query matches",
139 "query save": "save an issue and merge request query across repositories under a name", 139 "query save": "save an issue and merge request query across repositories under a name",
140 "query show": "show a saved query and how many rows it matches", 140 "query show": "show a saved query and how many rows it matches",
141 "query unpin": "take a saved query off your dashboard", 141 "query unpin": "take a saved query off your dashboard",
142 "register": "create an account (only meaningful for unregistered keys)", 142 "register": "create an account (only meaningful for unregistered keys)",
143 "release asset add": "upload an asset from stdin", 143 "release asset add": "upload an asset from stdin",
144 "release asset get": "write an asset to stdout", 144 "release asset get": "write an asset to stdout",
145 "release asset remove": "remove an asset", 145 "release asset remove": "remove an asset",
146 "release create": "create a release on a tag", 146 "release create": "create a release on a tag",
147 "release delete": "delete a release and its assets", 147 "release delete": "delete a release and its assets",
148 "release edit": "update a release's title and notes", 148 "release edit": "update a release's title and notes",
149 "release list": "list releases", 149 "release list": "list releases",
150 "release notes": "draft notes: one line per merge request since the previous release", 150 "release notes": "draft notes: one line per merge request since the previous release",
151 "release show": "show a release with assets", 151 "release show": "show a release with assets",
152 "repo access grant": "grant access", 152 "repo access grant": "grant access",
153 "repo access list": "list who can reach the repository, with the role and where it comes from", 153 "repo access list": "list who can reach the repository, with the role and where it comes from",
154 "repo access revoke": "revoke access", 154 "repo access revoke": "revoke access",
155 "repo archive": "archive a repository (read-only: pushes and issue/MR writes refused)", 155 "repo archive": "archive a repository (read-only: pushes and issue/MR writes refused)",
156 "repo blame": "attribute lines to commits", 156 "repo blame": "attribute lines to commits",
157 "repo bookmark": "bookmark a repository to come back to", 157 "repo bookmark": "bookmark a repository to come back to",
158 "repo bookmarks": "list the repositories you have bookmarked", 158 "repo bookmarks": "list the repositories you have bookmarked",
159 "repo cat": "read a file", 159 "repo cat": "read a file",
160 "repo commit": "show one commit with its patch", 160 "repo commit": "show one commit with its patch",
161 "repo commit-file": "write a file and commit it", 161 "repo commit-file": "write a file and commit it",
162 "repo create": "create a repository", 162 "repo create": "create a repository",
163 "repo definitions": "where each name a file uses is defined, from the symbol index", 163 "repo definitions": "where each name a file uses is defined, from the symbol index",
164 "repo delete": "delete a repository", 164 "repo delete": "delete a repository",
165 "repo deploy-key add": "bind a read-only (or --rw) key to one repository", 165 "repo deploy-key add": "bind a read-only (or --rw) key to one repository",
166 "repo deploy-key list": "list deploy keys", 166 "repo deploy-key list": "list deploy keys",
167 "repo deploy-key remove": "remove a deploy key", 167 "repo deploy-key remove": "remove a deploy key",
168 "repo deps disable": "stop checking dependencies", 168 "repo deps disable": "stop checking dependencies",
169 "repo deps enable": "check dependencies for updates and advisories", 169 "repo deps enable": "check dependencies for updates and advisories",
170 "repo deps status": "show dependency check state", 170 "repo deps status": "show dependency check state",
171 "repo diff": "the patch between two refs, from their merge base", 171 "repo diff": "the patch between two refs, from their merge base",
172 "repo domain add": "claim a custom pages domain (verify with a DNS TXT record)", 172 "repo domain add": "claim a custom pages domain (verify with a DNS TXT record)",
173 "repo domain list": "list custom pages domains", 173 "repo domain list": "list custom pages domains",
174 "repo domain remove": "remove a custom pages domain", 174 "repo domain remove": "remove a custom pages domain",
175 "repo domain verify": "check the DNS challenge and activate a claim", 175 "repo domain verify": "check the DNS challenge and activate a claim",
176 "repo download": "write a tar.gz of a ref to stdout", 176 "repo download": "write a tar.gz of a ref to stdout",
177 "repo fork": "fork a repository under your account", 177 "repo fork": "fork a repository under your account",
178 "repo grep": "search file contents", 178 "repo grep": "search file contents",
179 "repo import": "server-side mirror of a foreign repository", 179 "repo import": "server-side mirror of a foreign repository",
180 "repo import-issues": "import issue and PR history from GitHub or Forgejo", 180 "repo import-issues": "import issue and PR history from GitHub or Forgejo",
181 "repo list": "list repositories you own or can access", 181 "repo list": "list repositories you own or can access",
182 "repo log": "commit log with signature states", 182 "repo log": "commit log with signature states",
183 "repo mirror add": "mirror to or from a remote", 183 "repo mirror add": "mirror to or from a remote",
184 "repo mirror list": "list mirrors with sync status", 184 "repo mirror list": "list mirrors with sync status",
185 "repo mirror remove": "remove a mirror", 185 "repo mirror remove": "remove a mirror",
186 "repo mirror sync": "schedule an immediate sync", 186 "repo mirror sync": "schedule an immediate sync",
187 "repo mute": "mute a repository, including work you are part of", 187 "repo mute": "mute a repository, including work you are part of",
188 "repo pin": "pin a repository to your dashboard", 188 "repo pin": "pin a repository to your dashboard",
189 "repo readme": "print a repository's README", 189 "repo readme": "print a repository's README",
190 "repo refs": "list branches and tags", 190 "repo refs": "list branches and tags",
191 "repo rename": "rename a repository", 191 "repo rename": "rename a repository",
192 "repo runner add": "attach a runner's public key to a repository", 192 "repo runner add": "attach a runner's public key to a repository",
193 "repo runner list": "list the runners attached to a repository", 193 "repo runner list": "list the runners attached to a repository",
194 "repo runner remove": "detach a runner from a repository", 194 "repo runner remove": "detach a runner from a repository",
195 "repo search": "find repositories by name, description, or topic", 195 "repo search": "find repositories by name, description, or topic",
196 "repo secret list": "list build secret names", 196 "repo secret list": "list build secret names",
197 "repo secret remove": "remove a build secret", 197 "repo secret remove": "remove a build secret",
198 "repo secret set": "set a build secret", 198 "repo secret set": "set a build secret",
199 "repo secret-scan": "scan the whole history for credentials", 199 "repo secret-scan": "scan the whole history for credentials",
200 "repo security disable": "stop code scanning", 200 "repo security disable": "stop code scanning",
201 "repo security enable": "scan the default branch on the instance's code scanning schedule", 201 "repo security dismiss": "dismiss code scanning findings, which then stay out of the issue",
202 "repo security run": "queue a code scan of the default branch now", 202 "repo security enable": "scan the default branch on the instance's code scanning schedule",
203 "repo security status": "show code scanning state and the last scan's findings", 203 "repo security reopen": "take back dismissals of code scanning findings",
204 "repo settings default-branch": "set the default branch", 204 "repo security run": "queue a code scan of the default branch now",
205 "repo settings description": "set the repository description", 205 "repo security status": "show code scanning state and the last scan's findings",
206 "repo settings git-daemon": "expose over git://", 206 "repo settings default-branch": "set the default branch",
207 "repo settings protect": "protect a branch", 207 "repo settings description": "set the repository description",
208 "repo settings protect-tag": "protect tags matching a glob (created once, never moved or deleted)", 208 "repo settings git-daemon": "expose over git://",
209 "repo settings require-approvals": "require N fresh approvals to merge", 209 "repo settings protect": "protect a branch",
210 "repo settings require-checks": "gate merges on green statuses", 210 "repo settings protect-tag": "protect tags matching a glob (created once, never moved or deleted)",
211 "repo settings require-codeowners": "require an owner's approval for every file CODEOWNERS covers", 211 "repo settings require-approvals": "require N fresh approvals to merge",
212 "repo settings require-contexts": "name the statuses the checks gate waits for, and turn the gate on", 212 "repo settings require-checks": "gate merges on green statuses",
213 "repo settings require-mr": "protected branches take changes through merge requests only", 213 "repo settings require-codeowners": "require an owner's approval for every file CODEOWNERS covers",
214 "repo settings require-resolved": "require all review threads resolved to merge", 214 "repo settings require-contexts": "name the statuses the checks gate waits for, and turn the gate on",
215 "repo settings require-signed": "require verified commit signatures", 215 "repo settings require-mr": "protected branches take changes through merge requests only",
216 "repo settings secret-scan": "scan pushes for credentials: off, warn the pusher, or block the push", 216 "repo settings require-resolved": "require all review threads resolved to merge",
217 "repo settings secret-scan-allow": "paths whose secret scan findings are ignored", 217 "repo settings require-signed": "require verified commit signatures",
218 "repo settings show": "show settings", 218 "repo settings secret-scan": "scan pushes for credentials: off, warn the pusher, or block the push",
219 "repo settings unprotect": "unprotect a branch", 219 "repo settings secret-scan-allow": "paths whose secret scan findings are ignored",
220 "repo settings unprotect-tag": "drop a protected-tag glob", 220 "repo settings security-exclude-rule": "code scanning rules, or families of them, whose findings are set aside",
221 "repo settings visibility": "set repository visibility", 221 "repo settings security-ignore": "paths whose code scanning findings are set aside",
222 "repo settings website": "set the repository website", 222 "repo settings show": "show settings",
223 "repo show": "show repository details", 223 "repo settings unprotect": "unprotect a branch",
224 "repo symbols": "find where a name is defined, from the default branch's symbol index", 224 "repo settings unprotect-tag": "drop a protected-tag glob",
225 "repo topics add": "add topics", 225 "repo settings visibility": "set repository visibility",
226 "repo topics remove": "remove topics", 226 "repo settings website": "set the repository website",
227 "repo topics": "list topics", 227 "repo show": "show repository details",
228 "repo transfer": "move a repository to another owner", 228 "repo symbols": "find where a name is defined, from the default branch's symbol index",
229 "repo tree": "list a directory", 229 "repo topics add": "add topics",
230 "repo unarchive": "unarchive a repository", 230 "repo topics remove": "remove topics",
231 "repo unbookmark": "remove a bookmark", 231 "repo topics": "list topics",
232 "repo unpin": "unpin a repository", 232 "repo transfer": "move a repository to another owner",
233 "repo unwatch": "back to the default: only work you are part of", 233 "repo tree": "list a directory",
234 "repo watch": "hear about all activity on a repository", 234 "repo unarchive": "unarchive a repository",
235 "runner done": "finish a build", 235 "repo unbookmark": "remove a bookmark",
236 "runner log": "append a build's log from stdin", 236 "repo unpin": "unpin a repository",
237 "runner next": "claim the oldest pending build this key may run (runner protocol)", 237 "repo unwatch": "back to the default: only work you are part of",
238 "runner results": "upload a code scan's SARIF from stdin", 238 "repo watch": "hear about all activity on a repository",
239 "search": "find repositories, issues and merge requests across the instance", 239 "runner done": "finish a build",
240 "snippet create": "create a snippet from one file on stdin", 240 "runner log": "append a build's log from stdin",
241 "snippet delete": "delete a snippet and its files", 241 "runner next": "claim the oldest pending build this key may run (runner protocol)",
242 "snippet edit": "change a snippet's description or visibility", 242 "runner results": "upload a code scan's SARIF from stdin",
243 "snippet file get": "write a snippet file to stdout", 243 "search": "find repositories, issues and merge requests across the instance",
244 "snippet file remove": "remove a file from a snippet", 244 "snippet create": "create a snippet from one file on stdin",
245 "snippet file set": "add a file to a snippet, or replace one, from stdin", 245 "snippet delete": "delete a snippet and its files",
246 "snippet list": "list your snippets, or an owner's public ones", 246 "snippet edit": "change a snippet's description or visibility",
247 "snippet show": "show a snippet's metadata and files", 247 "snippet file get": "write a snippet file to stdout",
248 "status list": "statuses on a commit", 248 "snippet file remove": "remove a file from a snippet",
249 "status set": "report a commit status (CI)", 249 "snippet file set": "add a file to a snippet, or replace one, from stdin",
250 "token create": "mint an API token (shown once)", 250 "snippet list": "list your snippets, or an owner's public ones",
251 "token list": "list API tokens", 251 "snippet show": "show a snippet's metadata and files",
252 "token revoke": "revoke an API token by name", 252 "status list": "statuses on a commit",
253 "web diff set": "draw web diffs unified or side by side", 253 "status set": "report a commit status (CI)",
254 "web diff show": "the diff layout the web UI uses for you", 254 "token create": "mint an API token (shown once)",
255 "web login": "mint a one-time browser login URL", 255 "token list": "list API tokens",
256 "web sessions list": "list your browser sessions", 256 "token revoke": "revoke an API token by name",
257 "web sessions revoke": "end a browser session, or all of them", 257 "web diff set": "draw web diffs unified or side by side",
258 "web theme set": "follow the browser's scheme, or force light or dark", 258 "web diff show": "the diff layout the web UI uses for you",
259 "web theme show": "the colour scheme the web UI uses for you", 259 "web login": "mint a one-time browser login URL",
260 "webhook add": "add a webhook", 260 "web sessions list": "list your browser sessions",
261 "webhook deliveries": "recent deliveries", 261 "web sessions revoke": "end a browser session, or all of them",
262 "webhook list": "list webhooks", 262 "web theme set": "follow the browser's scheme, or force light or dark",
263 "webhook redeliver": "queue a delivery again", 263 "web theme show": "the colour scheme the web UI uses for you",
264 "webhook remove": "remove a webhook", 264 "webhook add": "add a webhook",
265 "whoami": "show the authenticated account", 265 "webhook deliveries": "recent deliveries",
266 "wiki list": "list a repository's wiki pages", 266 "webhook list": "list webhooks",
267 "wiki show": "print a wiki page", 267 "webhook redeliver": "queue a delivery again",
268 "webhook remove": "remove a webhook",
269 "whoami": "show the authenticated account",
270 "wiki list": "list a repository's wiki pages",
271 "wiki show": "print a wiki page",
268} 272}
internal/codescan/codescan.go +87 −10
@@ -21,9 +21,11 @@ import (
21// IssueTitle is fixed so the issue is recognizable across scans. 21// IssueTitle is fixed so the issue is recognizable across scans.
22const IssueTitle = "Code scanning findings" 22const IssueTitle = "Code scanning findings"
23 23
24// MaxUpload bounds one upload. MaxFindings bounds what is kept from it. 24// MaxUpload bounds one upload, maxResults what is read from it, and
25// MaxFindings what is reported of that once a Scope has set some aside.
25const ( 26const (
26 MaxUpload = 8 << 20 27 MaxUpload = 8 << 20
28 maxResults = 10 * MaxFindings
27 MaxFindings = 500 29 MaxFindings = 500
28) 30)
29 31
@@ -61,7 +63,7 @@ type sarif struct {
61 63
62// Parse returns the findings in a SARIF document, deduplicated by 64// Parse returns the findings in a SARIF document, deduplicated by
63// fingerprint and sorted by path and line. It reports whether results past 65// fingerprint and sorted by path and line. It reports whether results past
64// MaxFindings were dropped. 66// maxResults were dropped.
65func Parse(raw []byte) ([]store.SecurityFinding, bool, error) { 67func Parse(raw []byte) ([]store.SecurityFinding, bool, error) {
66 var doc sarif 68 var doc sarif
67 if err := json.Unmarshal(raw, &doc); err != nil { 69 if err := json.Unmarshal(raw, &doc); err != nil {
@@ -110,7 +112,7 @@ func Parse(raw []byte) ([]store.SecurityFinding, bool, error) {
110 continue 112 continue
111 } 113 }
112 seen[h.place()] = true 114 seen[h.place()] = true
113 if len(out) >= MaxFindings { 115 if len(out) >= maxResults {
114 truncated = true 116 truncated = true
115 break 117 break
116 } 118 }
@@ -248,6 +250,18 @@ func Same(a, b []store.SecurityFinding) bool {
248 return true 250 return true
249} 251}
250 252
253// ByPlace sorts a set by path and line, the way a scan lists it.
254func ByPlace(fs []store.SecurityFinding) []store.SecurityFinding {
255 out := append([]store.SecurityFinding(nil), fs...)
256 sort.SliceStable(out, func(i, j int) bool {
257 if out[i].Path != out[j].Path {
258 return out[i].Path < out[j].Path
259 }
260 return out[i].Line < out[j].Line
261 })
262 return out
263}
264
251// ByFingerprint sorts a set the way the store returns it, for Same. 265// ByFingerprint sorts a set the way the store returns it, for Same.
252func ByFingerprint(fs []store.SecurityFinding) []store.SecurityFinding { 266func ByFingerprint(fs []store.SecurityFinding) []store.SecurityFinding {
253 out := append([]store.SecurityFinding(nil), fs...) 267 out := append([]store.SecurityFinding(nil), fs...)
@@ -255,14 +269,25 @@ func ByFingerprint(fs []store.SecurityFinding) []store.SecurityFinding {
255 return out 269 return out
256} 270}
257 271
258// Body renders the issue: one list per tool, a line per finding. 272// Report is what the issue says: the findings still open, and how many
259func Body(branch string, findings []store.SecurityFinding, truncated bool) string { 273// others were dismissed or set aside.
274type Report struct {
275 Repo, Branch string
276 Open []store.SecurityFinding
277 Dismissed int
278 Aside Aside
279 Truncated bool
280}
281
282// Body renders the issue: one list per tool, a line per finding with its
283// fingerprint, then what is not listed.
284func Body(r Report) string {
260 var b strings.Builder 285 var b strings.Builder
261 fmt.Fprintf(&b, "The scheduled code scan of `%s` reported these findings. "+ 286 fmt.Fprintf(&b, "The scheduled code scan of `%s` reported these findings. "+
262 "This issue is kept up to date as that changes, and closed once a scan reports none.\n", branch) 287 "This issue is kept up to date as that changes, and closed once none are open.\n", r.Branch)
263 byTool := map[string][]store.SecurityFinding{} 288 byTool := map[string][]store.SecurityFinding{}
264 var tools []string 289 var tools []string
265 for _, f := range findings { 290 for _, f := range r.Open {
266 if _, ok := byTool[f.Tool]; !ok { 291 if _, ok := byTool[f.Tool]; !ok {
267 tools = append(tools, f.Tool) 292 tools = append(tools, f.Tool)
268 } 293 }
@@ -276,11 +301,63 @@ func Body(branch string, findings []store.SecurityFinding, truncated bool) strin
276 } 301 }
277 fmt.Fprintf(&b, "\n### %s\n\n", name) 302 fmt.Fprintf(&b, "\n### %s\n\n", name)
278 for _, f := range byTool[tool] { 303 for _, f := range byTool[tool] {
279 fmt.Fprintf(&b, "- `%s:%d` %s (%s): %s\n", f.Path, f.Line, f.Rule, f.Level, f.Message) 304 fmt.Fprintf(&b, "- `%s:%d` %s (%s): %s `%s`\n", f.Path, f.Line, f.Rule, f.Level, f.Message, Short(f.Fingerprint))
280 } 305 }
281 } 306 }
282 if truncated { 307 if r.Truncated {
283 fmt.Fprintf(&b, "\nOnly the first %d results are listed.\n", MaxFindings) 308 fmt.Fprintf(&b, "\nOnly the first %d findings are listed.\n", MaxFindings)
309 }
310 var notListed []string
311 if r.Dismissed > 0 {
312 notListed = append(notListed, fmt.Sprintf("%d dismissed", r.Dismissed))
313 }
314 if r.Aside.Ignored > 0 {
315 notListed = append(notListed, fmt.Sprintf("%d under paths or rules the repository ignores", r.Aside.Ignored))
316 }
317 if len(r.Aside.Unused) > 0 {
318 notListed = append(notListed, fmt.Sprintf("%d from rules for libraries it does not use (%s)", total(r.Aside.Unused), counts(r.Aside.Unused)))
319 }
320 if len(notListed) > 0 {
321 fmt.Fprintf(&b, "\nNot listed: %s. `repo security status %s` shows them.\n", strings.Join(notListed, "; "), r.Repo)
322 }
323 if len(r.Open) > 0 {
324 fmt.Fprintf(&b, "\nTo dismiss one: `repo security dismiss %s <fingerprint> --reason false-positive|wont-fix|test`.\n", r.Repo)
284 } 325 }
285 return b.String() 326 return b.String()
286} 327}
328
329// Short is the fingerprint as the issue prints it; Dismiss takes it back.
330func Short(fingerprint string) string {
331 return fingerprint[:min(len(fingerprint), ShortLen)]
332}
333
334// ShortLen is how much of a fingerprint is printed and the least that
335// names one.
336const ShortLen = 10
337
338func total(m map[string]int) int {
339 n := 0
340 for _, v := range m {
341 n += v
342 }
343 return n
344}
345
346// counts lists a map's keys with their counts, most first.
347func counts(m map[string]int) string {
348 keys := make([]string, 0, len(m))
349 for k := range m {
350 keys = append(keys, k)
351 }
352 sort.Slice(keys, func(i, j int) bool {
353 if m[keys[i]] != m[keys[j]] {
354 return m[keys[i]] > m[keys[j]]
355 }
356 return keys[i] < keys[j]
357 })
358 parts := make([]string, len(keys))
359 for i, k := range keys {
360 parts[i] = fmt.Sprintf("%s %d", k, m[k])
361 }
362 return strings.Join(parts, ", ")
363}
internal/codescan/codescan_test.go +2 −2
@@ -45,7 +45,7 @@ func TestParse(t *testing.T) {
45func TestParseTruncates(t *testing.T) { 45func TestParseTruncates(t *testing.T) {
46 var b strings.Builder 46 var b strings.Builder
47 b.WriteString(`{"runs":[{"tool":{"driver":{"name":"t"}},"results":[`) 47 b.WriteString(`{"runs":[{"tool":{"driver":{"name":"t"}},"results":[`)
48 for i := 0; i < MaxFindings+5; i++ { 48 for i := 0; i < maxResults+5; i++ {
49 if i > 0 { 49 if i > 0 {
50 b.WriteString(",") 50 b.WriteString(",")
51 } 51 }
@@ -53,7 +53,7 @@ func TestParseTruncates(t *testing.T) {
53 } 53 }
54 b.WriteString(`]}]}`) 54 b.WriteString(`]}]}`)
55 got, truncated, err := Parse([]byte(b.String())) 55 got, truncated, err := Parse([]byte(b.String()))
56 if err != nil || !truncated || len(got) != MaxFindings { 56 if err != nil || !truncated || len(got) != maxResults {
57 t.Fatalf("len %d truncated %v err %v", len(got), truncated, err) 57 t.Fatalf("len %d truncated %v err %v", len(got), truncated, err)
58 } 58 }
59} 59}
internal/codescan/libraries.go added +115
@@ -0,0 +1,115 @@
1package codescan
2
3import "strings"
4
5// library is a family of rules that is about one library: opengrep-rules
6// keeps those under <language>/<library>/, which a rule id carries as
7// language.library. Its findings apply only to a repository whose manifests
8// name one of packages, written ecosystem:name; a go: name ending in / is a
9// module path prefix, and root: is an entry at the top of the tree.
10// Directories that are about a language, a runtime or the standard library
11// (lang, browser, audit, correctness, template, node-crypto, …) are not
12// here, so their rules apply everywhere. Neither are languages whose
13// manifests gitbay does not read.
14type library struct {
15 rules string
16 packages []string
17}
18
19var libraries = []library{
20 {"go.aws-lambda", []string{"go:github.com/aws/aws-lambda-go"}},
21 {"go.gorilla", []string{"go:github.com/gorilla/"}},
22 {"go.gorm", []string{"go:gorm.io/gorm", "go:github.com/jinzhu/gorm"}},
23 {"go.grpc", []string{"go:google.golang.org/grpc"}},
24 {"go.jwt-go", []string{"go:github.com/dgrijalva/jwt-go", "go:github.com/golang-jwt/jwt/", "go:github.com/form3tech-oss/jwt-go"}},
25 {"go.otto", []string{"go:github.com/robertkrimen/otto"}},
26
27 {"javascript.ajv", []string{"npm:ajv"}},
28 {"javascript.angular", []string{"npm:angular", "npm:@angular/core"}},
29 {"javascript.apollo", []string{"npm:apollo-server", "npm:apollo-server-express", "npm:@apollo/server"}},
30 {"javascript.argon2", []string{"npm:argon2"}},
31 {"javascript.bluebird", []string{"npm:bluebird"}},
32 {"javascript.chrome-remote-interface", []string{"npm:chrome-remote-interface"}},
33 {"javascript.express", []string{"npm:express"}},
34 {"javascript.fbjs", []string{"npm:fbjs"}},
35 {"javascript.grpc", []string{"npm:grpc", "npm:@grpc/grpc-js"}},
36 {"javascript.jose", []string{"npm:jose", "npm:node-jose"}},
37 {"javascript.jquery", []string{"npm:jquery"}},
38 {"javascript.jsonwebtoken", []string{"npm:jsonwebtoken"}},
39 {"javascript.jwt-simple", []string{"npm:jwt-simple"}},
40 {"javascript.monaco-editor", []string{"npm:monaco-editor"}},
41 {"javascript.node-expat", []string{"npm:node-expat"}},
42 {"javascript.passport-jwt", []string{"npm:passport-jwt"}},
43 {"javascript.phantom", []string{"npm:phantom"}},
44 {"javascript.playwright", []string{"npm:playwright", "npm:@playwright/test"}},
45 {"javascript.puppeteer", []string{"npm:puppeteer", "npm:puppeteer-core"}},
46 {"javascript.react", []string{"npm:react"}},
47 {"javascript.sandbox", []string{"npm:sandbox"}},
48 {"javascript.sax", []string{"npm:sax"}},
49 {"javascript.sequelize", []string{"npm:sequelize"}},
50 {"javascript.serialize-javascript", []string{"npm:serialize-javascript"}},
51 {"javascript.shelljs", []string{"npm:shelljs"}},
52 {"javascript.thenify", []string{"npm:thenify"}},
53 {"javascript.vm2", []string{"npm:vm2"}},
54 {"javascript.vue", []string{"npm:vue"}},
55 {"javascript.wkhtmltoimage", []string{"npm:wkhtmltoimage"}},
56 {"javascript.wkhtmltopdf", []string{"npm:wkhtmltopdf"}},
57 {"javascript.xml2json", []string{"npm:xml2json"}},
58
59 {"typescript.angular", []string{"npm:@angular/core"}},
60 {"typescript.aws-cdk", []string{"npm:aws-cdk-lib", "npm:@aws-cdk/core"}},
61 {"typescript.nestjs", []string{"npm:@nestjs/core"}},
62 {"typescript.react", []string{"npm:react"}},
63 {"typescript.react.portability.i18next", []string{"npm:i18next", "npm:react-i18next"}},
64
65 {"python.airflow", []string{"pypi:apache-airflow"}},
66 {"python.attr", []string{"pypi:attrs"}},
67 {"python.bokeh", []string{"pypi:bokeh"}},
68 {"python.boto3", []string{"pypi:boto3"}},
69 {"python.click", []string{"pypi:click"}},
70 {"python.cryptography", []string{"pypi:cryptography"}},
71 {"python.distributed", []string{"pypi:distributed", "pypi:dask"}},
72 {"python.django", []string{"pypi:django"}},
73 {"python.docker", []string{"pypi:docker"}},
74 {"python.fastapi", []string{"pypi:fastapi"}},
75 {"python.flask", []string{"pypi:flask"}},
76 {"python.jinja2", []string{"pypi:jinja2", "pypi:flask"}},
77 {"python.jwt", []string{"pypi:pyjwt"}},
78 {"python.pycryptodome", []string{"pypi:pycryptodome", "pypi:pycryptodomex"}},
79 {"python.pymongo", []string{"pypi:pymongo"}},
80 {"python.pyramid", []string{"pypi:pyramid"}},
81 {"python.requests", []string{"pypi:requests"}},
82 {"python.sh", []string{"pypi:sh"}},
83 {"python.sqlalchemy", []string{"pypi:sqlalchemy", "pypi:flask-sqlalchemy"}},
84 {"python.twilio", []string{"pypi:twilio"}},
85
86 // Lints for writing Semgrep rules, which match any YAML with a list
87 // of rules that have ids.
88 {"yaml.semgrep", []string{"root:.semgrep", "root:.semgrep.yml", "root:.semgrep.yaml", "pypi:semgrep"}},
89}
90
91// libraryFor returns the most specific library a rule id belongs to. The
92// id may carry a prefix from where the rules were mounted
93// (opt.opengrep-rules.python.django…).
94func libraryFor(rule string) (library, bool) {
95 var best library
96 found := false
97 for _, l := range libraries {
98 if ruleIn(rule, l.rules) && len(l.rules) > len(best.rules) {
99 best, found = l, true
100 }
101 }
102 return best, found
103}
104
105// ruleIn reports whether a rule id is family or within it: family is a
106// whole id, a run of its dot-separated segments, or a suffix.
107func ruleIn(rule, family string) bool {
108 return rule == family || strings.HasPrefix(rule, family+".") ||
109 strings.HasSuffix(rule, "."+family) || strings.Contains(rule, "."+family+".")
110}
111
112// name is how the issue and status refer to a library.
113func (l library) name() string {
114 return l.rules[strings.LastIndex(l.rules, ".")+1:]
115}
internal/codescan/scope.go added +110
@@ -0,0 +1,110 @@
1package codescan
2
3import (
4 "bufio"
5 "bytes"
6 "path"
7 "strings"
8
9 "gitbay.org/gitbay/internal/ci"
10 "gitbay.org/gitbay/internal/store"
11)
12
13// Scope is what a repository sets aside from what its scan reports.
14type Scope struct {
15 // Ignore are path globs, as CI paths take them (security-ignore).
16 Ignore []string
17 // Vendored are the .gitattributes patterns Vendored returns.
18 Vendored []string
19 // ExcludeRules are rule ids or families (security-exclude-rule).
20 ExcludeRules []string
21 // Uses reports whether the scanned commit depends on any of packages,
22 // written as library lists them. Nil uses everything.
23 Uses func(packages []string) bool
24}
25
26// Aside counts the findings a scope set aside: Ignored by path or rule,
27// Unused by the library a rule is about.
28type Aside struct {
29 Ignored int `json:"ignored,omitempty"`
30 Unused map[string]int `json:"unused,omitempty"`
31}
32
33// Apply splits findings into those reported, at most MaxFindings of them,
34// and counts of those set aside. It reports whether reported ones were
35// dropped for the cap.
36func (s Scope) Apply(findings []store.SecurityFinding) ([]store.SecurityFinding, Aside, bool) {
37 var kept []store.SecurityFinding
38 var aside Aside
39 truncated := false
40 for _, f := range findings {
41 if s.ignored(f) {
42 aside.Ignored++
43 continue
44 }
45 if l, ok := libraryFor(f.Rule); ok && s.Uses != nil && !s.Uses(l.packages) {
46 if aside.Unused == nil {
47 aside.Unused = map[string]int{}
48 }
49 aside.Unused[l.name()]++
50 continue
51 }
52 if len(kept) == MaxFindings {
53 truncated = true
54 continue
55 }
56 kept = append(kept, f)
57 }
58 return kept, aside, truncated
59}
60
61func (s Scope) ignored(f store.SecurityFinding) bool {
62 for _, g := range s.Ignore {
63 if ci.Match(g, f.Path) {
64 return true
65 }
66 }
67 for _, p := range s.Vendored {
68 if attrMatch(p, f.Path) {
69 return true
70 }
71 }
72 for _, r := range s.ExcludeRules {
73 if ruleIn(f.Rule, r) {
74 return true
75 }
76 }
77 return false
78}
79
80// Vendored returns the patterns a .gitattributes file marks
81// linguist-vendored or linguist-generated. Unset and false values are left
82// out.
83func Vendored(gitattributes []byte) []string {
84 var out []string
85 sc := bufio.NewScanner(bytes.NewReader(gitattributes))
86 for sc.Scan() {
87 fields := strings.Fields(sc.Text())
88 if len(fields) < 2 || strings.HasPrefix(fields[0], "#") {
89 continue
90 }
91 for _, a := range fields[1:] {
92 if a == "linguist-vendored" || a == "linguist-generated" || a == "linguist-vendored=true" || a == "linguist-generated=true" {
93 out = append(out, fields[0])
94 break
95 }
96 }
97 }
98 return out
99}
100
101// attrMatch matches a path the way git matches a .gitattributes pattern: one
102// without a slash names files by base name at any depth, one with a slash is
103// relative to the top.
104func attrMatch(pattern, file string) bool {
105 if !strings.Contains(pattern, "/") {
106 ok, err := path.Match(pattern, path.Base(file))
107 return err == nil && ok
108 }
109 return ci.Match(strings.TrimPrefix(pattern, "/"), file)
110}
internal/control/repo.go +4
@@ -844,6 +844,8 @@ func runSettingsShow(c *Ctx, args []string) int {
844 "require signed commits", strconv.FormatBool(repo.Settings.RequireSignedCommits), 844 "require signed commits", strconv.FormatBool(repo.Settings.RequireSignedCommits),
845 "secret scan", leaks.Mode(repo.Settings.SecretScan), 845 "secret scan", leaks.Mode(repo.Settings.SecretScan),
846 "secret scan allow", strings.Join(repo.Settings.SecretScanAllow, ", "), 846 "secret scan allow", strings.Join(repo.Settings.SecretScanAllow, ", "),
847 "security ignore", strings.Join(repo.Settings.SecurityIgnore, ", "),
848 "security exclude rules", strings.Join(repo.Settings.SecurityExcludeRules, ", "),
847 "git daemon", strconv.FormatBool(repo.Settings.GitDaemon), 849 "git daemon", strconv.FormatBool(repo.Settings.GitDaemon),
848 "archived", strconv.FormatBool(repo.Settings.Archived), 850 "archived", strconv.FormatBool(repo.Settings.Archived),
849 ) 851 )
@@ -882,6 +884,8 @@ func runSettingsShow(c *Ctx, args []string) int {
882 {"Signed commits", []cell{onOff(set.RequireSignedCommits)}}, 884 {"Signed commits", []cell{onOff(set.RequireSignedCommits)}},
883 {"Secret scan", []cell{cText(leaks.Mode(set.SecretScan))}}, 885 {"Secret scan", []cell{cText(leaks.Mode(set.SecretScan))}},
884 {"Scan allows", []cell{list(set.SecretScanAllow)}}, 886 {"Scan allows", []cell{list(set.SecretScanAllow)}},
887 {"Scan ignores", []cell{list(set.SecurityIgnore)}},
888 {"Rules off", []cell{list(set.SecurityExcludeRules)}},
885 {"Git daemon", []cell{onOff(set.GitDaemon)}}, 889 {"Git daemon", []cell{onOff(set.GitDaemon)}},
886 {"Archived", []cell{onOff(set.Archived)}}, 890 {"Archived", []cell{onOff(set.Archived)}},
887 }, actions: []action{ 891 }, actions: []action{
internal/control/security.go +360 −23
@@ -1,15 +1,22 @@
1package control 1package control
2 2
3import ( 3import (
4 "encoding/json"
4 "errors" 5 "errors"
5 "fmt" 6 "fmt"
6 "io" 7 "io"
8 "path"
9 "slices"
10 "sort"
7 "strconv" 11 "strconv"
12 "strings"
8 "time" 13 "time"
9 14
10 "gitbay.org/gitbay/internal/botissue" 15 "gitbay.org/gitbay/internal/botissue"
11 "gitbay.org/gitbay/internal/ci" 16 "gitbay.org/gitbay/internal/ci"
12 "gitbay.org/gitbay/internal/codescan" 17 "gitbay.org/gitbay/internal/codescan"
18 "gitbay.org/gitbay/internal/deps"
19 "gitbay.org/gitbay/internal/gitutil"
13 "gitbay.org/gitbay/internal/policy" 20 "gitbay.org/gitbay/internal/policy"
14 "gitbay.org/gitbay/internal/protocol" 21 "gitbay.org/gitbay/internal/protocol"
15 "gitbay.org/gitbay/internal/store" 22 "gitbay.org/gitbay/internal/store"
@@ -40,6 +47,29 @@ func init() {
40 Usage: "repo security status <owner/name>", 47 Usage: "repo security status <owner/name>",
41 Examples: []string{"repo security status krz/gitbay"}, 48 Examples: []string{"repo security status krz/gitbay"},
42 ReadOnly: true, Run: runSecurityStatus}) 49 ReadOnly: true, Run: runSecurityStatus})
50 register(Command{Path: []string{"repo", "security", "dismiss"},
51 Summary: "dismiss code scanning findings, which then stay out of the issue",
52 Usage: "repo security dismiss <owner/name> <fingerprint>... --reason false-positive|wont-fix|test",
53 Flags: []Flag{
54 {"--reason", "false-positive|wont-fix|test", "why: not a problem, accepted, or only in tests", ""},
55 },
56 Examples: []string{"repo security dismiss krz/gitbay 3f9a0c1d2e --reason false-positive"},
57 Run: runSecurityDismiss})
58 register(Command{Path: []string{"repo", "security", "reopen"},
59 Summary: "take back dismissals of code scanning findings",
60 Usage: "repo security reopen <owner/name> <fingerprint>...",
61 Examples: []string{"repo security reopen krz/gitbay 3f9a0c1d2e"},
62 Run: runSecurityReopen})
63 register(Command{Path: []string{"repo", "settings", "security-ignore"},
64 Summary: "paths whose code scanning findings are set aside",
65 Usage: "repo settings security-ignore <owner/name> [<glob>...] (none clears the list)",
66 Examples: []string{"repo settings security-ignore krz/gitbay 'testdata/**' 'web/templates/*.html'"},
67 Run: runSecurityIgnore})
68 register(Command{Path: []string{"repo", "settings", "security-exclude-rule"},
69 Summary: "code scanning rules, or families of them, whose findings are set aside",
70 Usage: "repo settings security-exclude-rule <owner/name> [<rule>...] (none clears the list)",
71 Examples: []string{"repo settings security-exclude-rule krz/gitbay swift.lang.crypto.insecure-random"},
72 Run: runSecurityExcludeRule})
43 register(Command{Path: []string{"runner", "results"}, 73 register(Command{Path: []string{"runner", "results"},
44 Summary: "upload a code scan's SARIF from stdin", 74 Summary: "upload a code scan's SARIF from stdin",
45 Usage: "runner results <build-id>", 75 Usage: "runner results <build-id>",
@@ -47,25 +77,34 @@ func init() {
47 ReadsStdin: true, Run: runRunnerResults}) 77 ReadsStdin: true, Run: runRunnerResults})
48} 78}
49 79
50// SecurityOut is what `repo security status` emits. 80// SecurityOut is what `repo security status` emits: the last scan's open
81// findings, the ones dismissed, and how many it set aside.
51type SecurityOut struct { 82type SecurityOut struct {
52 Enabled bool `json:"enabled"` 83 Enabled bool `json:"enabled"`
53 NextRun string `json:"next_run,omitempty"` 84 NextRun string `json:"next_run,omitempty"`
54 LastResults string `json:"last_results,omitempty"` 85 LastResults string `json:"last_results,omitempty"`
55 IssueNumber int64 `json:"issue_number,omitempty"` 86 IssueNumber int64 `json:"issue_number,omitempty"`
56 Findings []SecurityFinding `json:"findings"` 87 Findings []SecurityFinding `json:"findings"`
88 Dismissed []SecurityFinding `json:"dismissed"`
89 Aside codescan.Aside `json:"aside"`
57} 90}
58 91
59// SecurityFinding is one result of the last scan. 92// SecurityFinding is one result of the last scan.
60type SecurityFinding struct { 93type SecurityFinding struct {
61 Tool string `json:"tool"` 94 Fingerprint string `json:"fingerprint"`
62 Rule string `json:"rule"` 95 Tool string `json:"tool"`
63 Level string `json:"level"` 96 Rule string `json:"rule"`
64 Path string `json:"path"` 97 Level string `json:"level"`
65 Line int `json:"line"` 98 Path string `json:"path"`
66 Message string `json:"message"` 99 Line int `json:"line"`
100 Message string `json:"message"`
101 Reason string `json:"reason,omitempty"`
102 DismissedBy string `json:"dismissed_by,omitempty"`
67} 103}
68 104
105// Reasons a finding is dismissed for.
106var dismissReasons = []string{"false-positive", "wont-fix", "test"}
107
69func runSecurityEnable(c *Ctx, args []string) int { 108func runSecurityEnable(c *Ctx, args []string) int {
70 if len(args) != 1 { 109 if len(args) != 1 {
71 return c.usage() 110 return c.usage()
@@ -153,10 +192,22 @@ func runSecurityStatus(c *Ctx, args []string) int {
153 if err != nil { 192 if err != nil {
154 return c.fail(protocol.ExitFailure, "%v", err) 193 return c.fail(protocol.ExitFailure, "%v", err)
155 } 194 }
195 dismissed, err := c.Store.SecurityDismissals(repo.ID)
196 if err != nil {
197 return c.fail(protocol.ExitFailure, "%v", err)
198 }
156 out := SecurityOut{Enabled: true, NextRun: scan.NextRun, LastResults: scan.LastResults, 199 out := SecurityOut{Enabled: true, NextRun: scan.NextRun, LastResults: scan.LastResults,
157 IssueNumber: scan.IssueNumber, Findings: []SecurityFinding{}} 200 IssueNumber: scan.IssueNumber, Findings: []SecurityFinding{}, Dismissed: []SecurityFinding{}}
158 for _, f := range reported { 201 json.Unmarshal([]byte(scan.Aside), &out.Aside)
159 out.Findings = append(out.Findings, SecurityFinding{f.Tool, f.Rule, f.Level, f.Path, f.Line, f.Message}) 202 for _, f := range codescan.ByPlace(reported) {
203 o := SecurityFinding{Fingerprint: f.Fingerprint, Tool: f.Tool, Rule: f.Rule, Level: f.Level,
204 Path: f.Path, Line: f.Line, Message: f.Message}
205 if d, ok := dismissed[f.Fingerprint]; ok {
206 o.Reason, o.DismissedBy = d.Reason, d.By
207 out.Dismissed = append(out.Dismissed, o)
208 continue
209 }
210 out.Findings = append(out.Findings, o)
160 } 211 }
161 return c.emitView(out, func(w io.Writer) { 212 return c.emitView(out, func(w io.Writer) {
162 tracked := "" 213 tracked := ""
@@ -169,12 +220,21 @@ func runSecurityStatus(c *Ctx, args []string) int {
169 "next scan", scan.NextRun, 220 "next scan", scan.NextRun,
170 "last results", c.when(scan.LastResults), 221 "last results", c.when(scan.LastResults),
171 "tracked in", tracked, 222 "tracked in", tracked,
223 "set aside", asideText(out.Aside),
172 ) 224 )
173 if len(out.Findings) > 0 { 225 if len(out.Findings) > 0 {
174 v.section("findings") 226 v.section("findings")
175 tb := c.table(w, "TOOL", "RULE", "LEVEL", "PATH", "LINE") 227 tb := c.table(w, "FINGERPRINT", "TOOL", "RULE", "LEVEL", "PATH", "LINE")
176 for _, f := range out.Findings { 228 for _, f := range out.Findings {
177 tb.row(cText(f.Tool), cText(f.Rule), cText(f.Level), cRef(f.Path), cNum(int64(f.Line))) 229 tb.row(cRef(codescan.Short(f.Fingerprint)), cText(f.Tool), cText(f.Rule), cText(f.Level), cRef(f.Path), cNum(int64(f.Line)))
230 }
231 tb.flush()
232 }
233 if len(out.Dismissed) > 0 {
234 v.section("dismissed")
235 tb := c.table(w, "FINGERPRINT", "REASON", "BY", "RULE", "PATH", "LINE")
236 for _, f := range out.Dismissed {
237 tb.row(cRef(codescan.Short(f.Fingerprint)), cText(f.Reason), cText(f.DismissedBy), cText(f.Rule), cRef(f.Path), cNum(int64(f.Line)))
178 } 238 }
179 tb.flush() 239 tb.flush()
180 } 240 }
@@ -188,12 +248,25 @@ func runSecurityStatus(c *Ctx, args []string) int {
188 n := strconv.FormatInt(scan.IssueNumber, 10) 248 n := strconv.FormatInt(scan.IssueNumber, 10)
189 s.fields = append(s.fields, field{"Tracked in", []cell{cLink("#"+n, c.siteURL(repo.Path(), "issues", n))}}) 249 s.fields = append(s.fields, field{"Tracked in", []cell{cLink("#"+n, c.siteURL(repo.Path(), "issues", n))}})
190 } 250 }
251 if a := asideText(out.Aside); a != "" {
252 s.fields = append(s.fields, field{"Set aside", []cell{cMeta(a)}})
253 }
191 found := section{title: "Findings", n: len(out.Findings)} 254 found := section{title: "Findings", n: len(out.Findings)}
192 for _, f := range out.Findings { 255 for _, f := range out.Findings {
193 found.rows = append(found.rows, rowOf(cRef(f.Path+":"+strconv.Itoa(f.Line)), cText(f.Rule), cMeta(f.Level))) 256 found.rows = append(found.rows, rowOf(cRef(f.Path+":"+strconv.Itoa(f.Line)), cText(f.Rule), cFlexRef(codescan.Short(f.Fingerprint))))
194 } 257 }
195 s.sections = []section{found} 258 s.sections = []section{found}
259 if len(out.Dismissed) > 0 {
260 gone := section{title: "Dismissed", n: len(out.Dismissed)}
261 for _, f := range out.Dismissed {
262 gone.rows = append(gone.rows, rowOf(cRef(f.Path+":"+strconv.Itoa(f.Line)), cState(f.Reason), cFlexRef(codescan.Short(f.Fingerprint))))
263 }
264 s.sections = append(s.sections, gone)
265 }
196 s.actions = []action{{"Security", []string{"repo", "security", "run", repo.Path()}}} 266 s.actions = []action{{"Security", []string{"repo", "security", "run", repo.Path()}}}
267 if len(out.Findings) > 0 {
268 s.actions = append(s.actions, action{"Security", []string{"repo", "security", "dismiss", repo.Path(), "<fp>", "--reason", "<why>"}})
269 }
197 return s 270 return s
198 }) 271 })
199} 272}
@@ -244,7 +317,7 @@ func runRunnerResults(c *Ctx, args []string) int {
244 if len(raw) > codescan.MaxUpload { 317 if len(raw) > codescan.MaxUpload {
245 return c.fail(protocol.ExitFailure, "results are over %d bytes", codescan.MaxUpload) 318 return c.fail(protocol.ExitFailure, "results are over %d bytes", codescan.MaxUpload)
246 } 319 }
247 findings, truncated, err := codescan.Parse(raw) 320 all, overflow, err := codescan.Parse(raw)
248 if err != nil { 321 if err != nil {
249 return c.fail(protocol.ExitFailure, "%v", err) 322 return c.fail(protocol.ExitFailure, "%v", err)
250 } 323 }
@@ -252,19 +325,19 @@ func runRunnerResults(c *Ctx, args []string) int {
252 if err != nil { 325 if err != nil {
253 return c.fail(protocol.ExitFailure, "%v", err) 326 return c.fail(protocol.ExitFailure, "%v", err)
254 } 327 }
328 findings, aside, truncated := scanScope(RepoDir(c.Cfg.Server.Root, repo.OwnerName, repo.Name), repo, b.SHA).Apply(all)
255 previous, err := c.Store.ReportedFindings(repo.ID) 329 previous, err := c.Store.ReportedFindings(repo.ID)
256 if err != nil { 330 if err != nil {
257 return c.fail(protocol.ExitFailure, "%v", err) 331 return c.fail(protocol.ExitFailure, "%v", err)
258 } 332 }
259 err = botissue.Maintain(c.Store, c.Cfg, repo, botissue.Tracked{ 333 asideJSON := ""
260 Title: codescan.IssueTitle, 334 if aside.Ignored > 0 || len(aside.Unused) > 0 {
261 Number: scan.IssueNumber, 335 j, _ := json.Marshal(aside)
262 Record: func(n int64) error { return c.Store.SetSecurityIssue(repo.ID, n) }, 336 asideJSON = string(j)
263 Empty: len(findings) == 0, 337 }
264 Changed: !codescan.Same(previous, codescan.ByFingerprint(findings)), 338 changed := !codescan.Same(previous, codescan.ByFingerprint(findings)) || asideJSON != scan.Aside
265 Save: func() error { return c.Store.ReplaceFindings(repo.ID, findings) }, 339 err = maintainFindings(c, repo, scan, findings, aside, truncated || overflow, changed,
266 Body: func() string { return codescan.Body(repo.DefaultBranch, findings, truncated) }, 340 func() error { return c.Store.ReplaceFindings(repo.ID, findings, asideJSON) })
267 })
268 if err != nil { 341 if err != nil {
269 return c.fail(protocol.ExitFailure, "%v", err) 342 return c.fail(protocol.ExitFailure, "%v", err)
270 } 343 }
@@ -277,3 +350,267 @@ func runRunnerResults(c *Ctx, args []string) int {
277 fmt.Fprintf(w, "%d findings recorded for build %d\n", len(findings), b.Number) 350 fmt.Fprintf(w, "%d findings recorded for build %d\n", len(findings), b.Number)
278 }) 351 })
279} 352}
353
354// maintainFindings brings the findings issue in line with a set of
355// findings: the open ones listed, the dismissed and set-aside ones
356// counted, closed when none are open.
357func maintainFindings(c *Ctx, repo store.Repo, scan store.SecurityScan, findings []store.SecurityFinding,
358 aside codescan.Aside, truncated, changed bool, save func() error) error {
359 dismissed, err := c.Store.SecurityDismissals(repo.ID)
360 if err != nil {
361 return err
362 }
363 var open []store.SecurityFinding
364 for _, f := range findings {
365 if _, ok := dismissed[f.Fingerprint]; !ok {
366 open = append(open, f)
367 }
368 }
369 return botissue.Maintain(c.Store, c.Cfg, repo, botissue.Tracked{
370 Title: codescan.IssueTitle,
371 Number: scan.IssueNumber,
372 Record: func(n int64) error { return c.Store.SetSecurityIssue(repo.ID, n) },
373 Empty: len(open) == 0,
374 Changed: changed,
375 Save: save,
376 Body: func() string {
377 return codescan.Body(codescan.Report{Repo: repo.Path(), Branch: repo.DefaultBranch, Open: codescan.ByPlace(open),
378 Dismissed: len(findings) - len(open), Aside: aside, Truncated: truncated})
379 },
380 })
381}
382
383// manifestLimit bounds a manifest or .gitattributes read for a scan's scope.
384const manifestLimit = 1 << 20
385
386// maxManifestDirs bounds how many directories' manifests a scope reads.
387const maxManifestDirs = 50
388
389// scanScope is what a repository sets aside from a scan of commit sha: its
390// settings, what .gitattributes marks vendored or generated, and findings
391// from rules about libraries no manifest at that commit names. Manifests are
392// read in every directory that has one, outside dependency directories, so
393// a monorepo's web/package.json counts.
394func scanScope(dir string, repo store.Repo, sha string) codescan.Scope {
395 sc := codescan.Scope{Ignore: repo.Settings.SecurityIgnore, ExcludeRules: repo.Settings.SecurityExcludeRules}
396 if raw, err := gitutil.ReadBlob(dir, sha, ".gitattributes", manifestLimit); err == nil {
397 sc.Vendored = codescan.Vendored(raw)
398 }
399 root := map[string]bool{}
400 if entries, err := gitutil.ListTree(dir, sha, ""); err == nil {
401 for _, e := range entries {
402 root[e.Name] = true
403 }
404 }
405 dirs := []string{""}
406 if entries, err := gitutil.ListTreeRecursive(dir, sha, ""); err == nil {
407 seen := map[string]bool{"": true}
408 for _, e := range entries {
409 d := path.Dir(e.Name)
410 if seen[d] || len(dirs) == maxManifestDirs || !slices.Contains(deps.ManifestFiles, path.Base(e.Name)) || dependencyDir(d) {
411 continue
412 }
413 seen[d] = true
414 dirs = append(dirs, d)
415 }
416 }
417 uses := map[string]bool{}
418 for _, d := range dirs {
419 read := func(p string) ([]byte, error) {
420 if d != "" {
421 p = d + "/" + p
422 }
423 return gitutil.ReadBlob(dir, sha, p, manifestLimit)
424 }
425 for _, dep := range deps.Names(read) {
426 uses[dep.Ecosystem+":"+dep.Name] = true
427 }
428 }
429 sc.Uses = func(packages []string) bool {
430 for _, p := range packages {
431 if name, ok := strings.CutPrefix(p, "root:"); ok {
432 if root[name] {
433 return true
434 }
435 continue
436 }
437 if strings.HasPrefix(p, "go:") && strings.HasSuffix(p, "/") {
438 for u := range uses {
439 if strings.HasPrefix(u, p) {
440 return true
441 }
442 }
443 continue
444 }
445 if uses[p] {
446 return true
447 }
448 }
449 return false
450 }
451 return sc
452}
453
454// dependencyDir reports whether a directory holds other projects' code
455// rather than the repository's own.
456func dependencyDir(d string) bool {
457 for _, part := range strings.Split(d, "/") {
458 if part == "node_modules" || part == "vendor" || part == "third_party" {
459 return true
460 }
461 }
462 return false
463}
464
465// asideText is what the last scan set aside, in a line.
466func asideText(a codescan.Aside) string {
467 var parts []string
468 if a.Ignored > 0 {
469 parts = append(parts, fmt.Sprintf("%d ignored", a.Ignored))
470 }
471 libs := make([]string, 0, len(a.Unused))
472 for l := range a.Unused {
473 libs = append(libs, l)
474 }
475 sort.Strings(libs)
476 for _, l := range libs {
477 parts = append(parts, fmt.Sprintf("%d %s (not used)", a.Unused[l], l))
478 }
479 return strings.Join(parts, ", ")
480}
481
482func runSecurityDismiss(c *Ctx, args []string) int {
483 f, err := c.parseArgs(args, flagSpec{Values: []string{"--reason"}, MaxPos: -1, Usage: c.Cmd.Usage})
484 if err != nil {
485 return c.fail(protocol.ExitUsage, "%v", err)
486 }
487 if len(f.Pos) < 2 {
488 return c.usage()
489 }
490 reason := f.Value("--reason")
491 if !slices.Contains(dismissReasons, reason) {
492 return c.fail(protocol.ExitUsage, "--reason is one of %s", strings.Join(dismissReasons, ", "))
493 }
494 return changeDismissals(c, f.Pos[0], f.Pos[1:], func(repoID int64, fps []string) error {
495 return c.Store.DismissFindings(repoID, c.User.ID, fps, reason)
496 }, "dismissed")
497}
498
499func runSecurityReopen(c *Ctx, args []string) int {
500 if len(args) < 2 {
501 return c.usage()
502 }
503 return changeDismissals(c, args[0], args[1:], c.Store.ReopenFindings, "reopened")
504}
505
506// changeDismissals resolves fingerprints, given in full or as the issue
507// prints them, against the last scan's findings, applies change, and
508// rewrites the issue to match. Writers may do it: it decides what the
509// issue lists, as editing the issue would.
510func changeDismissals(c *Ctx, name string, given []string, change func(int64, []string) error, verb string) int {
511 repo, code := resolveRepo(c, name, policy.CanWrite)
512 if code >= 0 {
513 return code
514 }
515 scan, err := c.Store.SecurityScanFor(repo.ID)
516 if errors.Is(err, store.ErrNotFound) {
517 return c.fail(protocol.ExitFailure, "code scanning is off on %s (repo security enable %s)", repo.Path(), repo.Path())
518 } else if err != nil {
519 return c.fail(protocol.ExitFailure, "%v", err)
520 }
521 findings, err := c.Store.ReportedFindings(repo.ID)
522 if err != nil {
523 return c.fail(protocol.ExitFailure, "%v", err)
524 }
525 var fps []string
526 for _, g := range given {
527 fp, msg := resolveFingerprint(findings, g)
528 if msg != "" {
529 return c.fail(protocol.ExitNotFound, "%s", msg)
530 }
531 if !slices.Contains(fps, fp) {
532 fps = append(fps, fp)
533 }
534 }
535 if err := change(repo.ID, fps); err != nil {
536 return c.fail(protocol.ExitFailure, "%v", err)
537 }
538 var aside codescan.Aside
539 json.Unmarshal([]byte(scan.Aside), &aside)
540 err = maintainFindings(c, repo, scan, findings, aside, len(findings) >= codescan.MaxFindings, true,
541 func() error { return nil })
542 if err != nil {
543 return c.fail(protocol.ExitFailure, "%v", err)
544 }
545 return c.emit(map[string]any{verb: fps}, func(w io.Writer) {
546 fmt.Fprintf(w, "%s %d on %s\n", verb, len(fps), repo.Path())
547 })
548}
549
550// resolveFingerprint finds the one finding a fingerprint or a prefix of at
551// least codescan.ShortLen names, or says why not.
552func resolveFingerprint(findings []store.SecurityFinding, given string) (string, string) {
553 if len(given) < codescan.ShortLen {
554 return "", fmt.Sprintf("fingerprint %q is too short; give at least %d characters", given, codescan.ShortLen)
555 }
556 match := ""
557 for _, f := range findings {
558 if strings.HasPrefix(f.Fingerprint, given) {
559 if match != "" {
560 return "", fmt.Sprintf("fingerprint %q names more than one finding", given)
561 }
562 match = f.Fingerprint
563 }
564 }
565 if match == "" {
566 return "", fmt.Sprintf("no finding %q in the last scan", given)
567 }
568 return match, ""
569}
570
571// maxSecurityList bounds the ignore and exclude-rule settings.
572const maxSecurityList = 50
573
574func runSecurityIgnore(c *Ctx, args []string) int {
575 return setSecurityList(c, args, "a glob", func(s *store.RepoSettings, xs []string) { s.SecurityIgnore = xs }, "ignores")
576}
577
578func runSecurityExcludeRule(c *Ctx, args []string) int {
579 return setSecurityList(c, args, "a rule", func(s *store.RepoSettings, xs []string) { s.SecurityExcludeRules = xs }, "excludes")
580}
581
582// setSecurityList replaces one of the code scanning lists, which apply from
583// the next scan.
584func setSecurityList(c *Ctx, args []string, what string, set func(*store.RepoSettings, []string), verb string) int {
585 if len(args) < 1 {
586 return c.usage()
587 }
588 var xs []string
589 for _, x := range args[1:] {
590 if x == "" || len(x) > 200 || strings.ContainsAny(x, " \t\r\n") {
591 return c.fail(protocol.ExitUsage, "%s is 1 to 200 characters with no spaces: %q", what, x)
592 }
593 if !slices.Contains(xs, x) {
594 xs = append(xs, x)
595 }
596 }
597 if len(xs) > maxSecurityList {
598 return c.fail(protocol.ExitUsage, "at most %d entries", maxSecurityList)
599 }
600 repo, code := resolveRepo(c, args[0], policy.CanAdmin)
601 if code >= 0 {
602 return code
603 }
604 s, err := c.Store.UpdateRepoSettings(repo.ID, func(s *store.RepoSettings) { set(s, xs) })
605 if err != nil {
606 return c.fail(protocol.ExitFailure, "%v", err)
607 }
608 return c.emit(s, func(w io.Writer) {
609 if len(xs) == 0 {
610 fmt.Fprintf(w, "code scanning %s cleared on %s\n", verb, repo.Path())
611 return
612 }
613 fmt.Fprintf(w, "code scanning %s on %s: %s; from the next scan (repo security run %s)\n",
614 verb, repo.Path(), strings.Join(xs, ", "), repo.Path())
615 })
616}
internal/control/security_test.go +182
@@ -2,6 +2,9 @@ package control
2 2
3import ( 3import (
4 "encoding/json" 4 "encoding/json"
5 "os"
6 "os/exec"
7 "path/filepath"
5 "strings" 8 "strings"
6 "testing" 9 "testing"
7 10
@@ -9,6 +12,7 @@ import (
9 "gitbay.org/gitbay/internal/codescan" 12 "gitbay.org/gitbay/internal/codescan"
10 "gitbay.org/gitbay/internal/config" 13 "gitbay.org/gitbay/internal/config"
11 "gitbay.org/gitbay/internal/protocol" 14 "gitbay.org/gitbay/internal/protocol"
15 "gitbay.org/gitbay/internal/store"
12) 16)
13 17
14const testSARIF = `{"version":"2.1.0","runs":[{"tool":{"driver":{"name":"opengrep"}},"results":[ 18const testSARIF = `{"version":"2.1.0","runs":[{"tool":{"driver":{"name":"opengrep"}},"results":[
@@ -113,3 +117,181 @@ func TestSecurityScanRoundTrip(t *testing.T) {
113 t.Errorf("results from another job: %d %s", code, out) 117 t.Errorf("results from another job: %d %s", code, out)
114 } 118 }
115} 119}
120
121// scopeRepo commits files to a bare repository and returns its directory
122// and the commit.
123func scopeRepo(t *testing.T, files map[string]string) (string, string) {
124 t.Helper()
125 root := t.TempDir()
126 src, dir := filepath.Join(root, "src"), filepath.Join(root, "bare.git")
127 git := func(cwd string, args ...string) string {
128 t.Helper()
129 cmd := exec.Command("git", args...)
130 cmd.Dir, cmd.Env = cwd, gitTestEnv()
131 out, err := cmd.CombinedOutput()
132 if err != nil {
133 t.Fatalf("git %v: %v\n%s", args, err, out)
134 }
135 return strings.TrimSpace(string(out))
136 }
137 for name, body := range files {
138 os.MkdirAll(filepath.Join(src, filepath.Dir(name)), 0o755)
139 os.WriteFile(filepath.Join(src, name), []byte(body), 0o644)
140 }
141 git(root, "init", "-q", "-b", "main", "src")
142 git(src, "add", ".")
143 git(src, "commit", "-q", "-m", "first")
144 git(root, "clone", "-q", "--bare", src, dir)
145 return dir, git(dir, "rev-parse", "HEAD")
146}
147
148func TestScanScope(t *testing.T) {
149 dir, sha := scopeRepo(t, map[string]string{
150 "requirements.txt": "flask>=3\n",
151 "web/package.json": `{"dependencies":{"react":"^19"}}`,
152 "node_modules/x/package.json": `{"dependencies":{"jquery":"3"}}`,
153 ".gitattributes": "Sources/Vendored/** linguist-vendored\n*.min.js linguist-generated\ndocs/** -linguist-vendored\n",
154 "app.py": "",
155 })
156 repo := store.Repo{Settings: store.RepoSettings{SecurityIgnore: []string{"testdata/**"},
157 SecurityExcludeRules: []string{"swift.lang.crypto.insecure-random"}}}
158 f := func(rule, path string) store.SecurityFinding {
159 return store.SecurityFinding{Rule: "opt.opengrep-rules." + rule, Path: path}
160 }
161 in := []store.SecurityFinding{
162 f("python.flask.security.xss.x", "app.py"), // flask is used
163 f("python.django.security.django-no-csrf-token", "t.html"), // django is not
164 f("javascript.jquery.security.audit.prohibit-jquery-html", "a.ts"), // only node_modules has it
165 f("typescript.react.security.audit.x", "web/a.tsx"), // web/package.json
166 f("typescript.react.portability.i18next.jsx-not-internationalized", "web/a.tsx"),
167 f("yaml.semgrep.missing-language-field", "rules.yaml"),
168 f("python.lang.security.audit.x", "testdata/a.py"),
169 f("swift.lang.crypto.insecure-random", "A.swift"),
170 f("c.lang.security.insecure-use-memset", "Sources/Vendored/a.c"),
171 f("javascript.browser.security.insecure-innerhtml", "lib/x.min.js"),
172 f("javascript.browser.security.insecure-innerhtml", "docs/x.js"),
173 }
174 kept, aside, truncated := scanScope(dir, repo, sha).Apply(in)
175 var got []string
176 for _, k := range kept {
177 got = append(got, k.Path)
178 }
179 if strings.Join(got, " ") != "app.py web/a.tsx docs/x.js" || truncated {
180 t.Errorf("kept %v truncated %v", got, truncated)
181 }
182 if aside.Ignored != 4 || aside.Unused["django"] != 1 || aside.Unused["jquery"] != 1 ||
183 aside.Unused["i18next"] != 1 || aside.Unused["semgrep"] != 1 || len(aside.Unused) != 4 {
184 t.Errorf("aside = %+v", aside)
185 }
186
187 // A repository that authors rules keeps the rule lints.
188 dir, sha = scopeRepo(t, map[string]string{".semgrep/rules.yaml": "rules: []\n"})
189 if kept, _, _ := scanScope(dir, store.Repo{}, sha).Apply(in[5:6]); len(kept) != 1 {
190 t.Errorf("rule lints set aside in a repository with .semgrep")
191 }
192}
193
194const twoFindings = `{"version":"2.1.0","runs":[{"tool":{"driver":{"name":"opengrep"}},"results":[
195 {"ruleId":"go.lang.security.audit.sqli","message":{"text":"SQL built from input"},
196 "locations":[{"physicalLocation":{"artifactLocation":{"uri":"store/q.go"},"region":{"startLine":12}}}]},
197 {"ruleId":"go.lang.security.audit.sqli","message":{"text":"SQL built from input"},
198 "locations":[{"physicalLocation":{"artifactLocation":{"uri":"store/q.go"},"region":{"startLine":40}}}]}]}]}`
199
200// Dismissing takes a finding out of the issue and closes it once none are
201// open; reopening brings it back.
202func TestSecurityDismiss(t *testing.T) {
203 st, repo, u, root := browseFixture(t)
204 p := repo.Path()
205 sec := config.Security{Image: "localhost/gitbay-security:1", Schedule: "0 4 * * *", Steps: config.DefaultSecuritySteps}
206 run := func(f func(*Ctx, []string) int, stdin string, args ...string) (int, string) {
207 t.Helper()
208 c, out := runnerCtx(st, u.ID, root)
209 c.Cfg.Security = sec
210 c.Stdin = strings.NewReader(stdin)
211 c.JSON = true
212 return f(c, args), out.String()
213 }
214 run(runSecurityEnable, "", p)
215 run(runSecurityRun, "", p)
216 _, claim := run(runRunnerNext, "")
217 var claimed struct {
218 Data struct {
219 ID int64 `json:"id"`
220 } `json:"data"`
221 }
222 json.Unmarshal([]byte(claim), &claimed)
223 if code, out := run(runRunnerResults, twoFindings, itoa(claimed.Data.ID)); code != protocol.ExitOK {
224 t.Fatalf("results: %d %s", code, out)
225 }
226 scan, _ := st.SecurityScanFor(repo.ID)
227 findings, _ := st.ReportedFindings(repo.ID)
228 if len(findings) != 2 {
229 t.Fatalf("two hits of one rule in one file stored as %d", len(findings))
230 }
231 issue, _ := st.IssueByNumber(repo.ID, scan.IssueNumber)
232 first, second := codescan.ByPlace(findings)[0].Fingerprint, codescan.ByPlace(findings)[1].Fingerprint
233 if !strings.Contains(issue.Body, codescan.Short(first)) || !strings.Contains(issue.Body, "repo security dismiss "+p) {
234 t.Fatalf("issue:\n%s", issue.Body)
235 }
236
237 dismiss := func(args ...string) (int, string) {
238 t.Helper()
239 var out strings.Builder
240 c, buf := runnerCtx(st, u.ID, root)
241 code := runSecurityDismiss(c, args)
242 out.WriteString(buf.String())
243 return code, out.String()
244 }
245 if code, out := dismiss(p, codescan.Short(first), "--reason", "nope"); code != protocol.ExitUsage {
246 t.Errorf("bad reason: %d %s", code, out)
247 }
248 if code, out := dismiss(p, first[:4], "--reason", "test"); code != protocol.ExitNotFound || !strings.Contains(out, "too short") {
249 t.Errorf("short prefix: %d %s", code, out)
250 }
251 if code, out := dismiss(p, codescan.Short(first), "--reason", "false-positive"); code != protocol.ExitOK {
252 t.Fatalf("dismiss: %d %s", code, out)
253 }
254 issue, _ = st.IssueByNumber(repo.ID, scan.IssueNumber)
255 if strings.Contains(issue.Body, codescan.Short(first)) || !strings.Contains(issue.Body, codescan.Short(second)) ||
256 !strings.Contains(issue.Body, "1 dismissed") || issue.State != "open" {
257 t.Errorf("after one dismissal (%s):\n%s", issue.State, issue.Body)
258 }
259
260 var status struct {
261 Data SecurityOut `json:"data"`
262 }
263 json.Unmarshal([]byte(dispatchIn(t, st, u, root, "", "repo", "security", "status", p, "--json")), &status)
264 if len(status.Data.Findings) != 1 || len(status.Data.Dismissed) != 1 ||
265 status.Data.Dismissed[0].Reason != "false-positive" || status.Data.Dismissed[0].DismissedBy != u.Username {
266 t.Errorf("status = %+v", status.Data)
267 }
268 screen := atTerminalIn(t, st, u, root, "repo", "security", "status", p)
269 if !strings.Contains(screen, "Dismissed (1)") {
270 t.Errorf("screen:\n%s", screen)
271 }
272 checkLegend(t, screen)
273
274 // The same set from the next scan leaves the issue alone.
275 run(runSecurityRun, "", p)
276 _, claim = run(runRunnerNext, "")
277 json.Unmarshal([]byte(claim), &claimed)
278 run(runRunnerResults, twoFindings, itoa(claimed.Data.ID))
279 if again, _ := st.IssueByNumber(repo.ID, scan.IssueNumber); again.Body != issue.Body || again.State != "open" {
280 t.Errorf("an unchanged scan rewrote the issue:\n%s", again.Body)
281 }
282
283 if code, out := dismiss(p, second, "--reason", "wont-fix"); code != protocol.ExitOK {
284 t.Fatalf("dismiss second: %d %s", code, out)
285 }
286 if closed, _ := st.IssueByNumber(repo.ID, scan.IssueNumber); closed.State != "closed" {
287 t.Errorf("issue with nothing open is %s", closed.State)
288 }
289 c, out := runnerCtx(st, u.ID, root)
290 if code := runSecurityReopen(c, []string{p, codescan.Short(second)}); code != protocol.ExitOK {
291 t.Fatalf("reopen: %d %s", code, out)
292 }
293 reopened, _ := st.SecurityScanFor(repo.ID)
294 if reopened.IssueNumber == scan.IssueNumber || reopened.IssueNumber == 0 {
295 t.Errorf("reopening a finding should open a fresh issue, got #%d", reopened.IssueNumber)
296 }
297}
internal/deps/manifest_test.go +15
@@ -174,3 +174,18 @@ func TestScanEmptyTree(t *testing.T) {
174 t.Errorf("Scan of empty tree = %v", got) 174 t.Errorf("Scan of empty tree = %v", got)
175 } 175 }
176} 176}
177
178// Names keeps what Scan leaves out for having no single version.
179func TestNames(t *testing.T) {
180 got := found(t, Names(tree(map[string]string{
181 "go.mod": "module x\n\nrequire github.com/gorilla/mux v1.8.1\n",
182 "requirements.txt": "Django>=4,<6\nrequests\n",
183 "package.json": `{"dependencies":{"jquery":"^3"},"devDependencies":{"vite":"git+https://x/y"},"peerDependencies":{"react":"*"}}`,
184 "Cargo.toml": "[dependencies]\nlocal = { path = \"../local\" }\n",
185 })))
186 for _, want := range []string{"go:github.com/gorilla/mux", "pypi:django", "pypi:requests", "npm:jquery", "npm:vite", "npm:react", "cargo:local"} {
187 if v, ok := got[want]; !ok || v != "" {
188 t.Errorf("%s: %q %v in %v", want, v, ok, got)
189 }
190 }
191}
internal/deps/names.go added +54
@@ -0,0 +1,54 @@
1package deps
2
3import (
4 "encoding/json"
5
6 "github.com/BurntSushi/toml"
7)
8
9// Names returns the direct dependencies the manifests in read name, whatever
10// version each asks for, with Current left empty. Code scanning uses it to
11// tell which libraries a repository uses; a range or a git source still
12// means the library is there.
13func Names(read ReadFile) []Dep {
14 var out []Dep
15 for _, d := range parseGoMod(read) {
16 out = append(out, Dep{Ecosystem: EcoGo, Name: d.Name})
17 }
18 for _, d := range pythonDeps(read, func(string) string { return "any" }) {
19 out = append(out, Dep{Ecosystem: EcoPyPI, Name: d.Name})
20 }
21 if raw, err := read("package.json"); err == nil {
22 var pkg struct {
23 Dependencies map[string]any `json:"dependencies"`
24 DevDependencies map[string]any `json:"devDependencies"`
25 PeerDependencies map[string]any `json:"peerDependencies"`
26 }
27 if json.Unmarshal(raw, &pkg) == nil {
28 for _, set := range []map[string]any{pkg.Dependencies, pkg.DevDependencies, pkg.PeerDependencies} {
29 for name := range set {
30 out = append(out, Dep{Ecosystem: EcoNPM, Name: name})
31 }
32 }
33 }
34 }
35 if raw, err := read("Cargo.toml"); err == nil {
36 var manifest struct {
37 Dependencies map[string]any `toml:"dependencies"`
38 DevDependencies map[string]any `toml:"dev-dependencies"`
39 BuildDependencies map[string]any `toml:"build-dependencies"`
40 }
41 if toml.Unmarshal(raw, &manifest) == nil {
42 for _, set := range []map[string]any{manifest.Dependencies, manifest.DevDependencies, manifest.BuildDependencies} {
43 for name := range set {
44 out = append(out, Dep{Ecosystem: EcoCargo, Name: name})
45 }
46 }
47 }
48 }
49 return out
50}
51
52// ManifestFiles are the files Names reads, for finding the directories of a
53// repository worth reading it in.
54var ManifestFiles = []string{"go.mod", "requirements.txt", "pyproject.toml", "package.json", "Cargo.toml"}
internal/httpd/depspage_test.go +28
@@ -4,6 +4,7 @@ import (
4 "strings" 4 "strings"
5 "testing" 5 "testing"
6 6
7 "gitbay.org/gitbay/internal/codescan"
7 "gitbay.org/gitbay/internal/control" 8 "gitbay.org/gitbay/internal/control"
8 "gitbay.org/gitbay/internal/web" 9 "gitbay.org/gitbay/internal/web"
9) 10)
@@ -46,3 +47,30 @@ func TestSettingsPageHidesDepsStatusWhenOff(t *testing.T) {
46 t.Error("the report shows with checks off") 47 t.Error("the report shows with checks off")
47 } 48 }
48} 49}
50
51// The code scanning section offers each open finding for dismissal, each
52// dismissed one for reopening, and says what the scan set aside.
53func TestSettingsPageRendersSecurityTriage(t *testing.T) {
54 var sb strings.Builder
55 err := web.Render(&sb, "settings.html", settingsPage{
56 repoPage: testRepoPage(),
57 Security: control.SecurityOut{
58 Enabled: true, LastResults: "2026-11-01T04:02:00Z",
59 Findings: []control.SecurityFinding{{Fingerprint: "aaaa1111bbbb", Rule: "go.lang.x", Path: "a.go", Line: 3}},
60 Dismissed: []control.SecurityFinding{{Fingerprint: "cccc2222dddd", Rule: "go.lang.y", Path: "b.go", Line: 9, Reason: "test", DismissedBy: "alice"}},
61 Aside: codescan.Aside{Unused: map[string]int{"django": 26}},
62 },
63 })
64 if err != nil {
65 t.Fatal(err)
66 }
67 page := sb.String()
68 for _, want := range []string{
69 `value="security-dismiss"`, `value="aaaa1111bbbb"`, `value="security-reopen"`, `value="cccc2222dddd"`,
70 "by alice", "26 from django rules", `name="security-ignore"`, `name="security-exclude-rule"`,
71 } {
72 if !strings.Contains(page, want) {
73 t.Errorf("settings page does not show %q", want)
74 }
75 }
76}
internal/httpd/settings.go +8
@@ -169,6 +169,14 @@ func (s *Server) settingsSubmit(w http.ResponseWriter, r *http.Request, u store.
169 verb = "enable" 169 verb = "enable"
170 } 170 }
171 argv = []string{"repo", "security", verb, repo} 171 argv = []string{"repo", "security", verb, repo}
172 case "security-dismiss":
173 argv = []string{"repo", "security", "dismiss", repo, v("fingerprint"), "--reason", v("reason")}
174 case "security-reopen":
175 argv = []string{"repo", "security", "reopen", repo, v("fingerprint")}
176 case "security-ignore":
177 argv = append([]string{"repo", "settings", "security-ignore", repo}, strings.Fields(v("security-ignore"))...)
178 case "security-exclude-rule":
179 argv = append([]string{"repo", "settings", "security-exclude-rule", repo}, strings.Fields(v("security-exclude-rule"))...)
172 case "secret-scan": 180 case "secret-scan":
173 argv = []string{"repo", "settings", "secret-scan", repo, v("secret-scan")} 181 argv = []string{"repo", "settings", "secret-scan", repo, v("secret-scan")}
174 case "secret-scan-allow": 182 case "secret-scan-allow":
internal/store/migrations/0083_security_dismissals.down.sql added +2
@@ -0,0 +1,2 @@
1ALTER TABLE security_scans DROP COLUMN aside;
2DROP TABLE security_dismissals;
internal/store/migrations/0083_security_dismissals.up.sql added +16
@@ -0,0 +1,16 @@
1-- A finding a repository dismissed, by fingerprint, so it stays dismissed
2-- across scans and line moves. Kept while scanning is off: turning it back
3-- on does not bring back what was decided. by_id is NULL once that
4-- account is gone.
5CREATE TABLE security_dismissals (
6 repo_id INTEGER NOT NULL REFERENCES repos(id) ON DELETE CASCADE,
7 fingerprint TEXT NOT NULL,
8 reason TEXT NOT NULL,
9 by_id INTEGER REFERENCES users(id) ON DELETE SET NULL,
10 created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
11 PRIMARY KEY (repo_id, fingerprint)
12);
13
14-- What the last scan set aside rather than reported (codescan.Aside, as
15-- JSON), so a change in it rewrites the issue.
16ALTER TABLE security_scans ADD COLUMN aside TEXT NOT NULL DEFAULT '';
internal/store/repos.go +4
@@ -42,6 +42,10 @@ type RepoSettings struct {
42 SecretScan string `json:"secret_scan,omitempty"` 42 SecretScan string `json:"secret_scan,omitempty"`
43 // SecretScanAllow are CI path globs whose findings are ignored. 43 // SecretScanAllow are CI path globs whose findings are ignored.
44 SecretScanAllow []string `json:"secret_scan_allow,omitempty"` 44 SecretScanAllow []string `json:"secret_scan_allow,omitempty"`
45 // SecurityIgnore are CI path globs, and SecurityExcludeRules rule ids
46 // or families, whose code scanning findings are set aside.
47 SecurityIgnore []string `json:"security_ignore,omitempty"`
48 SecurityExcludeRules []string `json:"security_exclude_rules,omitempty"`
45} 49}
46 50
47// Path returns the canonical owner/name form. 51// Path returns the canonical owner/name form.
internal/store/security.go +73 −4
@@ -12,6 +12,7 @@ type SecurityScan struct {
12 NextRun string 12 NextRun string
13 LastResults string // when results were last uploaded, '' for never 13 LastResults string // when results were last uploaded, '' for never
14 IssueNumber int64 14 IssueNumber int64
15 Aside string // what the last scan set aside, as JSON
15} 16}
16 17
17// SecurityFinding is one result from a scan, as last reported. 18// SecurityFinding is one result from a scan, as last reported.
@@ -49,8 +50,8 @@ func (s *Store) DisableSecurityScan(repoID int64) error {
49func (s *Store) SecurityScanFor(repoID int64) (SecurityScan, error) { 50func (s *Store) SecurityScanFor(repoID int64) (SecurityScan, error) {
50 var sc SecurityScan 51 var sc SecurityScan
51 err := s.DB.QueryRow( 52 err := s.DB.QueryRow(
52 "SELECT repo_id, next_run, last_results, issue_number FROM security_scans WHERE repo_id = ?", repoID). 53 "SELECT repo_id, next_run, last_results, issue_number, aside FROM security_scans WHERE repo_id = ?", repoID).
53 Scan(&sc.RepoID, &sc.NextRun, &sc.LastResults, &sc.IssueNumber) 54 Scan(&sc.RepoID, &sc.NextRun, &sc.LastResults, &sc.IssueNumber, &sc.Aside)
54 if errors.Is(err, sql.ErrNoRows) { 55 if errors.Is(err, sql.ErrNoRows) {
55 return sc, ErrNotFound 56 return sc, ErrNotFound
56 } 57 }
@@ -111,13 +112,17 @@ func (s *Store) ReportedFindings(repoID int64) ([]SecurityFinding, error) {
111 return out, rows.Err() 112 return out, rows.Err()
112} 113}
113 114
114// ReplaceFindings swaps in an upload's findings wholesale. 115// ReplaceFindings swaps in an upload's findings wholesale, with what it set
115func (s *Store) ReplaceFindings(repoID int64, findings []SecurityFinding) error { 116// aside.
117func (s *Store) ReplaceFindings(repoID int64, findings []SecurityFinding, aside string) error {
116 tx, err := s.DB.Begin() 118 tx, err := s.DB.Begin()
117 if err != nil { 119 if err != nil {
118 return err 120 return err
119 } 121 }
120 defer tx.Rollback() 122 defer tx.Rollback()
123 if _, err := tx.Exec("UPDATE security_scans SET aside = ? WHERE repo_id = ?", aside, repoID); err != nil {
124 return err
125 }
121 if _, err := tx.Exec("DELETE FROM security_findings WHERE repo_id = ?", repoID); err != nil { 126 if _, err := tx.Exec("DELETE FROM security_findings WHERE repo_id = ?", repoID); err != nil {
122 return err 127 return err
123 } 128 }
@@ -138,3 +143,67 @@ func (s *Store) StampSecurityResults(repoID int64) error {
138 WHERE repo_id = ?`, repoID) 143 WHERE repo_id = ?`, repoID)
139 return err 144 return err
140} 145}
146
147// SecurityDismissal is a finding a repository dismissed.
148type SecurityDismissal struct {
149 Fingerprint string
150 Reason string
151 By string // username, '' once the account is gone
152 CreatedAt string
153}
154
155// DismissFindings dismisses findings by fingerprint, replacing the reason
156// of any already dismissed.
157func (s *Store) DismissFindings(repoID, byID int64, fingerprints []string, reason string) error {
158 tx, err := s.DB.Begin()
159 if err != nil {
160 return err
161 }
162 defer tx.Rollback()
163 for _, fp := range fingerprints {
164 if _, err := tx.Exec(`
165 INSERT INTO security_dismissals (repo_id, fingerprint, reason, by_id) VALUES (?, ?, ?, ?)
166 ON CONFLICT (repo_id, fingerprint) DO UPDATE SET reason = excluded.reason, by_id = excluded.by_id,
167 created_at = strftime('%Y-%m-%dT%H:%M:%fZ','now')`,
168 repoID, fp, reason, byID); err != nil {
169 return err
170 }
171 }
172 return tx.Commit()
173}
174
175// ReopenFindings takes dismissals back.
176func (s *Store) ReopenFindings(repoID int64, fingerprints []string) error {
177 tx, err := s.DB.Begin()
178 if err != nil {
179 return err
180 }
181 defer tx.Rollback()
182 for _, fp := range fingerprints {
183 if _, err := tx.Exec("DELETE FROM security_dismissals WHERE repo_id = ? AND fingerprint = ?", repoID, fp); err != nil {
184 return err
185 }
186 }
187 return tx.Commit()
188}
189
190// SecurityDismissals returns a repo's dismissals by fingerprint.
191func (s *Store) SecurityDismissals(repoID int64) (map[string]SecurityDismissal, error) {
192 rows, err := s.DB.Query(`
193 SELECT d.fingerprint, d.reason, COALESCE(u.username, ''), d.created_at
194 FROM security_dismissals d LEFT JOIN users u ON u.id = d.by_id
195 WHERE d.repo_id = ?`, repoID)
196 if err != nil {
197 return nil, err
198 }
199 defer rows.Close()
200 out := map[string]SecurityDismissal{}
201 for rows.Next() {
202 var d SecurityDismissal
203 if err := rows.Scan(&d.Fingerprint, &d.Reason, &d.By, &d.CreatedAt); err != nil {
204 return nil, err
205 }
206 out[d.Fingerprint] = d
207 }
208 return out, rows.Err()
209}
internal/web/templates/settings.html +34 −2
@@ -182,16 +182,48 @@
182</form> 182</form>
183{{if .Security.Enabled}} 183{{if .Security.Enabled}}
184<p class="meta">Next scan {{.Security.NextRun}} · last results {{if .Security.LastResults}}{{when .Security.LastResults}}{{else}}never{{end}}{{if .Security.IssueNumber}} · <a href="/{{.Repo.OwnerName}}/{{.Repo.Name}}/issues/{{.Security.IssueNumber}}">#{{.Security.IssueNumber}}</a>{{end}}</p> 184<p class="meta">Next scan {{.Security.NextRun}} · last results {{if .Security.LastResults}}{{when .Security.LastResults}}{{else}}never{{end}}{{if .Security.IssueNumber}} · <a href="/{{.Repo.OwnerName}}/{{.Repo.Name}}/issues/{{.Security.IssueNumber}}">#{{.Security.IssueNumber}}</a>{{end}}</p>
185{{with .Security.Aside}}{{if or .Ignored .Unused}}<p class="meta">Set aside by the last scan:{{if .Ignored}} {{.Ignored}} under the paths or rules below, or marked vendored or generated in <code>.gitattributes</code>.{{end}}{{range $lib, $n := .Unused}} {{$n}} from {{$lib}} rules, which no manifest names.{{end}}</p>{{end}}{{end}}
185{{if .Security.Findings}}<div class="tablewrap"><table class="keys"> 186{{if .Security.Findings}}<div class="tablewrap"><table class="keys">
186<tr class="cols"><th scope="col">location</th><th scope="col">rule</th><th scope="col">level</th></tr> 187<tr class="cols"><th scope="col">location</th><th scope="col">rule</th><th scope="col">level</th><th scope="col"><span class="vh">dismiss</span></th></tr>
187{{range .Security.Findings}}<tr> 188{{range .Security.Findings}}<tr>
188 <td class="mono">{{.Path}}:{{.Line}}</td> 189 <td class="mono">{{.Path}}:{{.Line}}</td>
189 <td class="mono">{{.Rule}}</td> 190 <td class="mono">{{.Rule}}</td>
190 <td>{{.Level}}</td> 191 <td>{{.Level}}</td>
192 <td><form method="post" action="{{$base}}" class="inline">
193 <input type="hidden" name="field" value="security-dismiss">
194 <input type="hidden" name="fingerprint" value="{{.Fingerprint}}">
195 <select name="reason" aria-label="Reason"><option value="false-positive">False positive</option><option value="wont-fix">Won't fix</option><option value="test">Used in tests</option></select>
196 <button type="submit">Dismiss</button>
197 </form></td>
191</tr> 198</tr>
192{{end}}</table></div> 199{{end}}</table></div>
193{{else if .Security.LastResults}}<p class="none">The last scan found nothing.</p>{{end}} 200{{else if .Security.LastResults}}<p class="none">The last scan found nothing open.</p>{{end}}
201{{if .Security.Dismissed}}<h3>Dismissed</h3><div class="tablewrap"><table class="keys">
202<tr class="cols"><th scope="col">location</th><th scope="col">rule</th><th scope="col">reason</th><th scope="col"><span class="vh">reopen</span></th></tr>
203{{range .Security.Dismissed}}<tr>
204 <td class="mono">{{.Path}}:{{.Line}}</td>
205 <td class="mono">{{.Rule}}</td>
206 <td>{{.Reason}}{{if .DismissedBy}} <span class="meta">by {{.DismissedBy}}</span>{{end}}</td>
207 <td><form method="post" action="{{$base}}" class="inline">
208 <input type="hidden" name="field" value="security-reopen">
209 <input type="hidden" name="fingerprint" value="{{.Fingerprint}}">
210 <button type="submit">Reopen</button>
211 </form></td>
212</tr>
213{{end}}</table></div>{{end}}
194{{end}} 214{{end}}
215<form method="post" action="{{$base}}" class="setform">
216 <input type="hidden" name="field" value="security-ignore">
217 <div><label for="security-ignore">Ignored paths</label><p class="hint">Findings under these paths are set aside from the next scan. Path globs separated by spaces: <code>testdata/**</code>, <code>web/templates/*.html</code>. Paths <code>.gitattributes</code> marks <code>linguist-vendored</code> or <code>linguist-generated</code> are set aside without being listed.</p></div>
218 <div><input type="text" id="security-ignore" name="security-ignore" value="{{join .Repo.Settings.SecurityIgnore " "}}"></div>
219 <div><button type="submit" class="btn">Save</button></div>
220</form>
221<form method="post" action="{{$base}}" class="setform">
222 <input type="hidden" name="field" value="security-exclude-rule">
223 <div><label for="security-exclude-rule">Excluded rules</label><p class="hint">Findings from these rules are set aside from the next scan. A rule id or a family of them, separated by spaces: <code>swift.lang.crypto.insecure-random</code>, <code>javascript.browser</code>.</p></div>
224 <div><input type="text" id="security-exclude-rule" name="security-exclude-rule" value="{{join .Repo.Settings.SecurityExcludeRules " "}}"></div>
225 <div><button type="submit" class="btn">Save</button></div>
226</form>
195</section> 227</section>
196{{end}} 228{{end}}
197 229