Commit 9623fdfd99
Verified · cmc
Layout: unified · split
.gitbay/wiki/Parity.org +4 −1
| @@ -57,6 +57,7 @@ browser-only and the iOS build screen unable to say more than the log. | |||
| 57 | | request a review | yes | yes | yes | | 57 | | request a review | yes | yes | yes | |
| 58 | | choose body markup | yes | yes | yes | | 58 | | choose body markup | yes | yes | yes | |
| 59 | | preview body markup | n/a | yes | no | | 59 | | preview body markup | n/a | yes | no | |
| 60 | | TeX math as MathML | n/a | yes | no | | ||
| 60 | | stacked merge requests | yes | yes | yes | | 61 | | stacked merge requests | yes | yes | yes | |
| 61 | | revisions | yes | yes | yes | | 62 | | revisions | yes | yes | yes | |
| 62 | | range-diff | yes | yes | yes | | 63 | | range-diff | yes | yes | yes | |
| @@ -125,6 +126,7 @@ reviews, since an approval was of the diff against the old branch. | |||
| 125 | | labels: list, colour | yes | yes | yes | | 126 | | labels: list, colour | yes | yes | yes | |
| 126 | | choose body markup | yes | yes | yes | | 127 | | choose body markup | yes | yes | yes | |
| 127 | | preview body markup | n/a | yes | no | | 128 | | preview body markup | n/a | yes | no | |
| 129 | | TeX math as MathML | n/a | yes | no | | ||
| 128 | | issue templates | yes | yes | yes | | 130 | | issue templates | yes | yes | yes | |
| 129 | | milestone create, close, reopen | yes | yes | yes | | 131 | | milestone create, close, reopen | yes | yes | yes | |
| 130 | | org labels: set, list, remove | yes | yes | yes | | 132 | | org labels: set, list, remove | yes | yes | yes | |
| @@ -155,7 +157,8 @@ every surface now offers the choice: the web's create forms, and the iOS | |||
| 155 | composer on create, edit and comment. An edit starts on the format its | 157 | composer on create, edit and comment. An edit starts on the format its |
| 156 | body was stored in, since starting elsewhere would silently reinterpret | 158 | body was stored in, since starting elsewhere would silently reinterpret |
| 157 | it on the next save. Diff-line comments have no format column and are | 159 | it on the next save. Diff-line comments have no format column and are |
| 158 | always markdown. | 160 | always markdown. TeX math in either format renders as MathML on the |
| 161 | web (#294); the terminal shows the source, so the CLI row is =n/a=. | ||
| 159 | 162 | ||
| 160 | Every web form that takes markup has a Preview button beside its own | 163 | Every web form that takes markup has a Preview button beside its own |
| 161 | submit: issue and merge request create, their edit and comment boxes, | 164 | submit: issue and merge request create, their edit and comment boxes, |
.gitbay/wiki/Users.org +15
| @@ -299,6 +299,21 @@ does on the repository page, with relative links resolved against the | |||
| 299 | file's directory; =source= in the file's action bar (or =?view=source=) | 299 | file's directory; =source= in the file's action bar (or =?view=source=) |
| 300 | shows the text instead. Other files show the text with highlighting. | 300 | shows the text instead. Other files show the text with highlighting. |
| 301 | 301 | ||
| 302 | TeX math renders as MathML wherever markdown or org renders: READMEs, | ||
| 303 | files, wiki pages, issue, merge request and release bodies, comments, | ||
| 304 | and their previews. Markdown takes =$…$= inline and =$$…$$= for display, | ||
| 305 | either inline or with the =$$= lines on their own. A =$= followed by a | ||
| 306 | space, or a closing =$= preceded by a space or followed by a digit, is a | ||
| 307 | dollar sign, so =$5 and $10= stays prose; =\$= is always one. Org takes | ||
| 308 | =$…$=, =$$…$$=, =\(…\)=, =\[…\]= and =\begin{…}…\end{…}=. Code spans | ||
| 309 | and blocks are left alone. The supported TeX is a subset (letters, | ||
| 310 | numbers, operators, scripts, =\frac=, =\sqrt=, Greek and common symbols, | ||
| 311 | =\left=/=\right=, accents, =\text=, the =\math…= fonts, matrices, | ||
| 312 | =cases= and =aligned=; the full list heads =internal/texmath/texmath.go=). | ||
| 313 | Anything outside it, including macros, colours and links, shows as | ||
| 314 | source, as does an expression over 8 KiB or nested more than 64 deep. | ||
| 315 | The CLI shows the source. | ||
| 316 | |||
| 302 | * Organizations | 317 | * Organizations |
| 303 | 318 | ||
| 304 | Orgs share the owner namespace with users and own repositories at | 319 | Orgs share the owner namespace with users and own repositories at |
CHANGELOG.org +5
| @@ -47,6 +47,11 @@ anything beyond "replace the binary and restart" is needed. | |||
| 47 | with ={items, next}=. =dashboard= gains =queries= for pinned ones; the | 47 | with ={items, next}=. =dashboard= gains =queries= for pinned ones; the |
| 48 | web shows them on the dashboard and at =/<you>/-/queries=. An account | 48 | web shows them on the dashboard and at =/<you>/-/queries=. An account |
| 49 | keeps at most 50 saved queries, 10 pinned (#292). | 49 | keeps at most 50 saved queries, 10 pinned (#292). |
| 50 | - TeX math renders server-side as MathML in markdown (=$…$=, =$$…$$=) | ||
| 51 | and org (=$…$=, =\(…\)=, =\[…\]=, LaTeX environments) wherever | ||
| 52 | markup renders, through a subset converter in =internal/texmath=; | ||
| 53 | the sanitizer admits exactly the elements and attributes it emits, | ||
| 54 | and anything outside the subset shows as source (#294). | ||
| 50 | 55 | ||
| 51 | * v1.38.0 — 2026-09-29 | 56 | * v1.38.0 — 2026-09-29 |
| 52 | 57 | ||
internal/autolink/autolink.go +2 −2
| @@ -2,7 +2,7 @@ | |||
| 2 | // to the repository's issues and merge requests, owner/name#N (and !N) | 2 | // to the repository's issues and merge requests, owner/name#N (and !N) |
| 3 | // across repositories, and @user to owner pages. It operates on the HTML | 3 | // across repositories, and @user to owner pages. It operates on the HTML |
| 4 | // produced by the markdown/org pipeline, walking text nodes with a real | 4 | // produced by the markdown/org pipeline, walking text nodes with a real |
| 5 | // parser so nothing inside <a>, <code>, or <pre> is ever touched, and only | 5 | // parser so nothing inside <a>, <code>, <pre> or <math> is ever touched, and only |
| 6 | // references that actually resolve become links. | 6 | // references that actually resolve become links. |
| 7 | package autolink | 7 | package autolink |
| 8 | 8 | ||
| @@ -36,7 +36,7 @@ var ( | |||
| 36 | ) | 36 | ) |
| 37 | 37 | ||
| 38 | // skip lists elements whose text must never be rewritten. | 38 | // skip lists elements whose text must never be rewritten. |
| 39 | var skip = map[string]bool{"a": true, "code": true, "pre": true, "script": true, "style": true} | 39 | var skip = map[string]bool{"a": true, "code": true, "math": true, "pre": true, "script": true, "style": true} |
| 40 | 40 | ||
| 41 | // Rewrite processes an HTML fragment, linking references relative to | 41 | // Rewrite processes an HTML fragment, linking references relative to |
| 42 | // defaultOwner/defaultName. On any parse failure the input is returned | 42 | // defaultOwner/defaultName. On any parse failure the input is returned |
internal/httpd/math.go added +260
| @@ -0,0 +1,260 @@ | |||
| 1 | package httpd | ||
| 2 | |||
| 3 | import ( | ||
| 4 | "bytes" | ||
| 5 | "html/template" | ||
| 6 | "regexp" | ||
| 7 | "strings" | ||
| 8 | |||
| 9 | "github.com/microcosm-cc/bluemonday" | ||
| 10 | "github.com/niklasfasching/go-org/org" | ||
| 11 | "github.com/yuin/goldmark" | ||
| 12 | "github.com/yuin/goldmark/ast" | ||
| 13 | "github.com/yuin/goldmark/parser" | ||
| 14 | "github.com/yuin/goldmark/renderer" | ||
| 15 | "github.com/yuin/goldmark/text" | ||
| 16 | "github.com/yuin/goldmark/util" | ||
| 17 | |||
| 18 | "gitbay.org/gitbay/internal/texmath" | ||
| 19 | ) | ||
| 20 | |||
| 21 | // TeX math renders server-side as MathML, which browsers display natively, | ||
| 22 | // so the page needs no script and the CSP does not change (#294). | ||
| 23 | // | ||
| 24 | // The converter is internal/texmath rather than a library. The pure-Go | ||
| 25 | // options were treeblood (MIT), which writes \color and \class arguments | ||
| 26 | // into attributes unescaped, expands \def macros without a bound (a 180-byte | ||
| 27 | // input produced 3 MB), did not finish 5000 nested braces in 30 seconds and | ||
| 28 | // logs to stderr; goldmark-mathml, which runs Temml in a JavaScript VM; and | ||
| 29 | // converters inside large typesetting modules. texmath covers a documented | ||
| 30 | // subset, refuses everything else, and bounds input size and nesting. | ||
| 31 | |||
| 32 | // mathHTML renders one expression, or escapes its source when the converter | ||
| 33 | // refuses it. The result passes through ugcPolicy even on the markdown path, | ||
| 34 | // so the policy is the one statement of what math may emit. | ||
| 35 | func mathHTML(tex, source string, display bool) (string, bool) { | ||
| 36 | out, err := texmath.Convert(tex, display) | ||
| 37 | if err != nil { | ||
| 38 | return template.HTMLEscapeString(source), false | ||
| 39 | } | ||
| 40 | return ugcPolicy.Sanitize(out), true | ||
| 41 | } | ||
| 42 | |||
| 43 | // allowMath admits exactly the MathML texmath emits: its elements, and each | ||
| 44 | // attribute only on its element and only with the values it writes. | ||
| 45 | func allowMath(p *bluemonday.Policy) { | ||
| 46 | p.AllowElements(texmath.Elements...) | ||
| 47 | p.AllowNoAttrs().OnElements(texmath.Elements...) | ||
| 48 | for element, attrs := range texmath.Attrs { | ||
| 49 | for name, value := range attrs { | ||
| 50 | pattern := `^` + regexp.QuoteMeta(value) + `$` | ||
| 51 | if value == "<length>" { | ||
| 52 | pattern = `^-?[0-9]+(\.[0-9]+)?em$` | ||
| 53 | } | ||
| 54 | p.AllowAttrs(name).Matching(regexp.MustCompile(pattern)).OnElements(element) | ||
| 55 | } | ||
| 56 | } | ||
| 57 | } | ||
| 58 | |||
| 59 | // Markdown: $…$ inline and $$…$$ display, inline or as a block. | ||
| 60 | |||
| 61 | var ( | ||
| 62 | kindMath = ast.NewNodeKind("Math") | ||
| 63 | kindMathBlock = ast.NewNodeKind("MathBlock") | ||
| 64 | ) | ||
| 65 | |||
| 66 | type mathInline struct { | ||
| 67 | ast.BaseInline | ||
| 68 | tex string | ||
| 69 | display bool | ||
| 70 | } | ||
| 71 | |||
| 72 | func (n *mathInline) Kind() ast.NodeKind { return kindMath } | ||
| 73 | func (n *mathInline) Dump(src []byte, level int) { | ||
| 74 | ast.DumpHelper(n, src, level, map[string]string{"TeX": n.tex}, nil) | ||
| 75 | } | ||
| 76 | |||
| 77 | type mathBlock struct { | ||
| 78 | ast.BaseBlock | ||
| 79 | tex []byte | ||
| 80 | closed bool | ||
| 81 | } | ||
| 82 | |||
| 83 | func (n *mathBlock) Kind() ast.NodeKind { return kindMathBlock } | ||
| 84 | func (n *mathBlock) Dump(src []byte, level int) { | ||
| 85 | ast.DumpHelper(n, src, level, map[string]string{"TeX": string(n.tex)}, nil) | ||
| 86 | } | ||
| 87 | |||
| 88 | func isMathSpace(c byte) bool { return c == ' ' || c == '\t' || c == '\n' || c == '\r' } | ||
| 89 | |||
| 90 | // mathInlineParser follows pandoc's rule so prices stay prose: the opening | ||
| 91 | // $ has a non-space after it, and the closing $ a non-space before it and no | ||
| 92 | // digit after it. "$5 and $10" is text. A backslash escapes the next byte. | ||
| 93 | type mathInlineParser struct{} | ||
| 94 | |||
| 95 | func (mathInlineParser) Trigger() []byte { return []byte{'$'} } | ||
| 96 | |||
| 97 | func (mathInlineParser) Parse(parent ast.Node, block text.Reader, pc parser.Context) ast.Node { | ||
| 98 | line, seg := block.PeekLine() | ||
| 99 | if len(line) >= 2 && line[1] == '$' { | ||
| 100 | body := line[2:] | ||
| 101 | for i := 0; i+1 < len(body); i++ { | ||
| 102 | if body[i] == '\\' { | ||
| 103 | i++ | ||
| 104 | continue | ||
| 105 | } | ||
| 106 | if body[i] == '$' && body[i+1] == '$' { | ||
| 107 | if i == 0 { | ||
| 108 | break | ||
| 109 | } | ||
| 110 | block.Advance(i + 4) | ||
| 111 | return &mathInline{tex: string(body[:i]), display: true} | ||
| 112 | } | ||
| 113 | } | ||
| 114 | // Consume both dollars so the second does not open inline math. | ||
| 115 | block.Advance(2) | ||
| 116 | return ast.NewTextSegment(seg.WithStop(seg.Start + 2)) | ||
| 117 | } | ||
| 118 | body := line[1:] | ||
| 119 | if len(body) == 0 || isMathSpace(body[0]) { | ||
| 120 | return nil | ||
| 121 | } | ||
| 122 | for i := 0; i < len(body); i++ { | ||
| 123 | switch body[i] { | ||
| 124 | case '\\': | ||
| 125 | i++ | ||
| 126 | case '$': | ||
| 127 | if isMathSpace(body[i-1]) || i+1 < len(body) && body[i+1] >= '0' && body[i+1] <= '9' { | ||
| 128 | continue | ||
| 129 | } | ||
| 130 | block.Advance(i + 2) | ||
| 131 | return &mathInline{tex: string(body[:i])} | ||
| 132 | } | ||
| 133 | } | ||
| 134 | return nil | ||
| 135 | } | ||
| 136 | |||
| 137 | // mathBlockParser opens on a line that is $$ alone, or $$…$$ whole, and | ||
| 138 | // runs to the line that ends with $$. | ||
| 139 | type mathBlockParser struct{} | ||
| 140 | |||
| 141 | func (mathBlockParser) Trigger() []byte { return []byte{'$'} } | ||
| 142 | |||
| 143 | func (mathBlockParser) Open(parent ast.Node, reader text.Reader, pc parser.Context) (ast.Node, parser.State) { | ||
| 144 | line, _ := reader.PeekLine() | ||
| 145 | pos := pc.BlockOffset() | ||
| 146 | if pos < 0 || !bytes.HasPrefix(line[pos:], []byte("$$")) { | ||
| 147 | return nil, parser.NoChildren | ||
| 148 | } | ||
| 149 | rest := util.TrimRightSpace(line[pos+2:]) | ||
| 150 | n := &mathBlock{} | ||
| 151 | switch { | ||
| 152 | case len(rest) == 0: | ||
| 153 | case len(rest) > 2 && bytes.HasSuffix(rest, []byte("$$")): | ||
| 154 | n.tex, n.closed = rest[:len(rest)-2], true | ||
| 155 | default: | ||
| 156 | return nil, parser.NoChildren | ||
| 157 | } | ||
| 158 | reader.AdvanceToEOL() | ||
| 159 | return n, parser.NoChildren | ||
| 160 | } | ||
| 161 | |||
| 162 | func (mathBlockParser) Continue(node ast.Node, reader text.Reader, pc parser.Context) parser.State { | ||
| 163 | n := node.(*mathBlock) | ||
| 164 | if n.closed { | ||
| 165 | return parser.Close | ||
| 166 | } | ||
| 167 | line, _ := reader.PeekLine() | ||
| 168 | if line == nil { | ||
| 169 | return parser.Close | ||
| 170 | } | ||
| 171 | trimmed := util.TrimRightSpace(line) | ||
| 172 | if bytes.HasSuffix(trimmed, []byte("$$")) { | ||
| 173 | n.tex = append(n.tex, trimmed[:len(trimmed)-2]...) | ||
| 174 | n.closed = true | ||
| 175 | reader.AdvanceToEOL() | ||
| 176 | return parser.Close | ||
| 177 | } | ||
| 178 | n.tex = append(n.tex, line...) | ||
| 179 | reader.AdvanceToEOL() | ||
| 180 | return parser.Continue | parser.NoChildren | ||
| 181 | } | ||
| 182 | |||
| 183 | func (mathBlockParser) Close(ast.Node, text.Reader, parser.Context) {} | ||
| 184 | func (mathBlockParser) CanInterruptParagraph() bool { return true } | ||
| 185 | func (mathBlockParser) CanAcceptIndentedLine() bool { return false } | ||
| 186 | |||
| 187 | type mathRenderer struct{} | ||
| 188 | |||
| 189 | func (mathRenderer) RegisterFuncs(reg renderer.NodeRendererFuncRegisterer) { | ||
| 190 | reg.Register(kindMath, func(w util.BufWriter, _ []byte, node ast.Node, entering bool) (ast.WalkStatus, error) { | ||
| 191 | if entering { | ||
| 192 | n := node.(*mathInline) | ||
| 193 | delim := "$" | ||
| 194 | if n.display { | ||
| 195 | delim = "$$" | ||
| 196 | } | ||
| 197 | out, _ := mathHTML(n.tex, delim+n.tex+delim, n.display) | ||
| 198 | _, _ = w.WriteString(out) | ||
| 199 | } | ||
| 200 | return ast.WalkSkipChildren, nil | ||
| 201 | }) | ||
| 202 | reg.Register(kindMathBlock, func(w util.BufWriter, _ []byte, node ast.Node, entering bool) (ast.WalkStatus, error) { | ||
| 203 | if !entering { | ||
| 204 | return ast.WalkContinue, nil | ||
| 205 | } | ||
| 206 | n := node.(*mathBlock) | ||
| 207 | source := "$$" + string(n.tex) | ||
| 208 | if n.closed { | ||
| 209 | source += "$$" | ||
| 210 | } | ||
| 211 | out, ok := mathHTML(string(n.tex), source, true) | ||
| 212 | if !ok || !n.closed { | ||
| 213 | out = "<pre>" + template.HTMLEscapeString(source) + "</pre>" | ||
| 214 | } | ||
| 215 | _, _ = w.WriteString(out + "\n") | ||
| 216 | return ast.WalkSkipChildren, nil | ||
| 217 | }) | ||
| 218 | } | ||
| 219 | |||
| 220 | type mathExtension struct{} | ||
| 221 | |||
| 222 | func (mathExtension) Extend(m goldmark.Markdown) { | ||
| 223 | m.Parser().AddOptions( | ||
| 224 | parser.WithBlockParsers(util.Prioritized(mathBlockParser{}, 701)), | ||
| 225 | parser.WithInlineParsers(util.Prioritized(mathInlineParser{}, 501))) | ||
| 226 | m.Renderer().AddOptions(renderer.WithNodeRenderers(util.Prioritized(mathRenderer{}, 500))) | ||
| 227 | } | ||
| 228 | |||
| 229 | // Org: go-org already parses $…$, $$…$$, \(…\), \[…\] and \begin{…}…\end{…} | ||
| 230 | // as LaTeX fragments, and \begin{…} on its own lines as a LaTeX block; it | ||
| 231 | // writes them back out as text. These render them instead. | ||
| 232 | |||
| 233 | func (w *orgWriter) WriteLatexFragment(l org.LatexFragment) { | ||
| 234 | tex := org.String(l.Content...) | ||
| 235 | source := l.OpeningPair + tex + l.ClosingPair | ||
| 236 | // go-org takes any $…$; org's own rule keeps "$5 and $10" prose. | ||
| 237 | if l.OpeningPair == "$" && (tex == "" || isMathSpace(tex[0]) || isMathSpace(tex[len(tex)-1])) { | ||
| 238 | w.WriteText(org.Text{Content: source, IsRaw: true}) | ||
| 239 | return | ||
| 240 | } | ||
| 241 | display := l.OpeningPair != "$" && l.OpeningPair != `\(` | ||
| 242 | if strings.HasPrefix(l.OpeningPair, `\begin{`) { | ||
| 243 | tex = source | ||
| 244 | } | ||
| 245 | out, ok := mathHTML(tex, source, display) | ||
| 246 | if !ok { | ||
| 247 | w.WriteText(org.Text{Content: source, IsRaw: true}) | ||
| 248 | return | ||
| 249 | } | ||
| 250 | w.WriteString(out) | ||
| 251 | } | ||
| 252 | |||
| 253 | func (w *orgWriter) WriteLatexBlock(b org.LatexBlock) { | ||
| 254 | tex := org.String(b.Content...) | ||
| 255 | if out, ok := mathHTML(tex, tex, true); ok { | ||
| 256 | w.WriteString(out + "\n") | ||
| 257 | return | ||
| 258 | } | ||
| 259 | w.WriteString("<pre>" + template.HTMLEscapeString(tex) + "</pre>\n") | ||
| 260 | } | ||
internal/httpd/math_test.go added +183
| @@ -0,0 +1,183 @@ | |||
| 1 | package httpd | ||
| 2 | |||
| 3 | import ( | ||
| 4 | "net/http/httptest" | ||
| 5 | "strings" | ||
| 6 | "testing" | ||
| 7 | "time" | ||
| 8 | |||
| 9 | "gitbay.org/gitbay/internal/config" | ||
| 10 | "gitbay.org/gitbay/internal/store" | ||
| 11 | ) | ||
| 12 | |||
| 13 | func TestMarkdownMath(t *testing.T) { | ||
| 14 | cases := []struct { | ||
| 15 | name, src string | ||
| 16 | want []string | ||
| 17 | not []string | ||
| 18 | }{ | ||
| 19 | {"inline", "Euler: $e^{i\\pi}+1=0$.", []string{`<math><msup><mi>e</mi>`, `</math>.`}, nil}, | ||
| 20 | {"display inline", "so $$x^2$$ here", []string{`<math display="block"><msup>`}, nil}, | ||
| 21 | {"block", "text\n$$\n\\frac{a}{b}\n$$\nafter\n", []string{`<math display="block"><mfrac>`, `<p>after</p>`}, []string{"$$"}}, | ||
| 22 | {"one-line block", "$$x_1$$\n", []string{`<math display="block"><msub>`}, []string{"<p>"}}, | ||
| 23 | {"prices", "costs $5 and $10 today", []string{"costs $5 and $10 today"}, []string{"<math"}}, | ||
| 24 | {"space after open", "a $ x$ b", []string{"a $ x$ b"}, []string{"<math"}}, | ||
| 25 | {"space before close", "a $x $ b", []string{"a $x $ b"}, []string{"<math"}}, | ||
| 26 | {"digit after close", "$x$5", []string{"$x$5"}, []string{"<math"}}, | ||
| 27 | {"escaped dollars", `\$x\$`, []string{"$x$"}, []string{"<math"}}, | ||
| 28 | {"escaped dollar inside", `$a\$b$`, []string{`<mo>$</mo>`}, nil}, | ||
| 29 | {"code span", "`$x^2$`", []string{"<code>$x^2$</code>"}, []string{"<math"}}, | ||
| 30 | {"fenced code", "```\n$x^2$\n$$\ny\n$$\n```\n", []string{"$x^2$", "$$\ny\n$$"}, []string{"<math"}}, | ||
| 31 | {"indented code", " $x$\n", []string{"$x$"}, []string{"<math"}}, | ||
| 32 | {"invalid inline", `see $\frac{a$ here`, []string{`see $\frac{a$ here`}, []string{"<math"}}, | ||
| 33 | {"invalid block", "$$\n\\frac{\n$$\n", []string{"<pre tabindex=\"0\">$$\\frac{\n$$</pre>"}, []string{"<math"}}, | ||
| 34 | {"unclosed block", "$$\nx\n", []string{"<pre"}, []string{"<math"}}, | ||
| 35 | {"markup is escaped", "$\\text{<b>&</b>}$", []string{`<mtext><b>&</b></mtext>`}, []string{"<b>"}}, | ||
| 36 | } | ||
| 37 | for _, c := range cases { | ||
| 38 | out := string(ugcHTML(c.src, "md")) | ||
| 39 | for _, w := range c.want { | ||
| 40 | if !strings.Contains(out, w) { | ||
| 41 | t.Errorf("%s: lacks %q:\n%s", c.name, w, out) | ||
| 42 | } | ||
| 43 | } | ||
| 44 | for _, w := range c.not { | ||
| 45 | if strings.Contains(out, w) { | ||
| 46 | t.Errorf("%s: has %q:\n%s", c.name, w, out) | ||
| 47 | } | ||
| 48 | } | ||
| 49 | } | ||
| 50 | } | ||
| 51 | |||
| 52 | func TestOrgMath(t *testing.T) { | ||
| 53 | cases := []struct { | ||
| 54 | name, src string | ||
| 55 | want []string | ||
| 56 | not []string | ||
| 57 | }{ | ||
| 58 | {"dollar", "Euler: $e^x$ here", []string{`<math><msup><mi>e</mi><mi>x</mi></msup></math> here`}, nil}, | ||
| 59 | {"paren", `a \(x_1\) b`, []string{`<math><msub>`}, []string{`\(`}}, | ||
| 60 | {"bracket", `a \[x^2\] b`, []string{`<math display="block"><msup>`}, []string{`\[`}}, | ||
| 61 | {"double dollar", `a $$x$$ b`, []string{`<math display="block"><mi>x</mi></math>`}, nil}, | ||
| 62 | {"environment block", "\\begin{equation}\nx = \\frac{1}{2}\n\\end{equation}\n", []string{`<math display="block"><mrow><mi>x</mi><mo>=</mo><mfrac>`}, []string{`\begin`}}, | ||
| 63 | {"matrix block", "\\begin{pmatrix}\na & b \\\\\nc & d\n\\end{pmatrix}\n", []string{`<mtable><mtr><mtd><mi>a</mi></mtd>`}, nil}, | ||
| 64 | {"prices", "costs $5 and $10 today", []string{"costs $5 and $10 today"}, []string{"<math"}}, | ||
| 65 | {"verbatim", "=$x^2$= and ~$y$~", []string{"<code>$x^2$</code>", "<code>$y$</code>"}, []string{"<math"}}, | ||
| 66 | {"src block", "#+begin_src tex\n$x^2$\n#+end_src\n", []string{"<pre"}, []string{"<math"}}, | ||
| 67 | {"invalid", `see \(\frac{a\) here`, []string{`see \(\frac{a\) here`}, []string{"<math"}}, | ||
| 68 | {"invalid block", "\\begin{tabular}\nx\n\\end{tabular}\n", []string{`<pre tabindex="0">\begin{tabular}`}, []string{"<math"}}, | ||
| 69 | } | ||
| 70 | for _, c := range cases { | ||
| 71 | out := string(ugcHTML(c.src, "org")) | ||
| 72 | for _, w := range c.want { | ||
| 73 | if !strings.Contains(out, w) { | ||
| 74 | t.Errorf("%s: lacks %q:\n%s", c.name, w, out) | ||
| 75 | } | ||
| 76 | } | ||
| 77 | for _, w := range c.not { | ||
| 78 | if strings.Contains(out, w) { | ||
| 79 | t.Errorf("%s: has %q:\n%s", c.name, w, out) | ||
| 80 | } | ||
| 81 | } | ||
| 82 | } | ||
| 83 | } | ||
| 84 | |||
| 85 | // ugcPolicy admits the MathML texmath writes and nothing else, which is | ||
| 86 | // what an .html README or org's raw export would otherwise carry through. | ||
| 87 | func TestUGCPolicyMathML(t *testing.T) { | ||
| 88 | hostile := `<math display="block" xmlns:xlink="http://www.w3.org/1999/xlink" style="x" onclick="x()">` + | ||
| 89 | `<mi href="javascript:alert(1)" xlink:href="javascript:alert(2)" mathvariant="bold" style="color:red" onmouseover="x()">x</mi>` + | ||
| 90 | `<mo stretchy="true" form="prefix">(</mo><mspace width="expression(alert(3))"></mspace><mspace width="1em"></mspace>` + | ||
| 91 | `<semantics><annotation-xml encoding="text/html"><img src=x onerror="alert(4)"></annotation-xml></semantics>` + | ||
| 92 | `<maction actiontype="statusline"><mi>y</mi></maction><mstyle mathcolor="red"><mi>z</mi></mstyle>` + | ||
| 93 | `<mtext><style>*{}</style><script>alert(5)</script></mtext></math><math display="inline"></math>` | ||
| 94 | out := ugcPolicy.Sanitize(hostile) | ||
| 95 | for _, bad := range []string{"href", "xlink", "style", "onclick", "onmouseover", "onerror", "javascript", | ||
| 96 | "annotation", "semantics", "maction", "mstyle", "mathcolor", "script", "expression", | ||
| 97 | `mathvariant="bold"`, `stretchy="true"`, "form=", `display="inline"`} { | ||
| 98 | if strings.Contains(out, bad) { | ||
| 99 | t.Errorf("sanitized MathML keeps %q:\n%s", bad, out) | ||
| 100 | } | ||
| 101 | } | ||
| 102 | for _, good := range []string{`<math display="block">`, `<mspace width="1em">`, "<mi>x</mi>", "<mi>y</mi>"} { | ||
| 103 | if !strings.Contains(out, good) { | ||
| 104 | t.Errorf("sanitized MathML lacks %q:\n%s", good, out) | ||
| 105 | } | ||
| 106 | } | ||
| 107 | } | ||
| 108 | |||
| 109 | // Hostile TeX is refused and shown as escaped source, in bounded time and | ||
| 110 | // size, on both syntaxes. | ||
| 111 | func TestHostileMath(t *testing.T) { | ||
| 112 | long := strings.Repeat(`x+`, 1<<19) // 1 MiB in one expression | ||
| 113 | deep := strings.Repeat("{", 50000) + "x" + strings.Repeat("}", 50000) | ||
| 114 | for _, tex := range []string{ | ||
| 115 | `\href{javascript:alert(1)}{x}`, `\url{javascript:alert(1)}`, `\style{color:red}{x}`, | ||
| 116 | `\color{red" onmouseover="alert(1)}{x}`, `\class{a"b}{x}`, `\def\a{\a\a}\a`, | ||
| 117 | `\text{</math><script>alert(1)</script>}`, `\text{<img src=x onerror=alert(1)>}`, | ||
| 118 | deep, long, strings.Repeat(`\sqrt{`, 10000) + "x", | ||
| 119 | } { | ||
| 120 | for _, doc := range []struct{ src, format string }{ | ||
| 121 | {"$" + tex + "$", "md"}, {"$$\n" + tex + "\n$$\n", "md"}, | ||
| 122 | {`\(` + tex + `\)`, "org"}, {"\\[" + tex + "\\]", "org"}, | ||
| 123 | } { | ||
| 124 | start := time.Now() | ||
| 125 | out := string(ugcHTML(doc.src, doc.format)) | ||
| 126 | if d := time.Since(start); d > 2*time.Second { | ||
| 127 | t.Errorf("%.30q (%s) took %v", tex, doc.format, d) | ||
| 128 | } | ||
| 129 | if len(out) > 8*len(doc.src)+1024 { | ||
| 130 | t.Errorf("%.30q (%s): %d bytes out for %d in", tex, doc.format, len(out), len(doc.src)) | ||
| 131 | } | ||
| 132 | for _, bad := range []string{"<script", "<img", `href="`, `onmouseover="`, `style="`, `class="a`} { | ||
| 133 | if strings.Contains(out, bad) { | ||
| 134 | t.Errorf("%.30q (%s) emits %q:\n%.300s", tex, doc.format, bad, out) | ||
| 135 | } | ||
| 136 | } | ||
| 137 | } | ||
| 138 | } | ||
| 139 | // A refused \text keeps its payload as visible text, escaped. | ||
| 140 | out := string(ugcHTML(`$\href{javascript:alert(1)}{x}$`, "md")) | ||
| 141 | if !strings.Contains(out, `$\href{javascript:alert(1)}{x}$`) || strings.Contains(out, "<math") || strings.Contains(out, "<a") { | ||
| 142 | t.Errorf("refused \\href: %s", out) | ||
| 143 | } | ||
| 144 | } | ||
| 145 | |||
| 146 | // The issue page renders its body's math through the shared path, and | ||
| 147 | // autolinking leaves text inside <math> alone. | ||
| 148 | func TestIssuePageRendersMath(t *testing.T) { | ||
| 149 | st, err := store.Open(":memory:") | ||
| 150 | if err != nil { | ||
| 151 | t.Fatal(err) | ||
| 152 | } | ||
| 153 | defer st.Close() | ||
| 154 | if err := st.MigrateUp(); err != nil { | ||
| 155 | t.Fatal(err) | ||
| 156 | } | ||
| 157 | uid, err := st.CreateUser("alice", false) | ||
| 158 | if err != nil { | ||
| 159 | t.Fatal(err) | ||
| 160 | } | ||
| 161 | repoID, err := st.CreateRepo("user", uid, "app", "public") | ||
| 162 | if err != nil { | ||
| 163 | t.Fatal(err) | ||
| 164 | } | ||
| 165 | if _, err := st.CreateIssue(repoID, uid, "math", `Area is $\pi r^2$, see $\text{#1}$.`, "md"); err != nil { | ||
| 166 | t.Fatal(err) | ||
| 167 | } | ||
| 168 | cfg := config.Default() | ||
| 169 | cfg.Web.Mode = "accounts" | ||
| 170 | s := New(cfg, st, nil) | ||
| 171 | rr := httptest.NewRecorder() | ||
| 172 | s.Handler().ServeHTTP(rr, httptest.NewRequest("GET", "/alice/app/issues/1", nil)) | ||
| 173 | if rr.Code != 200 { | ||
| 174 | t.Fatalf("status %d", rr.Code) | ||
| 175 | } | ||
| 176 | body := rr.Body.String() | ||
| 177 | if !strings.Contains(body, `<math><mi>π</mi><msup><mi>r</mi><mn>2</mn></msup></math>`) { | ||
| 178 | t.Errorf("no MathML in the issue page:\n%s", body) | ||
| 179 | } | ||
| 180 | if !strings.Contains(body, `<mtext>#1</mtext>`) { | ||
| 181 | t.Errorf("autolink rewrote text inside <math>:\n%s", body) | ||
| 182 | } | ||
| 183 | } | ||
internal/httpd/web.go +4 −3
| @@ -1192,13 +1192,13 @@ var imageTypes = map[string]string{ | |||
| 1192 | 1192 | ||
| 1193 | // markdown is the shared renderer: GFM (tables, strikethrough, autolinks, | 1193 | // markdown is the shared renderer: GFM (tables, strikethrough, autolinks, |
| 1194 | // task lists) on top of CommonMark, with class-based fence highlighting | 1194 | // task lists) on top of CommonMark, with class-based fence highlighting |
| 1195 | // (the palette lives in the stylesheet, per scheme). Raw HTML is still | 1195 | // (the palette lives in the stylesheet, per scheme), and TeX math as |
| 1196 | // dropped. | 1196 | // MathML (math.go). Raw HTML is still dropped. |
| 1197 | // Headings carry ids so a README or wiki section can be linked to, the | 1197 | // Headings carry ids so a README or wiki section can be linked to, the |
| 1198 | // way org headings already are (#132). | 1198 | // way org headings already are (#132). |
| 1199 | var markdown = goldmark.New( | 1199 | var markdown = goldmark.New( |
| 1200 | goldmark.WithParserOptions(parser.WithAutoHeadingID()), | 1200 | goldmark.WithParserOptions(parser.WithAutoHeadingID()), |
| 1201 | goldmark.WithExtensions(extension.GFM, | 1201 | goldmark.WithExtensions(extension.GFM, mathExtension{}, |
| 1202 | highlighting.NewHighlighting(highlighting.WithFormatOptions(html.WithClasses(true))))) | 1202 | highlighting.NewHighlighting(highlighting.WithFormatOptions(html.WithClasses(true))))) |
| 1203 | 1203 | ||
| 1204 | // fenceHighlight renders one code block with chroma classes, for org and | 1204 | // fenceHighlight renders one code block with chroma classes, for org and |
| @@ -1353,6 +1353,7 @@ var ugcPolicy = func() *bluemonday.Policy { | |||
| 1353 | p.AllowAttrs("class"). | 1353 | p.AllowAttrs("class"). |
| 1354 | Matching(regexp.MustCompile(`^(chroma|[a-z0-9]{1,3})( (chroma|[a-z0-9]{1,3}))*$`)). | 1354 | Matching(regexp.MustCompile(`^(chroma|[a-z0-9]{1,3})( (chroma|[a-z0-9]{1,3}))*$`)). |
| 1355 | OnElements("span", "pre", "code", "div") | 1355 | OnElements("span", "pre", "code", "div") |
| 1356 | allowMath(p) | ||
| 1356 | return p | 1357 | return p |
| 1357 | }() | 1358 | }() |
| 1358 | 1359 | ||