markup: TeX math as MathML !532

merged merged by cmc on 2026-09-29 06:33 UTC · krz/gitbay:math-294 into main

11 files changed, +1906 −9

Layout: unified · split

.gitbay/wiki/Parity.org +4 −1

.gitbay/wiki/Users.org +19

	@@ -299,6 +299,25 @@ does on the repository page, with relative links resolved against the
299	file's directory; =source= in the file's action bar (or =?view=source=)	299	file's directory; =source= in the file's action bar (or =?view=source=)
300	shows the text instead. Other files show the text with highlighting.	300	shows the text instead. Other files show the text with highlighting.
301		301
		302	TeX math renders as MathML wherever markdown or org renders: READMEs,
		303	files, wiki pages, issue, merge request and release bodies, comments,
		304	and their previews. Markdown takes =$…$= inline and =$$…$$= for display,
		305	either inline or with the =$$= lines on their own. A =$= followed by a
		306	space, or a closing =$= preceded by a space or followed by a digit, is a
		307	dollar sign, so =$5 and $10= stays prose; =\$= is always one. Org takes
		308	=$…$=, =$$…$$=, =$…$=, =\[…\]= and =\begin{…}…\end{…}=. Code spans
		309	and blocks are left alone. The supported TeX is a subset (letters,
		310	numbers, operators, scripts, =\frac=, =\sqrt=, Greek and common symbols,
		311	=\left=/=\right=, accents, =\text=, the =\math…= fonts, matrices,
		312	=cases= and =aligned=; the full list heads =internal/texmath/texmath.go=).
		313	Anything outside it, including macros, colours and links, shows as
		314	source, as does an expression over 8 KiB or nested more than 64 deep,
		315	and so does all math in a document after its first 1000 expressions or
		316	256 KiB of TeX. A =$$= line opens display math only when a line ending
		317	in =$$= follows before a blank line. MathML typed as raw HTML is
		318	stripped like any other markup the sanitizer does not allow. The CLI
		319	shows the source.
		320
302	* Organizations	321	* Organizations
303		322
304	Orgs share the owner namespace with users and own repositories at	323	Orgs share the owner namespace with users and own repositories at

CHANGELOG.org +7

	@@ -47,6 +47,13 @@ anything beyond "replace the binary and restart" is needed.
47	with ={items, next}=. =dashboard= gains =queries= for pinned ones; the	47	with ={items, next}=. =dashboard= gains =queries= for pinned ones; the
48	web shows them on the dashboard and at =/<you>/-/queries=. An account	48	web shows them on the dashboard and at =/<you>/-/queries=. An account
49	keeps at most 50 saved queries, 10 pinned (#292).	49	keeps at most 50 saved queries, 10 pinned (#292).
		50	- TeX math renders server-side as MathML in markdown (=$…$=, =$$…$$=)
		51	and org (=$…$=, =$…$=, =\[…\]=, LaTeX environments) wherever
		52	markup renders, through a subset converter in =internal/texmath=.
		53	Its output is cleaned by a policy admitting exactly the elements and
		54	attributes it emits; MathML written as raw HTML is still stripped.
		55	Anything outside the subset, and math past 1000 expressions or
		56	256 KiB of TeX in one document, shows as source (#294).
50		57
51	* v1.38.0 — 2026-09-29	58	* v1.38.0 — 2026-09-29
52		59

internal/autolink/autolink.go +2 −2

	@@ -2,7 +2,7 @@
2	// to the repository's issues and merge requests, owner/name#N (and !N)	2	// to the repository's issues and merge requests, owner/name#N (and !N)
3	// across repositories, and @user to owner pages. It operates on the HTML	3	// across repositories, and @user to owner pages. It operates on the HTML
4	// produced by the markdown/org pipeline, walking text nodes with a real	4	// produced by the markdown/org pipeline, walking text nodes with a real
5	// parser so nothing inside <a>, <code>, or <pre> is ever touched, and only	5	// parser so nothing inside <a>, <code>, <pre> or <math> is ever touched, and only
6	// references that actually resolve become links.	6	// references that actually resolve become links.
7	package autolink	7	package autolink
8		8
	@@ -36,7 +36,7 @@ var (
36	)	36	)
37		37
38	// skip lists elements whose text must never be rewritten.	38	// skip lists elements whose text must never be rewritten.
39	var skip = map[string]bool{"a": true, "code": true, "pre": true, "script": true, "style": true}	39	var skip = map[string]bool{"a": true, "code": true, "math": true, "pre": true, "script": true, "style": true}
40		40
41	// Rewrite processes an HTML fragment, linking references relative to	41	// Rewrite processes an HTML fragment, linking references relative to
42	// defaultOwner/defaultName. On any parse failure the input is returned	42	// defaultOwner/defaultName. On any parse failure the input is returned

internal/httpd/math.go added +436

@@ -0,0 +1,436 @@
	1	package httpd
	2
	3	import (
	4	"bytes"
	5	"crypto/rand"
	6	"encoding/hex"
	7	"html/template"
	8	"regexp"
	9	"sort"
	10	"strconv"
	11	"strings"
	12
	13	"github.com/microcosm-cc/bluemonday"
	14	"github.com/niklasfasching/go-org/org"
	15	"github.com/yuin/goldmark"
	16	"github.com/yuin/goldmark/ast"
	17	"github.com/yuin/goldmark/parser"
	18	"github.com/yuin/goldmark/renderer"
	19	"github.com/yuin/goldmark/text"
	20	"github.com/yuin/goldmark/util"
	21
	22	"gitbay.org/gitbay/internal/texmath"
	23	)
	24
	25	// TeX math renders server-side as MathML, which browsers display natively,
	26	// so the page needs no script and the CSP does not change (#294).
	27	//
	28	// The converter is internal/texmath rather than a library. The pure-Go
	29	// options were treeblood (MIT), which writes \color and \class arguments
	30	// into attributes unescaped, expands \def macros without a bound (a 180-byte
	31	// input produced 3 MB), did not finish 5000 nested braces in 30 seconds and
	32	// logs to stderr; goldmark-mathml, which runs Temml in a JavaScript VM; and
	33	// converters inside large typesetting modules. texmath covers a documented
	34	// subset, refuses everything else, and bounds input size and nesting.
	35	//
	36	// MathML reaches a page only from the converter. ugcPolicy, which cleans
	37	// user-authored HTML, admits none of it; mathPolicy admits exactly what
	38	// texmath emits and cleans each converted expression.
	39
	40	// mathPolicy admits the MathML texmath emits: its elements, and each
	41	// attribute only on its element and only with the values it writes.
	42	var mathPolicy = func() *bluemonday.Policy {
	43	p := bluemonday.NewPolicy()
	44	p.AllowElements(texmath.Elements...)
	45	p.AllowNoAttrs().OnElements(texmath.Elements...)
	46	for element, attrs := range texmath.Attrs {
	47	for name, value := range attrs {
	48	pattern := `^` + regexp.QuoteMeta(value) + `$`
	49	if value == "<length>" {
	50	pattern = `^-?[0-9]+(\.[0-9]+)?em$`
	51	}
	52	p.AllowAttrs(name).Matching(regexp.MustCompile(pattern)).OnElements(element)
	53	}
	54	}
	55	return p
	56	}()
	57
	58	// mathHTML renders one expression, or escapes its source when the converter
	59	// refuses it.
	60	func mathHTML(tex, source string, display bool) (string, bool) {
	61	out, err := texmath.Convert(tex, display)
	62	if err != nil {
	63	return template.HTMLEscapeString(source), false
	64	}
	65	return mathPolicy.Sanitize(out), true
	66	}
	67
	68	// Per document, math stops rendering after this many expressions or this
	69	// much TeX; later delimiters stay literal text.
	70	const (
	71	maxMathExprs = 1000
	72	maxMathBytes = 256 << 10
	73	)
	74
	75	type mathBudget struct{ n, bytes int }
	76
	77	func (b *mathBudget) take(size int) bool {
	78	if b.n >= maxMathExprs \|\| b.bytes+size > maxMathBytes {
	79	return false
	80	}
	81	b.n++
	82	b.bytes += size
	83	return true
	84	}
	85
	86	// Markdown: $…$ inline and $$…$$ display, inline or as a block.
	87
	88	var (
	89	kindMath = ast.NewNodeKind("Math")
	90	kindMathBlock = ast.NewNodeKind("MathBlock")
	91
	92	mathBudgetKey = parser.NewContextKey()
	93	mathLineKey = parser.NewContextKey()
	94	)
	95
	96	func budgetFor(pc parser.Context) *mathBudget {
	97	b, _ := pc.Get(mathBudgetKey).(*mathBudget)
	98	if b == nil {
	99	b = &mathBudget{}
	100	pc.Set(mathBudgetKey, b)
	101	}
	102	return b
	103	}
	104
	105	type mathInline struct {
	106	ast.BaseInline
	107	tex string
	108	display bool
	109	}
	110
	111	func (n *mathInline) Kind() ast.NodeKind { return kindMath }
	112	func (n *mathInline) Dump(src []byte, level int) {
	113	ast.DumpHelper(n, src, level, map[string]string{"TeX": n.tex}, nil)
	114	}
	115
	116	type mathBlock struct {
	117	ast.BaseBlock
	118	tex []byte
	119	source []byte
	120	closed bool
	121	}
	122
	123	func (n *mathBlock) Kind() ast.NodeKind { return kindMathBlock }
	124	func (n *mathBlock) Dump(src []byte, level int) {
	125	ast.DumpHelper(n, src, level, map[string]string{"TeX": string(n.tex)}, nil)
	126	}
	127
	128	func isMathSpace(c byte) bool { return c == ' ' \|\| c == '\t' \|\| c == '\n' \|\| c == '\r' }
	129
	130	// mathLine is the valid closing dollars of one line, as source offsets,
	131	// found in one pass so each opener on the line looks its closer up rather
	132	// than rescanning the rest of the line.
	133	type mathLine struct {
	134	stop int
	135	closers []int
	136	}
	137
	138	// closers scans line, which starts at an opening $, for dollars that can
	139	// close inline math: a non-space before, no digit after, not escaped.
	140	func closers(line []byte, base int) []int {
	141	var out []int
	142	for i := 1; i < len(line); i++ {
	143	switch line[i] {
	144	case '\\':
	145	i++
	146	case '$':
	147	if isMathSpace(line[i-1]) \|\| i+1 < len(line) && line[i+1] >= '0' && line[i+1] <= '9' {
	148	continue
	149	}
	150	out = append(out, base+i)
	151	}
	152	}
	153	return out
	154	}
	155
	156	// mathInlineParser follows pandoc's rule so prices stay prose: the opening
	157	// $ has a non-space after it, and the closing $ a non-space before it and no
	158	// digit after it. "$5 and $10" is text. A backslash escapes the next byte.
	159	type mathInlineParser struct{}
	160
	161	func (mathInlineParser) Trigger() []byte { return []byte{'$'} }
	162
	163	func (mathInlineParser) Parse(parent ast.Node, block text.Reader, pc parser.Context) ast.Node {
	164	line, seg := block.PeekLine()
	165	if len(line) >= 2 && line[1] == '$' {
	166	// Scanning stops at the next $$, where the next attempt starts,
	167	// so each byte is scanned at most twice.
	168	body := line[2:]
	169	for i := 0; i+1 < len(body); i++ {
	170	if body[i] == '\\' {
	171	i++
	172	continue
	173	}
	174	if body[i] == '$' && body[i+1] == '$' {
	175	if i == 0 \|\| !budgetFor(pc).take(i) {
	176	break
	177	}
	178	block.Advance(i + 4)
	179	return &mathInline{tex: string(body[:i]), display: true}
	180	}
	181	}
	182	// Consume both dollars so the second does not open inline math.
	183	block.Advance(2)
	184	return ast.NewTextSegment(seg.WithStop(seg.Start + 2))
	185	}
	186	if len(line) < 2 \|\| isMathSpace(line[1]) {
	187	return nil
	188	}
	189	start := seg.Start - seg.Padding
	190	cache, _ := pc.Get(mathLineKey).(*mathLine)
	191	if cache == nil \|\| cache.stop != seg.Stop {
	192	cache = &mathLine{stop: seg.Stop, closers: closers(line, start)}
	193	pc.Set(mathLineKey, cache)
	194	}
	195	k := sort.SearchInts(cache.closers, start+2)
	196	if k == len(cache.closers) {
	197	return nil
	198	}
	199	end := cache.closers[k] - start
	200	if !budgetFor(pc).take(end - 1) {
	201	return nil
	202	}
	203	block.Advance(end + 1)
	204	return &mathInline{tex: string(line[1:end])}
	205	}
	206
	207	// mathBlockParser opens on a line that is $$ alone, when a line ending in
	208	// $$ follows before a blank line, or on a line that is $$…$$ whole.
	209	// Anything else stays paragraph text.
	210	type mathBlockParser struct{}
	211
	212	func (mathBlockParser) Trigger() []byte { return []byte{'$'} }
	213
	214	func (mathBlockParser) Open(parent ast.Node, reader text.Reader, pc parser.Context) (ast.Node, parser.State) {
	215	line, seg := reader.PeekLine()
	216	pos := pc.BlockOffset()
	217	if pos < 0 \|\| !bytes.HasPrefix(line[pos:], []byte("$$")) {
	218	return nil, parser.NoChildren
	219	}
	220	rest := util.TrimRightSpace(line[pos+2:])
	221	n := &mathBlock{}
	222	switch {
	223	case len(rest) == 0:
	224	// The lookahead stops at the first line ending in $$, which the
	225	// block then consumes, so no line is looked at twice by it.
	226	// It reads the source rather than moving the reader, whose
	227	// SetPosition keeps the line it last peeked.
	228	size, found := 0, false
	229	src := reader.Source()
	230	for i := seg.Stop; i < len(src); {
	231	end := len(src)
	232	if j := bytes.IndexByte(src[i:], '\n'); j >= 0 {
	233	end = i + j + 1
	234	}
	235	next := src[i:end]
	236	if util.IsBlank(next) {
	237	break
	238	}
	239	size += len(next)
	240	if bytes.HasSuffix(util.TrimRightSpace(next), []byte("$$")) {
	241	found = true
	242	break
	243	}
	244	i = end
	245	}
	246	if !found \|\| !budgetFor(pc).take(size) {
	247	return nil, parser.NoChildren
	248	}
	249	n.source = append(n.source, "$$\n"...)
	250	case len(rest) > 2 && bytes.HasSuffix(rest, []byte("$$")):
	251	if !budgetFor(pc).take(len(rest) - 2) {
	252	return nil, parser.NoChildren
	253	}
	254	n.tex, n.closed = rest[:len(rest)-2], true
	255	n.source = append([]byte("$$"), rest...)
	256	default:
	257	return nil, parser.NoChildren
	258	}
	259	reader.AdvanceToEOL()
	260	return n, parser.NoChildren
	261	}
	262
	263	func (mathBlockParser) Continue(node ast.Node, reader text.Reader, pc parser.Context) parser.State {
	264	n := node.(*mathBlock)
	265	if n.closed {
	266	return parser.Close
	267	}
	268	line, _ := reader.PeekLine()
	269	if line == nil \|\| util.IsBlank(line) {
	270	return parser.Close
	271	}
	272	trimmed := util.TrimRightSpace(line)
	273	if bytes.HasSuffix(trimmed, []byte("$$")) {
	274	n.tex = append(n.tex, trimmed[:len(trimmed)-2]...)
	275	n.source = append(n.source, trimmed...)
	276	n.closed = true
	277	reader.AdvanceToEOL()
	278	return parser.Close
	279	}
	280	n.tex = append(n.tex, line...)
	281	n.source = append(n.source, line...)
	282	reader.AdvanceToEOL()
	283	return parser.Continue \| parser.NoChildren
	284	}
	285
	286	func (mathBlockParser) Close(ast.Node, text.Reader, parser.Context) {}
	287	func (mathBlockParser) CanInterruptParagraph() bool { return true }
	288	func (mathBlockParser) CanAcceptIndentedLine() bool { return false }
	289
	290	type mathRenderer struct{}
	291
	292	func (mathRenderer) RegisterFuncs(reg renderer.NodeRendererFuncRegisterer) {
	293	reg.Register(kindMath, func(w util.BufWriter, _ []byte, node ast.Node, entering bool) (ast.WalkStatus, error) {
	294	if entering {
	295	n := node.(*mathInline)
	296	delim := "$"
	297	if n.display {
	298	delim = "$$"
	299	}
	300	out, _ := mathHTML(n.tex, delim+n.tex+delim, n.display)
	301	_, _ = w.WriteString(out)
	302	}
	303	return ast.WalkSkipChildren, nil
	304	})
	305	reg.Register(kindMathBlock, func(w util.BufWriter, _ []byte, node ast.Node, entering bool) (ast.WalkStatus, error) {
	306	if !entering {
	307	return ast.WalkContinue, nil
	308	}
	309	n := node.(*mathBlock)
	310	source := string(n.source)
	311	out, ok := mathHTML(string(n.tex), source, true)
	312	if !ok \|\| !n.closed {
	313	// A container that ended before the closing line leaves the
	314	// block unclosed; show what it held.
	315	out = "<pre>" + template.HTMLEscapeString(source) + "</pre>"
	316	}
	317	_, _ = w.WriteString(out + "\n")
	318	return ast.WalkSkipChildren, nil
	319	})
	320	}
	321
	322	type mathExtension struct{}
	323
	324	func (mathExtension) Extend(m goldmark.Markdown) {
	325	m.Parser().AddOptions(
	326	parser.WithBlockParsers(util.Prioritized(mathBlockParser{}, 701)),
	327	parser.WithInlineParsers(util.Prioritized(mathInlineParser{}, 501)))
	328	m.Renderer().AddOptions(renderer.WithNodeRenderers(util.Prioritized(mathRenderer{}, 500)))
	329	}
	330
	331	// Org: go-org already parses $…$, $$…$$, $…$, \[…\] and \begin{…}…\end{…}
	332	// as LaTeX fragments, and \begin{…} on its own lines as a LaTeX block; it
	333	// writes them back out as text. These render them instead.
	334	//
	335	// The whole org document goes through ugcPolicy, which admits no MathML, so
	336	// the writer leaves a placeholder for each expression and fill puts the
	337	// mathPolicy-cleaned MathML back after sanitizing. The placeholder carries a
	338	// random per-render prefix, so a document cannot spell one.
	339
	340	type mathSlots struct {
	341	prefix string
	342	html []string
	343	source []string
	344	budget mathBudget
	345	}
	346
	347	func newMathSlots() *mathSlots {
	348	var b [12]byte
	349	_, _ = rand.Read(b[:])
	350	return &mathSlots{prefix: "gitbaymath" + hex.EncodeToString(b[:]) + "n"}
	351	}
	352
	353	func (m *mathSlots) put(html, source string) string {
	354	m.html = append(m.html, html)
	355	m.source = append(m.source, source)
	356	return m.prefix + strconv.Itoa(len(m.html)-1) + "z"
	357	}
	358
	359	// fill replaces each placeholder in sanitized HTML with its MathML, or with
	360	// its escaped source where the placeholder landed inside a tag (an
	361	// attribute value). Sanitized output escapes < and > everywhere but in
	362	// tags, so the last of them seen says whether the text is inside one.
	363	func (m *mathSlots) fill(doc string) string {
	364	if len(m.html) == 0 {
	365	return doc
	366	}
	367	var b strings.Builder
	368	inTag := false
	369	for {
	370	i := strings.Index(doc, m.prefix)
	371	if i < 0 {
	372	b.WriteString(doc)
	373	return b.String()
	374	}
	375	before := doc[:i]
	376	if j := strings.LastIndexAny(before, "<>"); j >= 0 {
	377	inTag = before[j] == '<'
	378	}
	379	b.WriteString(before)
	380	doc = doc[i+len(m.prefix):]
	381	end := strings.IndexByte(doc, 'z')
	382	if end < 0 {
	383	b.WriteString(m.prefix)
	384	continue
	385	}
	386	k, err := strconv.Atoi(doc[:end])
	387	if err != nil \|\| k < 0 \|\| k >= len(m.html) {
	388	b.WriteString(m.prefix)
	389	continue
	390	}
	391	doc = doc[end+1:]
	392	if inTag {
	393	b.WriteString(template.HTMLEscapeString(m.source[k]))
	394	} else {
	395	b.WriteString(m.html[k])
	396	}
	397	}
	398	}
	399
	400	// writeMath writes an expression's placeholder, or its source as text when
	401	// the converter refuses it or the document's budget is spent.
	402	func (w *orgWriter) writeMath(tex, source string, display bool) {
	403	if w.math.budget.take(len(tex)) {
	404	if out, ok := mathHTML(tex, source, display); ok {
	405	w.WriteString(w.math.put(out, source))
	406	return
	407	}
	408	}
	409	w.WriteText(org.Text{Content: source, IsRaw: true})
	410	}
	411
	412	func (w *orgWriter) WriteLatexFragment(l org.LatexFragment) {
	413	tex := org.String(l.Content...)
	414	source := l.OpeningPair + tex + l.ClosingPair
	415	// go-org takes any $…$; org's own rule keeps "$5 and $10" prose.
	416	if l.OpeningPair == "$" && (tex == "" \|\| isMathSpace(tex[0]) \|\| isMathSpace(tex[len(tex)-1])) {
	417	w.WriteText(org.Text{Content: source, IsRaw: true})
	418	return
	419	}
	420	display := l.OpeningPair != "$" && l.OpeningPair != `\(`
	421	if strings.HasPrefix(l.OpeningPair, `\begin{`) {
	422	tex = source
	423	}
	424	w.writeMath(tex, source, display)
	425	}
	426
	427	func (w *orgWriter) WriteLatexBlock(b org.LatexBlock) {
	428	tex := org.String(b.Content...)
	429	if w.math.budget.take(len(tex)) {
	430	if out, ok := mathHTML(tex, tex, true); ok {
	431	w.WriteString(w.math.put(out, tex) + "\n")
	432	return
	433	}
	434	}
	435	w.WriteString("<pre>" + template.HTMLEscapeString(tex) + "</pre>\n")
	436	}

internal/httpd/math_test.go added +259

@@ -0,0 +1,259 @@
	1	package httpd
	2
	3	import (
	4	"net/http/httptest"
	5	"regexp"
	6	"strings"
	7	"testing"
	8	"time"
	9
	10	"gitbay.org/gitbay/internal/config"
	11	"gitbay.org/gitbay/internal/store"
	12	)
	13
	14	func TestMarkdownMath(t *testing.T) {
	15	cases := []struct {
	16	name, src string
	17	want []string
	18	not []string
	19	}{
	20	{"inline", "Euler: $e^{i\\pi}+1=0$.", []string{`<math><msup><mi>e</mi>`, `</math>.`}, nil},
	21	{"display inline", "so $$x^2$$ here", []string{`<math display="block"><msup>`}, nil},
	22	{"block", "text\n$$\n\\frac{a}{b}\n$$\nafter\n", []string{`<math display="block"><mfrac>`, `<p>after</p>`}, []string{"$$"}},
	23	{"one-line block", "$$x_1$$\n", []string{`<math display="block"><msub>`}, []string{"<p>"}},
	24	{"prices", "costs $5 and $10 today", []string{"costs $5 and $10 today"}, []string{"<math"}},
	25	{"space after open", "a $ x$ b", []string{"a $ x$ b"}, []string{"<math"}},
	26	{"space before close", "a $x $ b", []string{"a $x $ b"}, []string{"<math"}},
	27	{"digit after close", "$x$5", []string{"$x$5"}, []string{"<math"}},
	28	{"escaped dollars", `\$x\$`, []string{"$x$"}, []string{"<math"}},
	29	{"escaped dollar inside", `$a\$b$`, []string{`<mo>$</mo>`}, nil},
	30	{"code span", "`$x^2$`", []string{"<code>$x^2$</code>"}, []string{"<math"}},
	31	{"fenced code", "```\n$x^2$\n$$\ny\n$$\n```\n", []string{"$x^2$", "$$\ny\n$$"}, []string{"<math"}},
	32	{"indented code", " $x$\n", []string{"$x$"}, []string{"<math"}},
	33	{"invalid inline", `see $\frac{a$ here`, []string{`see $\frac{a$ here`}, []string{"<math"}},
	34	{"invalid block", "$$\n\\frac{\n$$\n", []string{"<pre tabindex=\"0\">$$\n\\frac{\n$$</pre>"}, []string{"<math"}},
	35	{"unclosed block", "$$\nx\n", []string{"<p>$$\nx</p>"}, []string{"<math", "<pre"}},
	36	{"blank line ends block", "$$\nx\n\ny $$\n", []string{"<p>$$\nx</p>", "<p>y $$</p>"}, []string{"<math", "<pre"}},
	37	{"block keeps source", "$$\na\nb $$\n", []string{`<math display="block"><mi>a</mi><mi>b</mi></math>`}, nil},
	38	{"markup is escaped", "$\\text{<b>&</b>}$", []string{`<mtext><b>&</b></mtext>`}, []string{"<b>"}},
	39	}
	40	for _, c := range cases {
	41	out := string(ugcHTML(c.src, "md"))
	42	for _, w := range c.want {
	43	if !strings.Contains(out, w) {
	44	t.Errorf("%s: lacks %q:\n%s", c.name, w, out)
	45	}
	46	}
	47	for _, w := range c.not {
	48	if strings.Contains(out, w) {
	49	t.Errorf("%s: has %q:\n%s", c.name, w, out)
	50	}
	51	}
	52	}
	53	}
	54
	55	func TestOrgMath(t *testing.T) {
	56	cases := []struct {
	57	name, src string
	58	want []string
	59	not []string
	60	}{
	61	{"dollar", "Euler: $e^x$ here", []string{`<math><msup><mi>e</mi><mi>x</mi></msup></math> here`}, nil},
	62	{"paren", `a $x_1$ b`, []string{`<math><msub>`}, []string{`\(`}},
	63	{"bracket", `a \[x^2\] b`, []string{`<math display="block"><msup>`}, []string{`\[`}},
	64	{"double dollar", `a $$x$$ b`, []string{`<math display="block"><mi>x</mi></math>`}, nil},
	65	{"environment block", "\\begin{equation}\nx = \\frac{1}{2}\n\\end{equation}\n", []string{`<math display="block"><mrow><mi>x</mi><mo>=</mo><mfrac>`}, []string{`\begin`}},
	66	{"matrix block", "\\begin{pmatrix}\na & b \\\\\nc & d\n\\end{pmatrix}\n", []string{`<mtable><mtr><mtd><mi>a</mi></mtd>`}, nil},
	67	{"prices", "costs $5 and $10 today", []string{"costs $5 and $10 today"}, []string{"<math"}},
	68	{"verbatim", "=$x^2$= and ~$y$~", []string{"<code>$x^2$</code>", "<code>$y$</code>"}, []string{"<math"}},
	69	{"src block", "#+begin_src tex\n$x^2$\n#+end_src\n", []string{"<pre"}, []string{"<math"}},
	70	{"invalid", `see $\frac{a$ here`, []string{`see $\frac{a$ here`}, []string{"<math"}},
	71	{"invalid block", "\\begin{tabular}\nx\n\\end{tabular}\n", []string{`<pre tabindex="0">\begin{tabular}`}, []string{"<math"}},
	72	}
	73	for _, c := range cases {
	74	out := string(ugcHTML(c.src, "org"))
	75	for _, w := range c.want {
	76	if !strings.Contains(out, w) {
	77	t.Errorf("%s: lacks %q:\n%s", c.name, w, out)
	78	}
	79	}
	80	for _, w := range c.not {
	81	if strings.Contains(out, w) {
	82	t.Errorf("%s: has %q:\n%s", c.name, w, out)
	83	}
	84	}
	85	}
	86	}
	87
	88	// mathPolicy admits the MathML texmath writes and nothing else.
	89	func TestMathPolicy(t *testing.T) {
	90	hostile := `<math display="block" xmlns:xlink="http://www.w3.org/1999/xlink" style="x" onclick="x()">` +
	91	`<mi href="javascript:alert(1)" xlink:href="javascript:alert(2)" mathvariant="bold" style="color:red" onmouseover="x()">x</mi>` +
	92	`<mo stretchy="true" form="prefix">(</mo><mspace width="expression(alert(3))"></mspace><mspace width="1em"></mspace>` +
	93	`<semantics><annotation-xml encoding="text/html"><img src=x onerror="alert(4)"></annotation-xml></semantics>` +
	94	`<maction actiontype="statusline"><mi>y</mi></maction><mstyle mathcolor="red"><mi>z</mi></mstyle>` +
	95	`<mtext><style>*{}</style><script>alert(5)</script></mtext></math><math display="inline"></math>`
	96	out := mathPolicy.Sanitize(hostile)
	97	for _, bad := range []string{"href", "xlink", "style", "onclick", "onmouseover", "onerror", "javascript",
	98	"annotation", "semantics", "maction", "mstyle", "mathcolor", "script", "expression",
	99	`mathvariant="bold"`, `stretchy="true"`, "form=", `display="inline"`} {
	100	if strings.Contains(out, bad) {
	101	t.Errorf("sanitized MathML keeps %q:\n%s", bad, out)
	102	}
	103	}
	104	for _, good := range []string{`<math display="block">`, `<mspace width="1em">`, "<mi>x</mi>", "<mi>y</mi>"} {
	105	if !strings.Contains(out, good) {
	106	t.Errorf("sanitized MathML lacks %q:\n%s", good, out)
	107	}
	108	}
	109	}
	110
	111	// Hostile TeX is refused and shown as escaped source, in bounded time and
	112	// size, on both syntaxes.
	113	func TestHostileMath(t *testing.T) {
	114	long := strings.Repeat(`x+`, 1<<19) // 1 MiB in one expression
	115	deep := strings.Repeat("{", 50000) + "x" + strings.Repeat("}", 50000)
	116	for _, tex := range []string{
	117	`\href{javascript:alert(1)}{x}`, `\url{javascript:alert(1)}`, `\style{color:red}{x}`,
	118	`\color{red" onmouseover="alert(1)}{x}`, `\class{a"b}{x}`, `\def\a{\a\a}\a`,
	119	`\text{</math><script>alert(1)</script>}`, `\text{<img src=x onerror=alert(1)>}`,
	120	deep, long, strings.Repeat(`\sqrt{`, 10000) + "x",
	121	} {
	122	for _, doc := range []struct{ src, format string }{
	123	{"$" + tex + "$", "md"}, {"$$\n" + tex + "\n$$\n", "md"},
	124	{`$` + tex + `$`, "org"}, {"\\[" + tex + "\\]", "org"},
	125	} {
	126	start := time.Now()
	127	out := string(ugcHTML(doc.src, doc.format))
	128	if d := time.Since(start); d > 2*time.Second {
	129	t.Errorf("%.30q (%s) took %v", tex, doc.format, d)
	130	}
	131	if len(out) > 8*len(doc.src)+1024 {
	132	t.Errorf("%.30q (%s): %d bytes out for %d in", tex, doc.format, len(out), len(doc.src))
	133	}
	134	for _, bad := range []string{"<script", "<img", `href="`, `onmouseover="`, `style="`, `class="a`} {
	135	if strings.Contains(out, bad) {
	136	t.Errorf("%.30q (%s) emits %q:\n%.300s", tex, doc.format, bad, out)
	137	}
	138	}
	139	}
	140	}
	141	// A refused \text keeps its payload as visible text, escaped.
	142	out := string(ugcHTML(`$\href{javascript:alert(1)}{x}$`, "md"))
	143	if !strings.Contains(out, `$\href{javascript:alert(1)}{x}$`) \|\| strings.Contains(out, "<math") \|\| strings.Contains(out, "<a") {
	144	t.Errorf("refused \\href: %s", out)
	145	}
	146	}
	147
	148	// The issue page renders its body's math through the shared path, and
	149	// autolinking leaves text inside <math> alone.
	150	func TestIssuePageRendersMath(t *testing.T) {
	151	st, err := store.Open(":memory:")
	152	if err != nil {
	153	t.Fatal(err)
	154	}
	155	defer st.Close()
	156	if err := st.MigrateUp(); err != nil {
	157	t.Fatal(err)
	158	}
	159	uid, err := st.CreateUser("alice", false)
	160	if err != nil {
	161	t.Fatal(err)
	162	}
	163	repoID, err := st.CreateRepo("user", uid, "app", "public")
	164	if err != nil {
	165	t.Fatal(err)
	166	}
	167	if _, err := st.CreateIssue(repoID, uid, "math", `Area is $\pi r^2$, see $\text{#1}$.`, "md"); err != nil {
	168	t.Fatal(err)
	169	}
	170	cfg := config.Default()
	171	cfg.Web.Mode = "accounts"
	172	s := New(cfg, st, nil)
	173	rr := httptest.NewRecorder()
	174	s.Handler().ServeHTTP(rr, httptest.NewRequest("GET", "/alice/app/issues/1", nil))
	175	if rr.Code != 200 {
	176	t.Fatalf("status %d", rr.Code)
	177	}
	178	body := rr.Body.String()
	179	if !strings.Contains(body, `<math><mi>π</mi><msup><mi>r</mi><mn>2</mn></msup></math>`) {
	180	t.Errorf("no MathML in the issue page:\n%s", body)
	181	}
	182	if !strings.Contains(body, `<mtext>#1</mtext>`) {
	183	t.Errorf("autolink rewrote text inside <math>:\n%s", body)
	184	}
	185	}
	186
	187	// MathML written by hand is user HTML, and ugcPolicy strips it: only the
	188	// converter's output, through mathPolicy, reaches the page.
	189	func TestRawMathMLStripped(t *testing.T) {
	190	raw := `<math display="block"><mi>q</mi></math>`
	191	for name, out := range map[string]string{
	192	"html readme": string(renderReadme("README.html", []byte(raw))),
	193	"markdown html": string(renderReadme("README.md", []byte("para "+raw+"\n\n"+raw+"\n"))),
	194	"org export": string(renderReadme("README.org", []byte("#+begin_export html\n"+raw+"\n#+end_export\n"))),
	195	"org inline": string(renderReadme("README.org", []byte("@@html:"+raw+"@@\n"))),
	196	} {
	197	if strings.Contains(out, "<math") \|\| strings.Contains(out, "<mi>") {
	198	t.Errorf("%s keeps raw MathML:\n%s", name, out)
	199	}
	200	}
	201	for name, out := range map[string]string{
	202	"markdown": string(renderReadme("README.md", []byte("$q$\n"))),
	203	"org": string(renderReadme("README.org", []byte("$q$\n"))),
	204	} {
	205	if !strings.Contains(out, "<math><mi>q</mi></math>") {
	206	t.Errorf("%s: no MathML:\n%s", name, out)
	207	}
	208	}
	209	}
	210
	211	// Org placeholders: a document cannot spell one, and one that lands in an
	212	// attribute is filled with escaped source, not markup.
	213	func TestMathSlots(t *testing.T) {
	214	out := string(ugcHTML("gitbaymath0z $x$ gitbaymath00000000000000000000000000n0z", "org"))
	215	if strings.Count(out, "<math>") != 1 \|\| !strings.Contains(out, "gitbaymath0z") {
	216	t.Errorf("forged placeholder: %s", out)
	217	}
	218	m := newMathSlots()
	219	a := m.put(`<math><mi>x</mi></math>`, `$x" onmouseover="y$`)
	220	b := m.put(`<math><mi>y</mi></math>`, `$y$`)
	221	got := m.fill(`<a title="` + a + `">` + b + `</a>`)
	222	want := `<a title="$x" onmouseover="y$"><math><mi>y</mi></math></a>`
	223	if got != want {
	224	t.Errorf("fill:\n got %s\nwant %s", got, want)
	225	}
	226	if other := newMathSlots(); other.prefix == m.prefix {
	227	t.Error("placeholder prefix repeats across renders")
	228	}
	229	}
	230
	231	// Many openers without closers on one line scan in linear time, and the
	232	// per-document budget leaves later math as text.
	233	func TestMathLinear(t *testing.T) {
	234	for _, src := range []string{
	235	strings.Repeat("$a ", 1<<20/3),
	236	strings.Repeat("$$a ", 1<<20/4),
	237	strings.Repeat("$$\na\n", 1<<20/5),
	238	} {
	239	for _, format := range []string{"md", "org"} {
	240	start := time.Now()
	241	ugcHTML(src, format)
	242	if d := time.Since(start); d > 2*time.Second {
	243	t.Errorf("%s: %.12q x %d took %v", format, src[:4], len(src), d)
	244	}
	245	}
	246	}
	247	src := strings.Repeat("$x$ ", maxMathExprs+10)
	248	out := string(ugcHTML(src, "md"))
	249	if n := strings.Count(out, "<math>"); n != maxMathExprs {
	250	t.Errorf("markdown rendered %d expressions, budget %d", n, maxMathExprs)
	251	}
	252	out = string(ugcHTML(src, "org"))
	253	if n := strings.Count(out, "<math>"); n != maxMathExprs {
	254	t.Errorf("org rendered %d expressions, budget %d", n, maxMathExprs)
	255	}
	256	if regexp.MustCompile(`gitbaymath[0-9a-f]+n`).MatchString(out) {
	257	t.Error("a placeholder survived")
	258	}
	259	}

internal/httpd/web.go +9 −6

 // markdown is the shared renderer: GFM (tables, strikethrough, autolinks,
 // task lists) on top of CommonMark, with class-based fence highlighting
-// (the palette lives in the stylesheet, per scheme). Raw HTML is still
+// (the palette lives in the stylesheet, per scheme), and TeX math as
-// dropped.
+// MathML (math.go). Raw HTML is still dropped.
 // Headings carry ids so a README or wiki section can be linked to, the
 // way org headings already are (#132).
 var markdown = goldmark.New(
 	goldmark.WithParserOptions(parser.WithAutoHeadingID()),
-	goldmark.WithExtensions(extension.GFM,
+	goldmark.WithExtensions(extension.GFM, mathExtension{},
 		highlighting.NewHighlighting(highlighting.WithFormatOptions(html.WithClasses(true)))))
 // fenceHighlight renders one code block with chroma classes, for org and
 		}
 		return fenceHighlight(source, lang)
 	}
-	writer.ExtendingWriter = &orgWriter{writer}
+	ow := &orgWriter{HTMLWriter: writer, math: newMathSlots()}
+	writer.ExtendingWriter = ow
 	out, err := doc.Write(writer)
 	if err != nil {
 		return fallback()
 	}
-	return imageAlt(template.HTML(ugcPolicy.Sanitize(out)))
+	return imageAlt(template.HTML(ow.math.fill(ugcPolicy.Sanitize(out))))
 }
 // orgWriter overrides go-org's autolink rendering. go-org ends a bare URL
 // at the first character outside RFC 3986's set, and that set includes
 // `.`, `,` and `)`, so a URL closing a sentence or a parenthesis took the
 // punctuation with it. Org stops a plain link before trailing punctuation
-// and keeps a `)` only when a `(` inside the link opened it.
+// and keeps a `)` only when a `(` inside the link opened it. It also
+// renders LaTeX fragments and blocks (math.go).
 type orgWriter struct {
 	*org.HTMLWriter
+	math *mathSlots
 }
 func (w *orgWriter) WriteRegularLink(l org.RegularLink) {

internal/texmath/symbols.go added +184

@@ -0,0 +1,184 @@
	1	package texmath
	2
	3	type sym struct {
	4	tag string
	5	text string
	6	upright bool // mathvariant="normal": TeX sets capital Greek upright
	7	fixed bool // stretchy="false": a bracket typed without \left
	8	limits bool
	9	}
	10
	11	func mi(s string) sym { return sym{tag: "mi", text: s} }
	12	func mo(s string) sym { return sym{tag: "mo", text: s} }
	13	func upright(s string) sym { return sym{tag: "mi", text: s, upright: true} }
	14	func bracket(s string) sym { return sym{tag: "mo", text: s, fixed: true} }
	15	func large(s string) sym { return sym{tag: "mo", text: s, limits: true} }
	16
	17	var symbols = map[string]sym{
	18	// Greek
	19	"alpha": mi("α"), "beta": mi("β"), "gamma": mi("γ"), "delta": mi("δ"),
	20	"epsilon": mi("ϵ"), "varepsilon": mi("ε"), "zeta": mi("ζ"), "eta": mi("η"),
	21	"theta": mi("θ"), "vartheta": mi("ϑ"), "iota": mi("ι"), "kappa": mi("κ"),
	22	"lambda": mi("λ"), "mu": mi("μ"), "nu": mi("ν"), "xi": mi("ξ"),
	23	"omicron": mi("ο"), "pi": mi("π"), "varpi": mi("ϖ"), "rho": mi("ρ"),
	24	"varrho": mi("ϱ"), "sigma": mi("σ"), "varsigma": mi("ς"), "tau": mi("τ"),
	25	"upsilon": mi("υ"), "phi": mi("ϕ"), "varphi": mi("φ"), "chi": mi("χ"),
	26	"psi": mi("ψ"), "omega": mi("ω"),
	27	"Gamma": upright("Γ"), "Delta": upright("Δ"), "Theta": upright("Θ"),
	28	"Lambda": upright("Λ"), "Xi": upright("Ξ"), "Pi": upright("Π"),
	29	"Sigma": upright("Σ"), "Upsilon": upright("Υ"), "Phi": upright("Φ"),
	30	"Psi": upright("Ψ"), "Omega": upright("Ω"),
	31
	32	// Letter-like
	33	"infty": mi("∞"), "partial": mi("∂"), "nabla": mi("∇"), "emptyset": mi("∅"),
	34	"varnothing": mi("∅"), "hbar": mi("ℏ"), "ell": mi("ℓ"), "Re": mi("ℜ"),
	35	"Im": mi("ℑ"), "aleph": mi("ℵ"), "wp": mi("℘"), "imath": mi("ı"), "jmath": mi("ȷ"),
	36
	37	// Binary operators
	38	"pm": mo("±"), "mp": mo("∓"), "times": mo("×"), "div": mo("÷"),
	39	"cdot": mo("⋅"), "ast": mo("∗"), "star": mo("⋆"), "circ": mo("∘"),
	40	"bullet": mo("∙"), "cap": mo("∩"), "cup": mo("∪"), "setminus": mo("∖"),
	41	"wedge": mo("∧"), "land": mo("∧"), "vee": mo("∨"), "lor": mo("∨"),
	42	"oplus": mo("⊕"), "ominus": mo("⊖"), "otimes": mo("⊗"), "odot": mo("⊙"),
	43	"sqcup": mo("⊔"), "sqcap": mo("⊓"), "dagger": mo("†"), "ddagger": mo("‡"),
	44	"lnot": mo("¬"), "neg": mo("¬"),
	45
	46	// Relations
	47	"le": mo("≤"), "leq": mo("≤"), "ge": mo("≥"), "geq": mo("≥"),
	48	"ne": mo("≠"), "neq": mo("≠"), "ll": mo("≪"), "gg": mo("≫"),
	49	"approx": mo("≈"), "equiv": mo("≡"), "sim": mo("∼"), "simeq": mo("≃"),
	50	"cong": mo("≅"), "propto": mo("∝"), "in": mo("∈"), "notin": mo("∉"),
	51	"ni": mo("∋"), "subset": mo("⊂"), "supset": mo("⊃"), "subseteq": mo("⊆"),
	52	"supseteq": mo("⊇"), "perp": mo("⊥"), "parallel": mo("∥"), "mid": mo("∣"),
	53	"vdash": mo("⊢"), "models": mo("⊨"), "coloneqq": mo("≔"), "prec": mo("≺"),
	54	"succ": mo("≻"), "preceq": mo("⪯"), "succeq": mo("⪰"), "doteq": mo("≐"),
	55
	56	// Arrows
	57	"to": mo("→"), "rightarrow": mo("→"), "leftarrow": mo("←"), "gets": mo("←"),
	58	"leftrightarrow": mo("↔"), "Rightarrow": mo("⇒"), "Leftarrow": mo("⇐"),
	59	"Leftrightarrow": mo("⇔"), "implies": mo("⟹"), "impliedby": mo("⟸"),
	60	"iff": mo("⟺"), "mapsto": mo("↦"), "longrightarrow": mo("⟶"),
	61	"longleftarrow": mo("⟵"), "Longrightarrow": mo("⟹"), "uparrow": mo("↑"),
	62	"downarrow": mo("↓"), "hookrightarrow": mo("↪"), "rightharpoonup": mo("⇀"),
	63
	64	// Logic and sets
	65	"forall": mo("∀"), "exists": mo("∃"), "nexists": mo("∄"), "therefore": mo("∴"),
	66	"because": mo("∵"), "top": mo("⊤"), "bot": mo("⊥"), "angle": mo("∠"),
	67	"prime": mo("′"), "degree": mo("°"),
	68
	69	// Dots
	70	"ldots": mo("…"), "dots": mo("…"), "cdots": mo("⋯"), "vdots": mo("⋮"),
	71	"ddots": mo("⋱"),
	72
	73	// Brackets typed without \left
	74	"{": bracket("{"), "}": bracket("}"), "\|": bracket("‖"),
	75	"langle": bracket("⟨"), "rangle": bracket("⟩"), "lvert": bracket("\|"),
	76	"rvert": bracket("\|"), "lVert": bracket("‖"), "rVert": bracket("‖"),
	77	"vert": bracket("\|"), "Vert": bracket("‖"), "lfloor": bracket("⌊"),
	78	"rfloor": bracket("⌋"), "lceil": bracket("⌈"), "rceil": bracket("⌉"),
	79	"lbrace": bracket("{"), "rbrace": bracket("}"), "backslash": mo("\\"),
	80
	81	// Escaped characters
	82	"%": mo("%"), "$": mo("$"), "#": mo("#"), "&": mo("&"), "_": mo("_"),
	83
	84	// Large operators
	85	"sum": large("∑"), "prod": large("∏"), "coprod": large("∐"),
	86	"bigcup": large("⋃"), "bigcap": large("⋂"), "bigvee": large("⋁"),
	87	"bigwedge": large("⋀"), "bigoplus": large("⨁"), "bigotimes": large("⨂"),
	88	"bigsqcup": large("⨆"),
	89	"int": mo("∫"), "iint": mo("∬"), "iiint": mo("∭"), "oint": mo("∮"),
	90	}
	91
	92	// spaces are the spacing commands and their widths.
	93	var spaces = map[string]string{
	94	",": "0.1667em", "thinspace": "0.1667em",
	95	":": "0.2222em", ">": "0.2222em", "medspace": "0.2222em",
	96	";": "0.2778em", "thickspace": "0.2778em",
	97	"!": "-0.1667em", "negthinspace": "-0.1667em",
	98	" ": "0.3333em", "enspace": "0.5em",
	99	"quad": "1em", "qquad": "2em",
	100	}
	101
	102	// functions are the named functions, set upright; true takes limits under
	103	// and over in display style.
	104	var functions = map[string]bool{
	105	"sin": false, "cos": false, "tan": false, "cot": false, "sec": false, "csc": false,
	106	"arcsin": false, "arccos": false, "arctan": false, "sinh": false, "cosh": false,
	107	"tanh": false, "coth": false, "log": false, "ln": false, "lg": false, "exp": false,
	108	"dim": false, "deg": false, "hom": false, "ker": false, "arg": false,
	109	"lim": true, "liminf": true, "limsup": true, "max": true, "min": true,
	110	"sup": true, "inf": true, "det": true, "gcd": true, "Pr": true,
	111	}
	112
	113	type accent struct {
	114	mark string
	115	accent bool // mover accent="true": the mark sits close, like an accent
	116	under bool
	117	}
	118
	119	var accents = map[string]accent{
	120	"hat": {mark: "^", accent: true}, "widehat": {mark: "^", accent: true},
	121	"tilde": {mark: "~", accent: true}, "widetilde": {mark: "~", accent: true},
	122	"bar": {mark: "¯", accent: true}, "overline": {mark: "‾", accent: true},
	123	"vec": {mark: "→", accent: true}, "overrightarrow": {mark: "→", accent: true},
	124	"dot": {mark: "˙", accent: true}, "ddot": {mark: "¨", accent: true},
	125	"acute": {mark: "´", accent: true}, "grave": {mark: "`", accent: true},
	126	"breve": {mark: "˘", accent: true}, "check": {mark: "ˇ", accent: true},
	127	"overbrace": {mark: "⏞"},
	128	"underline": {mark: "_", under: true},
	129	"underbrace": {mark: "⏟", under: true},
	130	}
	131
	132	var fonts = map[string]string{
	133	"mathrm": "rm", "mathbf": "bf", "boldsymbol": "bf", "bm": "bf",
	134	"mathbb": "bb", "mathcal": "cal", "mathscr": "cal", "mathfrak": "frak",
	135	"mathsf": "sf", "mathtt": "tt", "mathit": "it",
	136	}
	137
	138	var delimiters = map[string]string{
	139	"{": "{", "}": "}", "lbrace": "{", "rbrace": "}", "\|": "‖",
	140	"langle": "⟨", "rangle": "⟩", "lvert": "\|", "rvert": "\|", "vert": "\|",
	141	"lVert": "‖", "rVert": "‖", "Vert": "‖", "lfloor": "⌊", "rfloor": "⌋",
	142	"lceil": "⌈", "rceil": "⌉", "backslash": "\\",
	143	}
	144
	145	// alphabet is where a font's capitals, small letters and digits start in
	146	// Mathematical Alphanumeric Symbols (0 where the font has none), plus the
	147	// letters Unicode had already encoded elsewhere and left as holes.
	148	type alphabet struct {
	149	upper, lower, digit rune
	150	holes map[rune]rune
	151	}
	152
	153	var alphabets = map[string]alphabet{
	154	"bf": {0x1D400, 0x1D41A, 0x1D7CE, nil},
	155	"it": {0x1D434, 0x1D44E, 0, map[rune]rune{'h': 'ℎ'}},
	156	"bb": {0x1D538, 0x1D552, 0x1D7D8, map[rune]rune{
	157	'C': 'ℂ', 'H': 'ℍ', 'N': 'ℕ', 'P': 'ℙ', 'Q': 'ℚ', 'R': 'ℝ', 'Z': 'ℤ'}},
	158	"cal": {0x1D49C, 0x1D4B6, 0, map[rune]rune{
	159	'B': 'ℬ', 'E': 'ℰ', 'F': 'ℱ', 'H': 'ℋ', 'I': 'ℐ', 'L': 'ℒ', 'M': 'ℳ',
	160	'R': 'ℛ', 'e': 'ℯ', 'g': 'ℊ', 'o': 'ℴ'}},
	161	"frak": {0x1D504, 0x1D51E, 0, map[rune]rune{
	162	'C': 'ℭ', 'H': 'ℌ', 'I': 'ℑ', 'R': 'ℜ', 'Z': 'ℨ'}},
	163	"sf": {0x1D5A0, 0x1D5BA, 0x1D7E2, nil},
	164	"tt": {0x1D670, 0x1D68A, 0x1D7F6, nil},
	165	}
	166
	167	func alphanumeric(font string, r rune) rune {
	168	a, ok := alphabets[font]
	169	if !ok {
	170	return r
	171	}
	172	if h, ok := a.holes[r]; ok {
	173	return h
	174	}
	175	switch {
	176	case r >= 'A' && r <= 'Z':
	177	return a.upper + r - 'A'
	178	case r >= 'a' && r <= 'z':
	179	return a.lower + r - 'a'
	180	case r >= '0' && r <= '9' && a.digit != 0:
	181	return a.digit + r - '0'
	182	}
	183	return r
	184	}

internal/texmath/testdata/fuzz/FuzzConvert/1c174b99741b4473 added +3

@@ -0,0 +1,3 @@
	1	go test fuzz v1
	2	string("\\operatorname{\x00}")
	3	bool(false)

internal/texmath/texmath.go added +759

@@ -0,0 +1,759 @@
	1	// Package texmath converts a subset of TeX math to MathML.
	2	//
	3	// The subset: letters, numbers and operator characters; ^ and _; braces;
	4	// \frac and friends, \binom, \sqrt with an optional index; Greek letters and
	5	// the common symbols, relations and arrows in symbols.go; function names
	6	// (\sin, \lim, \operatorname{...}); large operators, which take their limits
	7	// above and below in display style; \left, \middle and \right; accents,
	8	// \overline, \underline and the braces; spacing commands; \text; the font
	9	// commands \mathrm, \mathbf, \mathbb, \mathcal, \mathscr, \mathfrak, \mathsf,
	10	// \mathtt, \mathit, \boldsymbol; and the environments matrix, pmatrix,
	11	// bmatrix, Bmatrix, vmatrix, Vmatrix, smallmatrix, cases, aligned, align,
	12	// gathered, gather, split, equation and displaymath.
	13	//
	14	// Anything else, including every command that defines macros, sets colors or
	15	// styles, or links, is an error; the caller shows the source instead. The
	16	// output uses only the elements and attributes listed in Elements and Attrs,
	17	// and its size is linear in the input, which is capped at MaxInput bytes and
	18	// MaxDepth levels of nesting.
	19	package texmath
	20
	21	import (
	22	"errors"
	23	"fmt"
	24	"html"
	25	"strings"
	26	"unicode"
	27	"unicode/utf8"
	28	)
	29
	30	// Limits on one expression.
	31	const (
	32	MaxInput = 8 << 10
	33	MaxDepth = 64
	34	)
	35
	36	// Elements is every MathML element Convert emits.
	37	var Elements = []string{
	38	"math", "mrow", "mi", "mn", "mo", "mtext", "mspace",
	39	"mfrac", "msqrt", "mroot", "msub", "msup", "msubsup",
	40	"munder", "mover", "munderover", "mtable", "mtr", "mtd",
	41	}
	42
	43	// Attrs is every attribute Convert emits, by element, with the values it
	44	// can take: a fixed value, or "<length>" for an em length.
	45	var Attrs = map[string]map[string]string{
	46	"math": {"display": "block"},
	47	"mi": {"mathvariant": "normal"},
	48	"mo": {"stretchy": "false"},
	49	"mfrac": {"linethickness": "0"},
	50	"mover": {"accent": "true"},
	51	"mspace": {"width": "<length>"},
	52	}
	53
	54	var (
	55	ErrTooLong = errors.New("texmath: expression too long")
	56	ErrTooDeep = errors.New("texmath: expression nested too deeply")
	57	)
	58
	59	// Convert renders tex as one <math> element; display selects block layout
	60	// and display-style limits.
	61	func Convert(tex string, display bool) (string, error) {
	62	if len(tex) > MaxInput {
	63	return "", ErrTooLong
	64	}
	65	if !utf8.ValidString(tex) {
	66	return "", errors.New("texmath: invalid UTF-8")
	67	}
	68	for _, r := range tex {
	69	if r < 0x20 && r != '\t' && r != '\n' && r != '\r' \|\| r == 0x7F \|\| r == 0xFFFE \|\| r == 0xFFFF {
	70	return "", errors.New("texmath: control character")
	71	}
	72	}
	73	p := &parser{src: tex, display: display}
	74	kids, err := p.list(func(t token) bool { return false })
	75	if err != nil {
	76	return "", err
	77	}
	78	if t := p.peek(); t.kind != eof {
	79	return "", p.unexpected(t)
	80	}
	81	root := &node{tag: "math", kids: kids}
	82	if display {
	83	root.attr("display", "block")
	84	}
	85	var b strings.Builder
	86	root.write(&b)
	87	return b.String(), nil
	88	}
	89
	90	type node struct {
	91	tag string
	92	attrs [][2]string
	93	text string
	94	kids []*node
	95	limits bool // a large operator: limits go under and over in display style
	96	fn bool // a function name: a thin space follows unless a delimiter does
	97	}
	98
	99	func (n node) attr(k, v string) node {
	100	n.attrs = append(n.attrs, [2]string{k, v})
	101	return n
	102	}
	103
	104	func (n node) write(b strings.Builder) {
	105	b.WriteByte('<')
	106	b.WriteString(n.tag)
	107	for _, a := range n.attrs {
	108	b.WriteString(" " + a[0] + `="` + html.EscapeString(a[1]) + `"`)
	109	}
	110	b.WriteByte('>')
	111	b.WriteString(html.EscapeString(n.text))
	112	for _, k := range n.kids {
	113	k.write(b)
	114	}
	115	b.WriteString("</" + n.tag + ">")
	116	}
	117
	118	func el(tag string, kids ...node) node { return &node{tag: tag, kids: kids} }
	119	func leaf(tag, text string) *node { return &node{tag: tag, text: text} }
	120	func row(kids []node) node { return &node{tag: "mrow", kids: kids} }
	121	func space(w string) *node { return (&node{tag: "mspace"}).attr("width", w) }
	122
	123	// fixed is an operator that must not stretch to the height of its row, the
	124	// way a bracket typed without \left does not in TeX.
	125	func fixed(s string) *node { return leaf("mo", s).attr("stretchy", "false") }
	126
	127	type kind int
	128
	129	const (
	130	eof kind = iota
	131	char // one character
	132	cmd // a control sequence; val is its name without the backslash
	133	)
	134
	135	type token struct {
	136	kind kind
	137	val string
	138	pos int
	139	}
	140
	141	type parser struct {
	142	src string
	143	pos int
	144	depth int
	145	display bool
	146	font string
	147	}
	148
	149	func (p *parser) skip() {
	150	for p.pos < len(p.src) {
	151	c := p.src[p.pos]
	152	switch {
	153	case c == ' ' \|\| c == '\t' \|\| c == '\n' \|\| c == '\r':
	154	p.pos++
	155	case c == '%':
	156	for p.pos < len(p.src) && p.src[p.pos] != '\n' {
	157	p.pos++
	158	}
	159	default:
	160	return
	161	}
	162	}
	163	}
	164
	165	func (p *parser) peek() token {
	166	save := p.pos
	167	t := p.next()
	168	p.pos = save
	169	return t
	170	}
	171
	172	func (p *parser) next() token {
	173	p.skip()
	174	start := p.pos
	175	if p.pos >= len(p.src) {
	176	return token{kind: eof, pos: start}
	177	}
	178	if p.src[p.pos] == '\\' {
	179	p.pos++
	180	if p.pos >= len(p.src) {
	181	return token{kind: cmd, val: "", pos: start}
	182	}
	183	if isASCIILetter(p.src[p.pos]) {
	184	end := p.pos
	185	for end < len(p.src) && isASCIILetter(p.src[end]) {
	186	end++
	187	}
	188	name := p.src[p.pos:end]
	189	p.pos = end
	190	// \operatorname* and friends: the star is part of the name.
	191	if p.pos < len(p.src) && p.src[p.pos] == '*' && starred[name] {
	192	p.pos++
	193	name += "*"
	194	}
	195	return token{kind: cmd, val: name, pos: start}
	196	}
	197	r, n := utf8.DecodeRuneInString(p.src[p.pos:])
	198	p.pos += n
	199	return token{kind: cmd, val: string(r), pos: start}
	200	}
	201	r, n := utf8.DecodeRuneInString(p.src[p.pos:])
	202	p.pos += n
	203	return token{kind: char, val: string(r), pos: start}
	204	}
	205
	206	var starred = map[string]bool{"operatorname": true}
	207
	208	func isASCIILetter(c byte) bool { return c >= 'a' && c <= 'z' \|\| c >= 'A' && c <= 'Z' }
	209
	210	func (p *parser) unexpected(t token) error {
	211	switch t.kind {
	212	case eof:
	213	return errors.New("texmath: unexpected end of input")
	214	case cmd:
	215	return fmt.Errorf(`texmath: unexpected \%s at %d`, t.val, t.pos)
	216	}
	217	return fmt.Errorf("texmath: unexpected %q at %d", t.val, t.pos)
	218	}
	219
	220	func (p *parser) expect(kind kind, val string) error {
	221	if t := p.next(); t.kind != kind \|\| t.val != val {
	222	return p.unexpected(t)
	223	}
	224	return nil
	225	}
	226
	227	func isChar(t token, s string) bool { return t.kind == char && t.val == s }
	228	func isCmd(t token, s string) bool { return t.kind == cmd && t.val == s }
	229
	230	// list parses atoms until the input ends or stop matches the next token,
	231	// which is left unread.
	232	func (p parser) list(stop func(token) bool) ([]node, error) {
	233	var out []*node
	234	for {
	235	t := p.peek()
	236	if t.kind == eof \|\| stop(t) {
	237	return out, nil
	238	}
	239	n, err := p.scripted()
	240	if err != nil {
	241	return nil, err
	242	}
	243	if n == nil {
	244	continue
	245	}
	246	out = append(out, n)
	247	if n.fn && !p.delimiterNext() {
	248	out = append(out, space("0.1667em"))
	249	}
	250	}
	251	}
	252
	253	// delimiterNext reports whether the next token closes a group or opens a
	254	// bracket, where TeX puts no space after a function name.
	255	func (p *parser) delimiterNext() bool {
	256	t := p.peek()
	257	if t.kind == eof {
	258	return true
	259	}
	260	if t.kind == char {
	261	return strings.Contains("()[]{}\|&.,;", t.val)
	262	}
	263	switch t.val {
	264	case "left", "right", "\\", "end", ",", ";", "!", "quad", "qquad", "{", "}":
	265	return true
	266	}
	267	return false
	268	}
	269
	270	// scripted parses one atom and any sub- and superscripts attached to it.
	271	func (p parser) scripted() (node, error) {
	272	var base *node
	273	if t := p.peek(); !isChar(t, "^") && !isChar(t, "_") {
	274	var err error
	275	if base, err = p.atom(); err != nil {
	276	return nil, err
	277	}
	278	}
	279	var sub, sup *node
	280	limits := base != nil && base.limits && p.display
	281	for {
	282	t := p.peek()
	283	switch {
	284	case isCmd(t, "limits"):
	285	p.next()
	286	limits = base != nil && base.limits
	287	continue
	288	case isCmd(t, "nolimits"):
	289	p.next()
	290	limits = false
	291	continue
	292	case isChar(t, "^"), isChar(t, "_"):
	293	default:
	294	if sub == nil && sup == nil {
	295	return base, nil
	296	}
	297	if base == nil {
	298	base = row(nil)
	299	}
	300	return script(base, sub, sup, limits), nil
	301	}
	302	p.next()
	303	arg, err := p.arg()
	304	if err != nil {
	305	return nil, err
	306	}
	307	if t.val == "^" {
	308	if sup != nil {
	309	return nil, fmt.Errorf("texmath: double superscript at %d", t.pos)
	310	}
	311	sup = arg
	312	} else {
	313	if sub != nil {
	314	return nil, fmt.Errorf("texmath: double subscript at %d", t.pos)
	315	}
	316	sub = arg
	317	}
	318	}
	319	}
	320
	321	func script(base, sub, sup node, limits bool) node {
	322	fn := base.fn
	323	var n *node
	324	switch {
	325	case limits && sub != nil && sup != nil:
	326	n = el("munderover", base, sub, sup)
	327	case limits && sub != nil:
	328	n = el("munder", base, sub)
	329	case limits:
	330	n = el("mover", base, sup)
	331	case sub != nil && sup != nil:
	332	n = el("msubsup", base, sub, sup)
	333	case sub != nil:
	334	n = el("msub", base, sub)
	335	default:
	336	n = el("msup", base, sup)
	337	}
	338	n.fn = fn
	339	return n
	340	}
	341
	342	// arg parses a command's argument: a braced group, or else one token.
	343	func (p parser) arg() (node, error) {
	344	t := p.peek()
	345	switch {
	346	case t.kind == eof:
	347	return nil, p.unexpected(t)
	348	case t.kind == char && strings.Contains("}&^_", t.val):
	349	return nil, p.unexpected(t)
	350	case t.kind == char && t.val >= "0" && t.val <= "9":
	351	p.next()
	352	return leaf("mn", p.styled(t.val)), nil
	353	}
	354	n, err := p.atom()
	355	if err == nil && n == nil {
	356	// \displaystyle and the like render nothing, so cannot be an argument.
	357	return nil, fmt.Errorf(`texmath: \%s cannot be an argument at %d`, t.val, t.pos)
	358	}
	359	return n, err
	360	}
	361
	362	func (p *parser) enter() error {
	363	p.depth++
	364	if p.depth > MaxDepth {
	365	return ErrTooDeep
	366	}
	367	return nil
	368	}
	369
	370	// atom parses one atom. A nil node with a nil error is a command that
	371	// renders nothing, such as \displaystyle.
	372	func (p parser) atom() (node, error) {
	373	if err := p.enter(); err != nil {
	374	return nil, err
	375	}
	376	defer func() { p.depth-- }()
	377	t := p.next()
	378	switch t.kind {
	379	case eof:
	380	return nil, p.unexpected(t)
	381	case char:
	382	return p.charAtom(t)
	383	}
	384	return p.command(t)
	385	}
	386
	387	func (p parser) charAtom(t token) (node, error) {
	388	r, _ := utf8.DecodeRuneInString(t.val)
	389	switch {
	390	case t.val == "{":
	391	font := p.font
	392	kids, err := p.list(func(t token) bool { return isChar(t, "}") })
	393	p.font = font
	394	if err != nil {
	395	return nil, err
	396	}
	397	if err := p.expect(char, "}"); err != nil {
	398	return nil, err
	399	}
	400	return row(kids), nil
	401	case r >= '0' && r <= '9':
	402	num := t.val
	403	for p.pos < len(p.src) {
	404	c := p.src[p.pos]
	405	if c >= '0' && c <= '9' \|\| c == '.' && p.pos+1 < len(p.src) && p.src[p.pos+1] >= '0' && p.src[p.pos+1] <= '9' {
	406	num += string(c)
	407	p.pos++
	408	continue
	409	}
	410	break
	411	}
	412	return leaf("mn", p.styled(num)), nil
	413	case unicode.IsLetter(r):
	414	n := leaf("mi", p.styled(t.val))
	415	if p.font == "rm" {
	416	n.attr("mathvariant", "normal")
	417	}
	418	return n, nil
	419	}
	420	switch t.val {
	421	case "}", "&", "$", "#", "\\":
	422	return nil, p.unexpected(t)
	423	case "(", ")", "[", "]", "\|", "/":
	424	return fixed(t.val), nil
	425	case "-":
	426	return leaf("mo", "−"), nil
	427	case "*":
	428	return leaf("mo", "∗"), nil
	429	case "'":
	430	return leaf("mo", "′"), nil
	431	case "~":
	432	return space("0.3333em"), nil
	433	}
	434	if r < 0x20 \|\| r == utf8.RuneError {
	435	return nil, p.unexpected(t)
	436	}
	437	return leaf("mo", t.val), nil
	438	}
	439
	440	func (p parser) command(t token) (node, error) {
	441	name := t.val
	442	if s, ok := symbols[name]; ok {
	443	n := leaf(s.tag, s.text)
	444	if s.upright {
	445	n.attr("mathvariant", "normal")
	446	}
	447	if s.fixed {
	448	n.attr("stretchy", "false")
	449	}
	450	n.limits = s.limits
	451	return n, nil
	452	}
	453	if w, ok := spaces[name]; ok {
	454	return space(w), nil
	455	}
	456	if f, ok := functions[name]; ok {
	457	n := leaf("mi", name)
	458	n.limits, n.fn = f, true
	459	return n, nil
	460	}
	461	if a, ok := accents[name]; ok {
	462	arg, err := p.arg()
	463	if err != nil {
	464	return nil, err
	465	}
	466	if a.under {
	467	return el("munder", arg, leaf("mo", a.mark)), nil
	468	}
	469	n := el("mover", arg, leaf("mo", a.mark))
	470	if a.accent {
	471	n.attr("accent", "true")
	472	}
	473	return n, nil
	474	}
	475	if f, ok := fonts[name]; ok {
	476	font := p.font
	477	p.font = f
	478	arg, err := p.arg()
	479	p.font = font
	480	return arg, err
	481	}
	482	switch name {
	483	case "frac", "dfrac", "tfrac", "cfrac":
	484	num, err := p.arg()
	485	if err != nil {
	486	return nil, err
	487	}
	488	den, err := p.arg()
	489	if err != nil {
	490	return nil, err
	491	}
	492	return el("mfrac", num, den), nil
	493	case "binom", "dbinom", "tbinom":
	494	top, err := p.arg()
	495	if err != nil {
	496	return nil, err
	497	}
	498	bottom, err := p.arg()
	499	if err != nil {
	500	return nil, err
	501	}
	502	frac := el("mfrac", top, bottom).attr("linethickness", "0")
	503	return row([]*node{leaf("mo", "("), frac, leaf("mo", ")")}), nil
	504	case "sqrt":
	505	var index *node
	506	if isChar(p.peek(), "[") {
	507	p.next()
	508	kids, err := p.list(func(t token) bool { return isChar(t, "]") })
	509	if err != nil {
	510	return nil, err
	511	}
	512	if err := p.expect(char, "]"); err != nil {
	513	return nil, err
	514	}
	515	index = row(kids)
	516	}
	517	arg, err := p.arg()
	518	if err != nil {
	519	return nil, err
	520	}
	521	if index != nil {
	522	return el("mroot", arg, index), nil
	523	}
	524	return el("msqrt", arg), nil
	525	case "left":
	526	open, err := p.delimiter()
	527	if err != nil {
	528	return nil, err
	529	}
	530	kids, err := p.list(func(t token) bool { return isCmd(t, "right") })
	531	if err != nil {
	532	return nil, err
	533	}
	534	if err := p.expect(cmd, "right"); err != nil {
	535	return nil, err
	536	}
	537	cls, err := p.delimiter()
	538	if err != nil {
	539	return nil, err
	540	}
	541	var out []*node
	542	if open != "" {
	543	out = append(out, leaf("mo", open))
	544	}
	545	out = append(out, kids...)
	546	if cls != "" {
	547	out = append(out, leaf("mo", cls))
	548	}
	549	return row(out), nil
	550	case "middle":
	551	d, err := p.delimiter()
	552	if err != nil {
	553	return nil, err
	554	}
	555	return leaf("mo", d), nil
	556	case "text", "textrm", "textnormal", "mbox":
	557	s, err := p.rawGroup()
	558	if err != nil {
	559	return nil, err
	560	}
	561	return leaf("mtext", s), nil
	562	case "operatorname", "operatorname*":
	563	s, err := p.rawGroup()
	564	if err != nil {
	565	return nil, err
	566	}
	567	n := leaf("mi", s)
	568	if utf8.RuneCountInString(s) == 1 {
	569	n.attr("mathvariant", "normal")
	570	}
	571	n.limits, n.fn = name == "operatorname*", true
	572	return n, nil
	573	case "begin":
	574	return p.environment()
	575	case "displaystyle", "textstyle", "scriptstyle", "scriptscriptstyle", "limits", "nolimits":
	576	return nil, nil
	577	}
	578	return nil, p.unexpected(t)
	579	}
	580
	581	// rawGroup reads a braced argument as text, for \text and \operatorname.
	582	// Nested braces must balance; \{, \}, \$, \%, \&, \#, \_ and \\ stand for
	583	// the character.
	584	func (p *parser) rawGroup() (string, error) {
	585	if err := p.expect(char, "{"); err != nil {
	586	return "", err
	587	}
	588	var b strings.Builder
	589	depth := 0
	590	for p.pos < len(p.src) {
	591	c := p.src[p.pos]
	592	switch {
	593	case c == '\\' && p.pos+1 < len(p.src) && strings.IndexByte(`{}$%&#_\`, p.src[p.pos+1]) >= 0:
	594	b.WriteByte(p.src[p.pos+1])
	595	p.pos += 2
	596	continue
	597	case c == '{':
	598	depth++
	599	case c == '}':
	600	if depth == 0 {
	601	p.pos++
	602	return b.String(), nil
	603	}
	604	depth--
	605	}
	606	b.WriteByte(c)
	607	p.pos++
	608	}
	609	return "", errors.New("texmath: unterminated group")
	610	}
	611
	612	// delimiter reads what follows \left, \middle or \right; "." is none.
	613	func (p *parser) delimiter() (string, error) {
	614	t := p.next()
	615	if t.kind == char {
	616	switch t.val {
	617	case ".":
	618	return "", nil
	619	case "<":
	620	return "⟨", nil
	621	case ">":
	622	return "⟩", nil
	623	case "(", ")", "[", "]", "\|", "/":
	624	return t.val, nil
	625	}
	626	}
	627	if t.kind == cmd {
	628	if d, ok := delimiters[t.val]; ok {
	629	return d, nil
	630	}
	631	}
	632	return "", p.unexpected(t)
	633	}
	634
	635	type env struct {
	636	open, close string
	637	table bool
	638	}
	639
	640	var environments = map[string]env{
	641	"matrix": {table: true},
	642	"smallmatrix": {table: true},
	643	"pmatrix": {"(", ")", true},
	644	"bmatrix": {"[", "]", true},
	645	"Bmatrix": {"{", "}", true},
	646	"vmatrix": {"\|", "\|", true},
	647	"Vmatrix": {"‖", "‖", true},
	648	"cases": {"{", "", true},
	649	"aligned": {table: true},
	650	"align": {table: true},
	651	"align*": {table: true},
	652	"gathered": {table: true},
	653	"gather": {table: true},
	654	"gather*": {table: true},
	655	"split": {table: true},
	656	"equation": {},
	657	"equation*": {},
	658	"displaymath": {},
	659	}
	660
	661	func (p *parser) envName() (string, error) {
	662	if err := p.expect(char, "{"); err != nil {
	663	return "", err
	664	}
	665	end := strings.IndexByte(p.src[p.pos:], '}')
	666	if end < 0 {
	667	return "", errors.New("texmath: unterminated environment name")
	668	}
	669	name := p.src[p.pos : p.pos+end]
	670	p.pos += end + 1
	671	return name, nil
	672	}
	673
	674	func (p parser) environment() (node, error) {
	675	name, err := p.envName()
	676	if err != nil {
	677	return nil, err
	678	}
	679	e, ok := environments[name]
	680	if !ok {
	681	return nil, fmt.Errorf("texmath: unsupported environment %q", name)
	682	}
	683	var body *node
	684	if e.table {
	685	body, err = p.table()
	686	} else {
	687	var kids []*node
	688	kids, err = p.list(func(t token) bool { return isCmd(t, "end") })
	689	body = row(kids)
	690	}
	691	if err != nil {
	692	return nil, err
	693	}
	694	if err := p.expect(cmd, "end"); err != nil {
	695	return nil, err
	696	}
	697	if end, err := p.envName(); err != nil {
	698	return nil, err
	699	} else if end != name {
	700	return nil, fmt.Errorf(`texmath: \begin{%s} ended by \end{%s}`, name, end)
	701	}
	702	if e.open == "" && e.close == "" {
	703	return body, nil
	704	}
	705	out := []*node{}
	706	if e.open != "" {
	707	out = append(out, leaf("mo", e.open))
	708	}
	709	out = append(out, body)
	710	if e.close != "" {
	711	out = append(out, leaf("mo", e.close))
	712	}
	713	return row(out), nil
	714	}
	715
	716	// table parses rows separated by \\ and cells separated by &, up to \end.
	717	func (p parser) table() (node, error) {
	718	stop := func(t token) bool { return isChar(t, "&") \|\| isCmd(t, "\\") \|\| isCmd(t, "end") }
	719	tbl := el("mtable")
	720	cur := el("mtr")
	721	for {
	722	kids, err := p.list(stop)
	723	if err != nil {
	724	return nil, err
	725	}
	726	cur.kids = append(cur.kids, el("mtd", kids...))
	727	t := p.peek()
	728	switch {
	729	case isChar(t, "&"):
	730	p.next()
	731	case isCmd(t, "\\"):
	732	p.next()
	733	tbl.kids = append(tbl.kids, cur)
	734	cur = el("mtr")
	735	case isCmd(t, "end"):
	736	// A trailing \\ leaves one empty cell; it is not a row.
	737	if len(cur.kids) > 1 \|\| len(cur.kids[0].kids) > 0 {
	738	tbl.kids = append(tbl.kids, cur)
	739	}
	740	return tbl, nil
	741	default:
	742	return nil, p.unexpected(t)
	743	}
	744	}
	745	}
	746
	747	// styled maps letters and digits into the current font's Mathematical
	748	// Alphanumeric Symbols, which is how MathML Core spells \mathbb and the rest:
	749	// mathvariant is honoured only as "normal".
	750	func (p *parser) styled(s string) string {
	751	if p.font == "" \|\| p.font == "rm" {
	752	return s
	753	}
	754	var b strings.Builder
	755	for _, r := range s {
	756	b.WriteRune(alphanumeric(p.font, r))
	757	}
	758	return b.String()
	759	}

internal/texmath/texmath_test.go added +224

@@ -0,0 +1,224 @@
	1	package texmath
	2
	3	import (
	4	"encoding/xml"
	5	"errors"
	6	"io"
	7	"regexp"
	8	"strings"
	9	"testing"
	10	"time"
	11	)
	12
	13	func TestConvert(t *testing.T) {
	14	cases := []struct {
	15	tex, want string
	16	}{
	17	{`x`, `<math><mi>x</mi></math>`},
	18	{`12.5+x`, `<math><mn>12.5</mn><mo>+</mo><mi>x</mi></math>`},
	19	{`a-b`, `<math><mi>a</mi><mo>−</mo><mi>b</mi></math>`},
	20	{`x^2`, `<math><msup><mi>x</mi><mn>2</mn></msup></math>`},
	21	{`x_i^2`, `<math><msubsup><mi>x</mi><mi>i</mi><mn>2</mn></msubsup></math>`},
	22	{`x^{2n}`, `<math><msup><mi>x</mi><mrow><mn>2</mn><mi>n</mi></mrow></msup></math>`},
	23	{`\frac{a}{b}`, `<math><mfrac><mrow><mi>a</mi></mrow><mrow><mi>b</mi></mrow></mfrac></math>`},
	24	{`\frac12`, `<math><mfrac><mn>1</mn><mn>2</mn></mfrac></math>`},
	25	{`\sqrt{x}`, `<math><msqrt><mrow><mi>x</mi></mrow></msqrt></math>`},
	26	{`\sqrt[3]{x}`, `<math><mroot><mrow><mi>x</mi></mrow><mrow><mn>3</mn></mrow></mroot></math>`},
	27	{`\alpha\Gamma`, `<math><mi>α</mi><mi mathvariant="normal">Γ</mi></math>`},
	28	{`a\le b`, `<math><mi>a</mi><mo>≤</mo><mi>b</mi></math>`},
	29	{`(a)`, `<math><mo stretchy="false">(</mo><mi>a</mi><mo stretchy="false">)</mo></math>`},
	30	{`\left( x \right.`, `<math><mrow><mo>(</mo><mi>x</mi></mrow></math>`},
	31	{`\left\{ x \middle\| y \right\}`, `<math><mrow><mo>{</mo><mi>x</mi><mo>\|</mo><mi>y</mi><mo>}</mo></mrow></math>`},
	32	{`\sin x`, `<math><mi>sin</mi><mspace width="0.1667em"></mspace><mi>x</mi></math>`},
	33	{`\sin(x)`, `<math><mi>sin</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></math>`},
	34	{`\text{if } x`, `<math><mtext>if </mtext><mi>x</mi></math>`},
	35	{`\mathbb{R}\mathbf{v}`, `<math><mrow><mi>ℝ</mi></mrow><mrow><mi>𝐯</mi></mrow></math>`},
	36	{`\mathrm{d}x`, `<math><mrow><mi mathvariant="normal">d</mi></mrow><mi>x</mi></math>`},
	37	{`\hat{x}`, `<math><mover accent="true"><mrow><mi>x</mi></mrow><mo>^</mo></mover></math>`},
	38	{`a\,b`, `<math><mi>a</mi><mspace width="0.1667em"></mspace><mi>b</mi></math>`},
	39	{`\binom{n}{k}`, `<math><mrow><mo>(</mo><mfrac linethickness="0"><mrow><mi>n</mi></mrow><mrow><mi>k</mi></mrow></mfrac><mo>)</mo></mrow></math>`},
	40	{`\begin{pmatrix}a&b\\c&d\end{pmatrix}`, `<math><mrow><mo>(</mo><mtable><mtr><mtd><mi>a</mi></mtd><mtd><mi>b</mi></mtd></mtr><mtr><mtd><mi>c</mi></mtd><mtd><mi>d</mi></mtd></mtr></mtable><mo>)</mo></mrow></math>`},
	41	{`\begin{matrix}a\\\end{matrix}`, `<math><mtable><mtr><mtd><mi>a</mi></mtd></mtr></mtable></math>`},
	42	{`x % comment` + "\n" + `+1`, `<math><mi>x</mi><mo>+</mo><mn>1</mn></math>`},
	43	{`a<b`, `<math><mi>a</mi><mo><</mo><mi>b</mi></math>`},
	44	}
	45	for _, c := range cases {
	46	got, err := Convert(c.tex, false)
	47	if err != nil {
	48	t.Errorf("Convert(%q): %v", c.tex, err)
	49	continue
	50	}
	51	if got != c.want {
	52	t.Errorf("Convert(%q)\n got %s\nwant %s", c.tex, got, c.want)
	53	}
	54	}
	55	}
	56
	57	func TestConvertDisplayLimits(t *testing.T) {
	58	got, err := Convert(`\sum_{i=1}^n i`, true)
	59	if err != nil {
	60	t.Fatal(err)
	61	}
	62	if !strings.HasPrefix(got, `<math display="block"><munderover><mo>∑</mo>`) {
	63	t.Errorf("display sum: %s", got)
	64	}
	65	got, _ = Convert(`\sum_{i=1}^n i`, false)
	66	if !strings.HasPrefix(got, `<math><msubsup><mo>∑</mo>`) {
	67	t.Errorf("inline sum: %s", got)
	68	}
	69	got, _ = Convert(`\lim_{x\to 0} f`, true)
	70	if !strings.HasPrefix(got, `<math display="block"><munder><mi>lim</mi>`) {
	71	t.Errorf("display lim: %s", got)
	72	}
	73	}
	74
	75	func TestConvertRefuses(t *testing.T) {
	76	for _, tex := range []string{
	77	`\frac{a}`, `x^`, `{x`, `x}`, `x^1^2`, `\left( x`, `\right)`,
	78	`\begin{pmatrix}a\end{bmatrix}`, `\begin{tabular}x\end{tabular}`,
	79	`a & b`, `a \\ b`, `\unknown`,
	80	`\href{javascript:alert(1)}{x}`, `\url{javascript:alert(1)}`,
	81	`\style{color:red}{x}`, `\color{red}{x}`, `\class{a}{x}`, `\htmlId{a}{x}`,
	82	`\def\a{x}\a`, `\newcommand{\a}{x}`, `\require{html}`, `\unicode{x}`,
	83	`\includegraphics{x}`, `\input{/etc/passwd}`,
	84	`\sqrt\displaystyle`, `\frac\displaystyle y`, `\hat\displaystyle`,
	85	`\overbrace\displaystyle`, `\binom\displaystyle1`, `\mathbf\limits`, `x^\nolimits`,
	86	} {
	87	if out, err := Convert(tex, false); err == nil {
	88	t.Errorf("Convert(%q) = %s, want an error", tex, out)
	89	}
	90	}
	91	}
	92
	93	func TestConvertBounds(t *testing.T) {
	94	deep := strings.Repeat("{", 100000) + "x" + strings.Repeat("}", 100000)
	95	fracs := strings.Repeat(`\frac{`, MaxDepth+1) + "x"
	96	for _, tex := range []string{deep, fracs, strings.Repeat("x", MaxInput+1)} {
	97	start := time.Now()
	98	if _, err := Convert(tex, false); err == nil {
	99	t.Errorf("Convert(%.20q...) succeeded", tex)
	100	}
	101	if d := time.Since(start); d > time.Second {
	102	t.Errorf("Convert(%.20q...) took %v", tex, d)
	103	}
	104	}
	105	// The largest accepted input: output stays linear in it.
	106	tex := strings.Repeat(`{}`, MaxInput/2)
	107	start := time.Now()
	108	out, err := Convert(tex, false)
	109	if err != nil {
	110	t.Fatal(err)
	111	}
	112	if len(out) > 16*MaxInput {
	113	t.Errorf("output %d bytes for %d bytes of input", len(out), len(tex))
	114	}
	115	if d := time.Since(start); d > time.Second {
	116	t.Errorf("took %v", d)
	117	}
	118	}
	119
	120	var tagAttr = regexp.MustCompile(`<([a-z]+)((?: [a-z]+="[^"]"))>`)
	121	var attrPair = regexp.MustCompile(` ([a-z]+)="([^"]*)"`)
	122
	123	// Every element and attribute in the output is one Elements and Attrs name,
	124	// so the sanitizer's allowlist built from them is complete.
	125	func TestConvertEmitsOnlyListed(t *testing.T) {
	126	allowed := map[string]bool{}
	127	for _, e := range Elements {
	128	allowed[e] = true
	129	}
	130	var inputs []string
	131	for name := range symbols {
	132	inputs = append(inputs, `\`+name)
	133	}
	134	for name := range functions {
	135	inputs = append(inputs, `\`+name+`_a^b x`)
	136	}
	137	for name := range accents {
	138	inputs = append(inputs, `\`+name+`{x}`)
	139	}
	140	for name := range spaces {
	141	inputs = append(inputs, `a\`+name+` b`)
	142	}
	143	for name := range fonts {
	144	inputs = append(inputs, `\`+name+`{Ab1}`)
	145	}
	146	for name := range environments {
	147	inputs = append(inputs, `\begin{`+name+`}a&b\\c&d\end{`+name+`}`)
	148	}
	149	inputs = append(inputs, `\binom12`, `\sqrt[3]{x}`, `\left(\middle\|\right)`, `\text{a}`,
	150	`\operatorname{rank}A`, `\operatorname*{arg\,max}_x`, `x_1^2`, `\sum_1^2`, `\sum_1`, `\sum^2`, `(a)~'`)
	151	for _, in := range inputs {
	152	for _, display := range []bool{false, true} {
	153	out, err := Convert(in, display)
	154	if err != nil {
	155	if strings.HasPrefix(in, `\begin{`) {
	156	continue // equation and friends take no & or \\
	157	}
	158	t.Errorf("Convert(%q): %v", in, err)
	159	continue
	160	}
	161	for _, m := range tagAttr.FindAllStringSubmatch(out, -1) {
	162	if !allowed[m[1]] {
	163	t.Errorf("%q emits <%s>", in, m[1])
	164	}
	165	for _, a := range attrPair.FindAllStringSubmatch(m[2], -1) {
	166	want, ok := Attrs[m[1]][a[1]]
	167	if !ok \|\| (want != "<length>" && want != a[2]) {
	168	t.Errorf("%q emits %s %s=%q", in, m[1], a[1], a[2])
	169	}
	170	}
	171	}
	172	}
	173	}
	174	}
	175
	176	// FuzzConvert: no panic, output bounded by the input, and output that is
	177	// well-formed XML using only the listed elements and attribute values.
	178	func FuzzConvert(f *testing.F) {
	179	for _, seed := range []string{
	180	`x^2`, `\frac{a}{b}`, `\sqrt[3]{x}`, `\left(\frac12\right)`, `\sum_{i=1}^n i`,
	181	`\begin{pmatrix}a&b\\c&d\end{pmatrix}`, `\text{a<b}`, `\mathbb{R}`, `\hat{x}~'`,
	182	`\sqrt\displaystyle`, `\operatorname*{argmax}_x`, `{{{x}}}`, `a\,b\quad c`,
	183	} {
	184	f.Add(seed, false)
	185	}
	186	allowed := map[string]bool{}
	187	for _, e := range Elements {
	188	allowed[e] = true
	189	}
	190	length := regexp.MustCompile(`^-?[0-9]+(\.[0-9]+)?em$`)
	191	f.Fuzz(func(t *testing.T, tex string, display bool) {
	192	out, err := Convert(tex, display)
	193	if err != nil {
	194	return
	195	}
	196	if len(out) > 64*len(tex)+256 {
	197	t.Fatalf("%d bytes out for %d in", len(out), len(tex))
	198	}
	199	d := xml.NewDecoder(strings.NewReader(out))
	200	for {
	201	tok, err := d.Token()
	202	if errors.Is(err, io.EOF) {
	203	break
	204	}
	205	if err != nil {
	206	t.Fatalf("not XML: %v\n%s", err, out)
	207	}
	208	start, ok := tok.(xml.StartElement)
	209	if !ok {
	210	continue
	211	}
	212	if !allowed[start.Name.Local] \|\| start.Name.Space != "" {
	213	t.Fatalf("element %v in %s", start.Name, out)
	214	}
	215	for _, a := range start.Attr {
	216	want, ok := Attrs[start.Name.Local][a.Name.Local]
	217	if !ok \|\| a.Name.Space != "" \|\| (want == "<length>" && !length.MatchString(a.Value)) \|\|
	218	(want != "<length>" && want != a.Value) {
	219	t.Fatalf("attribute %v=%q on %s in %s", a.Name, a.Value, start.Name.Local, out)
	220	}
	221	}
	222	}
	223	})
	224	}

	@@ -57,6 +57,7 @@ browser-only and the iOS build screen unable to say more than the log.
57	\| request a review \| yes \| yes \| yes \|	57	\| request a review \| yes \| yes \| yes \|
58	\| choose body markup \| yes \| yes \| yes \|	58	\| choose body markup \| yes \| yes \| yes \|
59	\| preview body markup \| n/a \| yes \| no \|	59	\| preview body markup \| n/a \| yes \| no \|
		60	\| TeX math as MathML \| n/a \| yes \| no \|
60	\| stacked merge requests \| yes \| yes \| yes \|	61	\| stacked merge requests \| yes \| yes \| yes \|
61	\| revisions \| yes \| yes \| yes \|	62	\| revisions \| yes \| yes \| yes \|
62	\| range-diff \| yes \| yes \| yes \|	63	\| range-diff \| yes \| yes \| yes \|
	@@ -125,6 +126,7 @@ reviews, since an approval was of the diff against the old branch.
125	\| labels: list, colour \| yes \| yes \| yes \|	126	\| labels: list, colour \| yes \| yes \| yes \|
126	\| choose body markup \| yes \| yes \| yes \|	127	\| choose body markup \| yes \| yes \| yes \|
127	\| preview body markup \| n/a \| yes \| no \|	128	\| preview body markup \| n/a \| yes \| no \|
		129	\| TeX math as MathML \| n/a \| yes \| no \|
128	\| issue templates \| yes \| yes \| yes \|	130	\| issue templates \| yes \| yes \| yes \|
129	\| milestone create, close, reopen \| yes \| yes \| yes \|	131	\| milestone create, close, reopen \| yes \| yes \| yes \|
130	\| org labels: set, list, remove \| yes \| yes \| yes \|	132	\| org labels: set, list, remove \| yes \| yes \| yes \|
	@@ -155,7 +157,8 @@ every surface now offers the choice: the web's create forms, and the iOS
155	composer on create, edit and comment. An edit starts on the format its	157	composer on create, edit and comment. An edit starts on the format its
156	body was stored in, since starting elsewhere would silently reinterpret	158	body was stored in, since starting elsewhere would silently reinterpret
157	it on the next save. Diff-line comments have no format column and are	159	it on the next save. Diff-line comments have no format column and are
158	always markdown.	160	always markdown. TeX math in either format renders as MathML on the
		161	web (#294); the terminal shows the source, so the CLI row is =n/a=.
159		162
160	Every web form that takes markup has a Preview button beside its own	163	Every web form that takes markup has a Preview button beside its own
161	submit: issue and merge request create, their edit and comment boxes,	164	submit: issue and merge request create, their edit and comment boxes,