Commit d7e13eb9 authored by songlinfeng's avatar songlinfeng
Browse files

add dtk-container-toolkit

parent fcdba4f3
package toml
// ValueStringRepresentation transforms an interface{} value into its toml string representation.
func ValueStringRepresentation(v interface{}, commented string, indent string, ord MarshalOrder, arraysOneElementPerLine bool) (string, error) {
return tomlValueStringRepresentation(v, commented, indent, ord, arraysOneElementPerLine)
}
*.out
*.swp
*.8
*.6
_obj
_test*
markdown
tags
sudo: false
language: go
go:
- "1.10.x"
- "1.11.x"
- tip
matrix:
fast_finish: true
allow_failures:
- go: tip
install:
- # Do nothing. This is needed to prevent default install action "go get -t -v ./..." from happening here (we want it to happen inside script step).
script:
- go get -t -v ./...
- diff -u <(echo -n) <(gofmt -d -s .)
- go tool vet .
- go test -v ./...
Blackfriday is distributed under the Simplified BSD License:
> Copyright © 2011 Russ Ross
> All rights reserved.
>
> Redistribution and use in source and binary forms, with or without
> modification, are permitted provided that the following conditions
> are met:
>
> 1. Redistributions of source code must retain the above copyright
> notice, this list of conditions and the following disclaimer.
>
> 2. Redistributions in binary form must reproduce the above
> copyright notice, this list of conditions and the following
> disclaimer in the documentation and/or other materials provided with
> the distribution.
>
> THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
> FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
> BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
> LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
> CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
> LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
> ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
> POSSIBILITY OF SUCH DAMAGE.
Blackfriday
[![Build Status][BuildV2SVG]][BuildV2URL]
[![PkgGoDev][PkgGoDevV2SVG]][PkgGoDevV2URL]
===========
Blackfriday is a [Markdown][1] processor implemented in [Go][2]. It
is paranoid about its input (so you can safely feed it user-supplied
data), it is fast, it supports common extensions (tables, smart
punctuation substitutions, etc.), and it is safe for all utf-8
(unicode) input.
HTML output is currently supported, along with Smartypants
extensions.
It started as a translation from C of [Sundown][3].
Installation
------------
Blackfriday is compatible with modern Go releases in module mode.
With Go installed:
go get github.com/russross/blackfriday/v2
will resolve and add the package to the current development module,
then build and install it. Alternatively, you can achieve the same
if you import it in a package:
import "github.com/russross/blackfriday/v2"
and `go get` without parameters.
Legacy GOPATH mode is unsupported.
Versions
--------
Currently maintained and recommended version of Blackfriday is `v2`. It's being
developed on its own branch: https://github.com/russross/blackfriday/tree/v2 and the
documentation is available at
https://pkg.go.dev/github.com/russross/blackfriday/v2.
It is `go get`-able in module mode at `github.com/russross/blackfriday/v2`.
Version 2 offers a number of improvements over v1:
* Cleaned up API
* A separate call to [`Parse`][4], which produces an abstract syntax tree for
the document
* Latest bug fixes
* Flexibility to easily add your own rendering extensions
Potential drawbacks:
* Our benchmarks show v2 to be slightly slower than v1. Currently in the
ballpark of around 15%.
* API breakage. If you can't afford modifying your code to adhere to the new API
and don't care too much about the new features, v2 is probably not for you.
* Several bug fixes are trailing behind and still need to be forward-ported to
v2. See issue [#348](https://github.com/russross/blackfriday/issues/348) for
tracking.
If you are still interested in the legacy `v1`, you can import it from
`github.com/russross/blackfriday`. Documentation for the legacy v1 can be found
here: https://pkg.go.dev/github.com/russross/blackfriday.
Usage
-----
For the most sensible markdown processing, it is as simple as getting your input
into a byte slice and calling:
```go
output := blackfriday.Run(input)
```
Your input will be parsed and the output rendered with a set of most popular
extensions enabled. If you want the most basic feature set, corresponding with
the bare Markdown specification, use:
```go
output := blackfriday.Run(input, blackfriday.WithNoExtensions())
```
### Sanitize untrusted content
Blackfriday itself does nothing to protect against malicious content. If you are
dealing with user-supplied markdown, we recommend running Blackfriday's output
through HTML sanitizer such as [Bluemonday][5].
Here's an example of simple usage of Blackfriday together with Bluemonday:
```go
import (
"github.com/microcosm-cc/bluemonday"
"github.com/russross/blackfriday/v2"
)
// ...
unsafe := blackfriday.Run(input)
html := bluemonday.UGCPolicy().SanitizeBytes(unsafe)
```
### Custom options
If you want to customize the set of options, use `blackfriday.WithExtensions`,
`blackfriday.WithRenderer` and `blackfriday.WithRefOverride`.
### `blackfriday-tool`
You can also check out `blackfriday-tool` for a more complete example
of how to use it. Download and install it using:
go get github.com/russross/blackfriday-tool
This is a simple command-line tool that allows you to process a
markdown file using a standalone program. You can also browse the
source directly on github if you are just looking for some example
code:
* <https://github.com/russross/blackfriday-tool>
Note that if you have not already done so, installing
`blackfriday-tool` will be sufficient to download and install
blackfriday in addition to the tool itself. The tool binary will be
installed in `$GOPATH/bin`. This is a statically-linked binary that
can be copied to wherever you need it without worrying about
dependencies and library versions.
### Sanitized anchor names
Blackfriday includes an algorithm for creating sanitized anchor names
corresponding to a given input text. This algorithm is used to create
anchors for headings when `AutoHeadingIDs` extension is enabled. The
algorithm has a specification, so that other packages can create
compatible anchor names and links to those anchors.
The specification is located at https://pkg.go.dev/github.com/russross/blackfriday/v2#hdr-Sanitized_Anchor_Names.
[`SanitizedAnchorName`](https://pkg.go.dev/github.com/russross/blackfriday/v2#SanitizedAnchorName) exposes this functionality, and can be used to
create compatible links to the anchor names generated by blackfriday.
This algorithm is also implemented in a small standalone package at
[`github.com/shurcooL/sanitized_anchor_name`](https://pkg.go.dev/github.com/shurcooL/sanitized_anchor_name). It can be useful for clients
that want a small package and don't need full functionality of blackfriday.
Features
--------
All features of Sundown are supported, including:
* **Compatibility**. The Markdown v1.0.3 test suite passes with
the `--tidy` option. Without `--tidy`, the differences are
mostly in whitespace and entity escaping, where blackfriday is
more consistent and cleaner.
* **Common extensions**, including table support, fenced code
blocks, autolinks, strikethroughs, non-strict emphasis, etc.
* **Safety**. Blackfriday is paranoid when parsing, making it safe
to feed untrusted user input without fear of bad things
happening. The test suite stress tests this and there are no
known inputs that make it crash. If you find one, please let me
know and send me the input that does it.
NOTE: "safety" in this context means *runtime safety only*. In order to
protect yourself against JavaScript injection in untrusted content, see
[this example](https://github.com/russross/blackfriday#sanitize-untrusted-content).
* **Fast processing**. It is fast enough to render on-demand in
most web applications without having to cache the output.
* **Thread safety**. You can run multiple parsers in different
goroutines without ill effect. There is no dependence on global
shared state.
* **Minimal dependencies**. Blackfriday only depends on standard
library packages in Go. The source code is pretty
self-contained, so it is easy to add to any project, including
Google App Engine projects.
* **Standards compliant**. Output successfully validates using the
W3C validation tool for HTML 4.01 and XHTML 1.0 Transitional.
Extensions
----------
In addition to the standard markdown syntax, this package
implements the following extensions:
* **Intra-word emphasis supression**. The `_` character is
commonly used inside words when discussing code, so having
markdown interpret it as an emphasis command is usually the
wrong thing. Blackfriday lets you treat all emphasis markers as
normal characters when they occur inside a word.
* **Tables**. Tables can be created by drawing them in the input
using a simple syntax:
```
Name | Age
--------|------
Bob | 27
Alice | 23
```
* **Fenced code blocks**. In addition to the normal 4-space
indentation to mark code blocks, you can explicitly mark them
and supply a language (to make syntax highlighting simple). Just
mark it like this:
```go
func getTrue() bool {
return true
}
```
You can use 3 or more backticks to mark the beginning of the
block, and the same number to mark the end of the block.
To preserve classes of fenced code blocks while using the bluemonday
HTML sanitizer, use the following policy:
```go
p := bluemonday.UGCPolicy()
p.AllowAttrs("class").Matching(regexp.MustCompile("^language-[a-zA-Z0-9]+$")).OnElements("code")
html := p.SanitizeBytes(unsafe)
```
* **Definition lists**. A simple definition list is made of a single-line
term followed by a colon and the definition for that term.
Cat
: Fluffy animal everyone likes
Internet
: Vector of transmission for pictures of cats
Terms must be separated from the previous definition by a blank line.
* **Footnotes**. A marker in the text that will become a superscript number;
a footnote definition that will be placed in a list of footnotes at the
end of the document. A footnote looks like this:
This is a footnote.[^1]
[^1]: the footnote text.
* **Autolinking**. Blackfriday can find URLs that have not been
explicitly marked as links and turn them into links.
* **Strikethrough**. Use two tildes (`~~`) to mark text that
should be crossed out.
* **Hard line breaks**. With this extension enabled newlines in the input
translate into line breaks in the output. This extension is off by default.
* **Smart quotes**. Smartypants-style punctuation substitution is
supported, turning normal double- and single-quote marks into
curly quotes, etc.
* **LaTeX-style dash parsing** is an additional option, where `--`
is translated into `&ndash;`, and `---` is translated into
`&mdash;`. This differs from most smartypants processors, which
turn a single hyphen into an ndash and a double hyphen into an
mdash.
* **Smart fractions**, where anything that looks like a fraction
is translated into suitable HTML (instead of just a few special
cases like most smartypant processors). For example, `4/5`
becomes `<sup>4</sup>&frasl;<sub>5</sub>`, which renders as
<sup>4</sup>&frasl;<sub>5</sub>.
Other renderers
---------------
Blackfriday is structured to allow alternative rendering engines. Here
are a few of note:
* [github_flavored_markdown](https://pkg.go.dev/github.com/shurcooL/github_flavored_markdown):
provides a GitHub Flavored Markdown renderer with fenced code block
highlighting, clickable heading anchor links.
It's not customizable, and its goal is to produce HTML output
equivalent to the [GitHub Markdown API endpoint](https://developer.github.com/v3/markdown/#render-a-markdown-document-in-raw-mode),
except the rendering is performed locally.
* [markdownfmt](https://github.com/shurcooL/markdownfmt): like gofmt,
but for markdown.
* [LaTeX output](https://gitlab.com/ambrevar/blackfriday-latex):
renders output as LaTeX.
* [bfchroma](https://github.com/Depado/bfchroma/): provides convenience
integration with the [Chroma](https://github.com/alecthomas/chroma) code
highlighting library. bfchroma is only compatible with v2 of Blackfriday and
provides a drop-in renderer ready to use with Blackfriday, as well as
options and means for further customization.
* [Blackfriday-Confluence](https://github.com/kentaro-m/blackfriday-confluence): provides a [Confluence Wiki Markup](https://confluence.atlassian.com/doc/confluence-wiki-markup-251003035.html) renderer.
* [Blackfriday-Slack](https://github.com/karriereat/blackfriday-slack): converts markdown to slack message style
TODO
----
* More unit testing
* Improve Unicode support. It does not understand all Unicode
rules (about what constitutes a letter, a punctuation symbol,
etc.), so it may fail to detect word boundaries correctly in
some instances. It is safe on all UTF-8 input.
License
-------
[Blackfriday is distributed under the Simplified BSD License](LICENSE.txt)
[1]: https://daringfireball.net/projects/markdown/ "Markdown"
[2]: https://golang.org/ "Go Language"
[3]: https://github.com/vmg/sundown "Sundown"
[4]: https://pkg.go.dev/github.com/russross/blackfriday/v2#Parse "Parse func"
[5]: https://github.com/microcosm-cc/bluemonday "Bluemonday"
[BuildV2SVG]: https://travis-ci.org/russross/blackfriday.svg?branch=v2
[BuildV2URL]: https://travis-ci.org/russross/blackfriday
[PkgGoDevV2SVG]: https://pkg.go.dev/badge/github.com/russross/blackfriday/v2
[PkgGoDevV2URL]: https://pkg.go.dev/github.com/russross/blackfriday/v2
//
// Blackfriday Markdown Processor
// Available at http://github.com/russross/blackfriday
//
// Copyright © 2011 Russ Ross <russ@russross.com>.
// Distributed under the Simplified BSD License.
// See README.md for details.
//
//
// Functions to parse block-level elements.
//
package blackfriday
import (
"bytes"
"html"
"regexp"
"strings"
"unicode"
)
const (
charEntity = "&(?:#x[a-f0-9]{1,8}|#[0-9]{1,8}|[a-z][a-z0-9]{1,31});"
escapable = "[!\"#$%&'()*+,./:;<=>?@[\\\\\\]^_`{|}~-]"
)
var (
reBackslashOrAmp = regexp.MustCompile("[\\&]")
reEntityOrEscapedChar = regexp.MustCompile("(?i)\\\\" + escapable + "|" + charEntity)
)
// Parse block-level data.
// Note: this function and many that it calls assume that
// the input buffer ends with a newline.
func (p *Markdown) block(data []byte) {
// this is called recursively: enforce a maximum depth
if p.nesting >= p.maxNesting {
return
}
p.nesting++
// parse out one block-level construct at a time
for len(data) > 0 {
// prefixed heading:
//
// # Heading 1
// ## Heading 2
// ...
// ###### Heading 6
if p.isPrefixHeading(data) {
data = data[p.prefixHeading(data):]
continue
}
// block of preformatted HTML:
//
// <div>
// ...
// </div>
if data[0] == '<' {
if i := p.html(data, true); i > 0 {
data = data[i:]
continue
}
}
// title block
//
// % stuff
// % more stuff
// % even more stuff
if p.extensions&Titleblock != 0 {
if data[0] == '%' {
if i := p.titleBlock(data, true); i > 0 {
data = data[i:]
continue
}
}
}
// blank lines. note: returns the # of bytes to skip
if i := p.isEmpty(data); i > 0 {
data = data[i:]
continue
}
// indented code block:
//
// func max(a, b int) int {
// if a > b {
// return a
// }
// return b
// }
if p.codePrefix(data) > 0 {
data = data[p.code(data):]
continue
}
// fenced code block:
//
// ``` go
// func fact(n int) int {
// if n <= 1 {
// return n
// }
// return n * fact(n-1)
// }
// ```
if p.extensions&FencedCode != 0 {
if i := p.fencedCodeBlock(data, true); i > 0 {
data = data[i:]
continue
}
}
// horizontal rule:
//
// ------
// or
// ******
// or
// ______
if p.isHRule(data) {
p.addBlock(HorizontalRule, nil)
var i int
for i = 0; i < len(data) && data[i] != '\n'; i++ {
}
data = data[i:]
continue
}
// block quote:
//
// > A big quote I found somewhere
// > on the web
if p.quotePrefix(data) > 0 {
data = data[p.quote(data):]
continue
}
// table:
//
// Name | Age | Phone
// ------|-----|---------
// Bob | 31 | 555-1234
// Alice | 27 | 555-4321
if p.extensions&Tables != 0 {
if i := p.table(data); i > 0 {
data = data[i:]
continue
}
}
// an itemized/unordered list:
//
// * Item 1
// * Item 2
//
// also works with + or -
if p.uliPrefix(data) > 0 {
data = data[p.list(data, 0):]
continue
}
// a numbered/ordered list:
//
// 1. Item 1
// 2. Item 2
if p.oliPrefix(data) > 0 {
data = data[p.list(data, ListTypeOrdered):]
continue
}
// definition lists:
//
// Term 1
// : Definition a
// : Definition b
//
// Term 2
// : Definition c
if p.extensions&DefinitionLists != 0 {
if p.dliPrefix(data) > 0 {
data = data[p.list(data, ListTypeDefinition):]
continue
}
}
// anything else must look like a normal paragraph
// note: this finds underlined headings, too
data = data[p.paragraph(data):]
}
p.nesting--
}
func (p *Markdown) addBlock(typ NodeType, content []byte) *Node {
p.closeUnmatchedBlocks()
container := p.addChild(typ, 0)
container.content = content
return container
}
func (p *Markdown) isPrefixHeading(data []byte) bool {
if data[0] != '#' {
return false
}
if p.extensions&SpaceHeadings != 0 {
level := 0
for level < 6 && level < len(data) && data[level] == '#' {
level++
}
if level == len(data) || data[level] != ' ' {
return false
}
}
return true
}
func (p *Markdown) prefixHeading(data []byte) int {
level := 0
for level < 6 && level < len(data) && data[level] == '#' {
level++
}
i := skipChar(data, level, ' ')
end := skipUntilChar(data, i, '\n')
skip := end
id := ""
if p.extensions&HeadingIDs != 0 {
j, k := 0, 0
// find start/end of heading id
for j = i; j < end-1 && (data[j] != '{' || data[j+1] != '#'); j++ {
}
for k = j + 1; k < end && data[k] != '}'; k++ {
}
// extract heading id iff found
if j < end && k < end {
id = string(data[j+2 : k])
end = j
skip = k + 1
for end > 0 && data[end-1] == ' ' {
end--
}
}
}
for end > 0 && data[end-1] == '#' {
if isBackslashEscaped(data, end-1) {
break
}
end--
}
for end > 0 && data[end-1] == ' ' {
end--
}
if end > i {
if id == "" && p.extensions&AutoHeadingIDs != 0 {
id = SanitizedAnchorName(string(data[i:end]))
}
block := p.addBlock(Heading, data[i:end])
block.HeadingID = id
block.Level = level
}
return skip
}
func (p *Markdown) isUnderlinedHeading(data []byte) int {
// test of level 1 heading
if data[0] == '=' {
i := skipChar(data, 1, '=')
i = skipChar(data, i, ' ')
if i < len(data) && data[i] == '\n' {
return 1
}
return 0
}
// test of level 2 heading
if data[0] == '-' {
i := skipChar(data, 1, '-')
i = skipChar(data, i, ' ')
if i < len(data) && data[i] == '\n' {
return 2
}
return 0
}
return 0
}
func (p *Markdown) titleBlock(data []byte, doRender bool) int {
if data[0] != '%' {
return 0
}
splitData := bytes.Split(data, []byte("\n"))
var i int
for idx, b := range splitData {
if !bytes.HasPrefix(b, []byte("%")) {
i = idx // - 1
break
}
}
data = bytes.Join(splitData[0:i], []byte("\n"))
consumed := len(data)
data = bytes.TrimPrefix(data, []byte("% "))
data = bytes.Replace(data, []byte("\n% "), []byte("\n"), -1)
block := p.addBlock(Heading, data)
block.Level = 1
block.IsTitleblock = true
return consumed
}
func (p *Markdown) html(data []byte, doRender bool) int {
var i, j int
// identify the opening tag
if data[0] != '<' {
return 0
}
curtag, tagfound := p.htmlFindTag(data[1:])
// handle special cases
if !tagfound {
// check for an HTML comment
if size := p.htmlComment(data, doRender); size > 0 {
return size
}
// check for an <hr> tag
if size := p.htmlHr(data, doRender); size > 0 {
return size
}
// no special case recognized
return 0
}
// look for an unindented matching closing tag
// followed by a blank line
found := false
/*
closetag := []byte("\n</" + curtag + ">")
j = len(curtag) + 1
for !found {
// scan for a closing tag at the beginning of a line
if skip := bytes.Index(data[j:], closetag); skip >= 0 {
j += skip + len(closetag)
} else {
break
}
// see if it is the only thing on the line
if skip := p.isEmpty(data[j:]); skip > 0 {
// see if it is followed by a blank line/eof
j += skip
if j >= len(data) {
found = true
i = j
} else {
if skip := p.isEmpty(data[j:]); skip > 0 {
j += skip
found = true
i = j
}
}
}
}
*/
// if not found, try a second pass looking for indented match
// but not if tag is "ins" or "del" (following original Markdown.pl)
if !found && curtag != "ins" && curtag != "del" {
i = 1
for i < len(data) {
i++
for i < len(data) && !(data[i-1] == '<' && data[i] == '/') {
i++
}
if i+2+len(curtag) >= len(data) {
break
}
j = p.htmlFindEnd(curtag, data[i-1:])
if j > 0 {
i += j - 1
found = true
break
}
}
}
if !found {
return 0
}
// the end of the block has been found
if doRender {
// trim newlines
end := i
for end > 0 && data[end-1] == '\n' {
end--
}
finalizeHTMLBlock(p.addBlock(HTMLBlock, data[:end]))
}
return i
}
func finalizeHTMLBlock(block *Node) {
block.Literal = block.content
block.content = nil
}
// HTML comment, lax form
func (p *Markdown) htmlComment(data []byte, doRender bool) int {
i := p.inlineHTMLComment(data)
// needs to end with a blank line
if j := p.isEmpty(data[i:]); j > 0 {
size := i + j
if doRender {
// trim trailing newlines
end := size
for end > 0 && data[end-1] == '\n' {
end--
}
block := p.addBlock(HTMLBlock, data[:end])
finalizeHTMLBlock(block)
}
return size
}
return 0
}
// HR, which is the only self-closing block tag considered
func (p *Markdown) htmlHr(data []byte, doRender bool) int {
if len(data) < 4 {
return 0
}
if data[0] != '<' || (data[1] != 'h' && data[1] != 'H') || (data[2] != 'r' && data[2] != 'R') {
return 0
}
if data[3] != ' ' && data[3] != '/' && data[3] != '>' {
// not an <hr> tag after all; at least not a valid one
return 0
}
i := 3
for i < len(data) && data[i] != '>' && data[i] != '\n' {
i++
}
if i < len(data) && data[i] == '>' {
i++
if j := p.isEmpty(data[i:]); j > 0 {
size := i + j
if doRender {
// trim newlines
end := size
for end > 0 && data[end-1] == '\n' {
end--
}
finalizeHTMLBlock(p.addBlock(HTMLBlock, data[:end]))
}
return size
}
}
return 0
}
func (p *Markdown) htmlFindTag(data []byte) (string, bool) {
i := 0
for i < len(data) && isalnum(data[i]) {
i++
}
key := string(data[:i])
if _, ok := blockTags[key]; ok {
return key, true
}
return "", false
}
func (p *Markdown) htmlFindEnd(tag string, data []byte) int {
// assume data[0] == '<' && data[1] == '/' already tested
if tag == "hr" {
return 2
}
// check if tag is a match
closetag := []byte("</" + tag + ">")
if !bytes.HasPrefix(data, closetag) {
return 0
}
i := len(closetag)
// check that the rest of the line is blank
skip := 0
if skip = p.isEmpty(data[i:]); skip == 0 {
return 0
}
i += skip
skip = 0
if i >= len(data) {
return i
}
if p.extensions&LaxHTMLBlocks != 0 {
return i
}
if skip = p.isEmpty(data[i:]); skip == 0 {
// following line must be blank
return 0
}
return i + skip
}
func (*Markdown) isEmpty(data []byte) int {
// it is okay to call isEmpty on an empty buffer
if len(data) == 0 {
return 0
}
var i int
for i = 0; i < len(data) && data[i] != '\n'; i++ {
if data[i] != ' ' && data[i] != '\t' {
return 0
}
}
if i < len(data) && data[i] == '\n' {
i++
}
return i
}
func (*Markdown) isHRule(data []byte) bool {
i := 0
// skip up to three spaces
for i < 3 && data[i] == ' ' {
i++
}
// look at the hrule char
if data[i] != '*' && data[i] != '-' && data[i] != '_' {
return false
}
c := data[i]
// the whole line must be the char or whitespace
n := 0
for i < len(data) && data[i] != '\n' {
switch {
case data[i] == c:
n++
case data[i] != ' ':
return false
}
i++
}
return n >= 3
}
// isFenceLine checks if there's a fence line (e.g., ``` or ``` go) at the beginning of data,
// and returns the end index if so, or 0 otherwise. It also returns the marker found.
// If info is not nil, it gets set to the syntax specified in the fence line.
func isFenceLine(data []byte, info *string, oldmarker string) (end int, marker string) {
i, size := 0, 0
// skip up to three spaces
for i < len(data) && i < 3 && data[i] == ' ' {
i++
}
// check for the marker characters: ~ or `
if i >= len(data) {
return 0, ""
}
if data[i] != '~' && data[i] != '`' {
return 0, ""
}
c := data[i]
// the whole line must be the same char or whitespace
for i < len(data) && data[i] == c {
size++
i++
}
// the marker char must occur at least 3 times
if size < 3 {
return 0, ""
}
marker = string(data[i-size : i])
// if this is the end marker, it must match the beginning marker
if oldmarker != "" && marker != oldmarker {
return 0, ""
}
// TODO(shurcooL): It's probably a good idea to simplify the 2 code paths here
// into one, always get the info string, and discard it if the caller doesn't care.
if info != nil {
infoLength := 0
i = skipChar(data, i, ' ')
if i >= len(data) {
if i == len(data) {
return i, marker
}
return 0, ""
}
infoStart := i
if data[i] == '{' {
i++
infoStart++
for i < len(data) && data[i] != '}' && data[i] != '\n' {
infoLength++
i++
}
if i >= len(data) || data[i] != '}' {
return 0, ""
}
// strip all whitespace at the beginning and the end
// of the {} block
for infoLength > 0 && isspace(data[infoStart]) {
infoStart++
infoLength--
}
for infoLength > 0 && isspace(data[infoStart+infoLength-1]) {
infoLength--
}
i++
i = skipChar(data, i, ' ')
} else {
for i < len(data) && !isverticalspace(data[i]) {
infoLength++
i++
}
}
*info = strings.TrimSpace(string(data[infoStart : infoStart+infoLength]))
}
if i == len(data) {
return i, marker
}
if i > len(data) || data[i] != '\n' {
return 0, ""
}
return i + 1, marker // Take newline into account.
}
// fencedCodeBlock returns the end index if data contains a fenced code block at the beginning,
// or 0 otherwise. It writes to out if doRender is true, otherwise it has no side effects.
// If doRender is true, a final newline is mandatory to recognize the fenced code block.
func (p *Markdown) fencedCodeBlock(data []byte, doRender bool) int {
var info string
beg, marker := isFenceLine(data, &info, "")
if beg == 0 || beg >= len(data) {
return 0
}
fenceLength := beg - 1
var work bytes.Buffer
work.Write([]byte(info))
work.WriteByte('\n')
for {
// safe to assume beg < len(data)
// check for the end of the code block
fenceEnd, _ := isFenceLine(data[beg:], nil, marker)
if fenceEnd != 0 {
beg += fenceEnd
break
}
// copy the current line
end := skipUntilChar(data, beg, '\n') + 1
// did we reach the end of the buffer without a closing marker?
if end >= len(data) {
return 0
}
// verbatim copy to the working buffer
if doRender {
work.Write(data[beg:end])
}
beg = end
}
if doRender {
block := p.addBlock(CodeBlock, work.Bytes()) // TODO: get rid of temp buffer
block.IsFenced = true
block.FenceLength = fenceLength
finalizeCodeBlock(block)
}
return beg
}
func unescapeChar(str []byte) []byte {
if str[0] == '\\' {
return []byte{str[1]}
}
return []byte(html.UnescapeString(string(str)))
}
func unescapeString(str []byte) []byte {
if reBackslashOrAmp.Match(str) {
return reEntityOrEscapedChar.ReplaceAllFunc(str, unescapeChar)
}
return str
}
func finalizeCodeBlock(block *Node) {
if block.IsFenced {
newlinePos := bytes.IndexByte(block.content, '\n')
firstLine := block.content[:newlinePos]
rest := block.content[newlinePos+1:]
block.Info = unescapeString(bytes.Trim(firstLine, "\n"))
block.Literal = rest
} else {
block.Literal = block.content
}
block.content = nil
}
func (p *Markdown) table(data []byte) int {
table := p.addBlock(Table, nil)
i, columns := p.tableHeader(data)
if i == 0 {
p.tip = table.Parent
table.Unlink()
return 0
}
p.addBlock(TableBody, nil)
for i < len(data) {
pipes, rowStart := 0, i
for ; i < len(data) && data[i] != '\n'; i++ {
if data[i] == '|' {
pipes++
}
}
if pipes == 0 {
i = rowStart
break
}
// include the newline in data sent to tableRow
if i < len(data) && data[i] == '\n' {
i++
}
p.tableRow(data[rowStart:i], columns, false)
}
return i
}
// check if the specified position is preceded by an odd number of backslashes
func isBackslashEscaped(data []byte, i int) bool {
backslashes := 0
for i-backslashes-1 >= 0 && data[i-backslashes-1] == '\\' {
backslashes++
}
return backslashes&1 == 1
}
func (p *Markdown) tableHeader(data []byte) (size int, columns []CellAlignFlags) {
i := 0
colCount := 1
for i = 0; i < len(data) && data[i] != '\n'; i++ {
if data[i] == '|' && !isBackslashEscaped(data, i) {
colCount++
}
}
// doesn't look like a table header
if colCount == 1 {
return
}
// include the newline in the data sent to tableRow
j := i
if j < len(data) && data[j] == '\n' {
j++
}
header := data[:j]
// column count ignores pipes at beginning or end of line
if data[0] == '|' {
colCount--
}
if i > 2 && data[i-1] == '|' && !isBackslashEscaped(data, i-1) {
colCount--
}
columns = make([]CellAlignFlags, colCount)
// move on to the header underline
i++
if i >= len(data) {
return
}
if data[i] == '|' && !isBackslashEscaped(data, i) {
i++
}
i = skipChar(data, i, ' ')
// each column header is of form: / *:?-+:? *|/ with # dashes + # colons >= 3
// and trailing | optional on last column
col := 0
for i < len(data) && data[i] != '\n' {
dashes := 0
if data[i] == ':' {
i++
columns[col] |= TableAlignmentLeft
dashes++
}
for i < len(data) && data[i] == '-' {
i++
dashes++
}
if i < len(data) && data[i] == ':' {
i++
columns[col] |= TableAlignmentRight
dashes++
}
for i < len(data) && data[i] == ' ' {
i++
}
if i == len(data) {
return
}
// end of column test is messy
switch {
case dashes < 3:
// not a valid column
return
case data[i] == '|' && !isBackslashEscaped(data, i):
// marker found, now skip past trailing whitespace
col++
i++
for i < len(data) && data[i] == ' ' {
i++
}
// trailing junk found after last column
if col >= colCount && i < len(data) && data[i] != '\n' {
return
}
case (data[i] != '|' || isBackslashEscaped(data, i)) && col+1 < colCount:
// something else found where marker was required
return
case data[i] == '\n':
// marker is optional for the last column
col++
default:
// trailing junk found after last column
return
}
}
if col != colCount {
return
}
p.addBlock(TableHead, nil)
p.tableRow(header, columns, true)
size = i
if size < len(data) && data[size] == '\n' {
size++
}
return
}
func (p *Markdown) tableRow(data []byte, columns []CellAlignFlags, header bool) {
p.addBlock(TableRow, nil)
i, col := 0, 0
if data[i] == '|' && !isBackslashEscaped(data, i) {
i++
}
for col = 0; col < len(columns) && i < len(data); col++ {
for i < len(data) && data[i] == ' ' {
i++
}
cellStart := i
for i < len(data) && (data[i] != '|' || isBackslashEscaped(data, i)) && data[i] != '\n' {
i++
}
cellEnd := i
// skip the end-of-cell marker, possibly taking us past end of buffer
i++
for cellEnd > cellStart && cellEnd-1 < len(data) && data[cellEnd-1] == ' ' {
cellEnd--
}
cell := p.addBlock(TableCell, data[cellStart:cellEnd])
cell.IsHeader = header
cell.Align = columns[col]
}
// pad it out with empty columns to get the right number
for ; col < len(columns); col++ {
cell := p.addBlock(TableCell, nil)
cell.IsHeader = header
cell.Align = columns[col]
}
// silently ignore rows with too many cells
}
// returns blockquote prefix length
func (p *Markdown) quotePrefix(data []byte) int {
i := 0
for i < 3 && i < len(data) && data[i] == ' ' {
i++
}
if i < len(data) && data[i] == '>' {
if i+1 < len(data) && data[i+1] == ' ' {
return i + 2
}
return i + 1
}
return 0
}
// blockquote ends with at least one blank line
// followed by something without a blockquote prefix
func (p *Markdown) terminateBlockquote(data []byte, beg, end int) bool {
if p.isEmpty(data[beg:]) <= 0 {
return false
}
if end >= len(data) {
return true
}
return p.quotePrefix(data[end:]) == 0 && p.isEmpty(data[end:]) == 0
}
// parse a blockquote fragment
func (p *Markdown) quote(data []byte) int {
block := p.addBlock(BlockQuote, nil)
var raw bytes.Buffer
beg, end := 0, 0
for beg < len(data) {
end = beg
// Step over whole lines, collecting them. While doing that, check for
// fenced code and if one's found, incorporate it altogether,
// irregardless of any contents inside it
for end < len(data) && data[end] != '\n' {
if p.extensions&FencedCode != 0 {
if i := p.fencedCodeBlock(data[end:], false); i > 0 {
// -1 to compensate for the extra end++ after the loop:
end += i - 1
break
}
}
end++
}
if end < len(data) && data[end] == '\n' {
end++
}
if pre := p.quotePrefix(data[beg:]); pre > 0 {
// skip the prefix
beg += pre
} else if p.terminateBlockquote(data, beg, end) {
break
}
// this line is part of the blockquote
raw.Write(data[beg:end])
beg = end
}
p.block(raw.Bytes())
p.finalize(block)
return end
}
// returns prefix length for block code
func (p *Markdown) codePrefix(data []byte) int {
if len(data) >= 1 && data[0] == '\t' {
return 1
}
if len(data) >= 4 && data[0] == ' ' && data[1] == ' ' && data[2] == ' ' && data[3] == ' ' {
return 4
}
return 0
}
func (p *Markdown) code(data []byte) int {
var work bytes.Buffer
i := 0
for i < len(data) {
beg := i
for i < len(data) && data[i] != '\n' {
i++
}
if i < len(data) && data[i] == '\n' {
i++
}
blankline := p.isEmpty(data[beg:i]) > 0
if pre := p.codePrefix(data[beg:i]); pre > 0 {
beg += pre
} else if !blankline {
// non-empty, non-prefixed line breaks the pre
i = beg
break
}
// verbatim copy to the working buffer
if blankline {
work.WriteByte('\n')
} else {
work.Write(data[beg:i])
}
}
// trim all the \n off the end of work
workbytes := work.Bytes()
eol := len(workbytes)
for eol > 0 && workbytes[eol-1] == '\n' {
eol--
}
if eol != len(workbytes) {
work.Truncate(eol)
}
work.WriteByte('\n')
block := p.addBlock(CodeBlock, work.Bytes()) // TODO: get rid of temp buffer
block.IsFenced = false
finalizeCodeBlock(block)
return i
}
// returns unordered list item prefix
func (p *Markdown) uliPrefix(data []byte) int {
i := 0
// start with up to 3 spaces
for i < len(data) && i < 3 && data[i] == ' ' {
i++
}
if i >= len(data)-1 {
return 0
}
// need one of {'*', '+', '-'} followed by a space or a tab
if (data[i] != '*' && data[i] != '+' && data[i] != '-') ||
(data[i+1] != ' ' && data[i+1] != '\t') {
return 0
}
return i + 2
}
// returns ordered list item prefix
func (p *Markdown) oliPrefix(data []byte) int {
i := 0
// start with up to 3 spaces
for i < 3 && i < len(data) && data[i] == ' ' {
i++
}
// count the digits
start := i
for i < len(data) && data[i] >= '0' && data[i] <= '9' {
i++
}
if start == i || i >= len(data)-1 {
return 0
}
// we need >= 1 digits followed by a dot and a space or a tab
if data[i] != '.' || !(data[i+1] == ' ' || data[i+1] == '\t') {
return 0
}
return i + 2
}
// returns definition list item prefix
func (p *Markdown) dliPrefix(data []byte) int {
if len(data) < 2 {
return 0
}
i := 0
// need a ':' followed by a space or a tab
if data[i] != ':' || !(data[i+1] == ' ' || data[i+1] == '\t') {
return 0
}
for i < len(data) && data[i] == ' ' {
i++
}
return i + 2
}
// parse ordered or unordered list block
func (p *Markdown) list(data []byte, flags ListType) int {
i := 0
flags |= ListItemBeginningOfList
block := p.addBlock(List, nil)
block.ListFlags = flags
block.Tight = true
for i < len(data) {
skip := p.listItem(data[i:], &flags)
if flags&ListItemContainsBlock != 0 {
block.ListData.Tight = false
}
i += skip
if skip == 0 || flags&ListItemEndOfList != 0 {
break
}
flags &= ^ListItemBeginningOfList
}
above := block.Parent
finalizeList(block)
p.tip = above
return i
}
// Returns true if the list item is not the same type as its parent list
func (p *Markdown) listTypeChanged(data []byte, flags *ListType) bool {
if p.dliPrefix(data) > 0 && *flags&ListTypeDefinition == 0 {
return true
} else if p.oliPrefix(data) > 0 && *flags&ListTypeOrdered == 0 {
return true
} else if p.uliPrefix(data) > 0 && (*flags&ListTypeOrdered != 0 || *flags&ListTypeDefinition != 0) {
return true
}
return false
}
// Returns true if block ends with a blank line, descending if needed
// into lists and sublists.
func endsWithBlankLine(block *Node) bool {
// TODO: figure this out. Always false now.
for block != nil {
//if block.lastLineBlank {
//return true
//}
t := block.Type
if t == List || t == Item {
block = block.LastChild
} else {
break
}
}
return false
}
func finalizeList(block *Node) {
block.open = false
item := block.FirstChild
for item != nil {
// check for non-final list item ending with blank line:
if endsWithBlankLine(item) && item.Next != nil {
block.ListData.Tight = false
break
}
// recurse into children of list item, to see if there are spaces
// between any of them:
subItem := item.FirstChild
for subItem != nil {
if endsWithBlankLine(subItem) && (item.Next != nil || subItem.Next != nil) {
block.ListData.Tight = false
break
}
subItem = subItem.Next
}
item = item.Next
}
}
// Parse a single list item.
// Assumes initial prefix is already removed if this is a sublist.
func (p *Markdown) listItem(data []byte, flags *ListType) int {
// keep track of the indentation of the first line
itemIndent := 0
if data[0] == '\t' {
itemIndent += 4
} else {
for itemIndent < 3 && data[itemIndent] == ' ' {
itemIndent++
}
}
var bulletChar byte = '*'
i := p.uliPrefix(data)
if i == 0 {
i = p.oliPrefix(data)
} else {
bulletChar = data[i-2]
}
if i == 0 {
i = p.dliPrefix(data)
// reset definition term flag
if i > 0 {
*flags &= ^ListTypeTerm
}
}
if i == 0 {
// if in definition list, set term flag and continue
if *flags&ListTypeDefinition != 0 {
*flags |= ListTypeTerm
} else {
return 0
}
}
// skip leading whitespace on first line
for i < len(data) && data[i] == ' ' {
i++
}
// find the end of the line
line := i
for i > 0 && i < len(data) && data[i-1] != '\n' {
i++
}
// get working buffer
var raw bytes.Buffer
// put the first line into the working buffer
raw.Write(data[line:i])
line = i
// process the following lines
containsBlankLine := false
sublist := 0
codeBlockMarker := ""
gatherlines:
for line < len(data) {
i++
// find the end of this line
for i < len(data) && data[i-1] != '\n' {
i++
}
// if it is an empty line, guess that it is part of this item
// and move on to the next line
if p.isEmpty(data[line:i]) > 0 {
containsBlankLine = true
line = i
continue
}
// calculate the indentation
indent := 0
indentIndex := 0
if data[line] == '\t' {
indentIndex++
indent += 4
} else {
for indent < 4 && line+indent < i && data[line+indent] == ' ' {
indent++
indentIndex++
}
}
chunk := data[line+indentIndex : i]
if p.extensions&FencedCode != 0 {
// determine if in or out of codeblock
// if in codeblock, ignore normal list processing
_, marker := isFenceLine(chunk, nil, codeBlockMarker)
if marker != "" {
if codeBlockMarker == "" {
// start of codeblock
codeBlockMarker = marker
} else {
// end of codeblock.
codeBlockMarker = ""
}
}
// we are in a codeblock, write line, and continue
if codeBlockMarker != "" || marker != "" {
raw.Write(data[line+indentIndex : i])
line = i
continue gatherlines
}
}
// evaluate how this line fits in
switch {
// is this a nested list item?
case (p.uliPrefix(chunk) > 0 && !p.isHRule(chunk)) ||
p.oliPrefix(chunk) > 0 ||
p.dliPrefix(chunk) > 0:
// to be a nested list, it must be indented more
// if not, it is either a different kind of list
// or the next item in the same list
if indent <= itemIndent {
if p.listTypeChanged(chunk, flags) {
*flags |= ListItemEndOfList
} else if containsBlankLine {
*flags |= ListItemContainsBlock
}
break gatherlines
}
if containsBlankLine {
*flags |= ListItemContainsBlock
}
// is this the first item in the nested list?
if sublist == 0 {
sublist = raw.Len()
}
// is this a nested prefix heading?
case p.isPrefixHeading(chunk):
// if the heading is not indented, it is not nested in the list
// and thus ends the list
if containsBlankLine && indent < 4 {
*flags |= ListItemEndOfList
break gatherlines
}
*flags |= ListItemContainsBlock
// anything following an empty line is only part
// of this item if it is indented 4 spaces
// (regardless of the indentation of the beginning of the item)
case containsBlankLine && indent < 4:
if *flags&ListTypeDefinition != 0 && i < len(data)-1 {
// is the next item still a part of this list?
next := i
for next < len(data) && data[next] != '\n' {
next++
}
for next < len(data)-1 && data[next] == '\n' {
next++
}
if i < len(data)-1 && data[i] != ':' && data[next] != ':' {
*flags |= ListItemEndOfList
}
} else {
*flags |= ListItemEndOfList
}
break gatherlines
// a blank line means this should be parsed as a block
case containsBlankLine:
raw.WriteByte('\n')
*flags |= ListItemContainsBlock
}
// if this line was preceded by one or more blanks,
// re-introduce the blank into the buffer
if containsBlankLine {
containsBlankLine = false
raw.WriteByte('\n')
}
// add the line into the working buffer without prefix
raw.Write(data[line+indentIndex : i])
line = i
}
rawBytes := raw.Bytes()
block := p.addBlock(Item, nil)
block.ListFlags = *flags
block.Tight = false
block.BulletChar = bulletChar
block.Delimiter = '.' // Only '.' is possible in Markdown, but ')' will also be possible in CommonMark
// render the contents of the list item
if *flags&ListItemContainsBlock != 0 && *flags&ListTypeTerm == 0 {
// intermediate render of block item, except for definition term
if sublist > 0 {
p.block(rawBytes[:sublist])
p.block(rawBytes[sublist:])
} else {
p.block(rawBytes)
}
} else {
// intermediate render of inline item
if sublist > 0 {
child := p.addChild(Paragraph, 0)
child.content = rawBytes[:sublist]
p.block(rawBytes[sublist:])
} else {
child := p.addChild(Paragraph, 0)
child.content = rawBytes
}
}
return line
}
// render a single paragraph that has already been parsed out
func (p *Markdown) renderParagraph(data []byte) {
if len(data) == 0 {
return
}
// trim leading spaces
beg := 0
for data[beg] == ' ' {
beg++
}
end := len(data)
// trim trailing newline
if data[len(data)-1] == '\n' {
end--
}
// trim trailing spaces
for end > beg && data[end-1] == ' ' {
end--
}
p.addBlock(Paragraph, data[beg:end])
}
func (p *Markdown) paragraph(data []byte) int {
// prev: index of 1st char of previous line
// line: index of 1st char of current line
// i: index of cursor/end of current line
var prev, line, i int
tabSize := TabSizeDefault
if p.extensions&TabSizeEight != 0 {
tabSize = TabSizeDouble
}
// keep going until we find something to mark the end of the paragraph
for i < len(data) {
// mark the beginning of the current line
prev = line
current := data[i:]
line = i
// did we find a reference or a footnote? If so, end a paragraph
// preceding it and report that we have consumed up to the end of that
// reference:
if refEnd := isReference(p, current, tabSize); refEnd > 0 {
p.renderParagraph(data[:i])
return i + refEnd
}
// did we find a blank line marking the end of the paragraph?
if n := p.isEmpty(current); n > 0 {
// did this blank line followed by a definition list item?
if p.extensions&DefinitionLists != 0 {
if i < len(data)-1 && data[i+1] == ':' {
return p.list(data[prev:], ListTypeDefinition)
}
}
p.renderParagraph(data[:i])
return i + n
}
// an underline under some text marks a heading, so our paragraph ended on prev line
if i > 0 {
if level := p.isUnderlinedHeading(current); level > 0 {
// render the paragraph
p.renderParagraph(data[:prev])
// ignore leading and trailing whitespace
eol := i - 1
for prev < eol && data[prev] == ' ' {
prev++
}
for eol > prev && data[eol-1] == ' ' {
eol--
}
id := ""
if p.extensions&AutoHeadingIDs != 0 {
id = SanitizedAnchorName(string(data[prev:eol]))
}
block := p.addBlock(Heading, data[prev:eol])
block.Level = level
block.HeadingID = id
// find the end of the underline
for i < len(data) && data[i] != '\n' {
i++
}
return i
}
}
// if the next line starts a block of HTML, then the paragraph ends here
if p.extensions&LaxHTMLBlocks != 0 {
if data[i] == '<' && p.html(current, false) > 0 {
// rewind to before the HTML block
p.renderParagraph(data[:i])
return i
}
}
// if there's a prefixed heading or a horizontal rule after this, paragraph is over
if p.isPrefixHeading(current) || p.isHRule(current) {
p.renderParagraph(data[:i])
return i
}
// if there's a fenced code block, paragraph is over
if p.extensions&FencedCode != 0 {
if p.fencedCodeBlock(current, false) > 0 {
p.renderParagraph(data[:i])
return i
}
}
// if there's a definition list item, prev line is a definition term
if p.extensions&DefinitionLists != 0 {
if p.dliPrefix(current) != 0 {
ret := p.list(data[prev:], ListTypeDefinition)
return ret
}
}
// if there's a list after this, paragraph is over
if p.extensions&NoEmptyLineBeforeBlock != 0 {
if p.uliPrefix(current) != 0 ||
p.oliPrefix(current) != 0 ||
p.quotePrefix(current) != 0 ||
p.codePrefix(current) != 0 {
p.renderParagraph(data[:i])
return i
}
}
// otherwise, scan to the beginning of the next line
nl := bytes.IndexByte(data[i:], '\n')
if nl >= 0 {
i += nl + 1
} else {
i += len(data[i:])
}
}
p.renderParagraph(data[:i])
return i
}
func skipChar(data []byte, start int, char byte) int {
i := start
for i < len(data) && data[i] == char {
i++
}
return i
}
func skipUntilChar(text []byte, start int, char byte) int {
i := start
for i < len(text) && text[i] != char {
i++
}
return i
}
// SanitizedAnchorName returns a sanitized anchor name for the given text.
//
// It implements the algorithm specified in the package comment.
func SanitizedAnchorName(text string) string {
var anchorName []rune
futureDash := false
for _, r := range text {
switch {
case unicode.IsLetter(r) || unicode.IsNumber(r):
if futureDash && len(anchorName) > 0 {
anchorName = append(anchorName, '-')
}
futureDash = false
anchorName = append(anchorName, unicode.ToLower(r))
default:
futureDash = true
}
}
return string(anchorName)
}
// Package blackfriday is a markdown processor.
//
// It translates plain text with simple formatting rules into an AST, which can
// then be further processed to HTML (provided by Blackfriday itself) or other
// formats (provided by the community).
//
// The simplest way to invoke Blackfriday is to call the Run function. It will
// take a text input and produce a text output in HTML (or other format).
//
// A slightly more sophisticated way to use Blackfriday is to create a Markdown
// processor and to call Parse, which returns a syntax tree for the input
// document. You can leverage Blackfriday's parsing for content extraction from
// markdown documents. You can assign a custom renderer and set various options
// to the Markdown processor.
//
// If you're interested in calling Blackfriday from command line, see
// https://github.com/russross/blackfriday-tool.
//
// Sanitized Anchor Names
//
// Blackfriday includes an algorithm for creating sanitized anchor names
// corresponding to a given input text. This algorithm is used to create
// anchors for headings when AutoHeadingIDs extension is enabled. The
// algorithm is specified below, so that other packages can create
// compatible anchor names and links to those anchors.
//
// The algorithm iterates over the input text, interpreted as UTF-8,
// one Unicode code point (rune) at a time. All runes that are letters (category L)
// or numbers (category N) are considered valid characters. They are mapped to
// lower case, and included in the output. All other runes are considered
// invalid characters. Invalid characters that precede the first valid character,
// as well as invalid character that follow the last valid character
// are dropped completely. All other sequences of invalid characters
// between two valid characters are replaced with a single dash character '-'.
//
// SanitizedAnchorName exposes this functionality, and can be used to
// create compatible links to the anchor names generated by blackfriday.
// This algorithm is also implemented in a small standalone package at
// github.com/shurcooL/sanitized_anchor_name. It can be useful for clients
// that want a small package and don't need full functionality of blackfriday.
package blackfriday
// NOTE: Keep Sanitized Anchor Name algorithm in sync with package
// github.com/shurcooL/sanitized_anchor_name.
// Otherwise, users of sanitized_anchor_name will get anchor names
// that are incompatible with those generated by blackfriday.
package blackfriday
// Extracted from https://html.spec.whatwg.org/multipage/entities.json
var entities = map[string]bool{
"&AElig": true,
"&AElig;": true,
"&AMP": true,
"&AMP;": true,
"&Aacute": true,
"&Aacute;": true,
"&Abreve;": true,
"&Acirc": true,
"&Acirc;": true,
"&Acy;": true,
"&Afr;": true,
"&Agrave": true,
"&Agrave;": true,
"&Alpha;": true,
"&Amacr;": true,
"&And;": true,
"&Aogon;": true,
"&Aopf;": true,
"&ApplyFunction;": true,
"&Aring": true,
"&Aring;": true,
"&Ascr;": true,
"&Assign;": true,
"&Atilde": true,
"&Atilde;": true,
"&Auml": true,
"&Auml;": true,
"&Backslash;": true,
"&Barv;": true,
"&Barwed;": true,
"&Bcy;": true,
"&Because;": true,
"&Bernoullis;": true,
"&Beta;": true,
"&Bfr;": true,
"&Bopf;": true,
"&Breve;": true,
"&Bscr;": true,
"&Bumpeq;": true,
"&CHcy;": true,
"&COPY": true,
"&COPY;": true,
"&Cacute;": true,
"&Cap;": true,
"&CapitalDifferentialD;": true,
"&Cayleys;": true,
"&Ccaron;": true,
"&Ccedil": true,
"&Ccedil;": true,
"&Ccirc;": true,
"&Cconint;": true,
"&Cdot;": true,
"&Cedilla;": true,
"&CenterDot;": true,
"&Cfr;": true,
"&Chi;": true,
"&CircleDot;": true,
"&CircleMinus;": true,
"&CirclePlus;": true,
"&CircleTimes;": true,
"&ClockwiseContourIntegral;": true,
"&CloseCurlyDoubleQuote;": true,
"&CloseCurlyQuote;": true,
"&Colon;": true,
"&Colone;": true,
"&Congruent;": true,
"&Conint;": true,
"&ContourIntegral;": true,
"&Copf;": true,
"&Coproduct;": true,
"&CounterClockwiseContourIntegral;": true,
"&Cross;": true,
"&Cscr;": true,
"&Cup;": true,
"&CupCap;": true,
"&DD;": true,
"&DDotrahd;": true,
"&DJcy;": true,
"&DScy;": true,
"&DZcy;": true,
"&Dagger;": true,
"&Darr;": true,
"&Dashv;": true,
"&Dcaron;": true,
"&Dcy;": true,
"&Del;": true,
"&Delta;": true,
"&Dfr;": true,
"&DiacriticalAcute;": true,
"&DiacriticalDot;": true,
"&DiacriticalDoubleAcute;": true,
"&DiacriticalGrave;": true,
"&DiacriticalTilde;": true,
"&Diamond;": true,
"&DifferentialD;": true,
"&Dopf;": true,
"&Dot;": true,
"&DotDot;": true,
"&DotEqual;": true,
"&DoubleContourIntegral;": true,
"&DoubleDot;": true,
"&DoubleDownArrow;": true,
"&DoubleLeftArrow;": true,
"&DoubleLeftRightArrow;": true,
"&DoubleLeftTee;": true,
"&DoubleLongLeftArrow;": true,
"&DoubleLongLeftRightArrow;": true,
"&DoubleLongRightArrow;": true,
"&DoubleRightArrow;": true,
"&DoubleRightTee;": true,
"&DoubleUpArrow;": true,
"&DoubleUpDownArrow;": true,
"&DoubleVerticalBar;": true,
"&DownArrow;": true,
"&DownArrowBar;": true,
"&DownArrowUpArrow;": true,
"&DownBreve;": true,
"&DownLeftRightVector;": true,
"&DownLeftTeeVector;": true,
"&DownLeftVector;": true,
"&DownLeftVectorBar;": true,
"&DownRightTeeVector;": true,
"&DownRightVector;": true,
"&DownRightVectorBar;": true,
"&DownTee;": true,
"&DownTeeArrow;": true,
"&Downarrow;": true,
"&Dscr;": true,
"&Dstrok;": true,
"&ENG;": true,
"&ETH": true,
"&ETH;": true,
"&Eacute": true,
"&Eacute;": true,
"&Ecaron;": true,
"&Ecirc": true,
"&Ecirc;": true,
"&Ecy;": true,
"&Edot;": true,
"&Efr;": true,
"&Egrave": true,
"&Egrave;": true,
"&Element;": true,
"&Emacr;": true,
"&EmptySmallSquare;": true,
"&EmptyVerySmallSquare;": true,
"&Eogon;": true,
"&Eopf;": true,
"&Epsilon;": true,
"&Equal;": true,
"&EqualTilde;": true,
"&Equilibrium;": true,
"&Escr;": true,
"&Esim;": true,
"&Eta;": true,
"&Euml": true,
"&Euml;": true,
"&Exists;": true,
"&ExponentialE;": true,
"&Fcy;": true,
"&Ffr;": true,
"&FilledSmallSquare;": true,
"&FilledVerySmallSquare;": true,
"&Fopf;": true,
"&ForAll;": true,
"&Fouriertrf;": true,
"&Fscr;": true,
"&GJcy;": true,
"&GT": true,
"&GT;": true,
"&Gamma;": true,
"&Gammad;": true,
"&Gbreve;": true,
"&Gcedil;": true,
"&Gcirc;": true,
"&Gcy;": true,
"&Gdot;": true,
"&Gfr;": true,
"&Gg;": true,
"&Gopf;": true,
"&GreaterEqual;": true,
"&GreaterEqualLess;": true,
"&GreaterFullEqual;": true,
"&GreaterGreater;": true,
"&GreaterLess;": true,
"&GreaterSlantEqual;": true,
"&GreaterTilde;": true,
"&Gscr;": true,
"&Gt;": true,
"&HARDcy;": true,
"&Hacek;": true,
"&Hat;": true,
"&Hcirc;": true,
"&Hfr;": true,
"&HilbertSpace;": true,
"&Hopf;": true,
"&HorizontalLine;": true,
"&Hscr;": true,
"&Hstrok;": true,
"&HumpDownHump;": true,
"&HumpEqual;": true,
"&IEcy;": true,
"&IJlig;": true,
"&IOcy;": true,
"&Iacute": true,
"&Iacute;": true,
"&Icirc": true,
"&Icirc;": true,
"&Icy;": true,
"&Idot;": true,
"&Ifr;": true,
"&Igrave": true,
"&Igrave;": true,
"&Im;": true,
"&Imacr;": true,
"&ImaginaryI;": true,
"&Implies;": true,
"&Int;": true,
"&Integral;": true,
"&Intersection;": true,
"&InvisibleComma;": true,
"&InvisibleTimes;": true,
"&Iogon;": true,
"&Iopf;": true,
"&Iota;": true,
"&Iscr;": true,
"&Itilde;": true,
"&Iukcy;": true,
"&Iuml": true,
"&Iuml;": true,
"&Jcirc;": true,
"&Jcy;": true,
"&Jfr;": true,
"&Jopf;": true,
"&Jscr;": true,
"&Jsercy;": true,
"&Jukcy;": true,
"&KHcy;": true,
"&KJcy;": true,
"&Kappa;": true,
"&Kcedil;": true,
"&Kcy;": true,
"&Kfr;": true,
"&Kopf;": true,
"&Kscr;": true,
"&LJcy;": true,
"&LT": true,
"&LT;": true,
"&Lacute;": true,
"&Lambda;": true,
"&Lang;": true,
"&Laplacetrf;": true,
"&Larr;": true,
"&Lcaron;": true,
"&Lcedil;": true,
"&Lcy;": true,
"&LeftAngleBracket;": true,
"&LeftArrow;": true,
"&LeftArrowBar;": true,
"&LeftArrowRightArrow;": true,
"&LeftCeiling;": true,
"&LeftDoubleBracket;": true,
"&LeftDownTeeVector;": true,
"&LeftDownVector;": true,
"&LeftDownVectorBar;": true,
"&LeftFloor;": true,
"&LeftRightArrow;": true,
"&LeftRightVector;": true,
"&LeftTee;": true,
"&LeftTeeArrow;": true,
"&LeftTeeVector;": true,
"&LeftTriangle;": true,
"&LeftTriangleBar;": true,
"&LeftTriangleEqual;": true,
"&LeftUpDownVector;": true,
"&LeftUpTeeVector;": true,
"&LeftUpVector;": true,
"&LeftUpVectorBar;": true,
"&LeftVector;": true,
"&LeftVectorBar;": true,
"&Leftarrow;": true,
"&Leftrightarrow;": true,
"&LessEqualGreater;": true,
"&LessFullEqual;": true,
"&LessGreater;": true,
"&LessLess;": true,
"&LessSlantEqual;": true,
"&LessTilde;": true,
"&Lfr;": true,
"&Ll;": true,
"&Lleftarrow;": true,
"&Lmidot;": true,
"&LongLeftArrow;": true,
"&LongLeftRightArrow;": true,
"&LongRightArrow;": true,
"&Longleftarrow;": true,
"&Longleftrightarrow;": true,
"&Longrightarrow;": true,
"&Lopf;": true,
"&LowerLeftArrow;": true,
"&LowerRightArrow;": true,
"&Lscr;": true,
"&Lsh;": true,
"&Lstrok;": true,
"&Lt;": true,
"&Map;": true,
"&Mcy;": true,
"&MediumSpace;": true,
"&Mellintrf;": true,
"&Mfr;": true,
"&MinusPlus;": true,
"&Mopf;": true,
"&Mscr;": true,
"&Mu;": true,
"&NJcy;": true,
"&Nacute;": true,
"&Ncaron;": true,
"&Ncedil;": true,
"&Ncy;": true,
"&NegativeMediumSpace;": true,
"&NegativeThickSpace;": true,
"&NegativeThinSpace;": true,
"&NegativeVeryThinSpace;": true,
"&NestedGreaterGreater;": true,
"&NestedLessLess;": true,
"&NewLine;": true,
"&Nfr;": true,
"&NoBreak;": true,
"&NonBreakingSpace;": true,
"&Nopf;": true,
"&Not;": true,
"&NotCongruent;": true,
"&NotCupCap;": true,
"&NotDoubleVerticalBar;": true,
"&NotElement;": true,
"&NotEqual;": true,
"&NotEqualTilde;": true,
"&NotExists;": true,
"&NotGreater;": true,
"&NotGreaterEqual;": true,
"&NotGreaterFullEqual;": true,
"&NotGreaterGreater;": true,
"&NotGreaterLess;": true,
"&NotGreaterSlantEqual;": true,
"&NotGreaterTilde;": true,
"&NotHumpDownHump;": true,
"&NotHumpEqual;": true,
"&NotLeftTriangle;": true,
"&NotLeftTriangleBar;": true,
"&NotLeftTriangleEqual;": true,
"&NotLess;": true,
"&NotLessEqual;": true,
"&NotLessGreater;": true,
"&NotLessLess;": true,
"&NotLessSlantEqual;": true,
"&NotLessTilde;": true,
"&NotNestedGreaterGreater;": true,
"&NotNestedLessLess;": true,
"&NotPrecedes;": true,
"&NotPrecedesEqual;": true,
"&NotPrecedesSlantEqual;": true,
"&NotReverseElement;": true,
"&NotRightTriangle;": true,
"&NotRightTriangleBar;": true,
"&NotRightTriangleEqual;": true,
"&NotSquareSubset;": true,
"&NotSquareSubsetEqual;": true,
"&NotSquareSuperset;": true,
"&NotSquareSupersetEqual;": true,
"&NotSubset;": true,
"&NotSubsetEqual;": true,
"&NotSucceeds;": true,
"&NotSucceedsEqual;": true,
"&NotSucceedsSlantEqual;": true,
"&NotSucceedsTilde;": true,
"&NotSuperset;": true,
"&NotSupersetEqual;": true,
"&NotTilde;": true,
"&NotTildeEqual;": true,
"&NotTildeFullEqual;": true,
"&NotTildeTilde;": true,
"&NotVerticalBar;": true,
"&Nscr;": true,
"&Ntilde": true,
"&Ntilde;": true,
"&Nu;": true,
"&OElig;": true,
"&Oacute": true,
"&Oacute;": true,
"&Ocirc": true,
"&Ocirc;": true,
"&Ocy;": true,
"&Odblac;": true,
"&Ofr;": true,
"&Ograve": true,
"&Ograve;": true,
"&Omacr;": true,
"&Omega;": true,
"&Omicron;": true,
"&Oopf;": true,
"&OpenCurlyDoubleQuote;": true,
"&OpenCurlyQuote;": true,
"&Or;": true,
"&Oscr;": true,
"&Oslash": true,
"&Oslash;": true,
"&Otilde": true,
"&Otilde;": true,
"&Otimes;": true,
"&Ouml": true,
"&Ouml;": true,
"&OverBar;": true,
"&OverBrace;": true,
"&OverBracket;": true,
"&OverParenthesis;": true,
"&PartialD;": true,
"&Pcy;": true,
"&Pfr;": true,
"&Phi;": true,
"&Pi;": true,
"&PlusMinus;": true,
"&Poincareplane;": true,
"&Popf;": true,
"&Pr;": true,
"&Precedes;": true,
"&PrecedesEqual;": true,
"&PrecedesSlantEqual;": true,
"&PrecedesTilde;": true,
"&Prime;": true,
"&Product;": true,
"&Proportion;": true,
"&Proportional;": true,
"&Pscr;": true,
"&Psi;": true,
"&QUOT": true,
"&QUOT;": true,
"&Qfr;": true,
"&Qopf;": true,
"&Qscr;": true,
"&RBarr;": true,
"&REG": true,
"&REG;": true,
"&Racute;": true,
"&Rang;": true,
"&Rarr;": true,
"&Rarrtl;": true,
"&Rcaron;": true,
"&Rcedil;": true,
"&Rcy;": true,
"&Re;": true,
"&ReverseElement;": true,
"&ReverseEquilibrium;": true,
"&ReverseUpEquilibrium;": true,
"&Rfr;": true,
"&Rho;": true,
"&RightAngleBracket;": true,
"&RightArrow;": true,
"&RightArrowBar;": true,
"&RightArrowLeftArrow;": true,
"&RightCeiling;": true,
"&RightDoubleBracket;": true,
"&RightDownTeeVector;": true,
"&RightDownVector;": true,
"&RightDownVectorBar;": true,
"&RightFloor;": true,
"&RightTee;": true,
"&RightTeeArrow;": true,
"&RightTeeVector;": true,
"&RightTriangle;": true,
"&RightTriangleBar;": true,
"&RightTriangleEqual;": true,
"&RightUpDownVector;": true,
"&RightUpTeeVector;": true,
"&RightUpVector;": true,
"&RightUpVectorBar;": true,
"&RightVector;": true,
"&RightVectorBar;": true,
"&Rightarrow;": true,
"&Ropf;": true,
"&RoundImplies;": true,
"&Rrightarrow;": true,
"&Rscr;": true,
"&Rsh;": true,
"&RuleDelayed;": true,
"&SHCHcy;": true,
"&SHcy;": true,
"&SOFTcy;": true,
"&Sacute;": true,
"&Sc;": true,
"&Scaron;": true,
"&Scedil;": true,
"&Scirc;": true,
"&Scy;": true,
"&Sfr;": true,
"&ShortDownArrow;": true,
"&ShortLeftArrow;": true,
"&ShortRightArrow;": true,
"&ShortUpArrow;": true,
"&Sigma;": true,
"&SmallCircle;": true,
"&Sopf;": true,
"&Sqrt;": true,
"&Square;": true,
"&SquareIntersection;": true,
"&SquareSubset;": true,
"&SquareSubsetEqual;": true,
"&SquareSuperset;": true,
"&SquareSupersetEqual;": true,
"&SquareUnion;": true,
"&Sscr;": true,
"&Star;": true,
"&Sub;": true,
"&Subset;": true,
"&SubsetEqual;": true,
"&Succeeds;": true,
"&SucceedsEqual;": true,
"&SucceedsSlantEqual;": true,
"&SucceedsTilde;": true,
"&SuchThat;": true,
"&Sum;": true,
"&Sup;": true,
"&Superset;": true,
"&SupersetEqual;": true,
"&Supset;": true,
"&THORN": true,
"&THORN;": true,
"&TRADE;": true,
"&TSHcy;": true,
"&TScy;": true,
"&Tab;": true,
"&Tau;": true,
"&Tcaron;": true,
"&Tcedil;": true,
"&Tcy;": true,
"&Tfr;": true,
"&Therefore;": true,
"&Theta;": true,
"&ThickSpace;": true,
"&ThinSpace;": true,
"&Tilde;": true,
"&TildeEqual;": true,
"&TildeFullEqual;": true,
"&TildeTilde;": true,
"&Topf;": true,
"&TripleDot;": true,
"&Tscr;": true,
"&Tstrok;": true,
"&Uacute": true,
"&Uacute;": true,
"&Uarr;": true,
"&Uarrocir;": true,
"&Ubrcy;": true,
"&Ubreve;": true,
"&Ucirc": true,
"&Ucirc;": true,
"&Ucy;": true,
"&Udblac;": true,
"&Ufr;": true,
"&Ugrave": true,
"&Ugrave;": true,
"&Umacr;": true,
"&UnderBar;": true,
"&UnderBrace;": true,
"&UnderBracket;": true,
"&UnderParenthesis;": true,
"&Union;": true,
"&UnionPlus;": true,
"&Uogon;": true,
"&Uopf;": true,
"&UpArrow;": true,
"&UpArrowBar;": true,
"&UpArrowDownArrow;": true,
"&UpDownArrow;": true,
"&UpEquilibrium;": true,
"&UpTee;": true,
"&UpTeeArrow;": true,
"&Uparrow;": true,
"&Updownarrow;": true,
"&UpperLeftArrow;": true,
"&UpperRightArrow;": true,
"&Upsi;": true,
"&Upsilon;": true,
"&Uring;": true,
"&Uscr;": true,
"&Utilde;": true,
"&Uuml": true,
"&Uuml;": true,
"&VDash;": true,
"&Vbar;": true,
"&Vcy;": true,
"&Vdash;": true,
"&Vdashl;": true,
"&Vee;": true,
"&Verbar;": true,
"&Vert;": true,
"&VerticalBar;": true,
"&VerticalLine;": true,
"&VerticalSeparator;": true,
"&VerticalTilde;": true,
"&VeryThinSpace;": true,
"&Vfr;": true,
"&Vopf;": true,
"&Vscr;": true,
"&Vvdash;": true,
"&Wcirc;": true,
"&Wedge;": true,
"&Wfr;": true,
"&Wopf;": true,
"&Wscr;": true,
"&Xfr;": true,
"&Xi;": true,
"&Xopf;": true,
"&Xscr;": true,
"&YAcy;": true,
"&YIcy;": true,
"&YUcy;": true,
"&Yacute": true,
"&Yacute;": true,
"&Ycirc;": true,
"&Ycy;": true,
"&Yfr;": true,
"&Yopf;": true,
"&Yscr;": true,
"&Yuml;": true,
"&ZHcy;": true,
"&Zacute;": true,
"&Zcaron;": true,
"&Zcy;": true,
"&Zdot;": true,
"&ZeroWidthSpace;": true,
"&Zeta;": true,
"&Zfr;": true,
"&Zopf;": true,
"&Zscr;": true,
"&aacute": true,
"&aacute;": true,
"&abreve;": true,
"&ac;": true,
"&acE;": true,
"&acd;": true,
"&acirc": true,
"&acirc;": true,
"&acute": true,
"&acute;": true,
"&acy;": true,
"&aelig": true,
"&aelig;": true,
"&af;": true,
"&afr;": true,
"&agrave": true,
"&agrave;": true,
"&alefsym;": true,
"&aleph;": true,
"&alpha;": true,
"&amacr;": true,
"&amalg;": true,
"&amp": true,
"&amp;": true,
"&and;": true,
"&andand;": true,
"&andd;": true,
"&andslope;": true,
"&andv;": true,
"&ang;": true,
"&ange;": true,
"&angle;": true,
"&angmsd;": true,
"&angmsdaa;": true,
"&angmsdab;": true,
"&angmsdac;": true,
"&angmsdad;": true,
"&angmsdae;": true,
"&angmsdaf;": true,
"&angmsdag;": true,
"&angmsdah;": true,
"&angrt;": true,
"&angrtvb;": true,
"&angrtvbd;": true,
"&angsph;": true,
"&angst;": true,
"&angzarr;": true,
"&aogon;": true,
"&aopf;": true,
"&ap;": true,
"&apE;": true,
"&apacir;": true,
"&ape;": true,
"&apid;": true,
"&apos;": true,
"&approx;": true,
"&approxeq;": true,
"&aring": true,
"&aring;": true,
"&ascr;": true,
"&ast;": true,
"&asymp;": true,
"&asympeq;": true,
"&atilde": true,
"&atilde;": true,
"&auml": true,
"&auml;": true,
"&awconint;": true,
"&awint;": true,
"&bNot;": true,
"&backcong;": true,
"&backepsilon;": true,
"&backprime;": true,
"&backsim;": true,
"&backsimeq;": true,
"&barvee;": true,
"&barwed;": true,
"&barwedge;": true,
"&bbrk;": true,
"&bbrktbrk;": true,
"&bcong;": true,
"&bcy;": true,
"&bdquo;": true,
"&becaus;": true,
"&because;": true,
"&bemptyv;": true,
"&bepsi;": true,
"&bernou;": true,
"&beta;": true,
"&beth;": true,
"&between;": true,
"&bfr;": true,
"&bigcap;": true,
"&bigcirc;": true,
"&bigcup;": true,
"&bigodot;": true,
"&bigoplus;": true,
"&bigotimes;": true,
"&bigsqcup;": true,
"&bigstar;": true,
"&bigtriangledown;": true,
"&bigtriangleup;": true,
"&biguplus;": true,
"&bigvee;": true,
"&bigwedge;": true,
"&bkarow;": true,
"&blacklozenge;": true,
"&blacksquare;": true,
"&blacktriangle;": true,
"&blacktriangledown;": true,
"&blacktriangleleft;": true,
"&blacktriangleright;": true,
"&blank;": true,
"&blk12;": true,
"&blk14;": true,
"&blk34;": true,
"&block;": true,
"&bne;": true,
"&bnequiv;": true,
"&bnot;": true,
"&bopf;": true,
"&bot;": true,
"&bottom;": true,
"&bowtie;": true,
"&boxDL;": true,
"&boxDR;": true,
"&boxDl;": true,
"&boxDr;": true,
"&boxH;": true,
"&boxHD;": true,
"&boxHU;": true,
"&boxHd;": true,
"&boxHu;": true,
"&boxUL;": true,
"&boxUR;": true,
"&boxUl;": true,
"&boxUr;": true,
"&boxV;": true,
"&boxVH;": true,
"&boxVL;": true,
"&boxVR;": true,
"&boxVh;": true,
"&boxVl;": true,
"&boxVr;": true,
"&boxbox;": true,
"&boxdL;": true,
"&boxdR;": true,
"&boxdl;": true,
"&boxdr;": true,
"&boxh;": true,
"&boxhD;": true,
"&boxhU;": true,
"&boxhd;": true,
"&boxhu;": true,
"&boxminus;": true,
"&boxplus;": true,
"&boxtimes;": true,
"&boxuL;": true,
"&boxuR;": true,
"&boxul;": true,
"&boxur;": true,
"&boxv;": true,
"&boxvH;": true,
"&boxvL;": true,
"&boxvR;": true,
"&boxvh;": true,
"&boxvl;": true,
"&boxvr;": true,
"&bprime;": true,
"&breve;": true,
"&brvbar": true,
"&brvbar;": true,
"&bscr;": true,
"&bsemi;": true,
"&bsim;": true,
"&bsime;": true,
"&bsol;": true,
"&bsolb;": true,
"&bsolhsub;": true,
"&bull;": true,
"&bullet;": true,
"&bump;": true,
"&bumpE;": true,
"&bumpe;": true,
"&bumpeq;": true,
"&cacute;": true,
"&cap;": true,
"&capand;": true,
"&capbrcup;": true,
"&capcap;": true,
"&capcup;": true,
"&capdot;": true,
"&caps;": true,
"&caret;": true,
"&caron;": true,
"&ccaps;": true,
"&ccaron;": true,
"&ccedil": true,
"&ccedil;": true,
"&ccirc;": true,
"&ccups;": true,
"&ccupssm;": true,
"&cdot;": true,
"&cedil": true,
"&cedil;": true,
"&cemptyv;": true,
"&cent": true,
"&cent;": true,
"&centerdot;": true,
"&cfr;": true,
"&chcy;": true,
"&check;": true,
"&checkmark;": true,
"&chi;": true,
"&cir;": true,
"&cirE;": true,
"&circ;": true,
"&circeq;": true,
"&circlearrowleft;": true,
"&circlearrowright;": true,
"&circledR;": true,
"&circledS;": true,
"&circledast;": true,
"&circledcirc;": true,
"&circleddash;": true,
"&cire;": true,
"&cirfnint;": true,
"&cirmid;": true,
"&cirscir;": true,
"&clubs;": true,
"&clubsuit;": true,
"&colon;": true,
"&colone;": true,
"&coloneq;": true,
"&comma;": true,
"&commat;": true,
"&comp;": true,
"&compfn;": true,
"&complement;": true,
"&complexes;": true,
"&cong;": true,
"&congdot;": true,
"&conint;": true,
"&copf;": true,
"&coprod;": true,
"&copy": true,
"&copy;": true,
"&copysr;": true,
"&crarr;": true,
"&cross;": true,
"&cscr;": true,
"&csub;": true,
"&csube;": true,
"&csup;": true,
"&csupe;": true,
"&ctdot;": true,
"&cudarrl;": true,
"&cudarrr;": true,
"&cuepr;": true,
"&cuesc;": true,
"&cularr;": true,
"&cularrp;": true,
"&cup;": true,
"&cupbrcap;": true,
"&cupcap;": true,
"&cupcup;": true,
"&cupdot;": true,
"&cupor;": true,
"&cups;": true,
"&curarr;": true,
"&curarrm;": true,
"&curlyeqprec;": true,
"&curlyeqsucc;": true,
"&curlyvee;": true,
"&curlywedge;": true,
"&curren": true,
"&curren;": true,
"&curvearrowleft;": true,
"&curvearrowright;": true,
"&cuvee;": true,
"&cuwed;": true,
"&cwconint;": true,
"&cwint;": true,
"&cylcty;": true,
"&dArr;": true,
"&dHar;": true,
"&dagger;": true,
"&daleth;": true,
"&darr;": true,
"&dash;": true,
"&dashv;": true,
"&dbkarow;": true,
"&dblac;": true,
"&dcaron;": true,
"&dcy;": true,
"&dd;": true,
"&ddagger;": true,
"&ddarr;": true,
"&ddotseq;": true,
"&deg": true,
"&deg;": true,
"&delta;": true,
"&demptyv;": true,
"&dfisht;": true,
"&dfr;": true,
"&dharl;": true,
"&dharr;": true,
"&diam;": true,
"&diamond;": true,
"&diamondsuit;": true,
"&diams;": true,
"&die;": true,
"&digamma;": true,
"&disin;": true,
"&div;": true,
"&divide": true,
"&divide;": true,
"&divideontimes;": true,
"&divonx;": true,
"&djcy;": true,
"&dlcorn;": true,
"&dlcrop;": true,
"&dollar;": true,
"&dopf;": true,
"&dot;": true,
"&doteq;": true,
"&doteqdot;": true,
"&dotminus;": true,
"&dotplus;": true,
"&dotsquare;": true,
"&doublebarwedge;": true,
"&downarrow;": true,
"&downdownarrows;": true,
"&downharpoonleft;": true,
"&downharpoonright;": true,
"&drbkarow;": true,
"&drcorn;": true,
"&drcrop;": true,
"&dscr;": true,
"&dscy;": true,
"&dsol;": true,
"&dstrok;": true,
"&dtdot;": true,
"&dtri;": true,
"&dtrif;": true,
"&duarr;": true,
"&duhar;": true,
"&dwangle;": true,
"&dzcy;": true,
"&dzigrarr;": true,
"&eDDot;": true,
"&eDot;": true,
"&eacute": true,
"&eacute;": true,
"&easter;": true,
"&ecaron;": true,
"&ecir;": true,
"&ecirc": true,
"&ecirc;": true,
"&ecolon;": true,
"&ecy;": true,
"&edot;": true,
"&ee;": true,
"&efDot;": true,
"&efr;": true,
"&eg;": true,
"&egrave": true,
"&egrave;": true,
"&egs;": true,
"&egsdot;": true,
"&el;": true,
"&elinters;": true,
"&ell;": true,
"&els;": true,
"&elsdot;": true,
"&emacr;": true,
"&empty;": true,
"&emptyset;": true,
"&emptyv;": true,
"&emsp13;": true,
"&emsp14;": true,
"&emsp;": true,
"&eng;": true,
"&ensp;": true,
"&eogon;": true,
"&eopf;": true,
"&epar;": true,
"&eparsl;": true,
"&eplus;": true,
"&epsi;": true,
"&epsilon;": true,
"&epsiv;": true,
"&eqcirc;": true,
"&eqcolon;": true,
"&eqsim;": true,
"&eqslantgtr;": true,
"&eqslantless;": true,
"&equals;": true,
"&equest;": true,
"&equiv;": true,
"&equivDD;": true,
"&eqvparsl;": true,
"&erDot;": true,
"&erarr;": true,
"&escr;": true,
"&esdot;": true,
"&esim;": true,
"&eta;": true,
"&eth": true,
"&eth;": true,
"&euml": true,
"&euml;": true,
"&euro;": true,
"&excl;": true,
"&exist;": true,
"&expectation;": true,
"&exponentiale;": true,
"&fallingdotseq;": true,
"&fcy;": true,
"&female;": true,
"&ffilig;": true,
"&fflig;": true,
"&ffllig;": true,
"&ffr;": true,
"&filig;": true,
"&fjlig;": true,
"&flat;": true,
"&fllig;": true,
"&fltns;": true,
"&fnof;": true,
"&fopf;": true,
"&forall;": true,
"&fork;": true,
"&forkv;": true,
"&fpartint;": true,
"&frac12": true,
"&frac12;": true,
"&frac13;": true,
"&frac14": true,
"&frac14;": true,
"&frac15;": true,
"&frac16;": true,
"&frac18;": true,
"&frac23;": true,
"&frac25;": true,
"&frac34": true,
"&frac34;": true,
"&frac35;": true,
"&frac38;": true,
"&frac45;": true,
"&frac56;": true,
"&frac58;": true,
"&frac78;": true,
"&frasl;": true,
"&frown;": true,
"&fscr;": true,
"&gE;": true,
"&gEl;": true,
"&gacute;": true,
"&gamma;": true,
"&gammad;": true,
"&gap;": true,
"&gbreve;": true,
"&gcirc;": true,
"&gcy;": true,
"&gdot;": true,
"&ge;": true,
"&gel;": true,
"&geq;": true,
"&geqq;": true,
"&geqslant;": true,
"&ges;": true,
"&gescc;": true,
"&gesdot;": true,
"&gesdoto;": true,
"&gesdotol;": true,
"&gesl;": true,
"&gesles;": true,
"&gfr;": true,
"&gg;": true,
"&ggg;": true,
"&gimel;": true,
"&gjcy;": true,
"&gl;": true,
"&glE;": true,
"&gla;": true,
"&glj;": true,
"&gnE;": true,
"&gnap;": true,
"&gnapprox;": true,
"&gne;": true,
"&gneq;": true,
"&gneqq;": true,
"&gnsim;": true,
"&gopf;": true,
"&grave;": true,
"&gscr;": true,
"&gsim;": true,
"&gsime;": true,
"&gsiml;": true,
"&gt": true,
"&gt;": true,
"&gtcc;": true,
"&gtcir;": true,
"&gtdot;": true,
"&gtlPar;": true,
"&gtquest;": true,
"&gtrapprox;": true,
"&gtrarr;": true,
"&gtrdot;": true,
"&gtreqless;": true,
"&gtreqqless;": true,
"&gtrless;": true,
"&gtrsim;": true,
"&gvertneqq;": true,
"&gvnE;": true,
"&hArr;": true,
"&hairsp;": true,
"&half;": true,
"&hamilt;": true,
"&hardcy;": true,
"&harr;": true,
"&harrcir;": true,
"&harrw;": true,
"&hbar;": true,
"&hcirc;": true,
"&hearts;": true,
"&heartsuit;": true,
"&hellip;": true,
"&hercon;": true,
"&hfr;": true,
"&hksearow;": true,
"&hkswarow;": true,
"&hoarr;": true,
"&homtht;": true,
"&hookleftarrow;": true,
"&hookrightarrow;": true,
"&hopf;": true,
"&horbar;": true,
"&hscr;": true,
"&hslash;": true,
"&hstrok;": true,
"&hybull;": true,
"&hyphen;": true,
"&iacute": true,
"&iacute;": true,
"&ic;": true,
"&icirc": true,
"&icirc;": true,
"&icy;": true,
"&iecy;": true,
"&iexcl": true,
"&iexcl;": true,
"&iff;": true,
"&ifr;": true,
"&igrave": true,
"&igrave;": true,
"&ii;": true,
"&iiiint;": true,
"&iiint;": true,
"&iinfin;": true,
"&iiota;": true,
"&ijlig;": true,
"&imacr;": true,
"&image;": true,
"&imagline;": true,
"&imagpart;": true,
"&imath;": true,
"&imof;": true,
"&imped;": true,
"&in;": true,
"&incare;": true,
"&infin;": true,
"&infintie;": true,
"&inodot;": true,
"&int;": true,
"&intcal;": true,
"&integers;": true,
"&intercal;": true,
"&intlarhk;": true,
"&intprod;": true,
"&iocy;": true,
"&iogon;": true,
"&iopf;": true,
"&iota;": true,
"&iprod;": true,
"&iquest": true,
"&iquest;": true,
"&iscr;": true,
"&isin;": true,
"&isinE;": true,
"&isindot;": true,
"&isins;": true,
"&isinsv;": true,
"&isinv;": true,
"&it;": true,
"&itilde;": true,
"&iukcy;": true,
"&iuml": true,
"&iuml;": true,
"&jcirc;": true,
"&jcy;": true,
"&jfr;": true,
"&jmath;": true,
"&jopf;": true,
"&jscr;": true,
"&jsercy;": true,
"&jukcy;": true,
"&kappa;": true,
"&kappav;": true,
"&kcedil;": true,
"&kcy;": true,
"&kfr;": true,
"&kgreen;": true,
"&khcy;": true,
"&kjcy;": true,
"&kopf;": true,
"&kscr;": true,
"&lAarr;": true,
"&lArr;": true,
"&lAtail;": true,
"&lBarr;": true,
"&lE;": true,
"&lEg;": true,
"&lHar;": true,
"&lacute;": true,
"&laemptyv;": true,
"&lagran;": true,
"&lambda;": true,
"&lang;": true,
"&langd;": true,
"&langle;": true,
"&lap;": true,
"&laquo": true,
"&laquo;": true,
"&larr;": true,
"&larrb;": true,
"&larrbfs;": true,
"&larrfs;": true,
"&larrhk;": true,
"&larrlp;": true,
"&larrpl;": true,
"&larrsim;": true,
"&larrtl;": true,
"&lat;": true,
"&latail;": true,
"&late;": true,
"&lates;": true,
"&lbarr;": true,
"&lbbrk;": true,
"&lbrace;": true,
"&lbrack;": true,
"&lbrke;": true,
"&lbrksld;": true,
"&lbrkslu;": true,
"&lcaron;": true,
"&lcedil;": true,
"&lceil;": true,
"&lcub;": true,
"&lcy;": true,
"&ldca;": true,
"&ldquo;": true,
"&ldquor;": true,
"&ldrdhar;": true,
"&ldrushar;": true,
"&ldsh;": true,
"&le;": true,
"&leftarrow;": true,
"&leftarrowtail;": true,
"&leftharpoondown;": true,
"&leftharpoonup;": true,
"&leftleftarrows;": true,
"&leftrightarrow;": true,
"&leftrightarrows;": true,
"&leftrightharpoons;": true,
"&leftrightsquigarrow;": true,
"&leftthreetimes;": true,
"&leg;": true,
"&leq;": true,
"&leqq;": true,
"&leqslant;": true,
"&les;": true,
"&lescc;": true,
"&lesdot;": true,
"&lesdoto;": true,
"&lesdotor;": true,
"&lesg;": true,
"&lesges;": true,
"&lessapprox;": true,
"&lessdot;": true,
"&lesseqgtr;": true,
"&lesseqqgtr;": true,
"&lessgtr;": true,
"&lesssim;": true,
"&lfisht;": true,
"&lfloor;": true,
"&lfr;": true,
"&lg;": true,
"&lgE;": true,
"&lhard;": true,
"&lharu;": true,
"&lharul;": true,
"&lhblk;": true,
"&ljcy;": true,
"&ll;": true,
"&llarr;": true,
"&llcorner;": true,
"&llhard;": true,
"&lltri;": true,
"&lmidot;": true,
"&lmoust;": true,
"&lmoustache;": true,
"&lnE;": true,
"&lnap;": true,
"&lnapprox;": true,
"&lne;": true,
"&lneq;": true,
"&lneqq;": true,
"&lnsim;": true,
"&loang;": true,
"&loarr;": true,
"&lobrk;": true,
"&longleftarrow;": true,
"&longleftrightarrow;": true,
"&longmapsto;": true,
"&longrightarrow;": true,
"&looparrowleft;": true,
"&looparrowright;": true,
"&lopar;": true,
"&lopf;": true,
"&loplus;": true,
"&lotimes;": true,
"&lowast;": true,
"&lowbar;": true,
"&loz;": true,
"&lozenge;": true,
"&lozf;": true,
"&lpar;": true,
"&lparlt;": true,
"&lrarr;": true,
"&lrcorner;": true,
"&lrhar;": true,
"&lrhard;": true,
"&lrm;": true,
"&lrtri;": true,
"&lsaquo;": true,
"&lscr;": true,
"&lsh;": true,
"&lsim;": true,
"&lsime;": true,
"&lsimg;": true,
"&lsqb;": true,
"&lsquo;": true,
"&lsquor;": true,
"&lstrok;": true,
"&lt": true,
"&lt;": true,
"&ltcc;": true,
"&ltcir;": true,
"&ltdot;": true,
"&lthree;": true,
"&ltimes;": true,
"&ltlarr;": true,
"&ltquest;": true,
"&ltrPar;": true,
"&ltri;": true,
"&ltrie;": true,
"&ltrif;": true,
"&lurdshar;": true,
"&luruhar;": true,
"&lvertneqq;": true,
"&lvnE;": true,
"&mDDot;": true,
"&macr": true,
"&macr;": true,
"&male;": true,
"&malt;": true,
"&maltese;": true,
"&map;": true,
"&mapsto;": true,
"&mapstodown;": true,
"&mapstoleft;": true,
"&mapstoup;": true,
"&marker;": true,
"&mcomma;": true,
"&mcy;": true,
"&mdash;": true,
"&measuredangle;": true,
"&mfr;": true,
"&mho;": true,
"&micro": true,
"&micro;": true,
"&mid;": true,
"&midast;": true,
"&midcir;": true,
"&middot": true,
"&middot;": true,
"&minus;": true,
"&minusb;": true,
"&minusd;": true,
"&minusdu;": true,
"&mlcp;": true,
"&mldr;": true,
"&mnplus;": true,
"&models;": true,
"&mopf;": true,
"&mp;": true,
"&mscr;": true,
"&mstpos;": true,
"&mu;": true,
"&multimap;": true,
"&mumap;": true,
"&nGg;": true,
"&nGt;": true,
"&nGtv;": true,
"&nLeftarrow;": true,
"&nLeftrightarrow;": true,
"&nLl;": true,
"&nLt;": true,
"&nLtv;": true,
"&nRightarrow;": true,
"&nVDash;": true,
"&nVdash;": true,
"&nabla;": true,
"&nacute;": true,
"&nang;": true,
"&nap;": true,
"&napE;": true,
"&napid;": true,
"&napos;": true,
"&napprox;": true,
"&natur;": true,
"&natural;": true,
"&naturals;": true,
"&nbsp": true,
"&nbsp;": true,
"&nbump;": true,
"&nbumpe;": true,
"&ncap;": true,
"&ncaron;": true,
"&ncedil;": true,
"&ncong;": true,
"&ncongdot;": true,
"&ncup;": true,
"&ncy;": true,
"&ndash;": true,
"&ne;": true,
"&neArr;": true,
"&nearhk;": true,
"&nearr;": true,
"&nearrow;": true,
"&nedot;": true,
"&nequiv;": true,
"&nesear;": true,
"&nesim;": true,
"&nexist;": true,
"&nexists;": true,
"&nfr;": true,
"&ngE;": true,
"&nge;": true,
"&ngeq;": true,
"&ngeqq;": true,
"&ngeqslant;": true,
"&nges;": true,
"&ngsim;": true,
"&ngt;": true,
"&ngtr;": true,
"&nhArr;": true,
"&nharr;": true,
"&nhpar;": true,
"&ni;": true,
"&nis;": true,
"&nisd;": true,
"&niv;": true,
"&njcy;": true,
"&nlArr;": true,
"&nlE;": true,
"&nlarr;": true,
"&nldr;": true,
"&nle;": true,
"&nleftarrow;": true,
"&nleftrightarrow;": true,
"&nleq;": true,
"&nleqq;": true,
"&nleqslant;": true,
"&nles;": true,
"&nless;": true,
"&nlsim;": true,
"&nlt;": true,
"&nltri;": true,
"&nltrie;": true,
"&nmid;": true,
"&nopf;": true,
"&not": true,
"&not;": true,
"&notin;": true,
"&notinE;": true,
"&notindot;": true,
"&notinva;": true,
"&notinvb;": true,
"&notinvc;": true,
"&notni;": true,
"&notniva;": true,
"&notnivb;": true,
"&notnivc;": true,
"&npar;": true,
"&nparallel;": true,
"&nparsl;": true,
"&npart;": true,
"&npolint;": true,
"&npr;": true,
"&nprcue;": true,
"&npre;": true,
"&nprec;": true,
"&npreceq;": true,
"&nrArr;": true,
"&nrarr;": true,
"&nrarrc;": true,
"&nrarrw;": true,
"&nrightarrow;": true,
"&nrtri;": true,
"&nrtrie;": true,
"&nsc;": true,
"&nsccue;": true,
"&nsce;": true,
"&nscr;": true,
"&nshortmid;": true,
"&nshortparallel;": true,
"&nsim;": true,
"&nsime;": true,
"&nsimeq;": true,
"&nsmid;": true,
"&nspar;": true,
"&nsqsube;": true,
"&nsqsupe;": true,
"&nsub;": true,
"&nsubE;": true,
"&nsube;": true,
"&nsubset;": true,
"&nsubseteq;": true,
"&nsubseteqq;": true,
"&nsucc;": true,
"&nsucceq;": true,
"&nsup;": true,
"&nsupE;": true,
"&nsupe;": true,
"&nsupset;": true,
"&nsupseteq;": true,
"&nsupseteqq;": true,
"&ntgl;": true,
"&ntilde": true,
"&ntilde;": true,
"&ntlg;": true,
"&ntriangleleft;": true,
"&ntrianglelefteq;": true,
"&ntriangleright;": true,
"&ntrianglerighteq;": true,
"&nu;": true,
"&num;": true,
"&numero;": true,
"&numsp;": true,
"&nvDash;": true,
"&nvHarr;": true,
"&nvap;": true,
"&nvdash;": true,
"&nvge;": true,
"&nvgt;": true,
"&nvinfin;": true,
"&nvlArr;": true,
"&nvle;": true,
"&nvlt;": true,
"&nvltrie;": true,
"&nvrArr;": true,
"&nvrtrie;": true,
"&nvsim;": true,
"&nwArr;": true,
"&nwarhk;": true,
"&nwarr;": true,
"&nwarrow;": true,
"&nwnear;": true,
"&oS;": true,
"&oacute": true,
"&oacute;": true,
"&oast;": true,
"&ocir;": true,
"&ocirc": true,
"&ocirc;": true,
"&ocy;": true,
"&odash;": true,
"&odblac;": true,
"&odiv;": true,
"&odot;": true,
"&odsold;": true,
"&oelig;": true,
"&ofcir;": true,
"&ofr;": true,
"&ogon;": true,
"&ograve": true,
"&ograve;": true,
"&ogt;": true,
"&ohbar;": true,
"&ohm;": true,
"&oint;": true,
"&olarr;": true,
"&olcir;": true,
"&olcross;": true,
"&oline;": true,
"&olt;": true,
"&omacr;": true,
"&omega;": true,
"&omicron;": true,
"&omid;": true,
"&ominus;": true,
"&oopf;": true,
"&opar;": true,
"&operp;": true,
"&oplus;": true,
"&or;": true,
"&orarr;": true,
"&ord;": true,
"&order;": true,
"&orderof;": true,
"&ordf": true,
"&ordf;": true,
"&ordm": true,
"&ordm;": true,
"&origof;": true,
"&oror;": true,
"&orslope;": true,
"&orv;": true,
"&oscr;": true,
"&oslash": true,
"&oslash;": true,
"&osol;": true,
"&otilde": true,
"&otilde;": true,
"&otimes;": true,
"&otimesas;": true,
"&ouml": true,
"&ouml;": true,
"&ovbar;": true,
"&par;": true,
"&para": true,
"&para;": true,
"&parallel;": true,
"&parsim;": true,
"&parsl;": true,
"&part;": true,
"&pcy;": true,
"&percnt;": true,
"&period;": true,
"&permil;": true,
"&perp;": true,
"&pertenk;": true,
"&pfr;": true,
"&phi;": true,
"&phiv;": true,
"&phmmat;": true,
"&phone;": true,
"&pi;": true,
"&pitchfork;": true,
"&piv;": true,
"&planck;": true,
"&planckh;": true,
"&plankv;": true,
"&plus;": true,
"&plusacir;": true,
"&plusb;": true,
"&pluscir;": true,
"&plusdo;": true,
"&plusdu;": true,
"&pluse;": true,
"&plusmn": true,
"&plusmn;": true,
"&plussim;": true,
"&plustwo;": true,
"&pm;": true,
"&pointint;": true,
"&popf;": true,
"&pound": true,
"&pound;": true,
"&pr;": true,
"&prE;": true,
"&prap;": true,
"&prcue;": true,
"&pre;": true,
"&prec;": true,
"&precapprox;": true,
"&preccurlyeq;": true,
"&preceq;": true,
"&precnapprox;": true,
"&precneqq;": true,
"&precnsim;": true,
"&precsim;": true,
"&prime;": true,
"&primes;": true,
"&prnE;": true,
"&prnap;": true,
"&prnsim;": true,
"&prod;": true,
"&profalar;": true,
"&profline;": true,
"&profsurf;": true,
"&prop;": true,
"&propto;": true,
"&prsim;": true,
"&prurel;": true,
"&pscr;": true,
"&psi;": true,
"&puncsp;": true,
"&qfr;": true,
"&qint;": true,
"&qopf;": true,
"&qprime;": true,
"&qscr;": true,
"&quaternions;": true,
"&quatint;": true,
"&quest;": true,
"&questeq;": true,
"&quot": true,
"&quot;": true,
"&rAarr;": true,
"&rArr;": true,
"&rAtail;": true,
"&rBarr;": true,
"&rHar;": true,
"&race;": true,
"&racute;": true,
"&radic;": true,
"&raemptyv;": true,
"&rang;": true,
"&rangd;": true,
"&range;": true,
"&rangle;": true,
"&raquo": true,
"&raquo;": true,
"&rarr;": true,
"&rarrap;": true,
"&rarrb;": true,
"&rarrbfs;": true,
"&rarrc;": true,
"&rarrfs;": true,
"&rarrhk;": true,
"&rarrlp;": true,
"&rarrpl;": true,
"&rarrsim;": true,
"&rarrtl;": true,
"&rarrw;": true,
"&ratail;": true,
"&ratio;": true,
"&rationals;": true,
"&rbarr;": true,
"&rbbrk;": true,
"&rbrace;": true,
"&rbrack;": true,
"&rbrke;": true,
"&rbrksld;": true,
"&rbrkslu;": true,
"&rcaron;": true,
"&rcedil;": true,
"&rceil;": true,
"&rcub;": true,
"&rcy;": true,
"&rdca;": true,
"&rdldhar;": true,
"&rdquo;": true,
"&rdquor;": true,
"&rdsh;": true,
"&real;": true,
"&realine;": true,
"&realpart;": true,
"&reals;": true,
"&rect;": true,
"&reg": true,
"&reg;": true,
"&rfisht;": true,
"&rfloor;": true,
"&rfr;": true,
"&rhard;": true,
"&rharu;": true,
"&rharul;": true,
"&rho;": true,
"&rhov;": true,
"&rightarrow;": true,
"&rightarrowtail;": true,
"&rightharpoondown;": true,
"&rightharpoonup;": true,
"&rightleftarrows;": true,
"&rightleftharpoons;": true,
"&rightrightarrows;": true,
"&rightsquigarrow;": true,
"&rightthreetimes;": true,
"&ring;": true,
"&risingdotseq;": true,
"&rlarr;": true,
"&rlhar;": true,
"&rlm;": true,
"&rmoust;": true,
"&rmoustache;": true,
"&rnmid;": true,
"&roang;": true,
"&roarr;": true,
"&robrk;": true,
"&ropar;": true,
"&ropf;": true,
"&roplus;": true,
"&rotimes;": true,
"&rpar;": true,
"&rpargt;": true,
"&rppolint;": true,
"&rrarr;": true,
"&rsaquo;": true,
"&rscr;": true,
"&rsh;": true,
"&rsqb;": true,
"&rsquo;": true,
"&rsquor;": true,
"&rthree;": true,
"&rtimes;": true,
"&rtri;": true,
"&rtrie;": true,
"&rtrif;": true,
"&rtriltri;": true,
"&ruluhar;": true,
"&rx;": true,
"&sacute;": true,
"&sbquo;": true,
"&sc;": true,
"&scE;": true,
"&scap;": true,
"&scaron;": true,
"&sccue;": true,
"&sce;": true,
"&scedil;": true,
"&scirc;": true,
"&scnE;": true,
"&scnap;": true,
"&scnsim;": true,
"&scpolint;": true,
"&scsim;": true,
"&scy;": true,
"&sdot;": true,
"&sdotb;": true,
"&sdote;": true,
"&seArr;": true,
"&searhk;": true,
"&searr;": true,
"&searrow;": true,
"&sect": true,
"&sect;": true,
"&semi;": true,
"&seswar;": true,
"&setminus;": true,
"&setmn;": true,
"&sext;": true,
"&sfr;": true,
"&sfrown;": true,
"&sharp;": true,
"&shchcy;": true,
"&shcy;": true,
"&shortmid;": true,
"&shortparallel;": true,
"&shy": true,
"&shy;": true,
"&sigma;": true,
"&sigmaf;": true,
"&sigmav;": true,
"&sim;": true,
"&simdot;": true,
"&sime;": true,
"&simeq;": true,
"&simg;": true,
"&simgE;": true,
"&siml;": true,
"&simlE;": true,
"&simne;": true,
"&simplus;": true,
"&simrarr;": true,
"&slarr;": true,
"&smallsetminus;": true,
"&smashp;": true,
"&smeparsl;": true,
"&smid;": true,
"&smile;": true,
"&smt;": true,
"&smte;": true,
"&smtes;": true,
"&softcy;": true,
"&sol;": true,
"&solb;": true,
"&solbar;": true,
"&sopf;": true,
"&spades;": true,
"&spadesuit;": true,
"&spar;": true,
"&sqcap;": true,
"&sqcaps;": true,
"&sqcup;": true,
"&sqcups;": true,
"&sqsub;": true,
"&sqsube;": true,
"&sqsubset;": true,
"&sqsubseteq;": true,
"&sqsup;": true,
"&sqsupe;": true,
"&sqsupset;": true,
"&sqsupseteq;": true,
"&squ;": true,
"&square;": true,
"&squarf;": true,
"&squf;": true,
"&srarr;": true,
"&sscr;": true,
"&ssetmn;": true,
"&ssmile;": true,
"&sstarf;": true,
"&star;": true,
"&starf;": true,
"&straightepsilon;": true,
"&straightphi;": true,
"&strns;": true,
"&sub;": true,
"&subE;": true,
"&subdot;": true,
"&sube;": true,
"&subedot;": true,
"&submult;": true,
"&subnE;": true,
"&subne;": true,
"&subplus;": true,
"&subrarr;": true,
"&subset;": true,
"&subseteq;": true,
"&subseteqq;": true,
"&subsetneq;": true,
"&subsetneqq;": true,
"&subsim;": true,
"&subsub;": true,
"&subsup;": true,
"&succ;": true,
"&succapprox;": true,
"&succcurlyeq;": true,
"&succeq;": true,
"&succnapprox;": true,
"&succneqq;": true,
"&succnsim;": true,
"&succsim;": true,
"&sum;": true,
"&sung;": true,
"&sup1": true,
"&sup1;": true,
"&sup2": true,
"&sup2;": true,
"&sup3": true,
"&sup3;": true,
"&sup;": true,
"&supE;": true,
"&supdot;": true,
"&supdsub;": true,
"&supe;": true,
"&supedot;": true,
"&suphsol;": true,
"&suphsub;": true,
"&suplarr;": true,
"&supmult;": true,
"&supnE;": true,
"&supne;": true,
"&supplus;": true,
"&supset;": true,
"&supseteq;": true,
"&supseteqq;": true,
"&supsetneq;": true,
"&supsetneqq;": true,
"&supsim;": true,
"&supsub;": true,
"&supsup;": true,
"&swArr;": true,
"&swarhk;": true,
"&swarr;": true,
"&swarrow;": true,
"&swnwar;": true,
"&szlig": true,
"&szlig;": true,
"&target;": true,
"&tau;": true,
"&tbrk;": true,
"&tcaron;": true,
"&tcedil;": true,
"&tcy;": true,
"&tdot;": true,
"&telrec;": true,
"&tfr;": true,
"&there4;": true,
"&therefore;": true,
"&theta;": true,
"&thetasym;": true,
"&thetav;": true,
"&thickapprox;": true,
"&thicksim;": true,
"&thinsp;": true,
"&thkap;": true,
"&thksim;": true,
"&thorn": true,
"&thorn;": true,
"&tilde;": true,
"&times": true,
"&times;": true,
"&timesb;": true,
"&timesbar;": true,
"&timesd;": true,
"&tint;": true,
"&toea;": true,
"&top;": true,
"&topbot;": true,
"&topcir;": true,
"&topf;": true,
"&topfork;": true,
"&tosa;": true,
"&tprime;": true,
"&trade;": true,
"&triangle;": true,
"&triangledown;": true,
"&triangleleft;": true,
"&trianglelefteq;": true,
"&triangleq;": true,
"&triangleright;": true,
"&trianglerighteq;": true,
"&tridot;": true,
"&trie;": true,
"&triminus;": true,
"&triplus;": true,
"&trisb;": true,
"&tritime;": true,
"&trpezium;": true,
"&tscr;": true,
"&tscy;": true,
"&tshcy;": true,
"&tstrok;": true,
"&twixt;": true,
"&twoheadleftarrow;": true,
"&twoheadrightarrow;": true,
"&uArr;": true,
"&uHar;": true,
"&uacute": true,
"&uacute;": true,
"&uarr;": true,
"&ubrcy;": true,
"&ubreve;": true,
"&ucirc": true,
"&ucirc;": true,
"&ucy;": true,
"&udarr;": true,
"&udblac;": true,
"&udhar;": true,
"&ufisht;": true,
"&ufr;": true,
"&ugrave": true,
"&ugrave;": true,
"&uharl;": true,
"&uharr;": true,
"&uhblk;": true,
"&ulcorn;": true,
"&ulcorner;": true,
"&ulcrop;": true,
"&ultri;": true,
"&umacr;": true,
"&uml": true,
"&uml;": true,
"&uogon;": true,
"&uopf;": true,
"&uparrow;": true,
"&updownarrow;": true,
"&upharpoonleft;": true,
"&upharpoonright;": true,
"&uplus;": true,
"&upsi;": true,
"&upsih;": true,
"&upsilon;": true,
"&upuparrows;": true,
"&urcorn;": true,
"&urcorner;": true,
"&urcrop;": true,
"&uring;": true,
"&urtri;": true,
"&uscr;": true,
"&utdot;": true,
"&utilde;": true,
"&utri;": true,
"&utrif;": true,
"&uuarr;": true,
"&uuml": true,
"&uuml;": true,
"&uwangle;": true,
"&vArr;": true,
"&vBar;": true,
"&vBarv;": true,
"&vDash;": true,
"&vangrt;": true,
"&varepsilon;": true,
"&varkappa;": true,
"&varnothing;": true,
"&varphi;": true,
"&varpi;": true,
"&varpropto;": true,
"&varr;": true,
"&varrho;": true,
"&varsigma;": true,
"&varsubsetneq;": true,
"&varsubsetneqq;": true,
"&varsupsetneq;": true,
"&varsupsetneqq;": true,
"&vartheta;": true,
"&vartriangleleft;": true,
"&vartriangleright;": true,
"&vcy;": true,
"&vdash;": true,
"&vee;": true,
"&veebar;": true,
"&veeeq;": true,
"&vellip;": true,
"&verbar;": true,
"&vert;": true,
"&vfr;": true,
"&vltri;": true,
"&vnsub;": true,
"&vnsup;": true,
"&vopf;": true,
"&vprop;": true,
"&vrtri;": true,
"&vscr;": true,
"&vsubnE;": true,
"&vsubne;": true,
"&vsupnE;": true,
"&vsupne;": true,
"&vzigzag;": true,
"&wcirc;": true,
"&wedbar;": true,
"&wedge;": true,
"&wedgeq;": true,
"&weierp;": true,
"&wfr;": true,
"&wopf;": true,
"&wp;": true,
"&wr;": true,
"&wreath;": true,
"&wscr;": true,
"&xcap;": true,
"&xcirc;": true,
"&xcup;": true,
"&xdtri;": true,
"&xfr;": true,
"&xhArr;": true,
"&xharr;": true,
"&xi;": true,
"&xlArr;": true,
"&xlarr;": true,
"&xmap;": true,
"&xnis;": true,
"&xodot;": true,
"&xopf;": true,
"&xoplus;": true,
"&xotime;": true,
"&xrArr;": true,
"&xrarr;": true,
"&xscr;": true,
"&xsqcup;": true,
"&xuplus;": true,
"&xutri;": true,
"&xvee;": true,
"&xwedge;": true,
"&yacute": true,
"&yacute;": true,
"&yacy;": true,
"&ycirc;": true,
"&ycy;": true,
"&yen": true,
"&yen;": true,
"&yfr;": true,
"&yicy;": true,
"&yopf;": true,
"&yscr;": true,
"&yucy;": true,
"&yuml": true,
"&yuml;": true,
"&zacute;": true,
"&zcaron;": true,
"&zcy;": true,
"&zdot;": true,
"&zeetrf;": true,
"&zeta;": true,
"&zfr;": true,
"&zhcy;": true,
"&zigrarr;": true,
"&zopf;": true,
"&zscr;": true,
"&zwj;": true,
"&zwnj;": true,
}
package blackfriday
import (
"html"
"io"
)
var htmlEscaper = [256][]byte{
'&': []byte("&amp;"),
'<': []byte("&lt;"),
'>': []byte("&gt;"),
'"': []byte("&quot;"),
}
func escapeHTML(w io.Writer, s []byte) {
escapeEntities(w, s, false)
}
func escapeAllHTML(w io.Writer, s []byte) {
escapeEntities(w, s, true)
}
func escapeEntities(w io.Writer, s []byte, escapeValidEntities bool) {
var start, end int
for end < len(s) {
escSeq := htmlEscaper[s[end]]
if escSeq != nil {
isEntity, entityEnd := nodeIsEntity(s, end)
if isEntity && !escapeValidEntities {
w.Write(s[start : entityEnd+1])
start = entityEnd + 1
} else {
w.Write(s[start:end])
w.Write(escSeq)
start = end + 1
}
}
end++
}
if start < len(s) && end <= len(s) {
w.Write(s[start:end])
}
}
func nodeIsEntity(s []byte, end int) (isEntity bool, endEntityPos int) {
isEntity = false
endEntityPos = end + 1
if s[end] == '&' {
for endEntityPos < len(s) {
if s[endEntityPos] == ';' {
if entities[string(s[end:endEntityPos+1])] {
isEntity = true
break
}
}
if !isalnum(s[endEntityPos]) && s[endEntityPos] != '&' && s[endEntityPos] != '#' {
break
}
endEntityPos++
}
}
return isEntity, endEntityPos
}
func escLink(w io.Writer, text []byte) {
unesc := html.UnescapeString(string(text))
escapeHTML(w, []byte(unesc))
}
//
// Blackfriday Markdown Processor
// Available at http://github.com/russross/blackfriday
//
// Copyright © 2011 Russ Ross <russ@russross.com>.
// Distributed under the Simplified BSD License.
// See README.md for details.
//
//
//
// HTML rendering backend
//
//
package blackfriday
import (
"bytes"
"fmt"
"io"
"regexp"
"strings"
)
// HTMLFlags control optional behavior of HTML renderer.
type HTMLFlags int
// HTML renderer configuration options.
const (
HTMLFlagsNone HTMLFlags = 0
SkipHTML HTMLFlags = 1 << iota // Skip preformatted HTML blocks
SkipImages // Skip embedded images
SkipLinks // Skip all links
Safelink // Only link to trusted protocols
NofollowLinks // Only link with rel="nofollow"
NoreferrerLinks // Only link with rel="noreferrer"
NoopenerLinks // Only link with rel="noopener"
HrefTargetBlank // Add a blank target
CompletePage // Generate a complete HTML page
UseXHTML // Generate XHTML output instead of HTML
FootnoteReturnLinks // Generate a link at the end of a footnote to return to the source
Smartypants // Enable smart punctuation substitutions
SmartypantsFractions // Enable smart fractions (with Smartypants)
SmartypantsDashes // Enable smart dashes (with Smartypants)
SmartypantsLatexDashes // Enable LaTeX-style dashes (with Smartypants)
SmartypantsAngledQuotes // Enable angled double quotes (with Smartypants) for double quotes rendering
SmartypantsQuotesNBSP // Enable « French guillemets » (with Smartypants)
TOC // Generate a table of contents
)
var (
htmlTagRe = regexp.MustCompile("(?i)^" + htmlTag)
)
const (
htmlTag = "(?:" + openTag + "|" + closeTag + "|" + htmlComment + "|" +
processingInstruction + "|" + declaration + "|" + cdata + ")"
closeTag = "</" + tagName + "\\s*[>]"
openTag = "<" + tagName + attribute + "*" + "\\s*/?>"
attribute = "(?:" + "\\s+" + attributeName + attributeValueSpec + "?)"
attributeValue = "(?:" + unquotedValue + "|" + singleQuotedValue + "|" + doubleQuotedValue + ")"
attributeValueSpec = "(?:" + "\\s*=" + "\\s*" + attributeValue + ")"
attributeName = "[a-zA-Z_:][a-zA-Z0-9:._-]*"
cdata = "<!\\[CDATA\\[[\\s\\S]*?\\]\\]>"
declaration = "<![A-Z]+" + "\\s+[^>]*>"
doubleQuotedValue = "\"[^\"]*\""
htmlComment = "<!---->|<!--(?:-?[^>-])(?:-?[^-])*-->"
processingInstruction = "[<][?].*?[?][>]"
singleQuotedValue = "'[^']*'"
tagName = "[A-Za-z][A-Za-z0-9-]*"
unquotedValue = "[^\"'=<>`\\x00-\\x20]+"
)
// HTMLRendererParameters is a collection of supplementary parameters tweaking
// the behavior of various parts of HTML renderer.
type HTMLRendererParameters struct {
// Prepend this text to each relative URL.
AbsolutePrefix string
// Add this text to each footnote anchor, to ensure uniqueness.
FootnoteAnchorPrefix string
// Show this text inside the <a> tag for a footnote return link, if the
// HTML_FOOTNOTE_RETURN_LINKS flag is enabled. If blank, the string
// <sup>[return]</sup> is used.
FootnoteReturnLinkContents string
// If set, add this text to the front of each Heading ID, to ensure
// uniqueness.
HeadingIDPrefix string
// If set, add this text to the back of each Heading ID, to ensure uniqueness.
HeadingIDSuffix string
// Increase heading levels: if the offset is 1, <h1> becomes <h2> etc.
// Negative offset is also valid.
// Resulting levels are clipped between 1 and 6.
HeadingLevelOffset int
Title string // Document title (used if CompletePage is set)
CSS string // Optional CSS file URL (used if CompletePage is set)
Icon string // Optional icon file URL (used if CompletePage is set)
Flags HTMLFlags // Flags allow customizing this renderer's behavior
}
// HTMLRenderer is a type that implements the Renderer interface for HTML output.
//
// Do not create this directly, instead use the NewHTMLRenderer function.
type HTMLRenderer struct {
HTMLRendererParameters
closeTag string // how to end singleton tags: either " />" or ">"
// Track heading IDs to prevent ID collision in a single generation.
headingIDs map[string]int
lastOutputLen int
disableTags int
sr *SPRenderer
}
const (
xhtmlClose = " />"
htmlClose = ">"
)
// NewHTMLRenderer creates and configures an HTMLRenderer object, which
// satisfies the Renderer interface.
func NewHTMLRenderer(params HTMLRendererParameters) *HTMLRenderer {
// configure the rendering engine
closeTag := htmlClose
if params.Flags&UseXHTML != 0 {
closeTag = xhtmlClose
}
if params.FootnoteReturnLinkContents == "" {
// U+FE0E is VARIATION SELECTOR-15.
// It suppresses automatic emoji presentation of the preceding
// U+21A9 LEFTWARDS ARROW WITH HOOK on iOS and iPadOS.
params.FootnoteReturnLinkContents = "<span aria-label='Return'>↩\ufe0e</span>"
}
return &HTMLRenderer{
HTMLRendererParameters: params,
closeTag: closeTag,
headingIDs: make(map[string]int),
sr: NewSmartypantsRenderer(params.Flags),
}
}
func isHTMLTag(tag []byte, tagname string) bool {
found, _ := findHTMLTagPos(tag, tagname)
return found
}
// Look for a character, but ignore it when it's in any kind of quotes, it
// might be JavaScript
func skipUntilCharIgnoreQuotes(html []byte, start int, char byte) int {
inSingleQuote := false
inDoubleQuote := false
inGraveQuote := false
i := start
for i < len(html) {
switch {
case html[i] == char && !inSingleQuote && !inDoubleQuote && !inGraveQuote:
return i
case html[i] == '\'':
inSingleQuote = !inSingleQuote
case html[i] == '"':
inDoubleQuote = !inDoubleQuote
case html[i] == '`':
inGraveQuote = !inGraveQuote
}
i++
}
return start
}
func findHTMLTagPos(tag []byte, tagname string) (bool, int) {
i := 0
if i < len(tag) && tag[0] != '<' {
return false, -1
}
i++
i = skipSpace(tag, i)
if i < len(tag) && tag[i] == '/' {
i++
}
i = skipSpace(tag, i)
j := 0
for ; i < len(tag); i, j = i+1, j+1 {
if j >= len(tagname) {
break
}
if strings.ToLower(string(tag[i]))[0] != tagname[j] {
return false, -1
}
}
if i == len(tag) {
return false, -1
}
rightAngle := skipUntilCharIgnoreQuotes(tag, i, '>')
if rightAngle >= i {
return true, rightAngle
}
return false, -1
}
func skipSpace(tag []byte, i int) int {
for i < len(tag) && isspace(tag[i]) {
i++
}
return i
}
func isRelativeLink(link []byte) (yes bool) {
// a tag begin with '#'
if link[0] == '#' {
return true
}
// link begin with '/' but not '//', the second maybe a protocol relative link
if len(link) >= 2 && link[0] == '/' && link[1] != '/' {
return true
}
// only the root '/'
if len(link) == 1 && link[0] == '/' {
return true
}
// current directory : begin with "./"
if bytes.HasPrefix(link, []byte("./")) {
return true
}
// parent directory : begin with "../"
if bytes.HasPrefix(link, []byte("../")) {
return true
}
return false
}
func (r *HTMLRenderer) ensureUniqueHeadingID(id string) string {
for count, found := r.headingIDs[id]; found; count, found = r.headingIDs[id] {
tmp := fmt.Sprintf("%s-%d", id, count+1)
if _, tmpFound := r.headingIDs[tmp]; !tmpFound {
r.headingIDs[id] = count + 1
id = tmp
} else {
id = id + "-1"
}
}
if _, found := r.headingIDs[id]; !found {
r.headingIDs[id] = 0
}
return id
}
func (r *HTMLRenderer) addAbsPrefix(link []byte) []byte {
if r.AbsolutePrefix != "" && isRelativeLink(link) && link[0] != '.' {
newDest := r.AbsolutePrefix
if link[0] != '/' {
newDest += "/"
}
newDest += string(link)
return []byte(newDest)
}
return link
}
func appendLinkAttrs(attrs []string, flags HTMLFlags, link []byte) []string {
if isRelativeLink(link) {
return attrs
}
val := []string{}
if flags&NofollowLinks != 0 {
val = append(val, "nofollow")
}
if flags&NoreferrerLinks != 0 {
val = append(val, "noreferrer")
}
if flags&NoopenerLinks != 0 {
val = append(val, "noopener")
}
if flags&HrefTargetBlank != 0 {
attrs = append(attrs, "target=\"_blank\"")
}
if len(val) == 0 {
return attrs
}
attr := fmt.Sprintf("rel=%q", strings.Join(val, " "))
return append(attrs, attr)
}
func isMailto(link []byte) bool {
return bytes.HasPrefix(link, []byte("mailto:"))
}
func needSkipLink(flags HTMLFlags, dest []byte) bool {
if flags&SkipLinks != 0 {
return true
}
return flags&Safelink != 0 && !isSafeLink(dest) && !isMailto(dest)
}
func isSmartypantable(node *Node) bool {
pt := node.Parent.Type
return pt != Link && pt != CodeBlock && pt != Code
}
func appendLanguageAttr(attrs []string, info []byte) []string {
if len(info) == 0 {
return attrs
}
endOfLang := bytes.IndexAny(info, "\t ")
if endOfLang < 0 {
endOfLang = len(info)
}
return append(attrs, fmt.Sprintf("class=\"language-%s\"", info[:endOfLang]))
}
func (r *HTMLRenderer) tag(w io.Writer, name []byte, attrs []string) {
w.Write(name)
if len(attrs) > 0 {
w.Write(spaceBytes)
w.Write([]byte(strings.Join(attrs, " ")))
}
w.Write(gtBytes)
r.lastOutputLen = 1
}
func footnoteRef(prefix string, node *Node) []byte {
urlFrag := prefix + string(slugify(node.Destination))
anchor := fmt.Sprintf(`<a href="#fn:%s">%d</a>`, urlFrag, node.NoteID)
return []byte(fmt.Sprintf(`<sup class="footnote-ref" id="fnref:%s">%s</sup>`, urlFrag, anchor))
}
func footnoteItem(prefix string, slug []byte) []byte {
return []byte(fmt.Sprintf(`<li id="fn:%s%s">`, prefix, slug))
}
func footnoteReturnLink(prefix, returnLink string, slug []byte) []byte {
const format = ` <a class="footnote-return" href="#fnref:%s%s">%s</a>`
return []byte(fmt.Sprintf(format, prefix, slug, returnLink))
}
func itemOpenCR(node *Node) bool {
if node.Prev == nil {
return false
}
ld := node.Parent.ListData
return !ld.Tight && ld.ListFlags&ListTypeDefinition == 0
}
func skipParagraphTags(node *Node) bool {
grandparent := node.Parent.Parent
if grandparent == nil || grandparent.Type != List {
return false
}
tightOrTerm := grandparent.Tight || node.Parent.ListFlags&ListTypeTerm != 0
return grandparent.Type == List && tightOrTerm
}
func cellAlignment(align CellAlignFlags) string {
switch align {
case TableAlignmentLeft:
return "left"
case TableAlignmentRight:
return "right"
case TableAlignmentCenter:
return "center"
default:
return ""
}
}
func (r *HTMLRenderer) out(w io.Writer, text []byte) {
if r.disableTags > 0 {
w.Write(htmlTagRe.ReplaceAll(text, []byte{}))
} else {
w.Write(text)
}
r.lastOutputLen = len(text)
}
func (r *HTMLRenderer) cr(w io.Writer) {
if r.lastOutputLen > 0 {
r.out(w, nlBytes)
}
}
var (
nlBytes = []byte{'\n'}
gtBytes = []byte{'>'}
spaceBytes = []byte{' '}
)
var (
brTag = []byte("<br>")
brXHTMLTag = []byte("<br />")
emTag = []byte("<em>")
emCloseTag = []byte("</em>")
strongTag = []byte("<strong>")
strongCloseTag = []byte("</strong>")
delTag = []byte("<del>")
delCloseTag = []byte("</del>")
ttTag = []byte("<tt>")
ttCloseTag = []byte("</tt>")
aTag = []byte("<a")
aCloseTag = []byte("</a>")
preTag = []byte("<pre>")
preCloseTag = []byte("</pre>")
codeTag = []byte("<code>")
codeCloseTag = []byte("</code>")
pTag = []byte("<p>")
pCloseTag = []byte("</p>")
blockquoteTag = []byte("<blockquote>")
blockquoteCloseTag = []byte("</blockquote>")
hrTag = []byte("<hr>")
hrXHTMLTag = []byte("<hr />")
ulTag = []byte("<ul>")
ulCloseTag = []byte("</ul>")
olTag = []byte("<ol>")
olCloseTag = []byte("</ol>")
dlTag = []byte("<dl>")
dlCloseTag = []byte("</dl>")
liTag = []byte("<li>")
liCloseTag = []byte("</li>")
ddTag = []byte("<dd>")
ddCloseTag = []byte("</dd>")
dtTag = []byte("<dt>")
dtCloseTag = []byte("</dt>")
tableTag = []byte("<table>")
tableCloseTag = []byte("</table>")
tdTag = []byte("<td")
tdCloseTag = []byte("</td>")
thTag = []byte("<th")
thCloseTag = []byte("</th>")
theadTag = []byte("<thead>")
theadCloseTag = []byte("</thead>")
tbodyTag = []byte("<tbody>")
tbodyCloseTag = []byte("</tbody>")
trTag = []byte("<tr>")
trCloseTag = []byte("</tr>")
h1Tag = []byte("<h1")
h1CloseTag = []byte("</h1>")
h2Tag = []byte("<h2")
h2CloseTag = []byte("</h2>")
h3Tag = []byte("<h3")
h3CloseTag = []byte("</h3>")
h4Tag = []byte("<h4")
h4CloseTag = []byte("</h4>")
h5Tag = []byte("<h5")
h5CloseTag = []byte("</h5>")
h6Tag = []byte("<h6")
h6CloseTag = []byte("</h6>")
footnotesDivBytes = []byte("\n<div class=\"footnotes\">\n\n")
footnotesCloseDivBytes = []byte("\n</div>\n")
)
func headingTagsFromLevel(level int) ([]byte, []byte) {
if level <= 1 {
return h1Tag, h1CloseTag
}
switch level {
case 2:
return h2Tag, h2CloseTag
case 3:
return h3Tag, h3CloseTag
case 4:
return h4Tag, h4CloseTag
case 5:
return h5Tag, h5CloseTag
}
return h6Tag, h6CloseTag
}
func (r *HTMLRenderer) outHRTag(w io.Writer) {
if r.Flags&UseXHTML == 0 {
r.out(w, hrTag)
} else {
r.out(w, hrXHTMLTag)
}
}
// RenderNode is a default renderer of a single node of a syntax tree. For
// block nodes it will be called twice: first time with entering=true, second
// time with entering=false, so that it could know when it's working on an open
// tag and when on close. It writes the result to w.
//
// The return value is a way to tell the calling walker to adjust its walk
// pattern: e.g. it can terminate the traversal by returning Terminate. Or it
// can ask the walker to skip a subtree of this node by returning SkipChildren.
// The typical behavior is to return GoToNext, which asks for the usual
// traversal to the next node.
func (r *HTMLRenderer) RenderNode(w io.Writer, node *Node, entering bool) WalkStatus {
attrs := []string{}
switch node.Type {
case Text:
if r.Flags&Smartypants != 0 {
var tmp bytes.Buffer
escapeHTML(&tmp, node.Literal)
r.sr.Process(w, tmp.Bytes())
} else {
if node.Parent.Type == Link {
escLink(w, node.Literal)
} else {
escapeHTML(w, node.Literal)
}
}
case Softbreak:
r.cr(w)
// TODO: make it configurable via out(renderer.softbreak)
case Hardbreak:
if r.Flags&UseXHTML == 0 {
r.out(w, brTag)
} else {
r.out(w, brXHTMLTag)
}
r.cr(w)
case Emph:
if entering {
r.out(w, emTag)
} else {
r.out(w, emCloseTag)
}
case Strong:
if entering {
r.out(w, strongTag)
} else {
r.out(w, strongCloseTag)
}
case Del:
if entering {
r.out(w, delTag)
} else {
r.out(w, delCloseTag)
}
case HTMLSpan:
if r.Flags&SkipHTML != 0 {
break
}
r.out(w, node.Literal)
case Link:
// mark it but don't link it if it is not a safe link: no smartypants
dest := node.LinkData.Destination
if needSkipLink(r.Flags, dest) {
if entering {
r.out(w, ttTag)
} else {
r.out(w, ttCloseTag)
}
} else {
if entering {
dest = r.addAbsPrefix(dest)
var hrefBuf bytes.Buffer
hrefBuf.WriteString("href=\"")
escLink(&hrefBuf, dest)
hrefBuf.WriteByte('"')
attrs = append(attrs, hrefBuf.String())
if node.NoteID != 0 {
r.out(w, footnoteRef(r.FootnoteAnchorPrefix, node))
break
}
attrs = appendLinkAttrs(attrs, r.Flags, dest)
if len(node.LinkData.Title) > 0 {
var titleBuff bytes.Buffer
titleBuff.WriteString("title=\"")
escapeHTML(&titleBuff, node.LinkData.Title)
titleBuff.WriteByte('"')
attrs = append(attrs, titleBuff.String())
}
r.tag(w, aTag, attrs)
} else {
if node.NoteID != 0 {
break
}
r.out(w, aCloseTag)
}
}
case Image:
if r.Flags&SkipImages != 0 {
return SkipChildren
}
if entering {
dest := node.LinkData.Destination
dest = r.addAbsPrefix(dest)
if r.disableTags == 0 {
//if options.safe && potentiallyUnsafe(dest) {
//out(w, `<img src="" alt="`)
//} else {
r.out(w, []byte(`<img src="`))
escLink(w, dest)
r.out(w, []byte(`" alt="`))
//}
}
r.disableTags++
} else {
r.disableTags--
if r.disableTags == 0 {
if node.LinkData.Title != nil {
r.out(w, []byte(`" title="`))
escapeHTML(w, node.LinkData.Title)
}
r.out(w, []byte(`" />`))
}
}
case Code:
r.out(w, codeTag)
escapeAllHTML(w, node.Literal)
r.out(w, codeCloseTag)
case Document:
break
case Paragraph:
if skipParagraphTags(node) {
break
}
if entering {
// TODO: untangle this clusterfuck about when the newlines need
// to be added and when not.
if node.Prev != nil {
switch node.Prev.Type {
case HTMLBlock, List, Paragraph, Heading, CodeBlock, BlockQuote, HorizontalRule:
r.cr(w)
}
}
if node.Parent.Type == BlockQuote && node.Prev == nil {
r.cr(w)
}
r.out(w, pTag)
} else {
r.out(w, pCloseTag)
if !(node.Parent.Type == Item && node.Next == nil) {
r.cr(w)
}
}
case BlockQuote:
if entering {
r.cr(w)
r.out(w, blockquoteTag)
} else {
r.out(w, blockquoteCloseTag)
r.cr(w)
}
case HTMLBlock:
if r.Flags&SkipHTML != 0 {
break
}
r.cr(w)
r.out(w, node.Literal)
r.cr(w)
case Heading:
headingLevel := r.HTMLRendererParameters.HeadingLevelOffset + node.Level
openTag, closeTag := headingTagsFromLevel(headingLevel)
if entering {
if node.IsTitleblock {
attrs = append(attrs, `class="title"`)
}
if node.HeadingID != "" {
id := r.ensureUniqueHeadingID(node.HeadingID)
if r.HeadingIDPrefix != "" {
id = r.HeadingIDPrefix + id
}
if r.HeadingIDSuffix != "" {
id = id + r.HeadingIDSuffix
}
attrs = append(attrs, fmt.Sprintf(`id="%s"`, id))
}
r.cr(w)
r.tag(w, openTag, attrs)
} else {
r.out(w, closeTag)
if !(node.Parent.Type == Item && node.Next == nil) {
r.cr(w)
}
}
case HorizontalRule:
r.cr(w)
r.outHRTag(w)
r.cr(w)
case List:
openTag := ulTag
closeTag := ulCloseTag
if node.ListFlags&ListTypeOrdered != 0 {
openTag = olTag
closeTag = olCloseTag
}
if node.ListFlags&ListTypeDefinition != 0 {
openTag = dlTag
closeTag = dlCloseTag
}
if entering {
if node.IsFootnotesList {
r.out(w, footnotesDivBytes)
r.outHRTag(w)
r.cr(w)
}
r.cr(w)
if node.Parent.Type == Item && node.Parent.Parent.Tight {
r.cr(w)
}
r.tag(w, openTag[:len(openTag)-1], attrs)
r.cr(w)
} else {
r.out(w, closeTag)
//cr(w)
//if node.parent.Type != Item {
// cr(w)
//}
if node.Parent.Type == Item && node.Next != nil {
r.cr(w)
}
if node.Parent.Type == Document || node.Parent.Type == BlockQuote {
r.cr(w)
}
if node.IsFootnotesList {
r.out(w, footnotesCloseDivBytes)
}
}
case Item:
openTag := liTag
closeTag := liCloseTag
if node.ListFlags&ListTypeDefinition != 0 {
openTag = ddTag
closeTag = ddCloseTag
}
if node.ListFlags&ListTypeTerm != 0 {
openTag = dtTag
closeTag = dtCloseTag
}
if entering {
if itemOpenCR(node) {
r.cr(w)
}
if node.ListData.RefLink != nil {
slug := slugify(node.ListData.RefLink)
r.out(w, footnoteItem(r.FootnoteAnchorPrefix, slug))
break
}
r.out(w, openTag)
} else {
if node.ListData.RefLink != nil {
slug := slugify(node.ListData.RefLink)
if r.Flags&FootnoteReturnLinks != 0 {
r.out(w, footnoteReturnLink(r.FootnoteAnchorPrefix, r.FootnoteReturnLinkContents, slug))
}
}
r.out(w, closeTag)
r.cr(w)
}
case CodeBlock:
attrs = appendLanguageAttr(attrs, node.Info)
r.cr(w)
r.out(w, preTag)
r.tag(w, codeTag[:len(codeTag)-1], attrs)
escapeAllHTML(w, node.Literal)
r.out(w, codeCloseTag)
r.out(w, preCloseTag)
if node.Parent.Type != Item {
r.cr(w)
}
case Table:
if entering {
r.cr(w)
r.out(w, tableTag)
} else {
r.out(w, tableCloseTag)
r.cr(w)
}
case TableCell:
openTag := tdTag
closeTag := tdCloseTag
if node.IsHeader {
openTag = thTag
closeTag = thCloseTag
}
if entering {
align := cellAlignment(node.Align)
if align != "" {
attrs = append(attrs, fmt.Sprintf(`align="%s"`, align))
}
if node.Prev == nil {
r.cr(w)
}
r.tag(w, openTag, attrs)
} else {
r.out(w, closeTag)
r.cr(w)
}
case TableHead:
if entering {
r.cr(w)
r.out(w, theadTag)
} else {
r.out(w, theadCloseTag)
r.cr(w)
}
case TableBody:
if entering {
r.cr(w)
r.out(w, tbodyTag)
// XXX: this is to adhere to a rather silly test. Should fix test.
if node.FirstChild == nil {
r.cr(w)
}
} else {
r.out(w, tbodyCloseTag)
r.cr(w)
}
case TableRow:
if entering {
r.cr(w)
r.out(w, trTag)
} else {
r.out(w, trCloseTag)
r.cr(w)
}
default:
panic("Unknown node type " + node.Type.String())
}
return GoToNext
}
// RenderHeader writes HTML document preamble and TOC if requested.
func (r *HTMLRenderer) RenderHeader(w io.Writer, ast *Node) {
r.writeDocumentHeader(w)
if r.Flags&TOC != 0 {
r.writeTOC(w, ast)
}
}
// RenderFooter writes HTML document footer.
func (r *HTMLRenderer) RenderFooter(w io.Writer, ast *Node) {
if r.Flags&CompletePage == 0 {
return
}
io.WriteString(w, "\n</body>\n</html>\n")
}
func (r *HTMLRenderer) writeDocumentHeader(w io.Writer) {
if r.Flags&CompletePage == 0 {
return
}
ending := ""
if r.Flags&UseXHTML != 0 {
io.WriteString(w, "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" ")
io.WriteString(w, "\"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\n")
io.WriteString(w, "<html xmlns=\"http://www.w3.org/1999/xhtml\">\n")
ending = " /"
} else {
io.WriteString(w, "<!DOCTYPE html>\n")
io.WriteString(w, "<html>\n")
}
io.WriteString(w, "<head>\n")
io.WriteString(w, " <title>")
if r.Flags&Smartypants != 0 {
r.sr.Process(w, []byte(r.Title))
} else {
escapeHTML(w, []byte(r.Title))
}
io.WriteString(w, "</title>\n")
io.WriteString(w, " <meta name=\"GENERATOR\" content=\"Blackfriday Markdown Processor v")
io.WriteString(w, Version)
io.WriteString(w, "\"")
io.WriteString(w, ending)
io.WriteString(w, ">\n")
io.WriteString(w, " <meta charset=\"utf-8\"")
io.WriteString(w, ending)
io.WriteString(w, ">\n")
if r.CSS != "" {
io.WriteString(w, " <link rel=\"stylesheet\" type=\"text/css\" href=\"")
escapeHTML(w, []byte(r.CSS))
io.WriteString(w, "\"")
io.WriteString(w, ending)
io.WriteString(w, ">\n")
}
if r.Icon != "" {
io.WriteString(w, " <link rel=\"icon\" type=\"image/x-icon\" href=\"")
escapeHTML(w, []byte(r.Icon))
io.WriteString(w, "\"")
io.WriteString(w, ending)
io.WriteString(w, ">\n")
}
io.WriteString(w, "</head>\n")
io.WriteString(w, "<body>\n\n")
}
func (r *HTMLRenderer) writeTOC(w io.Writer, ast *Node) {
buf := bytes.Buffer{}
inHeading := false
tocLevel := 0
headingCount := 0
ast.Walk(func(node *Node, entering bool) WalkStatus {
if node.Type == Heading && !node.HeadingData.IsTitleblock {
inHeading = entering
if entering {
node.HeadingID = fmt.Sprintf("toc_%d", headingCount)
if node.Level == tocLevel {
buf.WriteString("</li>\n\n<li>")
} else if node.Level < tocLevel {
for node.Level < tocLevel {
tocLevel--
buf.WriteString("</li>\n</ul>")
}
buf.WriteString("</li>\n\n<li>")
} else {
for node.Level > tocLevel {
tocLevel++
buf.WriteString("\n<ul>\n<li>")
}
}
fmt.Fprintf(&buf, `<a href="#toc_%d">`, headingCount)
headingCount++
} else {
buf.WriteString("</a>")
}
return GoToNext
}
if inHeading {
return r.RenderNode(&buf, node, entering)
}
return GoToNext
})
for ; tocLevel > 0; tocLevel-- {
buf.WriteString("</li>\n</ul>")
}
if buf.Len() > 0 {
io.WriteString(w, "<nav>\n")
w.Write(buf.Bytes())
io.WriteString(w, "\n\n</nav>\n")
}
r.lastOutputLen = buf.Len()
}
//
// Blackfriday Markdown Processor
// Available at http://github.com/russross/blackfriday
//
// Copyright © 2011 Russ Ross <russ@russross.com>.
// Distributed under the Simplified BSD License.
// See README.md for details.
//
//
// Functions to parse inline elements.
//
package blackfriday
import (
"bytes"
"regexp"
"strconv"
)
var (
urlRe = `((https?|ftp):\/\/|\/)[-A-Za-z0-9+&@#\/%?=~_|!:,.;\(\)]+`
anchorRe = regexp.MustCompile(`^(<a\shref="` + urlRe + `"(\stitle="[^"<>]+")?\s?>` + urlRe + `<\/a>)`)
// https://www.w3.org/TR/html5/syntax.html#character-references
// highest unicode code point in 17 planes (2^20): 1,114,112d =
// 7 dec digits or 6 hex digits
// named entity references can be 2-31 characters with stuff like &lt;
// at one end and &CounterClockwiseContourIntegral; at the other. There
// are also sometimes numbers at the end, although this isn't inherent
// in the specification; there are never numbers anywhere else in
// current character references, though; see &frac34; and &blk12;, etc.
// https://www.w3.org/TR/html5/syntax.html#named-character-references
//
// entity := "&" (named group | number ref) ";"
// named group := [a-zA-Z]{2,31}[0-9]{0,2}
// number ref := "#" (dec ref | hex ref)
// dec ref := [0-9]{1,7}
// hex ref := ("x" | "X") [0-9a-fA-F]{1,6}
htmlEntityRe = regexp.MustCompile(`&([a-zA-Z]{2,31}[0-9]{0,2}|#([0-9]{1,7}|[xX][0-9a-fA-F]{1,6}));`)
)
// Functions to parse text within a block
// Each function returns the number of chars taken care of
// data is the complete block being rendered
// offset is the number of valid chars before the current cursor
func (p *Markdown) inline(currBlock *Node, data []byte) {
// handlers might call us recursively: enforce a maximum depth
if p.nesting >= p.maxNesting || len(data) == 0 {
return
}
p.nesting++
beg, end := 0, 0
for end < len(data) {
handler := p.inlineCallback[data[end]]
if handler != nil {
if consumed, node := handler(p, data, end); consumed == 0 {
// No action from the callback.
end++
} else {
// Copy inactive chars into the output.
currBlock.AppendChild(text(data[beg:end]))
if node != nil {
currBlock.AppendChild(node)
}
// Skip past whatever the callback used.
beg = end + consumed
end = beg
}
} else {
end++
}
}
if beg < len(data) {
if data[end-1] == '\n' {
end--
}
currBlock.AppendChild(text(data[beg:end]))
}
p.nesting--
}
// single and double emphasis parsing
func emphasis(p *Markdown, data []byte, offset int) (int, *Node) {
data = data[offset:]
c := data[0]
if len(data) > 2 && data[1] != c {
// whitespace cannot follow an opening emphasis;
// strikethrough only takes two characters '~~'
if c == '~' || isspace(data[1]) {
return 0, nil
}
ret, node := helperEmphasis(p, data[1:], c)
if ret == 0 {
return 0, nil
}
return ret + 1, node
}
if len(data) > 3 && data[1] == c && data[2] != c {
if isspace(data[2]) {
return 0, nil
}
ret, node := helperDoubleEmphasis(p, data[2:], c)
if ret == 0 {
return 0, nil
}
return ret + 2, node
}
if len(data) > 4 && data[1] == c && data[2] == c && data[3] != c {
if c == '~' || isspace(data[3]) {
return 0, nil
}
ret, node := helperTripleEmphasis(p, data, 3, c)
if ret == 0 {
return 0, nil
}
return ret + 3, node
}
return 0, nil
}
func codeSpan(p *Markdown, data []byte, offset int) (int, *Node) {
data = data[offset:]
nb := 0
// count the number of backticks in the delimiter
for nb < len(data) && data[nb] == '`' {
nb++
}
// find the next delimiter
i, end := 0, 0
for end = nb; end < len(data) && i < nb; end++ {
if data[end] == '`' {
i++
} else {
i = 0
}
}
// no matching delimiter?
if i < nb && end >= len(data) {
return 0, nil
}
// trim outside whitespace
fBegin := nb
for fBegin < end && data[fBegin] == ' ' {
fBegin++
}
fEnd := end - nb
for fEnd > fBegin && data[fEnd-1] == ' ' {
fEnd--
}
// render the code span
if fBegin != fEnd {
code := NewNode(Code)
code.Literal = data[fBegin:fEnd]
return end, code
}
return end, nil
}
// newline preceded by two spaces becomes <br>
func maybeLineBreak(p *Markdown, data []byte, offset int) (int, *Node) {
origOffset := offset
for offset < len(data) && data[offset] == ' ' {
offset++
}
if offset < len(data) && data[offset] == '\n' {
if offset-origOffset >= 2 {
return offset - origOffset + 1, NewNode(Hardbreak)
}
return offset - origOffset, nil
}
return 0, nil
}
// newline without two spaces works when HardLineBreak is enabled
func lineBreak(p *Markdown, data []byte, offset int) (int, *Node) {
if p.extensions&HardLineBreak != 0 {
return 1, NewNode(Hardbreak)
}
return 0, nil
}
type linkType int
const (
linkNormal linkType = iota
linkImg
linkDeferredFootnote
linkInlineFootnote
)
func isReferenceStyleLink(data []byte, pos int, t linkType) bool {
if t == linkDeferredFootnote {
return false
}
return pos < len(data)-1 && data[pos] == '[' && data[pos+1] != '^'
}
func maybeImage(p *Markdown, data []byte, offset int) (int, *Node) {
if offset < len(data)-1 && data[offset+1] == '[' {
return link(p, data, offset)
}
return 0, nil
}
func maybeInlineFootnote(p *Markdown, data []byte, offset int) (int, *Node) {
if offset < len(data)-1 && data[offset+1] == '[' {
return link(p, data, offset)
}
return 0, nil
}
// '[': parse a link or an image or a footnote
func link(p *Markdown, data []byte, offset int) (int, *Node) {
// no links allowed inside regular links, footnote, and deferred footnotes
if p.insideLink && (offset > 0 && data[offset-1] == '[' || len(data)-1 > offset && data[offset+1] == '^') {
return 0, nil
}
var t linkType
switch {
// special case: ![^text] == deferred footnote (that follows something with
// an exclamation point)
case p.extensions&Footnotes != 0 && len(data)-1 > offset && data[offset+1] == '^':
t = linkDeferredFootnote
// ![alt] == image
case offset >= 0 && data[offset] == '!':
t = linkImg
offset++
// ^[text] == inline footnote
// [^refId] == deferred footnote
case p.extensions&Footnotes != 0:
if offset >= 0 && data[offset] == '^' {
t = linkInlineFootnote
offset++
} else if len(data)-1 > offset && data[offset+1] == '^' {
t = linkDeferredFootnote
}
// [text] == regular link
default:
t = linkNormal
}
data = data[offset:]
var (
i = 1
noteID int
title, link, altContent []byte
textHasNl = false
)
if t == linkDeferredFootnote {
i++
}
// look for the matching closing bracket
for level := 1; level > 0 && i < len(data); i++ {
switch {
case data[i] == '\n':
textHasNl = true
case isBackslashEscaped(data, i):
continue
case data[i] == '[':
level++
case data[i] == ']':
level--
if level <= 0 {
i-- // compensate for extra i++ in for loop
}
}
}
if i >= len(data) {
return 0, nil
}
txtE := i
i++
var footnoteNode *Node
// skip any amount of whitespace or newline
// (this is much more lax than original markdown syntax)
for i < len(data) && isspace(data[i]) {
i++
}
// inline style link
switch {
case i < len(data) && data[i] == '(':
// skip initial whitespace
i++
for i < len(data) && isspace(data[i]) {
i++
}
linkB := i
// look for link end: ' " )
findlinkend:
for i < len(data) {
switch {
case data[i] == '\\':
i += 2
case data[i] == ')' || data[i] == '\'' || data[i] == '"':
break findlinkend
default:
i++
}
}
if i >= len(data) {
return 0, nil
}
linkE := i
// look for title end if present
titleB, titleE := 0, 0
if data[i] == '\'' || data[i] == '"' {
i++
titleB = i
findtitleend:
for i < len(data) {
switch {
case data[i] == '\\':
i += 2
case data[i] == ')':
break findtitleend
default:
i++
}
}
if i >= len(data) {
return 0, nil
}
// skip whitespace after title
titleE = i - 1
for titleE > titleB && isspace(data[titleE]) {
titleE--
}
// check for closing quote presence
if data[titleE] != '\'' && data[titleE] != '"' {
titleB, titleE = 0, 0
linkE = i
}
}
// remove whitespace at the end of the link
for linkE > linkB && isspace(data[linkE-1]) {
linkE--
}
// remove optional angle brackets around the link
if data[linkB] == '<' {
linkB++
}
if data[linkE-1] == '>' {
linkE--
}
// build escaped link and title
if linkE > linkB {
link = data[linkB:linkE]
}
if titleE > titleB {
title = data[titleB:titleE]
}
i++
// reference style link
case isReferenceStyleLink(data, i, t):
var id []byte
altContentConsidered := false
// look for the id
i++
linkB := i
for i < len(data) && data[i] != ']' {
i++
}
if i >= len(data) {
return 0, nil
}
linkE := i
// find the reference
if linkB == linkE {
if textHasNl {
var b bytes.Buffer
for j := 1; j < txtE; j++ {
switch {
case data[j] != '\n':
b.WriteByte(data[j])
case data[j-1] != ' ':
b.WriteByte(' ')
}
}
id = b.Bytes()
} else {
id = data[1:txtE]
altContentConsidered = true
}
} else {
id = data[linkB:linkE]
}
// find the reference with matching id
lr, ok := p.getRef(string(id))
if !ok {
return 0, nil
}
// keep link and title from reference
link = lr.link
title = lr.title
if altContentConsidered {
altContent = lr.text
}
i++
// shortcut reference style link or reference or inline footnote
default:
var id []byte
// craft the id
if textHasNl {
var b bytes.Buffer
for j := 1; j < txtE; j++ {
switch {
case data[j] != '\n':
b.WriteByte(data[j])
case data[j-1] != ' ':
b.WriteByte(' ')
}
}
id = b.Bytes()
} else {
if t == linkDeferredFootnote {
id = data[2:txtE] // get rid of the ^
} else {
id = data[1:txtE]
}
}
footnoteNode = NewNode(Item)
if t == linkInlineFootnote {
// create a new reference
noteID = len(p.notes) + 1
var fragment []byte
if len(id) > 0 {
if len(id) < 16 {
fragment = make([]byte, len(id))
} else {
fragment = make([]byte, 16)
}
copy(fragment, slugify(id))
} else {
fragment = append([]byte("footnote-"), []byte(strconv.Itoa(noteID))...)
}
ref := &reference{
noteID: noteID,
hasBlock: false,
link: fragment,
title: id,
footnote: footnoteNode,
}
p.notes = append(p.notes, ref)
link = ref.link
title = ref.title
} else {
// find the reference with matching id
lr, ok := p.getRef(string(id))
if !ok {
return 0, nil
}
if t == linkDeferredFootnote {
lr.noteID = len(p.notes) + 1
lr.footnote = footnoteNode
p.notes = append(p.notes, lr)
}
// keep link and title from reference
link = lr.link
// if inline footnote, title == footnote contents
title = lr.title
noteID = lr.noteID
}
// rewind the whitespace
i = txtE + 1
}
var uLink []byte
if t == linkNormal || t == linkImg {
if len(link) > 0 {
var uLinkBuf bytes.Buffer
unescapeText(&uLinkBuf, link)
uLink = uLinkBuf.Bytes()
}
// links need something to click on and somewhere to go
if len(uLink) == 0 || (t == linkNormal && txtE <= 1) {
return 0, nil
}
}
// call the relevant rendering function
var linkNode *Node
switch t {
case linkNormal:
linkNode = NewNode(Link)
linkNode.Destination = normalizeURI(uLink)
linkNode.Title = title
if len(altContent) > 0 {
linkNode.AppendChild(text(altContent))
} else {
// links cannot contain other links, so turn off link parsing
// temporarily and recurse
insideLink := p.insideLink
p.insideLink = true
p.inline(linkNode, data[1:txtE])
p.insideLink = insideLink
}
case linkImg:
linkNode = NewNode(Image)
linkNode.Destination = uLink
linkNode.Title = title
linkNode.AppendChild(text(data[1:txtE]))
i++
case linkInlineFootnote, linkDeferredFootnote:
linkNode = NewNode(Link)
linkNode.Destination = link
linkNode.Title = title
linkNode.NoteID = noteID
linkNode.Footnote = footnoteNode
if t == linkInlineFootnote {
i++
}
default:
return 0, nil
}
return i, linkNode
}
func (p *Markdown) inlineHTMLComment(data []byte) int {
if len(data) < 5 {
return 0
}
if data[0] != '<' || data[1] != '!' || data[2] != '-' || data[3] != '-' {
return 0
}
i := 5
// scan for an end-of-comment marker, across lines if necessary
for i < len(data) && !(data[i-2] == '-' && data[i-1] == '-' && data[i] == '>') {
i++
}
// no end-of-comment marker
if i >= len(data) {
return 0
}
return i + 1
}
func stripMailto(link []byte) []byte {
if bytes.HasPrefix(link, []byte("mailto://")) {
return link[9:]
} else if bytes.HasPrefix(link, []byte("mailto:")) {
return link[7:]
} else {
return link
}
}
// autolinkType specifies a kind of autolink that gets detected.
type autolinkType int
// These are the possible flag values for the autolink renderer.
const (
notAutolink autolinkType = iota
normalAutolink
emailAutolink
)
// '<' when tags or autolinks are allowed
func leftAngle(p *Markdown, data []byte, offset int) (int, *Node) {
data = data[offset:]
altype, end := tagLength(data)
if size := p.inlineHTMLComment(data); size > 0 {
end = size
}
if end > 2 {
if altype != notAutolink {
var uLink bytes.Buffer
unescapeText(&uLink, data[1:end+1-2])
if uLink.Len() > 0 {
link := uLink.Bytes()
node := NewNode(Link)
node.Destination = link
if altype == emailAutolink {
node.Destination = append([]byte("mailto:"), link...)
}
node.AppendChild(text(stripMailto(link)))
return end, node
}
} else {
htmlTag := NewNode(HTMLSpan)
htmlTag.Literal = data[:end]
return end, htmlTag
}
}
return end, nil
}
// '\\' backslash escape
var escapeChars = []byte("\\`*_{}[]()#+-.!:|&<>~")
func escape(p *Markdown, data []byte, offset int) (int, *Node) {
data = data[offset:]
if len(data) > 1 {
if p.extensions&BackslashLineBreak != 0 && data[1] == '\n' {
return 2, NewNode(Hardbreak)
}
if bytes.IndexByte(escapeChars, data[1]) < 0 {
return 0, nil
}
return 2, text(data[1:2])
}
return 2, nil
}
func unescapeText(ob *bytes.Buffer, src []byte) {
i := 0
for i < len(src) {
org := i
for i < len(src) && src[i] != '\\' {
i++
}
if i > org {
ob.Write(src[org:i])
}
if i+1 >= len(src) {
break
}
ob.WriteByte(src[i+1])
i += 2
}
}
// '&' escaped when it doesn't belong to an entity
// valid entities are assumed to be anything matching &#?[A-Za-z0-9]+;
func entity(p *Markdown, data []byte, offset int) (int, *Node) {
data = data[offset:]
end := 1
if end < len(data) && data[end] == '#' {
end++
}
for end < len(data) && isalnum(data[end]) {
end++
}
if end < len(data) && data[end] == ';' {
end++ // real entity
} else {
return 0, nil // lone '&'
}
ent := data[:end]
// undo &amp; escaping or it will be converted to &amp;amp; by another
// escaper in the renderer
if bytes.Equal(ent, []byte("&amp;")) {
ent = []byte{'&'}
}
return end, text(ent)
}
func linkEndsWithEntity(data []byte, linkEnd int) bool {
entityRanges := htmlEntityRe.FindAllIndex(data[:linkEnd], -1)
return entityRanges != nil && entityRanges[len(entityRanges)-1][1] == linkEnd
}
// hasPrefixCaseInsensitive is a custom implementation of
// strings.HasPrefix(strings.ToLower(s), prefix)
// we rolled our own because ToLower pulls in a huge machinery of lowercasing
// anything from Unicode and that's very slow. Since this func will only be
// used on ASCII protocol prefixes, we can take shortcuts.
func hasPrefixCaseInsensitive(s, prefix []byte) bool {
if len(s) < len(prefix) {
return false
}
delta := byte('a' - 'A')
for i, b := range prefix {
if b != s[i] && b != s[i]+delta {
return false
}
}
return true
}
var protocolPrefixes = [][]byte{
[]byte("http://"),
[]byte("https://"),
[]byte("ftp://"),
[]byte("file://"),
[]byte("mailto:"),
}
const shortestPrefix = 6 // len("ftp://"), the shortest of the above
func maybeAutoLink(p *Markdown, data []byte, offset int) (int, *Node) {
// quick check to rule out most false hits
if p.insideLink || len(data) < offset+shortestPrefix {
return 0, nil
}
for _, prefix := range protocolPrefixes {
endOfHead := offset + 8 // 8 is the len() of the longest prefix
if endOfHead > len(data) {
endOfHead = len(data)
}
if hasPrefixCaseInsensitive(data[offset:endOfHead], prefix) {
return autoLink(p, data, offset)
}
}
return 0, nil
}
func autoLink(p *Markdown, data []byte, offset int) (int, *Node) {
// Now a more expensive check to see if we're not inside an anchor element
anchorStart := offset
offsetFromAnchor := 0
for anchorStart > 0 && data[anchorStart] != '<' {
anchorStart--
offsetFromAnchor++
}
anchorStr := anchorRe.Find(data[anchorStart:])
if anchorStr != nil {
anchorClose := NewNode(HTMLSpan)
anchorClose.Literal = anchorStr[offsetFromAnchor:]
return len(anchorStr) - offsetFromAnchor, anchorClose
}
// scan backward for a word boundary
rewind := 0
for offset-rewind > 0 && rewind <= 7 && isletter(data[offset-rewind-1]) {
rewind++
}
if rewind > 6 { // longest supported protocol is "mailto" which has 6 letters
return 0, nil
}
origData := data
data = data[offset-rewind:]
if !isSafeLink(data) {
return 0, nil
}
linkEnd := 0
for linkEnd < len(data) && !isEndOfLink(data[linkEnd]) {
linkEnd++
}
// Skip punctuation at the end of the link
if (data[linkEnd-1] == '.' || data[linkEnd-1] == ',') && data[linkEnd-2] != '\\' {
linkEnd--
}
// But don't skip semicolon if it's a part of escaped entity:
if data[linkEnd-1] == ';' && data[linkEnd-2] != '\\' && !linkEndsWithEntity(data, linkEnd) {
linkEnd--
}
// See if the link finishes with a punctuation sign that can be closed.
var copen byte
switch data[linkEnd-1] {
case '"':
copen = '"'
case '\'':
copen = '\''
case ')':
copen = '('
case ']':
copen = '['
case '}':
copen = '{'
default:
copen = 0
}
if copen != 0 {
bufEnd := offset - rewind + linkEnd - 2
openDelim := 1
/* Try to close the final punctuation sign in this same line;
* if we managed to close it outside of the URL, that means that it's
* not part of the URL. If it closes inside the URL, that means it
* is part of the URL.
*
* Examples:
*
* foo http://www.pokemon.com/Pikachu_(Electric) bar
* => http://www.pokemon.com/Pikachu_(Electric)
*
* foo (http://www.pokemon.com/Pikachu_(Electric)) bar
* => http://www.pokemon.com/Pikachu_(Electric)
*
* foo http://www.pokemon.com/Pikachu_(Electric)) bar
* => http://www.pokemon.com/Pikachu_(Electric))
*
* (foo http://www.pokemon.com/Pikachu_(Electric)) bar
* => foo http://www.pokemon.com/Pikachu_(Electric)
*/
for bufEnd >= 0 && origData[bufEnd] != '\n' && openDelim != 0 {
if origData[bufEnd] == data[linkEnd-1] {
openDelim++
}
if origData[bufEnd] == copen {
openDelim--
}
bufEnd--
}
if openDelim == 0 {
linkEnd--
}
}
var uLink bytes.Buffer
unescapeText(&uLink, data[:linkEnd])
if uLink.Len() > 0 {
node := NewNode(Link)
node.Destination = uLink.Bytes()
node.AppendChild(text(uLink.Bytes()))
return linkEnd, node
}
return linkEnd, nil
}
func isEndOfLink(char byte) bool {
return isspace(char) || char == '<'
}
var validUris = [][]byte{[]byte("http://"), []byte("https://"), []byte("ftp://"), []byte("mailto://")}
var validPaths = [][]byte{[]byte("/"), []byte("./"), []byte("../")}
func isSafeLink(link []byte) bool {
for _, path := range validPaths {
if len(link) >= len(path) && bytes.Equal(link[:len(path)], path) {
if len(link) == len(path) {
return true
} else if isalnum(link[len(path)]) {
return true
}
}
}
for _, prefix := range validUris {
// TODO: handle unicode here
// case-insensitive prefix test
if len(link) > len(prefix) && bytes.Equal(bytes.ToLower(link[:len(prefix)]), prefix) && isalnum(link[len(prefix)]) {
return true
}
}
return false
}
// return the length of the given tag, or 0 is it's not valid
func tagLength(data []byte) (autolink autolinkType, end int) {
var i, j int
// a valid tag can't be shorter than 3 chars
if len(data) < 3 {
return notAutolink, 0
}
// begins with a '<' optionally followed by '/', followed by letter or number
if data[0] != '<' {
return notAutolink, 0
}
if data[1] == '/' {
i = 2
} else {
i = 1
}
if !isalnum(data[i]) {
return notAutolink, 0
}
// scheme test
autolink = notAutolink
// try to find the beginning of an URI
for i < len(data) && (isalnum(data[i]) || data[i] == '.' || data[i] == '+' || data[i] == '-') {
i++
}
if i > 1 && i < len(data) && data[i] == '@' {
if j = isMailtoAutoLink(data[i:]); j != 0 {
return emailAutolink, i + j
}
}
if i > 2 && i < len(data) && data[i] == ':' {
autolink = normalAutolink
i++
}
// complete autolink test: no whitespace or ' or "
switch {
case i >= len(data):
autolink = notAutolink
case autolink != notAutolink:
j = i
for i < len(data) {
if data[i] == '\\' {
i += 2
} else if data[i] == '>' || data[i] == '\'' || data[i] == '"' || isspace(data[i]) {
break
} else {
i++
}
}
if i >= len(data) {
return autolink, 0
}
if i > j && data[i] == '>' {
return autolink, i + 1
}
// one of the forbidden chars has been found
autolink = notAutolink
}
i += bytes.IndexByte(data[i:], '>')
if i < 0 {
return autolink, 0
}
return autolink, i + 1
}
// look for the address part of a mail autolink and '>'
// this is less strict than the original markdown e-mail address matching
func isMailtoAutoLink(data []byte) int {
nb := 0
// address is assumed to be: [-@._a-zA-Z0-9]+ with exactly one '@'
for i := 0; i < len(data); i++ {
if isalnum(data[i]) {
continue
}
switch data[i] {
case '@':
nb++
case '-', '.', '_':
break
case '>':
if nb == 1 {
return i + 1
}
return 0
default:
return 0
}
}
return 0
}
// look for the next emph char, skipping other constructs
func helperFindEmphChar(data []byte, c byte) int {
i := 0
for i < len(data) {
for i < len(data) && data[i] != c && data[i] != '`' && data[i] != '[' {
i++
}
if i >= len(data) {
return 0
}
// do not count escaped chars
if i != 0 && data[i-1] == '\\' {
i++
continue
}
if data[i] == c {
return i
}
if data[i] == '`' {
// skip a code span
tmpI := 0
i++
for i < len(data) && data[i] != '`' {
if tmpI == 0 && data[i] == c {
tmpI = i
}
i++
}
if i >= len(data) {
return tmpI
}
i++
} else if data[i] == '[' {
// skip a link
tmpI := 0
i++
for i < len(data) && data[i] != ']' {
if tmpI == 0 && data[i] == c {
tmpI = i
}
i++
}
i++
for i < len(data) && (data[i] == ' ' || data[i] == '\n') {
i++
}
if i >= len(data) {
return tmpI
}
if data[i] != '[' && data[i] != '(' { // not a link
if tmpI > 0 {
return tmpI
}
continue
}
cc := data[i]
i++
for i < len(data) && data[i] != cc {
if tmpI == 0 && data[i] == c {
return i
}
i++
}
if i >= len(data) {
return tmpI
}
i++
}
}
return 0
}
func helperEmphasis(p *Markdown, data []byte, c byte) (int, *Node) {
i := 0
// skip one symbol if coming from emph3
if len(data) > 1 && data[0] == c && data[1] == c {
i = 1
}
for i < len(data) {
length := helperFindEmphChar(data[i:], c)
if length == 0 {
return 0, nil
}
i += length
if i >= len(data) {
return 0, nil
}
if i+1 < len(data) && data[i+1] == c {
i++
continue
}
if data[i] == c && !isspace(data[i-1]) {
if p.extensions&NoIntraEmphasis != 0 {
if !(i+1 == len(data) || isspace(data[i+1]) || ispunct(data[i+1])) {
continue
}
}
emph := NewNode(Emph)
p.inline(emph, data[:i])
return i + 1, emph
}
}
return 0, nil
}
func helperDoubleEmphasis(p *Markdown, data []byte, c byte) (int, *Node) {
i := 0
for i < len(data) {
length := helperFindEmphChar(data[i:], c)
if length == 0 {
return 0, nil
}
i += length
if i+1 < len(data) && data[i] == c && data[i+1] == c && i > 0 && !isspace(data[i-1]) {
nodeType := Strong
if c == '~' {
nodeType = Del
}
node := NewNode(nodeType)
p.inline(node, data[:i])
return i + 2, node
}
i++
}
return 0, nil
}
func helperTripleEmphasis(p *Markdown, data []byte, offset int, c byte) (int, *Node) {
i := 0
origData := data
data = data[offset:]
for i < len(data) {
length := helperFindEmphChar(data[i:], c)
if length == 0 {
return 0, nil
}
i += length
// skip whitespace preceded symbols
if data[i] != c || isspace(data[i-1]) {
continue
}
switch {
case i+2 < len(data) && data[i+1] == c && data[i+2] == c:
// triple symbol found
strong := NewNode(Strong)
em := NewNode(Emph)
strong.AppendChild(em)
p.inline(em, data[:i])
return i + 3, strong
case (i+1 < len(data) && data[i+1] == c):
// double symbol found, hand over to emph1
length, node := helperEmphasis(p, origData[offset-2:], c)
if length == 0 {
return 0, nil
}
return length - 2, node
default:
// single symbol found, hand over to emph2
length, node := helperDoubleEmphasis(p, origData[offset-1:], c)
if length == 0 {
return 0, nil
}
return length - 1, node
}
}
return 0, nil
}
func text(s []byte) *Node {
node := NewNode(Text)
node.Literal = s
return node
}
func normalizeURI(s []byte) []byte {
return s // TODO: implement
}
// Blackfriday Markdown Processor
// Available at http://github.com/russross/blackfriday
//
// Copyright © 2011 Russ Ross <russ@russross.com>.
// Distributed under the Simplified BSD License.
// See README.md for details.
package blackfriday
import (
"bytes"
"fmt"
"io"
"strings"
"unicode/utf8"
)
//
// Markdown parsing and processing
//
// Version string of the package. Appears in the rendered document when
// CompletePage flag is on.
const Version = "2.0"
// Extensions is a bitwise or'ed collection of enabled Blackfriday's
// extensions.
type Extensions int
// These are the supported markdown parsing extensions.
// OR these values together to select multiple extensions.
const (
NoExtensions Extensions = 0
NoIntraEmphasis Extensions = 1 << iota // Ignore emphasis markers inside words
Tables // Render tables
FencedCode // Render fenced code blocks
Autolink // Detect embedded URLs that are not explicitly marked
Strikethrough // Strikethrough text using ~~test~~
LaxHTMLBlocks // Loosen up HTML block parsing rules
SpaceHeadings // Be strict about prefix heading rules
HardLineBreak // Translate newlines into line breaks
TabSizeEight // Expand tabs to eight spaces instead of four
Footnotes // Pandoc-style footnotes
NoEmptyLineBeforeBlock // No need to insert an empty line to start a (code, quote, ordered list, unordered list) block
HeadingIDs // specify heading IDs with {#id}
Titleblock // Titleblock ala pandoc
AutoHeadingIDs // Create the heading ID from the text
BackslashLineBreak // Translate trailing backslashes into line breaks
DefinitionLists // Render definition lists
CommonHTMLFlags HTMLFlags = UseXHTML | Smartypants |
SmartypantsFractions | SmartypantsDashes | SmartypantsLatexDashes
CommonExtensions Extensions = NoIntraEmphasis | Tables | FencedCode |
Autolink | Strikethrough | SpaceHeadings | HeadingIDs |
BackslashLineBreak | DefinitionLists
)
// ListType contains bitwise or'ed flags for list and list item objects.
type ListType int
// These are the possible flag values for the ListItem renderer.
// Multiple flag values may be ORed together.
// These are mostly of interest if you are writing a new output format.
const (
ListTypeOrdered ListType = 1 << iota
ListTypeDefinition
ListTypeTerm
ListItemContainsBlock
ListItemBeginningOfList // TODO: figure out if this is of any use now
ListItemEndOfList
)
// CellAlignFlags holds a type of alignment in a table cell.
type CellAlignFlags int
// These are the possible flag values for the table cell renderer.
// Only a single one of these values will be used; they are not ORed together.
// These are mostly of interest if you are writing a new output format.
const (
TableAlignmentLeft CellAlignFlags = 1 << iota
TableAlignmentRight
TableAlignmentCenter = (TableAlignmentLeft | TableAlignmentRight)
)
// The size of a tab stop.
const (
TabSizeDefault = 4
TabSizeDouble = 8
)
// blockTags is a set of tags that are recognized as HTML block tags.
// Any of these can be included in markdown text without special escaping.
var blockTags = map[string]struct{}{
"blockquote": {},
"del": {},
"div": {},
"dl": {},
"fieldset": {},
"form": {},
"h1": {},
"h2": {},
"h3": {},
"h4": {},
"h5": {},
"h6": {},
"iframe": {},
"ins": {},
"math": {},
"noscript": {},
"ol": {},
"pre": {},
"p": {},
"script": {},
"style": {},
"table": {},
"ul": {},
// HTML5
"address": {},
"article": {},
"aside": {},
"canvas": {},
"figcaption": {},
"figure": {},
"footer": {},
"header": {},
"hgroup": {},
"main": {},
"nav": {},
"output": {},
"progress": {},
"section": {},
"video": {},
}
// Renderer is the rendering interface. This is mostly of interest if you are
// implementing a new rendering format.
//
// Only an HTML implementation is provided in this repository, see the README
// for external implementations.
type Renderer interface {
// RenderNode is the main rendering method. It will be called once for
// every leaf node and twice for every non-leaf node (first with
// entering=true, then with entering=false). The method should write its
// rendition of the node to the supplied writer w.
RenderNode(w io.Writer, node *Node, entering bool) WalkStatus
// RenderHeader is a method that allows the renderer to produce some
// content preceding the main body of the output document. The header is
// understood in the broad sense here. For example, the default HTML
// renderer will write not only the HTML document preamble, but also the
// table of contents if it was requested.
//
// The method will be passed an entire document tree, in case a particular
// implementation needs to inspect it to produce output.
//
// The output should be written to the supplied writer w. If your
// implementation has no header to write, supply an empty implementation.
RenderHeader(w io.Writer, ast *Node)
// RenderFooter is a symmetric counterpart of RenderHeader.
RenderFooter(w io.Writer, ast *Node)
}
// Callback functions for inline parsing. One such function is defined
// for each character that triggers a response when parsing inline data.
type inlineParser func(p *Markdown, data []byte, offset int) (int, *Node)
// Markdown is a type that holds extensions and the runtime state used by
// Parse, and the renderer. You can not use it directly, construct it with New.
type Markdown struct {
renderer Renderer
referenceOverride ReferenceOverrideFunc
refs map[string]*reference
inlineCallback [256]inlineParser
extensions Extensions
nesting int
maxNesting int
insideLink bool
// Footnotes need to be ordered as well as available to quickly check for
// presence. If a ref is also a footnote, it's stored both in refs and here
// in notes. Slice is nil if footnotes not enabled.
notes []*reference
doc *Node
tip *Node // = doc
oldTip *Node
lastMatchedContainer *Node // = doc
allClosed bool
}
func (p *Markdown) getRef(refid string) (ref *reference, found bool) {
if p.referenceOverride != nil {
r, overridden := p.referenceOverride(refid)
if overridden {
if r == nil {
return nil, false
}
return &reference{
link: []byte(r.Link),
title: []byte(r.Title),
noteID: 0,
hasBlock: false,
text: []byte(r.Text)}, true
}
}
// refs are case insensitive
ref, found = p.refs[strings.ToLower(refid)]
return ref, found
}
func (p *Markdown) finalize(block *Node) {
above := block.Parent
block.open = false
p.tip = above
}
func (p *Markdown) addChild(node NodeType, offset uint32) *Node {
return p.addExistingChild(NewNode(node), offset)
}
func (p *Markdown) addExistingChild(node *Node, offset uint32) *Node {
for !p.tip.canContain(node.Type) {
p.finalize(p.tip)
}
p.tip.AppendChild(node)
p.tip = node
return node
}
func (p *Markdown) closeUnmatchedBlocks() {
if !p.allClosed {
for p.oldTip != p.lastMatchedContainer {
parent := p.oldTip.Parent
p.finalize(p.oldTip)
p.oldTip = parent
}
p.allClosed = true
}
}
//
//
// Public interface
//
//
// Reference represents the details of a link.
// See the documentation in Options for more details on use-case.
type Reference struct {
// Link is usually the URL the reference points to.
Link string
// Title is the alternate text describing the link in more detail.
Title string
// Text is the optional text to override the ref with if the syntax used was
// [refid][]
Text string
}
// ReferenceOverrideFunc is expected to be called with a reference string and
// return either a valid Reference type that the reference string maps to or
// nil. If overridden is false, the default reference logic will be executed.
// See the documentation in Options for more details on use-case.
type ReferenceOverrideFunc func(reference string) (ref *Reference, overridden bool)
// New constructs a Markdown processor. You can use the same With* functions as
// for Run() to customize parser's behavior and the renderer.
func New(opts ...Option) *Markdown {
var p Markdown
for _, opt := range opts {
opt(&p)
}
p.refs = make(map[string]*reference)
p.maxNesting = 16
p.insideLink = false
docNode := NewNode(Document)
p.doc = docNode
p.tip = docNode
p.oldTip = docNode
p.lastMatchedContainer = docNode
p.allClosed = true
// register inline parsers
p.inlineCallback[' '] = maybeLineBreak
p.inlineCallback['*'] = emphasis
p.inlineCallback['_'] = emphasis
if p.extensions&Strikethrough != 0 {
p.inlineCallback['~'] = emphasis
}
p.inlineCallback['`'] = codeSpan
p.inlineCallback['\n'] = lineBreak
p.inlineCallback['['] = link
p.inlineCallback['<'] = leftAngle
p.inlineCallback['\\'] = escape
p.inlineCallback['&'] = entity
p.inlineCallback['!'] = maybeImage
p.inlineCallback['^'] = maybeInlineFootnote
if p.extensions&Autolink != 0 {
p.inlineCallback['h'] = maybeAutoLink
p.inlineCallback['m'] = maybeAutoLink
p.inlineCallback['f'] = maybeAutoLink
p.inlineCallback['H'] = maybeAutoLink
p.inlineCallback['M'] = maybeAutoLink
p.inlineCallback['F'] = maybeAutoLink
}
if p.extensions&Footnotes != 0 {
p.notes = make([]*reference, 0)
}
return &p
}
// Option customizes the Markdown processor's default behavior.
type Option func(*Markdown)
// WithRenderer allows you to override the default renderer.
func WithRenderer(r Renderer) Option {
return func(p *Markdown) {
p.renderer = r
}
}
// WithExtensions allows you to pick some of the many extensions provided by
// Blackfriday. You can bitwise OR them.
func WithExtensions(e Extensions) Option {
return func(p *Markdown) {
p.extensions = e
}
}
// WithNoExtensions turns off all extensions and custom behavior.
func WithNoExtensions() Option {
return func(p *Markdown) {
p.extensions = NoExtensions
p.renderer = NewHTMLRenderer(HTMLRendererParameters{
Flags: HTMLFlagsNone,
})
}
}
// WithRefOverride sets an optional function callback that is called every
// time a reference is resolved.
//
// In Markdown, the link reference syntax can be made to resolve a link to
// a reference instead of an inline URL, in one of the following ways:
//
// * [link text][refid]
// * [refid][]
//
// Usually, the refid is defined at the bottom of the Markdown document. If
// this override function is provided, the refid is passed to the override
// function first, before consulting the defined refids at the bottom. If
// the override function indicates an override did not occur, the refids at
// the bottom will be used to fill in the link details.
func WithRefOverride(o ReferenceOverrideFunc) Option {
return func(p *Markdown) {
p.referenceOverride = o
}
}
// Run is the main entry point to Blackfriday. It parses and renders a
// block of markdown-encoded text.
//
// The simplest invocation of Run takes one argument, input:
// output := Run(input)
// This will parse the input with CommonExtensions enabled and render it with
// the default HTMLRenderer (with CommonHTMLFlags).
//
// Variadic arguments opts can customize the default behavior. Since Markdown
// type does not contain exported fields, you can not use it directly. Instead,
// use the With* functions. For example, this will call the most basic
// functionality, with no extensions:
// output := Run(input, WithNoExtensions())
//
// You can use any number of With* arguments, even contradicting ones. They
// will be applied in order of appearance and the latter will override the
// former:
// output := Run(input, WithNoExtensions(), WithExtensions(exts),
// WithRenderer(yourRenderer))
func Run(input []byte, opts ...Option) []byte {
r := NewHTMLRenderer(HTMLRendererParameters{
Flags: CommonHTMLFlags,
})
optList := []Option{WithRenderer(r), WithExtensions(CommonExtensions)}
optList = append(optList, opts...)
parser := New(optList...)
ast := parser.Parse(input)
var buf bytes.Buffer
parser.renderer.RenderHeader(&buf, ast)
ast.Walk(func(node *Node, entering bool) WalkStatus {
return parser.renderer.RenderNode(&buf, node, entering)
})
parser.renderer.RenderFooter(&buf, ast)
return buf.Bytes()
}
// Parse is an entry point to the parsing part of Blackfriday. It takes an
// input markdown document and produces a syntax tree for its contents. This
// tree can then be rendered with a default or custom renderer, or
// analyzed/transformed by the caller to whatever non-standard needs they have.
// The return value is the root node of the syntax tree.
func (p *Markdown) Parse(input []byte) *Node {
p.block(input)
// Walk the tree and finish up some of unfinished blocks
for p.tip != nil {
p.finalize(p.tip)
}
// Walk the tree again and process inline markdown in each block
p.doc.Walk(func(node *Node, entering bool) WalkStatus {
if node.Type == Paragraph || node.Type == Heading || node.Type == TableCell {
p.inline(node, node.content)
node.content = nil
}
return GoToNext
})
p.parseRefsToAST()
return p.doc
}
func (p *Markdown) parseRefsToAST() {
if p.extensions&Footnotes == 0 || len(p.notes) == 0 {
return
}
p.tip = p.doc
block := p.addBlock(List, nil)
block.IsFootnotesList = true
block.ListFlags = ListTypeOrdered
flags := ListItemBeginningOfList
// Note: this loop is intentionally explicit, not range-form. This is
// because the body of the loop will append nested footnotes to p.notes and
// we need to process those late additions. Range form would only walk over
// the fixed initial set.
for i := 0; i < len(p.notes); i++ {
ref := p.notes[i]
p.addExistingChild(ref.footnote, 0)
block := ref.footnote
block.ListFlags = flags | ListTypeOrdered
block.RefLink = ref.link
if ref.hasBlock {
flags |= ListItemContainsBlock
p.block(ref.title)
} else {
p.inline(block, ref.title)
}
flags &^= ListItemBeginningOfList | ListItemContainsBlock
}
above := block.Parent
finalizeList(block)
p.tip = above
block.Walk(func(node *Node, entering bool) WalkStatus {
if node.Type == Paragraph || node.Type == Heading {
p.inline(node, node.content)
node.content = nil
}
return GoToNext
})
}
//
// Link references
//
// This section implements support for references that (usually) appear
// as footnotes in a document, and can be referenced anywhere in the document.
// The basic format is:
//
// [1]: http://www.google.com/ "Google"
// [2]: http://www.github.com/ "Github"
//
// Anywhere in the document, the reference can be linked by referring to its
// label, i.e., 1 and 2 in this example, as in:
//
// This library is hosted on [Github][2], a git hosting site.
//
// Actual footnotes as specified in Pandoc and supported by some other Markdown
// libraries such as php-markdown are also taken care of. They look like this:
//
// This sentence needs a bit of further explanation.[^note]
//
// [^note]: This is the explanation.
//
// Footnotes should be placed at the end of the document in an ordered list.
// Finally, there are inline footnotes such as:
//
// Inline footnotes^[Also supported.] provide a quick inline explanation,
// but are rendered at the bottom of the document.
//
// reference holds all information necessary for a reference-style links or
// footnotes.
//
// Consider this markdown with reference-style links:
//
// [link][ref]
//
// [ref]: /url/ "tooltip title"
//
// It will be ultimately converted to this HTML:
//
// <p><a href=\"/url/\" title=\"title\">link</a></p>
//
// And a reference structure will be populated as follows:
//
// p.refs["ref"] = &reference{
// link: "/url/",
// title: "tooltip title",
// }
//
// Alternatively, reference can contain information about a footnote. Consider
// this markdown:
//
// Text needing a footnote.[^a]
//
// [^a]: This is the note
//
// A reference structure will be populated as follows:
//
// p.refs["a"] = &reference{
// link: "a",
// title: "This is the note",
// noteID: <some positive int>,
// }
//
// TODO: As you can see, it begs for splitting into two dedicated structures
// for refs and for footnotes.
type reference struct {
link []byte
title []byte
noteID int // 0 if not a footnote ref
hasBlock bool
footnote *Node // a link to the Item node within a list of footnotes
text []byte // only gets populated by refOverride feature with Reference.Text
}
func (r *reference) String() string {
return fmt.Sprintf("{link: %q, title: %q, text: %q, noteID: %d, hasBlock: %v}",
r.link, r.title, r.text, r.noteID, r.hasBlock)
}
// Check whether or not data starts with a reference link.
// If so, it is parsed and stored in the list of references
// (in the render struct).
// Returns the number of bytes to skip to move past it,
// or zero if the first line is not a reference.
func isReference(p *Markdown, data []byte, tabSize int) int {
// up to 3 optional leading spaces
if len(data) < 4 {
return 0
}
i := 0
for i < 3 && data[i] == ' ' {
i++
}
noteID := 0
// id part: anything but a newline between brackets
if data[i] != '[' {
return 0
}
i++
if p.extensions&Footnotes != 0 {
if i < len(data) && data[i] == '^' {
// we can set it to anything here because the proper noteIds will
// be assigned later during the second pass. It just has to be != 0
noteID = 1
i++
}
}
idOffset := i
for i < len(data) && data[i] != '\n' && data[i] != '\r' && data[i] != ']' {
i++
}
if i >= len(data) || data[i] != ']' {
return 0
}
idEnd := i
// footnotes can have empty ID, like this: [^], but a reference can not be
// empty like this: []. Break early if it's not a footnote and there's no ID
if noteID == 0 && idOffset == idEnd {
return 0
}
// spacer: colon (space | tab)* newline? (space | tab)*
i++
if i >= len(data) || data[i] != ':' {
return 0
}
i++
for i < len(data) && (data[i] == ' ' || data[i] == '\t') {
i++
}
if i < len(data) && (data[i] == '\n' || data[i] == '\r') {
i++
if i < len(data) && data[i] == '\n' && data[i-1] == '\r' {
i++
}
}
for i < len(data) && (data[i] == ' ' || data[i] == '\t') {
i++
}
if i >= len(data) {
return 0
}
var (
linkOffset, linkEnd int
titleOffset, titleEnd int
lineEnd int
raw []byte
hasBlock bool
)
if p.extensions&Footnotes != 0 && noteID != 0 {
linkOffset, linkEnd, raw, hasBlock = scanFootnote(p, data, i, tabSize)
lineEnd = linkEnd
} else {
linkOffset, linkEnd, titleOffset, titleEnd, lineEnd = scanLinkRef(p, data, i)
}
if lineEnd == 0 {
return 0
}
// a valid ref has been found
ref := &reference{
noteID: noteID,
hasBlock: hasBlock,
}
if noteID > 0 {
// reusing the link field for the id since footnotes don't have links
ref.link = data[idOffset:idEnd]
// if footnote, it's not really a title, it's the contained text
ref.title = raw
} else {
ref.link = data[linkOffset:linkEnd]
ref.title = data[titleOffset:titleEnd]
}
// id matches are case-insensitive
id := string(bytes.ToLower(data[idOffset:idEnd]))
p.refs[id] = ref
return lineEnd
}
func scanLinkRef(p *Markdown, data []byte, i int) (linkOffset, linkEnd, titleOffset, titleEnd, lineEnd int) {
// link: whitespace-free sequence, optionally between angle brackets
if data[i] == '<' {
i++
}
linkOffset = i
for i < len(data) && data[i] != ' ' && data[i] != '\t' && data[i] != '\n' && data[i] != '\r' {
i++
}
linkEnd = i
if data[linkOffset] == '<' && data[linkEnd-1] == '>' {
linkOffset++
linkEnd--
}
// optional spacer: (space | tab)* (newline | '\'' | '"' | '(' )
for i < len(data) && (data[i] == ' ' || data[i] == '\t') {
i++
}
if i < len(data) && data[i] != '\n' && data[i] != '\r' && data[i] != '\'' && data[i] != '"' && data[i] != '(' {
return
}
// compute end-of-line
if i >= len(data) || data[i] == '\r' || data[i] == '\n' {
lineEnd = i
}
if i+1 < len(data) && data[i] == '\r' && data[i+1] == '\n' {
lineEnd++
}
// optional (space|tab)* spacer after a newline
if lineEnd > 0 {
i = lineEnd + 1
for i < len(data) && (data[i] == ' ' || data[i] == '\t') {
i++
}
}
// optional title: any non-newline sequence enclosed in '"() alone on its line
if i+1 < len(data) && (data[i] == '\'' || data[i] == '"' || data[i] == '(') {
i++
titleOffset = i
// look for EOL
for i < len(data) && data[i] != '\n' && data[i] != '\r' {
i++
}
if i+1 < len(data) && data[i] == '\n' && data[i+1] == '\r' {
titleEnd = i + 1
} else {
titleEnd = i
}
// step back
i--
for i > titleOffset && (data[i] == ' ' || data[i] == '\t') {
i--
}
if i > titleOffset && (data[i] == '\'' || data[i] == '"' || data[i] == ')') {
lineEnd = titleEnd
titleEnd = i
}
}
return
}
// The first bit of this logic is the same as Parser.listItem, but the rest
// is much simpler. This function simply finds the entire block and shifts it
// over by one tab if it is indeed a block (just returns the line if it's not).
// blockEnd is the end of the section in the input buffer, and contents is the
// extracted text that was shifted over one tab. It will need to be rendered at
// the end of the document.
func scanFootnote(p *Markdown, data []byte, i, indentSize int) (blockStart, blockEnd int, contents []byte, hasBlock bool) {
if i == 0 || len(data) == 0 {
return
}
// skip leading whitespace on first line
for i < len(data) && data[i] == ' ' {
i++
}
blockStart = i
// find the end of the line
blockEnd = i
for i < len(data) && data[i-1] != '\n' {
i++
}
// get working buffer
var raw bytes.Buffer
// put the first line into the working buffer
raw.Write(data[blockEnd:i])
blockEnd = i
// process the following lines
containsBlankLine := false
gatherLines:
for blockEnd < len(data) {
i++
// find the end of this line
for i < len(data) && data[i-1] != '\n' {
i++
}
// if it is an empty line, guess that it is part of this item
// and move on to the next line
if p.isEmpty(data[blockEnd:i]) > 0 {
containsBlankLine = true
blockEnd = i
continue
}
n := 0
if n = isIndented(data[blockEnd:i], indentSize); n == 0 {
// this is the end of the block.
// we don't want to include this last line in the index.
break gatherLines
}
// if there were blank lines before this one, insert a new one now
if containsBlankLine {
raw.WriteByte('\n')
containsBlankLine = false
}
// get rid of that first tab, write to buffer
raw.Write(data[blockEnd+n : i])
hasBlock = true
blockEnd = i
}
if data[blockEnd-1] != '\n' {
raw.WriteByte('\n')
}
contents = raw.Bytes()
return
}
//
//
// Miscellaneous helper functions
//
//
// Test if a character is a punctuation symbol.
// Taken from a private function in regexp in the stdlib.
func ispunct(c byte) bool {
for _, r := range []byte("!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~") {
if c == r {
return true
}
}
return false
}
// Test if a character is a whitespace character.
func isspace(c byte) bool {
return ishorizontalspace(c) || isverticalspace(c)
}
// Test if a character is a horizontal whitespace character.
func ishorizontalspace(c byte) bool {
return c == ' ' || c == '\t'
}
// Test if a character is a vertical character.
func isverticalspace(c byte) bool {
return c == '\n' || c == '\r' || c == '\f' || c == '\v'
}
// Test if a character is letter.
func isletter(c byte) bool {
return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z')
}
// Test if a character is a letter or a digit.
// TODO: check when this is looking for ASCII alnum and when it should use unicode
func isalnum(c byte) bool {
return (c >= '0' && c <= '9') || isletter(c)
}
// Replace tab characters with spaces, aligning to the next TAB_SIZE column.
// always ends output with a newline
func expandTabs(out *bytes.Buffer, line []byte, tabSize int) {
// first, check for common cases: no tabs, or only tabs at beginning of line
i, prefix := 0, 0
slowcase := false
for i = 0; i < len(line); i++ {
if line[i] == '\t' {
if prefix == i {
prefix++
} else {
slowcase = true
break
}
}
}
// no need to decode runes if all tabs are at the beginning of the line
if !slowcase {
for i = 0; i < prefix*tabSize; i++ {
out.WriteByte(' ')
}
out.Write(line[prefix:])
return
}
// the slow case: we need to count runes to figure out how
// many spaces to insert for each tab
column := 0
i = 0
for i < len(line) {
start := i
for i < len(line) && line[i] != '\t' {
_, size := utf8.DecodeRune(line[i:])
i += size
column++
}
if i > start {
out.Write(line[start:i])
}
if i >= len(line) {
break
}
for {
out.WriteByte(' ')
column++
if column%tabSize == 0 {
break
}
}
i++
}
}
// Find if a line counts as indented or not.
// Returns number of characters the indent is (0 = not indented).
func isIndented(data []byte, indentSize int) int {
if len(data) == 0 {
return 0
}
if data[0] == '\t' {
return 1
}
if len(data) < indentSize {
return 0
}
for i := 0; i < indentSize; i++ {
if data[i] != ' ' {
return 0
}
}
return indentSize
}
// Create a url-safe slug for fragments
func slugify(in []byte) []byte {
if len(in) == 0 {
return in
}
out := make([]byte, 0, len(in))
sym := false
for _, ch := range in {
if isalnum(ch) {
sym = false
out = append(out, ch)
} else if sym {
continue
} else {
out = append(out, '-')
sym = true
}
}
var a, b int
var ch byte
for a, ch = range out {
if ch != '-' {
break
}
}
for b = len(out) - 1; b > 0; b-- {
if out[b] != '-' {
break
}
}
return out[a : b+1]
}
package blackfriday
import (
"bytes"
"fmt"
)
// NodeType specifies a type of a single node of a syntax tree. Usually one
// node (and its type) corresponds to a single markdown feature, e.g. emphasis
// or code block.
type NodeType int
// Constants for identifying different types of nodes. See NodeType.
const (
Document NodeType = iota
BlockQuote
List
Item
Paragraph
Heading
HorizontalRule
Emph
Strong
Del
Link
Image
Text
HTMLBlock
CodeBlock
Softbreak
Hardbreak
Code
HTMLSpan
Table
TableCell
TableHead
TableBody
TableRow
)
var nodeTypeNames = []string{
Document: "Document",
BlockQuote: "BlockQuote",
List: "List",
Item: "Item",
Paragraph: "Paragraph",
Heading: "Heading",
HorizontalRule: "HorizontalRule",
Emph: "Emph",
Strong: "Strong",
Del: "Del",
Link: "Link",
Image: "Image",
Text: "Text",
HTMLBlock: "HTMLBlock",
CodeBlock: "CodeBlock",
Softbreak: "Softbreak",
Hardbreak: "Hardbreak",
Code: "Code",
HTMLSpan: "HTMLSpan",
Table: "Table",
TableCell: "TableCell",
TableHead: "TableHead",
TableBody: "TableBody",
TableRow: "TableRow",
}
func (t NodeType) String() string {
return nodeTypeNames[t]
}
// ListData contains fields relevant to a List and Item node type.
type ListData struct {
ListFlags ListType
Tight bool // Skip <p>s around list item data if true
BulletChar byte // '*', '+' or '-' in bullet lists
Delimiter byte // '.' or ')' after the number in ordered lists
RefLink []byte // If not nil, turns this list item into a footnote item and triggers different rendering
IsFootnotesList bool // This is a list of footnotes
}
// LinkData contains fields relevant to a Link node type.
type LinkData struct {
Destination []byte // Destination is what goes into a href
Title []byte // Title is the tooltip thing that goes in a title attribute
NoteID int // NoteID contains a serial number of a footnote, zero if it's not a footnote
Footnote *Node // If it's a footnote, this is a direct link to the footnote Node. Otherwise nil.
}
// CodeBlockData contains fields relevant to a CodeBlock node type.
type CodeBlockData struct {
IsFenced bool // Specifies whether it's a fenced code block or an indented one
Info []byte // This holds the info string
FenceChar byte
FenceLength int
FenceOffset int
}
// TableCellData contains fields relevant to a TableCell node type.
type TableCellData struct {
IsHeader bool // This tells if it's under the header row
Align CellAlignFlags // This holds the value for align attribute
}
// HeadingData contains fields relevant to a Heading node type.
type HeadingData struct {
Level int // This holds the heading level number
HeadingID string // This might hold heading ID, if present
IsTitleblock bool // Specifies whether it's a title block
}
// Node is a single element in the abstract syntax tree of the parsed document.
// It holds connections to the structurally neighboring nodes and, for certain
// types of nodes, additional information that might be needed when rendering.
type Node struct {
Type NodeType // Determines the type of the node
Parent *Node // Points to the parent
FirstChild *Node // Points to the first child, if any
LastChild *Node // Points to the last child, if any
Prev *Node // Previous sibling; nil if it's the first child
Next *Node // Next sibling; nil if it's the last child
Literal []byte // Text contents of the leaf nodes
HeadingData // Populated if Type is Heading
ListData // Populated if Type is List
CodeBlockData // Populated if Type is CodeBlock
LinkData // Populated if Type is Link
TableCellData // Populated if Type is TableCell
content []byte // Markdown content of the block nodes
open bool // Specifies an open block node that has not been finished to process yet
}
// NewNode allocates a node of a specified type.
func NewNode(typ NodeType) *Node {
return &Node{
Type: typ,
open: true,
}
}
func (n *Node) String() string {
ellipsis := ""
snippet := n.Literal
if len(snippet) > 16 {
snippet = snippet[:16]
ellipsis = "..."
}
return fmt.Sprintf("%s: '%s%s'", n.Type, snippet, ellipsis)
}
// Unlink removes node 'n' from the tree.
// It panics if the node is nil.
func (n *Node) Unlink() {
if n.Prev != nil {
n.Prev.Next = n.Next
} else if n.Parent != nil {
n.Parent.FirstChild = n.Next
}
if n.Next != nil {
n.Next.Prev = n.Prev
} else if n.Parent != nil {
n.Parent.LastChild = n.Prev
}
n.Parent = nil
n.Next = nil
n.Prev = nil
}
// AppendChild adds a node 'child' as a child of 'n'.
// It panics if either node is nil.
func (n *Node) AppendChild(child *Node) {
child.Unlink()
child.Parent = n
if n.LastChild != nil {
n.LastChild.Next = child
child.Prev = n.LastChild
n.LastChild = child
} else {
n.FirstChild = child
n.LastChild = child
}
}
// InsertBefore inserts 'sibling' immediately before 'n'.
// It panics if either node is nil.
func (n *Node) InsertBefore(sibling *Node) {
sibling.Unlink()
sibling.Prev = n.Prev
if sibling.Prev != nil {
sibling.Prev.Next = sibling
}
sibling.Next = n
n.Prev = sibling
sibling.Parent = n.Parent
if sibling.Prev == nil {
sibling.Parent.FirstChild = sibling
}
}
// IsContainer returns true if 'n' can contain children.
func (n *Node) IsContainer() bool {
switch n.Type {
case Document:
fallthrough
case BlockQuote:
fallthrough
case List:
fallthrough
case Item:
fallthrough
case Paragraph:
fallthrough
case Heading:
fallthrough
case Emph:
fallthrough
case Strong:
fallthrough
case Del:
fallthrough
case Link:
fallthrough
case Image:
fallthrough
case Table:
fallthrough
case TableHead:
fallthrough
case TableBody:
fallthrough
case TableRow:
fallthrough
case TableCell:
return true
default:
return false
}
}
// IsLeaf returns true if 'n' is a leaf node.
func (n *Node) IsLeaf() bool {
return !n.IsContainer()
}
func (n *Node) canContain(t NodeType) bool {
if n.Type == List {
return t == Item
}
if n.Type == Document || n.Type == BlockQuote || n.Type == Item {
return t != Item
}
if n.Type == Table {
return t == TableHead || t == TableBody
}
if n.Type == TableHead || n.Type == TableBody {
return t == TableRow
}
if n.Type == TableRow {
return t == TableCell
}
return false
}
// WalkStatus allows NodeVisitor to have some control over the tree traversal.
// It is returned from NodeVisitor and different values allow Node.Walk to
// decide which node to go to next.
type WalkStatus int
const (
// GoToNext is the default traversal of every node.
GoToNext WalkStatus = iota
// SkipChildren tells walker to skip all children of current node.
SkipChildren
// Terminate tells walker to terminate the traversal.
Terminate
)
// NodeVisitor is a callback to be called when traversing the syntax tree.
// Called twice for every node: once with entering=true when the branch is
// first visited, then with entering=false after all the children are done.
type NodeVisitor func(node *Node, entering bool) WalkStatus
// Walk is a convenience method that instantiates a walker and starts a
// traversal of subtree rooted at n.
func (n *Node) Walk(visitor NodeVisitor) {
w := newNodeWalker(n)
for w.current != nil {
status := visitor(w.current, w.entering)
switch status {
case GoToNext:
w.next()
case SkipChildren:
w.entering = false
w.next()
case Terminate:
return
}
}
}
type nodeWalker struct {
current *Node
root *Node
entering bool
}
func newNodeWalker(root *Node) *nodeWalker {
return &nodeWalker{
current: root,
root: root,
entering: true,
}
}
func (nw *nodeWalker) next() {
if (!nw.current.IsContainer() || !nw.entering) && nw.current == nw.root {
nw.current = nil
return
}
if nw.entering && nw.current.IsContainer() {
if nw.current.FirstChild != nil {
nw.current = nw.current.FirstChild
nw.entering = true
} else {
nw.entering = false
}
} else if nw.current.Next == nil {
nw.current = nw.current.Parent
nw.entering = false
} else {
nw.current = nw.current.Next
nw.entering = true
}
}
func dump(ast *Node) {
fmt.Println(dumpString(ast))
}
func dumpR(ast *Node, depth int) string {
if ast == nil {
return ""
}
indent := bytes.Repeat([]byte("\t"), depth)
content := ast.Literal
if content == nil {
content = ast.content
}
result := fmt.Sprintf("%s%s(%q)\n", indent, ast.Type, content)
for n := ast.FirstChild; n != nil; n = n.Next {
result += dumpR(n, depth+1)
}
return result
}
func dumpString(ast *Node) string {
return dumpR(ast, 0)
}
//
// Blackfriday Markdown Processor
// Available at http://github.com/russross/blackfriday
//
// Copyright © 2011 Russ Ross <russ@russross.com>.
// Distributed under the Simplified BSD License.
// See README.md for details.
//
//
//
// SmartyPants rendering
//
//
package blackfriday
import (
"bytes"
"io"
)
// SPRenderer is a struct containing state of a Smartypants renderer.
type SPRenderer struct {
inSingleQuote bool
inDoubleQuote bool
callbacks [256]smartCallback
}
func wordBoundary(c byte) bool {
return c == 0 || isspace(c) || ispunct(c)
}
func tolower(c byte) byte {
if c >= 'A' && c <= 'Z' {
return c - 'A' + 'a'
}
return c
}
func isdigit(c byte) bool {
return c >= '0' && c <= '9'
}
func smartQuoteHelper(out *bytes.Buffer, previousChar byte, nextChar byte, quote byte, isOpen *bool, addNBSP bool) bool {
// edge of the buffer is likely to be a tag that we don't get to see,
// so we treat it like text sometimes
// enumerate all sixteen possibilities for (previousChar, nextChar)
// each can be one of {0, space, punct, other}
switch {
case previousChar == 0 && nextChar == 0:
// context is not any help here, so toggle
*isOpen = !*isOpen
case isspace(previousChar) && nextChar == 0:
// [ "] might be [ "<code>foo...]
*isOpen = true
case ispunct(previousChar) && nextChar == 0:
// [!"] hmm... could be [Run!"] or [("<code>...]
*isOpen = false
case /* isnormal(previousChar) && */ nextChar == 0:
// [a"] is probably a close
*isOpen = false
case previousChar == 0 && isspace(nextChar):
// [" ] might be [...foo</code>" ]
*isOpen = false
case isspace(previousChar) && isspace(nextChar):
// [ " ] context is not any help here, so toggle
*isOpen = !*isOpen
case ispunct(previousChar) && isspace(nextChar):
// [!" ] is probably a close
*isOpen = false
case /* isnormal(previousChar) && */ isspace(nextChar):
// [a" ] this is one of the easy cases
*isOpen = false
case previousChar == 0 && ispunct(nextChar):
// ["!] hmm... could be ["$1.95] or [</code>"!...]
*isOpen = false
case isspace(previousChar) && ispunct(nextChar):
// [ "!] looks more like [ "$1.95]
*isOpen = true
case ispunct(previousChar) && ispunct(nextChar):
// [!"!] context is not any help here, so toggle
*isOpen = !*isOpen
case /* isnormal(previousChar) && */ ispunct(nextChar):
// [a"!] is probably a close
*isOpen = false
case previousChar == 0 /* && isnormal(nextChar) */ :
// ["a] is probably an open
*isOpen = true
case isspace(previousChar) /* && isnormal(nextChar) */ :
// [ "a] this is one of the easy cases
*isOpen = true
case ispunct(previousChar) /* && isnormal(nextChar) */ :
// [!"a] is probably an open
*isOpen = true
default:
// [a'b] maybe a contraction?
*isOpen = false
}
// Note that with the limited lookahead, this non-breaking
// space will also be appended to single double quotes.
if addNBSP && !*isOpen {
out.WriteString("&nbsp;")
}
out.WriteByte('&')
if *isOpen {
out.WriteByte('l')
} else {
out.WriteByte('r')
}
out.WriteByte(quote)
out.WriteString("quo;")
if addNBSP && *isOpen {
out.WriteString("&nbsp;")
}
return true
}
func (r *SPRenderer) smartSingleQuote(out *bytes.Buffer, previousChar byte, text []byte) int {
if len(text) >= 2 {
t1 := tolower(text[1])
if t1 == '\'' {
nextChar := byte(0)
if len(text) >= 3 {
nextChar = text[2]
}
if smartQuoteHelper(out, previousChar, nextChar, 'd', &r.inDoubleQuote, false) {
return 1
}
}
if (t1 == 's' || t1 == 't' || t1 == 'm' || t1 == 'd') && (len(text) < 3 || wordBoundary(text[2])) {
out.WriteString("&rsquo;")
return 0
}
if len(text) >= 3 {
t2 := tolower(text[2])
if ((t1 == 'r' && t2 == 'e') || (t1 == 'l' && t2 == 'l') || (t1 == 'v' && t2 == 'e')) &&
(len(text) < 4 || wordBoundary(text[3])) {
out.WriteString("&rsquo;")
return 0
}
}
}
nextChar := byte(0)
if len(text) > 1 {
nextChar = text[1]
}
if smartQuoteHelper(out, previousChar, nextChar, 's', &r.inSingleQuote, false) {
return 0
}
out.WriteByte(text[0])
return 0
}
func (r *SPRenderer) smartParens(out *bytes.Buffer, previousChar byte, text []byte) int {
if len(text) >= 3 {
t1 := tolower(text[1])
t2 := tolower(text[2])
if t1 == 'c' && t2 == ')' {
out.WriteString("&copy;")
return 2
}
if t1 == 'r' && t2 == ')' {
out.WriteString("&reg;")
return 2
}
if len(text) >= 4 && t1 == 't' && t2 == 'm' && text[3] == ')' {
out.WriteString("&trade;")
return 3
}
}
out.WriteByte(text[0])
return 0
}
func (r *SPRenderer) smartDash(out *bytes.Buffer, previousChar byte, text []byte) int {
if len(text) >= 2 {
if text[1] == '-' {
out.WriteString("&mdash;")
return 1
}
if wordBoundary(previousChar) && wordBoundary(text[1]) {
out.WriteString("&ndash;")
return 0
}
}
out.WriteByte(text[0])
return 0
}
func (r *SPRenderer) smartDashLatex(out *bytes.Buffer, previousChar byte, text []byte) int {
if len(text) >= 3 && text[1] == '-' && text[2] == '-' {
out.WriteString("&mdash;")
return 2
}
if len(text) >= 2 && text[1] == '-' {
out.WriteString("&ndash;")
return 1
}
out.WriteByte(text[0])
return 0
}
func (r *SPRenderer) smartAmpVariant(out *bytes.Buffer, previousChar byte, text []byte, quote byte, addNBSP bool) int {
if bytes.HasPrefix(text, []byte("&quot;")) {
nextChar := byte(0)
if len(text) >= 7 {
nextChar = text[6]
}
if smartQuoteHelper(out, previousChar, nextChar, quote, &r.inDoubleQuote, addNBSP) {
return 5
}
}
if bytes.HasPrefix(text, []byte("&#0;")) {
return 3
}
out.WriteByte('&')
return 0
}
func (r *SPRenderer) smartAmp(angledQuotes, addNBSP bool) func(*bytes.Buffer, byte, []byte) int {
var quote byte = 'd'
if angledQuotes {
quote = 'a'
}
return func(out *bytes.Buffer, previousChar byte, text []byte) int {
return r.smartAmpVariant(out, previousChar, text, quote, addNBSP)
}
}
func (r *SPRenderer) smartPeriod(out *bytes.Buffer, previousChar byte, text []byte) int {
if len(text) >= 3 && text[1] == '.' && text[2] == '.' {
out.WriteString("&hellip;")
return 2
}
if len(text) >= 5 && text[1] == ' ' && text[2] == '.' && text[3] == ' ' && text[4] == '.' {
out.WriteString("&hellip;")
return 4
}
out.WriteByte(text[0])
return 0
}
func (r *SPRenderer) smartBacktick(out *bytes.Buffer, previousChar byte, text []byte) int {
if len(text) >= 2 && text[1] == '`' {
nextChar := byte(0)
if len(text) >= 3 {
nextChar = text[2]
}
if smartQuoteHelper(out, previousChar, nextChar, 'd', &r.inDoubleQuote, false) {
return 1
}
}
out.WriteByte(text[0])
return 0
}
func (r *SPRenderer) smartNumberGeneric(out *bytes.Buffer, previousChar byte, text []byte) int {
if wordBoundary(previousChar) && previousChar != '/' && len(text) >= 3 {
// is it of the form digits/digits(word boundary)?, i.e., \d+/\d+\b
// note: check for regular slash (/) or fraction slash (⁄, 0x2044, or 0xe2 81 84 in utf-8)
// and avoid changing dates like 1/23/2005 into fractions.
numEnd := 0
for len(text) > numEnd && isdigit(text[numEnd]) {
numEnd++
}
if numEnd == 0 {
out.WriteByte(text[0])
return 0
}
denStart := numEnd + 1
if len(text) > numEnd+3 && text[numEnd] == 0xe2 && text[numEnd+1] == 0x81 && text[numEnd+2] == 0x84 {
denStart = numEnd + 3
} else if len(text) < numEnd+2 || text[numEnd] != '/' {
out.WriteByte(text[0])
return 0
}
denEnd := denStart
for len(text) > denEnd && isdigit(text[denEnd]) {
denEnd++
}
if denEnd == denStart {
out.WriteByte(text[0])
return 0
}
if len(text) == denEnd || wordBoundary(text[denEnd]) && text[denEnd] != '/' {
out.WriteString("<sup>")
out.Write(text[:numEnd])
out.WriteString("</sup>&frasl;<sub>")
out.Write(text[denStart:denEnd])
out.WriteString("</sub>")
return denEnd - 1
}
}
out.WriteByte(text[0])
return 0
}
func (r *SPRenderer) smartNumber(out *bytes.Buffer, previousChar byte, text []byte) int {
if wordBoundary(previousChar) && previousChar != '/' && len(text) >= 3 {
if text[0] == '1' && text[1] == '/' && text[2] == '2' {
if len(text) < 4 || wordBoundary(text[3]) && text[3] != '/' {
out.WriteString("&frac12;")
return 2
}
}
if text[0] == '1' && text[1] == '/' && text[2] == '4' {
if len(text) < 4 || wordBoundary(text[3]) && text[3] != '/' || (len(text) >= 5 && tolower(text[3]) == 't' && tolower(text[4]) == 'h') {
out.WriteString("&frac14;")
return 2
}
}
if text[0] == '3' && text[1] == '/' && text[2] == '4' {
if len(text) < 4 || wordBoundary(text[3]) && text[3] != '/' || (len(text) >= 6 && tolower(text[3]) == 't' && tolower(text[4]) == 'h' && tolower(text[5]) == 's') {
out.WriteString("&frac34;")
return 2
}
}
}
out.WriteByte(text[0])
return 0
}
func (r *SPRenderer) smartDoubleQuoteVariant(out *bytes.Buffer, previousChar byte, text []byte, quote byte) int {
nextChar := byte(0)
if len(text) > 1 {
nextChar = text[1]
}
if !smartQuoteHelper(out, previousChar, nextChar, quote, &r.inDoubleQuote, false) {
out.WriteString("&quot;")
}
return 0
}
func (r *SPRenderer) smartDoubleQuote(out *bytes.Buffer, previousChar byte, text []byte) int {
return r.smartDoubleQuoteVariant(out, previousChar, text, 'd')
}
func (r *SPRenderer) smartAngledDoubleQuote(out *bytes.Buffer, previousChar byte, text []byte) int {
return r.smartDoubleQuoteVariant(out, previousChar, text, 'a')
}
func (r *SPRenderer) smartLeftAngle(out *bytes.Buffer, previousChar byte, text []byte) int {
i := 0
for i < len(text) && text[i] != '>' {
i++
}
out.Write(text[:i+1])
return i
}
type smartCallback func(out *bytes.Buffer, previousChar byte, text []byte) int
// NewSmartypantsRenderer constructs a Smartypants renderer object.
func NewSmartypantsRenderer(flags HTMLFlags) *SPRenderer {
var (
r SPRenderer
smartAmpAngled = r.smartAmp(true, false)
smartAmpAngledNBSP = r.smartAmp(true, true)
smartAmpRegular = r.smartAmp(false, false)
smartAmpRegularNBSP = r.smartAmp(false, true)
addNBSP = flags&SmartypantsQuotesNBSP != 0
)
if flags&SmartypantsAngledQuotes == 0 {
r.callbacks['"'] = r.smartDoubleQuote
if !addNBSP {
r.callbacks['&'] = smartAmpRegular
} else {
r.callbacks['&'] = smartAmpRegularNBSP
}
} else {
r.callbacks['"'] = r.smartAngledDoubleQuote
if !addNBSP {
r.callbacks['&'] = smartAmpAngled
} else {
r.callbacks['&'] = smartAmpAngledNBSP
}
}
r.callbacks['\''] = r.smartSingleQuote
r.callbacks['('] = r.smartParens
if flags&SmartypantsDashes != 0 {
if flags&SmartypantsLatexDashes == 0 {
r.callbacks['-'] = r.smartDash
} else {
r.callbacks['-'] = r.smartDashLatex
}
}
r.callbacks['.'] = r.smartPeriod
if flags&SmartypantsFractions == 0 {
r.callbacks['1'] = r.smartNumber
r.callbacks['3'] = r.smartNumber
} else {
for ch := '1'; ch <= '9'; ch++ {
r.callbacks[ch] = r.smartNumberGeneric
}
}
r.callbacks['<'] = r.smartLeftAngle
r.callbacks['`'] = r.smartBacktick
return &r
}
// Process is the entry point of the Smartypants renderer.
func (r *SPRenderer) Process(w io.Writer, text []byte) {
mark := 0
for i := 0; i < len(text); i++ {
if action := r.callbacks[text[i]]; action != nil {
if i > mark {
w.Write(text[mark:i])
}
previousChar := byte(0)
if i > 0 {
previousChar = text[i-1]
}
var tmp bytes.Buffer
i += action(&tmp, previousChar, text[i:])
w.Write(tmp.Bytes())
mark = i + 1
}
}
if mark < len(text) {
w.Write(text[mark:])
}
}
run:
# do not run on test files yet
tests: false
# all available settings of specific linters
linters-settings:
errcheck:
# report about not checking of errors in type assetions: `a := b.(MyStruct)`;
# default is false: such cases aren't reported by default.
check-type-assertions: false
# report about assignment of errors to blank identifier: `num, _ := strconv.Atoi(numStr)`;
# default is false: such cases aren't reported by default.
check-blank: false
lll:
line-length: 100
tab-width: 4
prealloc:
simple: false
range-loops: false
for-loops: false
whitespace:
multi-if: false # Enforces newlines (or comments) after every multi-line if statement
multi-func: false # Enforces newlines (or comments) after every multi-line function signature
linters:
enable:
- megacheck
- govet
disable:
- maligned
- prealloc
disable-all: false
presets:
- bugs
- unused
fast: false
language: go
go_import_path: github.com/sirupsen/logrus
git:
depth: 1
env:
- GO111MODULE=on
go: 1.15.x
os: linux
install:
- ./travis/install.sh
script:
- cd ci
- go run mage.go -v -w ../ crossBuild
- go run mage.go -v -w ../ lint
- go run mage.go -v -w ../ test
# 1.8.1
Code quality:
* move magefile in its own subdir/submodule to remove magefile dependency on logrus consumer
* improve timestamp format documentation
Fixes:
* fix race condition on logger hooks
# 1.8.0
Correct versioning number replacing v1.7.1.
# 1.7.1
Beware this release has introduced a new public API and its semver is therefore incorrect.
Code quality:
* use go 1.15 in travis
* use magefile as task runner
Fixes:
* small fixes about new go 1.13 error formatting system
* Fix for long time race condiction with mutating data hooks
Features:
* build support for zos
# 1.7.0
Fixes:
* the dependency toward a windows terminal library has been removed
Features:
* a new buffer pool management API has been added
* a set of `<LogLevel>Fn()` functions have been added
# 1.6.0
Fixes:
* end of line cleanup
* revert the entry concurrency bug fix whic leads to deadlock under some circumstances
* update dependency on go-windows-terminal-sequences to fix a crash with go 1.14
Features:
* add an option to the `TextFormatter` to completely disable fields quoting
# 1.5.0
Code quality:
* add golangci linter run on travis
Fixes:
* add mutex for hooks concurrent access on `Entry` data
* caller function field for go1.14
* fix build issue for gopherjs target
Feature:
* add an hooks/writer sub-package whose goal is to split output on different stream depending on the trace level
* add a `DisableHTMLEscape` option in the `JSONFormatter`
* add `ForceQuote` and `PadLevelText` options in the `TextFormatter`
# 1.4.2
* Fixes build break for plan9, nacl, solaris
# 1.4.1
This new release introduces:
* Enhance TextFormatter to not print caller information when they are empty (#944)
* Remove dependency on golang.org/x/crypto (#932, #943)
Fixes:
* Fix Entry.WithContext method to return a copy of the initial entry (#941)
# 1.4.0
This new release introduces:
* Add `DeferExitHandler`, similar to `RegisterExitHandler` but prepending the handler to the list of handlers (semantically like `defer`) (#848).
* Add `CallerPrettyfier` to `JSONFormatter` and `TextFormatter` (#909, #911)
* Add `Entry.WithContext()` and `Entry.Context`, to set a context on entries to be used e.g. in hooks (#919).
Fixes:
* Fix wrong method calls `Logger.Print` and `Logger.Warningln` (#893).
* Update `Entry.Logf` to not do string formatting unless the log level is enabled (#903)
* Fix infinite recursion on unknown `Level.String()` (#907)
* Fix race condition in `getCaller` (#916).
# 1.3.0
This new release introduces:
* Log, Logf, Logln functions for Logger and Entry that take a Level
Fixes:
* Building prometheus node_exporter on AIX (#840)
* Race condition in TextFormatter (#468)
* Travis CI import path (#868)
* Remove coloured output on Windows (#862)
* Pointer to func as field in JSONFormatter (#870)
* Properly marshal Levels (#873)
# 1.2.0
This new release introduces:
* A new method `SetReportCaller` in the `Logger` to enable the file, line and calling function from which the trace has been issued
* A new trace level named `Trace` whose level is below `Debug`
* A configurable exit function to be called upon a Fatal trace
* The `Level` object now implements `encoding.TextUnmarshaler` interface
# 1.1.1
This is a bug fix release.
* fix the build break on Solaris
* don't drop a whole trace in JSONFormatter when a field param is a function pointer which can not be serialized
# 1.1.0
This new release introduces:
* several fixes:
* a fix for a race condition on entry formatting
* proper cleanup of previously used entries before putting them back in the pool
* the extra new line at the end of message in text formatter has been removed
* a new global public API to check if a level is activated: IsLevelEnabled
* the following methods have been added to the Logger object
* IsLevelEnabled
* SetFormatter
* SetOutput
* ReplaceHooks
* introduction of go module
* an indent configuration for the json formatter
* output colour support for windows
* the field sort function is now configurable for text formatter
* the CLICOLOR and CLICOLOR\_FORCE environment variable support in text formater
# 1.0.6
This new release introduces:
* a new api WithTime which allows to easily force the time of the log entry
which is mostly useful for logger wrapper
* a fix reverting the immutability of the entry given as parameter to the hooks
a new configuration field of the json formatter in order to put all the fields
in a nested dictionnary
* a new SetOutput method in the Logger
* a new configuration of the textformatter to configure the name of the default keys
* a new configuration of the text formatter to disable the level truncation
# 1.0.5
* Fix hooks race (#707)
* Fix panic deadlock (#695)
# 1.0.4
* Fix race when adding hooks (#612)
* Fix terminal check in AppEngine (#635)
# 1.0.3
* Replace example files with testable examples
# 1.0.2
* bug: quote non-string values in text formatter (#583)
* Make (*Logger) SetLevel a public method
# 1.0.1
* bug: fix escaping in text formatter (#575)
# 1.0.0
* Officially changed name to lower-case
* bug: colors on Windows 10 (#541)
* bug: fix race in accessing level (#512)
# 0.11.5
* feature: add writer and writerlevel to entry (#372)
# 0.11.4
* bug: fix undefined variable on solaris (#493)
# 0.11.3
* formatter: configure quoting of empty values (#484)
* formatter: configure quoting character (default is `"`) (#484)
* bug: fix not importing io correctly in non-linux environments (#481)
# 0.11.2
* bug: fix windows terminal detection (#476)
# 0.11.1
* bug: fix tty detection with custom out (#471)
# 0.11.0
* performance: Use bufferpool to allocate (#370)
* terminal: terminal detection for app-engine (#343)
* feature: exit handler (#375)
# 0.10.0
* feature: Add a test hook (#180)
* feature: `ParseLevel` is now case-insensitive (#326)
* feature: `FieldLogger` interface that generalizes `Logger` and `Entry` (#308)
* performance: avoid re-allocations on `WithFields` (#335)
# 0.9.0
* logrus/text_formatter: don't emit empty msg
* logrus/hooks/airbrake: move out of main repository
* logrus/hooks/sentry: move out of main repository
* logrus/hooks/papertrail: move out of main repository
* logrus/hooks/bugsnag: move out of main repository
* logrus/core: run tests with `-race`
* logrus/core: detect TTY based on `stderr`
* logrus/core: support `WithError` on logger
* logrus/core: Solaris support
# 0.8.7
* logrus/core: fix possible race (#216)
* logrus/doc: small typo fixes and doc improvements
# 0.8.6
* hooks/raven: allow passing an initialized client
# 0.8.5
* logrus/core: revert #208
# 0.8.4
* formatter/text: fix data race (#218)
# 0.8.3
* logrus/core: fix entry log level (#208)
* logrus/core: improve performance of text formatter by 40%
* logrus/core: expose `LevelHooks` type
* logrus/core: add support for DragonflyBSD and NetBSD
* formatter/text: print structs more verbosely
# 0.8.2
* logrus: fix more Fatal family functions
# 0.8.1
* logrus: fix not exiting on `Fatalf` and `Fatalln`
# 0.8.0
* logrus: defaults to stderr instead of stdout
* hooks/sentry: add special field for `*http.Request`
* formatter/text: ignore Windows for colors
# 0.7.3
* formatter/\*: allow configuration of timestamp layout
# 0.7.2
* formatter/text: Add configuration option for time format (#158)
The MIT License (MIT)
Copyright (c) 2014 Simon Eskildsen
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
# Logrus <img src="http://i.imgur.com/hTeVwmJ.png" width="40" height="40" alt=":walrus:" class="emoji" title=":walrus:"/> [![Build Status](https://github.com/sirupsen/logrus/workflows/CI/badge.svg)](https://github.com/sirupsen/logrus/actions?query=workflow%3ACI) [![Build Status](https://travis-ci.org/sirupsen/logrus.svg?branch=master)](https://travis-ci.org/sirupsen/logrus) [![Go Reference](https://pkg.go.dev/badge/github.com/sirupsen/logrus.svg)](https://pkg.go.dev/github.com/sirupsen/logrus)
Logrus is a structured logger for Go (golang), completely API compatible with
the standard library logger.
**Logrus is in maintenance-mode.** We will not be introducing new features. It's
simply too hard to do in a way that won't break many people's projects, which is
the last thing you want from your Logging library (again...).
This does not mean Logrus is dead. Logrus will continue to be maintained for
security, (backwards compatible) bug fixes, and performance (where we are
limited by the interface).
I believe Logrus' biggest contribution is to have played a part in today's
widespread use of structured logging in Golang. There doesn't seem to be a
reason to do a major, breaking iteration into Logrus V2, since the fantastic Go
community has built those independently. Many fantastic alternatives have sprung
up. Logrus would look like those, had it been re-designed with what we know
about structured logging in Go today. Check out, for example,
[Zerolog][zerolog], [Zap][zap], and [Apex][apex].
[zerolog]: https://github.com/rs/zerolog
[zap]: https://github.com/uber-go/zap
[apex]: https://github.com/apex/log
**Seeing weird case-sensitive problems?** It's in the past been possible to
import Logrus as both upper- and lower-case. Due to the Go package environment,
this caused issues in the community and we needed a standard. Some environments
experienced problems with the upper-case variant, so the lower-case was decided.
Everything using `logrus` will need to use the lower-case:
`github.com/sirupsen/logrus`. Any package that isn't, should be changed.
To fix Glide, see [these
comments](https://github.com/sirupsen/logrus/issues/553#issuecomment-306591437).
For an in-depth explanation of the casing issue, see [this
comment](https://github.com/sirupsen/logrus/issues/570#issuecomment-313933276).
Nicely color-coded in development (when a TTY is attached, otherwise just
plain text):
![Colored](http://i.imgur.com/PY7qMwd.png)
With `log.SetFormatter(&log.JSONFormatter{})`, for easy parsing by logstash
or Splunk:
```text
{"animal":"walrus","level":"info","msg":"A group of walrus emerges from the
ocean","size":10,"time":"2014-03-10 19:57:38.562264131 -0400 EDT"}
{"level":"warning","msg":"The group's number increased tremendously!",
"number":122,"omg":true,"time":"2014-03-10 19:57:38.562471297 -0400 EDT"}
{"animal":"walrus","level":"info","msg":"A giant walrus appears!",
"size":10,"time":"2014-03-10 19:57:38.562500591 -0400 EDT"}
{"animal":"walrus","level":"info","msg":"Tremendously sized cow enters the ocean.",
"size":9,"time":"2014-03-10 19:57:38.562527896 -0400 EDT"}
{"level":"fatal","msg":"The ice breaks!","number":100,"omg":true,
"time":"2014-03-10 19:57:38.562543128 -0400 EDT"}
```
With the default `log.SetFormatter(&log.TextFormatter{})` when a TTY is not
attached, the output is compatible with the
[logfmt](http://godoc.org/github.com/kr/logfmt) format:
```text
time="2015-03-26T01:27:38-04:00" level=debug msg="Started observing beach" animal=walrus number=8
time="2015-03-26T01:27:38-04:00" level=info msg="A group of walrus emerges from the ocean" animal=walrus size=10
time="2015-03-26T01:27:38-04:00" level=warning msg="The group's number increased tremendously!" number=122 omg=true
time="2015-03-26T01:27:38-04:00" level=debug msg="Temperature changes" temperature=-4
time="2015-03-26T01:27:38-04:00" level=panic msg="It's over 9000!" animal=orca size=9009
time="2015-03-26T01:27:38-04:00" level=fatal msg="The ice breaks!" err=&{0x2082280c0 map[animal:orca size:9009] 2015-03-26 01:27:38.441574009 -0400 EDT panic It's over 9000!} number=100 omg=true
```
To ensure this behaviour even if a TTY is attached, set your formatter as follows:
```go
log.SetFormatter(&log.TextFormatter{
DisableColors: true,
FullTimestamp: true,
})
```
#### Logging Method Name
If you wish to add the calling method as a field, instruct the logger via:
```go
log.SetReportCaller(true)
```
This adds the caller as 'method' like so:
```json
{"animal":"penguin","level":"fatal","method":"github.com/sirupsen/arcticcreatures.migrate","msg":"a penguin swims by",
"time":"2014-03-10 19:57:38.562543129 -0400 EDT"}
```
```text
time="2015-03-26T01:27:38-04:00" level=fatal method=github.com/sirupsen/arcticcreatures.migrate msg="a penguin swims by" animal=penguin
```
Note that this does add measurable overhead - the cost will depend on the version of Go, but is
between 20 and 40% in recent tests with 1.6 and 1.7. You can validate this in your
environment via benchmarks:
```
go test -bench=.*CallerTracing
```
#### Case-sensitivity
The organization's name was changed to lower-case--and this will not be changed
back. If you are getting import conflicts due to case sensitivity, please use
the lower-case import: `github.com/sirupsen/logrus`.
#### Example
The simplest way to use Logrus is simply the package-level exported logger:
```go
package main
import (
log "github.com/sirupsen/logrus"
)
func main() {
log.WithFields(log.Fields{
"animal": "walrus",
}).Info("A walrus appears")
}
```
Note that it's completely api-compatible with the stdlib logger, so you can
replace your `log` imports everywhere with `log "github.com/sirupsen/logrus"`
and you'll now have the flexibility of Logrus. You can customize it all you
want:
```go
package main
import (
"os"
log "github.com/sirupsen/logrus"
)
func init() {
// Log as JSON instead of the default ASCII formatter.
log.SetFormatter(&log.JSONFormatter{})
// Output to stdout instead of the default stderr
// Can be any io.Writer, see below for File example
log.SetOutput(os.Stdout)
// Only log the warning severity or above.
log.SetLevel(log.WarnLevel)
}
func main() {
log.WithFields(log.Fields{
"animal": "walrus",
"size": 10,
}).Info("A group of walrus emerges from the ocean")
log.WithFields(log.Fields{
"omg": true,
"number": 122,
}).Warn("The group's number increased tremendously!")
log.WithFields(log.Fields{
"omg": true,
"number": 100,
}).Fatal("The ice breaks!")
// A common pattern is to re-use fields between logging statements by re-using
// the logrus.Entry returned from WithFields()
contextLogger := log.WithFields(log.Fields{
"common": "this is a common field",
"other": "I also should be logged always",
})
contextLogger.Info("I'll be logged with common and other field")
contextLogger.Info("Me too")
}
```
For more advanced usage such as logging to multiple locations from the same
application, you can also create an instance of the `logrus` Logger:
```go
package main
import (
"os"
"github.com/sirupsen/logrus"
)
// Create a new instance of the logger. You can have any number of instances.
var log = logrus.New()
func main() {
// The API for setting attributes is a little different than the package level
// exported logger. See Godoc.
log.Out = os.Stdout
// You could set this to any `io.Writer` such as a file
// file, err := os.OpenFile("logrus.log", os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0666)
// if err == nil {
// log.Out = file
// } else {
// log.Info("Failed to log to file, using default stderr")
// }
log.WithFields(logrus.Fields{
"animal": "walrus",
"size": 10,
}).Info("A group of walrus emerges from the ocean")
}
```
#### Fields
Logrus encourages careful, structured logging through logging fields instead of
long, unparseable error messages. For example, instead of: `log.Fatalf("Failed
to send event %s to topic %s with key %d")`, you should log the much more
discoverable:
```go
log.WithFields(log.Fields{
"event": event,
"topic": topic,
"key": key,
}).Fatal("Failed to send event")
```
We've found this API forces you to think about logging in a way that produces
much more useful logging messages. We've been in countless situations where just
a single added field to a log statement that was already there would've saved us
hours. The `WithFields` call is optional.
In general, with Logrus using any of the `printf`-family functions should be
seen as a hint you should add a field, however, you can still use the
`printf`-family functions with Logrus.
#### Default Fields
Often it's helpful to have fields _always_ attached to log statements in an
application or parts of one. For example, you may want to always log the
`request_id` and `user_ip` in the context of a request. Instead of writing
`log.WithFields(log.Fields{"request_id": request_id, "user_ip": user_ip})` on
every line, you can create a `logrus.Entry` to pass around instead:
```go
requestLogger := log.WithFields(log.Fields{"request_id": request_id, "user_ip": user_ip})
requestLogger.Info("something happened on that request") # will log request_id and user_ip
requestLogger.Warn("something not great happened")
```
#### Hooks
You can add hooks for logging levels. For example to send errors to an exception
tracking service on `Error`, `Fatal` and `Panic`, info to StatsD or log to
multiple places simultaneously, e.g. syslog.
Logrus comes with [built-in hooks](hooks/). Add those, or your custom hook, in
`init`:
```go
import (
log "github.com/sirupsen/logrus"
"gopkg.in/gemnasium/logrus-airbrake-hook.v2" // the package is named "airbrake"
logrus_syslog "github.com/sirupsen/logrus/hooks/syslog"
"log/syslog"
)
func init() {
// Use the Airbrake hook to report errors that have Error severity or above to
// an exception tracker. You can create custom hooks, see the Hooks section.
log.AddHook(airbrake.NewHook(123, "xyz", "production"))
hook, err := logrus_syslog.NewSyslogHook("udp", "localhost:514", syslog.LOG_INFO, "")
if err != nil {
log.Error("Unable to connect to local syslog daemon")
} else {
log.AddHook(hook)
}
}
```
Note: Syslog hook also support connecting to local syslog (Ex. "/dev/log" or "/var/run/syslog" or "/var/run/log"). For the detail, please check the [syslog hook README](hooks/syslog/README.md).
A list of currently known service hooks can be found in this wiki [page](https://github.com/sirupsen/logrus/wiki/Hooks)
#### Level logging
Logrus has seven logging levels: Trace, Debug, Info, Warning, Error, Fatal and Panic.
```go
log.Trace("Something very low level.")
log.Debug("Useful debugging information.")
log.Info("Something noteworthy happened!")
log.Warn("You should probably take a look at this.")
log.Error("Something failed but I'm not quitting.")
// Calls os.Exit(1) after logging
log.Fatal("Bye.")
// Calls panic() after logging
log.Panic("I'm bailing.")
```
You can set the logging level on a `Logger`, then it will only log entries with
that severity or anything above it:
```go
// Will log anything that is info or above (warn, error, fatal, panic). Default.
log.SetLevel(log.InfoLevel)
```
It may be useful to set `log.Level = logrus.DebugLevel` in a debug or verbose
environment if your application has that.
Note: If you want different log levels for global (`log.SetLevel(...)`) and syslog logging, please check the [syslog hook README](hooks/syslog/README.md#different-log-levels-for-local-and-remote-logging).
#### Entries
Besides the fields added with `WithField` or `WithFields` some fields are
automatically added to all logging events:
1. `time`. The timestamp when the entry was created.
2. `msg`. The logging message passed to `{Info,Warn,Error,Fatal,Panic}` after
the `AddFields` call. E.g. `Failed to send event.`
3. `level`. The logging level. E.g. `info`.
#### Environments
Logrus has no notion of environment.
If you wish for hooks and formatters to only be used in specific environments,
you should handle that yourself. For example, if your application has a global
variable `Environment`, which is a string representation of the environment you
could do:
```go
import (
log "github.com/sirupsen/logrus"
)
func init() {
// do something here to set environment depending on an environment variable
// or command-line flag
if Environment == "production" {
log.SetFormatter(&log.JSONFormatter{})
} else {
// The TextFormatter is default, you don't actually have to do this.
log.SetFormatter(&log.TextFormatter{})
}
}
```
This configuration is how `logrus` was intended to be used, but JSON in
production is mostly only useful if you do log aggregation with tools like
Splunk or Logstash.
#### Formatters
The built-in logging formatters are:
* `logrus.TextFormatter`. Logs the event in colors if stdout is a tty, otherwise
without colors.
* *Note:* to force colored output when there is no TTY, set the `ForceColors`
field to `true`. To force no colored output even if there is a TTY set the
`DisableColors` field to `true`. For Windows, see
[github.com/mattn/go-colorable](https://github.com/mattn/go-colorable).
* When colors are enabled, levels are truncated to 4 characters by default. To disable
truncation set the `DisableLevelTruncation` field to `true`.
* When outputting to a TTY, it's often helpful to visually scan down a column where all the levels are the same width. Setting the `PadLevelText` field to `true` enables this behavior, by adding padding to the level text.
* All options are listed in the [generated docs](https://godoc.org/github.com/sirupsen/logrus#TextFormatter).
* `logrus.JSONFormatter`. Logs fields as JSON.
* All options are listed in the [generated docs](https://godoc.org/github.com/sirupsen/logrus#JSONFormatter).
Third party logging formatters:
* [`FluentdFormatter`](https://github.com/joonix/log). Formats entries that can be parsed by Kubernetes and Google Container Engine.
* [`GELF`](https://github.com/fabienm/go-logrus-formatters). Formats entries so they comply to Graylog's [GELF 1.1 specification](http://docs.graylog.org/en/2.4/pages/gelf.html).
* [`logstash`](https://github.com/bshuster-repo/logrus-logstash-hook). Logs fields as [Logstash](http://logstash.net) Events.
* [`prefixed`](https://github.com/x-cray/logrus-prefixed-formatter). Displays log entry source along with alternative layout.
* [`zalgo`](https://github.com/aybabtme/logzalgo). Invoking the Power of Zalgo.
* [`nested-logrus-formatter`](https://github.com/antonfisher/nested-logrus-formatter). Converts logrus fields to a nested structure.
* [`powerful-logrus-formatter`](https://github.com/zput/zxcTool). get fileName, log's line number and the latest function's name when print log; Sava log to files.
* [`caption-json-formatter`](https://github.com/nolleh/caption_json_formatter). logrus's message json formatter with human-readable caption added.
You can define your formatter by implementing the `Formatter` interface,
requiring a `Format` method. `Format` takes an `*Entry`. `entry.Data` is a
`Fields` type (`map[string]interface{}`) with all your fields as well as the
default ones (see Entries section above):
```go
type MyJSONFormatter struct {
}
log.SetFormatter(new(MyJSONFormatter))
func (f *MyJSONFormatter) Format(entry *Entry) ([]byte, error) {
// Note this doesn't include Time, Level and Message which are available on
// the Entry. Consult `godoc` on information about those fields or read the
// source of the official loggers.
serialized, err := json.Marshal(entry.Data)
if err != nil {
return nil, fmt.Errorf("Failed to marshal fields to JSON, %w", err)
}
return append(serialized, '\n'), nil
}
```
#### Logger as an `io.Writer`
Logrus can be transformed into an `io.Writer`. That writer is the end of an `io.Pipe` and it is your responsibility to close it.
```go
w := logger.Writer()
defer w.Close()
srv := http.Server{
// create a stdlib log.Logger that writes to
// logrus.Logger.
ErrorLog: log.New(w, "", 0),
}
```
Each line written to that writer will be printed the usual way, using formatters
and hooks. The level for those entries is `info`.
This means that we can override the standard library logger easily:
```go
logger := logrus.New()
logger.Formatter = &logrus.JSONFormatter{}
// Use logrus for standard log output
// Note that `log` here references stdlib's log
// Not logrus imported under the name `log`.
log.SetOutput(logger.Writer())
```
#### Rotation
Log rotation is not provided with Logrus. Log rotation should be done by an
external program (like `logrotate(8)`) that can compress and delete old log
entries. It should not be a feature of the application-level logger.
#### Tools
| Tool | Description |
| ---- | ----------- |
|[Logrus Mate](https://github.com/gogap/logrus_mate)|Logrus mate is a tool for Logrus to manage loggers, you can initial logger's level, hook and formatter by config file, the logger will be generated with different configs in different environments.|
|[Logrus Viper Helper](https://github.com/heirko/go-contrib/tree/master/logrusHelper)|An Helper around Logrus to wrap with spf13/Viper to load configuration with fangs! And to simplify Logrus configuration use some behavior of [Logrus Mate](https://github.com/gogap/logrus_mate). [sample](https://github.com/heirko/iris-contrib/blob/master/middleware/logrus-logger/example) |
#### Testing
Logrus has a built in facility for asserting the presence of log messages. This is implemented through the `test` hook and provides:
* decorators for existing logger (`test.NewLocal` and `test.NewGlobal`) which basically just adds the `test` hook
* a test logger (`test.NewNullLogger`) that just records log messages (and does not output any):
```go
import(
"github.com/sirupsen/logrus"
"github.com/sirupsen/logrus/hooks/test"
"github.com/stretchr/testify/assert"
"testing"
)
func TestSomething(t*testing.T){
logger, hook := test.NewNullLogger()
logger.Error("Helloerror")
assert.Equal(t, 1, len(hook.Entries))
assert.Equal(t, logrus.ErrorLevel, hook.LastEntry().Level)
assert.Equal(t, "Helloerror", hook.LastEntry().Message)
hook.Reset()
assert.Nil(t, hook.LastEntry())
}
```
#### Fatal handlers
Logrus can register one or more functions that will be called when any `fatal`
level message is logged. The registered handlers will be executed before
logrus performs an `os.Exit(1)`. This behavior may be helpful if callers need
to gracefully shutdown. Unlike a `panic("Something went wrong...")` call which can be intercepted with a deferred `recover` a call to `os.Exit(1)` can not be intercepted.
```
...
handler := func() {
// gracefully shutdown something...
}
logrus.RegisterExitHandler(handler)
...
```
#### Thread safety
By default, Logger is protected by a mutex for concurrent writes. The mutex is held when calling hooks and writing logs.
If you are sure such locking is not needed, you can call logger.SetNoLock() to disable the locking.
Situation when locking is not needed includes:
* You have no hooks registered, or hooks calling is already thread-safe.
* Writing to logger.Out is already thread-safe, for example:
1) logger.Out is protected by locks.
2) logger.Out is an os.File handler opened with `O_APPEND` flag, and every write is smaller than 4k. (This allows multi-thread/multi-process writing)
(Refer to http://www.notthewizard.com/2014/06/17/are-files-appends-really-atomic/)
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment