packages/docutils/rstgen

Search:
Source   Edit  

This module implements a generator of HTML/Latex from reStructuredText (see https://docutils.sourceforge.net/rst.html for information on this markup syntax) and is used by the compiler's docgen tools.

You can generate HTML output through the convenience proc rstToHtml, which provided an input string with rst markup returns a string with the generated HTML. The final output is meant to be embedded inside a full document you provide yourself, so it won't contain the usual <header> or <body> parts.

You can also create a RstGenerator structure and populate it with the other lower level methods to finally build complete documents. This requires many options and tweaking, but you are not limited to snippets and can generate LaTeX documents too.

Docutils configuration files are not supported. Instead HTML generation can be tweaked by editing file config/nimdoc.cfg.

There are stylistic difference between how this module renders some elements and how original Python Docutils does:

  • Backreferences to TOC in section headings are not generated. In HTML each section is also a link that points to the section itself: this is done for user to be able to copy the link into clipboard.
  • The same goes for footnotes/citations links: they point to themselves. No backreferences are generated since finding all references of a footnote can be done by simply searching for [footnoteName].

Types

EscapeMode = enum
  emText, emOption, emUrl
Source   Edit  
IndexedDocs = Table[IndexEntry, seq[IndexEntry]]

Contains the index sequences for doc types.

The key is a fake IndexEntry which will contain the title of the document in the keyword field and link will contain the html filename for the document. linkTitle and linkDesc will be empty.

The value indexed by this IndexEntry is a sequence with the real index entries found in the .idx file.

Source   Edit  
MetaEnum = enum
  metaNone, metaTitleRaw, metaTitle, metaSubtitle, metaAuthor, metaVersion
Source   Edit  
OutputTarget = enum
  outHtml, outLatex
which document type to generate Source   Edit  
RstGenerator = object of RootObj
  target*: OutputTarget
  config*: StringTableRef
  splitAfter*: int
  listingCounter*: int
  tocPart*: seq[PRstNode]
  hasToc*: bool
  findFile*: FindFileHandler
  msgHandler*: MsgHandler
  outDir*: string            ## output directory, initialized by docgen.nim
  destFile*: string          ## output (HTML) file, initialized by docgen.nim
  filenames*: RstFileTable
  filename*: string          ## source Nim or Rst file
  meta*: array[MetaEnum, string]
  ## \
  ## Stores the empty string or the last headline/overline found in the rst
  ## document, so it can be used as a prettier name for term index generation.
  ## \
  ## Keeps count of same text index terms to generate different identifiers
  ## for hyperlinks. See renderIndexTerm proc for details.
  id*: int                   ## A counter useful for generating IDs.
  onTestSnippet*: proc (d: var RstGenerator; filename, cmd: string; status: int;
                        content: string) {....gcsafe.}
  escMode*: EscapeMode
Source   Edit  

Consts

IndexExt = ".idx"
Source   Edit  

Procs

proc defaultConfig(): StringTableRef {....raises: [], tags: [], forbids: [].}

Returns a default configuration for embedded HTML generation.

The returned StringTableRef contains the parameters used by the HTML engine to build the final output. For information on what these parameters are and their purpose, please look up the file config/nimdoc.cfg bundled with the compiler.

The only difference between the contents of that file and the values provided by this proc is the doc.file variable. The doc.file variable of the configuration file contains HTML to build standalone pages, while this proc returns just the content for procs like rstToHtml to generate the bare minimum HTML.

Source   Edit  
proc esc(target: OutputTarget; s: string; splitAfter = -1; escMode = emText): string {.
    ...raises: [], tags: [], forbids: [].}
Escapes the HTML. Source   Edit  
proc escChar(target: OutputTarget; dest: var string; c: char;
             escMode: EscapeMode) {.inline, ...raises: [], tags: [], forbids: [].}
Source   Edit  
proc formatNamedVars(frmt: string; varnames: openArray[string];
                     varvalues: openArray[string]): string {.
    ...raises: [ValueError], tags: [], forbids: [].}
Source   Edit  
proc initRstGenerator(g: var RstGenerator; target: OutputTarget;
                      config: StringTableRef; filename: string;
                      findFile: FindFileHandler = nil;
                      msgHandler: MsgHandler = nil;
                      filenames = default(RstFileTable); hasToc = false) {.
    ...raises: [ValueError], tags: [], forbids: [].}

Initializes a RstGenerator.

You need to call this before using a RstGenerator with any other procs in this module. Pass a non nil StringTableRef value as config with parameters used by the HTML output generator. If you don't know what to use, pass the results of the defaultConfig() <#defaultConfig>_ proc.

The filename parameter will be used for error reporting and creating index hyperlinks to the file, but you can pass an empty string here if you are parsing a stream in memory. If filename ends with the .nim extension, the title for the document will be set by default to Module filename. This default title can be overridden by the embedded rst, but it helps to prettify the generated index if no title is found.

The RstParseOptions, FindFileHandler and MsgHandler types are defined in the packages/docutils/rst module. options selects the behaviour of the rst parser.

findFile is a proc used by the rst include directive among others. The purpose of this proc is to mangle or filter paths. It receives paths specified in the rst document and has to return a valid path to existing files or the empty string otherwise. If you pass nil, a default proc will be used which given a path returns the input path only if the file exists. One use for this proc is to transform relative paths found in the document to absolute path, useful if the rst file and the resources it references are not in the same directory as the current working directory.

The msgHandler is a proc used for user error reporting. It will be called with the filename, line, col, and type of any error found during parsing. If you pass nil, a default message handler will be used which writes the messages to the standard output.

Example:

import packages/docutils/rstgen

var gen: RstGenerator
gen.initRstGenerator(outHtml, defaultConfig(), "filename", {})

Source   Edit  
proc mergeIndexes(dir: string): string {....raises: [OSError, IOError, ValueError],
    tags: [ReadDirEffect, ReadIOEffect], forbids: [].}

Merges all index files in dir and returns the generated index as HTML.

This proc will first scan dir for index files with the .idx extension previously created by commands like nim doc|rst2html which use the --index:on switch. These index files are the result of calls to setIndexTerm() and writeIndexFile(), so they are simple tab separated files.

As convention this proc will split index files into two categories: documentation and API. API indices will be all joined together into a single big sorted index, making the bulk of the final index. This is good for API documentation because many symbols are repeated in different modules. On the other hand, documentation indices are essentially table of contents plus a few special markers. These documents will be rendered in a separate section which tries to maintain the order and hierarchy of the symbols in the index file.

To differentiate between a documentation and API file a convention is used: indices which contain one entry without the HTML hash character (#) will be considered documentation, since this hash-less entry is the explicit title of the document. Indices without this explicit entry will be considered generated API extracted out of a source .nim file.

Returns the merged and sorted indices into a single HTML block which can be further embedded into nimdoc templates.

Source   Edit  
proc nextSplitPoint(s: string; start: int): int {....raises: [], tags: [],
    forbids: [].}
Source   Edit  
proc readIndexDir(dir: string): tuple[modules: seq[string],
                                      symbols: seq[IndexEntry],
                                      docs: IndexedDocs] {.
    ...raises: [OSError, IOError, ValueError], tags: [ReadDirEffect, ReadIOEffect],
    forbids: [].}

Walks dir reading .idx files converting them in IndexEntry items.

Returns the list of found module names, the list of free symbol entries and the different documentation indexes. The list of modules is sorted. See the documentation of mergeIndexes for details.

Source   Edit  
proc renderCodeLang(result: var string; lang: SourceLanguage; code: string;
                    target: OutputTarget) {....raises: [ValueError], tags: [],
    forbids: [].}
Source   Edit  
proc renderIndexTerm(d: PDoc; n: PRstNode; result: var string) {.
    ...raises: [Exception, ValueError], tags: [RootEffect], forbids: [].}

Renders the string decorated within `foobar`:idx: markers.

Additionally adds the enclosed text to the index as a term. Since we are interested in different instances of the same term to have different entries, a table is used to keep track of the amount of times a term has previously appeared to give a different identifier value for each.

Source   Edit  
proc renderNimCode(result: var string; code: string; target: OutputTarget) {.
    ...raises: [ValueError], tags: [], forbids: [].}
Source   Edit  
proc renderRstToOut(d: var RstGenerator; n: PRstNode; result: var string) {.
    ...gcsafe, raises: [Exception, ValueError], tags: [RootEffect], forbids: [].}

Writes into result the rst ast n using the d configuration.

Before using this proc you need to initialise a RstGenerator with initRstGenerator and parse a rst file with rstParse from the packages/docutils/rst module. Example:

# ...configure gen and rst vars...
var generatedHtml = ""
renderRstToOut(gen, rst, generatedHtml)
echo generatedHtml

Source   Edit  
proc renderTocEntries(d: var RstGenerator; j: var int; lvl: int;
                      result: var string) {....raises: [Exception, ValueError],
    tags: [RootEffect], forbids: [].}
Source   Edit  
proc rstToHtml(s: string; options: RstParseOptions; config: StringTableRef;
               msgHandler: MsgHandler = rst.defaultMsgHandler): string {....gcsafe,
    raises: [Exception, ValueError, KeyError],
    tags: [RootEffect, ReadIOEffect, ReadEnvEffect], forbids: [].}

Converts an input rst string into embeddable HTML.

This convenience proc parses any input string using rst markup (it doesn't have to be a full document!) and returns an embeddable piece of HTML. The proc is meant to be used in online environments without access to a meaningful filesystem, and therefore rst include like directives won't work. For an explanation of the config parameter see the initRstGenerator proc. Example:

import packages/docutils/rstgen, strtabs

echo rstToHtml("*Hello* **world**!", {},
  newStringTable(modeStyleInsensitive))
# --> <em>Hello</em> <strong>world</strong>!

If you need to allow the rst include directive or tweak the generated output you have to create your own RstGenerator with initRstGenerator and related procs.

Source   Edit  
proc rstToLatex(rstSource: string; options: RstParseOptions): string {.inline,
    ...raises: [Exception, ValueError, KeyError],
    tags: [RootEffect, ReadIOEffect, ReadEnvEffect], forbids: [].}
Convenience proc for renderRstToOut and initRstGenerator.

Example:

doAssert rstToLatex("*Hello* **world**", {}) == """\emph{Hello} \textbf{world}"""
Source   Edit  
proc setIndexTerm(d: var RstGenerator; k: IndexEntryKind;
                  htmlFile, id, term: string; linkTitle, linkDesc = ""; line = 0) {.
    ...raises: [], tags: [], forbids: [].}

Adds a term to the index using the specified hyperlink identifier.

A new entry will be added to the index using the format term<tab>file#id. The file part will come from the htmlFile parameter.

The id will be appended with a hash character only if its length is not zero, otherwise no specific anchor will be generated. In general you should only pass an empty id value for the title of standalone rst documents (they are special for the mergeIndexes() proc, see Index (idx) file format for more information). Unlike other index terms, title entries are inserted at the beginning of the accumulated buffer to maintain a logical order of entries.

If linkTitle or linkDesc are not the empty string, two additional columns with their contents will be added.

The index won't be written to disk unless you call writeIndexFile(). The purpose of the index is documented in the docgen tools guide.

Source   Edit  
proc traverseForIndex(d: PDoc; n: PRstNode) {....raises: [Exception, ValueError],
    tags: [RootEffect], forbids: [].}
A version of renderRstToOut that only fills entries for .idx files. Source   Edit  
proc writeIndexFile(g: var RstGenerator; outfile: string) {....raises: [IOError],
    tags: [WriteIOEffect], forbids: [].}

Writes the current index buffer to the specified output file.

You previously need to add entries to the index with the setIndexTerm() proc. If the index is empty the file won't be created.

Source   Edit