The automarkup code tries to look up symbols once as a function, and once
as a macro. The Sphinx C domain code, though, totally ignores that
distinction and will return the same results either way. So just look
things up once and be done with it; the resulting output does not change,
but htmldocs build time drops by about 5%.
Tested-by: Akira Yokosawa <akiyks@gmail.com>
Link: https://lore.kernel.org/r/20220630163630.714673-3-corbet@lwn.net
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The automarkup code tries to create a lot of cross-references that don't
exist. Cross-reference lookups are expensive, especially in later versions
of Sphinx, so there is value in avoiding unnecessary ones. Remember
attempts that failed and do not retry them.
This improves the htmldocs build time by 5-10% depending on the phase of
the moon and other factors.
Tested-by: Akira Yokosawa <akiyks@gmail.com>
Link: https://lore.kernel.org/r/20220630163630.714673-2-corbet@lwn.net
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Since commit d18b01789a ("docs: Add automatic cross-reference for
documentation pages"), references that were already explicitly defined
with "ref:" and referred to other pages with a path have been doubled.
This is reported as the following error by Firefox:
Start tag "a" seen but an element of the same type was already open.
End tag "a" violates nesting rules.
As well as the invalid HTML, this also obscures the URI fragment links
to subsections because the second link overrides the first. For example
on the page admin-guide/hw-vuln/mds.html the last link should be to the
"Default Mitigations" subsection using a # URI fragment:
admin-guide/hw-vuln/l1tf.html#default-mitigations
But it is obsured by a second link to the whole page:
admin-guide/hw-vuln/l1tf.html
The full HTML with the double <a> tags looks like this:
<a class="reference internal" href="l1tf.html#default-mitigations">
<span class="std std-ref">
<a class="reference internal" href="l1tf.html">
<span class="doc">L1TF - L1 Terminal Fault</span>
</a>
</span>
</a>
After this commit, there is only a single link:
<a class="reference internal" href="l1tf.html#default-mitigations">
<span class="std std-ref">Documentation/admin-guide/hw-vuln//l1tf.rst</span>
</a>
Now that the second link is removed, the browser correctly jumps to the
default-mitigations subsection when clicking the link.
The fix is to check that nodes in the document to be modified are not
already references. A reference is counted as any text that is a
descendant of a reference type node. Only plain text should be converted
to new references, otherwise the doubling occurs.
Testing
=======
* Test that the build stdout is the same (ignoring ordering), and that
no new warnings are printed.
* Diff all .html files and check that the only modifications occur
to the bad double links.
* The auto linking of bare references to pages without "ref:" is still
working.
Fixes: d18b01789a ("docs: Add automatic cross-reference for documentation pages")
Reviewed-by: Nícolas F. R. A. Prado <n@nfraprado.net>
Signed-off-by: James Clark <james.clark@arm.com>
Link: https://lore.kernel.org/r/20220105143640.330602-2-james.clark@arm.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Previously, a cross-reference to another document could only be created
by writing the full path to the document starting from the
Documentation/ directory.
Extend this to also allow relative paths to be used. A relative path
would be just the path, like ../filename.rst, while the absolute path
still needs to start from Documentation, like Documentation/filename.rst.
As part of this change, the .rst extension is now required for both
types of paths, since not requiring it would cause the regex to be too
generic.
Suggested-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
Link: https://lore.kernel.org/r/20210128010028.58541-2-nfraprado@protonmail.com
[jc: Tweaked the regex to recognize .txt too]
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Sphinx 3.1 introduced namespaces for C cross-references. With this,
each C domain type/function declaration is put inside the namespace that
was active at the time of its declaration.
Add support for automatic cross-referencing inside C namespaces by
checking whether the corresponding source file had a C namespace Sphinx
directive, and if so, try cross-referencing inside of it before going to
the global scope.
This assumes there's only one namespace (if any) per rst file.
Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
Link: https://lore.kernel.org/r/20201117021107.214704-1-nfraprado@protonmail.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
It turns out that the Python 2 re module lacks the ASCII flag, so don't try
to use it there.
Fixes: f66e47f98c ("docs: automarkup.py: Fix regexes to solve sphinx 3 warnings")
Reported-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Sphinx 3 added support for declaring C macros with parameters using the
:c:macro role.
To support automarkup for both functions and parametrized macros using
the same regex (words ending in ()), try to cross-reference to both, and
only fall back to regular text if neither exist.
Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
With the transition to Sphinx 3, new warnings were caused by
automarkup, exposing bugs in the name matching.
When automarkup parsed a text like "struct struct" in the documentation,
it tried to cross-reference to a "struct" symbol, which is recognized as
a C reserved word by Sphinx 3, generating a warning.
Add some C reserved words (only the ones that were causing warnings) to
a list and skip them while trying to cross-reference.
Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
With the transition to Sphinx 3, new warnings were generated by
automarkup, exposing bugs in the regexes.
The warnings were caused by the expressions matching words in the
translated versions of the documentation, since any unicode character
was matched.
Fix the regular expression by making the C regexes use ASCII and
ensuring the expressions only match the beginning of words,
in order to avoid warnings like this:
WARNING: Unparseable C cross-reference: '调用debugfs_rename'
That's probably due to the lack of using spaces between words
on Chinese.
Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
While Sphinx 2 used a single c:type role for struct, union, enum and
typedef, Sphinx 3 uses a specific role for each one.
To keep backward compatibility, detect the Sphinx version and use the
correct roles for that version.
Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Cross-referencing to other documentation pages is possible using the
:doc:`doc-file` directive from Sphinx.
Add automatic markup for references to other documentation pages in the
format Documentation/subfolder/doc-file.rst (the extension being
optional).
This requires that the path be passed all the way from the Documentation
folder, which can be longer than passing a relative path through the
:doc: directive, but avoids the markup, making the text cleaner when
read in plain text.
Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
Link: https://lore.kernel.org/r/20200911133339.327721-3-nfraprado@protonmail.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The automarkup script previously matched expressions and substituted
them with markup to enable automatic cross-reference all in the same
function.
Split the expression matching iteration and the markup substitution into
different functions to make it easier to add new regular expressions and
functions to treat each of them.
Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
Link: https://lore.kernel.org/r/20200911133339.327721-2-nfraprado@protonmail.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
In order to cross-reference C types in the documentation, Sphinx
requires the syntax :c:type:`type_name`, or even :c:type:`struct
type_name <type_name>` in order to have the link text different from the
target text.
Extend automarkup to enable automatic cross-reference of C types by
matching any "struct|union|enum|typedef type_name" expression.
This makes the documentation's plain text cleaner and adds
cross-reference to types without any additional effort by the author.
Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
Link: https://lore.kernel.org/r/20200903005747.3900333-2-nfraprado@protonmail.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Sphinx 2.1 moved sphinx.environment.NoUri into sphinx.errors; that produced
this warning in the docs build:
/usr/lib/python3.7/site-packages/sphinx/registry.py:473:
RemovedInSphinx30Warning: sphinx.environment.NoUri is deprecated.
Grab NoUri from the right place and make the warning go away. That symbol
was only added to sphinx.errors in 2.1, so we must still import it from the
old location when running in older versions.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
With the introduction of Documentation/sphinx/automarkup.py, socket() is
parsed as a reference to the in-kernel definition of socket. Sphinx then
decides that struct socket is a good match, which is usually not
intended, when the syscall is meant instead. This was observed in
Documentation/networking/af_xdp.rst.
Prevent socket() from being misinterpreted by adding it to the Skipfuncs
list in automarkup.py.
Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
Reviewed-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
In Python, like in C, when a comma is omitted in a list of strings, the
two strings around the missing comma are concatenated.
Cc: stable@vger.kernel.org # v5.2 only
Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
Reviewed-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
When using the automarkup extension with:
make pdfdocs
without passing an specific book, the code will raise an exception:
File "/devel/v4l/docs/Documentation/sphinx/automarkup.py", line 86, in auto_markup
node.parent.replace(node, markup_funcs(name, app, node))
File "/devel/v4l/docs/Documentation/sphinx/automarkup.py", line 59, in markup_funcs
'function', target, pxref, lit_text)
File "/devel/v4l/docs/sphinx_2.0/lib/python3.7/site-packages/sphinx/domains/c.py", line 308, in resolve_xref
contnode, target)
File "/devel/v4l/docs/sphinx_2.0/lib/python3.7/site-packages/sphinx/util/nodes.py", line 450, in make_refnode
'#' + targetid)
File "/devel/v4l/docs/sphinx_2.0/lib/python3.7/site-packages/sphinx/builders/latex/__init__.py", line 159, in get_relative_uri
return self.get_target_uri(to, typ)
File "/devel/v4l/docs/sphinx_2.0/lib/python3.7/site-packages/sphinx/builders/latex/__init__.py", line 152, in get_target_uri
raise NoUri
sphinx.environment.NoUri
This happens because not all references will belong to a single
PDF/LaTeX document.
Better to just ignore those than breaking Sphinx build.
Fixes: d74b0d31dd ("Docs: An initial automarkup extension for sphinx")
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
[jc: Narrowed the "except" and tweaked the comment]
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Rather than fill our text files with :c:func:`function()` syntax, just do
the markup via a hook into the sphinx build process.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Jonathan Corbet