Allow '.. <figure/include/image/...>:: <path>' to appear anywhere within
a line, and find multiple directives on a single line. This is needed to
find files included e.g. within tables.
Implemented by making the <path> part of the regex more specific and
searching for matches anywhere within the contents of the file. Should
be a bit faster too.
Maybe there's some tiny potential for false positives, but this
generates the same file list as the old version for the current docs at
least.
Fixes: #21466
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
UnicodeDecodeError.object supports indexing, but pylint gets confused
for some reason.
doc/scripts/extract_content.py:70:16: E1136: Value 'e.object' is
unsubscriptable (unsubscriptable-object)
This warning is useful enough to want to have enabled in the upcoming
pylint CI check, so suppress it.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The dependencies in the scripts are figures, images, includes or literal
includes. Typically those dependencies are placed relative to the .rst
file that is being copied, but in some instances you might want them to
refer to an absolute path which, in fact, is relative to Sphinx's
top-level source directory. In this case, and as long as the top-level
source directory is provided, do not emit a warning.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Fixes pylint warnings like this one:
doc/conf.py:325:0: W1401: Anomalous backslash in string: '\s'.
String constant might be missing an r prefix.
(anomalous-backslash-in-string)
The reason for this warning is that backslash escapes are interpreted in
non-raw (non-r-prefixed) strings. For example, '\a' and r'\a' are not
the same string (first one has a single ASCII bell character, second one
has two characters).
It just happens that there's no \s (or \., or \/) escape for example,
and '\s' turns into two characters (as needed for a regex). It's risky
to rely on stuff like that regexes though. Best to make them raw strings
unless they're super trivial.
Also note that '\s' and '\\s' turn into the same string.
Another tip: A literal ' can be put into a string with "blah'blah"
instead of 'blah\'blah'.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Normalize the path of ignored folders so that variations in paths
formats on Windows are detected correctly.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
In a clean build, Ninja is creating copies of the extracted
documentation outputs regardless of whether they already exist or
not. I can't explain this behavior from the documentation of the build
command, but it's so, even if extract_content.py is patched to use
copyfile() instead of copy2() so that outputs (copied .rst files in
the build directory) have strictly newer mtimes than inputs (source
tree .rst files).
This extra copying doesn't happen for incremental builds, for some
reason.
To account for this behavior, add and use an extract_content.py option
that skips content extraction completely and leaves that all up to the
add_custom_command() which tracks each individual input->output build
command.
Reported-by: Carles Cufi <carles.cufi@nordicsemi.no>
Signed-off-by: Marti Bolivar <marti@foundries.io>
The logic which copies source documentation files into the build
directory could use some improvements to its dependency management, so
that when a source file changes, extract_content.py gets re-run.
Make these changes as follows:
- Add an --outputs flag to extract_content.py, so that the
sources it depends on and generates can be saved into a file and
thus made known to the build system
- Change the way the sources and destination are specified in the
extract_content.py command line so that the entire job can be done
in a single command, rather than multiple (to avoid having to
collate multiple --outputs files in CMake)
- Extract the content at configure time with execute_process(),
tracking all inputs and outputs within the build system itself. Use
this information to make sure that each individual output depends on
just its exact input file, ensuring updated inputs produce updated
outputs without having to call extract_content.py again.
- Ensure that the "content" build system target depends on all the
outputs, transitively triggering a rebuild any time an input
file (e.g. .rst documentation file or included image/source file)
changes.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Separate the phases of creating the list of outputs (and output
directories) from the actual creation of these outputs. This lays the
groundwork for just printing the inputs and outputs in a subsequent
patch, enabling CMake to track dependencies properly.
In the now-separate output generation phase, use shutil.copy2() to
preserve metadata like access and modification times when possible,
rather than re-reading every file and seeing if the contents are
different. This has the same semantics but is clearer and potentially
more efficient, as the initial copies have more accurate relative
timestamps.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Minor cleanups:
- Allow the user to run extract.content.py -h with ZEPHYR_BASE unset
without crashing
- Replace the use of the ZEPHYR_BUILD environment variable with an
--ignore argument (that can be given multiple times) which specifies
a path to a directory whose files should *not* be copied, and use
it to ignore the build directory
- rename, reorder, and tweak get_files() arguments a bit to go from
source to dest, with extra configuration at the end, getting rid of
local shadowing of global built-in functions
Signed-off-by: Marti Bolivar <marti@foundries.io>
To avoid having to refer to non-rst files artificially in order to get
them copied into the destination folder, make a raw full copy of the
doc/ folder instead of copying only the .rst files.
Fixes#9128
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The following changes have been made to support out-of-tree builds:
* In order to avoid using relative hardcoded paths, use CMake's
configure_file() to replace the paths in the doxygen input file
so that the output directory is set correctly.
* All .rst and additional required files are now copied from the doc/
folder into the build/rst folder using extract_content.py. The
samples/ and boards/ folder are copied twice (once into build/rst
and another into build/doc/rst) to manage relative paths.
* All paths are absolute where possible, including themes and static
content.
This patch ensures that the Zephyr repo is not contaminated by the
build at all.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Improve error reporting to include the filename being processed when a
problem occurs during a pre-generation phase when boards and samples
files are temporarily copied (by doc/scripts/extract_content.py) into
the documentation area for processing.
Two recent problems were noticed:
Some new files were using window-1252 encoding and included windows
printer quote marks and hyphens, for example 0x92 in window-1252
encoding is Unicode 0x2019 for 'RIGHT SINGLE QUOTATION MARK'.
An image file reference by a reST file was missing
Both of these threw an exception reporting the error, but did not
include any information about the file currently being processed, making
it hard to fix the problem (e.g., change the Windows right quote
character to an ASCII ').
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Depending on the version of Python 3 the user has installed, 'open'
may default to using 'ascii' encoding. Since the documentation is in
UTF-8, and contains characters beyond 'ascii', this will result in
failures when using these versions of Python.
Fix this by being explicit about the encoding to be used when reading
these files.
Signed-off-by: David Brown <david.brown@linaro.org>
Change-Id: I54c69334c6bf377f1135cec04f4e0ea96a8e9a5b
Replace the existing Apache 2.0 boilerplate header with an SPDX tag
throughout the zephyr code tree. This patch was generated via a
script run over the master branch.
Also updated doc/porting/application.rst that had a dependency on
line numbers in a literal include.
Manually updated subsys/logging/sys_log.c that had a malformed
header in the original file. Also cleanup several cases that already
had a SPDX tag and we either got a duplicate or missed updating.
Jira: ZEP-1457
Change-Id: I6131a1d4ee0e58f5b938300c2d2fc77d2e69572c
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Running documentation scripts on the top directory shifted all links one
level dowwn and is breaking all incoming links.
Use a script to copy all RST files into the doc/ directory before
running sphinx and keep structure intact.
Jira: ZEP-1579
Change-Id: Iccff068430e2ddb29e172cd8ae920475815d199e
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
