Commit graph

73 commits

Author SHA1 Message Date
Fabio Utzig 3f28f69477 doc: add --no-index-modules option to genrest.py
This add a new option `--no-index-modules` which works similarly to
`--modules` but does not generated index pages, only retains the
tweaking of how paths are displayed on symbol information pages,
showing '<title>/path/within/module/Kconfig' for paths that fall
within modules.

This is required by NCS, where there are more "modules" which we don't
want to have indexes for.

Signed-off-by: Fabio Utzig <fabio.utzig@nordicsemi.no>
2020-06-19 18:14:18 +02:00
Ulf Magnusson 178d3208da doc: genrest.py: Convert to use f-strings
Use f-strings instead of .format() to make the code easier to read,
especially for long multiline strings.

f-strings were added in Python 3.6, which Zephyr requires now.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-28 17:26:11 -05:00
Ulf Magnusson e181e1b773 kconfiglib: Update to hide tracebacks for expected errors
Update Kconfiglib to upstream revision 9c0b562c94 to get this commit in:

    Add Kconfig.__init__() helper flag for suppressing tracebacks

    Tools that don't use standard_kconfig() currently generate spammy
    tracebacks for e.g. syntax errors.

    Add a suppress_traceback flag to Kconfig.__init__() for catching
    "expected" exceptions and printing them to stderr and exiting with
    status 1. Use it to make all tools consistently hide tracebacks.

Use the new flag to hide tracebacks for expected exceptions in
kconfig.py, lint.py, and genrest.py.

Some menuconfig robustness tweaks for wonky terminals are included as
well, and a new feature for customizing .config and autoconf.h header
comments via environment variables.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-12-20 19:48:09 -05:00
Ulf Magnusson ea3041dbbe doc: extract_content.py: Detect included files in a more robust way
Allow '.. <figure/include/image/...>:: <path>' to appear anywhere within
a line, and find multiple directives on a single line. This is needed to
find files included e.g. within tables.

Implemented by making the <path> part of the regex more specific and
searching for matches anywhere within the contents of the file. Should
be a bit faster too.

Maybe there's some tiny potential for false positives, but this
generates the same file list as the old version for the current docs at
least.

Fixes: #21466

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-12-20 13:02:00 -05:00
Ulf Magnusson 5c28f39e7f doc: genrest: Use a separate index page for all symbols
Add an option --separate-all-index that makes genrest.py generate a
separate index-all.rst index page that lists all symbols, instead of
listing all symbols in index.rst. index-all.rst is linked from
index.rst.

This was originally motivated by an external project where index.rst
becomes the top-level page, which runs into a Sphinx bottleneck with
sphinx_rtd_theme. See https://github.com/sphinx-doc/sphinx/issues/6909.

This turned out pretty nice after some feedback from Ruth Fuchss, so use
it for Zephyr too.

Also unclutter the generated documentation a bit by removing some
headings. This makes the navigation menu on the left nicer too.

Piggyback making genrest.py executable, which is handy when running it
manually.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-12-11 12:32:30 -06:00
Ulf Magnusson f570f19bef doc: genrest: Use , instead of : as the separator in --modules
Using ':' as the separator in --modules breaks module specifications
like

    nrfxlib:nrfxlib:C:/Users/Carles/src/ncs/nrfxlib

The ':' in 'C:' gets seen as a separator.

Use ',' instead, which is unlikely to appear in paths (at least in
--modules). Treat a doubled ',,' as a literal ','.

Another option would be ';', but it clashes with how CMake represents
lists, which is awkward.

Also make quoting of arguments with spaces more robust by passing
VERBATIM to add_custom_target().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-12-05 16:23:10 +01:00
Ulf Magnusson 58e2057aaf genrest: Show symbol help texts on index page
Instead of showing the prompt for symbols on the index page, show their
help texts. This makes searching easier.

Fall back on the prompt if no help text is available.

Also change the code to only show one of the prompts if several are
available (happens if a symbol is defined in multiple locations and adds
a prompt in more than one of them). It's probably overkill to show them
all, and it doesn't come up that often.

Suggested by David B. Kinder.

Co-authored-by: David B. Kinder <david.b.kinder@intel.com>
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-12-03 17:02:22 +01:00
Ulf Magnusson 5492f2f8b6 genrest: List all symbols on main index page and refactor
Instead of having index.rst just link to other index pages in --modules
mode, list all symbols on it, and have links to the module index pages
at the top.

Get rid of index-all.rst and index-main.rst (and all related options).
index-all.rst is no longer needed, and index-main.rst (symbols outside
--modules) might not be that useful to people reading the documentation
(and could be added back if needed).

Also refactor and streamline the code a bunch. For the main index page,
the only difference between modules and non-modules mode is now whether
there's links to other index pages at the top.

Suggested by David B. Kinder.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-12-03 17:02:22 +01:00
Ulf Magnusson 711c4d8a48 doc: genrest: Support customizing the description on index pages
Extend the specifications passed to --modules to add the form

    <title>:<suffix>:<path>:<index description filename>

<index description filename> points to a file that contains RST that is
inserted at the top of the index page.

If no filename is passed, the old default description is used.

Also add three flags --top-index-desc, --non-module-index-desc, and
--all-index-desc for customizing the text at the top of non-module index
pages.

This functionality is currently unused in Zephyr, but will probably be
used later. It's being added for a downstream project.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-10-16 11:57:36 +02:00
Ulf Magnusson 91384ef01e scripts: genrest: Support splitting symbol documentation up by path
Add a --modules flag to genrest.py for generating separate index pages
for symbols defined within certain paths.

Passing

    --modules Shell🐚subsys/shell Storage:storage:subsys/storage

will generate these index pages, instead of a single index.rst file:

 - index-shell.rst: Lists the symbols defined in subsys/shell

 - index-storage.rst: Lists the symbols defined in subsys/storage

 - index-main.rst: Lists all symbols that are not in subsys/shell or
   subsys/storage

 - index-all.rst: Lists all symbols

 - index.rst: Contains links to the other index pages

The string before the first ':' ('Shell' and 'Storage' above) is used
when generating the title of the index page. 'Shell' gives
'Shell Configuration Options', for example.

The title for index-main.rst can be set by passing
'--non-module-title <title>'. It defaults to "Zephyr".

By default, paths in symbol information pages that are within modules
are shown as '<title>/<path relative to module>'. This can be disabled
by passing --keep-module-paths.

--keep-module-paths would make sense for the example above, but
stripping the path to the module is nice when dealing with modules
defined outside the Zephyr repository.

If a symbol is defined in multiple modules (or both in a module and
outside all modules), it will appear on multiple index pages.

This commit also simplifies how genrest.py is called a bit, making the
Kconfig filename optional (default: Kconfig).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-10-10 15:36:11 +02:00
David B. Kinder 4207e705f0 doc: fix genrest turbo mode output
PR #19493 cleaned up the turbo-mode process for doc generation but
introduced an error in the generated index.rst that causes use of the
"make htmldocs-fast" (so-called turbo mode) to fail with an error -
the generated list-table directive has no content (other than
the header row).  This PR adds one dummy row.

Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
2019-10-02 22:19:30 -04:00
Ulf Magnusson c15705f180 doc: genrest: Separate turbo mode logic from rest of code
KCONFIG_TURBO_MODE being twisted up in the normal logic made the code
hard to follow and change.

Use a separate write_dummy_index() function for KCONFIG_TURBO_MODE
instead, and add more documentation for it. Also get rid of options.rst
and just write all the symbol link targets directly to the dummy index
file, which is a bit simpler.

As a small piggybacked improvement for default values, make the heading
'default' instead of 'defaults' when there's only one. The menuconfigs
do this too.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-10-01 19:21:02 +02:00
Ulf Magnusson ca4200a2c7 doc: extract_content.py: Suppress spurious pylint warning
UnicodeDecodeError.object supports indexing, but pylint gets confused
for some reason.

    doc/scripts/extract_content.py:70:16: E1136: Value 'e.object' is
    unsubscriptable (unsubscriptable-object)

This warning is useful enough to want to have enabled in the upcoming
pylint CI check, so suppress it.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-09-07 08:13:22 -04:00
Ulf Magnusson a570b40252 genrest: De-spam docs by skipping direct deps. in more places
Similar deal to commit cc14c40a2d ("kconfiglib: Unclutter symbol
strings, avoid redundant writes, misc.").

Hide the direct dependencies in the defaults, selects, and implies
sections. Do the same in menuconfig/guiconfig as well.

This uses a new Kconfiglib API, so update Kconfiglib to upstream
revision 164ef007a8. This also includes some minor optimizations and
cleanups.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-06-24 12:14:06 -07:00
Ulf Magnusson e36e544cab genrest: Mention that dependencies are only propagated to 'depends on'
A recent Kconfiglib change turns

    config A
            bool
            prompt "foo" if C && D
            default A if B && C && D
            depends on C && D

into

    config A
            bool "foo"
            default A if B
            depends on C && D

Where e.g. D might be from a surrounding menu/if.

Update the note at the end of symbol pages to clarify that only the
'depends on' includes propagated dependencies.

Piggyback some minor cleanup.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-06-07 18:20:22 +02:00
Ulf Magnusson 6cfc13522c menuconfig: Small Space/Enter improvement + fix for obscure bug
Update menuconfig (and Kconfiglib, just to sync) to upstream revision
7e0b9f7ae1, to get these commits in:

  menuconfig: Improve space/enter behavior slightly

  Space toggles value is possible, and enters menus otherwise. Enter
  works the other way around.

  Make this explicit in the code, which also fixes some corner cases,
  like space doing nothing on a y-selected menuconfig symbol.

  -------------------------------------------------------------------

  menuconfig: Fix display issue for unsatisfied-deps selected symbol
  with children

  A symbol with unsatisfied direct dependencies can end up with visible
  children in an implicit submenu if it is selected (though that
  generates a warning), so the optimization in _shown_nodes() isn't
  safe, and causes the child nodes to not be shown outside show-all
  mode.

  Just remove the optimization. Trying things out some more,
  everything's plenty fast enough anyway.

  Checking the direct dependencies of the parent instead would be safe.

The menu path now says "(Top)" instead of "(top menu)" too, which is a
bit more compact. Make the same change in genrest.py.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-05-30 09:54:06 -04:00
Carles Cufi d9f53455e5 doc: extract_content: Allow for absolute paths in deps
The dependencies in the scripts are figures, images, includes or literal
includes. Typically those dependencies are placed relative to the .rst
file that is being copied, but in some instances you might want them to
refer to an absolute path which, in fact, is relative to Sphinx's
top-level source directory. In this case, and as long as the top-level
source directory is provided, do not emit a warning.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-04-18 17:52:49 -04:00
Ulf Magnusson 60c395024f genrest: Include all parents in menu paths
Previously, symbols not defined with 'menuconfig' with children weren't
listed in the children's menu paths. It was deliberate, but it's
probably an anti-feature in retrospect, because it can make it harder to
find stuff by following the menu path.

Don't try to be clever and just list all the parent nodes in the menu
path.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-04-18 08:47:15 -04:00
Ulf Magnusson a449c98db2 scripts: Fix risky uses of non-raw regex strings in Python scripts
Fixes pylint warnings like this one:

    doc/conf.py:325:0: W1401: Anomalous backslash in string: '\s'.
    String constant might be missing an r prefix.
    (anomalous-backslash-in-string)

The reason for this warning is that backslash escapes are interpreted in
non-raw (non-r-prefixed) strings. For example, '\a' and r'\a' are not
the same string (first one has a single ASCII bell character, second one
has two characters).

It just happens that there's no \s (or \., or \/) escape for example,
and '\s' turns into two characters (as needed for a regex). It's risky
to rely on stuff like that regexes though. Best to make them raw strings
unless they're super trivial.

Also note that '\s' and '\\s' turn into the same string.

Another tip: A literal ' can be put into a string with "blah'blah"
instead of 'blah\'blah'.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-28 14:41:32 -05:00
Ulf Magnusson 0aeba398fb doc: fix_tex: Remove redundant 'return'
Making a clean slate for some pylint CI tests.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-26 07:47:20 -05:00
Marc Herbert 5284231750 doc: fix incremental build by fixing doxygen output mtime
Doxygen doesn't support incremental builds.  It could be ok because it
doesn't take much time in this codebase.  However it's not because it
makes old output look new which has a cascade effect on sphinx:
https://sourceforge.net/p/doxygen/mailman/message/36580807/

Make doxygen artificially smarter by saving and restoring modify
timestamps on the filesystem for doxygen output files that haven't
changed.

On my system this brings down the time to run an incremental "make
htmldocs" from 75s down to 8s (cmake -DKCONFIG_TURBO_MODE=1 used in both
cases)

This optimization speeds-up non-doxygen documentation work only.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-02-13 06:58:52 -06:00
Anas Nashif e78ccdf4b8 doc: rename kconfig section
kconfig entries are options, symbols is probably an internal name to
kconfig.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-30 18:38:48 -05:00
Anas Nashif 940a931b08 doc: turbo mode for kconfig options
Building the documentation for all the Kconfig options significantly
adds to the total doc build time.  When making and testing major changes
to the documentation, we provide an option to temporarily stub-out
the auto-generated configuration documentation so the doc build process
runs much faster.

To enable this mode, set the following option when invoking cmake

    -DKCONFIG_TURBO_MODE=1

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-23 16:26:19 -05:00
Carles Cufi 24da65276a doc: extract_content: Normalize path of ignored folders
Normalize the path of ignored folders so that variations in paths
formats on Windows are detected correctly.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2018-10-23 10:28:27 +01:00
Marti Bolivar d7dc113783 doc: avoid copying content twice
In a clean build, Ninja is creating copies of the extracted
documentation outputs regardless of whether they already exist or
not. I can't explain this behavior from the documentation of the build
command, but it's so, even if extract_content.py is patched to use
copyfile() instead of copy2() so that outputs (copied .rst files in
the build directory) have strictly newer mtimes than inputs (source
tree .rst files).

This extra copying doesn't happen for incremental builds, for some
reason.

To account for this behavior, add and use an extract_content.py option
that skips content extraction completely and leaves that all up to the
add_custom_command() which tracks each individual input->output build
command.

Reported-by: Carles Cufi <carles.cufi@nordicsemi.no>
Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-10-16 15:10:18 -04:00
Marti Bolivar 0913c296cf doc: fix extract_content.py dependency tracking
The logic which copies source documentation files into the build
directory could use some improvements to its dependency management, so
that when a source file changes, extract_content.py gets re-run.

Make these changes as follows:

- Add an --outputs flag to extract_content.py, so that the
  sources it depends on and generates can be saved into a file and
  thus made known to the build system

- Change the way the sources and destination are specified in the
  extract_content.py command line so that the entire job can be done
  in a single command, rather than multiple (to avoid having to
  collate multiple --outputs files in CMake)

- Extract the content at configure time with execute_process(),
  tracking all inputs and outputs within the build system itself. Use
  this information to make sure that each individual output depends on
  just its exact input file, ensuring updated inputs produce updated
  outputs without having to call extract_content.py again.

- Ensure that the "content" build system target depends on all the
  outputs, transitively triggering a rebuild any time an input
  file (e.g. .rst documentation file or included image/source file)
  changes.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-10-12 11:37:48 +02:00
Marti Bolivar 9802f883f5 doc: rewrite extract_content.py for better input/output tracking
Separate the phases of creating the list of outputs (and output
directories) from the actual creation of these outputs. This lays the
groundwork for just printing the inputs and outputs in a subsequent
patch, enabling CMake to track dependencies properly.

In the now-separate output generation phase, use shutil.copy2() to
preserve metadata like access and modification times when possible,
rather than re-reading every file and seeing if the contents are
different. This has the same semantics but is clearer and potentially
more efficient, as the initial copies have more accurate relative
timestamps.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-10-12 11:37:48 +02:00
Marti Bolivar 0d4ca68051 doc: clean up extract_content.py use of environment variables
Minor cleanups:

- Allow the user to run extract.content.py -h with ZEPHYR_BASE unset
  without crashing
- Replace the use of the ZEPHYR_BUILD environment variable with an
  --ignore argument (that can be given multiple times) which specifies
  a path to a directory whose files should *not* be copied, and use
  it to ignore the build directory
- rename, reorder, and tweak get_files() arguments a bit to go from
  source to dest, with extra configuration at the end, getting rid of
  local shadowing of global built-in functions

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-10-12 11:37:48 +02:00
Marti Bolivar 5336f30def doc: make extract_content.py flake8 clean
Address flake8's complaints.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-10-12 11:37:48 +02:00
Ulf Magnusson 5cfe06b6bc genrest/menuconfig: Fix crash for promptless choices
Update menuconfig (and Kconfiglib, just to sync) to upstream revision
256e5b3e38e92 to get the fix below in, for an issue in an external
project. Also update genrest.py with a similar fix (the genrest issue
was what originally prompted it).

    menuconfig: Improve/fix promptless choice handling

    The code assumed that all parent (interface) menus always have a
    prompt, which is false for items in promptless choices. This led to
    a crash e.g. when viewing the symbol information for a symbol within
    a promptless choice.

    Promptless choices with children can show up "legitimately" when
    people define choices in multiple locations to add symbols, though
    this is broken in the C tools.

    Use standard_sc_expr_str(node.item) instead of the non-existing
    prompt for promptless choices. That way they show up as
    '<choice (name if any>)>', which is consistent with how they're
    shown elsewhere.

    This commit also changes how choice names are displayed in
    show-name/show-all mode, to the standard_sc_expr_str() format.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-10-10 16:16:28 +02:00
Daniel Leung 9945e7fda0 doc: add ability to generate PDF
This adds new targets to generate build documentation through
LaTEX to PDF.

There are a few notes:

1. pdflatex complains about the tex file generated by doxygen
   so it needs to be fixed with a Python script before feeding
   in through pdflatex.

2. SVG files are not recognized by pdflatex so they are converted
   to known good format on the fly, only for producing PDF. This
   uses the libRSVG's rsvg-convert tool.

Relates to #6782.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2018-08-29 18:16:56 -04:00
Ulf Magnusson c1f54cc5fd Kconfig: Show include paths in menuconfig and documentation
Update Kconfiglib and menuconfig to upstream revision a28bc4da9762e,
which adds include path information to menuconfig, showing how the
Kconfig file of the symbol got 'source'd in addition to showing the
definition location.

Add include path information to the auto-generated Kconfig documentation
too.

Some small Kconfiglib bugs are fixed to, like error reporting for
recursive 'source's being broken (crashing instead of printing the
error), a minor file descriptor leak, and syntax checking not catching
extra trailing tokens after 'if <expr>' and 'depends on <expr>'.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-26 09:47:20 -07:00
Ulf Magnusson f2acdffea3 genrest: List symbols selected by each symbol
Previously, you had to go to the Kconfig dump at the bottom of the
symbol reference page to see selected symbols (as opposed to the
select*ing* symbols, which were already listed). Might be easy to miss.

Use a format similar to the list of defaults
('- SELECTED_SYM if CONDITION').

Also list 'imply's, though I don't think those are used anywhere in
Zephyr yet.

Simplify the code generating the list of selecting symbols a bit as
well.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-19 11:19:24 -07:00
Ulf Magnusson bf8efdbf5b genrest: Generate RST links in a less hacky way
Use the new Kconfiglib expression printing customization functionality
to get rid of the hacky expr_str() overriding.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-13 19:31:46 -07:00
Carles Cufi a0dbc7aa4b doc: Copy all doc/ contents and not just .rst files
To avoid having to refer to non-rst files artificially in order to get
them copied into the destination folder, make a raw full copy of the
doc/ folder instead of copying only the .rst files.

Fixes #9128

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2018-08-06 21:56:57 +02:00
Carles Cufi e182dbc22e doc: cmake: Enable out-of-tree builds
The following changes have been made to support out-of-tree builds:

* In order to avoid using relative hardcoded paths, use CMake's
  configure_file() to replace the paths in the doxygen input file
  so that the output directory is set correctly.
* All .rst and additional required files are now copied from the doc/
  folder into the build/rst folder using extract_content.py. The
  samples/ and boards/ folder are copied twice (once into build/rst
  and another into build/doc/rst) to manage relative paths.
* All paths are absolute where possible, including themes and static
  content.

This patch ensures that the Zephyr repo is not contaminated by the
build at all.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2018-07-19 08:44:41 -04:00
Carles Cufi 60a9e1a7c7 doc: Remove unused filter-doc-log.sh
The filter-doc-log.sh script is no longer used since its functionality
has been moved to the CMakeLists.txt and filter-known-issues.py.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2018-07-12 01:26:05 +02:00
Ulf Magnusson 35ec18ac08 genrest: Mention implicit default values
It's common for people to put in a 'default n' "just in case", because
it might not be obvious that bool symbols implicitly default to 'n'. To
make it clearer, mention the implicit default value on the reference
pages of symbols without defaults.

Also mention that choices without defaults default to the first
(visible) symbol in the choice.

Note: Adding to the confusion, Kconfig used to generate a
'# CONFIG_FOO is not set' line for 'default n', but no output when there
was no default. I changed that recently in both the C tools and
Kconfiglib. There's no output either case now (unless the symbol is
visible).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-06-26 14:40:04 +02:00
David B. Kinder 934a4d2b13 doc: fix genrest.py to use utf-8 encoding
The genrest.py script (used to create the Kconfig configuration option
documentation) was depending on the environment's locale settings to
properly handle UTF-8 character encoding in the reST files it creates.
When we reused this script for another project, that environment's
LC_CTYPE was set to C, telling Python to use ASCII as the (default)
encoding.  This patch is a prophylactic measure to prevent potential
failures by adding the encoding option on the open() calls.

It also includes a small tweak to the introduction.

Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
2018-06-19 20:16:59 -04:00
David B. Kinder 3bdb52f1ad doc: improved error reporting for non-UTF8 docs
Improve error reporting to include the filename being processed when a
problem occurs during a pre-generation phase when boards and samples
files are temporarily copied (by doc/scripts/extract_content.py) into
the documentation area for processing.

Two recent problems were noticed:

  Some new files were using window-1252 encoding and included windows
  printer quote marks and hyphens, for example 0x92 in window-1252
  encoding is Unicode 0x2019 for 'RIGHT SINGLE QUOTATION MARK'.

  An image file reference by a reST file was missing

Both of these threw an exception reporting the error, but did not
include any information about the file currently being processed, making
it hard to fix the problem (e.g., change the Windows right quote
character to an ASCII ').

Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
2018-06-18 19:35:21 -04:00
Ulf Magnusson 31ab6bffb2 genrest: Generate documentation and links for choices
Until now, choices have kinda been a black box in the Kconfig reference,
and hid the dependencies of choice symbols (because choice symbols
depend directly on the choice rather than on whatever the choice
'depends on').

Generate separate information pages for choices and turn <choice>
dependencies into links.

One complication is that choices (usually) don't have names. Use the
index of each choice in the Kconfig files (first choice seen = choice 0,
then choice 1, etc.) instead to identify each choice.

Choice reference pages include the same information as symbol reference
pages (minus some things that don't apply for choices), and also list
the choice symbols contained in the choice.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-06-11 14:40:48 -04:00
Ulf Magnusson 972386d7a3 genrest: Highlight Kconfig as Kconfig, not Python
Sphinx defaults to Python highlighting, but a Kconfig lexer is available
as well
(http://pygments.org/docs/lexers/#pygments.lexers.configs.KconfigLexer).
Use it.

This only highlights Kconfig definitions without links (references to
other symbols), as Sphinx doesn't highlight '.. parsed-literal::' blocks
with links. It's an improvement over Python stuff getting highlighted at
least.

Side note: '.. highlight:: none' still gives '.. parsed-literal::'
blocks without links a different background color, for whatever reason.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-06-01 09:07:05 -04:00
Ulf Magnusson 06ecb287fc genrest: Move from doc/scripts/genrest/ to doc/scripts/
Having a single directory with just genrest.py feels a bit pointless.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-05-22 13:31:56 -04:00
Ulf Magnusson b45e152ff0 menuconfig/genrest: Exclude implicit submenus from menu paths
Make all menu paths ("(top menu) -> menu -> submenu -> ...") exclude
implicit submenus, which are shown indented in the menuconfig interface
and are created when items depend on the symbol before them.

Previously, implicit submenus were excluded in the menu path at the top
of the menuconfig interface, but were included in the menuconfig symbol
information dialog and in the docs generated by genrest.py.

This makes it consistent, and un-spams the documentation for some
symbols a bit.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-05-22 13:31:56 -04:00
Ulf Magnusson aa26289458 kconfig: Get rid of leading/trailing whitespace in prompts
Leading/trailing whitespace in prompts requires ugly workarounds in
genrest.py, as e.g. *prompt * is invalid RST. strip() all prompts in
Kconfiglib and get rid of the genrest.py workarounds. Add a warning too.

The Kconfiglib update has some unrelated cleanups and fixes (that won't
affect Zephyr).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-05-19 09:26:39 +03:00
Ulf Magnusson 09c813793b doc: genrest: Speed up documentation rebuilding
Skip writing index and symbol RST files whose content hasn't changed, to
avoid updating their timestamps. This makes documentation rebuilds much
faster, as Sphinx looks at the timestamp to determine if an RST file has
been updated.

Rebuilding docs with symbol reference up-to-date, before:

    $ time make html
    real    4m52.838s
    user    4m46.242s
    sys     0m4.249s

After:

    $ time make html
    real    0m48.731s
    user    0m47.571s
    sys     0m0.908s

Testing was done with 'make VERBOSE=1 SPHINXOPTS= html', suggested by
Marti Bolivar.

Piggyback a small cleanup to the code generating the select/imply
information.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-05-18 14:09:20 +03:00
Ulf Magnusson aa9fe26166 doc: genrest: Show properties on the correct symbol definition
Previously, symbols defined in multiple locations (e.g.,
Kconfig.defconfig symbols) would incorrectly get all their defaults,
selects, implies, and ranges displayed on the first definition.

This commit uses the Kconfiglib functionality added in 53af8db920066
("kconfiglib: Record which MenuNode has each property") to show such
symbols with all properties in the right location.

The definitions are now listed one-by-one, with the filename/linenr
displayed before each definition. The menu path (e.g.,
'(top menu) -> Device Drivers') is now listed for each definition as
well.

Fixes: #6879

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-05-17 23:55:07 +03:00
Ulf Magnusson 3a72cc9889 doc: genrest: Simplify select logic with split_expr()
The logic for walking reverse dependencies (from 'select's and 'imply's)
is a bit tricky to read (and write), and was essentially duplicated in a
few different spots (including in the upcoming menuconfig
implementation).

Use a new split_expr() helper function that's been added to Kconfiglib
to reduce code duplication and make the code more readable.

This also has another nice side effect: The list of symbols that select
a particular symbol now has the symbols in the order that they appear in
the Kconfig files.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-04-17 08:02:40 -07:00
Ulf Magnusson 4703ba7d31 doc: genrest: Use Kconfiglib 2 and improve output
The new version of the reference page for a symbol lists the following
essential and generally useful information at the top of the page:

 - Prompt

 - Type

 - Help text

 - Direct dependencies (from 'depends on', and inherited from if's and
   menus)

 - Defaults

 - Other symbols selecting or implying the symbol

The current symbol value and visibility are no longer displayed. They
don't make much sense in a reference document, as they depend on user
values from configuration files.

The bottom of the reference page also includes the symbol represented in
Kconfig format (including propagated dependencies).

All references to defined symbols appearing in expressions are turned
into links, which can be clicked on to jump to the reference page for
that symbol.

The following (slightly outdated) screenshot gives an idea of the
format:
https://www.dropbox.com/s/a34tlk2ncyus8po/promptandtype.png?dl=0

This commit also removes Kconfiglib 1, since it is no longer used.

Fixes #5622
Fixes #6821

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-29 14:25:24 -04:00
Ulf Magnusson 6188579063 filter-doc-log.sh: Fix error for empty log file
Running 'make html' in doc/ when doc.log is missing or empty gives the
following error if sphinx-build doesn't write anything on stdout/stderr:

    Error in ./scripts/filter-doc-log.sh: logfile "doc.log" not found.
    Makefile:84: recipe for target ”html” failed
    make: *** [html] Error 1

The problem is that scripts/filter-doc-log.sh tests for the existence of
the log file with [ -s ${LOG_FILE} ], which requires it to be nonempty.

Fix it by using -e instead, which only checks if the log file exists.
scripts/filter-known-issues.py ($KI_SCRIPT) seems to be able to deal
with empty files (and runs quickly).

Fixes #6854

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-29 08:26:47 -04:00