Advanced usage

This page contains more advanced topics in case you want to understand how to use Sphinx-Gallery more deeply.

Understanding warning and error outputs

Any warnings or errors that occur when executing code blocks in the gallery Python files will be printed in pink during building of the documentation. The .py file path and the line number that the error occurred in will also be printed. For example, the example Example that fails to execute will raise the following error:

File "<full_path>/examples/no_output/plot_raise.py", line 27, in <module>
    iae
NameError: name 'iae' is not defined

Problems in the text (reST) blocks of the gallery Python files will result in warnings or errors when Sphinx is converting the generated .rst files to HTML. These will be printed by Sphinx in pink, after code block errors, during building of the documentation. In this case, the .rst file path and .rst file line number will be printed. To fix the problem, you will need to amend the original .py file, not the generated .rst file. To figure out where the problem is, you will need to match the content of the .rst file at the line number printed to the original .py file.

Example .rst warning:

<full_path>/auto_examples/plot_example.rst:19: WARNING: Explicit markup ends without a blank line; unexpected unindent.

The warning above occurred due to line 19 in plot_example.rst. The original plot_example.py file will need to be amended to fix it. Sphinx-Gallery only (re)builds new, modified or failed examples, so re-running the documentation build should rebuild just the modified example, allowing for quick iteration.

Write a custom image scraper

Warning

The API for custom scrapers is currently experimental.

By default, Sphinx-Gallery supports image scraping for Matplotlib (matplotlib_scraper()). If you wish to capture output from other python packages, first determine if the object you wish to capture has a _repr_html_ method. If so, you can use the configuration capture_repr (Controlling what output is captured) to control the display of the object, without the need to write a custom scraper. This configuration allows capture of the raw html output, in a process similar to other html-based displays such as jupyter. If the first option does not work, this section describes how to write a custom scraper.

Image scrapers are functions (or callable class instances) that do the following things:

  1. Collect a list of images created in the latest execution of code.

  2. Write these images to disk in PNG, JPEG, SVG, GIF, or WebP format (with .png, .jpg, .svg, .gif, or .webp extensions, respectively)

  3. Return reST that embeds these figures in the built documentation.

The function should take the following inputs (in this order):

  1. block - a Sphinx-Gallery .py file is separated into consecutive lines of ‘code’ and reST ‘text’, called ‘blocks’. For each block, a tuple containing the (label, content, line_number) (e.g. ('code', 'print("Hello world")', 5)) of the block is created.

    • ‘label’ is a string that can either be 'text' or 'code'. In this context, it should only be 'code' as this function is only called for code blocks.

    • ‘content’ is a string containing the actual content of the code block.

    • ‘line_number’ is an integer, indicating the line number that the block starts at.

  2. block_vars - dictionary of configuration and runtime variables. Of interest for image scrapers is the element 'image_path_iterator' which is an iterable that returns an absolute path to an image file name adhering to Sphinx-Gallery naming convention. The path directs to the gallery_dirs/images directory (Configure and use Sphinx-Gallery) and the image file name is 'sphx_glr_' followed by the name of the source .py file then a number, which starts at 1 and increases by 1 at each iteration. The default file format is .'png'. For example: 'home/user/Documents/module/auto_examples/images/sphx_glr_plot_mymodule_001.png'. If a different image extension is desired, the scraper is responsible for replacing the default .png extension. Only supported image extensions (see above) will enable the file to be picked up by Sphinx-Gallery.

  3. gallery_conf - dictionary containing the configuration of Sphinx-Gallery, set under sphinx_gallery_conf in doc/conf.py (Configuration). Of note, the image_srcset configuration will provide user specified image resolutions (as floats) and can be used by your custom scraper to enable multi-resolution images.

It should return a string containing the reST for embedding this figure in the documentation. See matplotlib_scraper() for an example of a scraper function (click on ‘source’ below the function name to see the source code). The matplotlib_scraper() uses the helper function sphinx_gallery.scrapers.figure_rst() to help generate reST (see below).

This function will be called once for each code block of your examples. Sphinx-Gallery will take care of scaling images for the gallery index page thumbnails. PNG, JPEG and WebP images are scaled using Pillow, and SVG and GIF images are copied.

Warning

SVG images do not work with latex build modes, thus will not work while building a PDF version of your documentation. You may want to consider sphinxcontrib-svg2pdfconverter.

Example 1: a Matplotlib-style scraper

For example, we will show sample code for a scraper for a hypothetical package. It uses an approach similar to what sphinx_gallery.scrapers.matplotlib_scraper() does under the hood, which use the helper function sphinx_gallery.scrapers.figure_rst() to create the standardized reST. If your package will be used to write an image file to disk (e.g., PNG or JPEG), we recommend you use a similar approach:

def my_module_scraper(block, block_vars, gallery_conf):
    import mymodule
    # We use a list to collect references to image names
    image_names = list()
    # The `image_path_iterator` is created by Sphinx-Gallery, it will yield
    # a path to a file name that adheres to Sphinx-Gallery naming convention.
    image_path_iterator = block_vars['image_path_iterator']

    # Define a list of our already-created figure objects.
    list_of_my_figures = mymodule.get_figures()

    # Iterate through figure objects, save to disk, and keep track of paths.
    for fig, image_path in zip(list_of_my_figures, image_path_iterator):
        fig.save_png(image_path)
        image_names.append(image_path)

    # Close all references to figures so they aren't used later.
    mymodule.close('all')

    # Use the `figure_rst` helper function to generate the reST for this
    # code block's figures. Alternatively you can define your own reST.
    return figure_rst(image_names, gallery_conf['src_dir'])

This code could be defined either in your conf.py file, or as a module that you import into your conf.py file (see Importing callables). The configuration needed to use this scraper would look like:

sphinx_gallery_conf = {
    ...
    'image_scrapers': ('matplotlib', "my_module._scraper.my_module_scraper"),
}

Where Sphinx-Gallery will parse the string "my_module._scraper.my_module_scraper" to import the callable function.

Example 2: detecting image files on disk

Here’s another example that assumes that images have already been written to disk. In this case we won’t generate any image files, we’ll only generate the reST needed to embed them in the documentation. Note that the example scripts will still need to be executed to scrape the files, but the images don’t need to be produced during the execution.

We assume the function is defined within your package in a module called _scraper. Here is the scraper code:

from glob import glob
import shutil
import os
from sphinx_gallery.scrapers import figure_rst

def png_scraper(block, block_vars, gallery_conf):
    # Find all PNG files in the directory of this example.
    path_current_example = os.path.dirname(block_vars['src_file'])
    pngs = sorted(glob(os.path.join(path_current_example, '*.png')))

    # Iterate through PNGs, copy them to the Sphinx-Gallery output directory
    image_names = list()
    image_path_iterator = block_vars['image_path_iterator']
    seen = set()
    for png in pngs:
        if png not in seen:
            seen |= set(png)
            this_image_path = image_path_iterator.next()
            image_names.append(this_image_path)
            shutil.move(png, this_image_path)
    # Use the `figure_rst` helper function to generate reST for image files
    return figure_rst(image_names, gallery_conf['src_dir'])

Then, in our conf.py file, we include the following code:

sphinx_gallery_conf = {
    ...
    'image_scrapers': ('matplotlib', 'my_module._scraper.png_scraper'),
}

Example 3: matplotlib with SVG format

The sphinx_gallery.scrapers.matplotlib_scraper() supports **kwargs to pass to matplotlib.figure.Figure.savefig(), one of which is the format argument. See Write a custom image scraper for supported formats. To use SVG you can define the following function and ensure it is importable:

def matplotlib_svg_scraper(*args, **kwargs):
    return matplotlib_scraper(*args, format='svg', **kwargs)

Then in your conf.py:

sphinx_gallery_conf = {
    ...
    'image_scrapers': ("sphinxext.matplotlib_svg_scraper",),
    ...
}

You can also use different formats on a per-image basis, but this requires writing a customized scraper class or function.

Example 4: Mayavi scraper

Historically, Sphinx-Gallery supported scraping Mayavi figures as well as matplotlib figures. However, due to the complexity of maintaining the scraper, support was deprecated in version 0.12.0. To continue using a Mayavi scraping, consider using something like the following:

from sphinx_gallery.scrapers import figure_rst

def mayavi_scraper(self, block, block_vars, gallery_conf):
    from mayavi import mlab
    image_path_iterator = block_vars['image_path_iterator']
    image_paths = list()
    e = mlab.get_engine()
    for scene, image_path in zip(e.scenes, image_path_iterator):
        try:
            mlab.savefig(image_path, figure=scene)
        except Exception:
            mlab.close(all=True)
            raise
        # make sure the image is not too large
        scale_image(image_path, image_path, 850, 999)
        if 'images' in gallery_conf['compress_images']:
            optipng(image_path, gallery_conf['compress_images_args'])
        image_paths.append(image_path)
    mlab.close(all=True)
    return figure_rst(image_paths, gallery_conf['src_dir'])

Resetting before each example

Sphinx-Gallery supports ‘resetting’ function(s) that are run before and/or after each example script is executed. This is used natively in Sphinx-Gallery to ‘reset’ the behavior of the visualization packages matplotlib and seaborn (Resetting modules). However, this functionality could be used to reset other libraries, modify the resetting behavior for a natively-reset library or run an arbitrary custom function at the start and/or end of each script.

This is done by adding a custom function to the resetting tuple defined in conf.py. The function should take two variables: a dictionary called gallery_conf (which is your Sphinx-Gallery configuration) and a string called fname (which is the file name of the currently-executed Python script). These generally don’t need to be used in order to perform whatever resetting behavior you want, but must be included in the function definition for compatibility reasons.

For example, to reset matplotlib to always use the ggplot style, you could do:

def reset_mpl(gallery_conf, fname):
    from matplotlib import style
    style.use('ggplot')

Any custom functions can be defined (or imported) in conf.py and given to the reset_modules configuration key. To add the function defined above (assuming you’ve make it importable):

sphinx_gallery_conf = {
    ...
    'reset_modules': ("sphinxext.reset_mpl", "seaborn"),
}

In the config above "seaborn" refers to the native seaborn resetting function (see Resetting modules).

Note

Using resetters such as reset_mpl that deviate from the standard behavior that users will experience when manually running examples themselves is discouraged due to the inconsistency that results between the rendered examples and local outputs.

If the custom function needs to be aware of whether it is being run before or after an example, a function signature with three parameters can be used, where the third parameter is required to be named when:

def reset_mpl(gallery_conf, fname, when):
    import matplotlib as mpl
    mpl.rcParams['lines.linewidth'] = 2
    if when == 'after' and fname=='dashed_lines':
        mpl.rcParams['lines.linestyle'] = '-'

The value passed into when can be 'before' or 'after'. If reset_modules_order in the configuration is set to 'before' or 'after', when will always be the same value as what reset_modules_order is set to. This function signature is only useful when used in conjunction with reset_modules_order set to 'both'.

previous

Advanced topics

next

Frequently Asked Questions