Render Plotly Interactive Plot in Colab and Quarto

[s2t2]

Bug description

I have an IPYNB file (notebook downloaded from Google Colab). It has a cell in it that makes a chart in plotly:

import plotly.express as px

fig = px.line(x=[1,2,3], y=[1,2,3])
fig.show()

I see the chart in the colab notebook, but not when I render that notebook as a source file for a Quarto project.

Plotly figures do get displayed when rendered from a .QMD file, but I want to use .IPYNB because my notebook has taken advantage of Colab-specific functionality, like forms, notebook secrets, etc.

My question is: how can I get plotly figures to be properly displayed in Quarto, using a .IPYNB file source, in such a way that will work BOTH in colab and in Quarto?

Here is some investigation code. It appears different renderers work on colab vs quarto, but there is no overlap?

The solution documented here does not render in colab:
https://quarto.org/docs/interactive/widgets/jupyter.html#plotly

pio.renderers.default = "colab+plotly_mimetype+notebook_connected"
fig.show()

Here is a notebook that shows the results of all renderer options in Colab. Only "colab" and "sphinx_gallery" renderers work in Colab. But they don't work in Quarto. We need one option that will work for both.

https://colab.research.google.com/drive/1UpmfA_Ybj8i4rDtgPnQz1af79STG2enP?usp=sharing

https://stackoverflow.com/questions/78694049/plotly-figures-not-displaying-in-quarto

Steps to reproduce

I have an IPYNB file (notebook downloaded from Google Colab). It has a cell in it that makes a chart in plotly:

import plotly.express as px

fig = px.line(x=[1,2,3], y=[1,2,3])
fig.show()

I see the chart in the colab notebook, but not when I render that notebook as a source file for a Quarto project.

Expected behavior

No response

Actual behavior

No response

Your environment

No response

Quarto check output

(quarto-env) --->> quarto check
Quarto 1.4.554
[✓] Checking versions of quarto binary dependencies...
Pandoc version 3.1.11: OK
Dart Sass version 1.69.5: OK
Deno version 1.37.2: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
Version: 1.4.554
Path: /Applications/quarto/bin

[✓] Checking tools....................OK
TinyTeX: (not installed)
Chromium: (not installed)

[✓] Checking LaTeX....................OK
Using: Installation From Path
Path: /usr/local/bin
Version: undefined

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
Version: 3.11.9 (Conda)
Path: /opt/anaconda3/envs/quarto-env/bin/python
Jupyter: 5.7.2
Kernels: python3

[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........OK
Version: 4.3.1
Path: /usr/local/Cellar/r/4.3.1/lib/R
LibPaths:
- /usr/local/lib/R/4.3/site-library
- /usr/local/Cellar/r/4.3.1/lib/R/library
knitr: (None)
rmarkdown: (None)

  The knitr package is not available in this R installation.
  Install with install.packages("knitr")
  The rmarkdown package is not available in this R installation.
  Install with install.packages("rmarkdown")

[cscheid]

If the solution we document works for Quarto, then I think we don't consider this a bug. It would be nice to learn of a solution that's portable to other execution environments, but that's an enhancement.

[cderv]

I have an IPYNB file (notebook downloaded from Google Colab). It has a cell in it that makes a chart in plotly:

import plotly.express as px

fig = px.line(x=[1,2,3], y=[1,2,3])
fig.show()

I see the chart in the colab notebook, but not when I render that notebook as a source file for a Quarto project.

FWIW we have other issues / questions about this renderers thing for plotly. About this, my understanding is that plotly will choose the appropriate renderers. From https://plotly.com/python/renderers/#setting-the-default-renderer I read

In many contexts, an appropriate renderer will be chosen automatically and you will not need to perform any additional configuration.

Colab is one of the context mention. If you are executing your notebook on collab, the rendering will be done considering this context. So probably, the ipynb require new execution with Quarto quarto render from-collab.ipynb --execute. Especially if specific renderers are used like Google Colab dedicated one.

ipynb is a special format that store execution result. I don't know if there is a solution where the execution result from plotly for colab and quarto can be the same. It seems to me plotly expect a renderer to be associated to different environment, unless a more common renderer is used which can be more portable (like svg for example).

Now, the ipynb downloaded from collab should be ok to be rendered with quarto render - if not, then this is another issue as Quarto should be able to execute any ipynb when the right python environment is provided.

[kcarnold]

I faced the same thing and looked into it. In short: this should be doable, but it isn't easy.

Plotly uses a different renderer depending on the context in which it's running. Under quarto render, it uses a renderer called notebook_connected; under Colab it uses the colab renderer. They both use the same core output mechanism, but wrap it differently.

An "offline" conversion (without re-running) would require two things:

1. Imitating the activate method, which adds some init code the first time Plotly is used in a notebook, and
2. Editing each Plotly cell's output to reflect the different conditional paths taken in to_mimebundle.

The first step could be basically a copy-paste from a known-good notebook. The second step is more complex. Here's the approach I'd take:

1. Make a minimal ipynb that renders a single Plotly output.
2. Execute it in both Colab and quarto render with keep-ipynb.
3. Write a script to:
   1. find the output with Plotly.newPlot in both notebooks.
   2. Replace the randomly-generated ID with a placeholder value in both outputs.
   3. Compute and store the diff between the Colab and the Quarto output.
   4. Also find the activation stuff and store it too.
4. Write another script to, given the stored diff and a downloaded ipynb file, apply the diff to each Plotly output (watch out for different Plotly versions if Colab upgrades...). Also add the activation stuff to the output of the first code cell.

Another possible approach could be to parse out the args to Plotly.newPlot and shove them back into the plotly.io renderer function. I got partway with this before I realized it had some extra complications (like the renderer functions expecting to run inside of a Jupyter cell environment...). But here's the code I got to with that approach:

WIP code

'''Parse out the plotly call, which looks like:
                    Plotly.newPlot(\
                        "{id}",\
                        {data},\
                        {layout},\
                        {config}\
                    )
'''

def parse_plotly_call(output):
    # find the Plotly.newPlot(
    start_idx = output.find("Plotly.newPlot(")
    if start_idx == -1:
        return None
    # find the matching closing parenthesis. Note that there can be nested parentheses and quoting.
    comma_separated_parts = []
    start_idx = idx = start_idx + len("Plotly.newPlot(")
    open_delims = '('
    while len(open_delims) > 0:
        cur = output[idx]
        if len(open_delims) == 1 and cur == ",":
            comma_separated_parts.append(output[start_idx:idx])
            start_idx = idx + 1
        if cur == ")":
            assert open_delims[-1] == "("
            open_delims = open_delims[:-1]
        elif cur == "(":
            open_delims += "("
        elif cur == "{":
            open_delims += "{"
        elif cur == "}":
            assert open_delims[-1] == "{"
            open_delims = open_delims[:-1]
        elif cur == "[":
            open_delims += "["
        elif cur == "]":
            assert open_delims[-1] == "["
            open_delims = open_delims[:-1]
        elif cur in '"\'':
            if open_delims[-1] == cur:
                open_delims = open_delims[:-1]
            else:
                open_delims += cur
        idx += 1
    comma_separated_parts.append(output[start_idx:idx])
    return comma_separated_parts
parse_plotly_call(bad_cell_example.outputs[1]["data"]["text/html"])