Skip to content

nbconvert can occasionally fail with an error #3682

@JCGoran

Description

@JCGoran

Context

We use nbconvert to run the Jupyter notebooks, that are then shipped as part of the documentation.

Overview of the issue

Occasionally we may run into the following:

FAILED: CMakeFiles/notebook_basic_initialization_ipynb /nrn/build/docs/CMakeFiles/notebook_basic_initialization_ipynb
cd /nrn/docs && /usr/bin/cmake -E env NEURONHOME=/nrn/build/docs/share/nrn NRNHOME=/nrn/build/docs NMODLHOME=/nrn/build/docs NMODL_PYLIB=/usr/lib64/libpython3.13.so PATH=/nrn/build/docs/bin:/root/.local/bin:/root/bin:/usr/share/Modules/bin:/usr/lib64/ccache:/opt/venv/bin:/usr/lib64/openmpi/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin LD_LIBRARY_PATH=/nrn/build/docs/lib PYTHONPATH=/nrn/build/docs/lib/python:/nrn/test/rxd NEURON_MODULE_OPTIONS=\"-nogui\" jupyter nbconvert --to notebook --execute --inplace /nrn/docs/rxd-tutorials/basic-initialization.ipynb
  File "/opt/venv/lib64/python3.13/site-packages/ipykernel/kernelapp.py", line 331, in init_sockets
    self.shell_port = self._bind_socket(self.shell_socket, self.shell_port)
                      ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib64/python3.13/site-packages/ipykernel/kernelapp.py", line 253, in _bind_socket
    return self._try_bind_socket(s, port)
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/venv/lib64/python3.13/site-packages/ipykernel/kernelapp.py", line 229, in _try_bind_socket
    s.bind("tcp://%s:%i" % (self.ip, port))
    ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib64/python3.13/site-packages/zmq/sugar/socket.py", line 320, in bind
    super().bind(addr)
    ~~~~~~~~~~~~^^^^^^
  File "zmq/backend/cython/_zmq.py", line 1009, in zmq.backend.cython._zmq.Socket.bind
    _check_rc(rc)

  File "zmq/backend/cython/_zmq.py", line 190, in zmq.backend.cython._zmq._check_rc
    raise ZMQError(errno)

zmq.error.ZMQError: Address already in use (addr='tcp://127.0.0.1:37331')
[NbConvertApp] ERROR | Error occurred while starting new kernel client for kernel a30c58eb-ad46-4ebf-8cf0-1f48eb815012: Kernel died before replying to kernel_info

Seems to be an instance of jupyter/nbconvert#1066, i.e. the issue needs to be fixed upstream. One reason why this could occur is that we build all of the notebooks in parallel, so it's possible nbconvert tries to use the same port for 2 (or more) notebooks simultaneously, causing the error.

Expected result/behavior

The conversion process should always succeed, regardless of the concurrency scheme.

Minimal working example - MWE

CMake build commands:

cmake -DNRN_ENABLE_DOCS=ON -DNRN_ENABLE_INTERVIEWS=OFF -DNRN_ENABLE_MPI=ON -DNMODL_ENABLE_PYTHON_BINDINGS=ON -B build
cmake --build build --target docs

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions