Notebooks#
The JUPYTER tab is a JupyterLab served from inside the Web Terminal. Its
kernels import osprey.runtime, so a cell reads and writes the control
system through the same connector the agent’s own Python uses, under the same
write gates and with the same refusal text. Notebooks are ordinary files on a
durable path, so they outlive the container, and the OSPREY agent can edit
them alongside you.
The panel starts one kernel, OSPREY. A request for any other kernelspec,
the interpreter’s own python3 included, is refused. A notebook that names no
kernel gets the OSPREY one, so every cell runs under the gates below.
Turning the panel on#
The panel is a built-in with the id jupyter. The demo
preset selects it, so every tier of a standard build has the tab. The
hello-world preset and the bundled logbook, knowledge and channel-finder
logins do not select it.
Name it under web_panels to add it to a build profile of your own:
web_panels:
- jupyter # JUPYTER tab
In a hand-written config.yml it is enabled the way its peers are:
web:
panels:
jupyter: true
Note
JupyterLab, jupyter-server and ipykernel are core OSPREY
dependencies, so an image grows by roughly 50 MB whether or not the panel
is selected. Selecting it costs nothing beyond that. Leaving it out starts
no process and opens no port.
Where notebooks live#
Notebooks live in notebooks/ under the deployment’s agent-data root —
var/agent_data/notebooks/ in a container build. That directory is on the
durable volume, so notebooks survive osprey down && osprey up. Kernels do
not: stopping the deployment stops every kernel.
JupyterLab opens on that directory and cannot see above it. A path that
resolves outside it is refused with a 404, symlinks included — the check
is against the resolved path, not the text of it. Download and Copy
Download Link in the file browser obey the same rule: they serve files from
notebooks/ and nothing else.
Delete in the file browser removes the notebook from the volume. There is no trash to bring it back from.
When the terminal starts the sidecar and notebooks/ holds no notebook at
all, it writes getting-started.ipynb, a two-cell notebook holding one note
and one code cell. The note reads Cells read and write through the
deployment’s current control target and write posture. Any existing .ipynb
suppresses it, and it is never rewritten once written, so it is yours to edit
or delete.
The code cell reads a channel where the deployment already names one it can
serve, so running it returns a value rather than only importing two names.
The channel comes from the deployment’s own configuration, in this order: the
archiver_freshness health check’s channel, which a facility declares is
still moving, then a control target’s probe_channel. A deployment that
names neither — the hello-world preset serves the simulator in process and
declares no channel — gets the import line alone, because a cell naming a
channel nothing serves would fail on its first run.
Which machine a kernel writes to#
A kernel has no control-system identity of its own, and it follows no chat session. Cells read and write through the deployment’s current control target and write posture, the same ones the control-target chip in the sidebar shows and the agent uses. The write posture is the position of the write switch on that chip, for example writes asked, allowed or off (see The control-target chip for every state the chip shows).
The kernel re-reads that before every cell, so a switch made anywhere, by anyone, reaches the next cell you run. A cell can read back what it was routed with:
import os
os.environ["OSPREY_CONTROL_TARGET"] # the machine this cell writes to
os.environ["OSPREY_CONTROL_TARGET_GENERATION"] # which switch it belongs to
os.environ["OSPREY_LAUNCH_POSTURE"] # what this cell may write
Nothing restarts. Switch the control target from the chip, or move the write switch, and the next cell you run is routed by the new state.
Inside the web terminal, the chip in the sidebar is the only place to do that; the JUPYTER tab carries no chip of its own. A notebook in a window of its own has no sidebar beside it, so that window shows the same chip in its top-right corner. It is the same switch either way: a change made from either place lands deployment-wide.
A cell already running is not re-routed, but taking writes away still reaches it:
Change |
Reaches the running cell? |
Reaches the next cell? |
|---|---|---|
Turn writes off |
Yes, at its next write |
Yes |
Widen writes, or switch the machine |
No |
Yes |
Turning writes off lands at once. The connector re-reads the write posture on every write, so the running cell’s very next write is refused.
Widening waits for the next cell. A cell can never write more than it was allowed when it started, so widening cannot reach it.
Switching the machine waits for the next cell as well, so a switch never lands halfway through your work.
Re-run the cell to pick up either of the last two, and the refusal says which one you are in:
The control target changed while this cell ran. Re-run the cell.
Writes are off for this cell. Set the machine to asked or allowed on the chip, then re-run the cell.
A switch takes a moment to reach the machines, and a cell run in that window is routed nowhere rather than to a guess:
switch_in_progress:5150. A control-target switch is in progress on pid 5150; re-run the cell when the chip settles.
It works the other way too. From a cell’s first control-system call until that cell ends, a switch asked for anywhere in the deployment is refused and names the kernel holding the target. Whoever asked then knows to interrupt the kernel rather than wait on nothing. A cell that touches no channel holds nothing.
In a multi-user deployment, a cell reads the chip settings of the user whose terminal the kernel belongs to, and nobody else’s.
Under the hood: whose record a kernel reads, and what a write leaves
The record a cell reads is the one belonging to whoever the kernel’s
terminal is acting as, and a kernel finds it without being told. The build
points each terminal’s container at its own record directory through
OSPREY_CONTROL_CONTEXT_DIR, and a kernel reads that. Where nothing names
one, it falls back to the agent-data root and the name it is acting as.
Where its own audit records are filed is a separate question, answered by
that name rather than by a directory. A kernel starts with most of the
environment around it stripped away, and OSPREY_AUDIT_IDENTITY (the name
the deployment gives the container it runs in) is what survives the
stripping. In a multi-user deployment both answers are the user whose
terminal the kernel belongs to.
Each channel a cell writes leaves one allowed record on the
notebook_kernel surface, filed after the put. Its reason says how the
write ended: write_landed when the write was verified,
write_unconfirmed when the value was sent but not verified. The record’s
detail names the channel and the account and host the control system saw
the write come from. That is what joins it to a gateway’s put-log. A refused
write reaches no channel and leaves only its refusal record. See
Who the control system saw.
A cell carries no target at all in two cases: the deployment’s control-context record is missing or unreadable, or it names a machine this deployment cannot build — a target whose connector block was never rendered, or was removed under it. Reads then answer from the deployment’s baseline target and every write is refused. The first case clears itself as soon as the web terminal has written the record; the second is a configuration gap and stays until somebody fixes it.
A cell reaches the machine through osprey.runtime. Importing a
control-system client library — epics, aioca, epicscorelibs,
caproto, p4p, pvaccess, tango, PyTango or doocs4py —
or loading one through ctypes or cffi is refused, for reads as for
writes, whatever the chip says. It raises ClientLibraryRefusedError,
nothing is loaded, and the cell prints one line above the traceback:
Client libraries are not available in a notebook. Read and write through osprey.runtime.read_channel / write_channel instead.
No switch position changes that answer. Replace the import with the runtime’s calls, which take the same address and value:
# Refused:
# from epics import caget, caput
from osprey.runtime import read_channel, write_channel, write_channels
value = read_channel("DEMO:CORR1:SP")
write_channel("DEMO:CORR1:SP", 1.5)
write_channels({"DEMO:CORR1:SP": 1.5, "DEMO:CORR2:SP": -0.4})
ophyd, and ophyd-async’s EPICS modules such as ophyd_async.epics.core, are
refused with them, because they load a client library underneath; bluesky
and ophyd_async.core import. Submit a plan that drives devices to a Bluesky
lane queue instead, where it runs unchanged. Each refusal is filed on the
notebook_kernel surface with reason raw_client_import.
The refusal keeps a cell on the connector path; it is not a sandbox. A shell escape or a subprocess starts a fresh interpreter without it, and what closes that route is the deployment’s network; see Safety Chain.
A refusal for any other reason — a write ceiling, a limits violation — carries no extra line, because nothing you do in the notebook would change it.
A page reload keeps the kernel running, but the notebook comes back as it was last saved. Output from cells that ran since that save is gone.
What a cell’s environment holds#
The notebook server and its kernels start with the fixed set of environment
variables executed code gets (python_executor: — how long one agent script may run, and what it inherits): paths, locale,
the interpreter, TLS trust, proxies, the control-system client settings and
the osprey names a cell reads. The terminal’s provider key, its database
passwords and its telemetry login are not in that set, so os.environ in a
cell does not hold them. A cell that needs another variable gets it the way an
executor run does, by name:
config:
python_executor.child_env_passthrough: [HDF5_PLUGIN_PATH]
This keeps credentials out of a kernel’s own environment and is not a sandbox. A kernel runs as the same user as the terminal’s server and the agent, in the same container and on its network: what that user can read on disk or from another of its processes, a cell can read, and what the container can reach, a cell can reach (What the switch does not close).
For that reason the panel needs a login wall. A deployment on
auth.method: none trusts every request its proxy forwards, and a cell could
send one; osprey up refuses to start such a deployment while a render shows
this panel (open posture).
Notebooks the agent also edits#
The OSPREY agent may edit a notebook under notebooks/ (and, as before,
under artifacts/); an edit anywhere else is refused by the memory guard.
When it edits one, the JUPYTER row in the sidebar badges and the history row reads
agent edited <notebook>.
What you see next depends on what the notebook was doing at the time:
Not open — it opens with the agent’s change already in it.
Open with no unsaved edits of yours — pick File → Reload Notebook from Disk to take the change.
Open with unsaved edits of yours — JupyterLab notices the file moved under it and offers Overwrite or Revert when you save. Your work is never silently replaced.
Notebooks in a multi-user deployment#
The multi-user stack gives every user their own container and their own volumes, which a redeploy never touches (Multi-User Support), so notebooks are per user. There is no shared notebook folder. Sharing a notebook means handing over the file.
What a notebook output cannot run#
Output an agent or a kernel writes can no longer run script inside the console’s JupyterLab. The page’s policy admits only the scripts the terminal served itself, because script in an output would run on the console’s origin and could move the write switch.
No longer run:
IPython.display.Javascriptandapplication/javascriptoutputs.<script>in an HTML output, and inline handlers such asonerror.bokeh’s in-notebook renderer (
output_notebook()).Vega and Altair outputs, whose renderer is disabled with
@jupyterlab/javascript-extension.Frontend code a kernel supplies for itself, which is unsupported.
Still work: matplotlib, Plotly fig.show(), standard ipywidgets, tables,
markdown, math and images. Bokeh dashboards in the artifact gallery are
unaffected.
The policy keeps 'unsafe-eval', because Plotly and JupyterLab’s own
settings need it. Script in outputs is blocked by the nonce, not by that
keyword.
How the tab is themed#
The JUPYTER tab follows the light or dark side of the terminal’s theme. A dark theme shows JupyterLab Dark, and a light theme shows JupyterLab Light. When you change the terminal theme (Theming), the tab changes too. In the tab, the terminal theme replaces a theme that you pick from Settings → Theme.
The tab changes its theme after JupyterLab starts. Until then, JupyterLab
shows the theme that it stored last. If it has no stored theme, it shows the
pinned theme of the deployment. A web.theme that names a dark or a light
look pins the JupyterLab theme for that look. A family name on its own pins
nothing.
JupyterLab stores each theme change on the durable volume, so the theme comes back after a sidecar restart. A JupyterLab page that you open at its own address does not follow the terminal. It starts in the theme that the tab used last. A theme that you pick there from Settings → Theme stays until the tab follows the terminal again.
When the tab fails to start#
A dimmed JUPYTER entry whose tooltip reads JUPYTER failed to start:
<reason> means the sidecar did not start. On a host where the first start is
slow, the terminal gives up after 60 s. Set web.sidecar_ready_timeout_s
(web: — documentation, feedback, the bars and the chat pool) to give the sidecar longer. The reason is one line: what
went wrong, and the sidecar’s last error line when it has one. If the Jupyter
tile is open, it shows JupyterLab did not start, the reason, and a Retry
button. To see the last lines of the sidecar’s error output, open the tile’s
⋯ menu and select Show server log. The terminal log has the full error
output. The rest of the terminal is unaffected — only that one tab is
unavailable.
Click the entry, or Retry in the tile, to start the sidecar again. The terminal runs one attempt at a time and waits as long as it does at startup; the tooltip reads JUPYTER is starting and the tile shows Starting JupyterLab… until the attempt settles. A start that succeeds opens the tab; one that fails again shows its new reason.
A sidecar that dies after it started turns the entry the same way within about ten seconds. JupyterLab reads Disconnected first and saves fail, with nowhere else to save to. Copy any unsaved cells out of the browser before you start it again.
When JupyterLab stops answering but the sidecar still runs, click the Reload icon on the tile bar. This loads JupyterLab again; the kernels keep running. To stop the sidecar and start a new one, open the tile’s ⋯ menu and select Restart kernel server. The terminal asks you to confirm first, because running kernels stop and their unsaved state is lost.
osprey health never fetches the panel. It shows the same failed to start
sentence as a warning row when the terminal recorded a failure, and a skip row
otherwise, so it never claims a panel is healthy on evidence it does not have.
Limitations#
The agent cannot run cells. It edits notebook files; you run them.
No real-time collaborative editing. Two people in one notebook fall back to the save-time dialog above.
No health probe of the sidecar; the row reports what the terminal recorded.
See also
- Panels
The other tabs, and how the panel proxy treats credentials.
- Run the Web Terminal
Running the terminal that hosts them.