Import a Middle Layer export#
osprey facility import mml reads a MATLAB Middle Layer (MML) export and
writes it as sources of your facility description, under
data/facility/imported/mml/. osprey build merges those sources with
everything else under data/facility/; the import itself builds nothing.
Export from MATLAB#
The exporter ships with OSPREY. Print it, copy it to the MATLAB host and run it once per sub-machine:
osprey facility import mml --print-exporter > mml_export.m
Each run writes six files that share a stem. The import is given the
<stem>.ao.json and reads the other five from beside it.
osprey facility import mml --help lists the six files and what a run needs
of the Middle Layer.
What the files contain#
The lattice is saved first, before the export samples anything. Reading a nominal reaches its answer through the lattice’s own closed orbit and turns radiation off or the cavity on to get one, leaving the lattice in whichever state it needed, so a lattice saved afterwards would no longer be the one the rest of the export describes. A model measurement of the response matrix steps its correctors on a copy, so what it leaves behind is what its own readings needed rather than the columns it built.
Each file opens with an _export block recording the exporter version, the
MATLAB version, the machine, the sub-machine and when the export ran. After it
come the AO families (or the AD fields) as the Middle Layer holds them, with a
few values rewritten so JSON can carry them faithfully:
function handles become
{"$fn": "<name>", "file": "<path>"};padded char matrices become one trimmed string per row;
Inf,-InfandNaNare written as the strings"Inf","-Inf"and"NaN", never asnull, which would lose the value;logicals become
0/1, and theHandlesfield (plot handles) is left out.
Everything else is written as is. The script only reads from the Middle Layer; it sets nothing on the machine.
va.json is the part the Middle Layer cannot state as stored data. It opens
with a fingerprint of the lattice the file belongs to: element count, a digest
of the family names in lattice order, the model energy, and where any parameter
element sits. The importer recomputes it from the lattice file: a lattice that
disagrees on the elements is refused, and an energy that differs alone is
reported. Then one block per family: the Setpoint’s and the Monitor’s own
hardware-to-physics calibrations, and the Monitor’s physics-to-hardware
inverse, each sampled through the facility’s own conversion functions rather
than read out of their stored parameters; whether the Setpoint’s conversion
carries the beam’s rigidity; the per-device nominal the Middle Layer’s model
read gives; what the family’s own readings are corrected by, where it states
it; and, for a family the beam energy is read from, the energy table its
current maps to. The exporter’s own header lists every key.
The corrections (a gain, an offset, a roll and a crunch, one number per device) belong to whichever family states them, not to the beam monitors alone. A facility that calibrates its magnets and correctors from an orbit measurement keeps their gains and rolls under the same four names, so an export of a whole lattice usually carries a block of them on a dozen or more families. They are four numbers per device and nothing is resampled for them, so what that costs the file is negligible. The offset is in the family’s own hardware units (what its readback answers in, millimetres on most beam monitors) and not the physics units of the conversion beside it. A number the facility states nowhere is left out rather than defaulted, and one written in a shape that is not one per device is refused by name and costs that family only that number.
The numbers are looked up in the three places the Middle Layer looks: the
family’s Monitor field, the family itself, then the facility’s physics data
file, which a facility fills from an orbit fit and copies into the Accelerator
Objects at operating-mode set-up. Run the export from a session where that
set-up has run. An answer that came from the physics data can hold "NaN"
for single devices, because that file is stored against its own device list
and a device it does not cover comes back as no number, named on the console as
it happens. Such an entry means the facility states nothing for that one
device.
A family whose conversions refuse a sample keeps the facts already in hand and
records the MATLAB message under refused; it costs the export nothing else.
The same holds for the response matrix. A file the Accelerator Data names but
the Middle Layer cannot find ends in its own measurement of the model, and so
does an Accelerator Data that names no file at all: naming nothing is the one
state the Middle Layer answers with a file-chooser dialog, and an export runs
unattended, so the measurement is asked for directly instead. Each block’s
origin says whether the matrix was measured on the machine or computed from
a model, which is not the same question as whether a file answered, since a
facility may keep a computed matrix in a file like any other. A matrix computed
from a model says nothing about the machine: osprey facility validate then
compares a deck against a deck.
The <stem>.model.json file is not imported: it is what a check of OSPREY’s
model against the Middle Layer’s reads.
Run the import#
osprey facility import mml mymachine.storagering.ao.json mymachine.booster.ao.json
The first run finds no mapping, writes a draft to
data/facility/imported/mml/mapping.yaml and stops with
import mml: mapping-draft. No record is written. Review the draft, then
run the same command again.
The import takes every export the mapping names: a run given a subset is refused by the mapping check, which names each system no given export carries.
Each deck is held to the lattice its export was sampled over. A
<stem>.lattice.mat with another element count, other family names or other
parameter elements than the export states stops the import with
import mml: export-invalid and names the fact that differs: import the
lattice the export was sampled from, or export again over this one. An energy
that differs alone is reported and the import goes on.
Review the mapping#
The mapping holds every decision the export does not state. The draft fills in
what the export does say and leaves null in each slot only you can decide.
Every null must be replaced: the import stops with
import mml: mapping-undecided while one remains.
families.<raw>.devices says which device each slot of a family is. It takes
one of five forms:
Value |
Meaning |
|---|---|
|
The export’s |
|
The device segment of each slot’s address. |
a list of local names |
One name per device, in the family’s order. |
|
One device per |
|
The named family’s devices: matched by |
This fragment of mapping.yaml shows one devices answer per family; the
family’s other keys (aliases, description, provenance,
channels and fields, all required) stay as drafted:
families:
HCM:
devices: names
VCM:
devices: {same_as: HCM}
BPMx:
devices: {coordinates: BPM}
BPMy:
devices: {same_as: BPMx}
IonGauge:
devices: address
RF:
devices: [cavity1, cavity2]
The import stops with import mml: mapping-invalid for a list of the wrong
length, for names on a family the export leaves nameless, for
coordinates on a family with no DeviceList, for a same_as target that is
itself a same_as, and for a [sector, device] the same_as target
lacks.
Families that share addresses. One physical device listed under several
families must resolve to one id. The draft proposes this, giving the largest
family {coordinates: <stem>} and the others {same_as: <it>}, and says
so in a comment under each answer. An import under which a wired channel would
be two devices stops with import mml: mapping-invalid, naming the channel,
both families, both ids and the fix.
models.<name>.wiring lists the families the model drives or reads, each
with the field, engine attribute and calibration that bind it to the deck.
The optional models.<name>.tune names the channels the model’s tunes are
read on: one address per plane, {x: <address>, y: <address>} and
optionally s, or one waveform, {address: <address>, planes: [x, y]},
its planes optionally ending in s.
The optional top-level branches: block declares classes the shared
vocabulary lacks, each as <class>: {parent, description}. A family whose
class is neither a vocabulary class nor a declared branch names the class
it extends as its branch. The import writes each class the vocabulary
lacks to classes.yaml.
The facility: block seeds data/facility/identity.yaml on the first
import and is then removed from the mapping.
A mapping that disagrees with the exports prints one <key>: <message> line
per problem and writes nothing. Fix each line and run the import again.
What the import writes#
A clean run prints one wrote <path> line per file. The record files under
data/facility/imported/mml/ are rewritten by every import; mapping.yaml
is yours and is only drafted when absent. The authored files beside them —
limits.yaml, seeds.yaml, identity.yaml,
measurement/<model>.yaml and scenarios/readout.yaml — are created
only when absent and never rewritten, so your edits to them survive a
re-import. classes.yaml is created when absent; when it exists, the import
appends only the classes it lacks and leaves its rows as written.
measurement/<model>.yaml names, as quad and sext, the families the
export tags as tune and chromaticity correctors, and lists every kind those
groups and instruments allow; the import names the wired families it left out.
Each device carries the export’s CommonNames entry as its label. The
import writes one group per physical family: families that list the same
devices (BPMx and BPMy) are one group named by their common stem and
described by the family sentences. Each channel is described as
<device>: <field sentence>, from the mapping.
Each channel carries the role of its field’s direction and the signal the
mapping gives the field. The import writes no pair: osprey build pairs
each setpoint with the one readback on the same device whose signal names the
same quantity, so a field without a signal stays unpaired. A family whose
devices hold one coil per plane maps its current fields to the plane-qualified
signals (current_x_setpoint, current_x_readback, current_y_setpoint,
current_y_readback), so the build pairs each coil with its own readback.
Each device carries the export’s Position as s with its model, and
the build stops (place-conflict) when a wired device’s deck disagrees.
imported/mml/rows.json records which device each export DeviceList row
became, and the response check reads it.
Places from DeviceList#
The import writes places only for a model whose mapping entry answers
places; a mapping with no such answer writes no places.yaml and no
device place. The model is a place, and each sector sectors lists is a
place under it, named by token formatted with the sector number. This
fragment of mapping.yaml shows only the places answer; the model’s
description and provenance are required and stay as drafted:
models:
SR:
name: SR
places:
sectors: {count: 12, token: "SR{sector:02d}"} # SR/SR01 ... SR/SR12
model_wide: [RF, TUNE, DCCT] # these sit on SR
sector_override: {ExtractionKicker: 2} # this sits in SR/SR02
The sector list is stated, never read off the rows, so a place exists whether
or not a device sits there. Each device of the model takes the first of: its
family is in model_wide (the model); its family has a sector_override
(that sector); its DeviceList rows name several sectors (the model); its
rows name one sector (that sector); it has no row (the model). Both answers
are this model’s, so a family shared by two models can be sectored on one and
model-wide on the other. A device’s address is never where its place comes
from: a series supply’s one address feeds many magnets, each placed by its own
row.
Two placements stop the import before anything is written: a row naming a
sector beyond the list, and a device placed by its row and bound alone to an
address that begins with another sector’s token. Answer either with
sector_override or model_wide. The build records these places as
place_from: imported. A span that is the device’s sector or lies inside
it takes precedence, a span of the model above the sectors leaves the sector
standing, and a span of another sector stops the build place-conflict.
A setpoint takes its tolerance from its write field’s
Setpoint.Tolerance, one number per device or one for the family, written
{absolute: <x>} in the field’s HWUnits. A setpoint gets none when the
field states no unit, or its tolerance is not a finite number above 1e-12
(an Inf, or a machine-epsilon placeholder), and the import prints one line
counting the setpoint devices that export no usable Setpoint.Tolerance.
A later importer that reads EPICS records instead takes the default from the
record: MDEL when it is above 0, else 10^-PREC, and never HOPR or
LOPR.
Two more lines count what the wiring pass derived, one per family and only when
the count is not zero:
import mml: curves trimmed: <model>: family <family> (<n>) names a family
whose conversion curves the pass cut, and
import mml: supplies folded: <model>: family <family> (<n>) names a family
whose supplies feed several devices. Both go to stdout and never stop the
import.
Before it reads an export, the import looks at data/facility/:
Hand-written records of a kind this format writes refuse the import, which names each file, changes nothing and exits 1. The mml format writes channel, device, group, model, deck, seed, limit, identity and measurement records; the list format writes channel and device records. A file is hand-written once its bundled-origin line is gone. Corrections belong in
fixes.yaml.Once the facility holds an imported layer, records written by hand beside it stay and stop no import. They are the facility’s own additions, such as a model no export states, and the build combines them with the layers.
Bundled files are replaced, and one line says so:
import mml: replaced the bundled example facility (<n> files). The line prints before the export is read, so a stop that follows it, such as a mapping draft, comes after the replacement.Everything else of your own stays, such as
fixes.yaml, your knowledge pages and hand-written records of other kinds.
The import regenerates the knowledge indexes, so your kept pages are listed, and
leaves a blank knowledge/index.md when no page remains.
The build derives the simulator’s probe channel from the imported file. A stated
simulation.default_scenarios naming a scenario the imported file lacks stops
the next build with the file’s scenarios.
Check the result#
osprey facility validate
validate runs every check osprey build makes. For each model whose
export carried a response matrix it also compares that matrix with the one the
imported model computes, and prints one response check <model>: … line. A
failing check exits 1. When the check left rows out of the comparison, a second
line for that model follows on stderr, giving the number of rows left out and
the count per reason (unwired, no width, unsolved, table calibration); it does
not change the verdict or the exit code:
response check <model>: left out <n> rows (<k> unwired, <j> no width, <u> unsolved, <t> table calibration)
The check converts a monitor reading to position with one slope, taken at the centred beam. That is exact for a straight-line (gain and offset) calibration; a table-calibrated monitor is left out of the check and counted on the left-out line. If you need a more elaborate BPM calibration in the check, open an issue.
See also
- osprey facility
Every exit and message of
osprey facility validateandosprey facility import mml.