Design: references

Design: references#

Note

A design document, and the specification for how ChartBook addresses an asset. Every other design here writes addresses in this grammar, so read it before Design: the asset graph, Design: the index, Design: models or Design: data access. New to the project? Start with Design: start here.

Status: accepted 2026-10-12. The parser accepts the whole grammar (chartbook.refs, in 0.2.0). A local reference resolves to the path its manifest declares, and a reference to an asset of another pipeline resolves to the copy that chartbook.lock pins, for all five kinds. A part, a version and a registry parse, and are refused as “reserved and not yet supported”. See What works today.

The idea in one picture#

Data is the spine of ChartBook. An organization has pipelines, a pipeline has dataframes, and a dataframe has series (its columns):

ftsfr / fed_yield_curve : treasury_rates # dgs10
scope   pipeline          dataframe        series

Everything else a pipeline declares hangs off the pipeline, and says what kind of thing it is:

ftsfr/fed_yield_curve:chart/curve_today
ftsfr/fed_yield_curve:notebook/analysis
ftsfr/fed_yield_curve:note/methodology
ftsfr/fed_yield_curve:model/rate_forecaster@v3

That string is a reference. It is the one way to name an asset, wherever a name is needed: on the command line, in Python, in chartbook.toml, in the index, in a lockfile and in a URL.

Examples#

Reference

Means

treasury_rates

The dataframe treasury_rates in the pipeline you are in

chart/curve_today

The chart curve_today in the pipeline you are in

notebook/analysis

A notebook

note/methodology

A note

model/rate_forecaster

A model

treasury_rates#dgs10

The series dgs10 of that dataframe

model/rate_forecaster#weights.pt

One file inside that model

model/rate_forecaster@v3

Version 3 of that model

model/rate_forecaster@champion

Whichever version is currently labelled champion

fed_yield_curve:treasury_rates

A dataframe in another pipeline

ftsfr/fed_yield_curve:treasury_rates

The same, naming the organization too

ftsfr/fed_yield_curve:chart/curve_today

A chart in another pipeline

charthub.example/ftsfr/fed_yield_curve:treasury_rates

The same dataframe, from a named registry

The grammar#

reference   = [ pipeline-id ":" ] asset [ "#" part ] [ "@" rev ]

pipeline-id = [ registry "/" ] [ scope "/" ] pipeline
asset       = [ kind "/" ] name
kind        = "dataframe" | "chart" | "notebook" | "note" | "model"

scope, pipeline, name = one or more of  A-Z a-z 0-9 . _ -
part                  = one or more of  A-Z a-z 0-9 . _ - /
rev                   = one or more of  A-Z a-z 0-9 . _ - :
registry              = a host name, optionally with a port

There are four separators, and each asks a different question. Read left to right:

Separator

Question it answers

Example

/

One level down. Scope, then pipeline. Kind, then name.

ftsfr/fed_yield_curve, chart/curve_today

:

“From this pipeline, take…”

fed_yield_curve:treasury_rates

#

“…this part of it.”

treasury_rates#dgs10

@

“…at this version.”

model/rate_forecaster@v3

Because the separators differ, any piece can be left out and the rest still reads the same way. That is the whole reason there are four and not one.

Rules#

  1. No colon means local. A reference without : names an asset of the pipeline you are in. The current pipeline is found by walking up from the caller to the nearest chartbook.toml, as git finds a repository.

  2. No kind means dataframe. treasury_rates and dataframe/treasury_rates are the same reference. The short one is canonical: ChartBook always writes it that way.

  3. Kinds are singular. chart/gdp, never charts/gdp. The manifest’s tables are plural ([charts.gdp]); the parser says “did you mean chart?” when given a plural.

  4. Without a colon, x/y is always kind/name. It is never scope/pipeline. So ftsfr/fed_yield_curve is not an asset reference, and the error says to write ftsfr/fed_yield_curve:<asset>. This keeps the short forms unambiguous.

  5. Each kind has its own names. [dataframes.gdp] and [charts.gdp] are two assets, gdp and chart/gdp. Nothing is ambiguous, because the kind is part of the address.

  6. An asset may be named like a kind. chart alone is a dataframe called chart.

  7. A registry is only ever the first of exactly three components, and only if it looks like a host: it contains . or :, or is localhost. So charthub.example/acme/rates has a registry, and my.group/rates is a scope with a dot in it. No ID that was valid before registries were reserved changes meaning. (This is Docker’s rule, narrowed to three components.)

  8. Parsing goes right to left: take off @rev at the last @, then #part at the first #, then split pipeline from asset at the last :. That order is what lets a revision be a hash (@sha256:7f3a…) and a registry carry a port (localhost:5000/acme/rates:gdp).

A reference to a pipeline is a different thing#

Some places name a pipeline, not an asset: data.load(pipeline="ftsfr/fed_yield_curve", …), a key of [dependencies], a key of a catalog’s [pipelines]. Those take a pipeline ID, [registry/][scope/]name, optionally with @rev. They are handled by chartbook.identity, and they also accept a repository URL. The pipeline-id half of an asset reference follows the same rules, minus the URL form.

(Corrected 2026-10-12, proposed.) That is true of a lookup, such as data.load(pipeline=…). A key of [dependencies] or of [pipelines] is narrower: a plain ID, name or scope/name. A URL is never an identity, so it is not a key; and a dependency’s revision is written in its entry (rev = "a1b2c3d"), so @rev in a key is refused as reserved.

Where a pipeline is found, given its name, is one function: chartbook.resolve.locate_pipeline. It asks the [dependencies] of the pipeline that wants to know, then the catalog. A scoped name must match exactly. A bare name matches if exactly one entry has it, so with "ftsfr/fed_yield_curve" declared, fed_yield_curve:treasury_rates names that pipeline.

Kinds#

Kind

Manifest table

Index record

What #part means

Python front door

dataframe (default)

[dataframes.x]

Dataframe

A series: one column

chartbook.data

chart

[charts.x]

Chart

Nothing yet

chartbook.assets

notebook

[notebooks.x]

Notebook

Nothing yet

chartbook.assets

note

[notes.x]

Note

Nothing yet

chartbook.assets

model

[models.x]

Model

A file inside the model (not yet)

chartbook.models

Why dataframe is the default. Data is what gets passed between pipelines, loaded in notebooks and written in dependency lists. Charts, notebooks and notes are mostly looked at, not passed around. The common case gets the short spelling. (kubectl and Hugging Face do the same: the most common kind needs no prefix.)

Why names are per kind, not one namespace. An earlier draft made every asset name unique within a pipeline, so that scope/pipeline:name was enough. The maintainer rejected it: a chart of GDP and the GDP dataframe are naturally both called gdp, charts are not first-class in the way data is, and one namespace would have forced renames on existing projects.

Adding a kind means, in this order: add it to KINDS and KIND_TABLES in chartbook/refs.py; teach the manifest loader its table; add it to RESOLVABLE_KINDS in chartbook/resolve.py; add its record to the index schema; give it a page in the site; decide what #part means for it; and, if code will ask for it often, give it a front door module. model was the first kind to go through this; Design: models lists every place it touched, under “What is built”.

One address, three spellings#

The same address is written three ways, depending on who is reading it.

Text, for people, the command line, and chartbook.toml:

$ chartbook path chart/curve_today
$ chartbook path curve_today --kind chart

Python, where the kind can be an argument:

from chartbook import assets, data

assets.path("chart/curve_today")
assets.path("curve_today", kind="chart")     # the same call
data.path("treasury_rates")                  # dataframes only

JSON, in the index and anywhere a program reads an address. Named fields, nothing to parse:

{ "scope": "ftsfr", "pipeline": "fed_yield_curve",
  "kind": "model", "name": "rate_forecaster", "rev": "v3" }

kind and name are always present. The other keys (registry, scope, pipeline, part, rev) appear only when set, in the order the address is written.

What kind= means#

In Python and on the command line, kind says what you expect:

  • the reference names no kind → it takes this one;

  • the reference names the same kind → fine;

  • the reference names a different kind → KindMismatchError.

assets.path("gdp", kind="chart")          # chart/gdp
assets.path("chart/gdp", kind="chart")    # chart/gdp
assets.path("chart/gdp", kind="note")     # KindMismatchError
assets.path("gdp")                        # the dataframe gdp

Type safety, the small version#

Two things, and no more:

  1. Parse once. chartbook.refs.parse_asset_ref() turns text into an AssetRef, a frozen object with named fields that is checked when it is made. Code inside ChartBook passes that object around, never a raw string, and never splits a reference by hand.

  2. Typed front doors. chartbook.assets takes any kind. chartbook.data takes dataframes only: data.path("chart/gdp") raises, and says to use chartbook.assets. A later chartbook.models does the same for models. So a typo cannot hand back a chart’s HTML file where a parquet file was meant.

from chartbook import assets

ref = assets.parse("ftsfr/fed_yield_curve:model/rate_forecaster@v3")
ref.scope, ref.pipeline, ref.kind, ref.name, ref.rev
# ('ftsfr', 'fed_yield_curve', 'model', 'rate_forecaster', 'v3')

ref.pipeline_id     # 'ftsfr/fed_yield_curve'
ref.local_name      # 'model/rate_forecaster'
ref.table           # 'models'   (the manifest table)
ref.is_local        # False
str(ref)            # the canonical text
ref.to_dict()       # the JSON form
assets.AssetRef.from_dict(ref.to_dict()) == ref     # True

Parts: ##

A part is something inside an asset that deserves its own address. It needs no storage of its own; the asset already holds it.

(2026-10-13, decided by the maintainer: a part is an address, and never a way to load data. data.load("treasury_rates#dgs10") will not be built: ChartBook does not select columns, and a column is picked with polars or SQL. Design: start here, step 4, “4e”. What follows describes the address.)

  • In a dataframe, a part is a series. treasury_rates#dgs10 is one column, so that documentation, units and a frequency can attach to it. The key after # is the series’ key in [dataframes.treasury_rates.series.dgs10], which may name a differently-spelled column. See Addressing a series inside a dataframe in the asset graph.

  • In a model, a part is a file. A model is often a directory (weights, a tokenizer, a config). model/rate_forecaster#weights/best.pt is one file in it. This is why a part may contain /.

  • Charts, notebooks and notes have no parts yet. The syntax parses; nothing gives it meaning.

# was chosen because it maps onto the URL fragment of the asset’s page (…/treasury_rates.html#dgs10), and because it cannot appear in a name. In a shell it sometimes needs quoting (zsh with extended globbing reads # specially).

Versions: @#

@ selects a version. Two kinds of version name exist, and both are needed:

  • an exact version, which never changes meaning: @v3, or a content hash @sha256:7f3a…;

  • a label that moves: @champion points at whichever version was last given that label.

This is Docker’s lesson. A tag (:latest) and a digest (@sha256:…) are different things, and conflating them is how “it worked yesterday” happens.

Proposed, not yet settled

What follows is the working proposal for how a rev is looked up. It needs the store and the lockfile, so it is settled when those are built (step 4 in the Design: start here). The decided part is only: @ means a version, and exact versions and moving labels both exist.

  1. If it contains :, it is a content hash (sha256:<hex>): exactly those bytes.

  2. Otherwise, if the asset’s pipeline has recorded a version or an alias by that name for that asset (v3, champion), it is that.

  3. Otherwise it is a git revision of the pipeline (a commit, tag or branch): the asset as that revision of the pipeline built it.

Where version names and aliases are recorded is proposed in Design: models, since models are the first asset that needs them.

Where references appear#

Place

Example

Notes

Command line

chartbook path chart/curve_today

Python

assets.path(…), data.path(…)

Text or an AssetRef

chartbook.toml

dataframe = "repo_public" in [charts.x]

A local dataframe reference today. Future fields (inputs, trained_on) hold references in this grammar, with dataframe as the default kind

The index

"id": "ftsfr/fed_yield_curve:chart/curve_today" plus a ref object

IDs are fully qualified and carry no rev. See Design: the index

The lockfile

Keys of pinned assets

Same form as index IDs

SQL templates

{acme/rates:treasury_rates} in data.sql()

Dataframes only. See Design: data access

URLs

…/treasury_rates.html#dgs10

Why # is the part separator

What works today#

Form

Example

Today

Local dataframe

treasury_rates

Resolves

Local chart, notebook, note

chart/curve_today

Resolves

A dataframe through a catalog

data.load(pipeline="fed_yield_curve", dataframe="treasury_rates")

Resolves, by the older two-argument call

Another pipeline’s asset

fed_yield_curve:treasury_rates, fed_yield_curve:chart/curve_today

Resolves, to the pinned copy in the store, for a pipeline in [dependencies] that has been pinned. data.load() takes it too

A part

treasury_rates#dgs10

Parses. UnsupportedReferenceError

A version

treasury_rates@a1b2c3d

Parses. UnsupportedRevisionError

A registry

charthub.example/acme/rates:gdp

Parses. UnsupportedRegistryError

Local model

model/rate_forecaster

Resolves

Local is live; another pipeline’s is pinned. treasury_rates is a file of the pipeline you are in, which you are editing, so it resolves to that file. fed_yield_curve:treasury_rates is someone else’s, so it resolves to the copy your chartbook.lock pins, in the store, and does not change until chartbook update. The other pipeline must be in [dependencies]; one that is not raises DependencyNotDeclaredError, and an asset with no pinned copy raises AssetNotPinnedError. Both say what to add or run. (Design: the asset graph, “Pinning and isolation”.)

All three Unsupported… errors are ValueErrors whose message contains “reserved syntax and not yet supported”. A reference that breaks the grammar raises a plain ValueError instead. The difference matters: “not yet” tells the user the syntax is right.

What unlocks each row is in the Design: start here’s build order.

Where the code is#

Module

What it does

chartbook/refs.py

The grammar. AssetRef, parse_asset_ref(), KINDS, KIND_TABLES, KindMismatchError. Parses and formats; knows nothing about what resolves

chartbook/resolve.py

The one resolver. resolve(ref, kind=…) returns a typed Location: _resolve_local() for an asset of this pipeline, _resolve_pinned() for another’s. parse_declared_ref() says which forms are supported and refuses the reserved ones. locate_pipeline() is the resolution chain: where a pipeline is, from [dependencies] and then the catalog

chartbook/assets.py

The general front door: assets.path(), assets.parse(), assets.AssetRef

chartbook/data.py

The dataframe front door: data.path(), data.load(), data.scratch()

chartbook/models.py

The model front door: models.path()

chartbook/identity.py

Pipeline IDs: scope/name, the registry rule, deriving an ID from a git remote

chartbook/index.py

Writes each asset’s fully qualified id and ref into index.json, from one AssetRef (ref.in_pipeline(pipeline_id))

chartbook/cli.py

chartbook path REF [--kind KIND]

tests/test_refs.py

Every example in this document, as a test

Rules for whoever extends this:

  • There is one parser. Do not split a reference with string operations anywhere else. If a new form is needed, add it to refs.py and to this document in the same change.

  • There is one resolver. path(), load() and the CLI all call resolve(). Name resolution is singular by design (Not doing): two code paths that disagree about a name make dependencies unportable.

  • Refuse, do not guess. A form that cannot be resolved yet raises an Unsupported… error. It never falls back to something that happens to work.

  • To unlock a form, remove its refusal in parse_declared_ref() and add the resolution next to _resolve_local() and _resolve_pinned(). The grammar does not change. (There used to be a second function, parse_local_ref(), for what resolved, while another pipeline’s asset could be stated and not resolved. Since 2026-10-12 what may be stated is what resolves.)

Why this shape#

What other tools teach#

Tool

Address

Lesson taken

Docker

registry/org/image:tag@digest

Different separators let you leave parts out. A moving label and an exact version are different things; keep both

Bazel, Python entry points, git

//pkg:target, module:object, commit:path

A colon means “container, then the thing inside it”. That is our colon

URLs

page#section

# means a part of the thing

kubectl, Hugging Face

pod/web, datasets/org/name

Kind goes in front, and the most common kind needs no prefix

npm, pip, Hugging Face

pkg@1.2.3, repo@revision

@ means a version

SQL, Maven

db.schema.table, group:artifact:version

With one separator you must fill every slot, or depend on a hidden “current schema” setting

Alternatives that were turned down#

  • One separator for everything (ftsfr.fed_yield_curve.treasury_rates.dgs10). You could not tell pipeline.dataframe from dataframe.series, and . is legal inside a name.

  • The kind as a Python argument only (data.path("gdp", kind="chart")). It works in Python and nowhere else: not on a command line, not in TOML, not in the index. And it made chartbook.data hand out charts. The argument stays, as a statement of what you expect.

  • One namespace for all asset names. See “Kinds” above.

  • A URI scheme (chartbook://ftsfr/fed_yield_curve/dataframe/treasury_rates). Unambiguous, and nobody would type it. The short local form is the one people write a hundred times a day.

  • Plural kinds (charts/gdp), to match the manifest tables. Singular reads as a name (“the chart gdp”) and matches the index’s record kinds.

Open questions#

  • How @rev is looked up. Proposed above; settled with the store. (2026-10-13: built for one case, a version of one of this pipeline’s own models. model/x@7f3a9c2e41b0 is found among the pipeline’s snapshots by the start of the model’s content hash, and model/x@champion through a label in [models.x.aliases]. A revision of any other kind of asset, or of another pipeline’s, is still refused. Design: start here, “4d, as built”.)

  • Parts of charts, notebooks and notes. A notebook cell? A chart’s series? Nothing has asked for one yet.

  • Whether chartbook.toml tables may themselves be keyed by a reference (for example documenting a series of another pipeline’s dataframe). Not needed so far.