Build a data source

One JSON document naming a vendor, the endpoints a manager may ask for, and what the investor has to supply to reach it.

What a data source is

A data source is a vendor Aumos holds a credential for and will make requests to, and that is the whole of it. The host does three things and deliberately not a fourth: it authenticates, it refuses any path outside the allowlist your document declares, and it signs the request. What comes back is handed to the manager unread — not mapped, not renamed, not dated, not trimmed.

That is a decision rather than a gap. There used to be a port layer here that put every vendor behind one contract and dated every fact; removing it means a manager gets a vendor's own JSON and has to read the dates out of it itself. One capability, source:passthrough. One tool, source_request.

A source is one JSON document, not a directory of prose, and it can express far less than a program can. Every field is a literal, a number, or a member of a closed enum, and every object is strict — so a document cannot express something executable by shape rather than by inspection. There is no field whose contents are evaluated and no regular expression you supply.

The document

your-source-id/
  source.json
  README.md
  README.ko.md            ← optional
  translations/ko.json    ← optional; carries the one string a person reads

Field	What it is
`spec`	Always `"SourceSpec/1"`.
`id`	The source id, lowercase. Also the directory name, and what the investor names to install it.
`version`	Exact semver. Published once, dated once.
`vendor`	Who the data belongs to.
`caveat` optional	The one string in this format a person reads and the interpreter does not: where the claim breaks, what is not covered, what the dates in the response mean. The only field a translation carries.
`local` optional	Declares a document that reaches loopback only. A four-part bargain: loopback hosts only, no credentials at all, private and link-local still refused, and the investor has to name the id on their own machine.
`hosts`	Every host this document may reach. `https` only; loopback, link-local and private ranges are refused by name as well as by address — a credential pointed at a local address is how a credential fetch becomes a credential theft.
`headers` optional	Fixed request headers. Literals, and never a place to put a key.
`credentials` optional	What the investor has to supply, by name. ⚠️ A document can never name an environment variable — the host composes `AUMOS_SOURCE_CREDENTIAL_<ID>__<NAME>`, because the process that reads it is the one holding broker keys.
`auth` optional	One closed enum member — RFC 6749 client credentials — meaning this vendor issues a session. You name the token endpoint and which two of your credentials are the client id and secret. Cursor pagination is deliberately not expressible: a loop whose termination depends on a response is not a fixed behaviour that can be named.
`endpoints`	Every path an agent may ask for, each on a declared host. No `..` segment, no `{placeholder}` in the authority, and the filled URL is re-checked before every request rather than trusted from parse time.
`port` optional	Internal. Which slot the gateway fills this document into; there is one, and it is defaulted.

A published document, whole

sec-edgar, whole:

{
  "spec": "SourceSpec/1",
  "id": "sec-edgar",
  "version": "0.3.3",
  "vendor": "sec-edgar",
  "caveat": "SEC EDGAR's own endpoints, relayed unread. The `{symbol}` segment of `companyfacts` is a filename and not a ticker: it must be `CIK` + the ten-digit zero-padded CIK + `.json` (`CIK0000050863.json`), and a ticker there answers 404. So `/files/company_tickers.json` is a precondition of this path and not only a mapping table; its `cik_str` is an integer the caller pads. `companyfacts` is a filer's whole XBRL history; nothing here is bounded by asOf and nothing is dated by Aumos, so reading the filing dates out is the caller's work. SEC requires a contactable caller; that is the credential.",
  "hosts": ["www.sec.gov", "data.sec.gov"],
  "headers": {
    "accept": "application/json"
  },
  "credentials": [
    {
      "name": "user-agent",
      "header": "user-agent",
      "default": "Aumos/0.2 (ops@aumos.app)",
      "required": true
    }
  ],
  "endpoints": [
    {
      "host": "www.sec.gov",
      "path": "/files/company_tickers.json",
      "query": []
    },
    {
      "host": "data.sec.gov",
      "path": "/api/xbrl/companyfacts/{symbol}",
      "query": []
    }
  ]
}

What a document may and may not reach

It reaches only where it declared. https only; every endpoint's host must be one hosts names; no .. segment in a path; no {placeholder} in the authority; and the filled URL is re-checked before every request rather than trusted from parse time.
No loopback, link-local or private address, by name as well as by address. A credential pointed at a local address is how a credential fetch becomes a credential theft, and cloud metadata lives in exactly that range.
It may not relay a broker. A broker is the one vendor where the credential that reads is the credential that trades, so a document with an endpoint on a broker's host is refused — at the merge and again at the install.
local: true is a bargain, not a hole. Loopback hosts only, so it is never a bridge; no credentials at all, so there is nothing to carry to that address; private and link-local still refused; and it does not run until the investor names its id on their own machine.

Credentials, and sessions

You name a credential; you can never name a variable. The host composes AUMOS_SOURCE_CREDENTIAL_<ID>__<NAME>, and every variable under that prefix is a secret with no exception list — because the process that reads your document is the one holding broker keys, and a document that could choose a variable name could choose one of theirs.

If the vendor issues a session, auth names one behaviour: RFC 6749 client credentials, with the flow written once in Aumos rather than described by you. You name the token endpoint and which two of your declared credentials are the client id and secret, and both must be required — a document whose session cannot be established stands up and then fails every request with a 401, which reads on screen as a broken vendor rather than as a missing key.

Cursor pagination is not expressible, and that is not a near miss of the same thing: a cursor loop's termination depends on a response, so there is no fixed behaviour to name.

Your README is for a person

Your README is required and nothing checks what is in it, because it is the page an investor reads before typing a credential into their keychain. Say what the vendor is, what a manager gets from it, what a key costs and where to get one, and what the data does not cover.

caveat is the same obligation inside the document — the one string in SourceSpec/1 a person reads and the interpreter does not. Everything else in the format is a host, a path, a header or a credential name, which is the wire and is never translated.

Try it before you publish it

Point the gateway at your document and list its tools:

export AUMOS_SOURCE_SPECS="$PWD/path/to/your-source-id"
export AUMOS_SOURCE_CREDENTIAL_YOUR_SOURCE_ID__YOUR_KEY='…'
export AUMOS_CAPABILITIES='source:passthrough'
echo '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' \
  | node packages/skill-gateway/dist/main.js

Two documents claiming one id make the gateway refuse to start rather than pick one.

Publish it

The same public repository as a manager, untilled/aumos-catalogue, under sources/ instead of managers/. Its lint runs both halves of the real check — a schema generated from the same zod source Aumos parses with, and the very file Aumos runs its coherence rules from — so a green tick there means what the merge means, minus the one rule it cannot run: the broker check reads Aumos's own connector table, which is not published.

Sources are a second index rather than rows in the manager catalogue, so an id means one thing on both sides and neither can make the other unreadable. The published ones are here.