LWON 1.0 — Lightweight Object Notation

Created by Ahmet Zencirli. Licensed under Apache-2.0; see LICENSE and NOTICE.

LWON is a text format for JSON data. It writes the shape of the data once, in a header, and then writes only the values, by position, in a body. Any JSON value can be written as LWON and read back without loss. LWON is built for one job above all: putting structured data into an LLM prompt with fewer tokens.

id name role [tags]
[
[1 Alice admin [new paid]]
[2 Bob user []]
]

is the same data as

[{"id":1,"name":"Alice","role":"admin","tags":["new","paid"]},
 {"id":2,"name":"Bob","role":"user","tags":[]}]

1. Document

A document is a header followed by a body.

The header ends at the first line break that is not inside brackets. The header has one of three forms:

An encoder MUST use * when the root is neither an object nor an array whose elements include at least one object and no arrays.

2. Header

A header is a space-separated list of fields. Each field has a key and a kind.

The [ that opens inner fields MUST follow the key with no whitespace between them: a[b] is one object field, a [b] is two fields.

A field list MAY end with *, the keyed slot: the objects at that position may have keys besides the listed fields, and each record carries them by name (§3.3). * needs at least one field before it and must come last.

2.1 Keys

A key is written bare when it matches [\p{L}\p{N}_$][\p{L}\p{N}_$.\-]*. Every other key, including the empty key, is written as a JSON string: "first name", "@type", "".

2.2 Types and recursion

Every object or rows field defines a type: its list of inner fields. The type is named after its key. The root table or root object is a type named @.

A reference @name stands for the nearest enclosing type named name; @ alone stands for the root type. References make recursive data possible:

title link [children[@]]

describes a tree whose every node has title, link and children.

3. Body

Whitespace separates values. Commas outside embedded JSON are treated as whitespace, so [1, 2] and [1 2] read the same in LWON positions.

3.1 Primitives

A bare word starts with a letter (\p{L}) or _, contains no whitespace, no [ ] { } " \ , ~ and no control characters, and is not null, true or false. Alice, user0@example.com and https://x.dev/a?b=1 are bare words; "42", "true" and "New York" need quotes.

3.2 Values by kind

Where a field’s kind expects a record or an array, the value can still be any primitive or an embedded JSON object, so data that does not fit the header is never lost.

Embedded JSON is strict JSON (RFC 8259) and is read as such, commas and all: {"a":1}, [1,"x",[2]].

3.3 Records

A record is [ followed by one value per field, in header order, then ].

• ~ marks a field whose key is absent from the object.
• null marks a key that is present with the value null.
• A record MAY stop early: fields after its last value are absent.
• A record MUST NOT have more values than fields.
• When the field list ends in *, a record has one more slot after the fields: ~ (or nothing, at the end) when the object has no other keys, or a JSON object holding those keys and their values. A key in that object MUST NOT also be a listed field.

id level userId *
[
[1 info 7]
[2 warn ~ {"retry":2,"region":null}]
]

is [{"id":1,"level":"info","userId":7},{"id":2,"level":"warn","retry":2,"region":null}].

~ is valid only directly inside a record.

3.4 Empty values

4. Rules

4.1 Any values

An any value is a primitive (§3.1), an embedded JSON object, or an embedded JSON array.

4.2 Equality

Decoding the encoding of a JSON value MUST give back an equal value. Object key order is not significant (RFC 8259 §4) and MAY change: keys come back in header order.

4.3 Numbers

Numbers follow JSON number syntax. LWON adds no precision of its own; a decoder reads numbers the way its platform’s JSON parser does.

5. Grammar

document  = ws header ws body ws
header    = "*" / "{" fields "}" / fields          ; ends at a depth-0 line break
fields    = *( ws field ) [ ws "*" ] ws             ; "*" only after a field
field     = key [ "[" sub "]" ]                    ; no space before "["
          / "[" key [ "[" sub "]" ] "]"
          / "[" ref "]"
sub       = fields / ref
ref       = "@" [ name ]
key       = bare-key / json-string

table-body  = "[" *( ws row ) ws "]"
row         = record / primitive / json-object
record      = "[" *( ws ( value / "~" ) ) [ ws json-object ] ws "]"   ; object only in the "*" slot
value       = depends on field kind (§3.2)

6. Encoder guidance

The format allows many encodings of the same data. The reference encoder:

1. Takes the union of keys, in first-seen order, across all objects at a position.
2. Picks a kind per key: object when the values are objects and primitives, rows when they are arrays of objects and primitives, list for other arrays, any when objects and arrays are mixed.
3. Falls back to any when a position holds objects so sparse that ~ markers would cost more than repeating keys (fewer than 30% of slots filled).
4. Uses a reference instead of a new type when the objects’ keys are a subset of an enclosing type’s keys, missing at most one, and every value fits.
5. Moves keys that appear in fewer than half of the objects at a position to the * slot. Language models match values to fields by position with more reasoning, and rows full of ~ cost them the most; named sparse keys cut that reasoning to close to what JSON needs (see bench/eval).
6. Writes a value as embedded JSON only when it does not fit the header.
7. Omits trailing ~ in records and quotes only strings that are not bare words.

7. Changes from the 0.x draft

• {} means an empty object, not null. Null is null; an empty array is [].
• ~ marks an absent key.
• Strings use JSON escapes; bare words need no quotes.
• Values that do not fit the header are written as embedded JSON.
• {fields} and * headers cover root objects and any other root value.
• A field list may end in *, the keyed slot for keys most objects lack.
• The root wrapper form [menu[… [submenu[@menu]]]] is removed; use title link [submenu[@]].