webdevtodayjason Release Notes

TiinyBench 0.1.11

key() gains one more fallback, after every existing source (env var, farm config, saved config, TiinyOS scrape) comes up empty: it asks for the box's password and calls the device's own account API to get a working key directly, no TiinyOS desktop app needed. Only runs at an interactive terminal, never in a cron job.

Also carries the fixes since 0.1.10:

• Two numbers on a public page that could not be true
• Results survive an update now
• A rate divided by two tokens is not a rate
• A 404 that meant four different things
• The alternation was two backends, not a fussy validator
• A predicate that could raise instead of answering

Everything in this release came out of one thing: a sweep of all 52 installed models that came back with eleven measurements empty. Reading what the device actually said in those failures turned four of them into real numbers and four more into a diagnosis.

Numbers that did not exist before

• Music generation is measured. Foundation-1 runs at 0.29x real time, about seventy seconds of wall clock for twenty seconds of audio. It also ignores the duration you ask for and returns twenty seconds regardless, which the record now shows because it keeps both the ask and the result.
• Speech for Supertone-3, 2.10x real time.
• Both 0.6B embedding models, around 22 vectors a second at 1024 wide.

Why they were missing

• Music came back as a JSON envelope with the audio base64 inside it. The benchmark only understood a raw file or a job to poll, so it recorded a failure for a model that had just produced audio.
• Two music models disagree about their own request shape: one requires a duration, the other refuses it. One refuses the prompt and then says a prompt is required.
• Supertone's refusal listed the ten speakers it does have. Nobody read it.
• Two embedding models answered 502 because the device lists a model as running before its runtime accepts connections.

What changed

Refusals are now read rather than discarded, everywhere. The benchmark parses the field a 400 objects to, the speakers a 500 enumerates, and the …

The prebuilt shelf that LAST LIGHT downloads on first run. This is data, not code: there is no program in it. The app fetches it, checks it by length and then by hash, and unpacks it into its own data directory.

size 73196259 bytes sha256 85dacb04a56693295ffe23e2251f746a7250a366a475469eb1ce4fda43181e5c

What is on it, 5,955 documents:

• 5,934 Vikidia articles (https://en.vikidia.org), under CC BY-SA 3.0 or the GFDL, the same terms as Wikipedia.
• 21 survival and reference PDFs, 4,189 pages. Most are works of the US federal government and so are not under copyright in the United States: FEMA, USDA, EPA, USGS, HHS, the Army's Borden Institute, Oak Ridge. Two are public domain by age (Harding 1907, Gibson 1881, Hasluck 1917). One carries a written grant on its own page (the SODIS Safe Water School manual).
• Three of those 21 are Hesperian titles under CC BY-NC-SA: Where There Is No Doctor, Where There Is No Dentist, A Book for Midwives. Non-commercial.

Two plain text files travel with it. ATTRIBUTION.txt carries the Vikidia credit and licence, which a share-alike licence requires and which the app enforces: it refuses to unpack a bundle that does not have it. SOURCES.txt lists every document and why it is free to pass on.

Anything whose terms could not be established in writing was left off rather than guessed at. The corpus this was cut from holds 109 PDFs; 88 of them are not here.

The first run can now go and get its books.

0.1.0 shipped with nowhere to fetch a shelf from, so a fresh install served, said the shelf was not here and refused to download one. The shelf is published now, as its own release on this repo, and this build points at it.

shelf https://github.com/webdevtodayjason/archiver/releases/tag/shelf-v0.1.0 size 73196259 bytes sha256 85dacb04a56693295ffe23e2251f746a7250a366a475469eb1ce4fda43181e5c

5,955 documents: 5,934 Vikidia articles under CC BY-SA 3.0, and 21 survival and reference PDFs that are free to pass on. The download resumes if it breaks, is checked by length and then by hash before anything is unpacked, and is unpacked through a filter that refuses absolute paths, parent traversal and links. A bundle with no ATTRIBUTION.txt in it is refused outright.

All three shelf constants read the environment first (LAST_LIGHT_SHELF_URL, LAST_LIGHT_SHELF_SHA256, LAST_LIGHT_SHELF_BYTES), which is how a mirror gets pointed at. Nothing else changed, and the app still works with no device, no network and no corpus.

Everything in 0.1.8, plus the one thing a sweep of unmeasured model classes needs. 0.1.8 was tagged and released before this landed and never reached the farm catalog: install this one.

A missing model, and a reply this could not read

Both looked like a null in a result file, and the second costs a model load to reproduce once a sweep has moved on. Four of the nine model classes are being measured against real hardware for the first time, having only ever run against a fake built from the device's own documentation, so a response shape guessed wrong is the likely failure and a silent null is the worst way to find out about it.

A failed call now keeps the device's own words. The transport used to flatten an HTTP error to its status line, which threw away the only thing that separates the two cases: the gateway answers 503 with an envelope that says "No suitable model is currently running", and nothing else on the device uses that envelope.

So a test whose class of model is not resident returns a reason rather than nothing, and both the report and the Markdown export print "not measured because the model was not resident" instead of leaving a blank that reads as a failure. It is not a failed measurement: the box holds one accelerator and a hundred NPU units, and a class that was not loaded when the sweep reached it has nothing to report.

A reply that did come back and could not be used is captured instead: the …

Four more kinds of model can be measured, the report is rebuilt, and every run now records what makes it comparable to somebody else's box.

Every run carries its provenance

A run taken without this is a run that cannot be compared tomorrow. Each file now has a provenance block with a schema number on it.

Per run: the host that drove the benchmark (os, release, version, arch, python, logical CPUs, physical RAM, machine model), the transport (plane, gateway port, vhost if used, and the round trip measured before any work starts), the device (name, model, serial, TiinyOS, service version, NPU unit budget), and the tool (bench version, suite version, envelope schema, git commit). A benchmark driven from a Mac over a cable and one driven from a Windows box over Wi-Fi are not the same measurement, and until now nothing in the file said which.

Per model rather than per run, because a sweep loads and unloads as it goes and the conditions the fifth model met are not the ones the first met: the resident set with each model's NPU units at the moment that model's tests begin, units used and free, whether another process held the device lock, the sampling parameters actually sent, which tests ran and which were skipped, and the served context window. The lock is read and never taken, because a benchmark that fought for it would change the thing it measures.

Temperature and power are recorded as null with a reason rather than omitted. …

Four more kinds of model can be measured, and the run page will offer them.

Four new test types

Twenty-nine models are installed across nine classes and the suite had tests for five of them, so nine models wore a type with nothing behind it.

Class	Test	The figure
ASR	asr	seconds of audio transcribed per second of wall clock
Image-to-Text	ocr	seconds per page, and whether the digits came back
Music Generation	music	seconds of audio produced per second of wall clock
Text Reranking	rerank	query-document pairs scored per second

The audio clip and the page of digits are generated in code rather than committed: a fixture you cannot diff is one you cannot trust when a number moves. Both are byte for byte identical every run, and the tests decode them back and read the digits off the pixels.

OCR takes whichever of two paths answers and records which, because a box whose OCR model is a vision language model has no gateway route and has to be read through chat completions, and those two numbers are not comparable. Music handles both a blocking call and a polled job for the same reason.

Transcription accuracy is deliberately not scored: it needs a clip of known speech, which cannot be generated without a text-to-speech model loaded, and a benchmark that depends on a second model is measuring two things. Reranking does get a free sense check, because one of its four passages actually answers the query. …

The off-grid library, packaged for the Tiiny App Farm. Ask a question in plain words, get an answer built only out of the books on the device, with the book and the page printed under it. When the shelf does not cover the question it says so instead of inventing an answer.

This is the books half of the archiver repo. The notes half ships separately as tiiny-brain and is not affected by this tag.

What is in the farm build:

• ask and cite, browse the shelves, look a subject up by name
• no ingestion. Turning a PDF into text needs poppler, reaching poppler means starting another program, and the farm's archive scanner refuses a tree that can do that. The desk tool in the same repo still ingests.
• a prebuilt shelf instead: downloaded once, resumable, checked by length and then by hash before anything is unpacked, and unpacked through a filter that refuses absolute paths, parent traversal and links.
• with no shelf yet it serves and says so, and offers to fetch one.
• Python standard library only.

The shelf URL is still a placeholder. Nothing has been published to fetch, so --download-shelf refuses and says why rather than reaching for something that is not there.

python3 last-light --serve [--port N] python3 last-light --selfcheck python3 last-light --shelf

TiinyBench 0.1.6

Problem. 0.1.5 folded the sidebar to icons, but the main column was capped at 1500 pixels and left-aligned, so on a wide window the space the fold gave back sat empty on the right and the chat stayed the same width. Jason, looking at the folded chat: "the chat section stayed exactly the same width left to right."

Fix. The cap is gone. The main column now fills whatever the sidebar leaves, folded or wide, on every page. Prose keeps its own readable measure (the page blurbs and the empty-chat note are capped in characters); tables and the chat grow with the window.

Tests. The 100 offline tests pass unchanged; the chat page rendered at 2000 wide with the sidebar open runs to the right edge (grid: 236 px sidebar, main the rest).

TiinyBench 0.1.5

Problem. The Chat page is the first page in TiinyBench that wants the whole width of the window: a model card, the conversation and the loaded models side by side. The sidebar took 236 pixels of it for eleven labels the eye already knows by their glyphs. Jason, looking at a chat: "if I could collapse the left-hand nav to just icons, would I have a little bit more space?"

Fix. The sidebar folds to icons. A button in the brand row folds it to 64 pixels and back; the choice is remembered per browser and applied before the first paint, so a folded sidebar never flashes wide on load. Folded, each button carries its label as a tooltip, section headings become hairlines, and the footer keeps only the Tiiny mark. On a phone the sidebar is already a strip and the button is not offered. Nothing about the pages, the routes or the chat changes.

Tests. The 100 offline tests from 0.1.4 pass unchanged (the page's script order, the em dash rule, the chat and the catalogue); both states rendered at 1440 wide: 236 px and 64 px, the document exactly the viewport in each.

TiinyBench 0.1.4

Problem. TiinyBench could tell you what a model costs and never let you talk to one. Its sibling on the same hardware, AINode Pocket, has a chat tab that reports every reply's cost under the reply and the device beside the conversation, and asked for it here. The two apps also derived the same numbers from the same two gateway blocks in two places, which is the arrangement that drifts the first time the gateway renames a field and then has a chat bar and a saved benchmark disagreeing about the same request.

Fix. A Chat page, Pocket's three columns adapted to one Tiiny. Under every reply: time to first token, decode tok/s, total wall time, tokens in with the cache hits, tokens out, which device and model answered, and why generation stopped, all read off the gateway's own timings and usage on the stream's final chunk plus this machine's clock; the numbers are saved with the message. Left: a model card for whatever is selected (type, params, size, NPU units, input and output, where it is loaded, the vendor's description, a catalogue link) and the last twenty conversations, kept in the browser. Right: every model loaded on the box with its units and status and an Unload button, the NPU budget bar, and a Load panel that refuses in a sentence when a model is not installed, cannot chat, or would not fit, because the device accepts a load that does not fit and rolls it back without saying so. Replies render as markdown with highlighted code blocks a…

Loading notes should not have been a terminal command, and now it is not.

The empty board carries the door: "No notes yet. Point Tiiny Brain at a folder of markdown", with a button. The same button sits in the top bar and the same panel is under Settings. Give it a folder, or browse to one from this machine, and one press runs the walk, the filing, the chunking, the embedding on your Tiiny and the name index, telling you which of the five it is on. The board redraws itself when it finishes. Point it at the same folder later and it only does what changed.

The five commands still work exactly as they did. They are just no longer the only way in.

Every step that cannot run now says which step and keeps what came before it: a Tiiny nobody has set up, a Tiiny that did not answer, a Tiiny answering and serving no embedding model, a folder with no markdown. The names are built either way, so a sleeping Tiiny costs you the ability to ask and nothing else, and try again picks up where it stopped. An empty board is never left without words on it.

Three faults turned up on the way and are fixed here: a load started before the device was configured used to leave the page saying "loading" for ever; the board could stay empty after a load while the numbers under it counted the new notes; and the two routes the page polls no longer wait on the database during the slowest step. Two different vaults with an Inbox at the top now stay two notes. …

Story Lantern 0.1.3

Problem. The lantern was hard wired to tell every story with deepreinforce-ai/Ornith-1.0-35B. The farm listing only asks for a chat model, so a Tiiny holding Qwen3-8B or Qwen3.8-27B instead got 404 "Ornith-1.0-35B" is not loaded on the first planning call, and the lamp turned that into "Goodnight" and went back to the prompt with no story. Two people hit it on 2026-09-16; reproduced on Jason's unit in five seconds.

Fix. The storyteller is chosen once per story from what the Tiiny is actually holding: Ornith when it is there, then Qwen3.8, Qwen3.6, Qwen3-30B, Qwen3.5, gpt-oss, GLM, then any other chat model; coder, embedding, reranker, OCR, ASR, TTS, image and music models are never chosen. Ornith is started only when it is installed and fits the free NPU units, because the device rolls a load back silently when it does not. When no storyteller can be had, the lamp shows a calm card ("The lantern needs a storyteller. Ask a grown-up to check the Tiiny.") instead of a goodnight, and the parent view says exactly what is missing and what the device is holding. A story that fails after a page has been read still ends with the goodnight, on purpose. The parent view now shows who told each story. …

Running 0.1.2 from the launcher at about 2000px wide, the Connection page showed REACHED AT as 172.17.7.1 on a box that is at 172.17.7.177, and HOW as given, port 80 as p8800.api. with the rest gone. The tiles were drawn in the face the dashboard numbers use, at one size whether the value was three characters or twenty.

• The connection tile values now get a face sized for a short line of text, they wrap instead of running off the card, and hovering one shows the whole value.
• The tiles take the width their own value needs, so 1 TB and given, port 80 as p8800.api.tiiny are no longer made the same size.
• The discovery file on :39218 is asked once per address and fills in whatever the management API could not answer, so a box handed over by the launcher stops reading "found, unnamed" over a column of dashes.
• The tiles say what found the box, "the launcher" or "the farm", rather than naming this suite's own environment variables.

The browser job in CI now loads the page at 2000 as well as 1440 and 390, and fails if any tile value is wider than the box holding it.

A fresh install could not answer its own first two questions. /api/vitals and /api/settings both count rows in entity, and nothing on the cockpit's path created that table, so on a database nobody had built an entity index in the query raised, the handler dropped the socket, and the browser showed ERR_EMPTY_RESPONSE with an empty page behind it.

• Every table the app owns is now created on every connection, by one function the three modules share. A fresh database and one left half built by 0.1.1 both come out complete.
• A route that raises answers 500 with the reason instead of closing the connection in silence. The traceback still goes to farm.log.
• The corpus moves to the data directory the farm keeps beside the version folders, so an index survives an update. ARCHIVER_HOME still wins and a plain checkout is unchanged. Your notes were never involved: they are markdown files in your own folder.
• /favicon.ico answers 204 and the page carries its own mark.
• First CI: the tests and the selfcheck run on Linux, macOS and Windows.

Two testers reported the same thing in different words, and it was one cause: the page never ran.

The confirm dialog added in 0.1.1 sat above the script's own scope and called $ on its first line, so the whole <script> stopped on a ReferenceError before it asked for anything. Nothing below it executed. No navigation, no request for the device, no find button. The app came up looking finished with every control dead, on Windows and on a Mac alike.

Fixed in this release

• The page runs. Connection opens, the find button finds, the dashboard fills in.
• Discovery works on Windows. It used to shell out to ip and then ifconfig, and Windows has neither, so the candidate list was empty and a Tiiny on the same network could not be found. It is stdlib now and the same on every platform.
• The scan asks the responder the farm CLI asks, a GADGET_DISCOVER_V1 datagram on port 39217, which is the only probe that finds a box whose address moved.
• The Connection panel says what it is doing. It paints "Looking for your Tiiny" before the first request leaves, says so plainly when nothing answers, and offers the address box and the key box either way.
• The dashboard no longer dies on a fresh install. Asking for the leaderboard without a key was dropping the connection.
• tiiny-bench --where prints what it looked for and exits cleanly when there is no Tiiny attached.
• At 390 pixels wide the navigation used to push every panel off the right edge behind a sideways scroll. …