One network
One network for the painting and the whole thread
A real network, built here and trained here: the painting's patches, the depth the paint suggests and every letter of this conversation, in one shared space. Nothing is pre-trained.
The painting, patch by patch
The network looks at the painting in small patches, each with its own light, colour and place, and learns those words. The same weights read the letters of the thread.
- Steps trained
- 0
- Picture error
- —
- Letter error
- —
- Depth error
- —
- Patches that find their words
- —
- Patches of paint
- 0
- Words in its space
- 0
- Letters learned
- 0
- Characters known
- 0
The words it learns are the ones the paint can teach: how bright a patch is, which colour it is, where it sits. The text job learns the order of letters, not meaning. Nothing here is pre-trained; the gradients run in this tab.
Training, live
It starts on its own when the page opens and learns right here, in this tab. Every hundred steps it writes down what it measured.
Waiting for the network to be built…
The whole app, in Python and JAX
This page and the program below are one network. The program is the whole application: a GTK4 window, JAX gradients, the painting, the thread, the depth it predicts and the shared space. Copy it or save it, then run it on your own machine.
- Patches of paint
- —
- Words it learns
- —
- Letters it reads
- —
- Files to keep together
- net_gtk.py
What the window does
- Train 300 or 2 000 steps, or stop halfway; the learning runs in a thread so the window stays alive.
- Click any patch of the painting and it tells you the words it has for that patch.
- Ask it in words — warm gold, dark blue — and it lights the patches the question lands on.
- Give it a few letters and it lists the letters it expects next, with percentages.
- Paste your own writing and press Learn this instead: new letters get fresh rows in the same table, nothing it knew is thrown away.
- Two drawings under the painting: the depth it predicts, and the one space where dots are paint and squares are words.
- Save its weights to net_weights.json and load them again next time.
How to run it
One folder, one file, and a painting called painting.jpg if you have one. On Linux, in that folder:
-
Install Python packages
python3 -m pip install jax numpy pillow -
Install GTK4 and its Python bindings
sudo apt install python3-gi gir1.2-gtk-4.0 -
Open the window
python3 net_gtk.py
On Windows the short way is MSYS2: pacman -S mingw-w64-x86_64-python-gobject mingw-w64-x86_64-gtk4, then python -m pip install jax numpy pillow, then python net_gtk.py in that same MSYS2 terminal.
On macOS: brew install pygobject3 gtk4, then python3 -m pip install jax numpy pillow, then python3 net_gtk.py.
No display? python3 net_gtk.py --selftest trains in the terminal and prints the same numbers.
Measured by running this file in a terminal with JAX, on this painting and this thread: 400 steps → 67 % of patches find their own words (picture error 0.68), 2 000 steps → 95 % (0.53), 5 000 steps → 97 % (0.45). The depth error ends at 0.0003 and the letter error falls from 3.05 to 2.90.
A browser cannot run Python, so this tab shows the file rather than the window — the same network is running live above. The words it learns are the ones the paint can teach: this painting holds no blue, so a question about blue lands on gold patches instead. The letter head knows the order of letters, not meaning.
A compiler for the Python and JAX above
The box holds the forward pass of the one network. The compiler reads it, checks it, and writes it out twice: Python for JAX, and JavaScript that runs in this tab. One tree, two back ends.
- Tokens
- —
- Nodes in the tree
- —
- Functions
- —
What it computes
The check compares these numbers with the same program run in numpy and JAX on the same seeded weights — measured before this card existed, not guessed.
What the language allows
- def name(a, b): with an indented body, name = ..., and return ...
- Numbers, names, brackets, unary minus, and + - * / @ for a matrix product.
- .T for a transpose, and the calls tanh, sqrt, exp, log, sum, mean, maximum.
- Anything else stops the compiler with the line and the column instead of guessing.
This is a real compiler for a small language, not a Python runtime: no loops, no classes, no imports. The one network's forward pass fits inside it, which is why the same text can become JAX on your machine and JavaScript in this tab.
Virtual eyes: it only sees pixels
An eye pointed at a monitor: it gets the picture and nothing else — no page, no data, no network. From those pixels alone it finds edges, corners and how each point slides as the eye steps sideways, and from that it rebuilds where everything stands in space.
- Frames seen
- 0
- Corners
- 0
- Tracked
- 0
- Median motion
- —
- Depth found
- —
- Points in space
- 0
- One frame
- —
The depth is real geometry, but only up to the size of the eye's step, and only where the picture has texture — flat paint gives nothing to hold on to. Seeing where a wall stands is not knowing it is a wall, and nothing here invents a world: the room above is the pixels, measured.
Your idea, and what it would still need
- Eyes on the monitor are a real interface: a model that reads the screen the way a person does needs no plugin for each program — anything visible is already input.
- From two frames of a moving eye you get geometry, which is exactly what a world model has to agree with; that part this card does for real.
- A generated world in real time also needs a model that invents what nobody saw, keeps it consistent as you turn, and runs in milliseconds — that is not in this tab, and a page cannot train it.
- The honest limit: this eye recovers where things are, not what they mean. Speed of seeing is not understanding, and one is much easier than the other.
Image: public-domain scan of Leonardo da Vinci's Last Supper, Wikimedia Commons.