Hugging Face published an open-source pipeline that trains a coding model to produce watercolour paintings using reinforcement learning. The project reproduces an experiment by designer Surya Narreddi, who published a viral demonstration on August 23 showing a model generating botanical paintings with JavaScript.
The policy model outputs roughly 150 lines of JavaScript that execute against p5.brush, a canvas library created by developer @acamposuribe. The library simulates physical pigment bleeding across paper fibres, fluid mass, and flow fields. To preserve the aesthetic style, the system prompt restricts the language model to ten specific methods out of the 47 exposed by the library.
Restricting the drawing canvas
The ten permitted methods include beginShape, vertex, endShape, fill, noFill, fillBleed, fillTexture, circle, noStroke, and scaleBrushes. Restricting the API prevents the model from attempting hatching, lines, or custom brushes that alter the watercolour texture. Hugging Face also added an instruction prompting the model to paint petals two or three times, laying down a broad wash first and a smaller, more opaque shape inside it.
Headless Chromium renders each generated sketch on a central processing unit Space before aesthetic scoring occurs. The gate verifies that the script compiles, uses library methods rather than standard p5 calls, and covers the canvas with pigment. Renders that time out or fail return a null value rather than a zero, preventing failed executions from skewing advantage scores inside training groups.
Aesthetic rewards and reference pools
Training runs through Group Relative Policy Optimization using Hugging Face's TRL framework. The reward structure balances compile validation, the 7-billion-parameter HPSv3 aesthetic preference model, and a pairwise vision judge powered by Qwen3-VL-30B-A3B-Instruct. The pairwise judge ranks candidate rollouts against a reference pool of 178 hand-rated hibiscus sketches.
Four open-weight models generated the reference pool from iNaturalist photographs: GLM-5.2 produced 64 qualifying sketches, Kimi-K3 produced 57, Qwen3-Coder-Next produced 35, and Qwen3.5-122B-A10B produced 22. A human rater split the pool into two tiers, labeled "love" and "okay." During training, the pairwise judge compares candidates against two references drawn from each tier.
Hardware and model tuning
Target model Qwen3.5-35B-A3B trained on a single Nvidia H200 GPU. Standard low-rank adaptation settings initially trained only ten of forty layers because the mixture-of-experts model uses non-standard projection names. Hugging Face switched the adapter configuration to target all linear layers, enabling the policy to train despite leaving routed expert tensors frozen.
Software rendering accounted for 70 to 80 percent of execution time, with individual renders taking between 69 and 96 seconds. Training concluded at step 110 after roughly 34 hours of compute. Hugging Face also submitted an upstream patch to OpenEnv after discovering that cached, broken websocket connections caused consecutive training runs to fail.
