Perform
Your webcam becomes the lamp and screen. Bare hands drive the puppets — no controllers, no wearables — exactly the way a dalang's hands have always worked.
Our Focus
Preservation through technology
Physical puppets crack, masters pass away, and all-night performances are hard to stage for a new generation. Wayang.AI preserves wayang on three fronts: the craft — every puppet scanned, rigged and archived as a performable digital character; the performance — the dalang's gestures captured live by computer vision; and the knowledge — characters, stories and traditions documented in an open, community-built directory.
Rig
Upload any puppet image and the studio cuts it out and rigs it automatically, so fragile heirloom characters can take the stage again without being touched.
Document
A growing directory of characters, stories, wayang types and events — written and reviewed by the community, modelled on the national wayang data centre.
Computer Vision
Your hands are the rods
A plain webcam watches your hands. The vision pipeline finds 21 landmarks per hand — fingertips, knuckles, palm — and maps them onto the virtual stage sixty times a second. The middle fingertip carries the puppet's main rod (gapit); thumb and little finger steer the two arm rods (tuding); a closed fist plants the puppet in the digital banana trunk (debog).
- TrackingMediaPipe Hands, up to 4 hands / 4 puppets simultaneously
- SmoothingAdaptive 1€ filter — still in dialogue, instant in battle (sabetan)
- GesturesFist held 250 ms plants & retrieves puppets from the debog
- Latency≈0.13 ms per full engine cycle — over 99% of the frame budget stays free
- PrivacyVideo is processed locally in your browser and never uploaded
Automatic Rigging
From a photo to a living puppet
The built-in Studio turns a flat image into a performable character in four steps — the same pipeline our preservation team uses to archive heirloom puppets.
Cut out
Upload a puppet photo or drawing. Flood-fill background removal separates the character cleanly, even against busy backdrops.
Detect parts
Body and both arms are segmented so each limb can move on its own rods, with joint points placed along the shoulders and elbows.
Rig the joints
Adjust proportions, rod reach and joint coordinates on a live preview. Choose the character's kind once — it is locked forever, just like a real puppet's construction.
Save & perform
The finished kit is saved to your library, ready to be planted on stage, dressed with accessories, and driven by your hands.
On Stage
Features of the performance app
Swappable backgrounds
Change the kelir scenery in one click — palaces, forests, battlefields — or upload your own backdrop and save it to your library.
Dalang voice
Presets reshape your voice live — natural, deep elder, high and fine, or the booming voice of a giant — through pitch and formant shifting.
Accessories & weapons
Keris daggers, maces, crowns and shields anchor to a puppet's hands, head or body, in front of or behind the limbs — and follow every movement.
Monolithic puppets
Some wayang cannot be articulated: the gunungan, war banners, heirlooms and chariots are single rigid pieces. They glide as one body, steered by your palm — their kind is fixed at creation, permanently.
The digital debog
Clench your fist to pin a puppet into the banana trunk, freeing your hand for another character. Planted puppets are protected from reassignment until you retrieve them.
Gamelan & cempala
Slendro gamelan scenes — Ayak-ayak, Srepeg, Sampak — crossfade with the drama, while the dalang's dhodhog and kepyak percussion is synthesised live, never sampled.
The Directory
Characters & stories, documented
Modelled on the national wayang data centre (PDWI – Senawangi), our directory documents the repertoire for a global audience — loading the catalogue… Anyone with an approved account can contribute; every entry is reviewed by an admin before it is published.
Events
Training & performances
Loading events…