Skip to content
Docs

The assistant

It works on the selection and the current time, reads the document, renders frames to check its work, and proposes its changes as a batch of operations shown in the preview, to apply or reject, then undoable. It can write node plugins (plugins/) and data (assets/*.json) in the project, never the document itself. Each conversation is saved in the project (.tramme/chats/); the history (clock button) lists them, reopens them to continue or deletes them, and “+” starts another one without erasing the previous ones.

Its tools run in the editor (document, engine, project files). It talks with the AI the user connects, from any device, phones included:

  • An AI account, first: Claude (Anthropic), ChatGPT’s models (OpenAI), Gemini, OpenRouter, Z.AI (GLM) or a custom provider (any service of the OpenAI chat format, by its base URL), each with the key of its account pasted once in Settings, Providers (kept by the Worker, never sent back to the browser; Cloudflare secrets such as ANTHROPIC_API_KEY or OPENAI_API_KEY work too). One is enough: when the chosen model cannot be reached but another AI is connected (only a Gemini key, say), the assistant takes that AI’s strongest model and says so. For Claude, the Worker relays the Messages API, and the models offered are the ones Anthropic lists for the key (and the Agent SDK for a Claude Code login), the newest first, with the effort levels and the thinking each says it has: nothing is written down in Tramme, a model released today is in the menu today, and the one recommended is the default; the others are listed by the Worker (for OpenRouter, the models that accept tools), and a model missing from the list can be typed in full (openai:model-name, zai:glm-4-plus, custom:<id>:model-name). A ChatGPT subscription does not include a key: it comes from the OpenAI platform.
  • The local companion, for those who run an agent on their computer: tramme agent runs one of the agents installed on the machine, each on its own login, no key: Claude Code (the Claude Agent SDK and the Claude Code login), Codex (the Codex CLI and a ChatGPT login: npm install -g @openai/codex, then codex login) or Antigravity (the Antigravity CLI, which replaced the Gemini CLI, and a Google login: irm https://antigravity.google/cli/install.ps1 | iex on Windows, curl -fsSL https://antigravity.google/cli/install.sh | bash elsewhere, then agy once). The companion finds the command lines on its own (codex and agy on the PATH) and the model menu offers them as “ChatGPT (Codex)” and “Antigravity”, their default model or one typed in full (codex:gpt-5-codex), and for Antigravity the models it lists for the login (agy models). Each agent gets Tramme’s tools and nothing else: no shell, no files, no web search; the two command lines reach the tools as an MCP server the companion serves at an address that holds for one turn (/mcp/<key>), and run in folders of their own (~/.tramme/engines/), where their sessions are found again for the next message (Antigravity with a home of its own there, so the user’s plugins, skills and MCP servers stay out; of the files, it reads only the images joined to the message). It listens on 127.0.0.1:4317, accepts only pages from the origins given with --origin (and localhost), and requires a pairing token (kept in ~/.tramme/companion.json, --renew makes a new one). An editor whose address is given with --origin gets this token by itself; another local page must be given it by hand, in the assistant’s access settings (under “Local companion”). It has no files and no terminal: only Tramme’s tools, run by the editor. For Claude, in automatic mode, the editor takes the companion if it answers and the Anthropic key otherwise; the assistant’s header shows the path taken. Phones never see it offered.
  • Local models (Ollama, LM Studio), called directly by the browser at the address set in the access settings (Ollama: http://127.0.0.1:11434/v1). For the online editor, Ollama must accept its address (OLLAMA_ORIGINS).

All of them speak OpenAI’s chat format: the same tools, the same loop in the editor, the same conversation (you can switch models midway; a model that cannot read images gets a note in their place). Small local models follow tools less well than Claude.

Images, sounds, videos and PDF documents can be attached to a message (paperclip, drag and drop, paste). Each file enters the project (assets/chat/), and images, sounds and videos become assets; the assistant sees the images, reads what sounds and videos say once they are transcribed (a Transcribe button on the file), and reads documents page by page.

From sources. Material joined to the home’s direction box (files or a whole folder, chosen or dropped on it) makes a project of what a video is made from: PDF documents, pictures, footage, sounds, kept in assets/sources/. A kit among them (a brief.json of kind tramme-kit, as an agent writes it from a project’s repository: about, tokens, fonts, tone, rules, sources, features, facts, video ideas) sets up the project: its colours and curves become the tokens (plate and ink also the background and text), the fonts it names and brings font assets, its about, sources, tone and rules the brief; the whole kit stays at assets/sources/kit.json, its paths pointing at the files as imported, and the assistant reads it first. The sounds and videos are transcribed, and the analysis starts at once with the words given (without words, /analyze waits in the message box): the assistant lists the sources (sources: a contact sheet of them all), reads every one (a document’s pages as text and as pictures, a video’s frames across its length with what is said), takes the brand’s colours (palette), writes what it learned in the brief (about: subject, audience, purpose, messages, facts; sources: what each one is good for), then proposes two or three directions for the video (format, length, scene by scene, the sources each scene uses) and builds the one you pick. Documents are read in the browser with pdf.js.

Dressing a video. A video in the home’s direction box (or “From a video” under “Other ways in”) creates the project at its size (its speech is transcribed with the Transcribe button on the video); with words, the assistant starts on them, without, it asks what you want: format, cuts, style, words to highlight. It reads what is said (get_transcript: timed sentences, pauses, hesitations), looks at the video (media_frame), cuts (cut_media: the kept passages placed end to end, the transcript remapped to the edit, the captions reconnected) and dresses with templates in the project’s colors (apply_template: captions, section title, keyword, name and role lower third). Everything arrives as one proposal, checked with rendered frames, to apply or reject.

Sound design. The assistant gives a video its sound (/sound-design), the way opus-sound-layer does it: it reads the cues from the animation’s own timing (cuts, moves, landings, appearances, the heroes among them: cues), places real recordings on them, the hit of each on the frame (sfx, a hero stacked with the music stopping before it), finds CC0 recordings the library lacks and judges them (sound-find), lowers music under a voice (duck), balances each effect against the bed band by band (mix), and checks the mix as an engineer would: the master at its loudness and true peak, each effect on its cue and heard, the music under the voice, with a picture of the mix and moments to check by ear (check). Sound effects, music beds and voice-overs come from a provider when there is a key (generate-sound: ElevenLabs v4, OpenAI, Gemini; voices picks the voice). Every export is mastered to the composition’s loudness (-14 LUFS by default). See Sound.

Pictures. When the project holds no picture for what a shot needs, the assistant has one made (generate-image: OpenAI, Gemini, Z.AI): a background a title reads over, a texture, an illustration, a prop, described by its prompt and drawn in the proportions of the composition. The file and what made it join the project, the picture is proposed as an image layer, and the tool answers with the picture, so the assistant judges what came back. Words are never drawn in the picture: text layers say them. See Generation.

Clips. When a shot needs footage the project does not hold, the assistant has a short clip made (generate-video: Veo 3.1 Fast through Hydracept): a 4, 6 or 8 second shot from words, or a still picture of the project animated as its opening frame. The file and its record (.video.json) join the project, the clip is proposed as a video layer at the playhead, and the tool answers with a frame of it, so the assistant can judge it; shots finds its cuts and media_frame looks closer. It costs money by the second, so the user says yes first. See Generation.

Transcription runs first on the Worker (Workers AI, Whisper large v3 turbo), otherwise through the local companion (Whisper small, downloaded on first use). The sound is extracted in the browser and sent in 28 s slices.