Vidu S2: real-time avatars and video editing

What changes when video becomes interactive?

A conventional video API accepts a prompt, creates a task and eventually returns a clip. The Vidu S2 product page describes a different workflow: an ongoing video stream reacts to new input. An avatar can respond to speech, perform a motion or use a new reference image while the interaction continues. Editing changes a live source rather than repeatedly submitting unrelated short clips.

The official update log lists S2-Avatar and S2-Editing on September 15, 2026. The technical report describes 720p real-time avatars and explores spatial video for VR. Treat the VR work as research; a paper demonstration does not establish an available consumer product. We have not measured latency, frame rate or comparative quality.

Choose Avatar or Editing

The model map also lists an asynchronous Avatar workflow returning an MP4. Its existence is not proof that the offline endpoint accepts the same S2 model value as component mode. Pick a mode before copying an example: responsibility for voice and transport changes between them.

For Avatar, start with a clear portrait and one short movement instruction. Add a clothing or background reference after the initial stream is stable. For Editing, choose a camera, video or image source and replace one element at a time. A useful first comparison is the same source with two backgrounds; then inspect face consistency, object edges and motion between updates.

Try the official client first

1. Open the Vidu Stream developer documentation and its Quick Start. Obtain your own API key and confirm the account has real-time access and credits.
2. Use the official Python quickstart and accompanying browser client. The tutorial requires Python 3.10 or later and serves the client on localhost port 28890.
3. Match the endpoint to the key's environment: the tutorial defaults to api.vidu.cn; global access uses api.vidu.com. Do not assume a key works across both.
4. Choose the documented mode, create the session and connect the client. Grant microphone or camera permission only for the input you intend to send.
5. Try one short interaction, use the client's Hang Up action, and check that the provider session has ended. Inspect credit usage before another run.

Keep the API key on the server. The browser needs the session's connection information, not the permanent API credential. A successful session-creation response alone does not mean video is playing: confirm live state, connection acknowledgement and media joining in the official client.

API implementation: a session lifecycle

The component creation documentation specifies model: "vidu-s2" for S2, currently marked beta. The catalogue names vidu-s2-avatar and vidu-s2-editing are not interchangeable request values. The default in the component example remains vidu-s1; omitting the model can select the older generation.

For an application that owns its RTC channel, this is the documented component request shape. Replace every placeholder, use an image you are authorized to animate, and send the request from your backend with Authorization: Token $VIDU_API_KEY and JSON content type. The endpoint is POST https://api.vidu.cn/live/s_avatar/component, or the matching global host for a global key. This creates a billable session; the illustration has not been executed.

{
  "model": "vidu-s2",
  "image_uri": "https://example.com/avatar.jpg",
  "rtc_info": {
    "provider": "agora",
    "app_id": "your-rtc-app-id",
    "channel_id": "your-channel-id",
    "user_id": "uid-123",
    "token": "your-rtc-token"
  }
}

The response supplies live.id and a session-scoped client_secret. Follow the component WebSocket initialization and heartbeat protocol, then supply 24 kHz mono, signed 16-bit little-endian PCM audio. Video output joins your RTC channel. These transport details are specific to component mode; the full real-time Avatar control socket does not take binary media input.

Implement the documented sequence for your chosen mode: create a session, wait for readiness, establish its media connection, send audio or reference updates, and explicitly stop it. Add a maximum run time, a disconnect handler and a cleanup path for failed connections. Persist the session ID on your backend so a browser refresh does not lose the ability to stop a billable session.

Component mode is suitable when an application already owns the conversation pipeline. For example, your backend generates the reply text, your TTS produces audio, and S2 animates the character. Full real-time Avatar mode instead uses Vidu's conversational pipeline. An agent tool can coordinate a session, but it does not replace the browser media client.

Credits and session duration

These rates come from the real-time Avatar, component and Editing API documentation. A 60-second billable real-time Avatar session calculates to 90 credits; a 60-second component or Editing session calculates to 60 credits. For Avatar modes, the 45-credit threshold is an account-balance requirement, not a documented minimum session charge. Real-time Avatar billing begins after the documented live/connection acknowledgement condition; do not estimate cost solely from the seconds visible on screen.

The real-time documentation caps one session at 7,200 seconds. Set a much shorter application limit for a first test. Resource packages, regional storefront prices and concurrency allowances can differ; read your account's current offer instead of treating credits as a fixed dollar exchange rate. The hosting subscription and Vidu usage are separate.

Self-hosted Hermes Agent: connect your own bridge

Hermes can discover tools from a local MCP server using the upstream MCP configuration. The following is a configuration template for a bridge you have already implemented and tested. /srv/vidu-s2-bridge/server.py is your application path, not an official Vidu package supplied by this guide.

# ~/.hermes/config.yaml
mcp_servers:
  vidu_s2:
    command: python3
    args: ["/srv/vidu-s2-bridge/server.py"]

Start a new hermes chat session and confirm the bridge's tools appear. Keep key handling, billing limits and session ownership inside the bridge. Expose only the operations the agent needs, such as checking state and stopping its own session, and require explicit approval before starting paid streaming. Render the stream in your application's client.

Self-hosted OpenClaw: register the same bridge

Recent upstream OpenClaw documents a built-in MCP client. Verify your installed version supports these CLI commands, then register your working stdio bridge:

openclaw mcp add vidu-s2 \
  --command python3 \
  --arg /srv/vidu-s2-bridge/server.py
openclaw mcp doctor vidu-s2 --probe

Doctor should connect and list the tools your bridge actually advertises. A successful probe verifies MCP connectivity; it does not test Vidu authorization, streaming quality or billing cleanup. Validate those separately with one bounded session. Do not add guessed S2 entries to the chat-model provider configuration.

Managed hosting: what is available today?

Both hosted frameworks already share the Vidu MCP connection for ordinary short-video tasks. Its adapter currently supports viduq3-turbo, viduq3-pro, viduq2 and viduq1, with task creation and result queries. It does not implement S2's streaming session lifecycle.

For S2 today, use the official client or your separately deployed application. Selecting a chat model or entering a key in the existing Vidu connection does not add a real-time avatar interface. Custom self-hosted MCP configuration above describes a development route; it is not a managed S2 setup switch.

Sources checked October 4, 2026: the official Vidu product page, model map, update log, Stream documentation and technical report, plus both upstream MCP references linked above. No paid video session or independent performance benchmark was run for this guide.

Vidu S2 FAQ

Related guides

• Existing Vidu MCP short-video integration
• fal tools on Hermes Agent
• fal tools on OpenClaw
• 中文：Vidu S2 实时视频指南