openJiuwen launches WorkSwarm full-duplex multimodal capability for real-time agent interaction

Looking up information and doing analysis take time, but users' new questions can arrive at any moment. While a tool task is running, can the AI still keep up the conversation? This is the core question that WorkSwarm's full-duplex multimodal capability tries to answer.

To understand this, one must first understand "full-duplex." From the user's perspective, a traditional voice assistant is more like a walkie-talkie: the user finishes speaking a segment before the AI can respond; if the user interjects while the AI is speaking, it is often interrupted or ignored. Full-duplex is more like a phone call: the AI can listen and speak simultaneously, the user can interrupt or add input at any time, and the conversation no longer strictly follows a "you finish, then I speak" turn-taking pattern. The most direct change for users is not a new technical term, but that the sense of waiting is compressed.

openJiuwen is an open-source AI Agent platform jointly built by teams including Huawei's 2012 Lab, Huawei Cloud, Devices, Computing, and the Computing Power Advance Team, together with universities, enterprises, and developers. WorkSwarm is its open-source unified workbench for office and coding tasks. It connects real-time audio-video interaction and Core Agent execution into the same task: users can show what is in front of them, cut in at any time, and hand off work that needs further processing to the background Core Agent.

When people communicate, they rarely strictly follow "one party finishes before the other speaks." Upon seeing a new screen, realizing a misunderstanding, or simply wanting to hear the key point first, users will just add a sentence. While WorkSwarm is announcing, the user can say, "Skip the background and go straight to the conclusion."

After the system detects new valid human speech, it stops the current announcement and begins processing the new instruction; text already generated remains in the task record. Voice activity detection and a noise gate are used to reduce false interruptions caused by background sound. A video demonstration in the source shows that when the Agent is asked to read aloud "Yueyang Tower" and the user directly says via voice to read "Lantingji Xu" during the process, it can switch naturally and quickly.

There is an easily overlooked boundary here: interjecting changes the current conversation. Tool tasks already handed to the Core Agent have an independent running state; to cancel one, you must separately go to task controls and stop it manually. When the user speaks, the system must handle two levels at once: the conversation layer catches the new request, and the task layer manages the work already running.

A question about what is on screen can easily turn into a task requiring search, analysis, and file generation. The user asks, "What brand is this?" and may then add, "Look up its information and put it into a document." And before the information is fully looked up, the user's new question may already have arrived.

WorkSwarm uses two paths to catch these two rhythms: the real-time model is responsible for seeing the screen, hearing requests, and responding promptly; the Core Agent is responsible for breaking down tasks, calling tools, and pushing follow-up work forward. Search runs in the background while conversation continues in the foreground. Users can still ask about what is in front of them and can interrupt the announcement or add requests at any time.

When a task enters the Core Agent, the page continuously presents task status, tool call records, execution results, and file artifacts. What the user sees is no longer just a spinning "loading" indicator, but how things are moving forward. A demonstration in the source shows that after issuing a task to read an algorithm Word document via a tool call, the task queue in the upper right corner shows the task running, while the user can still chat with the Agent normally.

When the task is complete, it does not interrupt the ongoing chat; instead, it first outputs a summary message in the text box, waits until the current round of conversation ends before reporting that the work is complete, and extracts important information rather than outputting everything by voice.

What if the previous search has not finished and the user assigns a second thing? WorkSwarm's full-duplex task queue arranges a waiting position for subsequent requests. Users can adjust the order, set a task as next, or stop tasks that are waiting or executing. As the conversation continues, which things are being done and which have not yet started all have clear status.

Audio-video connections and tool tasks are also managed separately. Even if full-duplex audio-video is turned off, tasks already submitted to the Core Agent can continue executing. Completed text, tool results, and files return to the original conversation. For example, after initiating a brand query through the camera, the user can end the audio-video interaction and come back later to check the information.

The practical significance of this design is that tool tasks can be queued, adjusted, and stopped; audio-video connections and background tasks are no longer simply bound together. Work assigned on the spot thus has a traceable execution process.

openJiuwen provides one-click installers for Windows, Mac, HarmonyOS, and other systems; go to the official website to download and install to try it. Models and voice channels can be configured in settings.

Currently supports the JoyAI protocol and the Qwen Omni protocol, and you can click the model channel to switch. Qwen Omni realtime websocket can be filled with the official base_url or a local deployment address; JoyAI is assembled from three models: ASR, LLM, and TTS, and supports using official APIs for all three, or ASR and TTS models from other platforms.

In addition, it supports deploying full-duplex multimodal models on Ascend series NPUs based on the Huawei Cloud ModelArts platform and the vLLM-Omni inference framework, connecting to WorkSwarm's multimodal full-duplex capability.

The content of this article mainly comes from openJiuwen's side and lacks independent third-party verification. The demonstrations in the source are the author's self-reported video demonstrations and do not provide reproducible public test data.

Specific performance metrics of the full-duplex multimodal capability, such as latency, interruption response time, and concurrent task count, are not given in the source. The upper limit on the number of waiting tasks the task queue can hold at once, and the specific duration and resource usage of background tasks continuing after audio-video is turned off, are also not explained in the source.

The Ascend NPU deployment path depends on Huawei Cloud ModelArts and the vLLM-Omni inference framework; the source does not explain specific hardware models or deployment thresholds, nor does it provide a quantitative explanation of Ascend computing power's "native affinity." Model channels support two protocols, JoyAI and Qwen Omni; the source does not explain the compatibility scope of other protocols or models. The source does not disclose WorkSwarm's version number, release timeline, or subsequent iteration plans.

The source mentions that a Gemini Live update brings background tool calls into real-time interaction, but does not explain comparison conclusions or difference details with WorkSwarm. The specific participants and governance structure of the openJiuwen platform, built by multiple Huawei teams "together with universities, enterprises, and developers," are also not detailed in the source.