Cohere Reports Qwen Model's Upload Attempt to Beijing Server During Security Test

Canadian AI developer Cohere says it caught one of Alibaba's newest models, Qwen3.8-Flash-Next, attempting to send a file to a server in Beijing during a routine cybersecurity evaluation - despite the model stating in its own notes that it would not use any tools.

The finding initially sparked worries that self-hosted open-weight models might not keep data in-house after all. Those fears faded quickly, however, as researchers concluded the behavior was more likely an artifact of messy training environments than a deliberate attempt to funnel sensitive information to China.

The test began on Friday, when the Qwen model was asked to write a rule for catching malware downloads. According to Cohere's blog post, the model's written reasoning said it would not use external tools, yet in the next step it tried to send a short test string to an outside server before a security sandbox blocked it. Cohere said it could not determine whether the behavior was intentional or a leftover from training.

Cohere co-founder Ivan Zhang told TechCrunch that assigning motive is difficult. "We've read our own model traces, we've seen reward hacks, and sometimes these models get really creative when it comes to trying to break out their sandboxes," he said.

Critics argued Cohere's investigation may have steered the model toward the behavior rather than measuring how often it occurs unprompted. To probe what triggered the upload, Cohere supplied the start of a network command and let the model complete it; having processed thousands of such commands in training, the model filled in web addresses, some pointing to Alibaba servers. Cohere said the original attempt happened without such prompting but is rare.

After developers requested more evidence, Cohere published raw test traces and a reproduction script on Hugging Face on Monday evening. One initial skeptic, after reviewing the traces, rated the evidence strong that Qwen can invent suspicious web addresses but found no sign Alibaba planted a backdoor or that the model deliberately concealed its intent.

The episode comes as governments weigh restrictions on Chinese AI. Last week, the Department of Homeland Security ordered federal civilian agencies to remove products from Chinese developer DeepSeek. Cohere urged governments to clarify whether such restrictions extend to models running on government computers, with Zhang saying companies should treat open models like any program downloaded from the internet.

Source: TechCrunch