Update on Plagiarism Allegations Against Domestic AI Model

AI Translated Image

Upstage's Solar Open 100B model, which recently gained a lot of attention for its impressive performance in a domestic AI development project, was released on Hugging Face. However...

AI Translated Image

Sionic AI, another domestic AI firm, has raised allegations that the model is plagiarized. They claim it's actually just China's Z.ai open model, GLM-4.5-Air, with some additional fine-tuning. Here is their evidence: 1. Similarity in Model Weights In an AI model, weights are essentially the brain's information itself. The data accumulated during AI training is stored in the form of weights. However, analysis results have been released showing that the weights of Upstage's Solar Open model and GLM-4.5-Air are almost identical.

AI Translated Image

This graph compares the similarity between different layers within the GLM and Solar models versus the similarity between the same layers in both models. Generally, if they were completely different models, you wouldn't expect high similarity between the same layers. Yet, Solar and GLM showed nearly 99% cosine similarity in identical layers. In human terms, it's like having almost the exact same information stored in neurons located in the same parts of the brain. 2. Model Architecture Similarity Mixture-of-Experts (MoE), a widely used architecture nowadays, essentially means there are multiple models contained within one. Just as the brain is divided into regions for hearing, vision, and higher reasoning, an MoE model uses different internal 'experts' for fields like coding, math, conversation, or search to boost efficiency. (Note to angry experts: I've sacrificed some accuracy for the sake of a simple explanation. In reality, experts are assigned per token, which is a different concept from being split by field.) Details like the total number of internal models and how many are used at once often vary between models. However, the Solar model and the GLM model have the exact same total number of internal models and the same number of models used simultaneously.

AI Translated Image

This similarity in internal architecture suggests that the Solar model may have been derived from the GLM model. 3. Speculation on the Training Process

AI Translated Image

The company raising these doubts speculates that the training was done in this manner: They preserved the weights representing the base intelligence of the model and only retrained other components like the attention layers or the MoE router. 4. Significance The company raised concerns based on architecture and weights. In fact, referencing an existing model's architecture when developing a new one isn't really a huge deal. In the open-source ecosystem, 'yoinking' architecture ideas happens all the time and is even encouraged for the sake of AI advancement. However, the core issue is the weights. Using an existing model's weights as a starting point for further training is called fine-tuning. This is also a very common technique and isn't problematic in itself. But Upstage publicly committed to developing a 'Foundation Model' from scratch. Given that fine-tuning is inherently limited in its training scale, it doesn't fit a project aimed at securing national foundation model capabilities. This part needs to be clearly explained. 5. Upstage's Response Upstage has acknowledged these concerns and stated they will undergo a verification process of the model's training.

AI Translated Image

It looks like we'll have to wait and see the results.