Misc

How to Evaluate Domestic Chips Running Domestic Models: What to Check

Using Meituan's domestic-trained LongCat-2.5, evaluate domestic-chip-domestic-model claims by scale, end-to-end coverage, performance and ecosystem, not just params.

One-sentence summary

Using Meituan's fully domestic-trained LongCat-2.5, evaluate domestic-chip-domestic-model claims by training scale, end-to-end domestic coverage, performance and ecosystem, not just params.

Why evaluate separately ​

LLM training demands extreme compute-stack stability, interconnect and software tooling; whether domestic compute can run trillion-parameter models is its own question, not just a benchmark score.

Steps ​

  1. Check training scale: LongCat-2.5 at 1.6T tests domestic compute throughput and stability.
  2. Check end-to-end coverage: training and inference fully on domestic compute matters more than partial use.
  3. Check real performance: usability on long context, multimodal and agent tasks.
  4. Check ecosystem: whether compilers, scheduling and fault-tolerance tooling are ready.

Common Mistake ​

Stopping at it ran on domestic chips, ignoring end-to-end coverage and real performance, overstates or understates domestic maturity.

Sources ​