Reporter |
Editor | NUPIAO
Over the past two years, the AI compute industry has gone through a pretty obvious shift in focus.
At first, everyone was scrambling for GPUs. As large models took off, compute became the precondition for actually shipping AI applications. Local governments and enterprises rushed to build data centers and buy GPUs — at one point, the industry consensus was basically “if you’ve got cards, you’ve got compute.”
But once companies actually started deploying large models into their own business environments, the question changed from “do we have GPUs?” to “can these GPUs actually run?”
And in between those two questions sits a whole mess of complex software engineering.
GPUStack CEO Qin Xiaokang told NUPIAO that the market is flooded with GPUs, models, and inference backends, creating a tangled web of hardware-model-framework combinations. Even if you’ve got a GPU in hand, a model won’t just run — it needs inference backend adaptation and performance tuning specific to the hardware.

That’s why GPUStack evolved from basic GPU resource management into a full-stack package covering GPU management, model deployment, inference engine adaptation, and operations management.
Think of AI compute like water and electricity infrastructure. GPUs are the power generators, models and applications are the end users, and GPUStack is more like a “water utility company” — it doesn’t make GPUs, it doesn’t build large models. It organizes scattered compute, schedules it based on demand, and makes sure compute can reliably convert into tokens.
What’s really changing here is how the AI compute industry measures itself: from “how many cards do you have” to “how many tokens can those cards produce.”
From “Grabbing GPUs” to “Actually Using Them Well”
Domestic GPUs are going through a critical industry phase right now.
Qin Xiaokang sees two big shifts in AI infrastructure over the past two years: open-source models are catching up fast and entering more real-world scenarios, and non-NVIDIA GPUs are starting to get validated in actual business operations. For domestic compute, that means the industry’s attention is moving from the chips themselves toward software ecosystems and real-world efficiency.
That’s exactly where GPUStack has planted its flag. Getting a domestic GPU into actual production isn’t just a hardware problem. Qin told NUPIAO that the two most common issues are models and GPUs failing to adapt to each other — leaving GPUs sitting idle — and models that do run but can’t fully unleash the hardware’s performance.
He mentioned a financial client that had been running models on Huawei GPUs. After optimization, throughput jumped 8 to 9 times, and inference latency dropped by 80% to 90%. That said, this was an extreme optimization case. For scenarios that already had decent optimization foundations, the platform team considers a 20% to 30% utilization bump to be a solid industry benchmark.
The real story here isn’t about how many percentage points some platform can squeeze out of utilization — it’s that competition in the domestic GPU industry is extending into the software stack.
“Just doing GPU compute orchestration isn’t the hardest part. The really complex stuff is inference backend and inference engine adaptation and optimization.” GPUStack CTO Liang Sheng believes different GPUs map to different software ecosystems, different models have different runtime requirements, and model architectures are still evolving fast — which means AI compute infrastructure is still stuck in a phase similar to the early PC era, where “drivers needed constant adaptation.”
GPUStack tries to solve this with a pluggable inference backend mechanism: when a model vendor releases a new model or custom inference backend, there’s no need to rebuild the entire platform — just plug in through the interface.
That’s also one of the differences between GPUStack and traditional GPU clouds or pure GPU scheduling tools.
The company believes there’s no direct competitor that fully maps to GPUStack right now. Overseas, several GPU orchestration and AI infrastructure solutions have already taken shape — NVIDIA’s Run:ai mainly handles GPU scheduling, resource pooling, and cluster orchestration; open-source projects like Kueue and Volcano in the Kubernetes ecosystem tackle GPU workload scheduling; Anyscale approaches it more from distributed computing and AI workload scaling. When the European Commission analyzed the GPU orchestration software market, it also listed Run:ai alongside Slurm, ClearML, Anyscale, and Kubeflow as varying degrees of alternatives.
GPUStack is trying to push its competitive position further upstream: not just managing GPUs, but connecting GPUs, models, inference engines, and token services into one chain.
Which means its real competition isn’t just another GPU scheduling software company — it’s enterprises building their own platforms, chip vendors’ proprietary software stacks, cloud providers’ AI infrastructure, and all sorts of open-source project combinations.
This is also why the open-source model matters so much to GPUStack. The team believes AI hardware and models are iterating too fast for any single company to handle all the adaptation work — it requires hardware vendors, model vendors, and developers all chipping in. GPUStack has already reached 110,000 deployments, with real user usage and feedback becoming a key source of product iteration.
Compute Is Now a Game of Efficiency
The business model of the AI compute industry is shifting too.
Qin Xiaokang points out that many domestic token factories still operate on a project basis, mainly serving large-scale compute clusters of 10,000 cards or more. But not every company will rely on a handful of massive public compute centers in the future. For finance, energy, manufacturing, and plenty of other industries with requirements around data security, cost, and business scenarios, private deployment demand isn’t going anywhere.
That means the AI compute market could split into new tiers: a few super-large compute centers providing baseline compute, and lots of enterprises building their own “small token factories.”
GPUStack is targeting the latter. From a business logic standpoint, it’s somewhat similar to how cloud computing developed: once infrastructure becomes standardized, enterprises no longer need to build every underlying system themselves — they buy a mature software and service package that hides the underlying complexity.
The experience of Zhu Jiang, chairman of Diou Shuihua Group, offers a buyer-side perspective.
As a traditional industry enterprise, his team had already started building out compute operations and tried to develop its own compute management system. Zhu recalled to NUPIAO that the company had invested over a million yuan in self-development, but AI engineering talent was scarce, and GPU adaptation, software deployment, and bug fixing all demanded heavy investment. In the end, open-source GPUStack lowered the barrier to self-development, letting the company quickly achieve unified management across multiple GPU brands and build out compute services.
Zhu’s read on the industry: the GPU shortage won’t last forever. As GPU supply increases, the competitive focus will shift from “who can get GPUs” to “who can manage and utilize GPUs well.” His company has already made compute operations its second growth curve.
This is essentially the industry question GPUStack is trying to answer: when GPUs gradually go from scarce resource to a production material that can be purchased at scale, where’s the next bottleneck?
The answer probably isn’t more GPUs — it’s how to organize different GPU brands, different models, and different inference engines into a stable production system.
So “token factory” isn’t just a fancy new name for AI compute. It points to a real shift in AI infrastructure — the AI industry is moving from fighting over compute assets to fighting over compute efficiency; from building compute centers to building infrastructure that can continuously produce tokens.