AI large model tuning and deployment: Alibaba Cloud GPU/high-performance computer model selection and cost planning

cloud 2026-07-25 阅读 3

In the AI big model application explosion of the moment, many enterprises and developers are facing an extremely realistic problem:

The model is selected, how to build the computing environment? How to control the cost?

Fine-tuning (Fine-tuning) and inference deployment (Inference) of large models require very different hardware resources. The wrong calculation power specification is selected, either the "small horse-drawn cart" directly overflows (OOM), or the "big horse-drawn cart" causes a huge waste of calculation power.

From the perspective of actual combat, this article will disassemble how to select the most suitable GPU/high-performance computing instance for fine-tuning and deployment of large models on Alibaba Cloud, and provide a cost planning and avoidance guide for grounding gas.

1. first understand the core pain point: the difference in computing power requirements between fine-tuning and deployment.

Before selecting the model, we must clarify the difference in hardware requirements between fine-tuning and deployment.

Don't use deployment thinking to choose the calculation power of fine-tuning, and vice versa.

-----------------------------------------------------------------

| Large Model Hardware Requirements Comparison |

-----------------------------------------------------------------

----------------------------------------------------------------

-----------------------------------------------------------------

1. Model Fine-tuning (Fine-tuning): memory throughput is the last word

Fine-tuning (even for lightweight fine-tuning such as LoRA/QLoRA) requires the storage of model weights during training.

Gradients (Gradients), optimizer states (Optimizer States), and intermediate activation values (Activations)

Full parameter fine-tuning: the consumption of video memory is usually 4 to 6 times the amount of model parameters.

LoRA / QLoRA: Significantly reduces video memory requirements, but still requires enough video memory

Accommodates the extension of the context length (Context Length).

2. Model deployment (Inference): high throughput and low latency

The deployment focuses on KV Cache (key-value cache) management and concurrency capabilities. In the reasoning phase, the core looks

Memory Bandwidth (Memory Bandwidth) and Power Utilization

. In order to reduce costs, we usually combine quantization (quantization, such as INT4/FP8), vLLM, TGI and other acceleration frameworks.

2. Alibaba Cloud GPU/HPC Type Selection Panoramic Guide

The naming of Alibaba Cloud's GPU instances may seem complicated, but as long as you master the letter rules, the selection is actually very clear:

1. Lightweight deployment and small model fine-tuning (7B ~ 14B parameters)

If you are mainly running models with parameters 7B to 14B (such as Qwen2-7B, Llama-3-8B), or QLoRA light-weight fine-tuning:

Preferred model: gn7i / gn8is series GPU configuration: usually equipped with NVIDIA A10 or the same level of high-performance Tensor Core GPU(24GB ~ 48GB video memory). Applicable scenarios: high concurrency inference deployment, QLoRA fine tuning, small and medium-sized image and text generation. Advantages: extremely high cost performance, a single card can easily run 7B model FP16 reasoning or INT4 batch reasoning.

2. Medium-scale full-scale fine-tuning and large-scale model deployment (14B ~ 72B parameters)

For the 14B ~ 72B scale model, the demand for video memory is rising. Single card 24GB memory is far from enough, need multi-card parallel or single card large memory.

Preferred model: ebmgn7ex / gn7e series GPU configuration: equipped with NVIDIA A100 / V100 and other high-performance GPU(40GB/80GB video memory). Applicable scenarios: full parameters of 13B/70B model/LoRA fine-tuning, large model high concurrent deployment. Advantages: Equipped with high bandwidth NVLink interconnection, multi-card communication efficiency is extremely high, to avoid the distributed training of the card.

3. Ultra-large-scale pre-training and cluster fine-tuning (100 billion parameters/mixed expert model MoE)

For 100 billion-level parameter models (e. g. DeepSeek-R1 full version, Qwen-2.5-72B full fine-tuning):

Preferred model: high-performance heterogeneous computing cluster (SCC/lingjun intelligent computing) GPU configuration: NVIDIA H800/H100/ariyun self-developed high-performance computing cluster. Network support: RDMA (remote direct memory access) network, providing 200Gbps of non-blocking interconnection bandwidth. Applicable scenarios: large-scale distributed fine-tuning, multi-machine multi-card parallel training.

3. Ariyun large model calculation cost planning and pit avoidance guide.

Calculating power is a recognized pain point in the industry. Without scientific planning, tens of thousands or even hundreds of thousands of bills a month will hurt any team. The following are the four major money-saving strategies summarized in the actual battle:

1. Flexible use of billing mode: by volume, monthly and preemptive instances

Development and debugging phase: Be sure to use preemptible instances (Spot Instance). Aliyun preemptible instances can enjoy an ultra-low discount of up to 1 ~ 30% compared with pay-as-you-go instances. In the stage of writing code and adjusting Pipeline, preemptive instances can help you save more than 70% of trial and error costs.

Stable fine-tuning tasks: fine-tuning usually needs to run continuously for several hours to several days. it is recommended to use the pay-as-you-go + savings plan (Savings plans) or subscribe by week/month.

Online production deployment: The production environment requires 7 × 24 hours of high availability, and the overall cost is most controllable when you directly choose the package year and month, combined with Aliyun's renewal discount.

2. Account system and discount policy optimization (core money saving skills)

When an enterprise purchases Alibaba Cloud computing resources, it is often the least cost-effective way to purchase them directly from the official website at the standard original price.

Many enterprise teams build large model projects through compliance.

Alibaba Cloud account purchase

Channel or contact Alibaba Cloud enterprise-level service providers (vouchers/channel discounts) for account registration and bulk purchases. In this way, companies usually receive additional cash back, voucher credits, or exclusive model credits beyond official policies.

Tip: Especially when applying for high-end GPU quotas (Quotas) such as H800/A100, you can not only obtain a higher calculation quota, but also get a better price in the annual payment contract by purchasing and configuring the submitted Aliyun account.

3. Model compression techniques (quantization + pruning)

Half the video memory = half the cost.

When deploying, try not to use FP16 native weights directly. Models were quantified using AWQ, GPTQ, or FP8. The 70B model requires 140GB of video memory (at least 2 A100) under FP16, but only about 40GB of video memory is required after quantification to INT4, which is directly reduced from 2 cards to 1 card, and the cost is cut by 50% instantly.

4. Storage and data transfer optimization

Large model weights often dozens of GB, frequent reading and writing and cross-regional transmission will also have hidden costs:

NAS/CPFS high-performance parallel file system: Data sets are read frequently during fine-tuning. It is recommended to use Alibaba Cloud CPFS to prevent GPU from being in the Idle state waiting for data.

Deployment in the same region: Ensure that your OSS (object storage) and ECS/GPU instances are located in the same region (Region) and Availability Zone (AZ). Intranet transmission is not only free, but also dozens of times faster than the Internet.

4. selection decision flow chart (directly applied)

To help you make a quick decision, you can refer to the following path:

Just want to deploy a 7B/8B

The customer service/question and answer model?👉Select gn8is or gn7i (single card A10/24G) with vLLM framework INT4 quantization. The cost is controlled at the level of thousand yuan per month.

Need LoRA/domain fine-tuning for 14B ~ 32B models?👉Choose from gn7i (dual card) or ebmgn7ex (single card A100), combined with QLoRA technology. Pay-as-you-go/preemptible instances are settled by the hour.

Do you want to fine-tune the full parameters of the 70B model or deploy high-performance API services?👉Select ebmgn7ex(8-card A100/H800 cluster) and enable RDMA network acceleration. Combined with enterprise-level Aliyun account purchase scheme to obtain quotas and discounts, monthly package or savings plan.

Summary

There is no "absolute best" for large model computing power selection, only "best for the moment".

At the beginning of the project,

Lightweight fine-tuning quantization deployment preemptible instance

is the best combination for quickly validating MVP (minimum viable product); and when the business enters the run-up phase,

Reasonably evaluate the interconnection bandwidth of the model and match the purchase and renewal discounts of enterprise-level Aliyun accounts.

It is the key to keep AI projects profitable and healthy.