MindStudio ModelSlim User Guide¶
1. Functional Overview¶
MindStudio ModelSlim (msModelSlim) is a high-performance model compression tool in the Ascend ecosystem. It supports quantization and compression of dense LLMs, MoE models, and multimodal models. Developers can quickly tune models through Python APIs and export models adapted to frameworks such as MindIE and vLLM-Ascend for efficient deployment on Ascend AI Processors.
msModelSlim provides the following core functional modules, which can be selected based on actual requirements:
| Functional Module | Description | Applicable Scenarios |
|---|---|---|
| One-Click Quantization (V1) | Integrates best practices for mainstream large model quantization, automatically matches optimal configurations, and works out of the box. | Recommended as the first choice, applicable to models that support one-click quantization |
| Traditional Quantization (V0) | Performs quantization through Python scripts and supports fine-grained parameter adjustment. | Models not yet supported by one-click quantization |
| Sensitive Layer Analysis | Evaluates the quantization sensitivity of each layer from multiple dimensions, providing data support for quantization configuration tuning. | Quantization accuracy tuning and accuracy bottleneck locating |
| Automatic Tuning | Automatically and iteratively searches for quantization configurations based on accuracy targets, with full-process automation. | When accuracy does not meet the target, eliminating the need for repeated manual parameter adjustment |
| Autonomous Quantization | Provides a standard integration framework, supporting developers in quickly integrating their own models. | Integrating custom models into msModelSlim |
| Quantization Algorithm | Describes in detail the various quantization and outlier suppression algorithms supported by msModelSlim. | In-depth understanding of quantization principles and algorithm selection |
| Quantization Format | Describes the storage format of quantized weights and compatibility with various frameworks. | Multi-framework deployment and weight conversion |
2. Usage¶
2.1 One-Click Quantization¶
This feature provides corresponding usage instructions and operation guidance based on the capability support of msModelSlim under different architectures.
For detailed usage, refer to One-Click Quantization User Guide. You can select a specific functional module in the left navigation pane to view the related introduction.
2.2 Traditional Quantization¶
Traditional Quantization (V0) performs quantization through Python scripts and is applicable to model scenarios not yet supported by One-Click Quantization.
For detailed usage, see Traditional Quantization User Guide.
2.3 Sensitive Layer Analysis¶
Sensitive layer analysis is used to evaluate the sensitivity of each layer to quantization from multiple dimensions, helping accurately locate layers that should be rolled back or have their bit width increased, thereby providing data support for quantization configuration tuning.
For detailed usage, refer to Sensitive Layer Analysis User Guide.
2.4 Automatic Tuning¶
The Automatic Tuning feature can automatically and iteratively search for the optimal quantization configuration based on the accuracy target, automating the entire quantization and evaluation process without the need for repeated manual parameter adjustment.
For detailed usage, refer to Automatic Tuning User Guide.
2.5 Autonomous Quantization¶
This section provides complete model integration guidance for developers who want to integrate their own models into msModelSlim.
It is recommended that you first read Architecture Description to understand the overall design logic, and then refer to the following documents to complete model integration:
3. Quantization Algorithm¶
msModelSlim supports multiple quantization algorithms and outlier suppression algorithms. For details, see Quantization Algorithm Description.
4. Quantization Format¶
msModelSlim supports multiple quantized weight storage formats and weight conversion between different inference frameworks. For details, see Quantization Format Description.