msServiceProfiler Compare Tool

Overview

msServiceProfiler Compare Tool is designed for comparative analysis of performance data differences across different versions or frameworks in LLM inference service scenarios. It supports the generation of visualized reports and structured data output.

Concepts

• Service overall dimension: includes service-level core metrics such as throughput and latency.
• Request dimension: includes metrics for the complete processing cycle of a single request.
• Batch processing dimension: includes segmented performance metrics for batch processing tasks.

Supported Products

Note:
For details about Ascend product models, see Ascend Product Models.

Product Type	Supported (Yes/No)
Atlas A3 training products and Atlas A3 inference products	Yes
Atlas A2 training products and Atlas A2 inference products	Yes
Atlas 200I/500 A2 inference products	Yes
Atlas inference products	Yes
Atlas training products	No

Notice:
For Atlas A2 training products/Atlas A2 inference products, only the Atlas 800I A2 inference server is supported.
For Atlas inference products, only the Atlas 300I Duo inference card and Atlas 800 inference server (model 3000) are supported.

Preparations

• Install msServiceProfiler.
• If operator-level comparison is required, ensure that msprof-analyze is installed.

Data Preparation

• Collect service profile data to obtain two sets of data for comparison. For detailed collection methods, see msServiceProfiler.
• In operator comparison scenarios, use msServiceProfiler to collect profile data. Set acl_task_time to 3 to ensure that the directory of the collected profile data contains the operator data file suffixed with _ascend_pt.

Constraints

• Only CANN 8.1.RC1 and later versions are supported.
• MindIE 2.0.RC1 or later is required.
• The input data must be generated by parsing with msServiceProfiler.

Function

Description

Provides profile data difference comparison for serving scenarios, supporting the generation of structured comparison reports in Excel format.

Syntax

msserviceprofiler compare <input_path> <golden_path> [--output-path <output_path>] [--log-level <log_level>]

Parameters

Parameter	Mandatory (Yes/No)	Description
input_path	Yes	Path to a directory containing data to be analyzed (must include data parsed by msServiceProfiler).
golden_path	Yes	Path to a directory containing golden data.
--output-path	No	Path to a directory containing output results (Default: ./compare_result).
--log-level	No	Sets the log level. The options are as follows: • debug: debug level. Logs the debugging information for R&D or maintenance personnel to locate faults. • info: normal level. Logs normal tool operation information. This is the default value. • warning: warning level. Logs unexpected states that do not affect process execution. • error: minor error level. • fatal: major error level. • critical: critical level.

Example

# Perform default comparison.
msserviceprofiler compare ./profiling_data/v1 ./profiling_data/v2

Output File Description

Output Directory Description

span_comparation_result.csv displays the absolute error and relative error of all data pairs. It contains multiple span names, showing the time differences across spans.

The output directory structure is as follows:

|- output_path
    |- span_comparation_result.csv

Field Description

Table 1 span_comparation_result.csv

Field	Description
Golden-AVG	Average value of the golden data.
Golden-P50	50th percentile of the golden data.
Golden-P90	90th percentile of the golden data.
Input-AVG	Average value of the comparison data.
Input-P50	50th percentile of the comparison data.
Input-P90	90th percentile of the comparison data.
DIFF-AVG	Difference between the averages of the comparison data and the golden data. DIFF = Input - Golden
DIFF-P50	Difference between the 50th percentiles of the comparison data and the golden data.
DIFF-P90	Difference between the 90th percentiles of the comparison data and the golden data.
RDIFF-AVG(%)	Relative difference between the averages of the comparison data and the golden data. RDIFF = (Input - Golden)/Golden
RDIFF-P50(%)	Relative difference between the 50th percentiles of the comparison data and the golden data.
RDIFF-P90(%)	Relative difference between the 90th percentiles of the comparison data and the golden data.