4 Commits
10 changed files with 681 additions and 64 deletions
+312
View File
@@ -0,0 +1,312 @@
# LLM 速度测试台 · API 文档
> 系统所有页面功能均通过 REST API 提供,前端(网页)只是这些 API 的一个可视化客户端。
> 任何页面可见/可操作的数据都可以通过下面的接口调用或访问。
- **服务地址:** `http://<IP>:16097`
- **数据格式:** 请求/响应均为 `application/json`(导出类接口除外)
- **鉴权:** 内部工具,当前无鉴权;如需对外暴露,建议在网关/Nginx 层加访问控制
- **测试启动为异步**`POST /api/tests` 返回后,任务在后台线程执行,用 `GET /api/tests/<id>` 或日志轮询接口跟踪进度
---
## 目录
1. [健康检查](#1-健康检查)
2. [提供商配置](#2-提供商配置-configs)
3. [连接测试](#3-连接测试)
4. [速度测试](#4-速度测试-tests)
5. [测试详情与日志](#5-测试详情与日志)
6. [折线图(data-chart-tool](#6-折线图data-chart-tool)
7. [导出(Excel / JSON](#7-导出excel--json)
8. [数据模型](#8-数据模型)
9. [curl 使用示例](#9-curl-使用示例)
---
## 1. 健康检查
### `GET /api/health`
返回服务状态与正在运行的测试。
**响应:**
```json
{ "ok": true, "port": 16097, "running_tests": [] }
```
---
## 2. 提供商配置(Configs
### `GET /api/configs`
列出所有已保存配置(不返回 API Key 明文,仅 `has_key` 标记)。
**响应:**
```json
[
{ "id": 2, "name": "epyc-test", "provider": "openai",
"base_url": "http://121.40.164.32:18003/v1",
"model": "unsloth/Qwen3.8-27B-Q4_K_M", "temperature": 0.7, "has_key": 1 }
]
```
### `POST /api/configs`
新增配置。请求体字段:`name`(必填), `provider`, `base_url`, `api_key`, `model`, `temperature`
**请求:**
```json
{ "name": "DeepSeek", "provider": "openai",
"base_url": "https://api.deepseek.com/v1",
"api_key": "sk-xxx", "model": "deepseek-chat", "temperature": 0.7 }
```
**响应:** `{ "ok": true, "id": 3 }`
### `GET /api/configs/<id>`
获取单个配置(**含 API Key 明文**,用于前端回填;注意保管)。
### `PUT /api/configs/<id>`
更新配置,**局部更新**(只改请求里出现的字段)。
**请求:** `{ "model": "deepseek-v4-flash" }` → 响应 `{ "ok": true, "id": 3 }`
### `DELETE /api/configs/<id>`
删除配置。响应 `{ "ok": true }`
---
## 3. 连接测试
### `POST /api/configs/test`
验证 API Key / Base URL / 模型名连通性。只要流式请求成功返回(HTTP 200 + 收到响应流)即视为连通;支持推理型模型(Qwen3/DeepSeek 思维链)。
**请求:** 同配置对象(`provider`, `base_url`, `api_key`, `model`, `temperature`
**成功响应:**
```json
{
"ok": true, "total_ms": 1308.2,
"note": "",
"metrics": { "prompt_tokens": 63, "output_tokens": 32, "output_chars": 110,
"ttft_ms": 715.9, "prefill_speed": 86.7, "decode_speed": 52.7, "total_ms": 1308.2 }
}
```
> `note` 非空表示连接正常但未返回正文(可能为只输出思维链的模型)。
**失败响应:** `{ "ok": false, "error": "HTTP 401: ..." }`
---
## 4. 速度测试(Tests
### `POST /api/tests`
启动一次速度测试(异步,立即返回测试 id)。
**请求体:**
```json
{
"config": {
"provider": "openai",
"name": "Qwen3 对比",
"base_url": "http://121.40.164.32:18003/v1",
"api_key": "sk-xxx",
"model": "unsloth/Qwen3.8-27B-Q4_K_M",
"temperature": 0.7
},
"gen": {
"name": "Qwen3 不同上下文长度速度对比",
"context_lengths": [512, 2048, 4096, 8192, 16384, 32768, 65536, 131072],
"max_tokens": 128,
"samples": 2,
"warmup": true,
"avoid_cache": true
}
}
```
**gen 字段说明:**
| 字段 | 类型 | 默认 | 说明 |
|------|------|------|------|
| `name` | string | `""` | 测试名称/主题(会存入测试记录并展示在历史与详情) |
| `context_lengths` | number[] | `[512,2048,4096,8192,16384,32768,65536,131072]` | 要测试的上下文长度列表,每个长度独立校准+预热+采样 |
| `max_tokens` | number | `128` | 解码输出 token 长度 |
| `samples` | number | `2` | 每个上下文长度的采样次数 |
| `warmup` | bool | `true` | 测试前空转预热(不计速度) |
| `avoid_cache` | bool | `true` | 随机前缀避免缓存命中 |
**响应:** `{ "ok": true, "id": 9 }`
### `GET /api/tests?limit=<n>`
测试历史列表(按 id 倒序)。`limit` 默认 100,最大 1000。
**响应:**
```json
[
{ "id": 9, "created_at": "2026-08-23 18:52:00", "status": "done",
"provider": "openai", "model": "unsloth/Qwen3.8-27B-Q4_K_M",
"name": "Qwen3 不同上下文长度速度对比",
"error": "",
"summary": { "samples_ok": 2, "samples_total": 2, "avg_ttft_ms": 1808.7, ... } }
]
```
### `POST /api/tests/<id>/cancel`
停止正在运行的测试。响应 `{ "ok": true, "msg": "正在停止..." }`
### `DELETE /api/tests/<id>`
删除测试及其全部采样与日志。响应 `{ "ok": true }`
---
## 5. 测试详情与日志
### `GET /api/tests/<id>`
完整测试详情:基本信息 + 配置(API Key 已打码)+ 生成参数 + 汇总 + 每次采样 + 完整日志。
**响应结构:**
```json
{
"id": 9, "created_at": "...", "status": "done",
"provider": "openai", "model": "...", "name": "...", "error": "",
"config": { "base_url": "...", "api_key": "sk-x****", ... },
"gen": { "name": "...", "context_lengths": [512, 2048], "max_tokens": 128, "samples": 1, "warmup": true, "avoid_cache": true },
"summary": {
"samples_total": 2, "samples_ok": 2,
"calibration_chars_per_token": 1.82,
"avg_ttft_ms": 1808.7, "min_ttft_ms": 1122.8, "max_ttft_ms": 2494.6,
"avg_prefill_speed": 694.2, "min_prefill_speed": 515.7, "max_prefill_speed": 872.7,
"avg_decode_speed": 54.7, "min_decode_speed": 54.4, "max_decode_speed": 55.0,
"avg_prompt_tokens": 1378.0, "avg_output_tokens": 128.0,
"avg_total_ms": 4148.4, "min_total_ms": 3449.9, "max_total_ms": 4846.9,
"by_length": {
"512": { "samples_total": 1, "samples_ok": 1, "avg_ttft_ms": 1122.8, "avg_prefill_speed": 515.7, "avg_decode_speed": 55.0, "avg_prompt_tokens": 579, "avg_output_tokens": 128, "avg_total_ms": 3449.9 },
"2048": { "samples_total": 1, "samples_ok": 1, "avg_ttft_ms": 2494.6, "avg_prefill_speed": 872.7, "avg_decode_speed": 54.4, "avg_prompt_tokens": 2177, "avg_output_tokens": 128, "avg_total_ms": 4846.9 }
}
},
"runs": [
{ "run_index": 1, "context_length": 512,
"metrics": { "prompt_tokens": 579, "output_tokens": 128, "cached_tokens": 0,
"ttft_ms": 1122.8, "prefill_speed": 515.7, "decode_speed": 55.0, "total_ms": 3449.9 },
"error": "" }
],
"logs": [ { "id": 1, "level": "INFO", "msg": "═══ 开始速度测试 ═══", "rel": 0.0, "ts": "..." } ]
}
```
### `GET /api/tests/<id>/logs?after=<id>`
增量日志(前端轮询用)。`after` 为上次取到的最大日志 id,返回其后新增日志 + 最新状态/汇总/最后采样。
**响应:**
```json
{
"status": "running", "error": "",
"summary": {},
"last_run": { "...": "..." },
"logs": [ { "id": 68, "level": "METRIC", "msg": "...", "rel": 3.21 } ],
"after": 73
}
```
---
## 6. 折线图(data-chart-tool
> 调用 [data-chart-tool](http://192.168.2.8:12007/hz4th_coder/data-chart-tool.git) 的 `/api/chart` 接口生成**双Y轴折线图**:左轴=预填充速度(虚线)、右轴=解码速度(实线)、X 轴=上下文长度。本系统在服务端代理转发,前端只需请求本服务的两个接口。
### `GET /api/tests/<id>/chart`
生成折线图 PNG 图片(内部调用 data-chart-tool `/api/chart`,地址可在 `config.py``CHART_API_BASE` 修改)。
**成功响应:** `Content-Type: image/png`(可直接用于 `<img>` / 下载)
**失败响应:**
```json
{ "ok": false, "error": "无成功采样数据,无法画图" } // 400
{ "ok": false, "error": "测试不存在" } // 404
{ "ok": false, "error": "图表服务不可用: ..." } // 502
```
### `GET /api/tests/<id>/chart-data`
返回画图所用的 CSV 数据与图表请求配置,方便手动快速复制。
**响应:**
```json
{
"ok": true,
"csv": "上下文长度, 预填充速度(tok/s), 解码速度(tok/s)\n4096, 126.90, 22.80\n8192, 132.20, 22.40",
"rows": [[4096, 126.9, 22.8], [8192, 132.2, 22.4]],
"payload": {
"data": "...", "chartType": "line", "title": "...",
"dualYAxis": true,
"leftAxisName": "预填充速度(tok/s)", "rightAxisName": "解码速度(tok/s)",
"seriesTypes": ["line", "line"], "seriesAxis": [0, 1],
"seriesStyles": ["dashed", "solid"], "width": 1000, "height": 560, "pixelRatio": 2
}
}
```
> `csv` 即画图数据(第一列=上下文长度,第二列=预填充速度,第三列=解码速度),前端「复制画图数据」按钮复制的就是它。
---
## 7. 导出(Excel / JSON
### `GET /api/tests/<id>/export.xlsx`
导出 Excel 报告(**3 个 Sheet**:汇总 / 采样明细 / 日志),`Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet`
- **汇总**:测试信息 + 整体统计指标(平均/最大/最小)+ 按上下文长度分组
- **采样明细**:每次采样的上下文长度与全部指标
- **日志**:完整测试日志
### `GET /api/tests/<id>/export.json`
导出完整测试数据为 JSON(与 `GET /api/tests/<id>` 一致,API Key 打码),`Content-Type: application/json`
---
## 8. 数据模型
| 表 | 说明 | 关键字段 |
|----|------|----------|
| `configs` | 保存的接口配置 | id, name, provider, base_url, api_key, model, temperature |
| `tests` | 测试记录 | id, status(running/done/error/canceled), provider, model, **name**, config_json, gen_cfg_json, summary_json, error |
| `test_runs` | 每次采样 | id, test_id, run_index, **context_length**, metrics_json, error |
| `logs` | 测试日志 | id, test_id, level, msg, rel |
**summary 整体指标字段:**
`avg_/min_/max_` 前缀 × `ttft_ms` / `prefill_speed` / `decode_speed` / `total_ms`,以及 `avg_prompt_tokens` / `avg_output_tokens` / `avg_cached_tokens` / `best_ttft_ms`= min_ttft_ms)。
---
## 9. curl 使用示例
```bash
BASE=http://<IP>:16097
# 健康检查
curl $BASE/api/health
# 新增配置
curl -X POST $BASE/api/configs -H 'Content-Type: application/json' \
-d '{"name":"Qwen3","provider":"openai","base_url":"http://121.40.164.32:18003/v1","api_key":"sk-xxx","model":"unsloth/Qwen3.8-27B-Q4_K_M","temperature":0.7}'
# 测试连接
curl -X POST $BASE/api/configs/test -H 'Content-Type: application/json' \
-d '{"provider":"openai","base_url":"http://121.40.164.32:18003/v1","api_key":"sk-xxx","model":"unsloth/Qwen3.8-27B-Q4_K_M"}'
# 启动速度测试(异步)
curl -X POST $BASE/api/tests -H 'Content-Type: application/json' -d '{
"config": {"provider":"openai","base_url":"http://121.40.164.32:18003/v1","api_key":"sk-xxx","model":"unsloth/Qwen3.8-27B-Q4_K_M"},
"gen": {"name":"各长度对比","context_lengths":[512,2048,8192],"max_tokens":128,"samples":2,"warmup":true,"avoid_cache":true}
}'
# 查询测试列表 / 详情
curl "$BASE/api/tests?limit=10"
curl $BASE/api/tests/9
# 画图数据(CSV
curl $BASE/api/tests/9/chart-data
# 折线图 PNG(预填充左轴虚线 / 解码右轴实线)
curl -o chart.png $BASE/api/tests/9/chart
# 导出
curl -OJ $BASE/api/tests/9/export.xlsx
curl $BASE/api/tests/9/export.json
```
+27 -11
View File
@@ -4,7 +4,7 @@
- **访问地址:** `http://<IP>:16097/` - **访问地址:** `http://<IP>:16097/`
- **技术栈:** Python 3 + Flask + SQLite(纯 REST,无额外依赖) - **技术栈:** Python 3 + Flask + SQLite(纯 REST,无额外依赖)
- **版本:** v2.0.0 - **版本:** v2.2.0
--- ---
@@ -16,7 +16,8 @@
- 配置可**保存/加载/删除**,方便多模型对比 - 配置可**保存/加载/删除**,方便多模型对比
### 🚀 速度测试配置 ### 🚀 速度测试配置
- **多上下文长度测试**:默认为 `512 / 2048 / 8192 / 32768 / 131072` tokens,可通过标签点击启用/禁用,并支持手动添加任意自定义长度(≥16) - **测试名称(主题)**:可为每次测试命名,用于标注测试内容/主题,展示在历史列表、详情弹窗与导出报表中
- **多上下文长度测试**:默认为 `512 / 2048 / 4096 / 8192 / 16384 / 32768 / 65536 / 131072` tokens,可通过标签点击启用/禁用,并支持手动添加任意自定义长度(≥16)
- **解码输出长度(max tokens**:默认为 `128`,可手动自定义 - **解码输出长度(max tokens**:默认为 `128`,可手动自定义
- **每个长度采样次数**:默认为 `2`,可手动自定义 - **每个长度采样次数**:默认为 `2`,可手动自定义
- **测试前预热(空转)**:默认开启,先发一次不计速度的空转请求,避免冷启动/首请求偏慢污染真实采样数据 - **测试前预热(空转)**:默认开启,先发一次不计速度的空转请求,避免冷启动/首请求偏慢污染真实采样数据
@@ -24,11 +25,20 @@
### 📊 指标与结果 ### 📊 指标与结果
- 实时指标卡:首字延迟、预填充速度、解码速度、上文/输出 tokens、总耗时 - 实时指标卡:首字延迟、预填充速度、解码速度、上文/输出 tokens、总耗时
- **整体统计(平均/最小/最大)**:详情弹窗与 Excel 汇总展示首字延迟、预填充速度、解码速度、总耗时的平均/最小/最大
- **推理型模型兼容**:支持 Qwen3 / DeepSeek 等思维链模型(`reasoning_content` / `thinking` / `thought`),思维过程计入输出,不会误报“未收到输出”
- **采样失败不中断**:单次采样失败会记录并继续,不会让整个测试半途终止;全部失败才标记 error
- 实时控制台日志:校准、预热、每次采样明细全程可追溯 - 实时控制台日志:校准、预热、每次采样明细全程可追溯
- **每次完整测试**支持: - **每次完整测试**支持:
- **网页点击查看**:历史记录「查看」按钮弹出详情(整体平均 + 按上下文长度分组 + 每次采样明细 + 完整日志) - **网页点击查看**:历史记录「查看」按钮弹出详情(整体统计 + 按上下文长度分组 + 每次采样明细 + 完整日志)
- **📈 折线图(data-chart-tool)**:详情弹窗内点击「生成/刷新折线图」,调用 data-chart-tool 的 `/api/chart` 画**双Y轴折线图**——左轴=预填充速度(虚线),右轴=解码速度(实线),X 轴为上下文长度;并提供**画图 CSV 数据一键复制 / 下载 PNG**
- **文件下载 Excelxlsx)**:历史记录「Excel」按钮或详情弹窗「导出 Excel」,包含 汇总 / 采样明细 / 日志 三个 Sheet - **文件下载 Excelxlsx)**:历史记录「Excel」按钮或详情弹窗「导出 Excel」,包含 汇总 / 采样明细 / 日志 三个 Sheet
- 测试历史留存,可随时刷新、查看、导出、删除 - **文件下载 JSON**:详情弹窗「导出 JSON」
- 测试历史留存(含测试名称),可随时刷新、查看、导出、删除
### 🔌 开放 API
- 页面所有功能均通过 REST API 提供,前端只是可视化客户端
- **API 文档见 [API.md](API.md)**
--- ---
@@ -79,14 +89,19 @@ pip install -r requirements.txt
|------|------|------| |------|------|------|
| GET | `/api/health` | 健康检查 | | GET | `/api/health` | 健康检查 |
| GET/POST | `/api/configs` | 配置列表 / 新增配置 | | GET/POST | `/api/configs` | 配置列表 / 新增配置 |
| GET/DELETE | `/api/configs/<id>` | 单个配置 / 删除 | | GET/PUT/DELETE | `/api/configs/<id>` | 单个配置 / 更新(局部)/ 删除 |
| POST | `/api/configs/test` | 测试连接 | | POST | `/api/configs/test` | 测试连接 |
| GET/POST | `/api/tests` | 测试历史 / 启动测试 | | GET/POST | `/api/tests?limit=n` | 测试历史 / 启动测试(异步) |
| GET | `/api/tests/<id>` | 测试详情(含 runs / logs / summary | | GET | `/api/tests/<id>` | 测试详情(含 runs / logs / summary / 按长度分组 |
| GET | `/api/tests/<id>/logs?after=<id>` | 增量日志(前端轮询用) | | GET | `/api/tests/<id>/logs?after=<id>` | 增量日志(前端轮询用) |
| POST | `/api/tests/<id>/cancel` | 停止测试 | | POST | `/api/tests/<id>/cancel` | 停止测试 |
| DELETE | `/api/tests/<id>` | 删除测试 | | DELETE | `/api/tests/<id>` | 删除测试 |
| GET | `/api/tests/<id>/export.xlsx` | 导出 Excel 报告 | | GET | `/api/tests/<id>/export.xlsx` | 导出 Excel 报告(汇总/采样明细/日志三 Sheet |
| GET | `/api/tests/<id>/export.json` | 导出完整测试 JSON |
| GET | `/api/tests/<id>/chart` | 用 data-chart-tool 生成折线图 PNG(预填充左轴虚线 / 解码右轴实线) |
| GET | `/api/tests/<id>/chart-data` | 画图数据(CSV + 图表请求配置,供一键复制) |
> 完整字段说明、响应示例与 curl 示例见 **API.md**。
### 启动测试请求示例 ### 启动测试请求示例
```json ```json
@@ -101,7 +116,7 @@ POST /api/tests
"temperature": 0.7 "temperature": 0.7
}, },
"gen": { "gen": {
"context_lengths": [512, 2048, 8192, 32768, 131072], "context_lengths": [512, 2048, 4096, 8192, 16384, 32768, 65536, 131072],
"max_tokens": 128, "max_tokens": 128,
"samples": 2, "samples": 2,
"warmup": true, "warmup": true,
@@ -139,13 +154,14 @@ llm-speed-tester/
## 常见问题 ## 常见问题
- **连接测试提示“未收到任何输出内容”**:多为推理型模型(Qwen3/DeepSeek 思维链)或只返回 usage 的网关。已兼容 `reasoning_content` 等思维字段,连接成功即视为通过;若仍出现,请检查 API Key/Base URL/模型名。
- **无 openpyxl**`pip install openpyxl`(已加入 requirements.txt - **无 openpyxl**`pip install openpyxl`(已加入 requirements.txt
- **老版本数据库**:程序启动时自动迁移,为 `test_runs` 表补充 `context_length` 列,无需手动处理 - **老版本数据库**:程序启动时自动迁移,为 `test_runs` 表补充 `context_length` 列,无需手动处理
- **慢模型超时**:连接超时 30s、两次数据包间隔 120s,足够覆盖大多数慢模型;超长文(131072)生成慢属正常,请耐心等待 - **慢模型/长上文超时**:连接超时 60s、两次数据包间隔 300s`config.STREAM_READ_TIMEOUT` 可调);推理型模型思考阶段停顿不计超时,超长文(131072)生成慢属正常,请耐心等待
--- ---
## Git ## Git
- **仓库:** `hz4th_coder/llm-speed-tester` - **仓库:** `hz4th_coder/llm-speed-tester`
- **版本:** v2.0.0新增多上下文长度测试 + 预热 + Excel 导出 + 界面优化 - **版本:** v2.2.0(多上下文长度测试 + 预热 + Excel/JSON 导出 + 测试名称 + 整体统计平均/最小/最大 + 开放 API + 推理型模型兼容 + 📈 data-chart-tool 双Y轴折线图/画图数据复制 + 默认上下文长度增加 4096/16384/65536
+139 -15
View File
@@ -3,6 +3,7 @@
import io import io
import json import json
import requests
from flask import Flask, jsonify, request, send_file, send_from_directory from flask import Flask, jsonify, request, send_file, send_from_directory
import config import config
@@ -62,6 +63,16 @@ def get_one_config(cid):
return jsonify(c) return jsonify(c)
@app.route("/api/configs/<int:cid>", methods=["PUT"])
def update_config(cid):
cfg = request.get_json(force=True) or {}
old = db.get_config(cid)
if not old:
return jsonify({"ok": False, "error": "配置不存在"}), 404
db.update_config(cid, cfg)
return jsonify({"ok": True, "id": cid})
@app.route("/api/configs/<int:cid>", methods=["DELETE"]) @app.route("/api/configs/<int:cid>", methods=["DELETE"])
def del_config(cid): def del_config(cid):
db.delete_config(cid) db.delete_config(cid)
@@ -74,8 +85,13 @@ def test_config():
if not cfg.get("api_key"): if not cfg.get("api_key"):
return jsonify({"ok": False, "error": "请填写 API Key"}), 400 return jsonify({"ok": False, "error": "请填写 API Key"}), 400
try: try:
m = call_stream(cfg, "你好,请只回复:OK", {"max_tokens": 16, "avoid_cache": False}) # 连接测试:只要流式请求成功返回(哪怕正文为空/只有思维链)都算连通
return jsonify({"ok": True, "total_ms": m["total_ms"], "metrics": m}) m = call_stream(cfg, "你好,请简要回答:1+1=",
{"max_tokens": 32, "avoid_cache": False})
note = ""
if not (m.get("output_tokens") or m.get("output_chars")):
note = "(连接正常,但本次未返回正文内容,可能为推理型模型)"
return jsonify({"ok": True, "total_ms": m["total_ms"], "metrics": m, "note": note})
except ProviderError as e: except ProviderError as e:
return jsonify({"ok": False, "error": str(e)}) return jsonify({"ok": False, "error": str(e)})
except Exception as e: except Exception as e:
@@ -102,7 +118,11 @@ def start_test():
@app.route("/api/tests", methods=["GET"]) @app.route("/api/tests", methods=["GET"])
def list_tests(): def list_tests():
return jsonify(db.list_tests()) try:
limit = int(request.args.get("limit", 100))
except ValueError:
limit = 100
return jsonify(db.list_tests(max(1, min(limit, 1000))))
@app.route("/api/tests/<int:tid>", methods=["GET"]) @app.route("/api/tests/<int:tid>", methods=["GET"])
@@ -112,6 +132,7 @@ def get_test(tid):
return jsonify({"ok": False, "error": "测试不存在"}), 404 return jsonify({"ok": False, "error": "测试不存在"}), 404
t["runs"] = db.get_runs(tid) t["runs"] = db.get_runs(tid)
t["logs"] = db.get_logs(tid) t["logs"] = db.get_logs(tid)
_mask_cfg(t.get("config"))
return jsonify(t) return jsonify(t)
@@ -140,6 +161,107 @@ def del_test(tid):
return jsonify({"ok": True}) return jsonify({"ok": True})
# ───────────────────────── 图表(data-chart-tool 折线图:预填充左轴虚线 / 解码右轴实线) ─────────────────────────
def _build_chart_csv(t):
"""由测试汇总 by_length 构建画图 CSV:上下文长度, 预填充速度(tok/s), 解码速度(tok/s)"""
s = t.get("summary") or {}
by = s.get("by_length") or {}
lens = sorted(int(k) for k in by)
rows = []
for L in lens:
bl = by.get(str(L)) if str(L) in by else by.get(L) or {}
pre = bl.get("avg_prefill_speed")
dec = bl.get("avg_decode_speed")
if pre is None or dec is None:
continue
rows.append([L, round(pre, 2), round(dec, 2)])
if not rows:
return None
csv_lines = ["上下文长度, 预填充速度(tok/s), 解码速度(tok/s)"]
for L, pre, dec in rows:
csv_lines.append("%d, %.2f, %.2f" % (L, pre, dec))
return {"csv": "\n".join(csv_lines), "rows": rows}
def _chart_payload(t, csv_text):
"""组装 data-chart-tool /api/chart 请求体(双Y轴折线图)"""
title = ("%s %s" % (t.get("model") or "", t.get("name") or "速度对比")).strip()
return {
"data": csv_text,
"chartType": "line",
"title": title,
"theme": "default",
"showLegend": True,
"showGrid": True,
"showLabel": False,
"smoothLine": True,
"dualYAxis": True,
"leftAxisName": "预填充速度(tok/s)",
"rightAxisName": "解码速度(tok/s)",
"seriesTypes": ["line", "line"],
"seriesAxis": [0, 1],
"seriesStyles": ["dashed", "solid"], # 预填充=左轴虚线,解码=右轴实线
"width": 1000,
"height": 560,
"pixelRatio": 2,
}
@app.route("/api/tests/<int:tid>/chart-data")
def test_chart_data(tid):
t = db.get_test(tid)
if not t:
return jsonify({"ok": False, "error": "测试不存在"}), 404
built = _build_chart_csv(t)
if not built:
return jsonify({"ok": False, "error": "无成功采样数据,无法画图"}), 400
return jsonify({
"ok": True,
"csv": built["csv"],
"rows": built["rows"],
"payload": _chart_payload(t, built["csv"]),
})
@app.route("/api/tests/<int:tid>/chart")
def test_chart(tid):
"""用 data-chart-tool 生成折线图 PNG(预填充左轴虚线 / 解码右轴实线)"""
t = db.get_test(tid)
if not t:
return jsonify({"ok": False, "error": "测试不存在"}), 404
built = _build_chart_csv(t)
if not built:
return jsonify({"ok": False, "error": "无成功采样数据,无法画图"}), 400
try:
resp = requests.post(config.CHART_API_BASE + "/api/chart",
json=_chart_payload(t, built["csv"]), timeout=60)
except requests.RequestException as e:
return jsonify({"ok": False, "error": "图表服务不可用: %s" % e}), 502
if resp.status_code != 200:
return jsonify({"ok": False, "error": "图表生成失败(%d): %s" % (resp.status_code, resp.text[:300])}), 502
return send_file(io.BytesIO(resp.content), mimetype="image/png")
@app.route("/api/tests/<int:tid>/export.json")
def export_json(tid):
t = db.get_test(tid)
if not t:
return jsonify({"ok": False, "error": "测试不存在"}), 404
t["runs"] = db.get_runs(tid)
t["logs"] = db.get_logs(tid)
_mask_cfg(t.get("config"))
return jsonify(t)
def _mask_cfg(cfg):
"""对外隐藏 API Key,仅保留前几位便于识别"""
if cfg and cfg.get("api_key"):
k = cfg["api_key"]
cfg["api_key"] = k[:4] + "****" if len(k) > 6 else "****"
return cfg
# ───────────────────────── Excel 导出 ───────────────────────── # ───────────────────────── Excel 导出 ─────────────────────────
@app.route("/api/tests/<int:tid>/export.xlsx") @app.route("/api/tests/<int:tid>/export.xlsx")
@@ -188,6 +310,7 @@ def _build_xlsx(t):
ws.cell(1, 1).font = Font(bold=True, size=14) ws.cell(1, 1).font = Font(bold=True, size=14)
info = [ info = [
["测试编号", "#%d" % t["id"]], ["测试编号", "#%d" % t["id"]],
["测试名称", t.get("name") or "(未命名)"],
["创建时间", t.get("created_at", "")], ["创建时间", t.get("created_at", "")],
["状态", t.get("status", "")], ["状态", t.get("status", "")],
["提供商", t.get("provider", "")], ["提供商", t.get("provider", "")],
@@ -204,25 +327,26 @@ def _build_xlsx(t):
] ]
for row in info: for row in info:
ws.append(row) ws.append(row)
ws.cell(14, 1).font = title_font ws.cell(15, 1).font = title_font
r0 = len(info) + 2 r0 = len(info) + 2
overall = [ overall = [
["平均首字延迟(ms)", s.get("avg_ttft_ms")], ["首字延迟(ms)", s.get("avg_ttft_ms"), s.get("max_ttft_ms"), s.get("min_ttft_ms")],
["最佳首字延迟(ms)", s.get("best_ttft_ms")], ["预填充速度(tok/s)", s.get("avg_prefill_speed"), s.get("max_prefill_speed"), s.get("min_prefill_speed")],
["平均预填充速度(tok/s)", s.get("avg_prefill_speed")], ["解码速度(tok/s)", s.get("avg_decode_speed"), s.get("max_decode_speed"), s.get("min_decode_speed")],
["平均解码速度(tok/s)", s.get("avg_decode_speed")], ["提示词(tok)", s.get("avg_prompt_tokens"), None, None],
["平均提示词(tok)", s.get("avg_prompt_tokens")], ["输出(tok)", s.get("avg_output_tokens"), None, None],
["平均输出(tok)", s.get("avg_output_tokens")], ["总耗时(ms)", s.get("avg_total_ms"), s.get("max_total_ms"), s.get("min_total_ms")],
["平均总耗时(ms)", s.get("avg_total_ms")],
] ]
ws.cell(r0, 1, "整体平均指标").font = title_font ws.cell(r0, 1, "整体统计指标(平均 / 最大 / 最小)").font = title_font
for i, row in enumerate(overall, start=r0 + 1): for j, c in enumerate(["指标", "平均", "最大", "最小"], start=1):
ws.append([]) ws.cell(row=r0 + 1, column=j, value=c)
style_header(ws, r0 + 1, 4)
for i, row in enumerate(overall, start=r0 + 2):
for j, v in enumerate(row, start=1): for j, v in enumerate(row, start=1):
ws.cell(row=i, column=j, value=v) ws.cell(row=i, column=j, value=v)
# 按上下文长度分组 # 按上下文长度分组
r1 = r0 + len(overall) + 2 r1 = r0 + len(overall) + 3
ws.cell(r1, 1, "按上下文长度分组").font = title_font ws.cell(r1, 1, "按上下文长度分组").font = title_font
cols = ["上下文长度(tok)", "采样(成功/总数)", "首字ms", "预填充tok/s", "解码tok/s", "提示词tok", "输出tok", "总耗时ms"] cols = ["上下文长度(tok)", "采样(成功/总数)", "首字ms", "预填充tok/s", "解码tok/s", "提示词tok", "输出tok", "总耗时ms"]
ws.append([]) ws.append([])
+6 -3
View File
@@ -10,6 +10,9 @@ DATA_DIR = os.path.join(BASE_DIR, "data")
LOG_DIR = os.path.join(BASE_DIR, "logs") LOG_DIR = os.path.join(BASE_DIR, "logs")
DB_PATH = os.path.join(DATA_DIR, "llm_speed_tester.db") DB_PATH = os.path.join(DATA_DIR, "llm_speed_tester.db")
# 流式请求超时:连接 30s,两次数据包间隔最长 120s模型也够用) # 流式请求超时:连接 60s,两次数据包间隔最长 300s推理型/长上文模型也够用)
CONNECT_TIMEOUT = 30 CONNECT_TIMEOUT = 60
STREAM_READ_TIMEOUT = 120 STREAM_READ_TIMEOUT = 300
# data-chart-tool 图表服务地址(用它的 /api/chart 画折线图:预填充左轴虚线 / 解码右轴实线)
CHART_API_BASE = "http://127.0.0.1:16016"
+27 -4
View File
@@ -28,6 +28,7 @@ CREATE TABLE IF NOT EXISTS tests(
status TEXT DEFAULT 'running', status TEXT DEFAULT 'running',
provider TEXT DEFAULT '', provider TEXT DEFAULT '',
model TEXT DEFAULT '', model TEXT DEFAULT '',
name TEXT DEFAULT '',
config_json TEXT DEFAULT '{}', config_json TEXT DEFAULT '{}',
gen_cfg_json TEXT DEFAULT '{}', gen_cfg_json TEXT DEFAULT '{}',
summary_json TEXT DEFAULT '{}', summary_json TEXT DEFAULT '{}',
@@ -67,11 +68,15 @@ def _connect():
def _migrate(conn): def _migrate(conn):
"""老库升级:test_runs 补 context_length""" """老库升级:test_runs 补 context_length、tests 补 name"""
cur = conn.execute("PRAGMA table_info(test_runs)") cur = conn.execute("PRAGMA table_info(test_runs)")
cols = [r[1] for r in cur.fetchall()] cols = [r[1] for r in cur.fetchall()]
if "context_length" not in cols: if "context_length" not in cols:
conn.execute("ALTER TABLE test_runs ADD COLUMN context_length INTEGER DEFAULT 0") conn.execute("ALTER TABLE test_runs ADD COLUMN context_length INTEGER DEFAULT 0")
cur = conn.execute("PRAGMA table_info(tests)")
tcols = [r[1] for r in cur.fetchall()]
if "name" not in tcols:
conn.execute("ALTER TABLE tests ADD COLUMN name TEXT DEFAULT ''")
def init_db(): def init_db():
@@ -134,6 +139,23 @@ def delete_config(cid: int):
conn.close() conn.close()
def update_config(cid: int, cfg: dict):
"""局部更新:只更新请求里出现的字段"""
allow = {"name", "provider", "base_url", "api_key", "model", "temperature"}
fields = {k: v for k, v in cfg.items() if k in allow and v is not None}
if not fields:
return
sets = ",".join("%s=?" % k for k in fields)
vals = list(fields.values()) + [cid]
with _lock:
conn = _connect()
try:
conn.execute("UPDATE configs SET %s WHERE id=?" % sets, vals)
conn.commit()
finally:
conn.close()
# ───────────────────────── 测试记录 ───────────────────────── # ───────────────────────── 测试记录 ─────────────────────────
def create_test(cfg: dict, gen: dict) -> int: def create_test(cfg: dict, gen: dict) -> int:
@@ -141,9 +163,10 @@ def create_test(cfg: dict, gen: dict) -> int:
conn = _connect() conn = _connect()
try: try:
cur = conn.execute( cur = conn.execute(
"INSERT INTO tests(status,provider,model,config_json,gen_cfg_json,started_at) " "INSERT INTO tests(status,provider,model,name,config_json,gen_cfg_json,started_at) "
"VALUES('running',?,?,?,?,?)", "VALUES('running',?,?,?,?,?,?)",
(cfg.get("provider", "openai"), cfg.get("model", ""), (cfg.get("provider", "openai"), cfg.get("model", ""),
gen.get("name") or cfg.get("name") or "",
json.dumps(cfg, ensure_ascii=False), json.dumps(gen, ensure_ascii=False), json.dumps(cfg, ensure_ascii=False), json.dumps(gen, ensure_ascii=False),
time.time())) time.time()))
conn.commit() conn.commit()
@@ -186,7 +209,7 @@ def list_tests(limit=100):
conn = _connect() conn = _connect()
try: try:
rows = conn.execute( rows = conn.execute(
"SELECT id,created_at,status,provider,model,summary_json,error " "SELECT id,created_at,status,provider,model,name,summary_json,error "
"FROM tests ORDER BY id DESC LIMIT ?", (limit,)).fetchall() "FROM tests ORDER BY id DESC LIMIT ?", (limit,)).fetchall()
out = [] out = []
for r in rows: for r in rows:
+23 -11
View File
@@ -56,6 +56,8 @@ def _parse_sse_line(line):
def _metrics(start, first_token_at, end, prompt_tokens, output_tokens, def _metrics(start, first_token_at, end, prompt_tokens, output_tokens,
cached_tokens, output_chars, prompt_chars): cached_tokens, output_chars, prompt_chars):
if first_token_at is None:
first_token_at = end # 未收到正文但请求完成(如纯 usage 响应)
ttft_ms = (first_token_at - start) * 1000 ttft_ms = (first_token_at - start) * 1000
decode_ms = (end - first_token_at) * 1000 decode_ms = (end - first_token_at) * 1000
total_ms = (end - start) * 1000 total_ms = (end - start) * 1000
@@ -103,6 +105,7 @@ def stream_openai(cfg, prompt, gen, log, should_stop=None):
first_token_at = None first_token_at = None
output_chars = 0 output_chars = 0
prompt_tokens = output_tokens = cached_tokens = 0 prompt_tokens = output_tokens = cached_tokens = 0
event_count = 0
resp = None resp = None
try: try:
while True: while True:
@@ -126,13 +129,15 @@ def stream_openai(cfg, prompt, gen, log, should_stop=None):
for obj in _iter_json(resp): for obj in _iter_json(resp):
if should_stop and should_stop(): if should_stop and should_stop():
raise StopRequested() raise StopRequested()
event_count += 1
if obj.get("choices"): if obj.get("choices"):
delta = obj["choices"][0].get("delta") or {} delta = obj["choices"][0].get("delta") or {}
text = delta.get("content") or "" # 兼容推理型模型:Qwen3/DeepSeek 思维链在 reasoning_content
if text: piece = delta.get("content") or delta.get("reasoning_content") or ""
if piece:
if first_token_at is None: if first_token_at is None:
first_token_at = time.time() first_token_at = time.time()
output_chars += len(text) output_chars += len(piece)
usage = obj.get("usage") usage = obj.get("usage")
if usage: if usage:
prompt_tokens = usage.get("prompt_tokens") or 0 prompt_tokens = usage.get("prompt_tokens") or 0
@@ -148,8 +153,8 @@ def stream_openai(cfg, prompt, gen, log, should_stop=None):
if resp is not None: if resp is not None:
resp.close() resp.close()
if first_token_at is None: if event_count == 0:
raise ProviderError("未收到任何输出内容") raise ProviderError("未收到任何输出内容HTTP 200 但响应流为空)")
end = time.time() end = time.time()
return _metrics(start, first_token_at, end, prompt_tokens, output_tokens, return _metrics(start, first_token_at, end, prompt_tokens, output_tokens,
cached_tokens, output_chars, len(prompt)) cached_tokens, output_chars, len(prompt))
@@ -190,6 +195,7 @@ def stream_anthropic(cfg, prompt, gen, log, should_stop=None):
first_token_at = None first_token_at = None
output_chars = 0 output_chars = 0
prompt_tokens = output_tokens = 0 prompt_tokens = output_tokens = 0
event_count = 0
resp = None resp = None
try: try:
if should_stop and should_stop(): if should_stop and should_stop():
@@ -204,12 +210,15 @@ def stream_anthropic(cfg, prompt, gen, log, should_stop=None):
for obj in _iter_json(resp): for obj in _iter_json(resp):
if should_stop and should_stop(): if should_stop and should_stop():
raise StopRequested() raise StopRequested()
event_count += 1
etype = obj.get("type") etype = obj.get("type")
if etype == "message_start": if etype == "message_start":
usage = (obj.get("message") or {}).get("usage") or {} usage = (obj.get("message") or {}).get("usage") or {}
prompt_tokens = usage.get("input_tokens") or 0 prompt_tokens = usage.get("input_tokens") or 0
elif etype == "content_block_delta": elif etype == "content_block_delta":
text = (obj.get("delta") or {}).get("text") or "" delta = obj.get("delta") or {}
# 兼容 extended thinkingthinking 文本也算输出
text = delta.get("text") or delta.get("thinking") or ""
if text: if text:
if first_token_at is None: if first_token_at is None:
first_token_at = time.time() first_token_at = time.time()
@@ -225,8 +234,8 @@ def stream_anthropic(cfg, prompt, gen, log, should_stop=None):
if resp is not None: if resp is not None:
resp.close() resp.close()
if first_token_at is None: if event_count == 0:
raise ProviderError("未收到任何输出内容") raise ProviderError("未收到任何输出内容HTTP 200 但响应流为空)")
end = time.time() end = time.time()
return _metrics(start, first_token_at, end, prompt_tokens, output_tokens, return _metrics(start, first_token_at, end, prompt_tokens, output_tokens,
0, output_chars, len(prompt)) 0, output_chars, len(prompt))
@@ -252,6 +261,7 @@ def stream_google(cfg, prompt, gen, log, should_stop=None):
first_token_at = None first_token_at = None
output_chars = 0 output_chars = 0
prompt_tokens = output_tokens = cached_tokens = 0 prompt_tokens = output_tokens = cached_tokens = 0
event_count = 0
resp = None resp = None
try: try:
if should_stop and should_stop(): if should_stop and should_stop():
@@ -266,11 +276,13 @@ def stream_google(cfg, prompt, gen, log, should_stop=None):
for obj in _iter_json(resp): for obj in _iter_json(resp):
if should_stop and should_stop(): if should_stop and should_stop():
raise StopRequested() raise StopRequested()
event_count += 1
cands = obj.get("candidates") or [] cands = obj.get("candidates") or []
if cands: if cands:
parts = (cands[0].get("content") or {}).get("parts") or [] parts = (cands[0].get("content") or {}).get("parts") or []
for part in parts: for part in parts:
text = part.get("text") or "" # 兼容 thinking 模型:thought 文本也算输出
text = part.get("text") or part.get("thought") or ""
if text: if text:
if first_token_at is None: if first_token_at is None:
first_token_at = time.time() first_token_at = time.time()
@@ -288,8 +300,8 @@ def stream_google(cfg, prompt, gen, log, should_stop=None):
if resp is not None: if resp is not None:
resp.close() resp.close()
if first_token_at is None: if event_count == 0:
raise ProviderError("未收到任何输出内容") raise ProviderError("未收到任何输出内容HTTP 200 但响应流为空)")
end = time.time() end = time.time()
return _metrics(start, first_token_at, end, prompt_tokens, output_tokens, return _metrics(start, first_token_at, end, prompt_tokens, output_tokens,
cached_tokens, output_chars, len(prompt)) cached_tokens, output_chars, len(prompt))
+16
View File
@@ -184,6 +184,10 @@ table.history { width: 100%; border-collapse: collapse; font-size: 13px; }
.modal-body { padding: 16px 18px; overflow-y: auto; } .modal-body { padding: 16px 18px; overflow-y: auto; }
.modal-body h3 { font-size: 14px; margin: 16px 0 8px; color: var(--muted); } .modal-body h3 { font-size: 14px; margin: 16px 0 8px; color: var(--muted); }
.modal-body h3:first-child { margin-top: 0; } .modal-body h3:first-child { margin-top: 0; }
.detail-name {
background: rgba(79,140,255,.12); border: 1px solid rgba(79,140,255,.35); color: var(--accent);
border-radius: 10px; padding: 10px 14px; font-size: 14px; font-weight: 600; margin-bottom: 6px;
}
.kv { display: grid; grid-template-columns: repeat(auto-fill, minmax(180px, 1fr)); gap: 8px; } .kv { display: grid; grid-template-columns: repeat(auto-fill, minmax(180px, 1fr)); gap: 8px; }
.kv .kv-item { background: var(--panel2); border: 1px solid var(--border); border-radius: 8px; padding: 8px 10px; } .kv .kv-item { background: var(--panel2); border: 1px solid var(--border); border-radius: 8px; padding: 8px 10px; }
@@ -202,6 +206,18 @@ table.mini { width: 100%; border-collapse: collapse; font-size: 12.5px; }
.detail-log .ln.ERROR { color: var(--danger); } .detail-log .ln.ERROR { color: var(--danger); }
.detail-log .ln.WARN { color: var(--warn); } .detail-log .ln.WARN { color: var(--warn); }
/* ── 图表(data-chart-tool 折线图) ── */
.chart-block { background: var(--panel2); border: 1px solid var(--border); border-radius: 10px; padding: 12px; }
.chart-toolbar { display: flex; align-items: center; gap: 10px; flex-wrap: wrap; }
.chart-img-wrap { margin-top: 10px; min-height: 60px; display: flex; align-items: center; justify-content: center; border: 1px dashed var(--border); border-radius: 8px; padding: 10px; background: #0a0e17; }
.chart-img-wrap img { max-width: 100%; border-radius: 6px; }
.chart-data-head { margin-top: 10px; font-size: 12px; color: var(--muted); }
.chart-csv {
width: 100%; min-height: 120px; margin-top: 6px; background: #0a0e17;
border: 1px solid var(--border); border-radius: 8px; color: #a8f0d0;
font: 12px/1.7 var(--mono); padding: 10px; resize: vertical; outline: none;
}
pre.json-box { pre.json-box {
background: #0a0e17; border: 1px solid var(--border); border-radius: 8px; background: #0a0e17; border: 1px solid var(--border); border-radius: 8px;
padding: 10px; font: 11.5px/1.6 var(--mono); overflow: auto; max-height: 260px; color: #a8f0d0; padding: 10px; font: 11.5px/1.6 var(--mono); overflow: auto; max-height: 260px; color: #a8f0d0;
+5 -1
View File
@@ -68,6 +68,10 @@
<section class="card"> <section class="card">
<h2>🚀 速度测试配置</h2> <h2>🚀 速度测试配置</h2>
<div class="field">
<label>测试名称(主题)</label>
<input id="gen-name" type="text" placeholder="如:DeepSeek-V4 不同上下文长度速度对比">
</div>
<div class="field"> <div class="field">
<label>上下文长度(tokens,点击切换启停)</label> <label>上下文长度(tokens,点击切换启停)</label>
<div class="chips" id="gen-contexts"></div> <div class="chips" id="gen-contexts"></div>
@@ -136,7 +140,7 @@
<div class="table-wrap"> <div class="table-wrap">
<table class="history" id="history"> <table class="history" id="history">
<thead><tr> <thead><tr>
<th>#</th><th>时间</th><th>提供商</th><th>模型</th><th>采样</th> <th>#</th><th>时间</th><th>名称</th><th>提供商</th><th>模型</th><th>采样</th>
<th>首字 ms</th><th>预填充 tok/s</th><th>解码 tok/s</th><th>状态</th><th>操作</th> <th>首字 ms</th><th>预填充 tok/s</th><th>解码 tok/s</th><th>状态</th><th>操作</th>
</tr></thead> </tr></thead>
<tbody></tbody> <tbody></tbody>
+88 -14
View File
@@ -21,8 +21,8 @@ let pollTimer = null;
let lastLogId = 0; let lastLogId = 0;
let consoleLogs = []; // 当前测试已加载日志 [{id,level,msg,rel}] let consoleLogs = []; // 当前测试已加载日志 [{id,level,msg,rel}]
// 上下文长度:chips 列表 + 启用集合(默认 512/2048/8192/32768/131072 // 上下文长度:chips 列表 + 启用集合(默认 512/2048/4096/8192/16384/32768/65536/131072
const DEFAULT_CONTEXT_LENGTHS = [512, 2048, 8192, 32768, 131072]; const DEFAULT_CONTEXT_LENGTHS = [512, 2048, 4096, 8192, 16384, 32768, 65536, 131072];
let contextLengths = [...DEFAULT_CONTEXT_LENGTHS]; let contextLengths = [...DEFAULT_CONTEXT_LENGTHS];
let contextLengthsActive = new Set(contextLengths); let contextLengthsActive = new Set(contextLengths);
@@ -57,6 +57,7 @@ function currentConfig() {
function currentGen() { function currentGen() {
const lens = contextLengths.filter((l) => contextLengthsActive.has(l)); const lens = contextLengths.filter((l) => contextLengthsActive.has(l));
return { return {
name: $("#gen-name").value.trim(),
context_lengths: lens.length ? lens : [2048], context_lengths: lens.length ? lens : [2048],
max_tokens: parseInt($("#gen-max-tokens").value) || 128, max_tokens: parseInt($("#gen-max-tokens").value) || 128,
samples: parseInt($("#gen-samples").value) || 2, samples: parseInt($("#gen-samples").value) || 2,
@@ -158,7 +159,7 @@ async function testConnection() {
if (r.ok) { if (r.ok) {
const m = r.metrics || {}; const m = r.metrics || {};
showConn(true, showConn(true,
`✅ 连接成功(${r.total_ms}ms| 首字 ${fmt(m.ttft_ms)}ms | 提示词 ${fmt(m.prompt_tokens)} tok | 输出 ${fmt(m.output_tokens)} tok`); `✅ 连接成功(${r.total_ms}ms| 首字 ${fmt(m.ttft_ms)}ms | 提示词 ${fmt(m.prompt_tokens)} tok | 输出 ${fmt(m.output_tokens)} tok${r.note ? " " + r.note : ""}`);
} else { } else {
showConn(false, "❌ " + (r.error || "连接失败")); showConn(false, "❌ " + (r.error || "连接失败"));
} }
@@ -300,7 +301,7 @@ async function loadHistory() {
const tb = $("#history tbody"); const tb = $("#history tbody");
tb.innerHTML = ""; tb.innerHTML = "";
if (!list.length) { if (!list.length) {
tb.innerHTML = '<tr><td colspan="10" style="color:var(--muted);text-align:center">暂无测试记录</td></tr>'; tb.innerHTML = '<tr><td colspan="11" style="color:var(--muted);text-align:center">暂无测试记录</td></tr>';
return; return;
} }
for (const t of list) { for (const t of list) {
@@ -309,6 +310,7 @@ async function loadHistory() {
tr.innerHTML = ` tr.innerHTML = `
<td>#${t.id}</td> <td>#${t.id}</td>
<td>${esc(t.created_at)}</td> <td>${esc(t.created_at)}</td>
<td title="${esc(t.name || "")}">${esc(t.name || "—")}</td>
<td>${esc(PROVIDER_LABEL[t.provider] || t.provider)}</td> <td>${esc(PROVIDER_LABEL[t.provider] || t.provider)}</td>
<td>${esc(t.model)}</td> <td>${esc(t.model)}</td>
<td class="num">${fmt(s.samples_ok)}/${fmt(s.samples_total)}</td> <td class="num">${fmt(s.samples_ok)}/${fmt(s.samples_total)}</td>
@@ -390,28 +392,55 @@ async function viewDetail(id) {
).join("") + `</div>`; ).join("") + `</div>`;
$("#dt-id").textContent = id; $("#dt-id").textContent = id;
const overallRow = (label, avgV, minV, maxV) =>
`<tr><td>${label}</td><td class="num">${fmt(avgV)}</td><td class="num">${fmt(minV)}</td><td class="num">${fmt(maxV)}</td></tr>`;
$("#dt-body").innerHTML = ` $("#dt-body").innerHTML = `
${t.name ? `<div class="detail-name">🏷️ ${esc(t.name)}</div>` : ""}
<h3>📌 汇总指标</h3> <h3>📌 汇总指标</h3>
<div class="kv"> <div class="kv">
<div class="kv-item"><div class="kv-k">状态</div><div class="kv-v">${STATUS_LABEL[t.status] || t.status}</div></div> <div class="kv-item"><div class="kv-k">状态</div><div class="kv-v">${STATUS_LABEL[t.status] || t.status}</div></div>
<div class="kv-item"><div class="kv-k">创建时间</div><div class="kv-v">${esc(t.created_at)}</div></div> <div class="kv-item"><div class="kv-k">创建时间</div><div class="kv-v">${esc(t.created_at)}</div></div>
<div class="kv-item"><div class="kv-k">提供商 / 模型</div><div class="kv-v">${esc(PROVIDER_LABEL[t.provider] || t.provider)} / ${esc(t.model)}</div></div> <div class="kv-item"><div class="kv-k">提供商 / 模型</div><div class="kv-v">${esc(PROVIDER_LABEL[t.provider] || t.provider)} / ${esc(t.model)}</div></div>
<div class="kv-item"><div class="kv-k">采样(成功/总数)</div><div class="kv-v">${fmt(s.samples_ok)} / ${fmt(s.samples_total)}</div></div> <div class="kv-item"><div class="kv-k">采样(成功/总数)</div><div class="kv-v">${fmt(s.samples_ok)} / ${fmt(s.samples_total)}</div></div>
<div class="kv-item"><div class="kv-k">平均首字延迟</div><div class="kv-v">${fmt(s.avg_ttft_ms)} ms</div></div>
<div class="kv-item"><div class="kv-k">最佳首字延迟</div><div class="kv-v">${fmt(s.best_ttft_ms)} ms</div></div>
<div class="kv-item"><div class="kv-k">平均预填充速度</div><div class="kv-v">${fmt(s.avg_prefill_speed)} tok/s</div></div>
<div class="kv-item"><div class="kv-k">平均解码速度</div><div class="kv-v">${fmt(s.avg_decode_speed)} tok/s</div></div>
<div class="kv-item"><div class="kv-k">平均提示词</div><div class="kv-v">${fmt(s.avg_prompt_tokens)} tok</div></div>
<div class="kv-item"><div class="kv-k">平均输出</div><div class="kv-v">${fmt(s.avg_output_tokens)} tok</div></div>
<div class="kv-item"><div class="kv-k">平均缓存命中</div><div class="kv-v">${fmt(s.avg_cached_tokens, 0)} tok</div></div> <div class="kv-item"><div class="kv-k">平均缓存命中</div><div class="kv-v">${fmt(s.avg_cached_tokens, 0)} tok</div></div>
<div class="kv-item"><div class="kv-k">平均总耗时</div><div class="kv-v">${fmt(s.avg_total_ms)} ms</div></div> <div class="kv-item"><div class="kv-k">校准字符/token</div><div class="kv-v">${fmt(s.calibration_chars_per_token)}</div></div>
</div> </div>
<h3>📊 整体统计(平均 / 最小 / 最大)</h3>
<table class="mini"><thead><tr>
<th>指标</th><th>平均</th><th>最小</th><th>最大</th>
</tr></thead><tbody>
${overallRow("首字延迟(ms)", s.avg_ttft_ms, s.min_ttft_ms, s.max_ttft_ms)}
${overallRow("预填充速度(tok/s)", s.avg_prefill_speed, s.min_prefill_speed, s.max_prefill_speed)}
${overallRow("解码速度(tok/s)", s.avg_decode_speed, s.min_decode_speed, s.max_decode_speed)}
${overallRow("提示词(tok)", s.avg_prompt_tokens, null, null)}
${overallRow("输出(tok)", s.avg_output_tokens, null, null)}
${overallRow("总耗时(ms)", s.avg_total_ms, s.min_total_ms, s.max_total_ms)}
</tbody></table>
<h3>📏 按上下文长度汇总</h3> <h3>📏 按上下文长度汇总</h3>
${byLengthHtml} ${byLengthHtml}
<h3>📈 折线图(预填充左轴虚线 / 解码右轴实线)</h3>
<div class="chart-block">
<div class="chart-toolbar">
<button class="btn small primary" id="dt-chart-gen">🎨 生成/刷新折线图</button>
<span class="hint" id="dt-chart-status"></span>
</div>
<div class="chart-img-wrap" id="dt-chart-img">
<div class="hint">点击上方按钮,用 data-chart-tool 生成双Y轴折线图(左轴=预填充虚线,右轴=解码实线)</div>
</div>
<div class="chart-data-head">📋 画图数据(CSV,可一键复制)</div>
<textarea class="chart-csv" id="dt-chart-csv" readonly spellcheck="false" placeholder="(生成图表后自动填充,或手动复制)"></textarea>
<div class="chart-toolbar" style="margin-top:6px">
<button class="btn small" id="dt-chart-copy">📋 复制画图数据</button>
<button class="btn small" id="dt-chart-download">⬇ 下载图表 PNG</button>
</div>
</div>
<h3>⚙️ 测试参数</h3> <h3>⚙️ 测试参数</h3>
<div class="kv"> <div class="kv">
<div class="kv-item"><div class="kv-k">测试名称</div><div class="kv-v">${esc(t.name || "—")}</div></div>
<div class="kv-item"><div class="kv-k">上下文长度</div><div class="kv-v">${(g.context_lengths || []).join(" / ") || "—"} tok</div></div> <div class="kv-item"><div class="kv-k">上下文长度</div><div class="kv-v">${(g.context_lengths || []).join(" / ") || "—"} tok</div></div>
<div class="kv-item"><div class="kv-k">解码输出长度</div><div class="kv-v">${g.max_tokens ?? "—"} tok</div></div> <div class="kv-item"><div class="kv-k">解码输出长度</div><div class="kv-v">${g.max_tokens ?? "—"} tok</div></div>
<div class="kv-item"><div class="kv-k">每个长度采样</div><div class="kv-v">${g.samples ?? "—"}</div></div> <div class="kv-item"><div class="kv-k">每个长度采样</div><div class="kv-v">${g.samples ?? "—"}</div></div>
@@ -419,7 +448,6 @@ async function viewDetail(id) {
<div class="kv-item"><div class="kv-k">避免缓存</div><div class="kv-v">${g.avoid_cache ? "开" : "关"}</div></div> <div class="kv-item"><div class="kv-k">避免缓存</div><div class="kv-v">${g.avoid_cache ? "开" : "关"}</div></div>
<div class="kv-item"><div class="kv-k">温度</div><div class="kv-v">${fmt(cfg.temperature)}</div></div> <div class="kv-item"><div class="kv-k">温度</div><div class="kv-v">${fmt(cfg.temperature)}</div></div>
<div class="kv-item"><div class="kv-k">Base URL</div><div class="kv-v">${esc(cfg.base_url || "(默认)")}</div></div> <div class="kv-item"><div class="kv-k">Base URL</div><div class="kv-v">${esc(cfg.base_url || "(默认)")}</div></div>
<div class="kv-item"><div class="kv-k">校准字符/token</div><div class="kv-v">${fmt(s.calibration_chars_per_token)}</div></div>
</div> </div>
<h3>📊 每次采样明细</h3> <h3>📊 每次采样明细</h3>
@@ -433,13 +461,59 @@ async function viewDetail(id) {
$("#detail-mask").hidden = false; $("#detail-mask").hidden = false;
$("#dt-body").scrollTop = 0; $("#dt-body").scrollTop = 0;
window.__detail = t; window.__detail = t;
// 图表:生成/刷新 + 复制数据 + 下载
const genBtn = $("#dt-chart-gen");
const csvTa = $("#dt-chart-csv");
const imgWrap = $("#dt-chart-img");
const statusEl = $("#dt-chart-status");
let chartUrl = "";
const genChart = () => {
statusEl.textContent = "⏳ 正在生成...";
fetch(`/api/tests/${id}/chart-data`).then((r) => r.json()).then((d) => {
if (!d.ok) { statusEl.textContent = "❌ " + (d.error || "生成失败"); return; }
csvTa.value = d.csv;
chartUrl = `/api/tests/${id}/chart?t=${Date.now()}`;
const img = document.createElement("img");
img.src = chartUrl;
img.alt = "预填充/解码 折线图";
img.style.maxWidth = "100%";
img.onload = () => { imgWrap.innerHTML = ""; imgWrap.appendChild(img); statusEl.textContent = "✅ 生成完成"; };
img.onerror = () => { statusEl.textContent = "❌ 图表生成失败"; };
}).catch((e) => { statusEl.textContent = "❌ " + e.message; });
};
genBtn.addEventListener("click", genChart);
$("#dt-chart-copy").addEventListener("click", () => {
if (!csvTa.value) { toast("暂无画图数据,请先生成图表"); return; }
csvTa.select();
if (navigator.clipboard && navigator.clipboard.writeText) {
navigator.clipboard.writeText(csvTa.value).then(() => toast("画图数据已复制"));
} else {
document.execCommand("copy");
toast("画图数据已复制");
}
});
$("#dt-chart-download").addEventListener("click", () => {
if (!chartUrl) { toast("请先生成图表"); return; }
const a = document.createElement("a");
a.href = chartUrl;
a.download = `llm_speed_chart_${id}.png`;
document.body.appendChild(a); a.click(); a.remove();
});
} }
function closeDetail() { $("#detail-mask").hidden = true; } function closeDetail() { $("#detail-mask").hidden = true; }
function exportDetail() { async function exportDetail() {
if (!window.__detail) return; if (!window.__detail) return;
download(JSON.stringify(window.__detail, null, 2), `test_${window.__detail.id}.json`, "application/json"); try {
const resp = await fetch(`/api/tests/${window.__detail.id}/export.json`);
if (!resp.ok) throw new Error("服务端导出失败");
const j = await resp.json();
download(JSON.stringify(j, null, 2), `test_${j.id}.json`, "application/json");
} catch (e) {
toast("导出失败:" + e.message);
}
} }
async function exportXlsx(id) { async function exportXlsx(id) {
+38 -5
View File
@@ -21,6 +21,7 @@ class TestRunner(threading.Thread):
self.start_wall = time.time() self.start_wall = time.time()
self.ratio = None self.ratio = None
self.samples = [] self.samples = []
self.last_error = None
def request_cancel(self): def request_cancel(self):
self.cancel_flag = True self.cancel_flag = True
@@ -50,7 +51,7 @@ class TestRunner(threading.Thread):
model = self.cfg.get("model", "") model = self.cfg.get("model", "")
gen = self.gen gen = self.gen
# 上下文长度列表(支持手动自定义,默认 512/2048/8192/32768/131072 # 上下文长度列表(支持手动自定义,默认 512/2048/4096/8192/16384/32768/65536/131072
raw_lengths = gen.get("context_lengths") or [] raw_lengths = gen.get("context_lengths") or []
if not raw_lengths: if not raw_lengths:
# 兼容旧版单值配置 # 兼容旧版单值配置
@@ -62,6 +63,9 @@ class TestRunner(threading.Thread):
warmup = bool(gen.get("warmup", True)) # 测试前空转预热 warmup = bool(gen.get("warmup", True)) # 测试前空转预热
self.log("INFO", "═══ 开始速度测试 ═══") self.log("INFO", "═══ 开始速度测试 ═══")
name = gen.get("name") or self.cfg.get("name") or ""
if name:
self.log("INFO", "测试名称(主题): %s" % name)
self.log("INFO", "提供商: %s | 模型: %s" % (lp.PROVIDER_LABELS.get(provider, provider), model)) self.log("INFO", "提供商: %s | 模型: %s" % (lp.PROVIDER_LABELS.get(provider, provider), model))
self.log("INFO", "上下文长度: %s tokens | 生成长度: %d tokens | 每个长度采样: %d 次 | 预热: %s | 避免缓存: %s" self.log("INFO", "上下文长度: %s tokens | 生成长度: %d tokens | 每个长度采样: %d 次 | 预热: %s | 避免缓存: %s"
% (" / ".join(str(x) for x in lengths), max_tokens, n, % (" / ".join(str(x) for x in lengths), max_tokens, n,
@@ -97,14 +101,26 @@ class TestRunner(threading.Thread):
except StopRequested: except StopRequested:
raise raise
except ProviderError as e: except ProviderError as e:
# 单次采样失败:记录并继续后续采样,不让整个测试中断
self.last_error = str(e)
self.log("ERROR", "[%d tok] 采样 %d/%d 失败: %s" % (L, i, n, e)) self.log("ERROR", "[%d tok] 采样 %d/%d 失败: %s" % (L, i, n, e))
self.samples.append({"run_index": i, "context_length": L, "ok": False, "error": str(e)}) self.samples.append({"run_index": i, "context_length": L, "ok": False, "error": str(e)})
db.add_run(self.test_id, i, {}, str(e), context_length=L) db.add_run(self.test_id, i, {}, str(e), context_length=L)
raise e
summary = self._make_summary() summary = self._make_summary()
db.update_status(self.test_id, "done", summary=summary) ok_count = summary.get("samples_ok") or 0
self.log("INFO", "═══ 测试完成 ═══") fail_count = summary.get("samples_total", 0) - ok_count
if ok_count:
db.update_status(self.test_id, "done", summary=summary,
error=("%d 次采样失败:%s" % (fail_count, self.last_error)) if fail_count else "")
self.log("INFO", "═══ 测试完成 ═══")
if fail_count:
self.log("WARN", "%d 次采样失败(最后错误:%s" % (fail_count, self.last_error))
else:
db.update_status(self.test_id, "error", summary=summary,
error=self.last_error or "所有采样均失败")
self.log("ERROR", "所有采样均失败,测试标记为 error(最后错误:%s" % (self.last_error or "未知"))
return
self.log("INFO", "汇总: 平均首字 %.1f ms | 平均预填充 %.1f tok/s | 平均解码 %.1f tok/s" self.log("INFO", "汇总: 平均首字 %.1f ms | 平均预填充 %.1f tok/s | 平均解码 %.1f tok/s"
% (summary.get("avg_ttft_ms") or 0, % (summary.get("avg_ttft_ms") or 0,
summary.get("avg_prefill_speed") or 0, summary.get("avg_prefill_speed") or 0,
@@ -199,16 +215,33 @@ class TestRunner(threading.Thread):
} }
okm = [s["metrics"] for s in ok] okm = [s["metrics"] for s in ok]
def mn(k):
vals = [m[k] for m in okm if m.get(k) is not None]
return round(min(vals), 1) if vals else None
def mx(k):
vals = [m[k] for m in okm if m.get(k) is not None]
return round(max(vals), 1) if vals else None
summary = dict(base) summary = dict(base)
summary.update({ summary.update({
"by_length": by_length, "by_length": by_length,
"avg_ttft_ms": avg(okm, "ttft_ms"), "avg_ttft_ms": avg(okm, "ttft_ms"),
"min_ttft_ms": mn("ttft_ms"),
"max_ttft_ms": mx("ttft_ms"),
"avg_prefill_speed": avg(okm, "prefill_speed"), "avg_prefill_speed": avg(okm, "prefill_speed"),
"min_prefill_speed": mn("prefill_speed"),
"max_prefill_speed": mx("prefill_speed"),
"avg_decode_speed": avg(okm, "decode_speed"), "avg_decode_speed": avg(okm, "decode_speed"),
"min_decode_speed": mn("decode_speed"),
"max_decode_speed": mx("decode_speed"),
"avg_prompt_tokens": avg(okm, "prompt_tokens"), "avg_prompt_tokens": avg(okm, "prompt_tokens"),
"avg_output_tokens": avg(okm, "output_tokens"), "avg_output_tokens": avg(okm, "output_tokens"),
"avg_cached_tokens": avg(okm, "cached_tokens"), "avg_cached_tokens": avg(okm, "cached_tokens"),
"avg_total_ms": avg(okm, "total_ms"), "avg_total_ms": avg(okm, "total_ms"),
"best_ttft_ms": min([m["ttft_ms"] for m in okm if m.get("ttft_ms") is not None], default=None), "min_total_ms": mn("total_ms"),
"max_total_ms": mx("total_ms"),
"best_ttft_ms": mn("ttft_ms"),
}) })
return summary return summary