Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 hz4th_coder
|
2d889c80e0 | ||
|
hz4th_coder
|
577fbf69e3 | ||
|
hz4th_coder
|
15cd976fe1 | ||
|
hz4th_coder
|
5de118a7b5 |
@@ -0,0 +1,312 @@
|
||||
# LLM 速度测试台 · API 文档
|
||||
|
||||
> 系统所有页面功能均通过 REST API 提供，前端（网页）只是这些 API 的一个可视化客户端。
|
||||
> 任何页面可见/可操作的数据都可以通过下面的接口调用或访问。
|
||||
|
||||
- **服务地址：** `http://<IP>:16097`
|
||||
- **数据格式：** 请求/响应均为 `application/json`（导出类接口除外）
|
||||
- **鉴权：** 内部工具，当前无鉴权；如需对外暴露，建议在网关/Nginx 层加访问控制
|
||||
- **测试启动为异步**：`POST /api/tests` 返回后，任务在后台线程执行，用 `GET /api/tests/<id>` 或日志轮询接口跟踪进度
|
||||
|
||||
---
|
||||
|
||||
## 目录
|
||||
|
||||
1. [健康检查](#1-健康检查)
|
||||
2. [提供商配置](#2-提供商配置-configs)
|
||||
3. [连接测试](#3-连接测试)
|
||||
4. [速度测试](#4-速度测试-tests)
|
||||
5. [测试详情与日志](#5-测试详情与日志)
|
||||
6. [折线图（data-chart-tool）](#6-折线图data-chart-tool)
|
||||
7. [导出（Excel / JSON）](#7-导出excel--json)
|
||||
8. [数据模型](#8-数据模型)
|
||||
9. [curl 使用示例](#9-curl-使用示例)
|
||||
|
||||
---
|
||||
|
||||
## 1. 健康检查
|
||||
|
||||
### `GET /api/health`
|
||||
返回服务状态与正在运行的测试。
|
||||
|
||||
**响应：**
|
||||
```json
|
||||
{ "ok": true, "port": 16097, "running_tests": [] }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. 提供商配置（Configs）
|
||||
|
||||
### `GET /api/configs`
|
||||
列出所有已保存配置（不返回 API Key 明文，仅 `has_key` 标记）。
|
||||
|
||||
**响应：**
|
||||
```json
|
||||
[
|
||||
{ "id": 2, "name": "epyc-test", "provider": "openai",
|
||||
"base_url": "http://121.40.164.32:18003/v1",
|
||||
"model": "unsloth/Qwen3.8-27B-Q4_K_M", "temperature": 0.7, "has_key": 1 }
|
||||
]
|
||||
```
|
||||
|
||||
### `POST /api/configs`
|
||||
新增配置。请求体字段：`name`(必填), `provider`, `base_url`, `api_key`, `model`, `temperature`。
|
||||
|
||||
**请求：**
|
||||
```json
|
||||
{ "name": "DeepSeek", "provider": "openai",
|
||||
"base_url": "https://api.deepseek.com/v1",
|
||||
"api_key": "sk-xxx", "model": "deepseek-chat", "temperature": 0.7 }
|
||||
```
|
||||
**响应：** `{ "ok": true, "id": 3 }`
|
||||
|
||||
### `GET /api/configs/<id>`
|
||||
获取单个配置（**含 API Key 明文**，用于前端回填；注意保管）。
|
||||
|
||||
### `PUT /api/configs/<id>`
|
||||
更新配置，**局部更新**（只改请求里出现的字段）。
|
||||
|
||||
**请求：** `{ "model": "deepseek-v4-flash" }` → 响应 `{ "ok": true, "id": 3 }`
|
||||
|
||||
### `DELETE /api/configs/<id>`
|
||||
删除配置。响应 `{ "ok": true }`
|
||||
|
||||
---
|
||||
|
||||
## 3. 连接测试
|
||||
|
||||
### `POST /api/configs/test`
|
||||
验证 API Key / Base URL / 模型名连通性。只要流式请求成功返回（HTTP 200 + 收到响应流）即视为连通；支持推理型模型（Qwen3/DeepSeek 思维链）。
|
||||
|
||||
**请求：** 同配置对象（`provider`, `base_url`, `api_key`, `model`, `temperature`）
|
||||
|
||||
**成功响应：**
|
||||
```json
|
||||
{
|
||||
"ok": true, "total_ms": 1308.2,
|
||||
"note": "",
|
||||
"metrics": { "prompt_tokens": 63, "output_tokens": 32, "output_chars": 110,
|
||||
"ttft_ms": 715.9, "prefill_speed": 86.7, "decode_speed": 52.7, "total_ms": 1308.2 }
|
||||
}
|
||||
```
|
||||
> `note` 非空表示连接正常但未返回正文（可能为只输出思维链的模型）。
|
||||
|
||||
**失败响应：** `{ "ok": false, "error": "HTTP 401: ..." }`
|
||||
|
||||
---
|
||||
|
||||
## 4. 速度测试（Tests）
|
||||
|
||||
### `POST /api/tests`
|
||||
启动一次速度测试（异步，立即返回测试 id）。
|
||||
|
||||
**请求体：**
|
||||
```json
|
||||
{
|
||||
"config": {
|
||||
"provider": "openai",
|
||||
"name": "Qwen3 对比",
|
||||
"base_url": "http://121.40.164.32:18003/v1",
|
||||
"api_key": "sk-xxx",
|
||||
"model": "unsloth/Qwen3.8-27B-Q4_K_M",
|
||||
"temperature": 0.7
|
||||
},
|
||||
"gen": {
|
||||
"name": "Qwen3 不同上下文长度速度对比",
|
||||
"context_lengths": [512, 2048, 4096, 8192, 16384, 32768, 65536, 131072],
|
||||
"max_tokens": 128,
|
||||
"samples": 2,
|
||||
"warmup": true,
|
||||
"avoid_cache": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**gen 字段说明：**
|
||||
|
||||
| 字段 | 类型 | 默认 | 说明 |
|
||||
|------|------|------|------|
|
||||
| `name` | string | `""` | 测试名称/主题（会存入测试记录并展示在历史与详情） |
|
||||
| `context_lengths` | number[] | `[512,2048,4096,8192,16384,32768,65536,131072]` | 要测试的上下文长度列表，每个长度独立校准+预热+采样 |
|
||||
| `max_tokens` | number | `128` | 解码输出 token 长度 |
|
||||
| `samples` | number | `2` | 每个上下文长度的采样次数 |
|
||||
| `warmup` | bool | `true` | 测试前空转预热（不计速度） |
|
||||
| `avoid_cache` | bool | `true` | 随机前缀避免缓存命中 |
|
||||
|
||||
**响应：** `{ "ok": true, "id": 9 }`
|
||||
|
||||
### `GET /api/tests?limit=<n>`
|
||||
测试历史列表（按 id 倒序）。`limit` 默认 100，最大 1000。
|
||||
|
||||
**响应：**
|
||||
```json
|
||||
[
|
||||
{ "id": 9, "created_at": "2026-08-23 18:52:00", "status": "done",
|
||||
"provider": "openai", "model": "unsloth/Qwen3.8-27B-Q4_K_M",
|
||||
"name": "Qwen3 不同上下文长度速度对比",
|
||||
"error": "",
|
||||
"summary": { "samples_ok": 2, "samples_total": 2, "avg_ttft_ms": 1808.7, ... } }
|
||||
]
|
||||
```
|
||||
|
||||
### `POST /api/tests/<id>/cancel`
|
||||
停止正在运行的测试。响应 `{ "ok": true, "msg": "正在停止..." }`
|
||||
|
||||
### `DELETE /api/tests/<id>`
|
||||
删除测试及其全部采样与日志。响应 `{ "ok": true }`
|
||||
|
||||
---
|
||||
|
||||
## 5. 测试详情与日志
|
||||
|
||||
### `GET /api/tests/<id>`
|
||||
完整测试详情：基本信息 + 配置（API Key 已打码）+ 生成参数 + 汇总 + 每次采样 + 完整日志。
|
||||
|
||||
**响应结构：**
|
||||
```json
|
||||
{
|
||||
"id": 9, "created_at": "...", "status": "done",
|
||||
"provider": "openai", "model": "...", "name": "...", "error": "",
|
||||
"config": { "base_url": "...", "api_key": "sk-x****", ... },
|
||||
"gen": { "name": "...", "context_lengths": [512, 2048], "max_tokens": 128, "samples": 1, "warmup": true, "avoid_cache": true },
|
||||
"summary": {
|
||||
"samples_total": 2, "samples_ok": 2,
|
||||
"calibration_chars_per_token": 1.82,
|
||||
"avg_ttft_ms": 1808.7, "min_ttft_ms": 1122.8, "max_ttft_ms": 2494.6,
|
||||
"avg_prefill_speed": 694.2, "min_prefill_speed": 515.7, "max_prefill_speed": 872.7,
|
||||
"avg_decode_speed": 54.7, "min_decode_speed": 54.4, "max_decode_speed": 55.0,
|
||||
"avg_prompt_tokens": 1378.0, "avg_output_tokens": 128.0,
|
||||
"avg_total_ms": 4148.4, "min_total_ms": 3449.9, "max_total_ms": 4846.9,
|
||||
"by_length": {
|
||||
"512": { "samples_total": 1, "samples_ok": 1, "avg_ttft_ms": 1122.8, "avg_prefill_speed": 515.7, "avg_decode_speed": 55.0, "avg_prompt_tokens": 579, "avg_output_tokens": 128, "avg_total_ms": 3449.9 },
|
||||
"2048": { "samples_total": 1, "samples_ok": 1, "avg_ttft_ms": 2494.6, "avg_prefill_speed": 872.7, "avg_decode_speed": 54.4, "avg_prompt_tokens": 2177, "avg_output_tokens": 128, "avg_total_ms": 4846.9 }
|
||||
}
|
||||
},
|
||||
"runs": [
|
||||
{ "run_index": 1, "context_length": 512,
|
||||
"metrics": { "prompt_tokens": 579, "output_tokens": 128, "cached_tokens": 0,
|
||||
"ttft_ms": 1122.8, "prefill_speed": 515.7, "decode_speed": 55.0, "total_ms": 3449.9 },
|
||||
"error": "" }
|
||||
],
|
||||
"logs": [ { "id": 1, "level": "INFO", "msg": "═══ 开始速度测试 ═══", "rel": 0.0, "ts": "..." } ]
|
||||
}
|
||||
```
|
||||
|
||||
### `GET /api/tests/<id>/logs?after=<id>`
|
||||
增量日志（前端轮询用）。`after` 为上次取到的最大日志 id，返回其后新增日志 + 最新状态/汇总/最后采样。
|
||||
|
||||
**响应：**
|
||||
```json
|
||||
{
|
||||
"status": "running", "error": "",
|
||||
"summary": {},
|
||||
"last_run": { "...": "..." },
|
||||
"logs": [ { "id": 68, "level": "METRIC", "msg": "...", "rel": 3.21 } ],
|
||||
"after": 73
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. 折线图（data-chart-tool）
|
||||
|
||||
> 调用 [data-chart-tool](http://192.168.2.8:12007/hz4th_coder/data-chart-tool.git) 的 `/api/chart` 接口生成**双Y轴折线图**：左轴=预填充速度（虚线）、右轴=解码速度（实线）、X 轴=上下文长度。本系统在服务端代理转发，前端只需请求本服务的两个接口。
|
||||
|
||||
### `GET /api/tests/<id>/chart`
|
||||
生成折线图 PNG 图片（内部调用 data-chart-tool `/api/chart`，地址可在 `config.py` 的 `CHART_API_BASE` 修改）。
|
||||
|
||||
**成功响应：** `Content-Type: image/png`（可直接用于 `<img>` / 下载）
|
||||
|
||||
**失败响应：**
|
||||
```json
|
||||
{ "ok": false, "error": "无成功采样数据，无法画图" } // 400
|
||||
{ "ok": false, "error": "测试不存在" } // 404
|
||||
{ "ok": false, "error": "图表服务不可用: ..." } // 502
|
||||
```
|
||||
|
||||
### `GET /api/tests/<id>/chart-data`
|
||||
返回画图所用的 CSV 数据与图表请求配置，方便手动快速复制。
|
||||
|
||||
**响应：**
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"csv": "上下文长度, 预填充速度(tok/s), 解码速度(tok/s)\n4096, 126.90, 22.80\n8192, 132.20, 22.40",
|
||||
"rows": [[4096, 126.9, 22.8], [8192, 132.2, 22.4]],
|
||||
"payload": {
|
||||
"data": "...", "chartType": "line", "title": "...",
|
||||
"dualYAxis": true,
|
||||
"leftAxisName": "预填充速度(tok/s)", "rightAxisName": "解码速度(tok/s)",
|
||||
"seriesTypes": ["line", "line"], "seriesAxis": [0, 1],
|
||||
"seriesStyles": ["dashed", "solid"], "width": 1000, "height": 560, "pixelRatio": 2
|
||||
}
|
||||
}
|
||||
```
|
||||
> `csv` 即画图数据（第一列=上下文长度，第二列=预填充速度，第三列=解码速度），前端「复制画图数据」按钮复制的就是它。
|
||||
|
||||
---
|
||||
|
||||
## 7. 导出（Excel / JSON）
|
||||
|
||||
### `GET /api/tests/<id>/export.xlsx`
|
||||
导出 Excel 报告（**3 个 Sheet**：汇总 / 采样明细 / 日志），`Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet`。
|
||||
|
||||
- **汇总**：测试信息 + 整体统计指标（平均/最大/最小）+ 按上下文长度分组
|
||||
- **采样明细**：每次采样的上下文长度与全部指标
|
||||
- **日志**：完整测试日志
|
||||
|
||||
### `GET /api/tests/<id>/export.json`
|
||||
导出完整测试数据为 JSON（与 `GET /api/tests/<id>` 一致，API Key 打码），`Content-Type: application/json`。
|
||||
|
||||
---
|
||||
|
||||
## 8. 数据模型
|
||||
|
||||
| 表 | 说明 | 关键字段 |
|
||||
|----|------|----------|
|
||||
| `configs` | 保存的接口配置 | id, name, provider, base_url, api_key, model, temperature |
|
||||
| `tests` | 测试记录 | id, status(running/done/error/canceled), provider, model, **name**, config_json, gen_cfg_json, summary_json, error |
|
||||
| `test_runs` | 每次采样 | id, test_id, run_index, **context_length**, metrics_json, error |
|
||||
| `logs` | 测试日志 | id, test_id, level, msg, rel |
|
||||
|
||||
**summary 整体指标字段：**
|
||||
`avg_/min_/max_` 前缀 × `ttft_ms` / `prefill_speed` / `decode_speed` / `total_ms`，以及 `avg_prompt_tokens` / `avg_output_tokens` / `avg_cached_tokens` / `best_ttft_ms`（= min_ttft_ms）。
|
||||
|
||||
---
|
||||
|
||||
## 9. curl 使用示例
|
||||
|
||||
```bash
|
||||
BASE=http://<IP>:16097
|
||||
|
||||
# 健康检查
|
||||
curl $BASE/api/health
|
||||
|
||||
# 新增配置
|
||||
curl -X POST $BASE/api/configs -H 'Content-Type: application/json' \
|
||||
-d '{"name":"Qwen3","provider":"openai","base_url":"http://121.40.164.32:18003/v1","api_key":"sk-xxx","model":"unsloth/Qwen3.8-27B-Q4_K_M","temperature":0.7}'
|
||||
|
||||
# 测试连接
|
||||
curl -X POST $BASE/api/configs/test -H 'Content-Type: application/json' \
|
||||
-d '{"provider":"openai","base_url":"http://121.40.164.32:18003/v1","api_key":"sk-xxx","model":"unsloth/Qwen3.8-27B-Q4_K_M"}'
|
||||
|
||||
# 启动速度测试（异步）
|
||||
curl -X POST $BASE/api/tests -H 'Content-Type: application/json' -d '{
|
||||
"config": {"provider":"openai","base_url":"http://121.40.164.32:18003/v1","api_key":"sk-xxx","model":"unsloth/Qwen3.8-27B-Q4_K_M"},
|
||||
"gen": {"name":"各长度对比","context_lengths":[512,2048,8192],"max_tokens":128,"samples":2,"warmup":true,"avoid_cache":true}
|
||||
}'
|
||||
|
||||
# 查询测试列表 / 详情
|
||||
curl "$BASE/api/tests?limit=10"
|
||||
curl $BASE/api/tests/9
|
||||
|
||||
# 画图数据（CSV）
|
||||
curl $BASE/api/tests/9/chart-data
|
||||
# 折线图 PNG（预填充左轴虚线 / 解码右轴实线）
|
||||
curl -o chart.png $BASE/api/tests/9/chart
|
||||
|
||||
# 导出
|
||||
curl -OJ $BASE/api/tests/9/export.xlsx
|
||||
curl $BASE/api/tests/9/export.json
|
||||
```
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
- **访问地址：** `http://<IP>:16097/`
|
||||
- **技术栈：** Python 3 + Flask + SQLite（纯 REST，无额外依赖）
|
||||
- **版本：** v2.0.0
|
||||
- **版本：** v2.2.0
|
||||
|
||||
---
|
||||
|
||||
@@ -16,7 +16,8 @@
|
||||
- 配置可**保存/加载/删除**，方便多模型对比
|
||||
|
||||
### 🚀 速度测试配置
|
||||
- **多上下文长度测试**：默认为 `512 / 2048 / 8192 / 32768 / 131072` tokens，可通过标签点击启用/禁用，并支持手动添加任意自定义长度（≥16）
|
||||
- **测试名称（主题）**：可为每次测试命名，用于标注测试内容/主题，展示在历史列表、详情弹窗与导出报表中
|
||||
- **多上下文长度测试**：默认为 `512 / 2048 / 4096 / 8192 / 16384 / 32768 / 65536 / 131072` tokens，可通过标签点击启用/禁用，并支持手动添加任意自定义长度（≥16）
|
||||
- **解码输出长度（max tokens）**：默认为 `128`，可手动自定义
|
||||
- **每个长度采样次数**：默认为 `2`，可手动自定义
|
||||
- **测试前预热（空转）**：默认开启，先发一次不计速度的空转请求，避免冷启动/首请求偏慢污染真实采样数据
|
||||
@@ -24,11 +25,20 @@
|
||||
|
||||
### 📊 指标与结果
|
||||
- 实时指标卡：首字延迟、预填充速度、解码速度、上文/输出 tokens、总耗时
|
||||
- **整体统计（平均/最小/最大）**：详情弹窗与 Excel 汇总展示首字延迟、预填充速度、解码速度、总耗时的平均/最小/最大
|
||||
- **推理型模型兼容**：支持 Qwen3 / DeepSeek 等思维链模型（`reasoning_content` / `thinking` / `thought`），思维过程计入输出，不会误报“未收到输出”
|
||||
- **采样失败不中断**：单次采样失败会记录并继续，不会让整个测试半途终止；全部失败才标记 error
|
||||
- 实时控制台日志：校准、预热、每次采样明细全程可追溯
|
||||
- **每次完整测试**支持：
|
||||
- **网页点击查看**：历史记录「查看」按钮弹出详情（整体平均 + 按上下文长度分组 + 每次采样明细 + 完整日志）
|
||||
- **网页点击查看**：历史记录「查看」按钮弹出详情（整体统计 + 按上下文长度分组 + 每次采样明细 + 完整日志）
|
||||
- **📈 折线图（data-chart-tool）**：详情弹窗内点击「生成/刷新折线图」，调用 data-chart-tool 的 `/api/chart` 画**双Y轴折线图**——左轴=预填充速度（虚线），右轴=解码速度（实线），X 轴为上下文长度；并提供**画图 CSV 数据一键复制 / 下载 PNG**
|
||||
- **文件下载 Excel（xlsx）**：历史记录「Excel」按钮或详情弹窗「导出 Excel」，包含 汇总 / 采样明细 / 日志 三个 Sheet
|
||||
- 测试历史留存，可随时刷新、查看、导出、删除
|
||||
- **文件下载 JSON**：详情弹窗「导出 JSON」
|
||||
- 测试历史留存（含测试名称），可随时刷新、查看、导出、删除
|
||||
|
||||
### 🔌 开放 API
|
||||
- 页面所有功能均通过 REST API 提供，前端只是可视化客户端
|
||||
- **API 文档见 [API.md](API.md)**
|
||||
|
||||
---
|
||||
|
||||
@@ -79,14 +89,19 @@ pip install -r requirements.txt
|
||||
|------|------|------|
|
||||
| GET | `/api/health` | 健康检查 |
|
||||
| GET/POST | `/api/configs` | 配置列表 / 新增配置 |
|
||||
| GET/DELETE | `/api/configs/<id>` | 单个配置 / 删除 |
|
||||
| GET/PUT/DELETE | `/api/configs/<id>` | 单个配置 / 更新（局部）/ 删除 |
|
||||
| POST | `/api/configs/test` | 测试连接 |
|
||||
| GET/POST | `/api/tests` | 测试历史 / 启动测试 |
|
||||
| GET | `/api/tests/<id>` | 测试详情（含 runs / logs / summary） |
|
||||
| GET/POST | `/api/tests?limit=n` | 测试历史 / 启动测试（异步） |
|
||||
| GET | `/api/tests/<id>` | 测试详情（含 runs / logs / summary / 按长度分组） |
|
||||
| GET | `/api/tests/<id>/logs?after=<id>` | 增量日志（前端轮询用） |
|
||||
| POST | `/api/tests/<id>/cancel` | 停止测试 |
|
||||
| DELETE | `/api/tests/<id>` | 删除测试 |
|
||||
| GET | `/api/tests/<id>/export.xlsx` | 导出 Excel 报告 |
|
||||
| GET | `/api/tests/<id>/export.xlsx` | 导出 Excel 报告（汇总/采样明细/日志三 Sheet） |
|
||||
| GET | `/api/tests/<id>/export.json` | 导出完整测试 JSON |
|
||||
| GET | `/api/tests/<id>/chart` | 用 data-chart-tool 生成折线图 PNG（预填充左轴虚线 / 解码右轴实线） |
|
||||
| GET | `/api/tests/<id>/chart-data` | 画图数据（CSV + 图表请求配置，供一键复制） |
|
||||
|
||||
> 完整字段说明、响应示例与 curl 示例见 **API.md**。
|
||||
|
||||
### 启动测试请求示例
|
||||
```json
|
||||
@@ -101,7 +116,7 @@ POST /api/tests
|
||||
"temperature": 0.7
|
||||
},
|
||||
"gen": {
|
||||
"context_lengths": [512, 2048, 8192, 32768, 131072],
|
||||
"context_lengths": [512, 2048, 4096, 8192, 16384, 32768, 65536, 131072],
|
||||
"max_tokens": 128,
|
||||
"samples": 2,
|
||||
"warmup": true,
|
||||
@@ -139,13 +154,14 @@ llm-speed-tester/
|
||||
|
||||
## 常见问题
|
||||
|
||||
- **连接测试提示“未收到任何输出内容”**：多为推理型模型（Qwen3/DeepSeek 思维链）或只返回 usage 的网关。已兼容 `reasoning_content` 等思维字段，连接成功即视为通过；若仍出现，请检查 API Key/Base URL/模型名。
|
||||
- **无 openpyxl**：`pip install openpyxl`（已加入 requirements.txt）
|
||||
- **老版本数据库**：程序启动时自动迁移，为 `test_runs` 表补充 `context_length` 列，无需手动处理
|
||||
- **慢模型超时**：连接超时 30s、两次数据包间隔 120s，足够覆盖大多数慢模型；超长文（131072）生成慢属正常，请耐心等待
|
||||
- **慢模型/长上文超时**：连接超时 60s、两次数据包间隔 300s（`config.STREAM_READ_TIMEOUT` 可调）；推理型模型思考阶段停顿不计超时，超长文（131072）生成慢属正常，请耐心等待
|
||||
|
||||
---
|
||||
|
||||
## Git
|
||||
|
||||
- **仓库：** `hz4th_coder/llm-speed-tester`
|
||||
- **版本：** v2.0.0（新增多上下文长度测试 + 预热 + Excel 导出 + 界面优化）
|
||||
- **版本：** v2.2.0（多上下文长度测试 + 预热 + Excel/JSON 导出 + 测试名称 + 整体统计平均/最小/最大 + 开放 API + 推理型模型兼容 + 📈 data-chart-tool 双Y轴折线图/画图数据复制 + 默认上下文长度增加 4096/16384/65536）
|
||||
@@ -3,6 +3,7 @@
|
||||
import io
|
||||
import json
|
||||
|
||||
import requests
|
||||
from flask import Flask, jsonify, request, send_file, send_from_directory
|
||||
|
||||
import config
|
||||
@@ -62,6 +63,16 @@ def get_one_config(cid):
|
||||
return jsonify(c)
|
||||
|
||||
|
||||
@app.route("/api/configs/<int:cid>", methods=["PUT"])
|
||||
def update_config(cid):
|
||||
cfg = request.get_json(force=True) or {}
|
||||
old = db.get_config(cid)
|
||||
if not old:
|
||||
return jsonify({"ok": False, "error": "配置不存在"}), 404
|
||||
db.update_config(cid, cfg)
|
||||
return jsonify({"ok": True, "id": cid})
|
||||
|
||||
|
||||
@app.route("/api/configs/<int:cid>", methods=["DELETE"])
|
||||
def del_config(cid):
|
||||
db.delete_config(cid)
|
||||
@@ -74,8 +85,13 @@ def test_config():
|
||||
if not cfg.get("api_key"):
|
||||
return jsonify({"ok": False, "error": "请填写 API Key"}), 400
|
||||
try:
|
||||
m = call_stream(cfg, "你好，请只回复：OK", {"max_tokens": 16, "avoid_cache": False})
|
||||
return jsonify({"ok": True, "total_ms": m["total_ms"], "metrics": m})
|
||||
# 连接测试：只要流式请求成功返回（哪怕正文为空/只有思维链）都算连通
|
||||
m = call_stream(cfg, "你好，请简要回答：1+1=？",
|
||||
{"max_tokens": 32, "avoid_cache": False})
|
||||
note = ""
|
||||
if not (m.get("output_tokens") or m.get("output_chars")):
|
||||
note = "（连接正常，但本次未返回正文内容，可能为推理型模型）"
|
||||
return jsonify({"ok": True, "total_ms": m["total_ms"], "metrics": m, "note": note})
|
||||
except ProviderError as e:
|
||||
return jsonify({"ok": False, "error": str(e)})
|
||||
except Exception as e:
|
||||
@@ -102,7 +118,11 @@ def start_test():
|
||||
|
||||
@app.route("/api/tests", methods=["GET"])
|
||||
def list_tests():
|
||||
return jsonify(db.list_tests())
|
||||
try:
|
||||
limit = int(request.args.get("limit", 100))
|
||||
except ValueError:
|
||||
limit = 100
|
||||
return jsonify(db.list_tests(max(1, min(limit, 1000))))
|
||||
|
||||
|
||||
@app.route("/api/tests/<int:tid>", methods=["GET"])
|
||||
@@ -112,6 +132,7 @@ def get_test(tid):
|
||||
return jsonify({"ok": False, "error": "测试不存在"}), 404
|
||||
t["runs"] = db.get_runs(tid)
|
||||
t["logs"] = db.get_logs(tid)
|
||||
_mask_cfg(t.get("config"))
|
||||
return jsonify(t)
|
||||
|
||||
|
||||
@@ -140,6 +161,107 @@ def del_test(tid):
|
||||
return jsonify({"ok": True})
|
||||
|
||||
|
||||
# ───────────────────────── 图表（data-chart-tool 折线图：预填充左轴虚线 / 解码右轴实线） ─────────────────────────
|
||||
|
||||
def _build_chart_csv(t):
|
||||
"""由测试汇总 by_length 构建画图 CSV：上下文长度, 预填充速度(tok/s), 解码速度(tok/s)"""
|
||||
s = t.get("summary") or {}
|
||||
by = s.get("by_length") or {}
|
||||
lens = sorted(int(k) for k in by)
|
||||
rows = []
|
||||
for L in lens:
|
||||
bl = by.get(str(L)) if str(L) in by else by.get(L) or {}
|
||||
pre = bl.get("avg_prefill_speed")
|
||||
dec = bl.get("avg_decode_speed")
|
||||
if pre is None or dec is None:
|
||||
continue
|
||||
rows.append([L, round(pre, 2), round(dec, 2)])
|
||||
if not rows:
|
||||
return None
|
||||
csv_lines = ["上下文长度, 预填充速度(tok/s), 解码速度(tok/s)"]
|
||||
for L, pre, dec in rows:
|
||||
csv_lines.append("%d, %.2f, %.2f" % (L, pre, dec))
|
||||
return {"csv": "\n".join(csv_lines), "rows": rows}
|
||||
|
||||
|
||||
def _chart_payload(t, csv_text):
|
||||
"""组装 data-chart-tool /api/chart 请求体（双Y轴折线图）"""
|
||||
title = ("%s %s" % (t.get("model") or "", t.get("name") or "速度对比")).strip()
|
||||
return {
|
||||
"data": csv_text,
|
||||
"chartType": "line",
|
||||
"title": title,
|
||||
"theme": "default",
|
||||
"showLegend": True,
|
||||
"showGrid": True,
|
||||
"showLabel": False,
|
||||
"smoothLine": True,
|
||||
"dualYAxis": True,
|
||||
"leftAxisName": "预填充速度(tok/s)",
|
||||
"rightAxisName": "解码速度(tok/s)",
|
||||
"seriesTypes": ["line", "line"],
|
||||
"seriesAxis": [0, 1],
|
||||
"seriesStyles": ["dashed", "solid"], # 预填充=左轴虚线，解码=右轴实线
|
||||
"width": 1000,
|
||||
"height": 560,
|
||||
"pixelRatio": 2,
|
||||
}
|
||||
|
||||
|
||||
@app.route("/api/tests/<int:tid>/chart-data")
|
||||
def test_chart_data(tid):
|
||||
t = db.get_test(tid)
|
||||
if not t:
|
||||
return jsonify({"ok": False, "error": "测试不存在"}), 404
|
||||
built = _build_chart_csv(t)
|
||||
if not built:
|
||||
return jsonify({"ok": False, "error": "无成功采样数据，无法画图"}), 400
|
||||
return jsonify({
|
||||
"ok": True,
|
||||
"csv": built["csv"],
|
||||
"rows": built["rows"],
|
||||
"payload": _chart_payload(t, built["csv"]),
|
||||
})
|
||||
|
||||
|
||||
@app.route("/api/tests/<int:tid>/chart")
|
||||
def test_chart(tid):
|
||||
"""用 data-chart-tool 生成折线图 PNG（预填充左轴虚线 / 解码右轴实线）"""
|
||||
t = db.get_test(tid)
|
||||
if not t:
|
||||
return jsonify({"ok": False, "error": "测试不存在"}), 404
|
||||
built = _build_chart_csv(t)
|
||||
if not built:
|
||||
return jsonify({"ok": False, "error": "无成功采样数据，无法画图"}), 400
|
||||
try:
|
||||
resp = requests.post(config.CHART_API_BASE + "/api/chart",
|
||||
json=_chart_payload(t, built["csv"]), timeout=60)
|
||||
except requests.RequestException as e:
|
||||
return jsonify({"ok": False, "error": "图表服务不可用: %s" % e}), 502
|
||||
if resp.status_code != 200:
|
||||
return jsonify({"ok": False, "error": "图表生成失败(%d): %s" % (resp.status_code, resp.text[:300])}), 502
|
||||
return send_file(io.BytesIO(resp.content), mimetype="image/png")
|
||||
|
||||
|
||||
@app.route("/api/tests/<int:tid>/export.json")
|
||||
def export_json(tid):
|
||||
t = db.get_test(tid)
|
||||
if not t:
|
||||
return jsonify({"ok": False, "error": "测试不存在"}), 404
|
||||
t["runs"] = db.get_runs(tid)
|
||||
t["logs"] = db.get_logs(tid)
|
||||
_mask_cfg(t.get("config"))
|
||||
return jsonify(t)
|
||||
|
||||
|
||||
def _mask_cfg(cfg):
|
||||
"""对外隐藏 API Key，仅保留前几位便于识别"""
|
||||
if cfg and cfg.get("api_key"):
|
||||
k = cfg["api_key"]
|
||||
cfg["api_key"] = k[:4] + "****" if len(k) > 6 else "****"
|
||||
return cfg
|
||||
|
||||
|
||||
# ───────────────────────── Excel 导出 ─────────────────────────
|
||||
|
||||
@app.route("/api/tests/<int:tid>/export.xlsx")
|
||||
@@ -188,6 +310,7 @@ def _build_xlsx(t):
|
||||
ws.cell(1, 1).font = Font(bold=True, size=14)
|
||||
info = [
|
||||
["测试编号", "#%d" % t["id"]],
|
||||
["测试名称", t.get("name") or "（未命名）"],
|
||||
["创建时间", t.get("created_at", "")],
|
||||
["状态", t.get("status", "")],
|
||||
["提供商", t.get("provider", "")],
|
||||
@@ -204,25 +327,26 @@ def _build_xlsx(t):
|
||||
]
|
||||
for row in info:
|
||||
ws.append(row)
|
||||
ws.cell(14, 1).font = title_font
|
||||
ws.cell(15, 1).font = title_font
|
||||
r0 = len(info) + 2
|
||||
overall = [
|
||||
["平均首字延迟(ms)", s.get("avg_ttft_ms")],
|
||||
["最佳首字延迟(ms)", s.get("best_ttft_ms")],
|
||||
["平均预填充速度(tok/s)", s.get("avg_prefill_speed")],
|
||||
["平均解码速度(tok/s)", s.get("avg_decode_speed")],
|
||||
["平均提示词(tok)", s.get("avg_prompt_tokens")],
|
||||
["平均输出(tok)", s.get("avg_output_tokens")],
|
||||
["平均总耗时(ms)", s.get("avg_total_ms")],
|
||||
["首字延迟(ms)", s.get("avg_ttft_ms"), s.get("max_ttft_ms"), s.get("min_ttft_ms")],
|
||||
["预填充速度(tok/s)", s.get("avg_prefill_speed"), s.get("max_prefill_speed"), s.get("min_prefill_speed")],
|
||||
["解码速度(tok/s)", s.get("avg_decode_speed"), s.get("max_decode_speed"), s.get("min_decode_speed")],
|
||||
["提示词(tok)", s.get("avg_prompt_tokens"), None, None],
|
||||
["输出(tok)", s.get("avg_output_tokens"), None, None],
|
||||
["总耗时(ms)", s.get("avg_total_ms"), s.get("max_total_ms"), s.get("min_total_ms")],
|
||||
]
|
||||
ws.cell(r0, 1, "整体平均指标").font = title_font
|
||||
for i, row in enumerate(overall, start=r0 + 1):
|
||||
ws.append([])
|
||||
ws.cell(r0, 1, "整体统计指标（平均 / 最大 / 最小）").font = title_font
|
||||
for j, c in enumerate(["指标", "平均", "最大", "最小"], start=1):
|
||||
ws.cell(row=r0 + 1, column=j, value=c)
|
||||
style_header(ws, r0 + 1, 4)
|
||||
for i, row in enumerate(overall, start=r0 + 2):
|
||||
for j, v in enumerate(row, start=1):
|
||||
ws.cell(row=i, column=j, value=v)
|
||||
|
||||
# 按上下文长度分组
|
||||
r1 = r0 + len(overall) + 2
|
||||
r1 = r0 + len(overall) + 3
|
||||
ws.cell(r1, 1, "按上下文长度分组").font = title_font
|
||||
cols = ["上下文长度(tok)", "采样(成功/总数)", "首字ms", "预填充tok/s", "解码tok/s", "提示词tok", "输出tok", "总耗时ms"]
|
||||
ws.append([])
|
||||
|
||||
@@ -10,6 +10,9 @@ DATA_DIR = os.path.join(BASE_DIR, "data")
|
||||
LOG_DIR = os.path.join(BASE_DIR, "logs")
|
||||
DB_PATH = os.path.join(DATA_DIR, "llm_speed_tester.db")
|
||||
|
||||
# 流式请求超时：连接 30s，两次数据包间隔最长 120s（慢模型也够用）
|
||||
CONNECT_TIMEOUT = 30
|
||||
STREAM_READ_TIMEOUT = 120
|
||||
# 流式请求超时：连接 60s，两次数据包间隔最长 300s（推理型/长上文模型也够用）
|
||||
CONNECT_TIMEOUT = 60
|
||||
STREAM_READ_TIMEOUT = 300
|
||||
|
||||
# data-chart-tool 图表服务地址（用它的 /api/chart 画折线图：预填充左轴虚线 / 解码右轴实线）
|
||||
CHART_API_BASE = "http://127.0.0.1:16016"
|
||||
+27
-4
@@ -28,6 +28,7 @@ CREATE TABLE IF NOT EXISTS tests(
|
||||
status TEXT DEFAULT 'running',
|
||||
provider TEXT DEFAULT '',
|
||||
model TEXT DEFAULT '',
|
||||
name TEXT DEFAULT '',
|
||||
config_json TEXT DEFAULT '{}',
|
||||
gen_cfg_json TEXT DEFAULT '{}',
|
||||
summary_json TEXT DEFAULT '{}',
|
||||
@@ -67,11 +68,15 @@ def _connect():
|
||||
|
||||
|
||||
def _migrate(conn):
|
||||
"""老库升级：为 test_runs 补 context_length 列"""
|
||||
"""老库升级：test_runs 补 context_length、tests 补 name"""
|
||||
cur = conn.execute("PRAGMA table_info(test_runs)")
|
||||
cols = [r[1] for r in cur.fetchall()]
|
||||
if "context_length" not in cols:
|
||||
conn.execute("ALTER TABLE test_runs ADD COLUMN context_length INTEGER DEFAULT 0")
|
||||
cur = conn.execute("PRAGMA table_info(tests)")
|
||||
tcols = [r[1] for r in cur.fetchall()]
|
||||
if "name" not in tcols:
|
||||
conn.execute("ALTER TABLE tests ADD COLUMN name TEXT DEFAULT ''")
|
||||
|
||||
|
||||
def init_db():
|
||||
@@ -134,6 +139,23 @@ def delete_config(cid: int):
|
||||
conn.close()
|
||||
|
||||
|
||||
def update_config(cid: int, cfg: dict):
|
||||
"""局部更新：只更新请求里出现的字段"""
|
||||
allow = {"name", "provider", "base_url", "api_key", "model", "temperature"}
|
||||
fields = {k: v for k, v in cfg.items() if k in allow and v is not None}
|
||||
if not fields:
|
||||
return
|
||||
sets = ",".join("%s=?" % k for k in fields)
|
||||
vals = list(fields.values()) + [cid]
|
||||
with _lock:
|
||||
conn = _connect()
|
||||
try:
|
||||
conn.execute("UPDATE configs SET %s WHERE id=?" % sets, vals)
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
# ───────────────────────── 测试记录 ─────────────────────────
|
||||
|
||||
def create_test(cfg: dict, gen: dict) -> int:
|
||||
@@ -141,9 +163,10 @@ def create_test(cfg: dict, gen: dict) -> int:
|
||||
conn = _connect()
|
||||
try:
|
||||
cur = conn.execute(
|
||||
"INSERT INTO tests(status,provider,model,config_json,gen_cfg_json,started_at) "
|
||||
"VALUES('running',?,?,?,?,?)",
|
||||
"INSERT INTO tests(status,provider,model,name,config_json,gen_cfg_json,started_at) "
|
||||
"VALUES('running',?,?,?,?,?,?)",
|
||||
(cfg.get("provider", "openai"), cfg.get("model", ""),
|
||||
gen.get("name") or cfg.get("name") or "",
|
||||
json.dumps(cfg, ensure_ascii=False), json.dumps(gen, ensure_ascii=False),
|
||||
time.time()))
|
||||
conn.commit()
|
||||
@@ -186,7 +209,7 @@ def list_tests(limit=100):
|
||||
conn = _connect()
|
||||
try:
|
||||
rows = conn.execute(
|
||||
"SELECT id,created_at,status,provider,model,summary_json,error "
|
||||
"SELECT id,created_at,status,provider,model,name,summary_json,error "
|
||||
"FROM tests ORDER BY id DESC LIMIT ?", (limit,)).fetchall()
|
||||
out = []
|
||||
for r in rows:
|
||||
|
||||
+23
-11
@@ -56,6 +56,8 @@ def _parse_sse_line(line):
|
||||
|
||||
def _metrics(start, first_token_at, end, prompt_tokens, output_tokens,
|
||||
cached_tokens, output_chars, prompt_chars):
|
||||
if first_token_at is None:
|
||||
first_token_at = end # 未收到正文但请求完成（如纯 usage 响应）
|
||||
ttft_ms = (first_token_at - start) * 1000
|
||||
decode_ms = (end - first_token_at) * 1000
|
||||
total_ms = (end - start) * 1000
|
||||
@@ -103,6 +105,7 @@ def stream_openai(cfg, prompt, gen, log, should_stop=None):
|
||||
first_token_at = None
|
||||
output_chars = 0
|
||||
prompt_tokens = output_tokens = cached_tokens = 0
|
||||
event_count = 0
|
||||
resp = None
|
||||
try:
|
||||
while True:
|
||||
@@ -126,13 +129,15 @@ def stream_openai(cfg, prompt, gen, log, should_stop=None):
|
||||
for obj in _iter_json(resp):
|
||||
if should_stop and should_stop():
|
||||
raise StopRequested()
|
||||
event_count += 1
|
||||
if obj.get("choices"):
|
||||
delta = obj["choices"][0].get("delta") or {}
|
||||
text = delta.get("content") or ""
|
||||
if text:
|
||||
# 兼容推理型模型：Qwen3/DeepSeek 思维链在 reasoning_content
|
||||
piece = delta.get("content") or delta.get("reasoning_content") or ""
|
||||
if piece:
|
||||
if first_token_at is None:
|
||||
first_token_at = time.time()
|
||||
output_chars += len(text)
|
||||
output_chars += len(piece)
|
||||
usage = obj.get("usage")
|
||||
if usage:
|
||||
prompt_tokens = usage.get("prompt_tokens") or 0
|
||||
@@ -148,8 +153,8 @@ def stream_openai(cfg, prompt, gen, log, should_stop=None):
|
||||
if resp is not None:
|
||||
resp.close()
|
||||
|
||||
if first_token_at is None:
|
||||
raise ProviderError("未收到任何输出内容")
|
||||
if event_count == 0:
|
||||
raise ProviderError("未收到任何输出内容（HTTP 200 但响应流为空）")
|
||||
end = time.time()
|
||||
return _metrics(start, first_token_at, end, prompt_tokens, output_tokens,
|
||||
cached_tokens, output_chars, len(prompt))
|
||||
@@ -190,6 +195,7 @@ def stream_anthropic(cfg, prompt, gen, log, should_stop=None):
|
||||
first_token_at = None
|
||||
output_chars = 0
|
||||
prompt_tokens = output_tokens = 0
|
||||
event_count = 0
|
||||
resp = None
|
||||
try:
|
||||
if should_stop and should_stop():
|
||||
@@ -204,12 +210,15 @@ def stream_anthropic(cfg, prompt, gen, log, should_stop=None):
|
||||
for obj in _iter_json(resp):
|
||||
if should_stop and should_stop():
|
||||
raise StopRequested()
|
||||
event_count += 1
|
||||
etype = obj.get("type")
|
||||
if etype == "message_start":
|
||||
usage = (obj.get("message") or {}).get("usage") or {}
|
||||
prompt_tokens = usage.get("input_tokens") or 0
|
||||
elif etype == "content_block_delta":
|
||||
text = (obj.get("delta") or {}).get("text") or ""
|
||||
delta = obj.get("delta") or {}
|
||||
# 兼容 extended thinking：thinking 文本也算输出
|
||||
text = delta.get("text") or delta.get("thinking") or ""
|
||||
if text:
|
||||
if first_token_at is None:
|
||||
first_token_at = time.time()
|
||||
@@ -225,8 +234,8 @@ def stream_anthropic(cfg, prompt, gen, log, should_stop=None):
|
||||
if resp is not None:
|
||||
resp.close()
|
||||
|
||||
if first_token_at is None:
|
||||
raise ProviderError("未收到任何输出内容")
|
||||
if event_count == 0:
|
||||
raise ProviderError("未收到任何输出内容（HTTP 200 但响应流为空）")
|
||||
end = time.time()
|
||||
return _metrics(start, first_token_at, end, prompt_tokens, output_tokens,
|
||||
0, output_chars, len(prompt))
|
||||
@@ -252,6 +261,7 @@ def stream_google(cfg, prompt, gen, log, should_stop=None):
|
||||
first_token_at = None
|
||||
output_chars = 0
|
||||
prompt_tokens = output_tokens = cached_tokens = 0
|
||||
event_count = 0
|
||||
resp = None
|
||||
try:
|
||||
if should_stop and should_stop():
|
||||
@@ -266,11 +276,13 @@ def stream_google(cfg, prompt, gen, log, should_stop=None):
|
||||
for obj in _iter_json(resp):
|
||||
if should_stop and should_stop():
|
||||
raise StopRequested()
|
||||
event_count += 1
|
||||
cands = obj.get("candidates") or []
|
||||
if cands:
|
||||
parts = (cands[0].get("content") or {}).get("parts") or []
|
||||
for part in parts:
|
||||
text = part.get("text") or ""
|
||||
# 兼容 thinking 模型：thought 文本也算输出
|
||||
text = part.get("text") or part.get("thought") or ""
|
||||
if text:
|
||||
if first_token_at is None:
|
||||
first_token_at = time.time()
|
||||
@@ -288,8 +300,8 @@ def stream_google(cfg, prompt, gen, log, should_stop=None):
|
||||
if resp is not None:
|
||||
resp.close()
|
||||
|
||||
if first_token_at is None:
|
||||
raise ProviderError("未收到任何输出内容")
|
||||
if event_count == 0:
|
||||
raise ProviderError("未收到任何输出内容（HTTP 200 但响应流为空）")
|
||||
end = time.time()
|
||||
return _metrics(start, first_token_at, end, prompt_tokens, output_tokens,
|
||||
cached_tokens, output_chars, len(prompt))
|
||||
|
||||
@@ -184,6 +184,10 @@ table.history { width: 100%; border-collapse: collapse; font-size: 13px; }
|
||||
.modal-body { padding: 16px 18px; overflow-y: auto; }
|
||||
.modal-body h3 { font-size: 14px; margin: 16px 0 8px; color: var(--muted); }
|
||||
.modal-body h3:first-child { margin-top: 0; }
|
||||
.detail-name {
|
||||
background: rgba(79,140,255,.12); border: 1px solid rgba(79,140,255,.35); color: var(--accent);
|
||||
border-radius: 10px; padding: 10px 14px; font-size: 14px; font-weight: 600; margin-bottom: 6px;
|
||||
}
|
||||
|
||||
.kv { display: grid; grid-template-columns: repeat(auto-fill, minmax(180px, 1fr)); gap: 8px; }
|
||||
.kv .kv-item { background: var(--panel2); border: 1px solid var(--border); border-radius: 8px; padding: 8px 10px; }
|
||||
@@ -202,6 +206,18 @@ table.mini { width: 100%; border-collapse: collapse; font-size: 12.5px; }
|
||||
.detail-log .ln.ERROR { color: var(--danger); }
|
||||
.detail-log .ln.WARN { color: var(--warn); }
|
||||
|
||||
/* ── 图表（data-chart-tool 折线图） ── */
|
||||
.chart-block { background: var(--panel2); border: 1px solid var(--border); border-radius: 10px; padding: 12px; }
|
||||
.chart-toolbar { display: flex; align-items: center; gap: 10px; flex-wrap: wrap; }
|
||||
.chart-img-wrap { margin-top: 10px; min-height: 60px; display: flex; align-items: center; justify-content: center; border: 1px dashed var(--border); border-radius: 8px; padding: 10px; background: #0a0e17; }
|
||||
.chart-img-wrap img { max-width: 100%; border-radius: 6px; }
|
||||
.chart-data-head { margin-top: 10px; font-size: 12px; color: var(--muted); }
|
||||
.chart-csv {
|
||||
width: 100%; min-height: 120px; margin-top: 6px; background: #0a0e17;
|
||||
border: 1px solid var(--border); border-radius: 8px; color: #a8f0d0;
|
||||
font: 12px/1.7 var(--mono); padding: 10px; resize: vertical; outline: none;
|
||||
}
|
||||
|
||||
pre.json-box {
|
||||
background: #0a0e17; border: 1px solid var(--border); border-radius: 8px;
|
||||
padding: 10px; font: 11.5px/1.6 var(--mono); overflow: auto; max-height: 260px; color: #a8f0d0;
|
||||
|
||||
+5
-1
@@ -68,6 +68,10 @@
|
||||
|
||||
<section class="card">
|
||||
<h2>🚀 速度测试配置</h2>
|
||||
<div class="field">
|
||||
<label>测试名称（主题）</label>
|
||||
<input id="gen-name" type="text" placeholder="如：DeepSeek-V4 不同上下文长度速度对比">
|
||||
</div>
|
||||
<div class="field">
|
||||
<label>上下文长度（tokens，点击切换启停）</label>
|
||||
<div class="chips" id="gen-contexts"></div>
|
||||
@@ -136,7 +140,7 @@
|
||||
<div class="table-wrap">
|
||||
<table class="history" id="history">
|
||||
<thead><tr>
|
||||
<th>#</th><th>时间</th><th>提供商</th><th>模型</th><th>采样</th>
|
||||
<th>#</th><th>时间</th><th>名称</th><th>提供商</th><th>模型</th><th>采样</th>
|
||||
<th>首字 ms</th><th>预填充 tok/s</th><th>解码 tok/s</th><th>状态</th><th>操作</th>
|
||||
</tr></thead>
|
||||
<tbody></tbody>
|
||||
|
||||
+88
-14
@@ -21,8 +21,8 @@ let pollTimer = null;
|
||||
let lastLogId = 0;
|
||||
let consoleLogs = []; // 当前测试已加载日志 [{id,level,msg,rel}]
|
||||
|
||||
// 上下文长度：chips 列表 + 启用集合（默认 512/2048/8192/32768/131072）
|
||||
const DEFAULT_CONTEXT_LENGTHS = [512, 2048, 8192, 32768, 131072];
|
||||
// 上下文长度：chips 列表 + 启用集合（默认 512/2048/4096/8192/16384/32768/65536/131072）
|
||||
const DEFAULT_CONTEXT_LENGTHS = [512, 2048, 4096, 8192, 16384, 32768, 65536, 131072];
|
||||
let contextLengths = [...DEFAULT_CONTEXT_LENGTHS];
|
||||
let contextLengthsActive = new Set(contextLengths);
|
||||
|
||||
@@ -57,6 +57,7 @@ function currentConfig() {
|
||||
function currentGen() {
|
||||
const lens = contextLengths.filter((l) => contextLengthsActive.has(l));
|
||||
return {
|
||||
name: $("#gen-name").value.trim(),
|
||||
context_lengths: lens.length ? lens : [2048],
|
||||
max_tokens: parseInt($("#gen-max-tokens").value) || 128,
|
||||
samples: parseInt($("#gen-samples").value) || 2,
|
||||
@@ -158,7 +159,7 @@ async function testConnection() {
|
||||
if (r.ok) {
|
||||
const m = r.metrics || {};
|
||||
showConn(true,
|
||||
`✅ 连接成功（${r.total_ms}ms）| 首字 ${fmt(m.ttft_ms)}ms | 提示词 ${fmt(m.prompt_tokens)} tok | 输出 ${fmt(m.output_tokens)} tok`);
|
||||
`✅ 连接成功（${r.total_ms}ms）| 首字 ${fmt(m.ttft_ms)}ms | 提示词 ${fmt(m.prompt_tokens)} tok | 输出 ${fmt(m.output_tokens)} tok${r.note ? " " + r.note : ""}`);
|
||||
} else {
|
||||
showConn(false, "❌ " + (r.error || "连接失败"));
|
||||
}
|
||||
@@ -300,7 +301,7 @@ async function loadHistory() {
|
||||
const tb = $("#history tbody");
|
||||
tb.innerHTML = "";
|
||||
if (!list.length) {
|
||||
tb.innerHTML = '<tr><td colspan="10" style="color:var(--muted);text-align:center">暂无测试记录</td></tr>';
|
||||
tb.innerHTML = '<tr><td colspan="11" style="color:var(--muted);text-align:center">暂无测试记录</td></tr>';
|
||||
return;
|
||||
}
|
||||
for (const t of list) {
|
||||
@@ -309,6 +310,7 @@ async function loadHistory() {
|
||||
tr.innerHTML = `
|
||||
<td>#${t.id}</td>
|
||||
<td>${esc(t.created_at)}</td>
|
||||
<td title="${esc(t.name || "")}">${esc(t.name || "—")}</td>
|
||||
<td>${esc(PROVIDER_LABEL[t.provider] || t.provider)}</td>
|
||||
<td>${esc(t.model)}</td>
|
||||
<td class="num">${fmt(s.samples_ok)}/${fmt(s.samples_total)}</td>
|
||||
@@ -390,28 +392,55 @@ async function viewDetail(id) {
|
||||
).join("") + `</div>`;
|
||||
|
||||
$("#dt-id").textContent = id;
|
||||
const overallRow = (label, avgV, minV, maxV) =>
|
||||
`<tr><td>${label}</td><td class="num">${fmt(avgV)}</td><td class="num">${fmt(minV)}</td><td class="num">${fmt(maxV)}</td></tr>`;
|
||||
$("#dt-body").innerHTML = `
|
||||
${t.name ? `<div class="detail-name">🏷️ ${esc(t.name)}</div>` : ""}
|
||||
<h3>📌 汇总指标</h3>
|
||||
<div class="kv">
|
||||
<div class="kv-item"><div class="kv-k">状态</div><div class="kv-v">${STATUS_LABEL[t.status] || t.status}</div></div>
|
||||
<div class="kv-item"><div class="kv-k">创建时间</div><div class="kv-v">${esc(t.created_at)}</div></div>
|
||||
<div class="kv-item"><div class="kv-k">提供商 / 模型</div><div class="kv-v">${esc(PROVIDER_LABEL[t.provider] || t.provider)} / ${esc(t.model)}</div></div>
|
||||
<div class="kv-item"><div class="kv-k">采样（成功/总数）</div><div class="kv-v">${fmt(s.samples_ok)} / ${fmt(s.samples_total)}</div></div>
|
||||
<div class="kv-item"><div class="kv-k">平均首字延迟</div><div class="kv-v">${fmt(s.avg_ttft_ms)} ms</div></div>
|
||||
<div class="kv-item"><div class="kv-k">最佳首字延迟</div><div class="kv-v">${fmt(s.best_ttft_ms)} ms</div></div>
|
||||
<div class="kv-item"><div class="kv-k">平均预填充速度</div><div class="kv-v">${fmt(s.avg_prefill_speed)} tok/s</div></div>
|
||||
<div class="kv-item"><div class="kv-k">平均解码速度</div><div class="kv-v">${fmt(s.avg_decode_speed)} tok/s</div></div>
|
||||
<div class="kv-item"><div class="kv-k">平均提示词</div><div class="kv-v">${fmt(s.avg_prompt_tokens)} tok</div></div>
|
||||
<div class="kv-item"><div class="kv-k">平均输出</div><div class="kv-v">${fmt(s.avg_output_tokens)} tok</div></div>
|
||||
<div class="kv-item"><div class="kv-k">平均缓存命中</div><div class="kv-v">${fmt(s.avg_cached_tokens, 0)} tok</div></div>
|
||||
<div class="kv-item"><div class="kv-k">平均总耗时</div><div class="kv-v">${fmt(s.avg_total_ms)} ms</div></div>
|
||||
<div class="kv-item"><div class="kv-k">校准字符/token</div><div class="kv-v">${fmt(s.calibration_chars_per_token)}</div></div>
|
||||
</div>
|
||||
|
||||
<h3>📊 整体统计（平均 / 最小 / 最大）</h3>
|
||||
<table class="mini"><thead><tr>
|
||||
<th>指标</th><th>平均</th><th>最小</th><th>最大</th>
|
||||
</tr></thead><tbody>
|
||||
${overallRow("首字延迟(ms)", s.avg_ttft_ms, s.min_ttft_ms, s.max_ttft_ms)}
|
||||
${overallRow("预填充速度(tok/s)", s.avg_prefill_speed, s.min_prefill_speed, s.max_prefill_speed)}
|
||||
${overallRow("解码速度(tok/s)", s.avg_decode_speed, s.min_decode_speed, s.max_decode_speed)}
|
||||
${overallRow("提示词(tok)", s.avg_prompt_tokens, null, null)}
|
||||
${overallRow("输出(tok)", s.avg_output_tokens, null, null)}
|
||||
${overallRow("总耗时(ms)", s.avg_total_ms, s.min_total_ms, s.max_total_ms)}
|
||||
</tbody></table>
|
||||
|
||||
<h3>📏 按上下文长度汇总</h3>
|
||||
${byLengthHtml}
|
||||
|
||||
<h3>📈 折线图（预填充左轴虚线 / 解码右轴实线）</h3>
|
||||
<div class="chart-block">
|
||||
<div class="chart-toolbar">
|
||||
<button class="btn small primary" id="dt-chart-gen">🎨 生成/刷新折线图</button>
|
||||
<span class="hint" id="dt-chart-status"></span>
|
||||
</div>
|
||||
<div class="chart-img-wrap" id="dt-chart-img">
|
||||
<div class="hint">点击上方按钮，用 data-chart-tool 生成双Y轴折线图（左轴=预填充虚线，右轴=解码实线）</div>
|
||||
</div>
|
||||
<div class="chart-data-head">📋 画图数据（CSV，可一键复制）</div>
|
||||
<textarea class="chart-csv" id="dt-chart-csv" readonly spellcheck="false" placeholder="（生成图表后自动填充，或手动复制）"></textarea>
|
||||
<div class="chart-toolbar" style="margin-top:6px">
|
||||
<button class="btn small" id="dt-chart-copy">📋 复制画图数据</button>
|
||||
<button class="btn small" id="dt-chart-download">⬇ 下载图表 PNG</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<h3>⚙️ 测试参数</h3>
|
||||
<div class="kv">
|
||||
<div class="kv-item"><div class="kv-k">测试名称</div><div class="kv-v">${esc(t.name || "—")}</div></div>
|
||||
<div class="kv-item"><div class="kv-k">上下文长度</div><div class="kv-v">${(g.context_lengths || []).join(" / ") || "—"} tok</div></div>
|
||||
<div class="kv-item"><div class="kv-k">解码输出长度</div><div class="kv-v">${g.max_tokens ?? "—"} tok</div></div>
|
||||
<div class="kv-item"><div class="kv-k">每个长度采样</div><div class="kv-v">${g.samples ?? "—"}</div></div>
|
||||
@@ -419,7 +448,6 @@ async function viewDetail(id) {
|
||||
<div class="kv-item"><div class="kv-k">避免缓存</div><div class="kv-v">${g.avoid_cache ? "开" : "关"}</div></div>
|
||||
<div class="kv-item"><div class="kv-k">温度</div><div class="kv-v">${fmt(cfg.temperature)}</div></div>
|
||||
<div class="kv-item"><div class="kv-k">Base URL</div><div class="kv-v">${esc(cfg.base_url || "（默认）")}</div></div>
|
||||
<div class="kv-item"><div class="kv-k">校准字符/token</div><div class="kv-v">${fmt(s.calibration_chars_per_token)}</div></div>
|
||||
</div>
|
||||
|
||||
<h3>📊 每次采样明细</h3>
|
||||
@@ -433,13 +461,59 @@ async function viewDetail(id) {
|
||||
$("#detail-mask").hidden = false;
|
||||
$("#dt-body").scrollTop = 0;
|
||||
window.__detail = t;
|
||||
|
||||
// 图表：生成/刷新 + 复制数据 + 下载
|
||||
const genBtn = $("#dt-chart-gen");
|
||||
const csvTa = $("#dt-chart-csv");
|
||||
const imgWrap = $("#dt-chart-img");
|
||||
const statusEl = $("#dt-chart-status");
|
||||
let chartUrl = "";
|
||||
const genChart = () => {
|
||||
statusEl.textContent = "⏳ 正在生成...";
|
||||
fetch(`/api/tests/${id}/chart-data`).then((r) => r.json()).then((d) => {
|
||||
if (!d.ok) { statusEl.textContent = "❌ " + (d.error || "生成失败"); return; }
|
||||
csvTa.value = d.csv;
|
||||
chartUrl = `/api/tests/${id}/chart?t=${Date.now()}`;
|
||||
const img = document.createElement("img");
|
||||
img.src = chartUrl;
|
||||
img.alt = "预填充/解码 折线图";
|
||||
img.style.maxWidth = "100%";
|
||||
img.onload = () => { imgWrap.innerHTML = ""; imgWrap.appendChild(img); statusEl.textContent = "✅ 生成完成"; };
|
||||
img.onerror = () => { statusEl.textContent = "❌ 图表生成失败"; };
|
||||
}).catch((e) => { statusEl.textContent = "❌ " + e.message; });
|
||||
};
|
||||
genBtn.addEventListener("click", genChart);
|
||||
$("#dt-chart-copy").addEventListener("click", () => {
|
||||
if (!csvTa.value) { toast("暂无画图数据，请先生成图表"); return; }
|
||||
csvTa.select();
|
||||
if (navigator.clipboard && navigator.clipboard.writeText) {
|
||||
navigator.clipboard.writeText(csvTa.value).then(() => toast("画图数据已复制"));
|
||||
} else {
|
||||
document.execCommand("copy");
|
||||
toast("画图数据已复制");
|
||||
}
|
||||
});
|
||||
$("#dt-chart-download").addEventListener("click", () => {
|
||||
if (!chartUrl) { toast("请先生成图表"); return; }
|
||||
const a = document.createElement("a");
|
||||
a.href = chartUrl;
|
||||
a.download = `llm_speed_chart_${id}.png`;
|
||||
document.body.appendChild(a); a.click(); a.remove();
|
||||
});
|
||||
}
|
||||
|
||||
function closeDetail() { $("#detail-mask").hidden = true; }
|
||||
|
||||
function exportDetail() {
|
||||
async function exportDetail() {
|
||||
if (!window.__detail) return;
|
||||
download(JSON.stringify(window.__detail, null, 2), `test_${window.__detail.id}.json`, "application/json");
|
||||
try {
|
||||
const resp = await fetch(`/api/tests/${window.__detail.id}/export.json`);
|
||||
if (!resp.ok) throw new Error("服务端导出失败");
|
||||
const j = await resp.json();
|
||||
download(JSON.stringify(j, null, 2), `test_${j.id}.json`, "application/json");
|
||||
} catch (e) {
|
||||
toast("导出失败：" + e.message);
|
||||
}
|
||||
}
|
||||
|
||||
async function exportXlsx(id) {
|
||||
|
||||
@@ -21,6 +21,7 @@ class TestRunner(threading.Thread):
|
||||
self.start_wall = time.time()
|
||||
self.ratio = None
|
||||
self.samples = []
|
||||
self.last_error = None
|
||||
|
||||
def request_cancel(self):
|
||||
self.cancel_flag = True
|
||||
@@ -50,7 +51,7 @@ class TestRunner(threading.Thread):
|
||||
model = self.cfg.get("model", "")
|
||||
gen = self.gen
|
||||
|
||||
# 上下文长度列表（支持手动自定义，默认 512/2048/8192/32768/131072）
|
||||
# 上下文长度列表（支持手动自定义，默认 512/2048/4096/8192/16384/32768/65536/131072）
|
||||
raw_lengths = gen.get("context_lengths") or []
|
||||
if not raw_lengths:
|
||||
# 兼容旧版单值配置
|
||||
@@ -62,6 +63,9 @@ class TestRunner(threading.Thread):
|
||||
warmup = bool(gen.get("warmup", True)) # 测试前空转预热
|
||||
|
||||
self.log("INFO", "═══ 开始速度测试 ═══")
|
||||
name = gen.get("name") or self.cfg.get("name") or ""
|
||||
if name:
|
||||
self.log("INFO", "测试名称(主题): %s" % name)
|
||||
self.log("INFO", "提供商: %s | 模型: %s" % (lp.PROVIDER_LABELS.get(provider, provider), model))
|
||||
self.log("INFO", "上下文长度: %s tokens | 生成长度: %d tokens | 每个长度采样: %d 次 | 预热: %s | 避免缓存: %s"
|
||||
% (" / ".join(str(x) for x in lengths), max_tokens, n,
|
||||
@@ -97,14 +101,26 @@ class TestRunner(threading.Thread):
|
||||
except StopRequested:
|
||||
raise
|
||||
except ProviderError as e:
|
||||
# 单次采样失败：记录并继续后续采样，不让整个测试中断
|
||||
self.last_error = str(e)
|
||||
self.log("ERROR", "[%d tok] 采样 %d/%d 失败: %s" % (L, i, n, e))
|
||||
self.samples.append({"run_index": i, "context_length": L, "ok": False, "error": str(e)})
|
||||
db.add_run(self.test_id, i, {}, str(e), context_length=L)
|
||||
raise e
|
||||
|
||||
summary = self._make_summary()
|
||||
db.update_status(self.test_id, "done", summary=summary)
|
||||
self.log("INFO", "═══ 测试完成 ═══")
|
||||
ok_count = summary.get("samples_ok") or 0
|
||||
fail_count = summary.get("samples_total", 0) - ok_count
|
||||
if ok_count:
|
||||
db.update_status(self.test_id, "done", summary=summary,
|
||||
error=("%d 次采样失败：%s" % (fail_count, self.last_error)) if fail_count else "")
|
||||
self.log("INFO", "═══ 测试完成 ═══")
|
||||
if fail_count:
|
||||
self.log("WARN", "共 %d 次采样失败（最后错误：%s）" % (fail_count, self.last_error))
|
||||
else:
|
||||
db.update_status(self.test_id, "error", summary=summary,
|
||||
error=self.last_error or "所有采样均失败")
|
||||
self.log("ERROR", "所有采样均失败，测试标记为 error（最后错误：%s）" % (self.last_error or "未知"))
|
||||
return
|
||||
self.log("INFO", "汇总: 平均首字 %.1f ms | 平均预填充 %.1f tok/s | 平均解码 %.1f tok/s"
|
||||
% (summary.get("avg_ttft_ms") or 0,
|
||||
summary.get("avg_prefill_speed") or 0,
|
||||
@@ -199,16 +215,33 @@ class TestRunner(threading.Thread):
|
||||
}
|
||||
|
||||
okm = [s["metrics"] for s in ok]
|
||||
|
||||
def mn(k):
|
||||
vals = [m[k] for m in okm if m.get(k) is not None]
|
||||
return round(min(vals), 1) if vals else None
|
||||
|
||||
def mx(k):
|
||||
vals = [m[k] for m in okm if m.get(k) is not None]
|
||||
return round(max(vals), 1) if vals else None
|
||||
|
||||
summary = dict(base)
|
||||
summary.update({
|
||||
"by_length": by_length,
|
||||
"avg_ttft_ms": avg(okm, "ttft_ms"),
|
||||
"min_ttft_ms": mn("ttft_ms"),
|
||||
"max_ttft_ms": mx("ttft_ms"),
|
||||
"avg_prefill_speed": avg(okm, "prefill_speed"),
|
||||
"min_prefill_speed": mn("prefill_speed"),
|
||||
"max_prefill_speed": mx("prefill_speed"),
|
||||
"avg_decode_speed": avg(okm, "decode_speed"),
|
||||
"min_decode_speed": mn("decode_speed"),
|
||||
"max_decode_speed": mx("decode_speed"),
|
||||
"avg_prompt_tokens": avg(okm, "prompt_tokens"),
|
||||
"avg_output_tokens": avg(okm, "output_tokens"),
|
||||
"avg_cached_tokens": avg(okm, "cached_tokens"),
|
||||
"avg_total_ms": avg(okm, "total_ms"),
|
||||
"best_ttft_ms": min([m["ttft_ms"] for m in okm if m.get("ttft_ms") is not None], default=None),
|
||||
"min_total_ms": mn("total_ms"),
|
||||
"max_total_ms": mx("total_ms"),
|
||||
"best_ttft_ms": mn("ttft_ms"),
|
||||
})
|
||||
return summary
|
||||
Reference in New Issue
Block a user