v2.0.0：多上下文长度测试(默认512/2048/8192/32768/131072)+测试前空转预热(不计速度)+解码输出默认128+每长度采样默认2+Excel(xlsx)导出+详情按长度分组+接口输入框图标美化+README文档
This commit is contained in:
@@ -0,0 +1,151 @@
|
||||
# ⚡ LLM 速度测试台
|
||||
|
||||
大模型推理性能基准测试工具，用于测量不同上下文长度下的 **预填充速度（prefill）**、**首字延迟（TTFT）** 与 **解码吞吐（decode）**，并以网页 + Excel 形式输出结果。
|
||||
|
||||
- **访问地址：** `http://<IP>:16097/`
|
||||
- **技术栈：** Python 3 + Flask + SQLite（纯 REST，无额外依赖）
|
||||
- **版本：** v2.0.0
|
||||
|
||||
---
|
||||
|
||||
## 功能特性
|
||||
|
||||
### 🔌 大模型接口配置
|
||||
- 支持 **OpenAI 兼容**（OpenAI / DeepSeek / 火山方舟 / 任意兼容网关）、**Anthropic（Claude）**、**Google Gemini** 三类提供商
|
||||
- 配置项：配置名称、Base URL（留空自动使用官方默认）、API Key（可显示/隐藏）、模型名称、温度
|
||||
- 配置可**保存/加载/删除**，方便多模型对比
|
||||
|
||||
### 🚀 速度测试配置
|
||||
- **多上下文长度测试**：默认为 `512 / 2048 / 8192 / 32768 / 131072` tokens，可通过标签点击启用/禁用，并支持手动添加任意自定义长度（≥16）
|
||||
- **解码输出长度（max tokens）**：默认为 `128`，可手动自定义
|
||||
- **每个长度采样次数**：默认为 `2`，可手动自定义
|
||||
- **测试前预热（空转）**：默认开启，先发一次不计速度的空转请求，避免冷启动/首请求偏慢污染真实采样数据
|
||||
- **避免缓存**：默认开启，为每次采样追加随机前缀，测量真实预填充性能
|
||||
|
||||
### 📊 指标与结果
|
||||
- 实时指标卡：首字延迟、预填充速度、解码速度、上文/输出 tokens、总耗时
|
||||
- 实时控制台日志：校准、预热、每次采样明细全程可追溯
|
||||
- **每次完整测试**支持：
|
||||
- **网页点击查看**：历史记录「查看」按钮弹出详情（整体平均 + 按上下文长度分组 + 每次采样明细 + 完整日志）
|
||||
- **文件下载 Excel（xlsx）**：历史记录「Excel」按钮或详情弹窗「导出 Excel」，包含 汇总 / 采样明细 / 日志 三个 Sheet
|
||||
- 测试历史留存，可随时刷新、查看、导出、删除
|
||||
|
||||
---
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 环境要求
|
||||
- Python 3.10+（推荐使用 openclaw conda 环境）
|
||||
- 依赖见 `requirements.txt`
|
||||
|
||||
### 安装
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
# 或使用 conda 环境
|
||||
/home/hz1/miniconda3/envs/openclaw/bin/pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 启动
|
||||
```bash
|
||||
./start.sh # 启动（默认端口 16097）
|
||||
./start.sh stop # 停止
|
||||
./start.sh restart # 重启
|
||||
```
|
||||
|
||||
启动成功后访问 `http://<IP>:16097/`。
|
||||
|
||||
---
|
||||
|
||||
## 使用说明
|
||||
|
||||
1. **配置接口**：选择提供商 → 填写配置名称 / Base URL（可留空）/ API Key / 模型名称 → 点「保存」可留存，或直接点「🔍 测试连接」验证连通性
|
||||
2. **配置测试参数**：勾选要测试的上下文长度（默认 5 档），设置解码输出长度与采样次数，按需开关预热/避免缓存
|
||||
3. **开始测试**：点「▶ 开始测试」，右侧实时展示指标与日志；可随时「■ 停止」
|
||||
4. **查看与导出**：测试完成后，在「测试历史」中点「查看」看完整详情，点「Excel」或详情内「导出 Excel」下载 xlsx 报告
|
||||
|
||||
### 指标含义
|
||||
| 指标 | 含义 |
|
||||
|------|------|
|
||||
| 首字延迟 TTFT (ms) | 从请求发出到收到第一个 token 的时间（含预填充） |
|
||||
| 预填充速度 (tok/s) | prompt tokens / 首字延迟，衡量上文处理吞吐 |
|
||||
| 解码速度 (tok/s) | 输出 tokens / 解码阶段耗时，衡量逐 token 生成吞吐 |
|
||||
| 上下文/输出 tokens | 实际发送的提示词 token 数与模型返回的 token 数 |
|
||||
|
||||
---
|
||||
|
||||
## API
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
|------|------|------|
|
||||
| GET | `/api/health` | 健康检查 |
|
||||
| GET/POST | `/api/configs` | 配置列表 / 新增配置 |
|
||||
| GET/DELETE | `/api/configs/<id>` | 单个配置 / 删除 |
|
||||
| POST | `/api/configs/test` | 测试连接 |
|
||||
| GET/POST | `/api/tests` | 测试历史 / 启动测试 |
|
||||
| GET | `/api/tests/<id>` | 测试详情（含 runs / logs / summary） |
|
||||
| GET | `/api/tests/<id>/logs?after=<id>` | 增量日志（前端轮询用） |
|
||||
| POST | `/api/tests/<id>/cancel` | 停止测试 |
|
||||
| DELETE | `/api/tests/<id>` | 删除测试 |
|
||||
| GET | `/api/tests/<id>/export.xlsx` | 导出 Excel 报告 |
|
||||
|
||||
### 启动测试请求示例
|
||||
```json
|
||||
POST /api/tests
|
||||
{
|
||||
"config": {
|
||||
"provider": "openai",
|
||||
"name": "DeepSeek-V4",
|
||||
"base_url": "https://api.deepseek.com/v1",
|
||||
"api_key": "sk-xxx",
|
||||
"model": "deepseek-chat",
|
||||
"temperature": 0.7
|
||||
},
|
||||
"gen": {
|
||||
"context_lengths": [512, 2048, 8192, 32768, 131072],
|
||||
"max_tokens": 128,
|
||||
"samples": 2,
|
||||
"warmup": true,
|
||||
"avoid_cache": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 项目结构
|
||||
|
||||
```
|
||||
llm-speed-tester/
|
||||
├── app.py # Flask 主应用（路由 + Excel 导出）
|
||||
├── config.py # 全局配置（端口/超时/路径）
|
||||
├── database.py # SQLite 存储（配置/测试/采样/日志）
|
||||
├── tester.py # 测试执行器（校准 → 预热 → 多长度采样 → 汇总）
|
||||
├── llm_providers.py # 提供商适配器（OpenAI / Anthropic / Gemini）
|
||||
├── requirements.txt
|
||||
├── start.sh # 启动/停止脚本
|
||||
├── static/ # 前端（index.html / js/app.js / css/style.css）
|
||||
├── data/ # SQLite 数据库（llm_speed_tester.db）
|
||||
└── logs/ # 运行日志 + PID
|
||||
```
|
||||
|
||||
### 测试执行流程
|
||||
1. **校准**：发送一个短探测请求，估算 `token/字符` 比例
|
||||
2. **逐上下文长度**：对每个选中的长度构造对应 token 量的基准提示词
|
||||
3. **预热（空转，不计速度）**：每个长度先发一次请求，规避冷启动偏差
|
||||
4. **采样**：每个长度按采样次数重复请求，记录每次指标
|
||||
5. **汇总**：按长度分组 + 整体平均，写入测试记录
|
||||
|
||||
---
|
||||
|
||||
## 常见问题
|
||||
|
||||
- **无 openpyxl**：`pip install openpyxl`（已加入 requirements.txt）
|
||||
- **老版本数据库**：程序启动时自动迁移，为 `test_runs` 表补充 `context_length` 列，无需手动处理
|
||||
- **慢模型超时**：连接超时 30s、两次数据包间隔 120s，足够覆盖大多数慢模型；超长文（131072）生成慢属正常，请耐心等待
|
||||
|
||||
---
|
||||
|
||||
## Git
|
||||
|
||||
- **仓库：** `hz4th_coder/llm-speed-tester`
|
||||
- **版本：** v2.0.0（新增多上下文长度测试 + 预热 + Excel 导出 + 界面优化）
|
||||
@@ -1,8 +1,9 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""LLM 速度测试台 - Flask 主应用"""
|
||||
import io
|
||||
import json
|
||||
|
||||
from flask import Flask, jsonify, request, send_from_directory
|
||||
from flask import Flask, jsonify, request, send_file, send_from_directory
|
||||
|
||||
import config
|
||||
import database as db
|
||||
@@ -139,5 +140,150 @@ def del_test(tid):
|
||||
return jsonify({"ok": True})
|
||||
|
||||
|
||||
# ───────────────────────── Excel 导出 ─────────────────────────
|
||||
|
||||
@app.route("/api/tests/<int:tid>/export.xlsx")
|
||||
def export_xlsx(tid):
|
||||
t = db.get_test(tid)
|
||||
if not t:
|
||||
return jsonify({"ok": False, "error": "测试不存在"}), 404
|
||||
t["runs"] = db.get_runs(tid)
|
||||
t["logs"] = db.get_logs(tid)
|
||||
try:
|
||||
data = _build_xlsx(t)
|
||||
except Exception as e:
|
||||
return jsonify({"ok": False, "error": "导出失败: %s" % e}), 500
|
||||
return send_file(data, as_attachment=True,
|
||||
download_name="llm_speed_test_%d.xlsx" % tid,
|
||||
mimetype="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet")
|
||||
|
||||
|
||||
def _build_xlsx(t):
|
||||
from openpyxl import Workbook
|
||||
from openpyxl.styles import Alignment, Font, PatternFill
|
||||
|
||||
s = t.get("summary") or {}
|
||||
g = t.get("gen") or {}
|
||||
cfg = t.get("config") or {}
|
||||
by_length = s.get("by_length") or {}
|
||||
runs = t.get("runs") or []
|
||||
logs = t.get("logs") or []
|
||||
|
||||
wb = Workbook()
|
||||
head_fill = PatternFill("solid", fgColor="2A3550")
|
||||
head_font = Font(color="FFFFFF", bold=True)
|
||||
title_font = Font(bold=True, size=12)
|
||||
|
||||
def style_header(ws, row, ncol):
|
||||
for c in range(1, ncol + 1):
|
||||
cell = ws.cell(row=row, column=c)
|
||||
cell.fill = head_fill
|
||||
cell.font = head_font
|
||||
cell.alignment = Alignment(horizontal="center", vertical="center")
|
||||
|
||||
# ── Sheet1 汇总 ──
|
||||
ws = wb.active
|
||||
ws.title = "汇总"
|
||||
ws.append(["LLM 速度测试报告"])
|
||||
ws.cell(1, 1).font = Font(bold=True, size=14)
|
||||
info = [
|
||||
["测试编号", "#%d" % t["id"]],
|
||||
["创建时间", t.get("created_at", "")],
|
||||
["状态", t.get("status", "")],
|
||||
["提供商", t.get("provider", "")],
|
||||
["模型", t.get("model", "")],
|
||||
["Base URL", cfg.get("base_url") or "（默认）"],
|
||||
["上下文长度列表", " / ".join(str(x) for x in (g.get("context_lengths") or []))],
|
||||
["生成长度(max tokens)", g.get("max_tokens", 128)],
|
||||
["每个长度采样次数", g.get("samples", 2)],
|
||||
["预热(空转)", "开" if g.get("warmup", True) else "关"],
|
||||
["避免缓存", "开" if g.get("avoid_cache") else "关"],
|
||||
["采样(成功/总数)", "%s / %s" % (s.get("samples_ok"), s.get("samples_total"))],
|
||||
["校准 字符/token", s.get("calibration_chars_per_token") or "—"],
|
||||
["错误信息", t.get("error") or ""],
|
||||
]
|
||||
for row in info:
|
||||
ws.append(row)
|
||||
ws.cell(14, 1).font = title_font
|
||||
r0 = len(info) + 2
|
||||
overall = [
|
||||
["平均首字延迟(ms)", s.get("avg_ttft_ms")],
|
||||
["最佳首字延迟(ms)", s.get("best_ttft_ms")],
|
||||
["平均预填充速度(tok/s)", s.get("avg_prefill_speed")],
|
||||
["平均解码速度(tok/s)", s.get("avg_decode_speed")],
|
||||
["平均提示词(tok)", s.get("avg_prompt_tokens")],
|
||||
["平均输出(tok)", s.get("avg_output_tokens")],
|
||||
["平均总耗时(ms)", s.get("avg_total_ms")],
|
||||
]
|
||||
ws.cell(r0, 1, "整体平均指标").font = title_font
|
||||
for i, row in enumerate(overall, start=r0 + 1):
|
||||
ws.append([])
|
||||
for j, v in enumerate(row, start=1):
|
||||
ws.cell(row=i, column=j, value=v)
|
||||
|
||||
# 按上下文长度分组
|
||||
r1 = r0 + len(overall) + 2
|
||||
ws.cell(r1, 1, "按上下文长度分组").font = title_font
|
||||
cols = ["上下文长度(tok)", "采样(成功/总数)", "首字ms", "预填充tok/s", "解码tok/s", "提示词tok", "输出tok", "总耗时ms"]
|
||||
ws.append([])
|
||||
for j, c in enumerate(cols, start=1):
|
||||
ws.cell(row=r1 + 1, column=j, value=c)
|
||||
style_header(ws, r1 + 1, len(cols))
|
||||
if by_length:
|
||||
rr = r1 + 2
|
||||
for L in sorted(int(k) for k in by_length):
|
||||
bl = by_length[str(L)] if str(L) in by_length else by_length[L]
|
||||
ws.cell(row=rr, column=1, value=L)
|
||||
ws.cell(row=rr, column=2, value="%s / %s" % (bl.get("samples_ok"), bl.get("samples_total")))
|
||||
ws.cell(row=rr, column=3, value=bl.get("avg_ttft_ms"))
|
||||
ws.cell(row=rr, column=4, value=bl.get("avg_prefill_speed"))
|
||||
ws.cell(row=rr, column=5, value=bl.get("avg_decode_speed"))
|
||||
ws.cell(row=rr, column=6, value=bl.get("avg_prompt_tokens"))
|
||||
ws.cell(row=rr, column=7, value=bl.get("avg_output_tokens"))
|
||||
ws.cell(row=rr, column=8, value=bl.get("avg_total_ms"))
|
||||
rr += 1
|
||||
else:
|
||||
ws.cell(row=r1 + 2, column=1, value="（无成功采样数据）")
|
||||
for col, w in zip("ABCDEFGH", [22, 20, 12, 14, 14, 12, 12, 14]):
|
||||
ws.column_dimensions[col].width = w
|
||||
|
||||
# ── Sheet2 采样明细 ──
|
||||
ws2 = wb.create_sheet("采样明细")
|
||||
h2 = ["序号", "上下文长度tok", "提示词tok", "缓存tok", "首字ms", "预填充tok/s",
|
||||
"输出tok", "解码tok/s", "总耗时ms", "备注"]
|
||||
ws2.append(h2)
|
||||
style_header(ws2, 1, len(h2))
|
||||
for i, r in enumerate(runs, start=1):
|
||||
m = r.get("metrics") or {}
|
||||
ws2.append([
|
||||
i,
|
||||
r.get("context_length") or m.get("context_length") or "",
|
||||
m.get("prompt_tokens") or "",
|
||||
m.get("cached_tokens") if m.get("cached_tokens") else "",
|
||||
m.get("ttft_ms"),
|
||||
m.get("prefill_speed"),
|
||||
m.get("output_tokens"),
|
||||
m.get("decode_speed"),
|
||||
m.get("total_ms"),
|
||||
r.get("error") or "OK",
|
||||
])
|
||||
for col, w in zip("ABCDEFGHIJ", [8, 14, 12, 10, 12, 14, 12, 14, 12, 30]):
|
||||
ws2.column_dimensions[col].width = w
|
||||
|
||||
# ── Sheet3 日志 ──
|
||||
ws3 = wb.create_sheet("日志")
|
||||
ws3.append(["相对时间(s)", "级别", "内容"])
|
||||
style_header(ws3, 1, 3)
|
||||
for l in logs:
|
||||
ws3.append([l.get("rel", 0), l.get("level", ""), l.get("msg", "")])
|
||||
for col, w in zip("ABC", [14, 10, 90]):
|
||||
ws3.column_dimensions[col].width = w
|
||||
|
||||
bio = io.BytesIO()
|
||||
wb.save(bio)
|
||||
bio.seek(0)
|
||||
return bio
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
app.run(host=config.HOST, port=config.PORT, threaded=True, debug=False)
|
||||
+14
-4
@@ -39,6 +39,7 @@ CREATE TABLE IF NOT EXISTS test_runs(
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
test_id INTEGER NOT NULL,
|
||||
run_index INTEGER DEFAULT 0,
|
||||
context_length INTEGER DEFAULT 0,
|
||||
metrics_json TEXT DEFAULT '{}',
|
||||
error TEXT DEFAULT ''
|
||||
);
|
||||
@@ -65,11 +66,20 @@ def _connect():
|
||||
return conn
|
||||
|
||||
|
||||
def _migrate(conn):
|
||||
"""老库升级：为 test_runs 补 context_length 列"""
|
||||
cur = conn.execute("PRAGMA table_info(test_runs)")
|
||||
cols = [r[1] for r in cur.fetchall()]
|
||||
if "context_length" not in cols:
|
||||
conn.execute("ALTER TABLE test_runs ADD COLUMN context_length INTEGER DEFAULT 0")
|
||||
|
||||
|
||||
def init_db():
|
||||
with _lock:
|
||||
conn = _connect()
|
||||
try:
|
||||
conn.executescript(SCHEMA)
|
||||
_migrate(conn)
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
@@ -202,13 +212,13 @@ def delete_test(tid: int):
|
||||
|
||||
# ───────────────────────── 采样指标 ─────────────────────────
|
||||
|
||||
def add_run(tid: int, run_index: int, metrics: dict, error: str = ""):
|
||||
def add_run(tid: int, run_index: int, metrics: dict, error: str = "", context_length: int = 0):
|
||||
with _lock:
|
||||
conn = _connect()
|
||||
try:
|
||||
conn.execute(
|
||||
"INSERT INTO test_runs(test_id,run_index,metrics_json,error) VALUES(?,?,?,?)",
|
||||
(tid, run_index, json.dumps(metrics, ensure_ascii=False), error))
|
||||
"INSERT INTO test_runs(test_id,run_index,context_length,metrics_json,error) VALUES(?,?,?,?,?)",
|
||||
(tid, run_index, context_length, json.dumps(metrics, ensure_ascii=False), error))
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
@@ -219,7 +229,7 @@ def get_runs(tid: int):
|
||||
conn = _connect()
|
||||
try:
|
||||
rows = conn.execute(
|
||||
"SELECT run_index,metrics_json,error FROM test_runs "
|
||||
"SELECT run_index,context_length,metrics_json,error FROM test_runs "
|
||||
"WHERE test_id=? ORDER BY run_index", (tid,)).fetchall()
|
||||
out = []
|
||||
for r in rows:
|
||||
|
||||
@@ -1,2 +1,3 @@
|
||||
flask>=3.0
|
||||
requests>=2.31
|
||||
openpyxl>=3.1
|
||||
@@ -71,6 +71,26 @@ body {
|
||||
.row > select { flex: 1; }
|
||||
.row > input { flex: 1; }
|
||||
|
||||
/* 带图标前缀的输入框（配置名称 / Base URL / 模型名称） */
|
||||
.icon-input { display: flex; align-items: center; gap: 8px; background: var(--panel2); border: 1px solid var(--border); border-radius: 8px; padding: 0 10px; transition: border-color .15s; }
|
||||
.icon-input:focus-within { border-color: var(--accent); }
|
||||
.icon-input .icon { font-size: 14px; flex-shrink: 0; opacity: .9; }
|
||||
.icon-input input { flex: 1; background: transparent; border: none; outline: none; color: var(--text); font-size: 13px; padding: 8px 0; }
|
||||
|
||||
/* 上下文长度 chips */
|
||||
.chips { display: flex; flex-wrap: wrap; gap: 6px; min-height: 30px; }
|
||||
.chips-empty { color: var(--muted); font-size: 12px; align-self: center; }
|
||||
.chip {
|
||||
display: inline-flex; align-items: center; gap: 6px;
|
||||
padding: 4px 8px 4px 10px; border-radius: 16px; cursor: pointer; user-select: none;
|
||||
background: rgba(79,140,255,.16); border: 1px solid var(--accent); color: var(--accent);
|
||||
font-family: var(--mono); font-size: 12px; transition: .15s;
|
||||
}
|
||||
.chip:hover { filter: brightness(1.15); }
|
||||
.chip.off { background: var(--panel2); border-color: var(--border); color: var(--muted); text-decoration: line-through; }
|
||||
.chip .chip-x { font-size: 11px; line-height: 1; border-radius: 50%; padding: 1px 3px; }
|
||||
.chip .chip-x:hover { background: rgba(255,92,108,.25); color: var(--danger); }
|
||||
|
||||
/* 开关 */
|
||||
.switch-field { display: flex; align-items: center; justify-content: space-between; }
|
||||
.switch { position: relative; display: inline-block; width: 46px; height: 26px; flex-shrink: 0; }
|
||||
|
||||
+21
-9
@@ -40,11 +40,11 @@
|
||||
</div>
|
||||
<div class="field">
|
||||
<label>配置名称</label>
|
||||
<input id="cfg-name" placeholder="例如：DeepSeek-V4">
|
||||
<div class="icon-input"><span class="icon">📛</span><input id="cfg-name" placeholder="例如：DeepSeek-V4"></div>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label>Base URL</label>
|
||||
<input id="cfg-baseurl" placeholder="留空使用官方默认地址">
|
||||
<div class="icon-input"><span class="icon">🌐</span><input id="cfg-baseurl" placeholder="留空使用官方默认地址"></div>
|
||||
<div class="hint" id="cfg-default-url"></div>
|
||||
</div>
|
||||
<div class="field">
|
||||
@@ -56,7 +56,7 @@
|
||||
</div>
|
||||
<div class="field">
|
||||
<label>模型名称</label>
|
||||
<input id="cfg-model" placeholder="如 gpt-4o / deepseek-chat / claude-sonnet-4">
|
||||
<div class="icon-input"><span class="icon">🤖</span><input id="cfg-model" placeholder="如 gpt-4o / deepseek-chat / claude-sonnet-4"></div>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label>温度 Temperature <b id="tmp-val">0.7</b></label>
|
||||
@@ -69,16 +69,27 @@
|
||||
<section class="card">
|
||||
<h2>🚀 速度测试配置</h2>
|
||||
<div class="field">
|
||||
<label>上文长度（目标 tokens）</label>
|
||||
<input id="gen-prompt-tokens" type="number" min="16" step="16" value="2048">
|
||||
<label>上下文长度（tokens，点击切换启停）</label>
|
||||
<div class="chips" id="gen-contexts"></div>
|
||||
<div class="row" style="margin-top:6px">
|
||||
<input id="gen-context-add" type="number" min="16" step="16" placeholder="自定义长度（≥16）">
|
||||
<button class="btn small" id="btn-context-add">＋ 添加</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label>生成长度（max tokens）</label>
|
||||
<input id="gen-max-tokens" type="number" min="1" step="1" value="256">
|
||||
<label>解码输出长度（max tokens）</label>
|
||||
<input id="gen-max-tokens" type="number" min="1" step="1" value="128">
|
||||
</div>
|
||||
<div class="field">
|
||||
<label>采样次数</label>
|
||||
<input id="gen-samples" type="number" min="1" max="50" value="3">
|
||||
<label>每个长度采样次数</label>
|
||||
<input id="gen-samples" type="number" min="1" max="50" value="2">
|
||||
</div>
|
||||
<div class="field switch-field">
|
||||
<label>测试前预热<span class="hint-inline">空转不计速度，避免冷启动偏差</span></label>
|
||||
<label class="switch">
|
||||
<input type="checkbox" id="gen-warmup" checked>
|
||||
<span class="slider"></span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="field switch-field">
|
||||
<label>避免缓存<span class="hint-inline">随机前缀，测真实预填充</span></label>
|
||||
@@ -141,6 +152,7 @@
|
||||
<div class="modal-head">
|
||||
<h2>测试详情 #<span id="dt-id"></span></h2>
|
||||
<div>
|
||||
<button class="btn small primary" id="dt-export-xlsx">⬇ 导出 Excel</button>
|
||||
<button class="btn small primary" id="dt-export">导出 JSON</button>
|
||||
<button class="btn small" id="dt-close">✕</button>
|
||||
</div>
|
||||
|
||||
+112
-7
@@ -21,6 +21,11 @@ let pollTimer = null;
|
||||
let lastLogId = 0;
|
||||
let consoleLogs = []; // 当前测试已加载日志 [{id,level,msg,rel}]
|
||||
|
||||
// 上下文长度：chips 列表 + 启用集合（默认 512/2048/8192/32768/131072）
|
||||
const DEFAULT_CONTEXT_LENGTHS = [512, 2048, 8192, 32768, 131072];
|
||||
let contextLengths = [...DEFAULT_CONTEXT_LENGTHS];
|
||||
let contextLengthsActive = new Set(contextLengths);
|
||||
|
||||
const esc = (s) => String(s ?? "").replace(/[&<>"']/g,
|
||||
(c) => ({ "&": "&", "<": "<", ">": ">", '"': """, "'": "'" }[c]));
|
||||
const fmt = (v, d = "—") => (v === null || v === undefined || isNaN(v) ? d : v);
|
||||
@@ -50,14 +55,58 @@ function currentConfig() {
|
||||
}
|
||||
|
||||
function currentGen() {
|
||||
const lens = contextLengths.filter((l) => contextLengthsActive.has(l));
|
||||
return {
|
||||
prompt_tokens: parseInt($("#gen-prompt-tokens").value) || 2048,
|
||||
max_tokens: parseInt($("#gen-max-tokens").value) || 256,
|
||||
samples: parseInt($("#gen-samples").value) || 3,
|
||||
context_lengths: lens.length ? lens : [2048],
|
||||
max_tokens: parseInt($("#gen-max-tokens").value) || 128,
|
||||
samples: parseInt($("#gen-samples").value) || 2,
|
||||
avoid_cache: $("#gen-avoid-cache").checked,
|
||||
warmup: $("#gen-warmup").checked,
|
||||
};
|
||||
}
|
||||
|
||||
/* ───────────────────────── 上下文长度 chips ───────────────────────── */
|
||||
|
||||
function renderChips() {
|
||||
const box = $("#gen-contexts");
|
||||
box.innerHTML = "";
|
||||
if (!contextLengths.length) {
|
||||
box.innerHTML = '<span class="chips-empty">暂无长度，点击下方“添加”自定义</span>';
|
||||
return;
|
||||
}
|
||||
for (const len of contextLengths) {
|
||||
const active = contextLengthsActive.has(len);
|
||||
const c = document.createElement("span");
|
||||
c.className = "chip" + (active ? "" : " off");
|
||||
c.title = "点击启用/禁用";
|
||||
c.innerHTML = `<span class="chip-v">${len}</span><span class="chip-x">✕</span>`;
|
||||
c.addEventListener("click", (e) => {
|
||||
if (e.target.closest(".chip-x")) {
|
||||
contextLengths = contextLengths.filter((x) => x !== len);
|
||||
contextLengthsActive.delete(len);
|
||||
renderChips();
|
||||
} else {
|
||||
if (contextLengthsActive.has(len)) contextLengthsActive.delete(len);
|
||||
else contextLengthsActive.add(len);
|
||||
renderChips();
|
||||
}
|
||||
});
|
||||
box.appendChild(c);
|
||||
}
|
||||
}
|
||||
|
||||
function addContextLength() {
|
||||
const v = parseInt($("#gen-context-add").value);
|
||||
if (!v || v < 16) { toast("请输入有效长度（≥16）"); return; }
|
||||
if (!contextLengths.includes(v)) {
|
||||
contextLengths.push(v);
|
||||
contextLengthsActive.add(v);
|
||||
contextLengths.sort((a, b) => a - b);
|
||||
renderChips();
|
||||
}
|
||||
$("#gen-context-add").value = "";
|
||||
}
|
||||
|
||||
function updateDefaultUrlHint() {
|
||||
const p = $("#cfg-provider").value;
|
||||
$("#cfg-default-url").textContent = "默认地址：" + PROVIDER_DEFAULT_URL[p];
|
||||
@@ -269,6 +318,7 @@ async function loadHistory() {
|
||||
<td><span class="status-pill ${esc(t.status)}">${STATUS_LABEL[t.status] || t.status}</span></td>
|
||||
<td>
|
||||
<button class="btn small" data-view="${t.id}">查看</button>
|
||||
<button class="btn small" data-xlsx="${t.id}">Excel</button>
|
||||
<button class="btn small danger" data-del="${t.id}">删除</button>
|
||||
</td>`;
|
||||
tb.appendChild(tr);
|
||||
@@ -284,17 +334,43 @@ async function viewDetail(id) {
|
||||
const cfg = t.config || {};
|
||||
const runs = t.runs || [];
|
||||
const logs = t.logs || [];
|
||||
const byLength = s.by_length || {};
|
||||
|
||||
let byLengthHtml;
|
||||
const lens = Object.keys(byLength).sort((a, b) => a - b);
|
||||
if (lens.length) {
|
||||
byLengthHtml = `<table class="mini"><thead><tr>
|
||||
<th>上下文长度tok</th><th>采样(成功/总数)</th><th>首字ms</th><th>预填充tok/s</th>
|
||||
<th>解码tok/s</th><th>提示词tok</th><th>输出tok</th><th>总耗时ms</th>
|
||||
</tr></thead><tbody>` +
|
||||
lens.map((L) => {
|
||||
const bl = byLength[L] || {};
|
||||
return `<tr>
|
||||
<td class="num">${L}</td>
|
||||
<td class="num">${fmt(bl.samples_ok)}/${fmt(bl.samples_total)}</td>
|
||||
<td class="num">${fmt(bl.avg_ttft_ms)}</td>
|
||||
<td class="num">${fmt(bl.avg_prefill_speed)}</td>
|
||||
<td class="num">${fmt(bl.avg_decode_speed)}</td>
|
||||
<td class="num">${fmt(bl.avg_prompt_tokens)}</td>
|
||||
<td class="num">${fmt(bl.avg_output_tokens)}</td>
|
||||
<td class="num">${fmt(bl.avg_total_ms)}</td>
|
||||
</tr>`;
|
||||
}).join("") + `</tbody></table>`;
|
||||
} else {
|
||||
byLengthHtml = '<div class="hint">无成功采样数据</div>';
|
||||
}
|
||||
|
||||
let runsHtml;
|
||||
if (runs.length) {
|
||||
runsHtml = `<table class="mini"><thead><tr>
|
||||
<th>采样</th><th>提示词tok</th><th>缓存tok</th><th>首字ms</th><th>预填充tok/s</th>
|
||||
<th>采样</th><th>上下文tok</th><th>提示词tok</th><th>缓存tok</th><th>首字ms</th><th>预填充tok/s</th>
|
||||
<th>输出tok</th><th>解码tok/s</th><th>总耗时ms</th><th>备注</th>
|
||||
</tr></thead><tbody>` +
|
||||
runs.map((r, i) => {
|
||||
const m = r.metrics || {};
|
||||
return `<tr class="${r.error ? "err" : ""}">
|
||||
<td>${i + 1}</td>
|
||||
<td class="num">${r.context_length || m.context_length || "—"}</td>
|
||||
<td class="num">${fmt(m.prompt_tokens)}</td>
|
||||
<td class="num">${fmt(m.cached_tokens, 0)}</td>
|
||||
<td class="num">${fmt(m.ttft_ms)}</td>
|
||||
@@ -331,11 +407,15 @@ async function viewDetail(id) {
|
||||
<div class="kv-item"><div class="kv-k">平均总耗时</div><div class="kv-v">${fmt(s.avg_total_ms)} ms</div></div>
|
||||
</div>
|
||||
|
||||
<h3>📏 按上下文长度汇总</h3>
|
||||
${byLengthHtml}
|
||||
|
||||
<h3>⚙️ 测试参数</h3>
|
||||
<div class="kv">
|
||||
<div class="kv-item"><div class="kv-k">上文长度</div><div class="kv-v">${g.prompt_tokens ?? "—"} tok</div></div>
|
||||
<div class="kv-item"><div class="kv-k">生成长度</div><div class="kv-v">${g.max_tokens ?? "—"} tok</div></div>
|
||||
<div class="kv-item"><div class="kv-k">采样次数</div><div class="kv-v">${g.samples ?? "—"}</div></div>
|
||||
<div class="kv-item"><div class="kv-k">上下文长度</div><div class="kv-v">${(g.context_lengths || []).join(" / ") || "—"} tok</div></div>
|
||||
<div class="kv-item"><div class="kv-k">解码输出长度</div><div class="kv-v">${g.max_tokens ?? "—"} tok</div></div>
|
||||
<div class="kv-item"><div class="kv-k">每个长度采样</div><div class="kv-v">${g.samples ?? "—"}</div></div>
|
||||
<div class="kv-item"><div class="kv-k">预热(空转)</div><div class="kv-v">${g.warmup === false ? "关" : "开"}</div></div>
|
||||
<div class="kv-item"><div class="kv-k">避免缓存</div><div class="kv-v">${g.avoid_cache ? "开" : "关"}</div></div>
|
||||
<div class="kv-item"><div class="kv-k">温度</div><div class="kv-v">${fmt(cfg.temperature)}</div></div>
|
||||
<div class="kv-item"><div class="kv-k">Base URL</div><div class="kv-v">${esc(cfg.base_url || "（默认）")}</div></div>
|
||||
@@ -362,6 +442,25 @@ function exportDetail() {
|
||||
download(JSON.stringify(window.__detail, null, 2), `test_${window.__detail.id}.json`, "application/json");
|
||||
}
|
||||
|
||||
async function exportXlsx(id) {
|
||||
try {
|
||||
const resp = await fetch(`/api/tests/${id}/export.xlsx`);
|
||||
if (!resp.ok) {
|
||||
const j = await resp.json().catch(() => ({}));
|
||||
throw new Error(j.error || "导出失败");
|
||||
}
|
||||
const blob = await resp.blob();
|
||||
const a = document.createElement("a");
|
||||
a.href = URL.createObjectURL(blob);
|
||||
a.download = `llm_speed_test_${id}.xlsx`;
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
setTimeout(() => { URL.revokeObjectURL(a.href); a.remove(); }, 100);
|
||||
} catch (e) {
|
||||
toast("导出失败：" + e.message);
|
||||
}
|
||||
}
|
||||
|
||||
/* ───────────────────────── 导出 / 下载 ───────────────────────── */
|
||||
|
||||
function download(text, filename, type = "text/plain") {
|
||||
@@ -424,6 +523,8 @@ function bind() {
|
||||
$("#btn-test-conn").addEventListener("click", testConnection);
|
||||
$("#btn-start").addEventListener("click", startTest);
|
||||
$("#btn-cancel").addEventListener("click", stopTest);
|
||||
$("#btn-context-add").addEventListener("click", addContextLength);
|
||||
$("#gen-context-add").addEventListener("keydown", (e) => { if (e.key === "Enter") addContextLength(); });
|
||||
|
||||
$("#btn-clear-console").addEventListener("click", clearConsole);
|
||||
$("#btn-export-log").addEventListener("click", exportCurrentLog);
|
||||
@@ -431,8 +532,10 @@ function bind() {
|
||||
|
||||
$("#history tbody").addEventListener("click", (e) => {
|
||||
const v = e.target.closest("[data-view]");
|
||||
const x = e.target.closest("[data-xlsx]");
|
||||
const d = e.target.closest("[data-del]");
|
||||
if (v) viewDetail(Number(v.dataset.view));
|
||||
if (x) exportXlsx(Number(x.dataset.xlsx));
|
||||
if (d) {
|
||||
const id = Number(d.dataset.del);
|
||||
if (confirm(`确定删除测试 #${id} 及其全部日志？`)) {
|
||||
@@ -444,6 +547,7 @@ function bind() {
|
||||
$("#dt-close").addEventListener("click", closeDetail);
|
||||
$("#detail-mask").addEventListener("click", (e) => { if (e.target === $("#detail-mask")) closeDetail(); });
|
||||
$("#dt-export").addEventListener("click", exportDetail);
|
||||
$("#dt-export-xlsx").addEventListener("click", () => { if (window.__detail) exportXlsx(window.__detail.id); });
|
||||
|
||||
document.addEventListener("keydown", (e) => { if (e.key === "Escape") closeDetail(); });
|
||||
}
|
||||
@@ -452,6 +556,7 @@ function bind() {
|
||||
|
||||
(async function init() {
|
||||
bind();
|
||||
renderChips();
|
||||
updateDefaultUrlHint();
|
||||
loadConfigs();
|
||||
loadHistory();
|
||||
|
||||
@@ -48,26 +48,41 @@ class TestRunner(threading.Thread):
|
||||
def _run(self):
|
||||
provider = self.cfg.get("provider", "openai")
|
||||
model = self.cfg.get("model", "")
|
||||
n = max(1, int(self.gen.get("samples", 3)))
|
||||
target_tokens = max(16, int(self.gen.get("prompt_tokens", 2048)))
|
||||
max_tokens = max(1, int(self.gen.get("max_tokens", 256)))
|
||||
avoid_cache = bool(self.gen.get("avoid_cache"))
|
||||
gen = self.gen
|
||||
|
||||
# 上下文长度列表（支持手动自定义，默认 512/2048/8192/32768/131072）
|
||||
raw_lengths = gen.get("context_lengths") or []
|
||||
if not raw_lengths:
|
||||
# 兼容旧版单值配置
|
||||
raw_lengths = [int(gen.get("prompt_tokens", 2048))]
|
||||
lengths = sorted(set(int(x) for x in raw_lengths if int(x) >= 16)) or [2048]
|
||||
n = max(1, int(gen.get("samples", 2))) # 每个长度采样次数
|
||||
max_tokens = max(1, int(gen.get("max_tokens", 128))) # 解码输出长度
|
||||
avoid_cache = bool(gen.get("avoid_cache"))
|
||||
warmup = bool(gen.get("warmup", True)) # 测试前空转预热
|
||||
|
||||
self.log("INFO", "═══ 开始速度测试 ═══")
|
||||
self.log("INFO", "提供商: %s | 模型: %s" % (lp.PROVIDER_LABELS.get(provider, provider), model))
|
||||
self.log("INFO", "目标上文: %d tokens | 生成长度: %d tokens | 采样: %d 次 | 避免缓存: %s"
|
||||
% (target_tokens, max_tokens, n, "开" if avoid_cache else "关"))
|
||||
self.log("INFO", "上下文长度: %s tokens | 生成长度: %d tokens | 每个长度采样: %d 次 | 预热: %s | 避免缓存: %s"
|
||||
% (" / ".join(str(x) for x in lengths), max_tokens, n,
|
||||
"开" if warmup else "关", "开" if avoid_cache else "关"))
|
||||
|
||||
ratio = self._calibrate()
|
||||
self.ratio = ratio
|
||||
base_prompt = self._build_prompt(target_tokens, ratio)
|
||||
self.log("INFO", "构造基准提示词完成，目标约 %d tokens" % target_tokens)
|
||||
self.log("INFO", "校准完成: %.3f tok/字符（%.2f 字符/token）" % (ratio, 1.0 / ratio))
|
||||
|
||||
for L in lengths:
|
||||
if self.should_stop():
|
||||
raise StopRequested()
|
||||
base_prompt = self._build_prompt(L, ratio)
|
||||
self.log("INFO", "▸▸ 上下文长度 %d tokens（基准提示词构造完成）" % L)
|
||||
if warmup:
|
||||
self._warmup(base_prompt)
|
||||
for i in range(1, n + 1):
|
||||
if self.should_stop():
|
||||
raise StopRequested()
|
||||
prompt = self._finalize_prompt(base_prompt)
|
||||
self.log("INFO", "── 采样 %d/%d 开始 ──" % (i, n))
|
||||
self.log("INFO", "── [%d tok] 采样 %d/%d 开始 ──" % (L, i, n))
|
||||
try:
|
||||
m = lp.call_stream(
|
||||
self.cfg, prompt,
|
||||
@@ -75,15 +90,16 @@ class TestRunner(threading.Thread):
|
||||
log=lambda lv, msg: self.log(lv, msg),
|
||||
should_stop=self.should_stop)
|
||||
m["run_index"] = i
|
||||
self.samples.append({"run_index": i, "ok": True, "metrics": m})
|
||||
db.add_run(self.test_id, i, m)
|
||||
self.log("METRIC", self._fmt_metric(i, n, m))
|
||||
m["context_length"] = L
|
||||
self.samples.append({"run_index": i, "context_length": L, "ok": True, "metrics": m})
|
||||
db.add_run(self.test_id, i, m, context_length=L)
|
||||
self.log("METRIC", self._fmt_metric(L, i, n, m))
|
||||
except StopRequested:
|
||||
raise
|
||||
except ProviderError as e:
|
||||
self.log("ERROR", "采样 %d/%d 失败: %s" % (i, n, e))
|
||||
self.samples.append({"run_index": i, "ok": False, "error": str(e)})
|
||||
db.add_run(self.test_id, i, {}, str(e))
|
||||
self.log("ERROR", "[%d tok] 采样 %d/%d 失败: %s" % (L, i, n, e))
|
||||
self.samples.append({"run_index": i, "context_length": L, "ok": False, "error": str(e)})
|
||||
db.add_run(self.test_id, i, {}, str(e), context_length=L)
|
||||
raise e
|
||||
|
||||
summary = self._make_summary()
|
||||
@@ -94,6 +110,20 @@ class TestRunner(threading.Thread):
|
||||
summary.get("avg_prefill_speed") or 0,
|
||||
summary.get("avg_decode_speed") or 0))
|
||||
|
||||
def _warmup(self, base_prompt):
|
||||
"""空转预热：不计入任何速度统计，用于避免冷启动/首次请求偏慢影响采样"""
|
||||
self.log("INFO", "预热（空转，不计速度）...")
|
||||
try:
|
||||
lp.call_stream(self.cfg, base_prompt,
|
||||
{"max_tokens": 8, "avoid_cache": False},
|
||||
log=lambda lv, msg: self.log(lv, msg),
|
||||
should_stop=self.should_stop)
|
||||
self.log("INFO", "预热完成（不纳入统计）")
|
||||
except StopRequested:
|
||||
raise
|
||||
except Exception as e:
|
||||
self.log("WARN", "预热失败（继续测试）: %s" % e)
|
||||
|
||||
# ───────────────────────── 工具方法 ─────────────────────────
|
||||
|
||||
def _calibrate(self):
|
||||
@@ -129,15 +159,15 @@ class TestRunner(threading.Thread):
|
||||
return "[cache-bust %s]\n%s" % (uuid.uuid4().hex, base)
|
||||
return base
|
||||
|
||||
def _fmt_metric(self, i, n, m):
|
||||
return ("采样 %d/%d 完成 | 提示词 %d tok | 缓存 %d tok | 首字 %s ms | 预填充 %s tok/s"
|
||||
def _fmt_metric(self, L, i, n, m):
|
||||
return ("[%d tok] 采样 %d/%d 完成 | 提示词 %d tok | 缓存 %d tok | 首字 %s ms | 预填充 %s tok/s"
|
||||
" | 输出 %d tok | 解码 %s tok/s | 总耗时 %s ms"
|
||||
% (i, n, m.get("prompt_tokens") or 0, m.get("cached_tokens") or 0,
|
||||
% (L, i, n, m.get("prompt_tokens") or 0, m.get("cached_tokens") or 0,
|
||||
m.get("ttft_ms"), m.get("prefill_speed"), m.get("output_tokens") or 0,
|
||||
m.get("decode_speed"), m.get("total_ms")))
|
||||
|
||||
def _make_summary(self):
|
||||
ok = [s["metrics"] for s in self.samples if s.get("ok")]
|
||||
ok = [s for s in self.samples if s.get("ok")]
|
||||
base = {
|
||||
"provider": self.cfg.get("provider"),
|
||||
"model": self.cfg.get("model"),
|
||||
@@ -149,19 +179,36 @@ class TestRunner(threading.Thread):
|
||||
if not ok:
|
||||
return base
|
||||
|
||||
def avg(k):
|
||||
vals = [m[k] for m in ok if m.get(k) is not None]
|
||||
def avg(ms, k):
|
||||
vals = [m[k] for m in ms if m.get(k) is not None]
|
||||
return round(statistics.mean(vals), 1) if vals else None
|
||||
|
||||
# 按上下文长度分组汇总
|
||||
by_length = {}
|
||||
for L in sorted(set(s["context_length"] for s in ok)):
|
||||
group = [s["metrics"] for s in ok if s["context_length"] == L]
|
||||
by_length[L] = {
|
||||
"samples_total": sum(1 for s in self.samples if s["context_length"] == L),
|
||||
"samples_ok": len(group),
|
||||
"avg_ttft_ms": avg(group, "ttft_ms"),
|
||||
"avg_prefill_speed": avg(group, "prefill_speed"),
|
||||
"avg_decode_speed": avg(group, "decode_speed"),
|
||||
"avg_prompt_tokens": avg(group, "prompt_tokens"),
|
||||
"avg_output_tokens": avg(group, "output_tokens"),
|
||||
"avg_total_ms": avg(group, "total_ms"),
|
||||
}
|
||||
|
||||
okm = [s["metrics"] for s in ok]
|
||||
summary = dict(base)
|
||||
summary.update({
|
||||
"avg_ttft_ms": avg("ttft_ms"),
|
||||
"avg_prefill_speed": avg("prefill_speed"),
|
||||
"avg_decode_speed": avg("decode_speed"),
|
||||
"avg_prompt_tokens": avg("prompt_tokens"),
|
||||
"avg_output_tokens": avg("output_tokens"),
|
||||
"avg_cached_tokens": avg("cached_tokens"),
|
||||
"avg_total_ms": avg("total_ms"),
|
||||
"best_ttft_ms": min([m["ttft_ms"] for m in ok if m.get("ttft_ms") is not None], default=None),
|
||||
"by_length": by_length,
|
||||
"avg_ttft_ms": avg(okm, "ttft_ms"),
|
||||
"avg_prefill_speed": avg(okm, "prefill_speed"),
|
||||
"avg_decode_speed": avg(okm, "decode_speed"),
|
||||
"avg_prompt_tokens": avg(okm, "prompt_tokens"),
|
||||
"avg_output_tokens": avg(okm, "output_tokens"),
|
||||
"avg_cached_tokens": avg(okm, "cached_tokens"),
|
||||
"avg_total_ms": avg(okm, "total_ms"),
|
||||
"best_ttft_ms": min([m["ttft_ms"] for m in okm if m.get("ttft_ms") is not None], default=None),
|
||||
})
|
||||
return summary
|
||||
Reference in New Issue
Block a user