auto 任务运行中实时显示已爬/待爬缓存数; 人工停止后继续爬取验证
- engine: CrawlJob 增加 _auto_visited/_auto_pending 运行中实时状态 (含发现新链接入队后刷新) - app.py 详情接口: 任务运行中优先读内存实时值, 不再显示上次运行结束的旧数字 - 端到端验证: 人工停止(已爬3,缓存18) → 继续爬取(跳过seed,爬完18页,缓存0) ✅ - 运行中采样: visited 1→2 实时增长, pending 15→14 同步递减 ✅
This commit is contained in:
@@ -308,11 +308,19 @@ def api_task_detail(tid):
|
||||
result["run_page"] = page
|
||||
result["run_pages"] = pages
|
||||
result["run_page_size"] = page_size
|
||||
# auto 状态数量 (独立文件)
|
||||
# auto 状态数量: 运行中优先读内存实时值, 否则读状态文件
|
||||
if task.get("mode") == "auto":
|
||||
live_v = live_p = None
|
||||
with JOBS_LOCK:
|
||||
job = JOBS.get(tid)
|
||||
if job:
|
||||
live_v = getattr(job, "_auto_visited", None)
|
||||
live_p = getattr(job, "_auto_pending", None)
|
||||
st = store.load_auto_state(tid, task)
|
||||
result["auto_pending_count"] = len(st.get("pending", []))
|
||||
result["auto_visited_count"] = len(st.get("visited", []))
|
||||
result["auto_pending_count"] = (
|
||||
live_p if live_p is not None else len(st.get("pending", [])))
|
||||
result["auto_visited_count"] = (
|
||||
live_v if live_v is not None else len(st.get("visited", [])))
|
||||
return jsonify(result)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user