Skip to content

part-0002 相关工作与定位:continuing 任务谱系

摘要

Table 1 把 E-Commerce Bench 与五个 continuing business-operation 基准并排对比(列:领域 / Steps per run / 并发运营单元数 / 真实数据 / 谈判 / multi-agent / adversarial / reproducible / open source):Vending-Bench(≤2000 步、1 单元、无真实数据‡、有谈判与对抗但不可复现不开源)、 Vending-Bench 2(3000–6000 步、同样无真实数据‡、对抗供应商加入但仍 LLM 驱动)、YC-Bench (~500 步、确定性创业年、rule-based 客户、无谈判、可复现开源)、MerchantBench(~700 步、 365 天卖家侧、真实数据、供应商以随机事件到达、无谈判无对抗)、RetailBench(~1000 步、 180 天超市、privileged oracle、无谈判无对抗)。E-Commerce Bench:1000–4000 步、≤4 店并发、 真实数据、有谈判、multi-agent(有对手方)、有 adversarial、可复现(§3.5.2 确定性内核)、开源 ——六项属性一次集齐。

§2.1 Episodic Long-Horizon:episodic 基准设一个终态目标、奖励达到它的多步执行。三类: work-value(GDPval 对知识工作按专家判断打分;TheAgentCompany 把模拟员工的活设在软件公司里)、 environment-grounded(SWE-bench 用自己测试集解决真实 GitHub issue;OSWorld 在多步 GUI 上 驱动真实 OS)、exploration-driven(UltraHorizon 隐藏部分可观测世界的规则;EdgeBench 连续 学一个环境 12–72 小时)。共同局限:目标一达成评测即止 → 测不出动态环境里跨数百子任务 的依赖跟踪、意外失败恢复、用累积历史优化未来行动。

§2.2 Continuing Long-Horizon:continuing 任务无终态、拿累积得分。形态包括交易组合求累计 收益(Qian et al. 2025;Chen et al. 2026)、自我进化 agent 长技能库/改自己代码(Voyager、 Darwin Godel Machine)。E-Commerce Bench 所属的 business-operation 线:Vending-Bench 起 头(一台售货机 ~2000 条消息、暴露长期连贯性失败、供应商回复 LLM 生成 → run 随机); Vending-Bench 2 扩到整年 + 对抗供应商(仍 LLM 驱动);YC-Bench 确定性创业年(rule-based 客户、无谈判);MerchantBench 365 天卖家侧真实数据(供应商随机事件);RetailBench 180 天 超市对 privileged oracle——两者都无谈判无对抗。TERMS-Bench 在 business-operation 线之外: 用 rule-based 对手在 sampled regimes 里诊断双边价格谈判,有脚本对手但没有让成交落地生效的 经济。

要点归纳

  • 竞争格局一句话:Vending-Bench 线用 LLM 对手换来了谈判+对抗,丢了可复现与真实经济; 真实数据零售对用确定性换掉了谈判与对抗;E-Commerce Bench 全要
  • Table 1 的「Op. units ≤4」是其它基准(全部 1 单元)没有的复杂度维度——并发多店管理
  • 空白定位:没有任何其它条目把「会谈判的对手」与「可复现的供应商经济」配对

术语 / 概念

  • work-value / environment-grounded / exploration-driven — §2.1 对 episodic 基准的三分
  • business-operation simulation(经营模拟线) — continuing 任务谱系中 E-Commerce Bench 所属的分支:agent 经营一家 going concern、按累积余额评判
  • privileged oracle(特权预言机) — RetailBench 的设定(对手/环境对 agent 过度可见), 与本文的信息不对称设计相对
  • rule-based counterpart(基于规则的对手) — TERMS-Bench / YC-Bench 式确定性对手, 本文内核设计的前身

原句摘录

However long these tasks run, each ends the moment its goal is met, reducing evaluation solely to the quality of the final deliverable. Consequently, this setup fails to assess an agent's ability in dynamic environments to track dependencies across hundreds of sub-tasks, recover from unexpected failures, or leverage accumulated history to optimize future actions.

The Vending-Bench line couples negotiation with adversarial suppliers in its sequel, but only through sampled LLM parties and without an open, real-data economy, while the data-grounded retail pair buys reproducibility by giving those dynamics up. E-Commerce Bench holds all six properties at once, and no other entry pairs a negotiating counterpart with reproducible supplier economics.

疑问 / 待查

  • Table 1 的 Steps/run 是「taken or inferred from each paper」——对比口径本身带推断成分
  • MerchantBench 与 RetailBench 的具体机制(随机事件供应商 / oracle 设计)值得对照原文评估 「六项全有」主张的边界
Backlinks (1)
flowchart LR
  n0["ch-0001 凶案、侦探与母亲的信(第 2–163 章)"]
  n1["ch-0002 坦白、出逃与和解(第 163–233 章)"]
  n2["The Curious Incident of the Dog in the Night-Time 人物"]
  n3["The Curious Incident of the Dog in the Night-Time(整理完成)"]
  n4["The Curious Incident of the Dog in the Night-Time 笔记"]
  n5["The Curious Incident of the Dog in the Night-Time 故事线"]
  n6["E-Commerce Bench(整理完成)"]
  n7["E-Commerce Bench 笔记"]
  n8["part-0001 摘要与引言:一年期电商运营基准(Abstract + §1)"]
  n9["part-0002 相关工作与定位:continuing 任务谱系(Table 1 + §2)"]
  n10["part-0003 基准设计:四层架构与确定性经济(§3 主体)"]
  n11["part-0004 实验结果:18 模型的多维画像(Table 2 + §4)"]
  n12["part-0005 结论与参考文献(§5 + References)"]
  n13["part-0006 工具集与 agent harness:18 工具与上下文管理(Table 3 + Appendix A)"]
  n14["part-0007 数据层:类目、店型、日历与供应商(Appendix B)"]
  n15["part-0008 经济引擎:13 步结算与需求/退货/成本公式(Table 7 + Appendix C)"]
  n16["part-0009 确定性谈判内核:决策函数与骗局目录(Appendix D)"]
  n17["part-0010 评测指标定义:六个维度与统计口径(Appendix E)"]
  n18["part-0011 每维度结果导览与模型/实验设置(Table 10 + Appendix F)"]
  n19["part-0012 失败案例研究:骗局实录与一次破产(Appendix G)"]
  n20["part-0013 失败模式规则表与提示词设计(Table 18 + Appendix H)"]
  n21["The End of Software Engineering(整理完成)"]
  n22["The End of Software Engineering 笔记"]
  n23["part-0001 摘要与引言:范式重构的宣告(Abstract + §1 前段)"]
  n24["part-0002 第一性原理:传统软件与 Agent 系统的形式化模型(§1 后段 + §2)"]
  n25["part-0003 三代交付史与 AI→Software→Result 的失败(§3.1–3.2)"]
  n26["part-0004 Agent→Result 与 Agentic Engineering 学科(§3.3 + §4)"]
  n27["part-0005 实证证据与 EvoClaw 落差(§5)"]
  n28["part-0006 四阶段演进路线图(前段:表 3 + Stage I–III)"]
  n29["part-0007 路线图后段与建议:实践者与研究者(§6.3 后段 + §6.4 + §7.1–7.2)"]
  n30["part-0008 组织建议、结论与参考文献(§7.3 + §8)"]
  n31["ch-0001 Introduction(为什么做数据可视化)"]
  n32["ch-0002 选工具讲你的数据故事(Ch 1)"]
  n33["ch-0003 强化电子表格技能(Ch 2)"]
  n34["ch-0004 找到并质询你的数据(Ch 3)"]
  n35["ch-0005 清洗脏数据(Ch 4)"]
  n36["ch-0006 做有意义的比较(Ch 5)"]
  n37["ch-0007 图表化你的数据(Ch 6)"]
  n38["ch-0008 地图化你的数据(Ch 7)"]
  n39["ch-0009 表格化你的数据(Ch 8)"]
  n40["ch-0010 嵌入网页(Ch 9)"]
  n41["ch-0011 用 GitHub 编辑与托管代码(Ch 10)"]
  n42["ch-0012 Chart.js 与 Highcharts 模板(Ch 11)"]
  n43["ch-0013 Leaflet 地图模板(Ch 12)"]
  n44["ch-0014 转换你的地图数据(Ch 13)"]
  n45["ch-0015 识别谎言、减少偏差(Ch 14)"]
  n46["ch-0016 讲述并展示你的数据故事(Ch 15)"]
  n47["ch-0017 附录 A 排查常见问题"]
  n48["Hands-On Data Visualization(整理完成)"]
  n49["Reading"]
  n0 --> n4
  n3 --> n0
  n3 --> n1
  n3 --> n2
  n3 --> n4
  n3 --> n5
  n5 --> n3
  n6 --> n7
  n6 --> n8
  n6 --> n9
  n6 --> n10
  n6 --> n11
  n6 --> n12
  n6 --> n13
  n6 --> n14
  n6 --> n15
  n6 --> n16
  n6 --> n17
  n6 --> n18
  n6 --> n19
  n6 --> n20
  n21 --> n22
  n21 --> n23
  n21 --> n24
  n21 --> n25
  n21 --> n26
  n21 --> n27
  n21 --> n28
  n21 --> n29
  n21 --> n30
  n22 --> n21
  n30 --> n22
  n48 --> n31
  n48 --> n32
  n48 --> n33
  n48 --> n34
  n48 --> n35
  n48 --> n36
  n48 --> n37
  n48 --> n38
  n48 --> n39
  n48 --> n40
  n48 --> n41
  n48 --> n42
  n48 --> n43
  n48 --> n44
  n48 --> n45
  n48 --> n46
  n48 --> n47
  n49 --> n3
  n49 --> n6
  n49 --> n21
  n49 --> n48
  click n0 "../../curious-incident/ch-0001/" "ch-0001 凶案、侦探与母亲的信(第 2–163 章)"
  click n1 "../../curious-incident/ch-0002/" "ch-0002 坦白、出逃与和解(第 163–233 章)"
  click n2 "../../curious-incident/characters/" "The Curious Incident of the Dog in the Night-Time 人物"
  click n3 "../../curious-incident/" "The Curious Incident of the Dog in the Night-Time(整理完成)"
  click n4 "../../curious-incident/notes/" "The Curious Incident of the Dog in the Night-Time 笔记"
  click n5 "../../curious-incident/storyline/" "The Curious Incident of the Dog in the Night-Time 故事线"
  click n6 "../" "E-Commerce Bench(整理完成)"
  click n7 "../notes/" "E-Commerce Bench 笔记"
  click n8 "../part-0001/" "part-0001 摘要与引言:一年期电商运营基准(Abstract + §1)"
  click n9 "./" "part-0002 相关工作与定位:continuing 任务谱系(Table 1 + §2)"
  click n10 "../part-0003/" "part-0003 基准设计:四层架构与确定性经济(§3 主体)"
  click n11 "../part-0004/" "part-0004 实验结果:18 模型的多维画像(Table 2 + §4)"
  click n12 "../part-0005/" "part-0005 结论与参考文献(§5 + References)"
  click n13 "../part-0006/" "part-0006 工具集与 agent harness:18 工具与上下文管理(Table 3 + Appendix A)"
  click n14 "../part-0007/" "part-0007 数据层:类目、店型、日历与供应商(Appendix B)"
  click n15 "../part-0008/" "part-0008 经济引擎:13 步结算与需求/退货/成本公式(Table 7 + Appendix C)"
  click n16 "../part-0009/" "part-0009 确定性谈判内核:决策函数与骗局目录(Appendix D)"
  click n17 "../part-0010/" "part-0010 评测指标定义:六个维度与统计口径(Appendix E)"
  click n18 "../part-0011/" "part-0011 每维度结果导览与模型/实验设置(Table 10 + Appendix F)"
  click n19 "../part-0012/" "part-0012 失败案例研究:骗局实录与一次破产(Appendix G)"
  click n20 "../part-0013/" "part-0013 失败模式规则表与提示词设计(Table 18 + Appendix H)"
  click n21 "../../end-of-software-engineering/" "The End of Software Engineering(整理完成)"
  click n22 "../../end-of-software-engineering/notes/" "The End of Software Engineering 笔记"
  click n23 "../../end-of-software-engineering/part-0001/" "part-0001 摘要与引言:范式重构的宣告(Abstract + §1 前段)"
  click n24 "../../end-of-software-engineering/part-0002/" "part-0002 第一性原理:传统软件与 Agent 系统的形式化模型(§1 后段 + §2)"
  click n25 "../../end-of-software-engineering/part-0003/" "part-0003 三代交付史与 AI→Software→Result 的失败(§3.1–3.2)"
  click n26 "../../end-of-software-engineering/part-0004/" "part-0004 Agent→Result 与 Agentic Engineering 学科(§3.3 + §4)"
  click n27 "../../end-of-software-engineering/part-0005/" "part-0005 实证证据与 EvoClaw 落差(§5)"
  click n28 "../../end-of-software-engineering/part-0006/" "part-0006 四阶段演进路线图(前段:表 3 + Stage I–III)"
  click n29 "../../end-of-software-engineering/part-0007/" "part-0007 路线图后段与建议:实践者与研究者(§6.3 后段 + §6.4 + §7.1–7.2)"
  click n30 "../../end-of-software-engineering/part-0008/" "part-0008 组织建议、结论与参考文献(§7.3 + §8)"
  click n31 "../../hands-on-data-visualization/ch-0001/" "ch-0001 Introduction(为什么做数据可视化)"
  click n32 "../../hands-on-data-visualization/ch-0002/" "ch-0002 选工具讲你的数据故事(Ch 1)"
  click n33 "../../hands-on-data-visualization/ch-0003/" "ch-0003 强化电子表格技能(Ch 2)"
  click n34 "../../hands-on-data-visualization/ch-0004/" "ch-0004 找到并质询你的数据(Ch 3)"
  click n35 "../../hands-on-data-visualization/ch-0005/" "ch-0005 清洗脏数据(Ch 4)"
  click n36 "../../hands-on-data-visualization/ch-0006/" "ch-0006 做有意义的比较(Ch 5)"
  click n37 "../../hands-on-data-visualization/ch-0007/" "ch-0007 图表化你的数据(Ch 6)"
  click n38 "../../hands-on-data-visualization/ch-0008/" "ch-0008 地图化你的数据(Ch 7)"
  click n39 "../../hands-on-data-visualization/ch-0009/" "ch-0009 表格化你的数据(Ch 8)"
  click n40 "../../hands-on-data-visualization/ch-0010/" "ch-0010 嵌入网页(Ch 9)"
  click n41 "../../hands-on-data-visualization/ch-0011/" "ch-0011 用 GitHub 编辑与托管代码(Ch 10)"
  click n42 "../../hands-on-data-visualization/ch-0012/" "ch-0012 Chart.js 与 Highcharts 模板(Ch 11)"
  click n43 "../../hands-on-data-visualization/ch-0013/" "ch-0013 Leaflet 地图模板(Ch 12)"
  click n44 "../../hands-on-data-visualization/ch-0014/" "ch-0014 转换你的地图数据(Ch 13)"
  click n45 "../../hands-on-data-visualization/ch-0015/" "ch-0015 识别谎言、减少偏差(Ch 14)"
  click n46 "../../hands-on-data-visualization/ch-0016/" "ch-0016 讲述并展示你的数据故事(Ch 15)"
  click n47 "../../hands-on-data-visualization/ch-0017/" "ch-0017 附录 A 排查常见问题"
  click n48 "../../hands-on-data-visualization/" "Hands-On Data Visualization(整理完成)"
  click n49 "../../" "Reading"