CASE STUDY
2026.08.20 · 候选版本验收 · 个人项目 · AI 协作Aug 20, 2026 · release-candidate review · Personal project · AI-assisted
H2 Manual Copilot:引用点得开,答案就已经可靠吗?
H2 Manual Copilot 面向设备手册问答:读者提出故障代码或操作问题,系统给出有出处的回答,证据不足时拒答。项目在 2026 年 8 月收窄到一个本地可验收的候选版本。
我与 AI 编程工具先把验收对象列清楚:数据能否导入、检索能否命中、引用能否打开对应页面,以及模型能否回答。这些问题依赖的证据不同,不能靠一段顺利的演示一起确认。
原始材料没齐,先做什么
当时缺少正式模型配置和足够的真实手册。等待全部资料会挡住接口和交互开发,直接把模拟输出写成产品能力又会造成误导。
我们使用明确标注的合成手册和确定性模型替身,先检查提问、引用、原始 PDF 与反馈落库的路径。验收表继续保留真实材料、真实 embedding 和语义跨语言检索的缺口,没有把待完成项从表里删掉。
这一阶段可以判断软件是否把引用交到读者手上,不能判断真实设备问题的答案是否可靠。报告里的局部评测值也只对应那组可跑的样本与替身条件。
引用要能回到文档、版本与页码
问答页上的来源标签如果只能显示一段文字,读者仍然无法核查。原型因此保留来源实体,将引用与具体文档、版本和页码关联,并提供打开 PDF 的路径。
这让读者可以对照原文,却没有自动证明模型忠实使用了那段证据。页码跳转和回答正确性仍需分别检查。更复杂的解析回退、页内高亮及真实手册质量也没有在这一轮完成。
创建数据库成功,还要试着撤回
验收中的一次迁移回滚发现:表撤回后,枚举类型没有一起删除。仅从空库向前安装,不会走到这个失败路径。修复后,历史记录保留了测试库里的降级与重新升级检查。
这个问题也改变了我对“可交付”的检查范围:除了能启动,还需要知道更新失败后能退回哪里。发布与回滚清单因此成为候选版本的一部分,实际部署则继续单独记录。
本轮核对固定版本验收报告与实现,没有重新运行模型或设备评估。此处记录的是替身条件下的工程验收,不提供设备操作建议。下一轮需要授权手册与真实模型,再复查回答、拒答和引用是否一致。
H2 Manual Copilot: a clickable citation still needs a trustworthy answer
H2 Manual Copilot explores questions about equipment manuals, with traceable answers and refusal when evidence is insufficient. In August 2026 I narrowed the project with AI coding tools to a locally reviewable release candidate.
We listed separate checks for ingestion, retrieval, source navigation, and answer quality. A smooth demonstration could not establish them all.
Work with explicitly incomplete inputs
Real model configuration and sufficient authentic manuals were missing. We used labeled synthetic manuals and a deterministic model substitute to check the path from question to citation, original PDF, and saved feedback.
The acceptance table retained gaps for real documents, embeddings, and cross-language semantic retrieval. Partial evaluation values applied only to the runnable samples and substitute conditions. They did not measure answers to real equipment questions.
Make the source inspectable
A source label alone leaves the reader unable to check the answer. The prototype links citations to documents, versions, and pages, with a route to the original PDF.
Opening that page supports inspection. It does not establish that a model used the passage faithfully. Answer correctness, richer parsing fallback, and page-level highlighting require their own evidence.
Test the route back
A migration rollback exposed an enum type left behind after tables were removed. Forward installation into an empty database would not encounter that failure. The historical record includes the fix and a downgrade/upgrade check in a test database.
This expanded my delivery criteria to include recovery after an unsuccessful update. The candidate included release and rollback checklists; deployment remained a separate action.
I reviewed the fixed-version acceptance report and implementation without rerunning models or evaluating equipment use. This is an engineering account under substitute conditions. The next check needs authorized manuals and a real model to examine answers, refusals, and citations together.