← 返回全部案例

CASE STUDY

2026.08.20 · 候选版本验收 · 个人项目 · AI 协作

整理于约 5 分钟

H2 Manual Copilot:引用点得开,答案就已经可靠吗?

H2 Manual Copilot 面向设备手册问答:读者提出故障代码或操作问题,系统给出有出处的回答,证据不足时拒答。项目在 2026 年 8 月收窄到一个本地可验收的候选版本。

我与 AI 编程工具先把验收对象列清楚:数据能否导入、检索能否命中、引用能否打开对应页面,以及模型能否回答。这些问题依赖的证据不同,不能靠一段顺利的演示一起确认。

原始材料没齐,先做什么

当时缺少正式模型配置和足够的真实手册。等待全部资料会挡住接口和交互开发,直接把模拟输出写成产品能力又会造成误导。

我们使用明确标注的合成手册和确定性模型替身,先检查提问、引用、原始 PDF 与反馈落库的路径。验收表继续保留真实材料、真实 embedding 和语义跨语言检索的缺口,没有把待完成项从表里删掉。

这一阶段可以判断软件是否把引用交到读者手上,不能判断真实设备问题的答案是否可靠。报告里的局部评测值也只对应那组可跑的样本与替身条件。

引用要能回到文档、版本与页码

问答页上的来源标签如果只能显示一段文字,读者仍然无法核查。原型因此保留来源实体,将引用与具体文档、版本和页码关联,并提供打开 PDF 的路径。

这让读者可以对照原文,却没有自动证明模型忠实使用了那段证据。页码跳转和回答正确性仍需分别检查。更复杂的解析回退、页内高亮及真实手册质量也没有在这一轮完成。

创建数据库成功,还要试着撤回

验收中的一次迁移回滚发现:表撤回后,枚举类型没有一起删除。仅从空库向前安装,不会走到这个失败路径。修复后,历史记录保留了测试库里的降级与重新升级检查。

这个问题也改变了我对“可交付”的检查范围:除了能启动,还需要知道更新失败后能退回哪里。发布与回滚清单因此成为候选版本的一部分,实际部署则继续单独记录。

本轮核对固定版本验收报告与实现,没有重新运行模型或设备评估。此处记录的是替身条件下的工程验收,不提供设备操作建议。下一轮需要授权手册与真实模型,再复查回答、拒答和引用是否一致。

H2 Manual Copilot: a clickable citation still needs a trustworthy answer

H2 Manual Copilot explores questions about equipment manuals, with traceable answers and refusal when evidence is insufficient. In August 2026 I narrowed the project with AI coding tools to a locally reviewable release candidate.

We listed separate checks for ingestion, retrieval, source navigation, and answer quality. A smooth demonstration could not establish them all.

Work with explicitly incomplete inputs

Real model configuration and sufficient authentic manuals were missing. We used labeled synthetic manuals and a deterministic model substitute to check the path from question to citation, original PDF, and saved feedback.

The acceptance table retained gaps for real documents, embeddings, and cross-language semantic retrieval. Partial evaluation values applied only to the runnable samples and substitute conditions. They did not measure answers to real equipment questions.

Make the source inspectable

A source label alone leaves the reader unable to check the answer. The prototype links citations to documents, versions, and pages, with a route to the original PDF.

Opening that page supports inspection. It does not establish that a model used the passage faithfully. Answer correctness, richer parsing fallback, and page-level highlighting require their own evidence.

Test the route back

A migration rollback exposed an enum type left behind after tables were removed. Forward installation into an empty database would not encounter that failure. The historical record includes the fix and a downgrade/upgrade check in a test database.

This expanded my delivery criteria to include recovery after an unsuccessful update. The candidate included release and rollback checklists; deployment remained a separate action.

I reviewed the fixed-version acceptance report and implementation without rerunning models or evaluating equipment use. This is an engineering account under substitute conditions. The next check needs authorized manuals and a real model to examine answers, refusals, and citations together.