多出来的一片叶子：我重新认识空间智能的过程

The Extra Leaf: Rethinking Spatial Intelligence

最近，因为参加群核科技举办的 AI 黑客松和一些小型 Workshop，我开始接触 AI 3D 与空间智能相关的技术和工具。

在这些活动中，我逐渐了解到 Lux3D、Aholo，以及利用图片、视频生成三维物体和空间的不同方法。

为了更系统地理解这些技术，我开始学习由群核科技与观猹合作推出的空间智能课程。

最初，我只是想知道：这些工具能够做什么？如果想开发一个真正可以交互的 3D 应用，需要掌握哪些知识？

但随着学习深入，我发现自己关注的问题发生了变化。

从「AI 能不能生成一个三维模型」，逐渐变成了「AI 究竟理解了这个三维世界的多少信息」。

而让我印象很深的，是课程里一只背面多出了一片叶子的杯子。

01｜一只杯子背面多出来的叶子

在 Lux3D 的课程中，有一个生成陶瓷纪念杯的案例。

目标很明确：白色圆筒形杯身，右侧连接着完整的把手，正面中央有一片绿色叶子，而背面应该保持纯白。

生成后的杯子，正面看起来没有什么问题。

但当课程作者把它转到背面时，发现背面竟然也出现了一片叶子。

这件事让我觉得很有意思。

对于一个二维图像生成任务，我们可能只需要判断画面是否符合预期。但对于一个可以旋转的三维物体，正面漂亮并不意味着整体正确。

它还有背面、侧面、顶部和底部。

课程中还介绍了白模与贴图的区别。通过切换白模，可以判断把手是否连接完整、杯口是否敞开；通过检查贴图，则可以确认颜色、材质和图案位置是否正确。

这让我开始意识到：

一个模型看起来合理，不代表它在三维空间中处处一致。

更进一步说，模型能够生成一个具有合理外观的物体，并不等于它已经像人一样理解这个物体的结构。

在没有充分信息的地方，AI 仍然可能生成看似合理、实际却不符合要求的内容。

02｜从二维图像到三维世界，并没有想象中简单

在此之前，我对 3D 技术的理解相对直观：通过照片或文字生成模型，再把它放到网页里展示。

但在学习三维视觉的过程中，我开始接触相机成像、多视角几何、深度估计，以及 COLMAP、VGGT、NeRF、3D Gaussian Splatting（3DGS）等技术。

我才逐渐理解，一张普通照片本身并不会直接告诉计算机物体的真实距离。

照片中的二维像素，是三维世界经过相机投影形成的结果。

要从中恢复空间结构，就需要更多线索。

例如，从不同位置观察同一物体时，物体相对背景的位置会发生变化。这种视差可以帮助我们估计深度。

但如果拍摄时只在原地转动手机，没有足够的位置变化，即使录制了很多画面，也不代表得到了充分的三维信息。

在 3DGS 手机采集课程里，我还学到了图像重叠、拍摄路线与遮挡之间的关系。

原来，一段能让人流畅观看的视频，不一定就是一段适合三维重建的视频。

拍摄时还需要考虑相邻画面的共同区域、运动模糊、光照变化，以及那些始终没有被相机观察到的表面。

这让我对「重建」这个词有了新的认识。

三维重建不是简单地把照片变成立体图片，而是根据有限的观察信息，恢复或推断一个可被重新观察的空间。

不同技术的表示方式和能力边界也并不相同。

点云、网格、NeRF 和 3DGS 都与三维场景有关，却不能被视为完全等价的结果。

一个 3DGS 场景可以具有逼真的视觉效果，但如果想进一步测量尺寸、实现物理碰撞或者编辑具体物体，通常还需要额外的数据或处理。

视觉上的真实，并不自动意味着几何上的准确。

03｜空间智能的四个方向

课程把空间智能的相关能力组织为四个主要方向：

空间重建（Spatial Reconstruction）：根据照片、视频等观察信息，重建三维物体或场景。

空间理解（Spatial Understanding）：识别空间中的物体、位置、形状，以及它们之间的关系。

空间生成（Spatial Generation）：根据文字、图片或其他条件，生成新的三维内容。

空间行动（Spatial Action）：让智能体在环境中进行导航、操作和反馈调整。

这样的分类帮助我把原本比较零散的技术概念组织了起来。

尤其是空间理解和空间行动之间的区别，让我印象很深。

一个视觉模型能够识别图片中的桌子、椅子和门，并不代表它已经知道如何绕开椅子走到门前。

同样，一个虚拟角色能够播放走路动画，也不代表它真正完成了空间导航。

在多智能体空间模拟课程里，有一个很适合初学者理解的例子：让一个小人在房间里寻找出口。

从俯视图看，门在哪里可能一目了然。

但对于房间中的角色来说，墙壁和家具会遮挡视线。它需要根据当前观察选择行动，并在移动后重新观察环境。

一个完整的行动过程至少包括：

观察 → 决策 → 执行 → 环境变化 → 再次观察。

如果模型说出了「向前走」，但角色的位置没有更新，那么行动实际上没有发生。

如果角色成功找到门，是因为程序直接告诉了它出口坐标，也不能把结果简单解释成自主视觉探索。

这些区别让我开始更关注一项 AI 能力背后的实现与验证方式，而不只是最终演示效果。

04｜我开始把课程整理成自己的知识卡片

在学习过程中，我没有只收藏课程链接，而是尝试和 AI 一起，把课程内容重新整理成更容易理解与复习的知识卡片。

从空间智能导论、3D 视觉，到空间重建、理解、生成和行动，再到 Lux3D、Aholo API、3DGS 手机采集与多智能体模拟，我们逐渐整理出了多个独立的学习页面。

这些页面包含概念解释、技术对比、交互示意和自测。

当课程越来越多时，我又遇到了另一个问题：

这些知识应该如何组织，才能让它们之间的联系更清楚？

最后，我决定把它们统一放进 Lumi Lab 的「空间智能学习手帖」中。

我希望它不只是课程笔记的集合，而是一套可以持续扩展的个人学习档案。

不过，这个过程也让我意识到一个值得警惕的问题。

AI 可以很快帮我整理概念、生成页面，甚至制作具有交互效果的技术示意。

但生成了一张专业的知识卡片，并不意味着我已经掌握了卡片上的技术。

我需要区分：

• 阅读过的知识；
• 能够用自己的语言解释的概念；
• 已经通过实验验证的现象；
• 尚未动手实践的技术能力。

尤其是在 AI 辅助开发的过程中，页面做出来了、代码能够运行，不代表我已经理解其中所有实现细节。

所以，相比单纯积累更多漂亮的页面，我更希望这些知识卡片能够帮助自己发现尚未理解的地方，并逐渐把理论和实践连接起来。

05｜从技术演示到真实产品，还差什么？

这次学习的最后一部分，开始从技术知识转向产品实践。

我发现，这也是自己比较感兴趣的方向。

在一个 Workshop 或黑客松里，我们很容易被有趣的技术能力吸引。

一张图片生成 3D 模型、一段视频重建真实空间，或者让多个 AI 角色在房间中活动，这些都很适合作为技术演示。

但如果要把它们做成产品，问题就不同了。

用户为什么需要它？

例如，假设我们开发一个可以旋转查看三维模型的网页。

如果用户只是欣赏一件虚拟展品，那么流畅的视角控制和良好的视觉效果可能已经足够。

但如果用户想在购买家具前判断它能否放进自己的房间，那么准确的尺寸、比例和空间关系就变得更加重要。

如果用户想在真实空间重建的场景中玩游戏，还需要处理碰撞、可走区域和交互规则。

同样是 AI 3D，不同的使用目标，会带来完全不同的产品要求。

课程中关于 MVP 的思路也让我重新思考开发顺序。

与其一开始就尝试做一个完整的三维虚拟校园，不如先选择一个具体地点，让用户能够打开场景、找到一处信息，再观察他是否能够独立完成这个任务。

先验证一段有意义的体验，再决定是否值得扩展。

技术能力决定了我们可能做什么，而用户需求帮助我们判断应该先做什么。

06｜下一步：把理解变成实际验证

这次学习让我初步建立了关于空间智能的知识框架，但其中的判断还要靠实践来检验。

「Lux3D 模型生成 → 质量检查 → GLB 导出 → 网页加载」这条流程，已经在我的 3D 益智游戏《最后一块地》里真实走过一遍：工厂、住宅、商店和公园四栋建筑，都是先用参考图生成带贴图的 GLB，在网页查看器上做正面、侧面和背面的检查——正是那只杯子教我的检查方式——再导出、记录文件结构和哈希，最后接入 three.js，并处理好加载失败时的几何回退。

但我要诚实地区分：那次实践中，生成、检查和接入大多由 AI 辅助执行，我做的是授权、判断和验收；真实手机、低配设备和慢网下的加载性能也还没有验证。所以这条流程对我来说是「见过、用过」，还不是「独立完成过」。我希望之后亲手把它再走一遍，把每个环节变成自己能复现的判断。

另一个还没有开始的实践，是真实场景的 3DGS 手机采集：观察拍摄路线、遮挡和图像重叠对重建质量的影响。

之后，再从具体使用场景出发，判断这些能力能够帮助用户完成什么任务。

在这个过程中，我想继续记录真实的输入、生成结果、失败案例和修改过程。

我并不希望把「接触过某个工具」直接写成「掌握了某项技术」，也不想因为 AI 辅助完成了代码，就忽略自己尚未理解的部分。

对我来说，更有价值的记录可能是：

我提出过什么问题，尝试过什么方法，得到了什么结果，又是根据什么证据调整了原先的想法。

---

回看这次学习，我最大的变化，是开始从「AI 能生成什么」转向关注「AI 究竟理解了什么，又能在多大程度上可靠地完成任务」。

从一只背面多出叶子的杯子，到一间可以被重建的房间，再到能够观察和行动的虚拟角色，空间智能让我看到了一条从视觉生成通往环境理解与交互的技术路线。

这条路线还有许多值得继续探索的地方。

一个能被看见的三维世界，只是开始。

真正让我感兴趣的，是机器如何理解这个世界、与这个世界发生交互，以及我们怎样把这些能力转化成真正有用的产品。

---

参考资料

• 观猹 AI 3D 空间智能课程：Lux3D 从 0 到 1
• 3DGS 手机采集
• 多智能体空间模拟
• 从课程项目到产品实践
• 最后一块地 The Last Plot：使用真实 Lux3D 资产的 3D 益智游戏

Recently, through the AI hackathon and several small workshops hosted by Manycore, I began getting hands-on with the technologies and tools behind AI 3D and spatial intelligence.

Through these events, I gradually learned about Lux3D, Aholo, and the different approaches that generate 3D objects and spaces from images and video.

To understand these technologies more systematically, I started working through the spatial intelligence course co-produced by Manycore and Guancha (观猹).

At first, I only wanted to know: what can these tools do? And if I wanted to build a genuinely interactive 3D application, what would I need to learn?

But as I went deeper, I noticed the question I cared about had changed.

It shifted from “Can AI generate a 3D model?” to “How much of this 3D world does AI actually understand?”

And what left the deepest impression was a cup in the course that had grown an extra leaf on its back.

01 | The extra leaf on the back of a cup

In the Lux3D course, there is a case about generating a ceramic souvenir mug.

The goal was clear: a white cylindrical body, a complete handle attached on the right, a single green leaf centered on the front — and a plain white back.

The generated mug looked fine from the front.

But when the course author rotated it around, the back had grown a leaf of its own.

I found this fascinating.

For a 2D image generation task, we might only need to check whether the picture matches expectations. But for a 3D object you can rotate, looking good from the front does not mean it is correct as a whole.

There is still the back, the sides, the top and the bottom.

The course also explains the difference between the untextured model and the texture. Switching to the untextured model shows whether the handle is fully connected and whether the rim is open; checking the texture confirms colors, materials, and where the patterns sit.

This made me realize:

A model that looks plausible is not the same as a model that is consistent everywhere in 3D space.

To go one step further: an AI that can generate an object with a reasonable appearance has not necessarily understood its structure the way a person does.

Where information is insufficient, AI can still produce content that looks plausible but fails the actual requirement.

02 | From 2D images to a 3D world is harder than it looks

Before this, my understanding of 3D technology was fairly intuitive: generate a model from photos or text, then put it on a web page.

But while learning 3D vision, I started to encounter camera imaging, multi-view geometry, depth estimation, and technologies such as COLMAP, VGGT, NeRF, and 3D Gaussian Splatting (3DGS).

Only then did I gradually understand that an ordinary photo never directly tells the computer an object’s true distance.

The 2D pixels in a photo are the result of the 3D world being projected through a camera.

Recovering spatial structure from them requires more clues.

For example, when you observe the same object from different positions, its position relative to the background shifts. This parallax helps us estimate depth.

But if you only rotate your phone in place while shooting, without enough change in position, then even a long recording does not mean you captured sufficient 3D information.

In the 3DGS phone-capture lessons, I also learned how image overlap, shooting paths, and occlusion relate to each other.

It turns out that a video that is pleasant to watch is not necessarily a video suited for 3D reconstruction.

When shooting, you also have to consider the shared area between adjacent frames, motion blur, lighting changes, and the surfaces the camera never observed at all.

This gave me a new understanding of the word “reconstruction.”

3D reconstruction is not simply turning photos into stereo pictures. It is recovering — or inferring — a space that can be observed again, from limited observations.

Different representations also differ in how they work and where their limits are.

Point clouds, meshes, NeRF, and 3DGS all relate to 3D scenes, but they cannot be treated as fully equivalent results.

A 3DGS scene can look visually convincing, but if you want to measure dimensions, run physical collisions, or edit specific objects, you usually need additional data or processing.

Visual realism does not automatically mean geometric accuracy.

03 | The four directions of spatial intelligence

The course organizes the capabilities of spatial intelligence into four main directions:

Spatial Reconstruction: rebuilding 3D objects or scenes from observations such as photos and video.

Spatial Understanding: recognizing the objects, positions, and shapes in a space, and the relations between them.

Spatial Generation: creating new 3D content from text, images, or other conditions.

Spatial Action: letting an agent navigate, manipulate, and adjust through feedback in an environment.

This framing helped me organize concepts that had felt scattered.

The distinction between spatial understanding and spatial action impressed me most.

A vision model that can identify a table, a chair, and a door in a picture does not yet know how to walk around the chair to reach the door.

Likewise, a virtual character that plays a walking animation has not actually completed spatial navigation.

In the multi-agent simulation lessons there is an example well suited for beginners: a small character looking for the exit in a room.

From a top-down view, the door is obvious.

But for the character inside the room, walls and furniture block its view. It has to choose actions based on its current observation and observe the environment again after moving.

A complete action loop includes at least:

Observe → Decide → Act → Environment changes → Observe again.

If the model says “walk forward” but the character’s position never updates, no action actually happened.

And if the character finds the door because the program directly told it the exit coordinates, the result cannot be described as autonomous visual exploration.

These distinctions made me pay more attention to how an AI capability is implemented and verified — not just the final demo.

04 | Turning the course into my own knowledge cards

While learning, I did not just bookmark the course links. Together with AI, I reorganized the course material into knowledge cards that are easier to understand and review.

From the introduction to spatial intelligence and 3D vision, through reconstruction, understanding, generation, and action, to Lux3D, the Aholo API, 3DGS phone capture, and multi-agent simulation, we gradually built several independent learning pages.

These pages contain concept explanations, technology comparisons, interactive diagrams, and self-tests.

As the courses piled up, I ran into another question:

How should this knowledge be organized so the connections between pieces become clearer?

In the end, I decided to put everything into the Spatial Intelligence Learning Notebook in Lumi Lab.

I want it to be more than a collection of course notes — a personal learning archive that can keep growing.

But this process also alerted me to something worth being wary of.

AI can quickly help me organize concepts, generate pages, and even build interactive technical illustrations.

But generating a professional-looking knowledge card does not mean I have mastered the technology on it.

I need to distinguish between:

• Knowledge I have read;
• Concepts I can explain in my own words;
• Phenomena I have verified through experiments;
• Skills I have not yet practiced hands-on.

Especially in AI-assisted development, a finished page and runnable code do not mean I understand every implementation detail.

So rather than simply accumulating more beautiful pages, I hope these knowledge cards help me discover what I have not yet understood, and gradually connect theory with practice.

05 | From technical demo to real product: what is still missing?

The last part of this learning journey turned from technical knowledge toward product practice — which, I discovered, is the direction I enjoy most.

In a workshop or hackathon, it is easy to be drawn to interesting capabilities.

Generating a 3D model from one image, reconstructing a real space from one video, or letting multiple AI characters move around a room — all of these make great demos.

But turning them into a product raises different questions.

Why would a user need it?

For example, suppose we build a web page where you can rotate and inspect a 3D model.

If the user is simply admiring a virtual exhibit, smooth camera control and good visuals may be enough.

But if the user wants to know whether a piece of furniture fits in their room before buying it, accurate dimensions, proportions, and spatial relations become essential.

If the user wants to play a game inside a reconstructed real space, collisions, walkable areas, and interaction rules have to be handled too.

Same AI 3D, different goals — completely different product requirements.

The course’s take on MVP also made me rethink build order.

Rather than trying to build a complete 3D virtual campus from the start, pick one specific spot, let users open the scene and find one piece of information, and watch whether they can complete the task on their own.

Validate one meaningful experience first, then decide whether it is worth expanding.

Capability decides what we could build; user needs decide what we should build first.

06 | Next step: turning understanding into verification

This learning gave me an initial framework for spatial intelligence, but its judgments still need to be tested by practice.

The flow of “Lux3D model generation → quality check → GLB export → web page loading” has already been walked once, in my 3D puzzle game The Last Plot: all four buildings — the factory, the houses, the shop, and the park — were generated as textured GLBs from reference images, checked in the web viewer from the front, the side, and the back — exactly the checking method that cup taught me — then exported with their file structure and hashes recorded, and finally loaded in three.js with a geometric fallback for load failures.

But I should be honest about the distinction: in that practice, most of the generating, checking, and integration was executed with AI assistance; what I did was authorize, judge, and accept the results. Loading performance on real phones, low-end devices, and slow networks is also still unverified. So for me, this flow is something I have “seen and used,” not something I have “completed independently.” I want to walk it again with my own hands and turn every step into a judgment I can reproduce.

The practice that has not started yet is a real-world 3DGS phone capture: observing how the shooting path, occlusion, and image overlap affect reconstruction quality.

After that, starting from concrete use cases, I want to judge what tasks these capabilities can actually help users complete.

Along the way, I want to keep recording the real inputs, generated results, failure cases, and revisions.

I do not want to write “I have touched a tool” as “I have mastered a technique,” nor ignore the parts I do not yet understand just because AI helped write the code.

To me, the more valuable record might be:

What questions did I ask, what approaches did I try, what results did I get, and on what evidence did I revise my earlier assumptions?

---

Looking back, my biggest change is a shift of attention: from “What can AI generate?” to “What does AI actually understand, and how reliably can it complete a task?”

From a cup with an extra leaf on its back, to a room that can be reconstructed, to virtual characters that observe and act — spatial intelligence showed me a technical path from visual generation toward environmental understanding and interaction.

There is still much worth exploring along this path.

A 3D world you can see is only the beginning.

What really interests me is how machines understand this world and interact with it — and how we turn those capabilities into products that are genuinely useful.

---

References

• Guancha AI 3D Spatial Intelligence Course: Lux3D from 0 to 1
• 3DGS Phone Capture
• Multi-Agent Spatial Simulation
• From Course Projects to Product Practice
• The Last Plot: a 3D puzzle game built with real Lux3D assets