19 Commits
Author SHA1 Message Date
m1saka 264a5e664a refactor(web): remove DSH workspace status bar duplicated in settings 2026-10-22 16:55:03 +08:00
m1saka ed72985e7b fix: preserve snapshot symlinks and guard incomplete upgrade copies 2026-10-18 10:12:53 +08:00
m1saka 1c244e135f fix: recover login state after network interruptions 2026-10-17 22:44:10 +08:00
m1saka f77f3d77c0 feat: add full snapshot downloads and safe manual restore 2026-10-17 01:36:21 +08:00
m1saka c6615faa49 fix: extend proxy timeout for Office rendering 2026-10-17 00:52:20 +08:00
m1saka 602917698a feat: add safe backup management and theme preferences
Add snapshot usage and protected cleanup, preserve unsaved profile edits, avoid holding SQL connections during snapshot deletion, and fix download filenames. Include regression tests and API documentation.
2026-10-17 00:36:21 +08:00
m1saka 09e202796f docs: synchronize documentation with current implementation 2026-10-16 11:37:18 +08:00
m1saka a8ba3d74dc feat(workspace): add text editing and image previews 2026-10-15 23:10:13 +08:00
m1saka d380e18db0 feat(web): follow system dark mode and simplify workspace hint 2026-10-15 21:25:58 +08:00
m1saka 7aa540af04 fix(auth): recover refresh rotations and configure session lifetime 2026-10-15 17:22:03 +08:00
m1saka 60ff16bcdd chore: ignore Android build leftovers and signing files 2026-10-15 16:54:01 +08:00
m1saka 22a5f64edc fix: delete instance data and remove legacy shared workspace support 2026-10-15 10:55:21 +08:00
m1saka ee22603726 fix: keep panel default gateway stable across instance networks 2026-10-15 10:12:03 +08:00
m1saka f6fc681a68 fix: normalize default arm64 v8 image platform 2026-10-15 09:50:30 +08:00
m1saka 56c8f76e32 feat: redesign responsive console and fix restore regressions 2026-10-15 01:08:55 +08:00
m1saka 3bf197206a fix: prune missing image indexes and align instance creation payload 2026-10-14 19:37:52 +08:00
m1saka 15910a086f chore: ignore Vite cache in main branch 2026-10-14 15:55:20 +08:00
m1saka a2dd6ee27d refactor!: separate runtime releases from panel source 2026-10-14 15:05:10 +08:00
m1saka 20635ffaec feat: build runtime releases from pinned Git branch commits 2026-10-14 14:25:27 +08:00
144 changed files with 9901 additions and 2928 deletions
-1
View File
@@ -10,4 +10,3 @@ output
**/*.tsbuildinfo **/*.tsbuildinfo
.playwright-cli .playwright-cli
**/*.sqlite* **/*.sqlite*
image/*.test.mjs
+6
View File
@@ -19,3 +19,9 @@ UPLOAD_ALLOWED_EXTENSIONS=
PANEL_PUBLIC_URL=http://127.0.0.1:8088 PANEL_PUBLIC_URL=http://127.0.0.1:8088
# Optional proxy networks. X-Forwarded-For is ignored unless the peer is in these CIDRs. # Optional proxy networks. X-Forwarded-For is ignored unless the peer is in these CIDRs.
TRUSTED_PROXY_CIDRS= TRUSTED_PROXY_CIDRS=
# Absolute login lifetime (Go duration: 24h, 168h for 7 days). Applies to new logins.
# Refresh does not extend this deadline. Invalid/non-positive values prevent startup.
SESSION_TTL=24h
# Trusted executable runtime build source. Do not embed credentials in this URL.
DSH_RELEASE_REPOSITORY=https://git.misaka.ren/m1saka/dsh-web.git
DSH_RELEASE_BRANCH=dockerfile
+29
View File
@@ -7,3 +7,32 @@ bin/
output/ output/
.playwright-cli/ .playwright-cli/
*.sqlite* *.sqlite*
# Local IDE files
.idea/
*.iml
# Generated files left when switching from android-app to main
/.dart_tool/
/.flutter-plugins-dependencies
/build/
/android/.gradle/
/android/.kotlin/
/android/build/
/android/app/build/
/android/local.properties
/android/gradlew
/android/gradlew.bat
/android/gradle/wrapper/gradle-wrapper.jar
/android/app/src/main/java/io/flutter/plugins/GeneratedPluginRegistrant.java
# Android packages and signing credentials
*.apk
*.aab
*.jks
*.keystore
*.p12
*.pfx
keystore.properties
signing.properties
/android/key.properties
+55 -19
View File
@@ -1,6 +1,40 @@
# DESIGN.md 验收记录 # DESIGN.md 验收记录
## 恢复宽限期与轮询缓存修复(独立复审后) > 发布架构变更说明:下文为各阶段的历史验收证据,保留当时命令、测试数量及结论,不作为当前操作指南。main 已移除内置 runtime 构建资产;`release.json`、完整 `image/` 和初始化测试仅在 `dockerfile` 发布分支维护,新构建仅允许管理员通过 Git 发布检查并确认精确 commit。旧 npm/实验构建及构建后创建预约流程已被取代,这是 breaking change:移除 `/api/latest`、`/api/compatibility`、`POST /api/builds` 及 `/api/intents` 的全部查询、创建和取消接口,不提供 410 兼容入口。仅保留 `GET /api/builds` 和 `GET /api/builds/{id}` 构建历史读取。不迁移或恢复旧构建及旧创建意图,不在后台续跑或完成旧预约;这不意味着删除已有实例或数据。旧 `scripts/build-image.sh` 与 `scripts/test-intent-live.mjs` 已删除;`scripts/test-panel.mjs` 现在复用预先验证且准入的本地镜像。历史通过记录不等于新 Git 发布架构已通过真实部署验收,当前使用方式见 README.md、API.md、OPERATIONS.md。
## 当前实现与验收状态(2026-10-16 文档核对)
本节优先于后面的历史记录。已完整阅读两份文档并对照当前源码;本地验证由本次维护任务的独立验证执行者完成并回报,不将代码存在、模拟测试或旧部署证据冒充当前真实环境通过。
### 本次实际验证
- `bash scripts/check-trial.sh`:退出0;Go普通测试/race各5包通过,vet/build通过;前端25文件198测试通过,TypeScript/Vite生产构建通过;`node --check scripts/test-panel.mjs`通过。
- `node --test scripts/check-trial.test.mjs`:10/10通过。
- 两项Docker live测试因未启用`DSH_DOCKER_LIVE_TEST`跳过,**不计真实Docker通过**。本次未执行真实浏览器、公网入口或完整Git发布部署验收,未调用模型或搜索。
- 历史测试数量、镜像ID、截图及/tmp路径仅证明当时记录;本轮未逐一重新获取这些临时证据,不代表当前线上状态。
### 当前能力矩阵
| 范围 | 当前代码与本地验证 | 当前版本仍缺的真实证据 |
|---|---|---|
| 前端与架构 | React19/TS/Vite、Tailwind4/Radix、Monaco;Go标准库Docker HTTP client、SQLite;本轮自动化/构建通过 | 当前UI浏览器全流程与真机;Flutter/Android仅历史工作区残留,无当前原生发布验收 |
| Git发布与准入 | 管理员检查发布/精确commit、本机构建验证/历史;模板7与当前平台身份检查;本地测试通过 | 真实发布→构建→验证→准入→创建完整链路、故障及重启恢复;不能沿用旧npm/实验构建结论 |
| 实例与存储 | 两实例事务名额、每实例bridge/home,/workspace为DSH_HOME;删除清理实例数据及备份;无预约或共享入口 | 当前删除语义、并发名额与不确定创建恢复的真实Docker复验;旧“删除保留数据”证据不适用 |
| 代理与会话 | 归属鉴权、服务端token/cookie桥接、WS/SSE与续期;本地测试通过 | 当前部署iframe、独立页、长时/后台/多页续期与真实WS/SSE;历史615秒结果不是当前整体重验 |
| 文件与配额 | 完整home、上传下载/ZIP/文本编辑/图片预览、账号软配额;本地测试通过 | 当前浏览器编辑冲突/图片显示/敏感确认、跨用户访问、真实配额耗尽及容器直接写盘边界 |
| 配置与升级 | Profile校验/预览、密码加密配置备份/导入/恢复、快照/回滚;本地测试通过 | 当前跨版本/插件重建/离线失败/旧包兼容及回滚数据保留;同版本替换不证明跨版本迁移 |
| 部署与安全 | 登录/refresh/CSRF/审计/改密、非root/只读/资源限制;本地测试通过 | 实际域名HTTPS/代理、socket权限、真实浏览器;不宣称不可信公网多租户隔离或生产SLA |
| 可选增强 | XFS/ZFS硬配额、配额sidecar、socket-proxy未作为当前默认交付 | 未实现/未验收,不标通过;不列为默认软配额流程失败项 |
关键纠偏:除usage外文件API不带instance返回410,新容器不挂/shared;准入布局仅支持模板7,旧模板3–6不能靠说明正向准入。实例删除会清理自身目录和备份,孤儿回收是另一流程。创建不确定时不凭超时404清除依据。见`internal/api/files.go`、`runtime.go`、`instance_data.go`、`admission.go`、`resource_recovery.go`。
结论:**本次本地自动化与构建通过;当前完整真实部署/浏览器验收未完成,不作M1–M7整体结案。**
## 历史阶段记录(以下均非当前版本整体验收)
各节“当前”“本轮”“最终”“已部署”均指该节当时版本;旧命令可能已移除。旧结案不覆盖文首矩阵;各历史阶段之间的覆盖关系仅适用于对应迭代。
## 恢复宽限期与轮询缓存修复(历史,已被后续修复替代)
范围:本轮为对上述工作树的独立代码审计与全量重测后的修复。审计确认八项修复中六项正确,发现三处残留:创建结果不确定的挂起标志永不过期导致实例永久无法删除(高)、构建校验不确定行无限重试并持续新增审计(中)、意图轮询每秒对每个成功镜像发起 Docker inspect(中)。修复未改变凭据文件管理边界或准入策略;未部署、未操作真实 Docker 实例。 范围:本轮为对上述工作树的独立代码审计与全量重测后的修复。审计确认八项修复中六项正确,发现三处残留:创建结果不确定的挂起标志永不过期导致实例永久无法删除(高)、构建校验不确定行无限重试并持续新增审计(中)、意图轮询每秒对每个成功镜像发起 Docker inspect(中)。修复未改变凭据文件管理边界或准入策略;未部署、未操作真实 Docker 实例。
@@ -183,36 +217,36 @@
用户确认的孤儿策略:管理员二次确认回收身份/挂载核验通过、无实例/升级引用的停止容器;不自动纳管,不触碰运行中或来源不明容器,不删数据/网络。其他架构替代见下表;不声称逐字遵循SDK、组件库或目录示意。 用户确认的孤儿策略:管理员二次确认回收身份/挂载核验通过、无实例/升级引用的停止容器;不自动纳管,不触碰运行中或来源不明容器,不删数据/网络。其他架构替代见下表;不声称逐字遵循SDK、组件库或目录示意。
## 早期进度记录(以下由上方最新结论覆盖) ## 后续UI改版记录(历史独立阶段,不由2026-10-10结案覆盖)
## 当前实现与待验收 ## UI 改版与功能回归(2026-10-15)
| 设计章节 | 代码已接入 | 尚需真实证据 | 本节为本轮证据,后文保留历史记录;不代表所有历史待验收项均已关闭。
|---|---|---|
| §1–6 | sibling路径映射、每用户两实例、持久预约/取消/失败恢复、生命周期与身份对账 | 模板2非root全流程、刷新/重启预约恢复、并发名额 |
| §5 | 精确npm版本、模板/平台身份、构建去重/限频/磁盘预检、镜像usage/missing及管理员清理 | 真实构建与删除、外部删镜像、空间/网络故障及重启对账 |
| §7–8 | 路径反代、归属鉴权、服务端token/cookie桥接、WS/SSE | 新部署的新浏览器完整流程、重启重新认证、长时续期 |
| §9 | 文件管理、目录ZIP、写前配额检查/串行化、周期巡检、管理员额度更新 | 新UI文件流程、agent实际读写、面板各写入路径配额覆盖 |
| §10 | 密码加密tar.gz v2、配置/资源目录/manifest、外部包导入、旧key包兼容、停启/健康检查/回滚 | 真实资源往返、原密码/错误密码、无key新包、旧key包、插件恢复失败回滚 |
| §11 | 本地Monaco、YAML校验、bundle与脱敏dump-config预览、保存后启动验证 | 新浏览器编辑/预览/错误路径;当前仍需先停止再保存 |
| §12 | 预升级密码备份、home快照、成功删旧、失败回滚及恢复原运行状态 | 不同DSH版本、真实插件重建/离线失败、故障矩阵与数据保留 |
| §13–16 | 短JWT/refresh轮换/失败锁定、审计、资源限制、非root部署配置 | 模板2非root数据写入和socket访问、新浏览器及DSH独立页长JWT续期 |
| §17–19 | 默认路径反代与软配额;已有回归测试及旧版集成记录 | 必需流程未关闭,不标记M1–M7整体完成;硬quota为可选增强 |
备份包含六个固定配置文件及skins、pets、agent-presets、.agent-presets资源树和manifest,不包含node_modules、会话或工作区。新密码包不依赖backup.key;保留旧密钥包时仍须保管原key。备份/恢复/升级不再统一要求用户手动停机;操作中断或回滚失败不保证自动灾难恢复。 - 前端改为 Tailwind v4、Radix 原语的 shadcn 风格组件及 lucide 图标;桌面侧栏、移动导航、响应式卡片、确认弹窗和备份操作统一。
- 最终前端 20 个测试文件、131 个用例通过,TypeScript/Vite 生产构建通过;Go 全量测试、race 测试、vet、build 均通过;check-trial 脚本的 10 个测试通过,git diff --check 通过。
- 浏览器验证桌面及 320/375/667px 移动视口:无页面横向溢出、主要控件触控高度至少 44px、弹窗关闭焦点恢复、准入编辑器只挂载一份且跨断点保留草稿。配额读回、重设密码后登录、自助修改密码后退出、删除取消/确认、重命名冲突在弹窗内提示并可修正重试、错误备份密码在当前弹窗提示均通过。截图位于本地 .playwright-cli/。
- 隔离真实 Docker 面板执行 scripts/test-panel.mjs,TEST_UPGRADE=1,最终退出码 0:登录/刷新、镜像选择、创建/代理、文件夹/上传/下载/ZIP/重命名/删除、Profile 保存、加密备份/下载/导入/恢复、启停、同镜像替换升级、实例删除通过。补充验证未认证 401、缺少 CSRF/外部 Origin 403、普通用户不能访问用户管理、配额更新与负值拒绝。
- 真实回归发现并修复两个后端缺陷:空 Profile 规范化为合法空数组 YAML;回滚停止容器接受 Docker 304,但必须继续核验确实停止。新增回归用例,修复后完整集成流程重新通过。
- 本次升级仅覆盖 0.2.0-rc.2 到相同版本的容器替换;跨版本升级、插件重建、离线失败回滚、长时会话续期,以及真实配额耗尽/跨用户实例访问未在本轮完整实测,不计为通过。未调用付费模型。
- 一次性测试实例及其容器/网络均已清理;原有 DSH 实例未重启。测试日志及隔离备份保留于 /tmp/dsh-isolated-integration-BFUCkK/(本地临时证据,不随提交分发)。
## 合理替代方案(不改变验收目标) ## 早期实现与待验收表(已归档)
原表描述模板2、npm构建及预约阶段,已由文首当前矩阵替代。早期未闭项须按各阶段证据解释,不能推导当前Git发布通过。配置备份为白名单包,不含完整工作区、依赖或会话;旧密钥包仍须保管原key。操作中断或回滚失败不保证自动灾难恢复。
## 历史合理替代方案(不改变验收目标)
| DESIGN选型 | 当前实现 | 等价目标与边界 | | DESIGN选型 | 当前实现 | 等价目标与边界 |
|---|---|---| |---|---|---|
| 官方Docker SDK | stdlib net/http Docker Engine client,API v1.47 | 仍须验证编排、构建流、错误处理和daemon兼容性,不能以未用SDK认定功能缺失 | | 官方Docker SDK | stdlib net/http Docker Engine client,API v1.47 | 仍须验证编排、构建流、错误处理和daemon兼容性,不能以未用SDK认定功能缺失 |
| 共享bridge | 每实例独立bridge,面板接入 | 减少实例直接互通;不等于完整出站或宿主网络隔离 | | 共享bridge | 每实例独立bridge,面板接入 | 减少实例直接互通;不等于完整出站或宿主网络隔离 |
| DSH cookie改写后下发浏览器 | 服务端交换/缓存cookie,反代注入并移除上游Set-Cookie | 保留面板登录与实例归属鉴权;WS/SSE、重启失效与续期仍需实测 | | DSH cookie改写后下发浏览器 | 服务端交换/缓存cookie,反代注入并移除上游Set-Cookie | 保留面板登录与实例归属鉴权;WS/SSE、重启失效与续期仍需实测 |
| shadcn/ui | 普通React组件及CSS,保留本地Monaco | 不要求替换组件库,但交互、错误反馈和可用性仍须浏览器验收 | | shadcn/ui | 当时为普通React/CSS;现为Tailwind v4及Radix原语的shadcn风格组件,保留Monaco | 历史确认不替代当前UI浏览器验收 |
代码主要按internal/api、internal/docker、internal/backup、internal/store组织,与设计目录示意不同;不宣称逐字一致。默认软配额符合§9/§18路线,XFS/ZFS硬quota依赖运维选择,不列为默认必过项。npm 0.2.0-rc.2不支持--public-url,当前使用路径代理而不传该参数。 代码主要按internal/api、internal/docker、internal/backup、internal/store组织,与设计目录示意不同;不宣称逐字一致。默认软配额符合§9/§18路线,XFS/ZFS硬quota依赖运维选择,不列为默认必过项。npm 0.2.0-rc.2不支持--public-url,当前使用路径代理而不传该参数。
## 历史证据与本轮边界 ## 早期历史证据与当时边界(非当前待办结论)
本轮已取得真实证据: 本轮已取得真实证据:
- 非root面板 `dsh-design-check`(1000:1000、只读rootfs、socket附加组989)已启动;33项HTTP认证/刷新/配额/ZIP/Range检查通过,报告 `/tmp/dsh-design-check.8ekWaC/api-check-results.txt`。 - 非root面板 `dsh-design-check`(1000:1000、只读rootfs、socket附加组989)已启动;33项HTTP认证/刷新/配额/ZIP/Range检查通过,报告 `/tmp/dsh-design-check.8ekWaC/api-check-results.txt`。
@@ -233,7 +267,9 @@
- 面板已部署 `dsh-mgr:workspace6`;正常构建 API 记录 `e9a368bbb66e51e6c8f2439a8e94698a` 成功,recipe_rev=6,镜像 `sha256:38e04308da602da22e8df49d54166da1835922d50a3fa485711b7c5f6479d287`。 - 面板已部署 `dsh-mgr:workspace6`;正常构建 API 记录 `e9a368bbb66e51e6c8f2439a8e94698a` 成功,recipe_rev=6,镜像 `sha256:38e04308da602da22e8df49d54166da1835922d50a3fa485711b7c5f6479d287`。
- 面板部署保留配置和挂载,旧面板停止留作回滚;现有 DSH 实例未重启,构建前后实例记录一致。 - 面板部署保留配置和挂载,旧面板停止留作回滚;现有 DSH 实例未重启,构建前后实例记录一致。
## 模板 7:完整 DSH_HOME 改为 /workspace ## 历史模板7迁移:完整 DSH_HOME 改为 /workspace
> 本节的/shared挂载、共享入口与迁移结果仅描述当时方案;当前共享入口和新容器共享挂载均已移除,实例删除会清理自身数据及备份,详见文首矩阵。
### 管理交互优化验收 ### 管理交互优化验收
+98 -30
View File
@@ -1,10 +1,38 @@
# DSH Web 面板 API(MVP) # DSH Web 面板 API(MVP)
默认监听 127.0.0.1:8088。JSON 错误统一为 {error:string}。登录使用 HttpOnly session/access/refresh cookie;写请求需 X-CSRF-Token 和精确同源 Origin(未配置 PANEL_PUBLIC_URL 时也校验)。access JWT 有效期5分钟,refresh轮换且受24小时session限制。 默认监听 127.0.0.1:8088。面板处理器通过 fail 返回的错误为 JSON {error:string};未注册路由的标准 404、静态文件/下载错误及运行时上游响应不保证此格式。面板非 GET 请求(包括 HEAD/OPTIONS)需 X-CSRF-Token 和精确同源 Origin,登录仅要求 Origin(未配置 PANEL_PUBLIC_URL 时按请求 TLS 与 Host 校验)。运行时代理使用下文独立规则。access JWT 有效期5分钟,refresh轮换但不延长 session 的绝对到期时间,session 默认24小时,可用 SESSION_TTL 修改。
除另有说明,成功返回 200;创建用户、实例、备份也不是 201。下述处理器状态码以通过前置认证、Origin/CSRF 和资源检查为前提,前置失败可能优先返回 401/403/409。普通业务接口需要同时有效的 dsh_session 与 dsh_access,不支持以 Authorization Bearer 替代。
- /healthz:无需认证,返回 200 和纯文本 ok;当前实现不限制方法,也不检查 Docker/数据库健康。
- GET /api/runtime-session.js:无需登录,返回 JavaScript、Cache-Control: no-store,用于运行时页面续期。
## 配置 ## 配置
- UPLOAD_ALLOWED_EXTENSIONS:可选逗号分隔扩展名白名单(例如 .txt,.md);空表示不限制。大小写不敏感,非内容/恶意软件检测。 - UPLOAD_ALLOWED_EXTENSIONS:可选逗号分隔扩展名白名单(例如 .txt,.md);空表示不限制。大小写不敏感,非内容/恶意软件检测。
## 实例备份与快照管理
### 完整 home 快照恢复
`GET /api/instances/:id/snapshots` 的每项另含 `restorable:boolean`、`restore_reason:string`、可选 `source_version` 与 `source_image`。来源记录在独立的 `snapshot_metadata` 表中,不会从当前实例推断旧快照版本。已知镜像或版本不一致时不可恢复;来源未知时 `restorable:true` 但 `restore_reason` 为 `snapshot source image/version unknown; explicit acknowledgement required`,客户端必须显示风险警告并要求额外确认。保护状态及大小读取错误时不可恢复。列表只是预检提示,最终恢复还会检查实时配额、容器布局、镜像准入与读者占用。
`POST /api/instances/:id/snapshots` 接收 `{id,password,confirm:true,allow_unknown_version?:true}`(JSON 最大4096字节,在获取锁前读取)。`password` 是当前登录密码,不是旧备份密码;需正常登录、同源 Origin 与 CSRF。来源未知必须明确传入 `allow_unknown_version:true`;此开关不能绕过已知不兼容。成功返回 `200 {ok:true,snapshot_id:"pre-restore-<32hex>"}`,源快照保持不变。
恢复替换整个 home,不是仅覆盖配置。仅支持经核验的模板7独立 /workspace bind 挂载与当前实例镜像;拒绝旧共享布局、被阻止镜像、恢复保护状态及活跃文件读者。先进行配额和保留数量检查,再使用 rooted 路径验证复制到同文件系统暂存目录;外部/绝对/悬空符号链接及特殊文件拒绝。停止运行时后重新检查配额,将当前 home 重命名保留为 pre-restore 快照,再安装暂存目录并验证就绪;原来停止的实例会临时启动验证后再次停止。失败时独立超时上下文执行回滚,失败的新 home 保留为 failed-restore 快照;无法安全回滚时保持 error,要求管理员恢复。
非法请求或不安全内容400,当前密码错误403,目标不存在/所有权不符404,保护/读者占用/不兼容/缺少未知版本确认/保留上限409,请求或配额超限413,存储/数据库/回滚故障500,容器检查/停止失败502。正常恢复前会记录 snapshot.restore.started,完成或回滚分别记录 snapshot.restore.completed / snapshot.restore.rolled_back。此接口不改变实例镜像。
- `DELETE /api/instances/:id/backups?id=<32hex>` 删除当前用户实例的加密 `.dshbak` 包及其索引,记录 `backup.delete` 审计。ID 为32位小写十六进制;支持无索引的旧包,以及文件已不存在的残留索引。
- `GET /api/instances/:id/snapshots` 返回数组 `{id,size,created_at,protected,protection_reason,size_error?}`。`size` 为普通文件逻辑字节总和(与配额口径相同,硬链接逐路径计数,不跟随符号链接);扫描失败为 `-1` 并提供 `size_error`,不能显示为零。`created_at` 为目录修改时间的 RFC3339 字符串,不保证真实创建时间。未保护时 `protection_reason` 是空字符串。
- `DELETE /api/instances/:id/snapshots?id=<id>` 删除选定快照。先持久化 `snapshot.delete.started`,删除成功后记录 `snapshot.delete.completed`,删除失败则记录 `snapshot.delete.failed`;target 为实例 ID,detail 为快照 ID。初始审计失败时不会删除,终态审计失败返回500并保留 started 记录。快照另支持 GET 下载,见下方接口清单及 OPERATIONS.md 的归档限制。
仅识别实例目录直属的 `pre-restore-<32hex>`、`failed-restore-<32hex>`、`upgrade-<32hex>-snapshot`、`failed-home-<32hex>` 真实目录;不列出或删除 `home`、其它命名或符号链接目录。删除成功返回 `200 {ok:true}`,非法ID/不安全路径400,不存在/所有权不符404,恢复保护409,存储/索引/审计错误500。不支持的方法405。删除仍要求同源 Origin 与 CSRF,不要求备份密码,不调用 Docker inspect/stop/restart。
实例 `error`、未完成升级(仅 `complete` / `rolled_back` 为终态)、恢复日志查询错误时拒绝两类删除。快照列表仍可读取并报告保护原因。所有权在实例锁与用户锁内重新检查。无自动清理;文件系统和数据库无法原子提交,包删除后的索引提交失败会返回500,重试同一删除可清理残留索引。
快照递归删除期间保留实例锁与用户锁,但不持有 SQL 连接或事务。仅明确识别的非法/不安全路径返回400;权限、I/O 等操作系统故障返回500。快照目录不存在时列表返回空数组,删除返回404。
## 管理员孤儿回收 ## 管理员孤儿回收
GET /api/orphans 返回可回收的完整容器ID数组。DELETE /api/orphans?id=<id>&confirm=<相同完整id> 二次核验后非强制删除,保留bind数据及网络。仅接受本项目身份/挂载匹配、已停止、无实例或升级引用的容器;普通用户403,确认不匹配400,状态不安全409。 GET /api/orphans 返回可回收的完整容器ID数组。DELETE /api/orphans?id=<id>&confirm=<相同完整id> 二次核验后非强制删除,保留bind数据及网络。仅接受本项目身份/挂载匹配、已停止、无实例或升级引用的容器;普通用户403,确认不匹配400,状态不安全409。
@@ -12,89 +40,129 @@ GET /api/orphans 返回可回收的完整容器ID数组。DELETE /api/orphans?id
## 其他配置 ## 其他配置
- LISTEN_ADDR(默认 127.0.0.1:8088) - LISTEN_ADDR(默认 127.0.0.1:8088)
- DATA_ROOT(默认 ./data) - DATA_ROOT(默认 ./data)
- HOST_DATA_ROOT(默认等于 DATA_ROOT 的绝对路径,用于 Docker bind source;容器部署必须显式设置宿主绝对路径) - HOST_DATA_ROOT(默认等于实际 CONTAINER_DATA_ROOT 的绝对路径,用于 Docker bind source;容器部署必须显式设置宿主绝对路径)
- CONTAINER_DATA_ROOT(默认 DATA_ROOT) - CONTAINER_DATA_ROOT(默认 DATA_ROOT)
- DOCKER_HOST(默认 unix:///var/run/docker.sock) - DOCKER_HOST(默认 unix:///var/run/docker.sock)
- PANEL_PUBLIC_URL(可选) - PANEL_PUBLIC_URL(可选 http/https origin,可带根路径 /,不可带其它路径、用户信息、查询或片段;非法值拒绝启动)
- TRUSTED_PROXY_CIDRS(可选逗号分隔 CIDR;仅信任这些代理的 X-Forwarded-For 来计算登录来源限流,不替代 Origin 校验;非法值拒绝启动)
- SESSION_TTL(默认 24h;Go duration,至少1秒且必须为整秒;非法值拒绝启动,只影响新建 session)
- ADMIN_USERNAME / ADMIN_PASSWORD(首次启动创建管理员;密码必须 12–72 字节,不设置则拒绝启动,不打印密码) - ADMIN_USERNAME / ADMIN_PASSWORD(首次启动创建管理员;密码必须 12–72 字节,不设置则拒绝启动,不打印密码)
- WEB_DIR(可选静态前端目录) - WEB_DIR(静态前端目录,默认 web/dist)
- PANEL_CONTAINER_ID(面板容器名/ID,用于加入实例网络;宿主进程运行时留空) - PANEL_CONTAINER_ID(面板容器名/ID,用于加入实例网络;宿主进程运行时留空)
- DEFAULT_QUOTA_BYTES(默认10GiB) - DEFAULT_QUOTA_BYTES(默认10GiB)
- DSH_DOCKER_ROOT_VISIBLE(与DockerRootDir相同路径的明确映射,用于构建前容量检查;未知容量拒绝新构建) - DSH_DOCKER_ROOT_VISIBLE(与DockerRootDir相同路径的明确映射,用于构建前容量检查;未知容量拒绝新构建)
- DSH_RELEASE_REPOSITORY(默认 https://git.misaka.ren/m1saka/dsh-web.git;仅接受无用户信息、查询或片段的 HTTPS 仓库地址)
- DSH_RELEASE_BRANCH(默认 dockerfile;Git 发布检查和构建使用的分支)
## 认证 ## 认证
- POST /api/login {username,password} → {user,csrf_token};user 字段为 id,username,admin,quota_bytes,created_at - POST /api/login {username,password} → {user,csrf_token};user 字段为 id,username,admin,quota_bytes,created_at
- POST /api/logout → {ok:true} - 登录请求体最多4096字节,非法/多段 JSON 返回400,凭据错误401,Origin错误403;每来源每分钟最多30次、账户累计5次失败锁定15分钟,限流/锁定429,认证不可用或并发槽耗尽503。
- POST /api/refresh → {user,csrf_token}(刷新HttpOnly cookie,需Origin和CSRF;并发轮换冲突且session仍有效时返回409,不签发cookie,客户端可复验 `/api/me` 后重试原请求;会话撤销/过期返回401,存储异常不能当作注销) - cookie 均为 SameSite=Lax:HttpOnly 的 dsh_session/dsh_access 路径 /,dsh_refresh 路径 /api;另有可供脚本读取的 dsh_csrf(路径 /,非 HttpOnly)。请求使用 TLS 或 PANEL_PUBLIC_URL 为 HTTPS 时设置 Secure。
- POST /api/logout → {ok:true}(不要求有效 access,但仍需有效 session、Origin 和 CSRF;撤销当前会话并清除四个 cookie,不是无条件幂等的匿名退出接口)
- POST /api/refresh → {user,csrf_token}(需 session、refresh cookie、Origin 和 CSRF,不要求 access 尚有效;接受当前 refresh 或紧邻的上一代 refresh,上一代重试幂等返回同一个后继 token 并签发 cookie,不返回轮换冲突409。更旧/无效 refresh 返回401;过期/撤销 session 通常先由 guard 返回403,处理器复查失败也可能401;存储异常在处理器返回500/503,但 guard 的 session 查询失败也会403)
- GET /api/me → {user,csrf_token} - GET /api/me → {user,csrf_token}
- POST /api/me/password {current_password,password} → {ok:true,reauthenticate:true}(普通用户及管理员仅修改本人;不接收目标 ID,未知 JSON 字段返回 400)。需有效 access/session、同源 Origin 和 X-CSRF-Token;新密码为 UTF-8 12–72 字节,非法 JSON、缺少当前密码或长度不符返回 400。错误当前密码返回 400 而非 401,客户端不应因此退出登录。未登录 401,Origin/CSRF 不符 403(失效 session 在 guard 中也可能返回 403),已认证的非 POST 请求返回 405。 - POST /api/me/password {current_password,password} → {ok:true,reauthenticate:true}(普通用户及管理员仅修改本人;不接收目标 ID,未知 JSON 字段返回 400)。需有效 access/session、同源 Origin 和 X-CSRF-Token;新密码为 UTF-8 12–72 字节,非法 JSON、缺少当前密码或长度不符返回 400。错误当前密码返回 400 而非 401,客户端不应因此退出登录。未登录 401,Origin/CSRF 不符 403(失效 session 在 guard 中也可能返回 403),已认证的非 POST 请求返回 405。
- 与登录、管理员重设共用 authLoginMu,等待后复验会话与当前密码;沿用账户登录失败计数,累计 5 次错误锁定 15 分钟,锁定期间返回 429(即使当前密码正确),已登录不能绕过。认证存储故障返回 503。 - 与登录、管理员重设共用 authLoginMu,等待后复验会话与当前密码;沿用账户登录失败计数,累计 5 次错误锁定 15 分钟,锁定期间返回 429(即使当前密码正确),已登录不能绕过。认证存储故障返回 503。
- 同一事务保存 bcrypt hash、撤销本人全部 session/refresh、清除失败计数,并记录 user.change_password 审计(操作者和目标均为本人 ID,detail 为空,无密码/hash/token);事务失败返回 500 并完整回滚,其他用户不受影响。成功后客户端应清理登录状态并重新登录;旧密码、旧 access/session/refresh 不再可用。已建立的 runtime WS/SSE 不主动断开,下一次请求或重连重新鉴权。 - 同一事务保存 bcrypt hash、撤销本人全部 session/refresh、清除失败计数,并记录 user.change_password 审计(操作者和目标均为本人 ID,detail 为空,无密码/hash/token);事务失败返回 500 并完整回滚,其他用户不受影响。成功后客户端应清理登录状态并重新登录;旧密码、旧 access/session/refresh 不再可用。已建立的 runtime WS/SSE 不主动断开,下一次请求或重连重新鉴权。
## 管理 ## 管理
- GET /api/users(管理员) - GET /api/users → User[](管理员,按创建时间排序)
- POST /api/users {username,password,admin?,quota_bytes?}(管理员) - POST /api/users {username,password,admin?,quota_bytes?} → User(管理员;username为3–40个字母/数字/_/-,password为12–72字节;admin默认false,quota_bytes省略或0使用默认配额,负数400;用户名重复或创建存储失败409)
- PATCH /api/users {id,quota_bytes}(管理员修改正数配额) - PATCH /api/users {id,quota_bytes} → {ok:true}(管理员修改正数配额;非法输入400,用户不存在404)
- POST /api/users/reset-password {id,password} → {ok:true,reauthenticate:boolean}(仅管理员;需有效 access/session、同源 Origin 和 X-CSRF-Token)。密码按 UTF-8 字节计为 12–72 字节,以 bcrypt 保存;缺少 ID、非法 JSON 或长度不符返回 400,目标不存在 404,未认证 401,非管理员或 Origin/CSRF 不符 403(失效 session 在 guard 中也可能返回 403),已认证的非 POST 请求返回 405。 - POST /api/users/reset-password {id,password} → {ok:true,reauthenticate:boolean}(仅管理员;需有效 access/session、同源 Origin 和 X-CSRF-Token)。密码按 UTF-8 字节计为 12–72 字节,以 bcrypt 保存;缺少 ID、非法 JSON 或长度不符返回 400,目标不存在 404,未认证 401,非管理员或 Origin/CSRF 不符 403(失效 session 在 guard 中也可能返回 403),已认证的非 POST 请求返回 405。
- 同一事务更新密码、删除目标全部 session/refresh、清除目标登录失败锁定,并记录 user.reset_password 审计(操作者 ID、目标 ID、时间,detail 为空,不含密码/hash/token)。任何数据库失败返回 500 并回滚,不影响其他用户会话。与登录串行,等待后重新检查管理员会话和权限。 - 同一事务更新密码、删除目标全部 session/refresh、清除目标登录失败锁定,并记录 user.reset_password 审计(操作者 ID、目标 ID、时间,detail 为空,不含密码/hash/token)。任何数据库失败返回 500 并回滚,不影响其他用户会话。与登录串行,等待后重新检查管理员会话和权限。
- 允许重设自身,此时 reauthenticate:true,当前会话也立即失效,客户端应清理登录状态并回到登录页;重设他人返回 false。旧密码不能再登录,旧 access/refresh 不能发起新的认证请求。已建立的 runtime WS/SSE 不会被主动立即断开:代理仅在请求/握手时鉴权,下一次请求或重连会被拒绝;现有 runtime 页面刷新认证脚本不是即时断连保证。 - 允许重设自身,此时 reauthenticate:true,当前会话也立即失效,客户端应清理登录状态并回到登录页;重设他人返回 false。旧密码不能再登录,旧 access/refresh 不能发起新的认证请求。已建立的 runtime WS/SSE 不会被主动立即断开:代理仅在请求/握手时鉴权,下一次请求或重连会被拒绝;现有 runtime 页面刷新认证脚本不是即时断连保证。
- GET /api/audit(管理员) - GET /api/audit → [{id,user_id,action,target,detail,created_at}](管理员;按id倒序最多500条,无分页)
## 镜像与构建 ## 镜像与构建
- GET /api/images(每项含 admission、admission_reason、selectable;只有 status=available 且 selectable=true 才可新建/升级)
- GET /api/compatibility(登录后可读;返回 recipe_rev、platform、entries[{version,recipe_rev,platform,state,build_allowed,reason}]、unknown{state,build_allowed,reason}、experimental_admin_only。此为普通版本构建清单,不等于每个实际镜像的准入结果) 这是 breaking change:`/api/latest`、`/api/compatibility`、`POST /api/builds` 及 `/api/intents`(包括子路径的查询、创建、取消)已移除,不提供 410 兼容入口。`GET /api/builds` 与 `GET /api/builds/{id}` 仅用于构建历史读取。旧构建及旧创建意图不迁移、不恢复执行,不在后台续跑或完成旧预约;不因此删除已有实例或数据。管理员改用 Git 发布接口构建,用户以已验证且准入的精确本地镜像 ID 创建实例。
- POST /api/images/admission {image_id,state,evidence}(管理员、同源和CSRF;state为recommended/compatible/existing_only/blocked,evidence必填。正向准入要求精确镜像身份、成功验证记录、受支持挂载布局与平台;策略与审计持久化。单镜像准入不自动扩大普通版本构建白名单)
已认证请求中,`POST /api/builds` 返回 405;`/api/latest`、`/api/compatibility` 和 `/api/intents`(含子路径)返回 404。历史意图记录留在数据库中,但不再提供存储操作、轮询或恢复流程,也不计入配额或作为镜像删除引用。
- GET /api/releases/latest(仅管理员;获取并校验配置的 Git 发布分支,返回 release_id、version、commit、repository、branch、recipe_rev、platforms、compatible、reason,以及可选的 build。检查会下载构建文件,但不执行 Dockerfile;未知部署协议或平台不允许构建)
- POST /api/releases/build {commit}(仅管理员、同源和 CSRF;提交检查时看到的精确 commit,分支已变化返回 409,需重新检查。返回 202 构建任务;不自动升级实例)
- Git 构建任务附带 release_id、source_commit、source_repository 来源信息;发布身份独立于 DSH 版本及部署布局,允许同一 DSH 版本构建新的脚本修订。可信发布通过现有镜像身份和启动验证后可准入,显式禁用仍优先。新构建仅允许管理员通过 Git 发布接口提交;main 不再包含内置 runtime 构建资产。
- GET /api/images(仅返回 Docker 本地实际存在的镜像;每项含 admission、admission_reason、selectable;只有 status=available 且 selectable=true 才可新建/升级。inspect 确认 404 后原子删除镜像索引行并将成功构建记录标为 missing,保留构建历史、显式准入策略和实例;missing 历史不会重建索引。Docker 列表或检查失败时请求失败,不删除索引行)
- POST /api/images/admission {image_id,state,evidence}(管理员、同源和CSRF;state为recommended/compatible/existing_only/blocked,evidence必填。正向准入要求精确镜像身份、成功验证记录、受支持挂载布局与平台;策略与审计持久化。单镜像准入不授权新构建)
- DELETE /api/images?image_id=sha256:...(管理员;保护实例、活动任务、升级及全部Docker容器引用,不force) - DELETE /api/images?image_id=sha256:...(管理员;保护实例、活动任务、升级及全部Docker容器引用,不force)
- GET /api/latest(查询 npm dist-tags.latest;附带 supported、build_allowed、admission、admission_reason,最新版不一定受支持)
- GET /api/builds?limit=20&offset=0(摘要数组,不含日志正文;limit默认20、最大100,offset非负;X-Has-More响应头表示后续是否有记录) - GET /api/builds?limit=20&offset=0(摘要数组,不含日志正文;limit默认20、最大100,offset非负;X-Has-More响应头表示后续是否有记录)
- GET /api/builds/{id}(按需获取单个任务及脱敏日志;仅已登录用户,日志最多64 KiB) - GET /api/builds/{id}(按需获取单个任务及脱敏日志;仅已登录用户,日志最多64 KiB)
- POST /api/builds {version,experimental?}(固定版本,异步;experimental=true仅管理员可用。未知版本普通构建会预先拒绝;实验仍执行完整性、依赖和启动验证,success不自动准入。返回任务含experimental标记。版本/模板/平台去重,新任务每用户每小时最多3个,共享复用不扣次数)
镜像策略:recommended/compatible可在实际验证与身份检查通过后用于新建/升级;existing_only允许已有实例运行和启停,不能新建/作为升级目标;blocked还拒绝新的start/restart及受控配置非GET操作(含Profile保存、备份和恢复),但不会自动停止已运行容器或断开工作台,通用文件管理仍允许。experimental是成功实验镜像未准入时的展示状态,不是兼容性结论。missing/unverified不可选。 镜像策略:recommended/compatible可在实际验证与身份检查通过后用于新建/升级;existing_only允许已有实例运行和启停,不能新建/作为升级目标;blocked还拒绝新的start/restart及受控配置非GET操作(含Profile保存、备份和恢复),但不会自动停止已运行容器或断开工作台,通用文件管理仍允许。experimental是成功实验镜像未准入时的展示状态,不是兼容性结论。missing/unverified不可选。
内置普通构建清单为 linux/amd64、模板7、0.2.0-rc.1(compatible)和0.2.0-rc.2(recommended)。其他平台/版本不继承这些验收结论。模板3–6的历史成功镜像默认仅存量,具有精确验证记录并经管理员验收可准入;未知模板布局不可正向准入,模板7向旧共享布局降级仍拒绝。 历史兼容基线为 linux/amd64、模板7、0.2.0-rc.1(compatible)和0.2.0-rc.2(recommended)。其他平台/版本不继承这些验收结论。当前正向准入仅支持模板7且平台必须匹配面板的 GOOS/GOARCH;模板3–6的历史成功验证记录保留,但不能靠管理员 evidence 正向准入。未知模板布局不可正向准入,模板7向旧共享布局降级仍拒绝。
镜像项字段为 id,image_id,version,status,created_at,usage_count,admission,admission_reason,selectable,以及可省略的 tag,recipe_rev,platform。构建项字段为 id,version,status,created_at,experimental,以及非空时返回的 image_id,error,requested_by,recipe_rev,platform,finished_at,release_id,source_repository,source_commit;详情另有非空时返回的 logs。
发布构建 commit 必须为40位小写十六进制,未知字段或多段 JSON 返回400;相同发布已有 queued/running/validating/success 任务时复用,仍返回202。每管理员一小时最多3个新构建请求,超限429。准入 evidence 不得为空白,原始字符串最多4096 UTF-8字节,成功返回镜像项。镜像删除成功返回 {image_id,status:"missing"};未索引镜像404,引用/活动任务冲突409,Docker异常502;活动构建或未完成升级会全局阻止镜像删除,不仅限于目标镜像引用。构建分页参数重复、非整数或越界返回400,并非自动截断至100。
## 实例 ## 实例
- GET /api/instances → Instance[];字段 id,uid,name,version,image_id,status,container_id,created_at - GET /api/instances → Instance[];字段 id,uid,name,version,image_id,status,container_id,created_at
- POST /api/instances {name,image_id,version} → Instance(每用户最多 2 个) - POST /api/instances {name,image_id} → Instance(每用户最多 2 个,超限409;image_id 必须为 sha256: 加64位小写十六进制且指向已验证、准入的精确本地镜像。version 由镜像标签推导,提交 version 或其他未知字段返回400,不接受构建预约。name 去除首尾空白后非空,原始长度最多200字节;请求体最多64 KiB)
- POST /api/instances/{id}/start|stop|restart - GET /api/instances/{id} → Instance
- POST /api/instances/{id}/start|stop|restart → Instance
- DELETE /api/instances/{id} → {ok:true} - DELETE /api/instances/{id} → {ok:true}
- POST /api/intents {name,version}(固定精确版本,持久预留构建后创建名额)
- GET /api/intents(当前用户意图)
- DELETE /api/intents/{id}(仅取消等待,不取消共享构建)
所有成功 JSON 列表均为顶层数组(不是 data 包装);错误均为 {error:string}。实现与验收仍在进行,不应仅凭接口列在此处认定已验收。 实例列表仅包含当前用户的实例;管理员也不能通过实例接口访问其他用户实例(404)。列表 Docker inspect 失败时该项 status 为 unavailable,详情 inspect 失败则502。生命周期操作遇到恢复状态或文件读租约冲突409,Docker操作失败通常502。
- GET /api/instances/{id}/logs
- /dsh/{id}/...(登录和 owner 校验;需同 Docker 网络) 列表型接口返回顶层数组(不是 data 包装),复合对象内的列表除外。实现与验收仍在进行,不应仅凭接口列在此处认定已验收。
- GET /api/instances/{id}/logs → {logs:string}(一次性脱敏日志,不是 SSE)
## 运行时 HTTP / WebSocket / SSE
- /dsh/{id}/...:校验有效 session/access 和 owner,需同 Docker 网络;不存在或非本人404,恢复状态409,运行时未启动、网络或认证不可用502。/dsh/{id} 重定向至带尾斜杠路径(307)。
- WS/SSE 通过相同前缀透明代理,不是 /api/ws/(后者是文件管理)。WS握手必须携带精确同源 Origin;普通 GET/HEAD 可省略 Origin,但提供时必须匹配;其他方法必须同源。这里不校验 X-CSRF-Token,也不将该头转发给上游。
- 认证只在请求/握手时检查,不因 access 到期、退出或改密主动关闭已建立流。未认证的普通请求/WS返回401;满足浏览器 document/iframe 导航条件的 GET 会303跳转到面板 /#runtime-renew 续期桥。
- 代理剥离面板 Cookie、Authorization、Proxy-Authorization 和查询中的运行时 token,使用服务端交换得到的运行时 Cookie;上游 Set-Cookie 不下发,Location 重写到实例前缀。WS帧及 SSE 事件结构由运行时决定,面板未定义独立消息协议。上游认证失败不会自动重放写请求。
## 工作区 ## 工作区
指定实例时文件根为该所有者实例的完整home;允许下载凭据及目录ZIP,不承诺write-only。下载提示仅在界面防误操作,API不按敏感文件名过滤。通用文件修改不强制走专用Profile的停机/YAML校验流程;归属、路径安全、配额、恢复状态和现有文件操作并发保护仍有效。上传不覆盖同名文件(409)。 除 usage 外,所有 /api/ws/ 请求必须带查询参数 instance={id},缺少或空值返回410(共享文件入口已移除);实例不存在或不属本人返回404,管理员无跨用户豁免。文件根为该实例的完整home;允许下载凭据及目录ZIP,不承诺write-only。下载提示仅在界面防误操作,API不按敏感文件名过滤。通用文件修改不强制走专用Profile的停机/YAML校验流程;归属、路径安全、配额、恢复状态和现有文件操作并发保护仍有效。上传不覆盖同名文件(409)。下列简写路径同样必须补上 instance 查询参数。
- GET /api/ws/tree?path=. - GET /api/ws/tree?path=. → [{name,path,is_dir,size,mod_time}](按名称排序,目录 size 为0)
- GET /api/ws/usage - GET /api/ws/usage → {used_bytes,quota_bytes}(当前用户总用量,不是单实例用量;可不带 instance,带上时仍检查归属)
- POST /api/ws/mkdir {path} - POST /api/ws/mkdir {path}
- POST /api/ws/rename {path,new_path} - POST /api/ws/rename {path,new_path}
- DELETE /api/ws/delete?path=... - DELETE /api/ws/delete?path=...
- POST /api/ws/upload multipart file,path - POST /api/ws/upload multipart file,path
- mkdir/rename/delete/upload 成功均返回 {ok:true};upload 的 path 是目标目录,instance 仍须在 URL 查询中提供。配额/文件大小限制413,扩展名白名单拒绝415,并发上传限流429;multipart解析失败(含部分超限请求)400。
- GET /api/ws/image?instance=...&path=...:只读图片预览,沿用工作区认证、实例归属检查和实例读租约;仅支持 GET。
- path 与 text 接口一致:保留空白和反斜杠,必须是非空、规范的相对文件路径。Linux openat2 限制在实例 home 内,拒绝符号链接、跨挂载点、多硬链接及特殊文件;不写入文件。
- 有界读取最多 16 MiB 的内存快照,读取期间检测到文件变化返回 409,超限返回 413。PNG/JPEG/GIF/WebP 按 magic 识别;SVG 必须有 .svg 扩展名(不区分大小写),完整 XML 的唯一根元素必须是 SVG 命名空间的 svg,拒绝 HTML、XML directives/DOCTYPE 和未知实体,不加载外部实体。其他格式或目录返回 415,缺失/不安全文件返回 404,非法路径返回 400。
- 返回原始图片字节及 image MIME;固定 `X-Content-Type-Options: nosniff`、`Content-Security-Policy: sandbox; default-src 'none'; style-src 'unsafe-inline'`、`Content-Disposition: attachment`、`Cache-Control: no-store`。SVG 不做内容改写,依靠图片上下文和 CSP 限制脚本及外部资源;直接导航下载附件。
- GET /api/ws/text?instance=...&path=... → {content:string,version:string}
- PUT /api/ws/text?instance=...&path=... JSON {content:string,version:string} → {version:string}
- 仅编辑已有普通文件;沿用工作区认证、实例归属和恢复保护。必须提交 GET 返回的不透明 version;缺少 content/version 返回 400,版本不匹配或保存前文件已删除返回 409,不创建新文件。GET 不存在/不安全路径返回 404;非法相对路径返回 400。
- 内容最多 1 MiB(UTF-8 字节数,含边界),超限 413;无效 UTF-8、二进制/控制字符返回 415(仅允许 tab、LF、CR 控制字符)。JSON 请求上限为 6 MiB + 4096 字节以容纳转义,超限 413。
- Linux openat2 限定访问工作区内路径,拒绝符号链接、跨挂载点和多硬链接文件;不支持 openat2 时安全失败。固定父目录描述符,在同目录写临时文件、保留原文件 mode/UID/GID、同步文件,再原子 rename 替换;不保留扩展属性/ACL。审计仅记录操作和路径,不记录内容。
- 用户锁串行化面板保存;锁内读取当前配额和用量,要求现有用量 + 新内容大小不超配额(包括临时文件开销,否则 413)。因此接近配额时即使缩小文件也可能无法保存。外部进程不受该锁和配额检查约束。
- rename 前立即重新读取目标并核对最新内容/版本,但检查与 rename 不是文件系统 CAS:外部写入者仍可能在两者之间写入或删除目标,不能保证绝不覆盖外部修改。父目录被外部重命名时操作仍针对固定目录。需严格避免丢失修改时,先停止外部写入;不承诺断电后的目录项持久性。
- GET /api/ws/download?path=...:普通文件保持原始流与 Range 支持;目录返回 `application/zip` 附件(目录名.zip),空 path 或 `.` 下载整个工作区(workspace.zip)。ZIP 条目相对所选目录,保留空目录,不额外包裹顶层目录,使用 Store 模式(不压缩),不支持 ZIP Range。 - GET /api/ws/download?path=...:普通文件保持原始流与 Range 支持;目录返回 `application/zip` 附件(目录名.zip),空 path 或 `.` 下载整个工作区(workspace.zip)。ZIP 条目相对所选目录,保留空目录,不额外包裹顶层目录,使用 Store 模式(不压缩),不支持 ZIP Range。
- ZIP 先校验有界文件清单,再逐文件流式输出,不生成临时归档,不持 workspace/global 锁跨网络传输。每次最多 10,000 个文件/目录条目、64 层、10 GiB 文件总大小;进程最多同时 4 个目录下载,超出分别返回 413 / 429;扫描和传输合计限时 2 分钟。 - ZIP 先校验有界文件清单,再逐文件流式输出,不生成临时归档,不持 workspace/global 锁跨网络传输。每次最多 10,000 个文件/目录条目、64 层、10 GiB 文件总大小;进程最多同时 4 个目录下载,超出分别返回 413 / 429;扫描和传输合计限时 2 分钟。
- 路径必须位于工作区内;下载拒绝任意路径分量的符号链接、跨挂载点、特殊文件和多硬链接普通文件。目录含上述条目或含反斜杠/冒号的不安全 ZIP 文件名时整体拒绝(400),不静默漏文件。依赖 Linux openat2,内核不支持时安全失败。 - 路径必须位于工作区内;下载拒绝任意路径分量的符号链接、跨挂载点、特殊文件和多硬链接普通文件。目录含上述条目或含反斜杠/冒号的不安全 ZIP 文件名时整体拒绝(400),不静默漏文件。依赖 Linux openat2,内核不支持时安全失败。
- 下载不是原子快照:流式读取前复核文件身份、大小、类型及修改时间;并发修改、取消或传输错误可能导致连接中断/不完整 ZIP,请停止写入后重试。单文件下载不受 ZIP 数量、大小、并发和时限约束。 - 下载不是原子快照:流式读取前复核文件身份、大小、类型及修改时间;并发修改、取消或传输错误可能导致连接中断/不完整 ZIP,请停止写入后重试。单文件下载不受 ZIP 数量、大小、并发和时限约束。
## profile / 备份 ## profile / 备份
- GET /api/instances/{id}/profile - GET /api/instances/{id}/profile → {content:string}(配置不存在时 content 为空)
- PUT /api/instances/{id}/profile {content,restart?}(停机原子写;restart=true启动并检查就绪) - PUT /api/instances/{id}/profile {content,restart?}(停机原子写;restart=true启动并检查就绪)
- GET /api/instances/{id}/profile-preview(bundle清单和脱敏dump-config,字符串值隐藏;运行中才能dump) - GET /api/instances/{id}/profile-preview(bundle清单和脱敏dump-config,字符串值隐藏;运行中才能dump)
- POST /api/instances/{id}/backups {password}(当前登录密码加密;自动恢复原运行状态) - POST /api/instances/{id}/backups {password}(当前登录密码加密;自动恢复原运行状态)
- POST /api/instances/{id}/backups multipart file,password(导入外部包,password为原包密码;不立即覆盖home) - POST /api/instances/{id}/backups multipart file,password(导入外部包,password为原包密码;不立即覆盖home)
- GET /api/instances/{id}/backups - GET /api/instances/{id}/backups
- GET /api/instances/{id}/backups?download={backup_id} - GET /api/instances/{id}/backups?download={backup_id}
- DELETE /api/instances/{id}/backups?id={backup_id} → {ok:true}
- GET /api/instances/{id}/snapshots → [{id,created_at,size,protected,protection_reason,restorable,restore_reason,source_version?,source_image?,size_error?}]
- DELETE /api/instances/{id}/snapshots?id={snapshot_id} → {ok:true}
- GET /api/instances/{id}/snapshots?download={snapshot_id} → 完整 home 的 `.tar.gz` 附件(未加密,可能含凭据)
- POST /api/instances/{id}/snapshots {id,password,confirm:true,allow_unknown_version?} → {ok:true,snapshot_id}(当前登录密码;snapshot_id 为切换前 home 的保留快照)
- POST /api/instances/{id}/restore {backup_id,password}(原备份密码;旧平台key包可留空) - POST /api/instances/{id}/restore {backup_id,password}(原备份密码;旧平台key包可留空)
- POST /api/instances/{id}/upgrade {image_id,password}(当前登录密码,加密pre-upgrade备份) - POST /api/instances/{id}/upgrade {image_id,password}(当前登录密码,加密pre-upgrade备份)
Profile写入要求实例先停止。备份/还原/升级会受控停启;恢复保留原运行状态,升级成功运行新版,失败恢复旧状态。模板2入口先准备插件再启动应用。删除实例仅删除容器与网络并移除索引,保留数据目录。升级失败恢复实例home,不撤销共享workspace或外部系统副作用。 Profile PUT 与 restore 成功返回 {ok:true},upgrade 返回 Instance。profile-preview 返回 {bundles,config,redacted:true,preview_error?};无法运行 dump 时仍可200并携带 preview_error,不保证 config 非空。备份列表为 [{id,size,created_at,kind,schema}],创建/导入返回同形单个对象,download 返回二进制附件。备份数量上限20,达到上限409;恢复快照达到保留上限也返回409,需管理员清理。配置非GET请求体有界读取,超限413、读取超时408、并发槽耗尽503;Profile JSON最多2 MiB,普通备份/restore JSON最多4096字节。
Profile写入要求实例先停止。备份/还原/升级会受控停启;恢复保留原运行状态,升级成功运行新版,失败尝试恢复旧状态,回滚失败需人工处理。运行镜像的插件准备行为由发布分支维护,应针对精确镜像验证。删除实例先清理容器与网络,再永久删除该实例的 home、快照及备份,最后移除实例索引;数据清理失败保留 error 实例以便重试。升级仅支持实例home到 /workspace 的单一bind布局,不迁移旧共享挂载;home回滚不撤销外部系统副作用。
创建恢复:Docker 创建响应丢失时,首次 inspect 404 不代表原请求已结束。实例保留为可见的 error 状态并继续占用名额,用户可重试删除;后台不会仅凭名称删除身份不匹配资源。不确定创建记录不会自动过期:若资源始终不存在,需管理员确认 Docker 原创建请求已终止后再处理保留记录,不能仅凭等待时长判定安全。构建验证容器在创建前持久登记任务身份,启动时即使没有排队构建也会回收并重试失败清理;持续不存在的未知结果仍保留登记以捕获晚到容器。构建终态和镜像索引事务失败会独立重试并记录日志,不阻塞后续构建;数据库关闭停止重试,重启将未完成构建标记失败。 创建恢复:Docker 创建响应丢失时,首次 inspect 404 不代表原请求已结束。实例保留为可见的 error 状态并继续占用名额,用户可重试删除;后台不会仅凭名称删除身份不匹配资源。不确定创建记录不会自动过期:若资源始终不存在,需管理员确认 Docker 原创建请求已终止后再处理保留记录,不能仅凭等待时长判定安全。构建验证容器在创建前持久登记任务身份,启动时即使没有排队构建也会回收并重试失败清理;持续不存在的未知结果仍保留登记以捕获晚到容器。构建终态和镜像索引事务失败会独立重试并记录日志,不阻塞后续构建;数据库关闭停止重试,重启将未完成构建标记失败。
已实现并通过核心集成检查:镜像复用验证、Docker 生命周期、token/cookie 桥接、加密备份恢复、同镜像替换升级。WS/SSE 代理有单元测试,真实完整会话待验证。硬配额与跨公网租户域名隔离未实现;当前 dsh 发行版不支持 --public-url。默认仅 loopback 监听,不应直接暴露公网。完整限制见 README.md。 镜像验证、Docker 生命周期、token/cookie 桥接、加密备份恢复和同镜像替换升级已有实现与历史集成记录,具体版本和本次验证范围见 ACCEPTANCE.md,不能据此推断当前全部链路已重新验收。WS/SSE 代理有单元测试,真实完整会话需单独验证。硬配额与跨公网租户域名隔离未实现;已验证的 DSH rc.1/rc.2 启动流程不使用 --public-url。默认仅 loopback 监听,不应直接暴露公网。完整限制见 README.md。
+129 -231
View File
@@ -1,6 +1,8 @@
# DeepSeek Harness 多租户容器管理平台 — 设计文档 # DeepSeek Harness 多租户容器管理平台 — 设计文档
> 状态:本机管理面板核心流程已实现并通过真实 Docker 集成检查;生产隔离、硬配额等仍有差距,以 README.md 的实现边界与验证范围为准。下文是目标设计,不代表所有条目已实现。 > 当前发布方式为管理员检查Git发布并确认精确commit,本地构建runtime;release.json、image/与初始化测试仅在dockerfile发布分支维护。旧npm/实验构建和构建预约已移除,详见§5。历史验收不自动适用于当前架构,验证范围见ACCEPTANCE.md,部署操作见OPERATIONS.md。
> 状态:当前代码具备可信团队管理面板核心流程;历史真实 Docker 验收仅适用于各自记录的版本,不代表当前 Git 发布与全部新增功能已验收。本文“当前实现基线”优先于后面的历史设计;未实现规划不作为已交付能力。验收证据与缺口见 ACCEPTANCE.md。
> 目标项目:`deepseek-ai/deepseek-harness`(npm CLI `@deepseek-ai/dsh`,命令 `dsh web`) > 目标项目:`deepseek-ai/deepseek-harness`(npm CLI `@deepseek-ai/dsh`,命令 `dsh web`)
> 控制面技术栈:Go + 前端 SPA > 控制面技术栈:Go + 前端 SPA
> 部署形态:单机 Docker daemon;面板自身亦运行于容器中 > 部署形态:单机 Docker daemon;面板自身亦运行于容器中
@@ -8,7 +10,54 @@
--- ---
## 0. 调研结论与硬约束 ## 当前实现基线(本次源码核对)
本节描述仓库代码,而不是部署状态。本次本地测试与构建结果见 ACCEPTANCE.md 当前记录;代码已实现和自动化通过均不等于真实环境已验证。正文标为“当前实现”的章节是现行方案,历史调研与可选规划另行标注。
### 技术栈与历史 Flutter 的边界
- 当前前端是 `web/` 下的 React 19 + TypeScript + Vite,Tailwind CSS v4、Radix 原语的 shadcn 风格组件、lucide 图标及按需加载的本地 Monaco;入口为 `web/src/main.tsx`、`web/src/App.tsx`。登录默认进入内嵌 DSH,桌面侧栏与移动底栏共用响应式页面;主题跟随系统。
- Flutter/Android 属于历史本地工程痕迹:工作区可能仍有 `android/`、`.dart_tool/`、`build/`、IDE 文件,但当前 Git 跟踪文件没有 Dart 源码或 pubspec。它们不是现行应用、构建输入或受支持的 Android 发布产物,不能据目录存在宣称 Flutter 客户端仍在维护。
- 后端模块要求 Go 1.25(`go.mod`),Docker 构建阶段使用 Go 1.26;标准库 `net/http` + 自有 Docker Engine HTTP client(API v1.47),不是官方 Docker SDK。SQLite 使用 `modernc.org/sqlite`,密码为 bcrypt,配置备份采用 Argon2id + AES-GCM。
```text
cmd/server/ Go 启动、静态 web/dist 服务
internal/api/ HTTP、鉴权、发布构建、实例/文件/代理/配置/升级/恢复编排
internal/docker/ 标准库 Docker Engine HTTP client
internal/backup/ 配置打包与加密
internal/store/ SQLite、会话、实例名额事务
web/src/ React 页面、组件、样式与前端测试
scripts/ 本地检查及需显式运行的集成/镜像冒烟脚本
Dockerfile 构建面板及 React 静态资源,不构建 DSH runtime
compose.yaml 当前面板部署模板
```
`internal/store/intents.go` 当前承载实例名额事务,不表示仍有预约 API。runtime 的 `release.json`、`image/` 与初始化测试在 `dockerfile` 发布分支,main 不包含这些构建资产。
### 架构与功能边界索引
现行网络、路径、目录、构建、生命周期与认证详见§2–8;文件/配置/升级见§9–12。特别注意:每实例home独立,实例删除会清理自身数据及备份;共享文件入口与构建预约均已移除;旧不确定创建“宽限期后404释放”策略已被替代。
### 当前发布与准入流程(详见 §5)
1. 管理员通过 `GET /api/releases/latest` 检查配置的 Git 发布仓库/分支及 manifest;通过 `POST /api/releases/build` 确认精确 commit。分支已变化返回冲突,需重新检查。
2. 构建任务绑定仓库、commit、release ID、版本、部署协议与平台;执行时重新验证信任源和管理员权限,获取锁定 commit 的上下文,本地构建并验证,保留分页历史及详情日志(`releases.go`、`releases_git.go`、`images.go`、`builds_read.go`)。不自动升级已有实例。
3. 镜像准入仍区分推荐、兼容、仅存量与禁止;验证记录及实际镜像身份必须匹配。当前 `supportedLayout` 仅接受协议/模板 7(`admission.go`),不能套用历史“模板 3–6 可正向准入并使用双挂载”的记录。
4. 普通用户仅选择可用本地镜像。旧 latest/compatibility、写 builds、全部 intents 接口已经移除;实验构建、构建预约、旧任务续跑不属于当前能力。
### 当前部署、未实现规划与验证边界
- 当前 `compose.yaml` 为面板 UID/GID 1000、socket 附加组、只读根、tmpfs、资源限制,默认发布 `127.0.0.1:8088`;需配置数据根、DockerRootDir、公开 URL 与可信代理。部署说明见 §16、仓库实际 Compose 和 OPERATIONS.md。
- XFS/ZFS 硬配额、特权配额 sidecar、socket-proxy、子域名入口属于未交付的可选设计,不是现行默认能力;实际公网 TLS/代理、真机与长期稳定性需独立验收。软配额不能阻止容器直接写盘瞬时超限。
- `scripts/check-trial.sh` 当前检查 Go 测试/race/vet/build、前端测试/build 和集成脚本语法,不自动运行真实 Docker、浏览器或发布分支初始化测试。完整新 Git 发布链路及部署验证的通过状态以 ACCEPTANCE.md 的当前矩阵为准,不由历史 M1–M7 结案推导。
---
## 详细设计与历史调研
下文当前实现章节已同步;§0为历史上游调研,§9中的硬配额为可选规划。上游链接及版本调研未在本轮重新验证,不能据此推断未来发行版行为。
## 0. 历史上游调研结论与硬约束
> 实现阶段实测更正:本地 npm `0.2.0-rc.2` 支持 `--trusted-host`,但拒绝 `--public-url`;下文基于 upstream master 的 public-url 描述不能直接用于该发行版。实际启动仅传 web/no-open/port/trusted-host,代理负责外部路径。`scripts/probe-runtime.mjs` 已验证默认 token 交换为 303 + cookie、认证首页 200;完整代理以集成测试为准。 > 实现阶段实测更正:本地 npm `0.2.0-rc.2` 支持 `--trusted-host`,但拒绝 `--public-url`;下文基于 upstream master 的 public-url 描述不能直接用于该发行版。实际启动仅传 web/no-open/port/trusted-host,代理负责外部路径。`scripts/probe-runtime.mjs` 已验证默认 token 交换为 303 + cookie、认证首页 200;完整代理以集成测试为准。
@@ -33,199 +82,113 @@
## 1. 目标与范围 ## 1. 目标与范围
### 已确认的访问与镜像策略(覆盖下文早期假设) ### 已确认的访问与镜像策略
- 面向可信团队,实例所有者可通过文件管理读取、下载、修改完整 DSH_HOME,包括自己的凭据。面板不承诺凭据 write-only;跨用户隔离、路径安全、配额与恢复状态保护保持不变。下载文件/打包目录会提示泄密风险,提示属于交互防误操作,不是阻止所有者访问的安全边界。 - 面向可信团队,实例所有者可通过文件管理读取、下载、修改完整 DSH_HOME,包括自己的凭据。面板不承诺凭据 write-only;跨用户隔离、路径安全、配额与恢复状态保护保持不变。下载文件/打包目录会提示泄密风险,提示属于交互防误操作,不是阻止所有者访问的安全边界。
- 专用 Profile 编辑器提供 YAML 校验与受控停启流程,但通用文件操作、DSH 终端和 agent 不强制经过该流程;修改运行中配置存在不一致风险,建议先停机并备份。 - 专用 Profile 编辑器提供 YAML 校验与受控停启流程,但通用文件操作、DSH 终端和 agent 不强制经过该流程;修改运行中配置存在不一致风险,建议先停机并备份。
- 镜像准入分为推荐、仍兼容、仅存量运行、禁止使用。兼容依据绑定精确版本、模板修订、平台与镜像身份;不再仅因历史模板不是当前模板就一律判为不可用,也不能仅因历史构建成功就认定兼容。 - 镜像准入分为推荐、仍兼容、仅存量运行、禁止使用。兼容依据绑定精确版本、模板修订、平台与镜像身份;不再仅因历史模板不是当前模板就一律判为不可用,也不能仅因历史构建成功就认定兼容。
- 未验证 DSH 新版仅管理员可显式实验构建;保留既有完整性、依赖与启动检查。实验构建成功不是兼容验收,未明确准入的镜像不可新建实例或作为升级目标。管理员正向准入须填写验收依据并记录审计,不绕过实际镜像身份与验证记录检查。 - 新版本仅通过管理员 Git 发布流程构建;实验构建已移除。管理员正向准入须填写验收依据并记录审计,不绕过布局、平台、实际镜像身份与验证记录检查。
- 通过 docker.sock 在单机上**创建/启停/删除**多个 `dsh web` 容器,**每用户最多 2 个**。 - 通过 docker.sock 在单机上**创建/启停/删除**多个 `dsh web` 容器,**每用户最多 2 个**。
- 每用户一个**独立工作区**,支持文件**上传/下载/删除/重命名**,并有**容量配额**。 - 每实例独立完整 home,支持文件上传/下载/删除/重命名、文本编辑和图片预览,按账号统计软配额。
- 工作区**映射进每个 dsh 容器**(agent 在工作区内干活)。 - 实例 home 挂入自身容器的 /workspace,不与同账号其他实例共享挂载。
- **备份/还原 dsh 配置**(含模型与插件清单)。 - **备份/还原 dsh 配置**(含模型与插件清单)。
- **编辑 profile 文件**(`cordis.patch.yml`)。 - **编辑 profile 文件**(`cordis.patch.yml`)。
- **升级**:保留数据,用新版本镜像重启容器。 - **升级**:保留数据,用新版本镜像重启容器。
- **本地镜像管理**:面板查询 DSH 最新版本,用户手动触发本机构建;创建实例时选择本地已有版本,或手动构建最新版后创建。不需要私有镜像仓库,不推送生成的镜像。 - **本地镜像管理**:管理员检查 Git 发布并确认精确 commit 构建;用户选择经验证且准入的本地镜像创建,不提供构建预约。不推送生成的镜像。
- 控制面:Go 后端 + SPA 前端;内置账号 + 目录配额;**面板自身容器化运行**。 - 控制面:Go 后端 + SPA 前端;内置账号 + 目录配额;**面板自身容器化运行**。
--- ---
## 2. 总体架构 ## 2. 总体架构(当前实现)
面板不是宿主进程,而是挂了 `docker.sock` 的**兄弟容器**(sibling),通过 sock 编排 dsh 兄弟容器,并通过同一 bridge 网络反代它们。 面板是通过 docker.sock 编排 sibling runtime 的 Go 容器,提供 React 静态资源、API、路径反代和 SQLite 持久化。每个实例独立 bridge,面板接入对应网络,runtime 不发布主机端口。容器内 DSH 监听 loopback:3080,由发布镜像内 Caddy 在8080转发;镜像内部实现须以精确发布 commit 为准。
``` 浏览器 → 面板 SPA/API 或 /dsh/<id>/ → 实例专属 bridge → runtime:8080 → DSH:3080。
┌────────────── 宿主 ──────────────────┐
│ docker.sock ──┐ │
│ /var/lib/dsh-mgr ──┐ (bind) │
│ │ │
│ ┌─────────────────▼─────────────▼───┐ │ dsh-net (bridge)
│ │ 面板容器 dsh-mgr │ │ :80/:443 发布
│ │ - /data ← /var/lib/dsh-mgr │ │
│ │ - /var/run/docker.sock │ │
│ │ - SPA + API + 反代 + 编排引擎 │ │
│ └─────────┬───────────────────────────┘ │
│ │ sock │
│ ┌─────────▼───────────┐ ┌───────────────┐ │
│ │ dsh-web:<v> 容器1 │ │ 容器2 ... │ │ 不发布端口
│ │ caddy:8080 → 3080 │ │ │ │ /data/dsh-home、/workspace ← 宿主 bind
│ └─────────────────────┘ └───────────────┘ │
└─────────────────────────────────────────────┘
```
组件: socket-proxy 仍为可选规划,当前 Compose 直接挂 socket,不代表具备宿主级隔离。
1. **面板服务**(Go 容器):REST API + 静态 SPA + 反向代理子系统 + docker client + 配额/备份/升级引擎。
2. **dsh-web 镜像**(自建):Node + `@deepseek-ai/dsh` + caddy + 入口脚本。
3. **dsh-net**:内部 bridge 网络,容器不发布主机端口,仅面板可达。
4. **(可选)socket-proxy**:面板与 sock 之间的白名单代理。
--- ---
## 3. 宿主路径 ↔ 面板内路径映射(关键) ## 3. 宿主路径 ↔ 面板内路径映射(当前实现)
Docker bind 的 `Source` 必须是**守护进程所在宿主路径**,不是面板容器内路径。面板容器把宿主 `/var/lib/dsh-mgr` 挂到 `/data`,于是同一文件树有两个视角: Docker bind.Source 必须使用 daemon 宿主路径。面板文件操作使用 CONTAINER_DATA_ROOT(Compose 为 /data),通过 internal/api/paths.go 的 toHostPath 映射到 HOST_DATA_ROOT。
| 用途 | 面板直接读写(容器视角) | 创建 dsh 容器 bind.Source(宿主视角) | | 用途 | 面板路径 | Docker bind.Source / Target |
|---|---|---| |---|---|---|
| 用户工作区 | `/data/workspaces/<uid>/files/...` | `/var/lib/dsh-mgr/workspaces/<uid>/files` | | 实例完整 home | /data/workspaces/<uid>/instances/<id>/home | <HOST_DATA_ROOT>/workspaces/<uid>/instances/<id>/home → /workspace |
| 实例 DSH_HOME | `/data/workspaces/<uid>/instances/<id>/home/...` | `/var/lib/dsh-mgr/workspaces/<uid>/instances/<id>/home` | | 配置备份 | /data/backups/<uid>/<id>/ | 不挂入 runtime |
| 备份包 | `/data/backups/<uid>/<id>/...` | (不入容器,仅面板读写) |
面板配置: 构建上下文是从锁定 Git commit 获取并提交的归档流,不应把历史内置 image/ 当作宿主 bind 源。
```
HOST_DATA_ROOT=/var/lib/dsh-mgr # 传给 docker API 的 bind 源
CONTAINER_DATA_ROOT=/data # 面板自己 fs 操作的根
```
- 所有"面板直接动文件"(文件 API、备份 tar、profile 读写)用 `CONTAINER_DATA_ROOT`。
- 所有"喂给 docker API 的 bind.Source / build context"用 `HOST_DATA_ROOT` 拼接。
- 封装 `toHostPath(containerPath)` 翻译函数集中调用,避免散落拼接出错——这是 sibling 容器最常见的 bug。
--- ---
## 4. 数据与目录布局(宿主机) ## 4. 数据与目录布局(当前实现)
``` ```
/var/lib/dsh-mgr/ <HOST_DATA_ROOT>/
db.sqlite # 用户、实例、配额、备份索引 db.sqlite
workspaces/<uid>/ workspaces/<uid>/
files/ # 用户工作区(面板文件 API + 各 dsh 容器共享挂载) instances/<instId>/home/ # 完整 DSH_HOME → /workspace
└─ <用户自由目录结构> .credentials.yaml
instances/<instId>/ cordis.patch.yml
home/ # DSH_HOME(bind → 容器 /data/dsh-home) profiles/web/ # 配置、插件清单、可重建依赖
.credentials.yaml # API Key(机密) storages/ # 实例工作区数据等
cordis.patch.yml # home 级 patch files/ # 可能遗留;当前无共享入口或新容器挂载
profiles/web/ backups/<uid>/<instId>/ # 加密配置备份;不是完整 home 快照备份
package.json # 插件依赖 + dsh.profile bundle 清单
cordis.patch.yml # profile 级 patch(= "profile 文件")
node_modules/ # 插件实体(可重建,不入备份)
skins/ pets/ agent-presets/ .agent-presets/
dsh-ssh.json # 0600
sessions/ worktrees/ # 运行态(不入配置备份)
backups/<uid>/<instId>/
<timestamp>.tar.gz # 配置备份包
# 镜像层由本机 Docker daemon 管理,不作为此目录下的文件保存;DB 记录镜像索引
``` ```
**配额作用域**:`workspaces/<uid>/` 整棵树(files + 所有 instances 的 home)。每用户一个配额单位。 账号软配额统计 workspaces/<uid>/,并非只统计当前实例。实例删除会清理该实例目录及对应备份;历史共享 files 不自动合并到 home。旧目录结构和旧删除保留策略不能作为现行承诺。
--- ---
## 5. 自建 DSH 镜像设计 ## 5. 自建 DSH 镜像(当前 Git 发布流程)
``` 管理员检查 GET /api/releases/latest,再以 POST /api/releases/build 确认精确 commit;分支变化必须重新确认。后端从受信任仓库锁定 commit 获取 release.json 与 image/,校验部署协议7及当前面板平台,构建本地镜像并验证。构建记录绑定仓库/commit/release ID/版本/平台,任务去重、限频、容量检查与失败记录仍保留。GET /api/builds 及 GET /api/builds/{id} 提供历史与日志。
FROM node:22-slim
# 1) 装 dsh CLI(版本参数化)
ARG DSH_VERSION=0.2.0-rc.2
RUN npm i -g @deepseek-ai/dsh@${DSH_VERSION}
# 2) 装 caddy(容器内反代,绕过 dsh 拒绝 0.0.0.0)
COPY caddy /usr/bin/caddy
COPY Caddyfile /etc/caddy/Caddyfile # :8080 { reverse_proxy 127.0.0.1:3080 }
COPY entrypoint.sh /entrypoint.sh
ENV DSH_HOME=/data/dsh-home
WORKDIR /workspace
ENTRYPOINT ["/entrypoint.sh"]
# CMD 由面板按实例注入:dsh web --no-open --port 3080 --public-url <url> --trusted-host <host>
```
- `entrypoint.sh`:`tini` 接管信号 → 起 caddy(后台)→ `exec dsh "$@"`。dsh 绑 loopback:3080,caddy 绑 0.0.0.0:8080。 不再从npm查询latest,不接受普通用户实验构建,不打包main内置image/,不提供构建后创建预约。创建实例必须另行选择经验证且准入的本地image ID;管理员构建成功不等于已完成完整兼容验收或自动升级实例。
- **存储方式**:生成的镜像仅保存在本机 Docker daemon,不推送 registry,也不从 registry 拉取成品 `dsh-web` 镜像。所有用户共享面板管理的本地镜像目录。
- **版本查询**:面板从 npm 查询 `@deepseek-ai/dsh` 的 `dist-tags.latest`,展示精确版本和查询时间;查询不触发构建或升级。不通过版本字符串排序猜测最新版。查询失败时保留本地版本列表,缓存结果需标明过期。
- **版本固定**:用户确认构建时锁定页面展示的精确版本,后端校验其确实存在于 npm 元数据中;构建过程中不再次解析 `latest`,Dockerfile 不安装浮动的 `@latest`。
- **本机构建**:Docker API `POST /build` 接受 tar 流;面板将内置 `/image/` 的 Dockerfile+caddy+entrypoint+Caddyfile 打包后提交给 daemon,并通过 Docker API 的 buildargs 传入 `DSH_VERSION`。用户不能提交任意 Dockerfile、构建上下文、镜像地址或其他构建参数。
- **镜像身份**:tag 使用 `dsh-web:<dshver>-<recipeRev>`,其中 `recipeRev` 标识镜像模板修订;记录精确 DSH 版本、模板修订、架构、本地 image ID 和构建时间。创建实例使用已确认的 image ID,不依赖可能被覆盖的 tag。
- **构建任务**:手动触发后异步执行,展示排队、构建中、验证中、成功、失败状态及脱敏日志;任务持久化。同版本、模板修订和架构的并发请求复用同一任务,已有可用镜像直接复用。默认全局串行构建,并限制用户触发频率、任务时长及日志大小,构建前检查磁盘空间。
- **可用性判定**:Docker build 成功后,用临时隔离容器验证版本和 DSH 启动就绪;验证通过才加入可选择列表。失败保留错误记录,不自动替换已有可用镜像。面板重启后对账未完成任务及本地镜像,不盲目重复构建。
- **本地清单**:以 Docker inspect/list 和面板管理标签对账,展示精确版本、构建时间、可用状态及使用情况。外部删除导致 image ID 不存在时标记缺失,不再允许新建;管理员清理时保护被实例、待执行任务或回滚流程引用的镜像,不自动运行全局 prune。
- **网络要求**:仅本地保存不代表离线构建;首次构建仍可能需要拉取基础镜像及下载 npm 包。网络故障不影响选择已存在的本地可用镜像。
**创建实例交互**: 镜像准入绑定实际身份和成功验证记录;supportedLayout只接受模板7,平台须匹配当前面板平台,内置defaultAdmission规则另有限定linux/amd64。旧模板记录可以保留,不意味着可通过填写说明正向准入。镜像清理保护有效引用,不自动全局prune。构建仍可能需要基础镜像和依赖网络,不能宣称离线构建。
1. **选择本地版本**:从本地可用列表选择版本,校验 image ID 仍存在后创建。
2. **构建最新版并创建**:查询并展示精确版本 → 用户确认 → 事务预留实例名额 → 构建或复用任务 → 验证成功 → 用该 image ID 创建实例。等待构建的实例也计入每用户 2 个名额;构建/创建失败或取消时清理未完成资源并释放名额,不回退到其他版本。取消某个用户的等待不取消其他用户共享的构建任务。
3. 面板另提供独立的“构建最新版”按钮,仅生成本地镜像,不创建或升级实例,也不占实例名额。
--- ---
## 6. 容器生命周期(docker.sock + socket-proxy) ## 6. 容器生命周期(当前实现)
Go 端用 `github.com/docker/docker/client`: 标准库 Docker HTTP client 使用 API v1.47。创建固定 image ID,事务限制每用户最多两个实例,无构建预约。每实例独立网络与 home,UID/GID 1000,工作目录 /workspace,DSH_HOME/HOME 同为 /workspace,XDG 路径位于其内。启动参数为 web --no-open --port 3080 --trusted-host <host>,不传 --public-url。
- **创建**: 当前资源约束为只读 rootfs、drop ALL capabilities、no-new-privileges、2GiB 内存、2 CPU、256 PID 及有界可执行 /tmp。实例支持启停、重启、日志、删除;删除回收容器/网络并清理实例数据与备份。创建不确定时保留恢复依据,不能靠一次或超时后的404清除。孤儿资源仅管理员明确确认并核验后处理,不自动纳管。
- Image = 本地可用镜像的固定 image ID(对应精确 DSH 版本及模板修订,见 §5)
- Env:`DSH_HOME=/data/dsh-home`(密钥不入 env;用户在 dsh UI 里设 key,落到 `.credentials.yaml`)
- Binds(Source 用 **HOST 路径**):
- `workspaces/<uid>/instances/<instId>/home` → `/data/dsh-home`
- `workspaces/<uid>/files` → `/workspace`
- Network = `dsh-net`(不发布端口)
- Cmd = `web --no-open --port 3080 --public-url https://<panel>/dsh/<instId>/ --trusted-host <panel>`
- Labels:`dsh-mgr.uid`、`dsh-mgr.instId`、`dsh-mgr.version`、`dsh-mgr.role=dsh`
- 资源:`--memory=2g --cpus=2`(可配);`--tmpfs /tmp`;只读 rootfs + 必要写入挂载
- **操作**:start/stop/restart/remove/logs/inspect/exec,全走 SDK。
- **2 实例上限**:按逻辑实例在 DB 事务中预留名额,等待构建/创建中的实例也占名额;失败清理后释放。Docker labels 用于对账,不作为并发名额控制。
- **对账**:面板启动时按 label 枚举容器与 DB 对账(孤儿容器回收或纳管)。
- **sock 安全**:面板挂裸 sock = 等效宿主 root。缓解:
1. **socket-proxy**(推荐):只放行白名单 API(containers/exec/images/networks/volumes/logs/build),关 plugins/daemon/swarm。面板连 proxy 而非裸 sock。
2. 面板自身非 root、只读 rootfs + tmpfs /tmp、资源限制。dsh 容器不 `--privileged`、去 cap。
--- ## 7. 网络与访问方式(当前实现)
## 7. 网络与访问方式 默认已经选择路径前缀代理,不是待拍板选项。面板将 /dsh/<id>/ 转发至对应实例网络中的 runtime:8080;归属检查在代理之前。Compose 默认仅向 loopback 发布面板8088,公网需管理员另配 HTTPS 入口与可信代理。子域名、直接发布 runtime 端口不是当前默认能力。
- 面板**启动时 ensure** `dsh-net`(bridge)存在,并把自身加入(compose 声明或运行时 `network connect`)。 ## 8. 访问与认证(当前实现)
- dsh 容器挂 `dsh-net`,**不发布主机端口**。
- 面板反代 `/dsh/<instId>/` → `http://<container-name>.dsh-net:8080`,容器名 = instId。
- **访问方式(待定,见 §18)**:默认推荐"路径前缀反代 + token 桥接"(§8),面板天然为 ingress。备选:子域名(需泛 DNS+TLS)/ 直发端口(MVP,隔离弱)。
--- 面板校验账号和实例归属,从启动日志取得 token 并在服务端交换/缓存 DSH cookie;token 按容器身份/启动时间加密持久化。反代剥前缀、处理 Host 与转发头,注入服务端 cookie 并移除上游 Set-Cookie,不把 DSH cookie 改写下发给浏览器。客户端持有的是面板认证 cookie。
## 8. 访问与认证(token 桥接 + cookie 改写) WS/SSE 由反代支持;会话续期由面板/runtime 注入脚本协调。上游拒绝 cookie 时失效缓存,下次请求重新交换,不重放写请求。已建立 WS/SSE 不承诺即时撤销;真实部署的长会话、插件与公开入口仍需分别验证。
> 此为最复杂点。推荐方案:路径前缀反代 + token 桥接。
- 面板路由 `/dsh/<instId>/*` → 反代到 `http://<instId>.dsh-net:8080`,**剥掉 `/dsh/<instId>` 前缀**,**保留浏览器 Host**,加 `X-Forwarded-*`。
- **token 桥接**:容器启动后,面板从 `docker logs` 流式抓 dsh 打印的 `http://127.0.0.1:3080/?token=xxx` 里的 token;用户首次打开 `/dsh/<instId>/` 时,面板服务端用该 token 打一次容器换 cookie,再把 `Set-Cookie` 的 `Path=/` 改写为 `Path=/dsh/<instId>`、加 `Secure`/`HttpOnly`/`SameSite=Lax` 下发给浏览器。后续请求带该 cookie 透传。
- **鉴权**:面板先校验用户登录 + 该 instId 归属当前 uid,再放行反代。dsh 自身 token cookie 为第二层。
- **WebSocket/SSE**:反代需支持 WS 升级;cookie 改写后需实测 WS 握手是否正常(关键风险)。
## 9. 工作区文件管理 + 容量配额 ## 9. 工作区文件管理 + 容量配额
**文件 API**(path 校验:解析后必须在 `files/` 内,拒 `..`/绝对路径): **文件 API**(除 usage 外必须提供 instance 并校验归属;path 限于该实例 home,拒穿越、绝对路径和不安全符号链接):
- `GET /api/ws/tree?path=` 列目录(带 size/mtime) - `GET /api/ws/tree?path=` 列目录(带 size/mtime)
- `POST /api/ws/mkdir`、`POST /api/ws/rename`、`DELETE /api/ws/delete` - `POST /api/ws/mkdir`、`POST /api/ws/rename`、`DELETE /api/ws/delete`
- `POST /api/ws/upload`(分片/流式,**写前查配额**) - `POST /api/ws/upload`(有界上传、写前查配额;不是分片续传协议)
- `GET/PUT /api/ws/text`(文本读取/带版本冲突检查的保存)、`GET /api/ws/image`(受控图片读取)
- `GET /api/ws/download?path=`(单文件流;目录自动 zip) - `GET /api/ws/download?path=`(单文件流;目录自动 zip)
- `GET /api/ws/usage`(当前用量/配额) - `GET /api/ws/usage`(当前用量/配额)
**配额实现**(容器化面板的约束): **配额实现**(容器化面板的约束):
- **默认:软配额**(无特权,随处可跑)。面板写前 `du` 校验 + 周期巡检 + 写入并发限流。并发大文件可能短暂超限。
- **默认:软配额**。面板写前遍历统计账号目录用量,写入串行化并限制并发,每五分钟巡检记录超限。容器/agent直接写盘不受面板写锁约束,仍可超过软配额;不通过删用户数据恢复容量。
- **可选硬配额**(二选一,需运维确认): - **可选硬配额**(二选一,需运维确认):
- **XFS prjquota**:面板 `--privileged` + `cap_add:[SYS_ADMIN,CHOWN]` + 把宿主 XFS 挂载点 bind 进来,面板内对 `/data` 跑 `xfs_quota`/`chattr -p`。每用户唯一 project ID(uid+1000)。 - **XFS prjquota**:面板 `--privileged` + `cap_add:[SYS_ADMIN,CHOWN]` + 把宿主 XFS 挂载点 bind 进来,面板内对 `/data` 跑 `xfs_quota`/`chattr -p`。每用户唯一 project ID(uid+1000)。
- **ZFS refquota**(若宿主用 ZFS):面板 `--privileged` + zfsutils,每用户 `zfs create pool/users/<uid> && zfs set refquota=10G`,再 bind 该 dataset 到 `/data/workspaces/<uid>`。 - **ZFS refquota**(若宿主用 ZFS):面板 `--privileged` + zfsutils,每用户 `zfs create pool/users/<uid> && zfs set refquota=10G`,再 bind 该 dataset 到 `/data/workspaces/<uid>`。
- 或在宿主跑**极小特权配额 sidecar**(gRPC/HTTP),面板远程调它做 quota 操作——面板自身保持非特权。 - 或在宿主跑**极小特权配额 sidecar**(gRPC/HTTP),面板远程调它做 quota 操作——面板自身保持非特权。
- 配额值存 DB,按用户可配(默认 10GiB)。面板展示用量。 - 配额值存 DB,按用户可配(默认 10GiB)。面板展示用量。
> 具体选型见 §18 待拍板。 > 当前仅软配额;上述 XFS/ZFS 和特权 sidecar 是未实现规划,不是可直接启用的配置。
--- ---
@@ -244,21 +207,21 @@ Go 端用 `github.com/docker/docker/client`:
| `dsh-ssh.json` | SSH 凭据(0600,加密入包) | | `dsh-ssh.json` | SSH 凭据(0600,加密入包) |
| `manifest.json` | dsh 版本、备份 schema 版、时间戳 | | `manifest.json` | dsh 版本、备份 schema 版、时间戳 |
**排除**:`node_modules/`(pnpm 重建)、`worktrees/`、`sessions/`、日志。 **范围**:仅上述六个固定配置文件、四类资源树和生成的manifest;不是完整home备份,通常位于白名单外的工作区、会话和插件node_modules不会入包。资源树按路径递归纳入普通文件,不根据文件内容识别运行态或凭据;不安全路径、符号链接及特殊文件会被拒绝。
**备份包加密**:用用户密码派生密钥(Argon2id → AES-GCM)加密整包;密钥不落库。下载/上传均加密包。 **备份包加密**:用用户密码派生密钥(Argon2id → AES-GCM)加密整包;密钥不落库。下载/上传均加密包。
**备份流程**:停容器(或仅停写)→ tar 指定文件 → 加密 → 落 `backups/<uid>/<instId>/` + 索引入库 → 重启容器。支持立即下载。 **备份流程**:按受控流程停止原本运行的容器,打包白名单并加密落盘,再恢复原运行状态;支持下载。不是任意运行中文件的一致性全量快照。
**还原流程**:校验+解密包 → 停容器 → 备份当前 home 为 `.pre-restore.bak` → 覆盖配置文件(不动 node_modules)→ **按 `package.json` 重建插件**(`dsh plugin --profile web install` 或 entrypoint 首启时 pnpm reconcile)→ 启容器 → 健康检查 → 失败则回滚 `.pre-restore.bak`。 **还原流程**:校验并解密包,受控停机、保存home恢复快照,应用配置与协调插件,启动验证后恢复应有运行状态;失败尝试快照回滚,失败恢复可能需人工介入。实际快照命名和恢复记录以config.go及运维文档为准,不提供任意第三方插件兼容保证。
--- ---
## 11. profile 文件编辑 ## 11. profile 文件编辑
- 编辑对象:`profiles/web/cordis.patch.yml`(YAML)。 - 编辑对象:`profiles/web/cordis.patch.yml`(YAML)。
- 前端用 Monaco 编辑器;后端 `GET` 返回原文、`POST` 校验(YAML 合法 + 可选 `--dump-config-schema` 比对)后原子写(临时文件 + rename,`dsh-atomic-write` 风格)。 - 前端用本地 Monaco;后端 `GET` 返回原文,`PUT` 校验 YAML 后原子写入,专用编辑器修改前需停止实例。未实现的 schema 比对不作为已交付能力。
- 保存后**重启容器**使生效(dsh 有 `dsh-hmr`,但配置类变更稳妥起见重启;后续可探热更)。 - 页面“保存并启动”发送 `restart:true`,写入后启动并检查就绪;API省略该参数时仅保存,不自动启动。
- 只读展示 `dsh.profile`(bundle 清单)与 `--dump-config` 预览(面板 `docker exec` 跑 `dsh --profile web --dump-config` 取回渲染)。 - 只读展示 `dsh.profile`(bundle 清单)与 `--dump-config` 预览(面板 `docker exec` 跑 `dsh --profile web --dump-config` 取回渲染)。
- 专用配置预览保持脱敏;完整 home 文件管理允许实例所有者读取 `.credentials.yaml` 原文,不承诺 write-only。 - 专用配置预览保持脱敏;完整 home 文件管理允许实例所有者读取 `.credentials.yaml` 原文,不承诺 write-only。
@@ -267,23 +230,23 @@ Go 端用 `github.com/docker/docker/client`:
## 12. 升级流程(保留数据 + 重启新版容器) ## 12. 升级流程(保留数据 + 重启新版容器)
``` ```
1. 用户选择本地可用目标版本;若需最新版,先手动触发 §5 构建并等待验证成功。锁定本地 image ID 后才进入备份/停机流程,不自动 pull 成品镜像 1. 用户选择本地可用目标版本;若无目标镜像,先由管理员完成 §5 Git 发布构建。锁定已验证且准入的 image ID 后才进入备份/停机流程
2. 自动备份当前配置(§10)→ 标记 pre-upgrade 备份 2. 停止原本运行的旧容器并检查确实已停机(不 remove,保留作回滚)
3. docker stop <old>(不 remove,保留作回滚) 3. 创建有界home快照,再用当前登录密码创建 pre-upgrade 配置备份
4. docker create <new>:同名 instId、同 binds(home + files)、同 labels、新 image、 4. 保留旧容器回滚依据,创建替换容器:同逻辑实例、同home挂载、新image,不挂共享files
Cmd 按新版本要求重算 --public-url/--trusted-host Cmd 使用 web/no-open/port/trusted-host,不传 --public-url
5. entrypoint 首启:若 profiles/web/node_modules 缺失/与 package.json 不一致 → pnpm install 重建插件 5. 启动新容器;发布镜像入口负责插件协调,其离线能力须针对精确镜像验证
6. docker start <new> → 面板抓 token、健康检查(轮询 caddy :8080) 6. 面板抓 token 并验证就绪
7. 成功:docker rm <old>;失败:stop <new>、start <old>、标记回滚、提示用户 7. 成功删除旧容器;失败尝试清理新容器、恢复home快照及旧容器原运行状态,回滚失败保留恢复依据并提示人工处理
``` ```
关键:**bind 不变 ⇒ 数据零丢失**;`node_modules` 在新镜像里按 `package.json` 重建(跨版本兼容性由 pnpm/dsh-plugin-manager 保证)。 历史设计意图:保持 bind 并重建插件依赖以尽量保留数据;**这不保证零丢失或任意插件跨版本兼容**。当前升级还依赖备份、快照、健康检查和回滚,故障及兼容性必须实测。
--- ---
## 13. 安全与资源 ## 13. 安全与资源
- **docker.sock**:面板容器能访问 sock 即等效宿主 root;socket-proxy 白名单 + 面板非 root + 只读 rootfs 缓解。 - **docker.sock**:面板容器能访问 sock 即等效宿主 root。当前非root进程和只读rootfs不能消除该权限;socket-proxy白名单只是尚未实施的可选增强。
- dsh 容器:**不 `--privileged`**、去 CAP_NET_RAW 等、只读 rootfs、tmpfs /tmp、mem/cpu 限制、不挂 sock。 - dsh 容器:**不 `--privileged`**、去 CAP_NET_RAW 等、只读 rootfs、tmpfs /tmp、mem/cpu 限制、不挂 sock。
- 面板账号:bcrypt + JWT(短期 access + refresh),失败锁定。 - 面板账号:bcrypt + JWT(短期 access + refresh),失败锁定。
- 配置备份包加密;实例所有者通过文件 API 下载的凭据或目录 ZIP 不属于加密配置备份,应按机密数据保管。 - 配置备份包加密;实例所有者通过文件 API 下载的凭据或目录 ZIP 不属于加密配置备份,应按机密数据保管。
@@ -292,118 +255,53 @@ Go 端用 `github.com/docker/docker/client`:
--- ---
## 14. 技术栈与项目结构 ## 14. 技术栈与项目结构(当前实现)
- **后端**:Go 1.22+,`net/http`(或 chi/gin),`github.com/docker/docker/client`,`github.com/docker/go-connections`,sqlite(`modernc.org/sqlite` 纯 Go 驱动,免 CGO),Argon2id、AES-GCM。 当前 Go/React 栈和实际目录见文首基线:Go 模块版本1.25、net/http、自有 Docker HTTP client、modernc SQLite;React 19、TypeScript、Vite、Tailwind v4、Radix 原语、Monaco。不是 Flutter 客户端,也不是独立 auth/workspace/proxy/upgrade 包的原设计拆分;多数编排位于 internal/api。
- **前端**:Vite + React + TypeScript,Monaco(profile 编辑),shadcn/ui。
``` 旧设想中的 main/image/、deploy/ 不能用于当前构建或部署:runtime 资产在 dockerfile 发布分支,面板使用根目录 Dockerfile 和 compose.yaml。
dsh-mgr/
cmd/server/ # Go 入口
internal/
auth/ # 账号/JWT
docker/ # 容器编排(镜像、生命周期、对账、路径翻译)
workspace/ # 文件 API、配额(软 du + 可选 xfs/zfs)
proxy/ # /dsh/<id> 反代 + token 桥接 + cookie 改写
backup/ # 备份/还原/加密
upgrade/ # 升级编排
profile/ # cordis.patch.yml 编辑
store/ # sqlite schema/查询
api/ # HTTP handlers
web/ # SPA
image/ # 自建 dsh-web 镜像(Dockerfile + Caddyfile + entrypoint.sh + caddy 二进制)
deploy/ # 面板自身 docker-compose / socket-proxy / systemd
```
--- ---
## 15. 数据模型(sqlite 概要) ## 15. 数据模型(当前实现概要)
- `users(id, username, pass_hash, quota_bytes, created_at)` SQLite基础表users、sessions、instances、audit由internal/store初始化;认证轮换、构建/镜像准入、发布身份、runtime创建/网络恢复和升级恢复等表由相应API模块维护。实际字段与迁移以源码为准,不将早期设计的示意DDL当作数据库接口。
- `instances(id, uid, name, dsh_version, image_id, build_id, status, container_id, created_at, ...)`
- `backups(id, uid, inst_id, path, size, schema_ver, created_at, kind=auto|manual|pre-upgrade|pre-restore)` image_builds包含任务、验证及发布来源信息;image_admissions绑定精确image ID/version/recipe/platform及验收依据;runtime_creates/runtime_networks保留资源恢复依据。旧表或文件名存在不代表旧预约服务仍运行。
- `images(image_id, version, recipe_rev, platform, tag, status, built_at)`(本地镜像索引,status=validating|available|failed|missing;与 Docker 实际状态对账)
- `image_builds(id, requested_by, version, recipe_rev, platform, status, image_id, error, created_at, finished_at)`(持久化构建任务;同版本+模板修订+架构的活动任务去重)
- `audit(id, uid, action, target, detail, at)`
--- ---
## 16. 面板自身部署(compose 示例) ## 16. 面板自身部署(当前实现)
```yaml 以根目录 compose.yaml、.env.example 和 OPERATIONS.md 为操作入口,不复制历史80/443或共享bridge样例。默认面板绑定127.0.0.1:8088,运行用户1000:1000,附加宿主socket组;只读rootfs、/tmp tmpfs、1GiB内存、2 CPU、512 PID及健康检查。
services:
dsh-mgr:
image: dsh-mgr:latest
restart: unless-stopped
ports:
- "80:80" # 前置 TLS 反代或直接 443
user: "65532:65532" # 非 root
environment:
HOST_DATA_ROOT: /var/lib/dsh-mgr
CONTAINER_DATA_ROOT: /data
PANEL_PUBLIC_URL: https://dsh.example.com
DSH_NET_NAME: dsh-net
DEFAULT_QUOTA_BYTES: 10737418240
DOCKER_HOST: unix:///var/run/docker.sock # 或指向 socket-proxy
volumes:
- /var/lib/dsh-mgr:/data
- /var/run/docker.sock:/var/run/docker.sock:ro # 或 socket-proxy
networks: [dsh-net]
read_only: true
tmpfs: [/tmp]
deploy: { resources: { limits: { memory: 1g } } }
# 硬配额(XFS)可选: privileged: true; cap_add: [SYS_ADMIN,CHOWN]; volumes: [xfs mount]
# 可选 socket-proxy 配置 HOST_DATA_ROOT / CONTAINER_DATA_ROOT、DockerRootDir只读容量探测挂载、PANEL_CONTAINER_ID,以及实际公开URL和可信代理CIDR。裸docker.sock挂载并未被只读rootfs降权,仍需信任面板与账号。当前模板不部署socket-proxy或硬配额sidecar。
socket-proxy:
image: tecnativa/docker-socket-proxy
environment:
CONTAINERS: 1; EXEC: 1; IMAGES: 1; NETWORKS: 1; VOLUMES: 1; LOGS: 1; BUILD: 1
POST: 1
volumes: [/var/run/docker.sock:/var/run/docker.sock:ro]
networks: [dsh-net]
restart: unless-stopped
networks:
dsh-net: { driver: bridge }
```
--- ---
## 17. 关键风险与待验证 ## 17. 关键风险与待验证
1. **token/cookie 桥接**(§8)最脆——需实测 dsh token 换 cookie 在反代+前缀下的行为,确认 cookie 改写后 **WS/SSE 正常**(dsh web 可能有 WebSocket)。 1. **token/cookie 桥接**(§8):需在实际入口验证服务端cookie交换与注入、前缀代理及WS/SSE,不以本地模拟替代部署验收。
2. **插件首启重建**:跨 dsh 版本 `pnpm install` 在 entrypoint 无网时失败——可能需升级时由面板 `docker exec` 联网重建后再切流量。 2. **插件首启重建**:跨 dsh 版本 `pnpm install` 在 entrypoint 无网时失败——可能需升级时由面板 `docker exec` 联网重建后再切流量。
3. **XFS/ZFS 硬配额**依赖宿主 FS 与特权/agent;否则退化为软配额。 3. **XFS/ZFS 硬配额**依赖宿主 FS 与特权/agent;否则退化为软配额。
4. **dsh agent 工作目录**:`dsh-fs-local`/`dsh-fs-sandbox` 的根是否等于 cwd——需确认 `/workspace` 作为 cwd 即可让 agent 在工作区操作,或需额外配置 fs root。 4. **dsh agent 工作目录**:`dsh-fs-local`/`dsh-fs-sandbox` 的根是否等于 cwd——需确认 `/workspace` 作为 cwd 即可让 agent 在工作区操作,或需额外配置 fs root。
5. dsh 版本仍为 `0.2.0-rc.*`(RC),API/flag 可能变动——需维护版本兼容矩阵。 5. dsh 版本仍为 `0.2.0-rc.*`(RC),API/flag 可能变动——需维护版本兼容矩阵。
6. **sibling bind 路径翻译**:`toHostPath` 出错会导致 bind 挂错目录或挂空目录,需单测覆盖。 6. **sibling bind 路径翻译**:`toHostPath` 出错会导致 bind 挂错目录或挂空目录,需单测覆盖。
7. **本地镜像构建**:验证 npm 查询失败仍可选本地版本、并发构建去重、失败释放实例名额、面板重启对账、镜像被外部删除,以及构建磁盘不足等路径;构建最新版不保证兼容,必须通过启动验证。 7. **Git 发布构建**:须验证精确 commit 确认与分支变化、信任源/权限变化、构建去重及恢复、外部删除镜像、容量及网络故障。构建成功不等于完整兼容验收,不存在旧构建预约释放名额流程。
--- ---
## 18. 待拍板决策点(open) ## 18. 已决策与后续规划
**已确认:镜像仅保存在本机;面板支持查询并手动构建最新版;创建实例可选本地已有版本,或构建最新版后创建。不需要 registry/CI 镜像分发。** 已选择每实例独立bridge、路径前缀反代、服务端cookie桥接、账号软配额、React响应式Web及管理员Git发布本地镜像。旧npm最新构建/预约、共享工作区和Flutter发布不是当前路线。
> 以下访问方式和配额选型仍待确认;此前审查发现的隔离、文件安全和升级回滚问题仍需另行修订。 可选后续增强:XFS/ZFS硬配额或宿主配额sidecar、socket-proxy、独立域名隔离。上述未纳入当前默认交付;是否实施需另行设计、授权和验收,不能以文档列出视为支持。
1. **访问方式**:路径前缀反代 + token 桥接(推荐,面板天然 ingress)/子域名(需泛 DNS+TLS)/直发端口(MVP,隔离弱)。 ## 19. 里程碑与交付判定
- **推荐默认**:路径前缀反代。
2. **配额**:宿主 FS 是 XFS / ZFS / 其他?是否接受"特权面板"或"宿主配额 sidecar"做硬配额?还是软配额默认即可?
- **推荐默认**:软配额(du)起步,硬配额作为后续增强;若宿主为 XFS/ZFS 且可接受特权,则上硬配额。
--- M1镜像、M2账号/生命周期、M3代理、M4文件/软配额、M5配置备份、M6升级回滚、M7界面/管理安全保留为能力分组,不再以旧npm构建及预约作为当前M2标准。当前Git发布、实例home独占、删除清理、文本编辑/图片预览与主题等变更须按ACCEPTANCE.md当前矩阵分别补证据。
## 19. 里程碑 历史M1–M7结案不等于当前commit或真实部署整体通过;本地测试通过、真实Docker通过、浏览器通过与公网验收必须分开记录。
1. **M1 镜像 + 单容器手动跑通**:自建本地 `dsh-web` 镜像,验证精确版本、caddy+dsh 启动与就绪。
2. **M2 面板骨架**:账号/鉴权 + npm 最新版查询 + 本地镜像列表/手动构建任务 + 创建时选本地版本或构建最新版 + docker 编排(创建/启停/删除/2 上限)+ 对账 + 路径翻译。
3. **M3 反代 + token 桥接**:`/dsh/<id>/` 可用(含 WS)。
4. **M4 工作区文件 API + 软配额**。
5. **M5 备份/还原 + profile 编辑**。
6. **M6 升级流程 + 回滚**。
7. **M7 SPA 完善 + 审计 + 资源限制 + 可选硬配额**。
--- ---
+1 -3
View File
@@ -11,15 +11,13 @@ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
COPY cmd/ ./cmd/ COPY cmd/ ./cmd/
COPY internal/ ./internal/ COPY internal/ ./internal/
COPY image/ ./image/
RUN CGO_ENABLED=0 go build -trimpath -o /out/dsh-mgr ./cmd/server RUN CGO_ENABLED=0 go build -trimpath -o /out/dsh-mgr ./cmd/server
FROM debian:bookworm-slim FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates && rm -rf /var/lib/apt/lists/* RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates git && rm -rf /var/lib/apt/lists/*
WORKDIR /app WORKDIR /app
COPY --from=backend /out/dsh-mgr /usr/local/bin/dsh-mgr COPY --from=backend /out/dsh-mgr /usr/local/bin/dsh-mgr
COPY --from=web /src/web/dist ./web/dist COPY --from=web /src/web/dist ./web/dist
COPY image/ ./image/
USER 1000:1000 USER 1000:1000
EXPOSE 8088 EXPOSE 8088
HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 CMD ["/usr/local/bin/dsh-mgr", "-healthcheck"] HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 CMD ["/usr/local/bin/dsh-mgr", "-healthcheck"]
+78 -10
View File
@@ -1,12 +1,14 @@
# 本机或可信团队试运行运维 # 本机或可信团队试运行运维
当前仅支持模板 7 的独立实例布局:实例 home 挂载到 /workspace,不再创建或使用 workspaces/<uid>/files,也不再迁移旧共享挂载布局。历史 files 目录可在确认旧容器已停止并移除挂载后,由管理员在维护窗口删除;面板不会在启动或查询用量时自动递归删除历史数据。旧版本已经删除实例所留下的目录也需核对数据库及容器引用后单独清理。
本版本面向本机或受信任团队,不应直接用于公网不可信租户。DSH 与面板使用同源路径代理;Docker socket 权限等价于宿主 root。 本版本面向本机或受信任团队,不应直接用于公网不可信租户。DSH 与面板使用同源路径代理;Docker socket 权限等价于宿主 root。
## 启动 ## 启动
1. 复制 `.env.example` 为 `.env`,填写专用绝对目录 `DSH_HOST_DATA_ROOT` 和 12–72 字节的随机管理员密码。限制 `.env` 的读取权限,不要提交。管理员账号创建后该密码不再被读取,应从 `.env` 中移除,再运行 `docker compose up -d --force-recreate panel` 重建面板容器,清除容器环境变量中保留的明文密码(`docker inspect` 可见)。仅修改 `.env` 或重启原容器不会更新其环境变量;重建面板会短暂中断面板访问。 1. 复制 `.env.example` 为 `.env`,填写专用绝对目录 `DSH_HOST_DATA_ROOT` 和 12–72 字节的随机管理员密码。限制 `.env` 的读取权限,不要提交。管理员账号创建后该密码不再被读取,应从 `.env` 中移除,再运行 `docker compose up -d --force-recreate panel` 重建面板容器,清除容器环境变量中保留的明文密码(`docker inspect` 可见)。仅修改 `.env` 或重启原容器不会更新其环境变量;重建面板会短暂中断面板访问。
2. 面板以 UID/GID 1000 运行。首次部署前由管理员创建专用数据目录并赋予 1000:1000 所有权、0700 权限;不要对已有未知目录递归改权限。将 `stat -c %g /var/run/docker.sock` 的结果填入 `.env` 的 `DOCKER_GID`,以附加组访问 socket。然后运行 `docker compose up --build -d`。默认访问 `http://127.0.0.1:8088`;远程管理优先使用 SSH 本地端口转发,保留此访问地址。Compose 定义了 `dsh-mgr -healthcheck` 健康检查(探测本机 `/healthz`);进程无响应时容器会被标记为 unhealthy,配合 `restart: unless-stopped` 在退出时自动拉起。 2. 面板以 UID/GID 1000 运行。首次部署前由管理员创建专用数据目录并赋予 1000:1000 所有权、0700 权限;不要对已有未知目录递归改权限。将 `stat -c %g /var/run/docker.sock` 的结果填入 `.env` 的 `DOCKER_GID`,以附加组访问 socket。然后运行 `docker compose up --build -d`。默认访问 `http://127.0.0.1:8088`;远程管理优先使用 SSH 本地端口转发,保留此访问地址。Compose 定义了 `dsh-mgr -healthcheck` 健康检查(探测本机 `/healthz`);连续探测失败时容器会被标记为 unhealthy,但该状态本身不会触发重启。`restart: unless-stopped` 用于进程退出后的自动拉起;`/healthz` 仅检查HTTP存活,不检查数据库、Docker或实例健康。
3. 登录后在镜像页提交精确 DSH 版本或查询最新版再手动构建。已有本地镜像会复用,但必须通过启动验证才允许创建实例。镜像不推送至仓库。 3. 管理员登录后在镜像页检查 Git 发布并确认精确 commit 后构建;普通用户选择已验证且准入的本地镜像创建实例。没有可选镜像时等待管理员构建,不再按 npm 版本或预约构建后创建。镜像不推送至仓库。
4. 创建实例后从面板进入 DSH。不要给 runtime 发布宿主端口或挂载 Docker socket。 4. 创建实例后从面板进入 DSH。不要给 runtime 发布宿主端口或挂载 Docker socket。
首次启动创建管理员。修改环境变量不会重设已有账号密码。容器内 `/data` 必须对应 `HOST_DATA_ROOT` 指向的同一宿主目录,否则 sibling 容器会挂载错误数据。 首次启动创建管理员。修改环境变量不会重设已有账号密码。容器内 `/data` 必须对应 `HOST_DATA_ROOT` 指向的同一宿主目录,否则 sibling 容器会挂载错误数据。
@@ -15,9 +17,9 @@
## 公网 HTTPS 入口(仅可信账号) ## 公网 HTTPS 入口(仅可信账号)
反向代理与证书由部署管理员管理,面板仍建议只监听代理可达的私有入口。设置 `PANEL_PUBLIC_URL=https://你的域名`(仅 origin,不含业务子路径);Compose 默认值仍为本机 HTTP。此值决定 Cookie 的 Secure 属性及 Origin/CSRF 校验,不从客户端的 X-Forwarded-Proto/Host 推断。 反向代理与证书由部署管理员管理,面板仍建议只监听代理可达的私有入口。设置 `PANEL_PUBLIC_URL=https://你的域名`(仅 origin,不含业务子路径);Compose 默认值仍为本机 HTTP。此值用于 Origin/CSRF 校验,HTTPS值会启用 Cookie 的 Secure 属性(直接TLS请求也会启用),不从客户端的 X-Forwarded-Proto/Host 推断。
默认只按直连来源 IP 限流,不信任 X-Forwarded-For。经反代部署时,设置 `TRUSTED_PROXY_CIDRS` 为面板实际看到的代理来源地址/CIDR,且确保只有这些代理能连接后端、代理正确维护转发链;不要填写全网 CIDR。未设置时,代理后所有用户将共享该代理来源的每分钟登录额度。公网开放前应实际验证不同客户端来源识别、登录/刷新/退出、DSH iframe、WS/SSE及插件设置接口。 默认只按直连来源 IP 限流,不信任 X-Forwarded-For。经反代部署时,设置 `TRUSTED_PROXY_CIDRS` 为面板实际看到的、逗号分隔的代理来源CIDR(单个IPv4/IPv6地址须写成 `/32` 或 `/128`),且确保只有这些代理能连接后端、代理正确维护转发链;不要填写全网 CIDR。未设置时,代理后所有用户将共享该代理来源的每分钟登录额度。公网开放前应实际验证不同客户端来源识别、登录/刷新/退出、DSH iframe、WS/SSE及插件设置接口。
账号由管理员创建,不支持不可信租户。软配额、同源插件与 Docker socket 权限边界不因 HTTPS 而改变;本版本不承诺抵御分布式拒绝服务。 账号由管理员创建,不支持不可信租户。软配额、同源插件与 Docker socket 权限边界不因 HTTPS 而改变;本版本不承诺抵御分布式拒绝服务。
@@ -35,11 +37,11 @@
### 镜像兼容验收与准入 ### 镜像兼容验收与准入
管理员实验构建只负责生成并执行基础验证,不代表上线批准。准入前应在独立测试环境核验精确镜像的启动、认证/代理、终端与流式连接、文件读写、插件协调、配置备份恢复,以及所需升级/回滚路径;记录版本、模板、平台、镜像ID、测试范围和证据位置。不要在验收依据中粘贴凭据,也不要仅填写“构建成功”。本次实现不自动启动待验收镜像供普通用户使用,不自动运行真实部署测试。 管理员 Git 发布构建会执行基础验证,但不能代替完整上线验收。准入前应在独立测试环境核验精确镜像的启动、认证/代理、终端与流式连接、文件读写、插件协调、配置备份恢复,以及所需升级/回滚路径;记录版本、模板、平台、镜像ID、测试范围和证据位置。不要在验收依据中粘贴凭据,也不要仅填写“构建成功”。本次实现不自动启动待验收镜像供普通用户使用,不自动运行真实部署测试。
在镜像页“准入管理”选择推荐或兼容并填写依据,后端还会检查精确镜像身份、成功验证记录和可支持的挂载布局。审批只作用于该镜像;新镜像需要自己的验证与审批。退为仅存量后不允许新建/升级目标;标为禁止使用还阻止新的启动/重启及受控配置修改、备份/恢复写操作,避免其自动重启绕过禁用。已有配置读取和通用home文件管理仍保留,不会自动停止正在运行的实例。若需要立即隔离已运行实例,管理员必须另行确认并执行停止操作。 在镜像页“准入管理”选择推荐或兼容并填写依据,后端还会检查精确镜像身份、成功验证记录和可支持的挂载布局。审批只作用于该镜像;新镜像需要自己的验证与审批。退为仅存量后不允许新建/升级目标;标为禁止使用还阻止新的启动/重启及受控配置修改、备份/恢复写操作,避免其自动重启绕过禁用。已有配置读取和通用home文件管理仍保留,不会自动停止正在运行的实例。若需要立即隔离已运行实例,管理员必须另行确认并执行停止操作。
历史home布局不同:模板7使用完整home作为 `/workspace`,受支持旧模板使用独立 `/data/dsh-home` 与用户共享 `/workspace`。不要把旧模板准入理解为数据布局可任意互换;不支持的反向迁移仍会拒绝,先做好完整备份。 当前准入仅支持模板7,使用完整home作为 `/workspace`;旧模板的独立 `/data/dsh-home` 与用户共享 `/workspace` 布局不再支持准入。升级也要求原容器只有实例home到 `/workspace` 的单一bind挂载,不会迁移旧共享布局;不符合时返回409。处理历史实例前先做好完整备份。
运行时认证材料由面板首次成功连接实例时提取,并使用 `auth-signing.key` 派生的密钥加密保存在数据根的 `runtime-auth/`(不在用户home内)。按容器 ID 和启动时间绑定,不会将旧启动周期凭据用于新容器;不同 Host 分别缓存 cookie。全量备份应连同签名密钥和该目录保存,不要向用户公开。 运行时认证材料由面板首次成功连接实例时提取,并使用 `auth-signing.key` 派生的密钥加密保存在数据根的 `runtime-auth/`(不在用户home内)。按容器 ID 和启动时间绑定,不会将旧启动周期凭据用于新容器;不同 Host 分别缓存 cookie。全量备份应连同签名密钥和该目录保存,不要向用户公开。
@@ -47,11 +49,43 @@
升级到本版本后应在维护窗口逐个访问现有实例,完成首次认证持久化。若在首次成功连接前,启动token已被Docker日志轮转删除,本版本无法凭空恢复;先保存工作,再由管理员安排重启相关实例并访问验证。不要为此删除或重建实例。初次日志扫描上限64 MiB,常规每文件10 MiB/3份保留在此范围内。 升级到本版本后应在维护窗口逐个访问现有实例,完成首次认证持久化。若在首次成功连接前,启动token已被Docker日志轮转删除,本版本无法凭空恢复;先保存工作,再由管理员安排重启相关实例并访问验证。不要为此删除或重建实例。初次日志扫描上限64 MiB,常规每文件10 MiB/3份保留在此范围内。
- 实例升级选择已经启动验证的本地镜像并提供当前登录密码,面板自动停机、创建密码配置备份与home快照、启动验证及失败回滚,无需先手动停止。home快照最大1 GiB;允许受约束的内部相对链接以支持pnpm,拒绝不安全链接/特殊文件。模板7的 `/workspace` 是实例home,包含在home回滚内;旧共享数据不回滚。跨DSH版本历史验证范围见 ACCEPTANCE.md。 - 实例升级选择已经启动验证且准入的本地模板7镜像并提供当前登录密码,面板自动停机、创建密码配置备份与home快照、启动验证及失败回滚,无需先手动停止。home快照最大1 GiB;软链接按链接本身保存,保留相对、绝对、失效及循环链接的目标文本,不读取或备份链接指向的外部内容;仍拒绝 socket、FIFO 等特殊文件。模板7的 `/workspace` 是实例home,包含在home回滚内;旧共享挂载布局会在升级前被拒绝。跨DSH版本历史验证范围见 ACCEPTANCE.md。
- 升级快照先复制到 `.upgrade-<ID>-snapshot-partial`,完成后才发布正式目录并记录完成状态。复制失败会清理临时目录;清理失败会报告并记录 `upgrade.snapshot.cleanup_failed`,管理员核实后处理。进程中断也可能留下临时目录,不会作为可恢复快照列出。已发布但未记录完成的快照会显示不可恢复。旧版残留没有可靠的完成记录,本版本不会自动判定其完整或自动删除,建议保留证据并重新生成快照。
- 完整快照恢复和下载同样保留软链接原文;失效链接不会自动修复,外部目标不属于恢复范围。下载的 tar.gz 可能包含绝对或目录外链接,应在隔离目录使用安全解压工具,不要跟随链接写入宿主机。独立的加密配置备份仍只接受白名单普通文件,配置路径和资源中的软链接仍会被拒绝。
- Profile是例外:当前需先停止实例,再使用“保存并启动”;页面包含Monaco和脱敏配置预览。不要把备份/升级自动停启理解为Profile可在线改写。 - Profile是例外:当前需先停止实例,再使用“保存并启动”;页面包含Monaco和脱敏配置预览。不要把备份/升级自动停启理解为Profile可在线改写。
- 若实例进入 `error`,不要直接修改数据库状态强制启动。保留原容器、实例目录与 `upgrade_journal` 记录,检查实际容器与 journal 的旧/新容器 ID、阶段、快照路径是否一致。 - 若实例进入 `error`,不要直接修改数据库状态强制启动。保留原容器、实例目录与 `upgrade_journal` 记录,检查实际容器与 journal 的旧/新容器 ID、阶段、快照路径是否一致。
- 人工恢复前停止相关容器并另做完整备份。中断点不同,不能统一用一次目录覆盖或状态修改处理;当前无自动灾难恢复命令。 - 人工恢复前停止相关容器并另做完整备份。中断点不同,不能统一用一次目录覆盖或状态修改处理;当前无自动灾难恢复命令。
- 删除实例会删除其容器与网络,但保留数据目录。升级快照暂无自动清理;不要删除仍被未完成 journal 引用的快照。 - 删除实例会删除其容器、网络、实例目录(含 home 和快照)及专属备份,无法撤销;请事先下载需要保留的数据。清理失败保留 error 实例,修复存储问题后重试删除。未完成升级恢复的实例仍禁止删除。
## Git 分支发布运行镜像构建配置
发布源默认为本仓库 `https://git.misaka.ren/m1saka/dsh-web.git` 的 `dockerfile` 分支,通过 `DSH_RELEASE_REPOSITORY` / `DSH_RELEASE_BRANCH` 配置。它发布的是构建文件,不是预构建镜像,也不是面板程序自身的更新。修改配置后需重新创建面板容器。
直接运行面板二进制时需要安装 `git`;面板 Dockerfile 已包含该依赖。当前通道仅支持无需认证的 HTTPS 仓库,不读取宿主 Git 凭据、不跟随重定向,也不支持 SSH 或 URL 内嵌令牌。私有仓库认证需另行扩展,不能把访问令牌写入仓库地址。
`release.json` 和完整的 `image/` 目录仅在 `dockerfile` 发布分支维护,main 不再保留或打包这些文件。发布分支根目录放 `release.json`;发布时一起提交 Dockerfile、Caddyfile、entrypoint 和全部安装/适配脚本。只把 Dockerfile 单独放到分支上是不完整的发布。不要把 `.env`、私钥、凭据、用户数据或 `.git` 放进构建目录;`.dockerignore` 只是辅助措施,发布者仍须审查内容。
Git 通道直接向 Docker API 发送构建上下文,不执行 Docker CLI 的 `.dockerignore` 过滤。面板会排除 `.git`、`.env` 系列和常见凭据文件,并拒绝链接、特殊文件及越界路径,但不能识别任意文件名下的秘密。构建归档最大 32 MiB;Git 获取有 45 秒超时及文件大小限制,建议发布分支仅保留发布所需文件。
`release.json` 示例:
```json
{
"schema_version": 1,
"release_id": "runtime-7.1",
"version": "0.2.0-rc.2",
"deployment_schema": 7,
"platforms": ["linux/amd64"]
}
```
`version` 是精确 DSH 版本;`release_id` 是构建配置发布编号,修改脚本或 Dockerfile 后即使 DSH 版本不变,也应使用新的编号。`deployment_schema` 声明运行布局兼容协议,当前 Git 发布通道支持协议 7:实例 home 是唯一bind挂载,目标为 `/workspace`,不挂载共享目录 `/shared`,代理端口为8080,并保留现有认证与启动约定。它不应随普通构建修订递增;不兼容的运行约定需要面板新增支持。`platforms` 只能声明发布者实际验证的平台。
首次发布时,在独立 checkout 或 worktree 准备 `dockerfile` 分支,将 `release.json` 和完整 `image/` 目录提交并推送到同一远程仓库。无需切换正在开发的主工作目录,也无需 CI。此功能的代码安装本身不会替你创建或推送远程分支;分支未发布前,检查失败是预期状态,旧实例不受影响,但不会回退到内置构建。这是 breaking change:移除 `/api/latest`、`/api/compatibility`、`POST /api/builds` 及 `/api/intents` 的全部查询、创建和取消接口,不提供 410 兼容入口。仅保留 `GET /api/builds` 和 `GET /api/builds/{id}` 构建历史读取。不迁移或恢复旧构建及旧创建意图,不在后台续跑或完成旧预约;这不意味着删除已有实例或数据。
管理员打开镜像页会检查 Git 发布,页面保持打开时每五分钟复查,也可手动刷新。确认构建后面板以固定 commit 获取文件;确认期间分支发生变化会拒绝并要求重新检查。构建任务记录发布编号、仓库和 commit,同一 DSH 版本的新构建发布不会误用旧版本构建任务。构建与验证成功后才可选择镜像;不会自动升级任何已有实例,升级仍走密码确认、备份、快照和失败回滚流程。
该分支是可执行代码信任边界:限制写权限,发布前审查 Dockerfile 与脚本,不向不可信仓库开放构建。可信发布只替代内置版本名单,不替代完整的上线验收;应在发布前验证启动、代理/认证、文件访问、插件以及升级恢复。管理员对精确镜像设置的禁用策略仍然有效。上游依赖、基础镜像和网络变化可能影响重建结果,commit 固定并不意味着所有外部依赖都已固定。Git 故障不会停止已有实例。
## 创建与构建恢复记录 ## 创建与构建恢复记录
@@ -82,6 +116,40 @@
bash scripts/check-trial.sh bash scripts/check-trial.sh
``` ```
该脚本执行本地自动化测试、race、静态检查及前后端构建,不启动容器、不安装依赖、不部署。需提前准备 Go/Node/npm 和前端依赖。可通过 `TEST_TMPDIR` 指定已存在的临时目录。镜像构建单测已隔离真实磁盘空闲量;生产磁盘检查仍保留。 该脚本执行本地自动化测试、race、静态检查及前后端构建,不启动容器、不安装依赖、不部署。需提前准备 Go/Node/npm 和前端依赖。可通过 `TEST_TMPDIR` 指定已存在的临时目录。面板构建单测已隔离真实磁盘空闲量;生产磁盘检查仍保留。`image/init-profile.test.mjs` 属于发布分支的 runtime 测试,不在 main 的检查脚本中运行。
`scripts/test-panel.mjs` 必须指向独立测试部署,通过 `TEST_PANEL_URL`、`TEST_USERNAME`、`TEST_PASSWORD` 指定访问参数;`TEST_UPGRADE=1` 包含同镜像替换升级。它会创建及删除测试实例、留下保留策略要求的数据,并可能构建或复用本地 runtime 镜像。不要对生产账号运行。 `scripts/test-panel.mjs` 必须指向独立测试部署,通过 `TEST_PANEL_URL`、`TEST_USERNAME`、`TEST_PASSWORD` 指定访问参数;`TEST_UPGRADE=1` 包含同镜像替换升级。运行前需由管理员通过 Git 发布构建准备已验证且准入的本地 runtime 镜像;可用 `TEST_FROM_VERSION` / `TEST_TO_VERSION` 选择源和目标版本。脚本不提交构建,只复用这些镜像,会创建及删除测试实例,删除时一并清理实例数据。不要对生产账号运行。`scripts/smoke-image.sh <image-id>` 和 `scripts/test-mobile-image.mjs <image-id>` 仅测试已有镜像的运行行为,不读取 main 的构建源;仅在隔离测试环境使用。旧构建脚本和构建后创建意图的 live 测试已删除。
## Office 文档预览超时
Office 文档预览的运行时接口 `/api/officeToPdf/render`(面板路径为 `/dsh/:id/api/officeToPdf/render`)使用独立代理连接池,响应头等待上限为120秒;其它运行时请求及认证仍保持原有20秒上限。此设置避免转换尚未完成就被面板提前截断为502,不修改DSH转换器自身的超时(当前镜像默认60秒),也不保证转换一定成功。浏览器取消请求仍会传递至上游。若面板前还有Nginx等代理,其等待上限也应允许转换完成。此修复只需更新面板,无需为此重建运行镜像。
## 手动备份空间管理
实例备份包和历史完整工作区快照必须由用户显式删除,不执行自动清理。快照列表与删除接口见 API.md。删除快照会降低工作区逻辑用量;加密包在独立 backups 目录,不计入当前工作区软配额。快照大小不等于物理磁盘占用,硬链接按路径重复计数;`size_error` 表示无法完整扫描,应显示未知而非零。
恢复失败、未完成升级及恢复日志不可读时,列表仍报告保护状态,但禁止删除包和快照。先完成恢复并核实运行状态和升级日志,不要为了释放空间直接修改日志状态。`home` 永远不属于历史快照管理对象。删除不需要停机或 Docker 可用,也不会触发重启。
文件删除与 SQLite 提交不具备跨系统原子性。若加密包已删除而索引提交失败,接口明确返回500;重试相同删除会清理残留索引。快照删除先持久化 `snapshot.delete.started`,失败则不触碰目录;递归删除期间不占用 SQL 连接或事务,仍保留实例锁和用户锁。随后分别记录 `snapshot.delete.completed` 或 `snapshot.delete.failed`。终态审计写入失败返回500;只有 started 的记录也可能表示进程中断或操作仍在进行,不代表删除成功,应由管理员核实目录和审计记录。目录删除中途遇到存储错误可能部分完成,排除故障后再重试。明确不安全路径返回400,权限或 I/O 故障返回500,缺失快照删除返回404(列表为空)。
## 完整快照下载与手动恢复
实例的备份面板提供完整快照下载和恢复入口。下载的是未加密 `.tar.gz`,不同于仅包含配置资源的加密 `.dshbak`;可能包含密钥、会话和依赖,请妥善保管。归档保留权限及安全的相对符号链接,不跟随链接读取外部文件,遇到特殊文件或逃逸链接整体拒绝。下载先在系统临时目录完成归档,再释放实例锁传输;临时文件在请求结束时清理,需预留临时磁盘空间。文件内容总量上限1 GiB、100000条目、128层,另有限额容纳归档头及压缩开销;归档和传输分别有两分钟时限,共用最多4个目录归档下载槽。
恢复针对服务器保留的快照,不支持上传任意 tar 包恢复。需当前登录密码和明确确认;源快照保留不动,先复制到暂存目录,再停机并保留当前 home,切换后启动检查,失败尝试回退。原先停止的实例检查后仍停止。恢复需要额外容纳一份所选快照的配额和磁盘空间;已有恢复快照达到3个时先清理。已知源镜像或版本与当前实例不一致时禁止恢复;历史快照缺少来源信息时须额外确认风险。新升级、配置恢复和完整快照恢复会记录来源信息。
error 状态、未完成升级、活跃文件操作、禁用镜像或不兼容挂载布局会阻止恢复。恢复中断或回退失败保留 error 状态及数据,需管理员结合 `snapshot.restore.*` 审计核查 home、pre-restore、failed-restore 和暂存目录;不要直接把状态改成 running。快照只恢复 home,不回滚镜像、外部服务或启动产生的外部副作用。
### 完整快照恢复安全说明
快照恢复会替换整个实例 home,必须由所有者重新输入当前登录密码并明确确认。仅恢复到相同的已知镜像与版本;历史快照没有可靠来源记录时必须显示未知版本风险并额外确认,不得把当前实例版本当作历史来源。旧升级日志的 image_id 是目标镜像,不是旧 home 的可靠来源。新快照的来源写入 SQLite snapshot_metadata 表,升级回滚留下的 failed-home 使用尝试升级的目标镜像来源。
源快照不会被消耗。操作将源复制到实例目录下的 .snapshot-restore-<32hex>,确认 Docker 已停止后,把原 home 重命名为 pre-restore-<32hex>,再以同文件系统重命名安装副本并 fsync 父目录。运行时启动并通过就绪验证后恢复原运行/停止状态。原来停止的实例也会临时启动验证,因此应把恢复视为可执行该快照内容的操作,仅恢复可信数据。
开始停止/切换前实例状态持久化为 error;中途进程退出不会自动清除此状态。错误恢复时先停止容器,核对 snapshot.restore.started 审计中的源、pre-restore、failed-restore ID,以及 home 和暂存目录。不要盲目删除或覆盖这些目录;先确认哪一个保存原 home,保留全部恢复证据与源快照。只有核对目录、挂载和运行时就绪后才解除恢复状态。无法停止、重命名、同步或回滚就绪时接口保持 error 并要求人工处理。
恢复前按源副本大小检查用户配额,停止运行时后再次核验总用量;原 home 通过重命名保留,不重复复制。已有 pre-restore / failed-restore 数量达到3时拒绝,需显式清理不再需要的快照。没有自动保留清理;进程异常退出的隐藏暂存副本须在维护窗口核验后清理。文件复制期间不持有数据库事务。此流程仍需部署环境提供同文件系统原子重命名及 fsync 持久性,不能替代异机备份。
## 会话有效期
在 `.env` 设置 `SESSION_TTL=24h`(默认),七天可设置为 `168h`。使用 Go duration 单位 `h`、`m`、`s`,不支持 `d`;必须至少一秒且为整秒,非法值会阻止启动。修改后运行 `docker compose up -d --force-recreate panel`。新登录使用新期限;已有会话保留原绝对到期时间,刷新不会延长。access token签发有效期为五分钟,但数据库会话过期或被撤销后,即使token尚未到期也不能认证。修改自己的密码或管理员重设密码会撤销该用户所有会话,需要重新登录。`.env` 由 Compose 读取,直接运行二进制时应在进程环境中导出 `SESSION_TTL`。
+31 -14
View File
@@ -8,17 +8,23 @@
登录后默认进入内嵌 DSH 工作台。在“设置”中选择当前账号的实例;选择仅保存在当前浏览器并按账号区分,不保存登录凭据。没有实例时先到“实例”创建,停止或异常的实例需先启动或处理。切换实例会重新加载内嵌页面,但不会停止原容器。 登录后默认进入内嵌 DSH 工作台。在“设置”中选择当前账号的实例;选择仅保存在当前浏览器并按账号区分,不保存登录凭据。没有实例时先到“实例”创建,停止或异常的实例需先启动或处理。切换实例会重新加载内嵌页面,但不会停止原容器。
移动端使用底部导航:DSH、实例、文件、设置;桌面保留侧栏。镜像管理和管理员功能从设置进入。切换面板栏目会保留内嵌 DSH 页面,提供重新加载及独立窗口入口。此为同源 iframe,不是安全隔离边界,仍仅适合本机或受信任团队。 移动端使用底部导航:DSH、实例、文件、设置;桌面保留侧栏,管理员可直接从侧栏进入镜像与用户管理,移动端这两个入口仍在“设置”中。切换面板栏目会保留内嵌 DSH 页面,重新加载与独立窗口入口收在工具栏的溢出菜单中。此为同源 iframe,不是安全隔离边界,仍仅适合本机或受信任团队。
## 当前实现 ## 当前实现
文件管理中的 PNG、JPEG、GIF、WebP、SVG 可通过“预览”按钮查看(最大 16 MiB),支持适应窗口、原始尺寸、缩放和下载。预览沿用实例归属校验,SVG 只作为图片加载,响应限制脚本与外部资源;直接打开预览地址会下载文件,不作为网页执行。
文件管理支持通过“编辑”按钮在线查看和保存不超过 1 MiB 的 UTF-8 文本,使用本地 Monaco 编辑器并跟随系统深浅色。二进制或超限文件请下载处理。关闭、重新加载及离开浏览器前会提醒未保存修改;保存失败保留草稿。保存按读取版本检查冲突,不提供强制覆盖;原子替换需要额外的临时文件配额。DSH 等外部写入不受面板锁控制,最终检查与替换之间仍有竞态,修改运行配置前建议停机并备份。通用编辑不执行专用 Profile 流程的 YAML 校验或强制停机。
DSH 新运行镜像仅由管理员从本仓库 `dockerfile` 分支获取独立发布的构建配置,无需 CI 预构建镜像。`release.json` 和完整的 `image/` 目录仅维护在发布分支,main 不再内置运行镜像构建资产;面板锁定 Git commit,在本地构建并验证。管理员在镜像页检查并确认构建,实例升级仍需另行确认,不会因分支更新自动停机。发布约定和首次启用步骤见 [OPERATIONS.md](OPERATIONS.md)。
实例所有者可管理完整 home,包括读取自己的配置和凭据;不承诺密钥 write-only,跨用户隔离保持不变。通用文件操作不受专用 Profile 编辑器的 YAML 校验和停机要求约束。下载、打包与敏感修改提示用于防误操作,不是权限限制;文件下载和目录 ZIP 不会自动加密,修改配置前建议先停止实例并备份。 实例所有者可管理完整 home,包括读取自己的配置和凭据;不承诺密钥 write-only,跨用户隔离保持不变。通用文件操作不受专用 Profile 编辑器的 YAML 校验和停机要求约束。下载、打包与敏感修改提示用于防误操作,不是权限限制;文件下载和目录 ZIP 不会自动加密,修改配置前建议先停止实例并备份。
- 内置账号、管理员配额、短JWT与refresh轮换、失败锁定及审计。 - 内置账号、管理员配额、短JWT与refresh轮换、失败锁定及审计。
- 精确版本查询、本地镜像异步构建/验证/清理、持久创建预约与事务名额(每用户最多两个实例)、启动对账。预约不再仅保存在当前页面。 - 管理员 Git 发布检查、本地镜像异步构建/验证/清理、使用已验证且准入的本地镜像创建实例与事务名额(每用户最多两个实例)、启动对账。
- 实例启停删除、独立home(模板7挂载为 `/workspace`)、每实例bridge、归属鉴权及路径反代;DSH认证cookie由服务端保管。 - 实例启停删除、独立home(模板7挂载为 `/workspace`)、每实例bridge、归属鉴权及路径反代;DSH认证cookie由服务端保管。
- 文件上传/下载/重命名/删除及目录ZIP;软配额写前检查、写入串行化与周期巡检。上传最大32 MiB且不覆盖已有文件,删除目录须先清空;删除实例保留数据。 - 文件上传/下载/重命名/删除及目录ZIP;软配额写前检查、写入串行化与周期巡检。上传最大32 MiB且不覆盖已有文件,删除目录须先清空;删除实例会永久删除其 home、快照及备份。
- 密码加密配置备份、资源目录与manifest、外部加密包导入、恢复健康检查及回滚。备份不再限于六个固定文件;不包含会话、工作区及node_modules。 - 密码加密配置备份、资源目录与manifest、外部加密包导入、恢复健康检查及回滚。备份只包含六个配置文件及 `skins/`、`pets/`、`agent-presets/`、`.agent-presets/` 资源目录,不是整个 home 的备份;会话、普通工作文件及node_modules不在备份范围内。
- 备份/恢复/升级自动编排停启;升级保存密码配置备份和home快照,失败尝试回滚。Profile使用本地Monaco及脱敏预览;目前仍需先停止实例,页面“保存并启动”随后验证就绪。 - 备份/恢复/升级自动编排停启;升级保存密码配置备份和home快照,失败尝试回滚。Profile使用本地Monaco及脱敏预览;目前仍需先停止实例,页面“保存并启动”随后验证就绪。
新备份使用创建时的登录密码经Argon2id派生密钥、AES-GCM加密,不依赖或创建 `backup.key`。导入/恢复须使用原备份密码,修改登录密码不会重加密旧包。旧版密钥包仍兼容,保留这些包时必须保管原 `backup.key`。 新备份使用创建时的登录密码经Argon2id派生密钥、AES-GCM加密,不依赖或创建 `backup.key`。导入/恢复须使用原备份密码,修改登录密码不会重加密旧包。旧版密钥包仍兼容,保留这些包时必须保管原 `backup.key`。
@@ -168,13 +174,13 @@ sudo systemctl reload caddy
## 本地runtime构建 ## 本地runtime构建
### 兼容清单与实验构建 ### Git 发布与镜像准入
镜像的构建验证与使用准入分开管理:推荐/兼容镜像还必须通过精确身份与实际验证检查才可选择;仅存量镜像允许原实例继续运行和启停,不可新建或作为升级目标;禁止使用镜像还拒绝新的启动/重启,但不会自动停止现有容器或断开工作台。 镜像的构建验证与使用准入分开管理:推荐/兼容镜像还必须通过精确身份与实际验证检查才可选择;仅存量镜像允许原实例继续运行和启停,不可新建或作为升级目标;禁止使用镜像还拒绝新的启动/重启,但不会自动停止现有容器或断开工作台。
未知版本仅管理员可在镜像页显式提交实验构建。实验保留镜像模板的完整性、依赖和启动检查,可能因不兼容而失败;不会为让新版通过而跳过检查。实验成功后仍不可创建实例,管理员需完成独立兼容验收,再在“准入管理”填写可追溯依据并确认。准入绑定精确镜像ID、版本、模板和平台,不自动信任同tag的新镜像,也不扩大普通版本构建白名单;已准入的未知版本请通过本地镜像选择创建。 管理员在镜像页检查 Git 发布并确认精确 commit 后构建。普通用户只能选择已验证且准入的本地镜像创建实例;没有可选镜像时需等待管理员发布与构建,不再提供 npm 版本、实验构建或构建后自动创建入口。这是 breaking change:移除 `/api/latest`、`/api/compatibility`、`POST /api/builds` 及 `/api/intents` 的全部查询、创建和取消接口,不提供 410 兼容入口。仅保留 `GET /api/builds` 和 `GET /api/builds/{id}` 构建历史读取。不迁移或恢复旧构建及旧创建意图,不在后台续跑或完成旧预约;这不意味着删除已有实例或数据。准入绑定精确镜像ID、版本、模板和平台,不自动信任同tag的新镜像。
历史模板按其自身验证记录与挂载布局判断,不因模板号不是当前版本就标为缺失。有成功验证记录、布局受支持的旧镜像可经管理员验收准入;无法识别的布局不会仅凭一段说明放行。界面展示实际准入原因,以后端校验为准。 历史模板保留其精确身份验证记录,不因模板号不是当前版本就标为缺失。但当前新建/升级及正向准入只支持模板7的挂载布局,且镜像平台必须匹配当前面板平台;旧模板即使有成功验证记录也不能靠管理员填写说明放行。内置默认推荐/兼容策略针对 `linux/amd64`,其他平台不能据此推断已经验收。存量实例的启停仍受实际资源状态与禁止使用策略约束;界面展示具体准入原因,以后端校验为准。
当前模板为修订7,同时内置 `[email protected]` 与 `[email protected]`(MIT,https://github.com/DDDMUC/dsh-free-search)。Free Search及其普通依赖固定版本、校验SHA-512并保留镜像内原始tarball;新profile默认正式安装并启用,已有profile不自动注入。模板4的mobile恢复路径继续保留。仅包含内置固定依赖的profile在启动前离线安装,不限制运行期插件管理器联网。 当前模板为修订7,同时内置 `[email protected]` 与 `[email protected]`(MIT,https://github.com/DDDMUC/dsh-free-search)。Free Search及其普通依赖固定版本、校验SHA-512并保留镜像内原始tarball;新profile默认正式安装并启用,已有profile不自动注入。模板4的mobile恢复路径继续保留。仅包含内置固定依赖的profile在启动前离线安装,不限制运行期插件管理器联网。
@@ -186,23 +192,34 @@ Free Search默认引擎为Bing,注册的统一provider ID为ddg;未设置或
依赖固定引用 `file:/usr/local/lib/dsh-mgr/dsh-web-mobile-3.0.5.tgz`,首次安装与备份恢复无需下载该包。备份包含依赖和bundle清单,恢复目标必须提供相同tarball路径(模板4及兼容后续镜像);初始化会在此依赖存在且workspace缺失时补建官方no-auto-peers配置,保留已有workspace及禁用状态。 依赖固定引用 `file:/usr/local/lib/dsh-mgr/dsh-web-mobile-3.0.5.tgz`,首次安装与备份恢复无需下载该包。备份包含依赖和bundle清单,恢复目标必须提供相同tarball路径(模板4及兼容后续镜像);初始化会在此依赖存在且workspace缺失时补建官方no-auto-peers配置,保留已有workspace及禁用状态。
需要Docker、Bash、npm及基础镜像/npm网络访问;不推送镜像。 运行镜像的构建源与初始化单测在 `dockerfile` 分支维护;main 只构建面板,不再提供 `scripts/build-image.sh`。镜像构建仍需要基础镜像与依赖网络访问,不推送镜像。
注意:dsh-web-mobile还包含响应压缩、移动会话删除入口和自定义模型推理档位补全,并非纯CSS。内置版本随镜像固定,不跟踪GitHub main或npm latest。 注意:dsh-web-mobile还包含响应压缩、移动会话删除入口和自定义模型推理档位补全,并非纯CSS。内置版本随镜像固定,不跟踪GitHub main或npm latest。
模板4目前限定DSH 0.2.0-rc.1/rc.2,并校验已安装宿主的Cordis/DSH peer版本;其他版本需先更新兼容验证,不会自动安装第二套宿主依赖。插件没有独立参数配置页;它修改DSH原有移动界面与设置页。 模板4目前限定DSH 0.2.0-rc.1/rc.2,并校验已安装宿主的Cordis/DSH peer版本;其他版本需先更新兼容验证,不会自动安装第二套宿主依赖。插件没有独立参数配置页;它修改DSH原有移动界面与设置页。
```sh ```sh
bash scripts/build-image.sh # 查询latest并锁定精确版本 bash scripts/smoke-image.sh "$RUNTIME_IMAGE_ID" # 已由管理员构建的本地镜像 ID
bash scripts/build-image.sh 0.2.0-rc.2 # 或指定精确版本
bash scripts/smoke-image.sh dsh-web:0.2.0-rc.2-7
``` ```
当前tag为 `dsh-web:<version>-7`(模板修订7)。runtime以UID/GID 1000运行,home和workspace需允许该身份读写。DSH监听loopback:3080,Caddy监听容器内8080;不要发布runtime端口或挂载socket。原生模块需要可执行的 `/tmp`,保留nosuid/nodev和容量限制。存在插件清单时先协调插件,失败不启动应用;真实验证范围见 ACCEPTANCE.md。 `scripts/smoke-image.sh` 只检查已有镜像运行,不读取源码、不构建镜像;需 Docker 和 Bash,会创建并清理临时容器。请从镜像页取得精确镜像 ID,不依赖旧版本 tag 推断 Git 发布身份。当前部署协议为7。runtime以UID/GID 1000运行,home和workspace需允许该身份读写。DSH监听loopback:3080,Caddy监听容器内8080;不要发布runtime端口或挂载socket。原生模块需要可执行的 `/tmp`,保留nosuid/nodev和容量限制。存在插件清单时先协调插件,失败不启动应用;真实验证范围见 ACCEPTANCE.md。
## 设计替代与边界 ## 设计替代与边界
构建历史接口现默认分页(20条,最大100条),列表不再携带日志正文;日志通过 `/api/builds/{id}` 按需读取。升级面板后请刷新已打开的旧页面;外部脚本需按 API.md 更新,不能再依赖 `/api/builds` 返回全部历史或日志。 构建历史接口现默认分页(20条,最大100条),列表不再携带日志正文;日志通过 `/api/builds/{id}` 按需读取。升级面板后请刷新已打开的旧页面;外部脚本需按 API.md 更新,不能再依赖 `/api/builds` 返回全部历史或日志。
合理替代方案为:stdlib HTTP Docker client而非官方SDK;每实例独立bridge而非共享bridge;服务端保管DSH cookie而非下发浏览器;普通React组件而非shadcn/ui(保留Monaco)。这些是选型差异,不降低功能、安全或验收标准,详见验收表。 合理替代方案为:stdlib HTTP Docker client而非官方SDK;每实例独立bridge而非共享bridge;服务端保管DSH cookie而非下发浏览器。前端已改用 Tailwind CSS + Radix 原语的 shadcn 风格组件(保留本地Monaco),桌面与移动端共用一套响应式界面。界面跟随系统深浅色设置自动切换夜间模式(含 Monaco 编辑器),与内嵌 DSH 行为一致,无独立开关。这些是选型差异,不降低功能、安全或验收标准,详见验收表。
同源路径代理仅适用于可信用户/插件,尚无独立origin及宿主/局域网出站防护。默认软配额不能阻止agent直接写满磁盘;硬quota是依赖宿主的可选增强,不是默认路线必做项。升级/恢复快照需额外容量,home快照上限1 GiB,模板7的 `/workspace` 就是实例home,会随home快照回滚;旧共享目录不随回滚恢复;中断及回滚失败仍可能需要管理员恢复。 同源路径代理仅适用于可信用户/插件,尚无独立origin及宿主/局域网出站防护。默认软配额不能阻止agent直接写满磁盘;硬quota是依赖宿主的可选增强,不是默认路线必做项。升级/恢复快照需额外容量,home快照上限1 GiB,模板7的 `/workspace` 就是实例home,会随home快照回滚。旧的用户级共享工作区接口和新容器共享挂载已经移除;磁盘上可能遗留的旧共享目录不属于实例home快照,也不会自动迁移。中断及回滚失败仍可能需要管理员恢复。
## 开发验证与文档维护
安装前端依赖后,可运行本地检查:
```sh
bash scripts/check-trial.sh
node --test scripts/check-trial.test.mjs
```
前者依次执行 Go 单测、race 检查、vet、构建、前端单测与生产构建,以及浏览器验收脚本的语法检查;后者验证检查脚本自身的失败传播行为。它们不等于真实 Docker、浏览器或公网 HTTPS 验收,需要外部环境的测试及历史记录见 [ACCEPTANCE.md](ACCEPTANCE.md)。
修改接口时同步 [API.md](API.md) 和 [前端对接说明](web/API_REQUEST.md);修改用户流程时同步本页和 [前端说明](web/README.md);修改部署、认证或数据布局时同步 [OPERATIONS.md](OPERATIONS.md)、[认证说明](internal/api/AUTH_DESIGN.md) 与 [DESIGN.md](DESIGN.md)。验收记录应注明验证范围,不把历史通过记录视为当前版本的全量验证。
-3
View File
@@ -125,9 +125,6 @@ func run() error {
srv := &http.Server{Addr: getenv("LISTEN_ADDR", "127.0.0.1:8088"), Handler: handler, ReadHeaderTimeout: 10 * time.Second, IdleTimeout: 60 * time.Second, MaxHeaderBytes: 1 << 20} srv := &http.Server{Addr: getenv("LISTEN_ADDR", "127.0.0.1:8088"), Handler: handler, ReadHeaderTimeout: 10 * time.Second, IdleTimeout: 60 * time.Second, MaxHeaderBytes: 1 << 20}
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop() defer stop()
if err := panel.StartIntents(ctx); err != nil {
return fmt.Errorf("creation intent recovery: %w", err)
}
go panel.RunQuotaMonitor(ctx) go panel.RunQuotaMonitor(ctx)
go panel.RunSessionCleanup(ctx) go panel.RunSessionCleanup(ctx)
return serve(ctx, srv) return serve(ctx, srv)
+7
View File
@@ -10,6 +10,10 @@ services:
- ${DOCKER_GID:?Set the numeric group ID of the Docker socket} - ${DOCKER_GID:?Set the numeric group ID of the Docker socket}
ports: ports:
- '127.0.0.1:8088:8088' - '127.0.0.1:8088:8088'
networks:
default:
# Keep the default gateway stable when joining runtime instance networks.
gw_priority: 100
environment: environment:
LISTEN_ADDR: 0.0.0.0:8088 LISTEN_ADDR: 0.0.0.0:8088
DATA_ROOT: /data DATA_ROOT: /data
@@ -17,6 +21,9 @@ services:
HOST_DATA_ROOT: ${DSH_HOST_DATA_ROOT:?Set an absolute host data directory} HOST_DATA_ROOT: ${DSH_HOST_DATA_ROOT:?Set an absolute host data directory}
PANEL_PUBLIC_URL: ${PANEL_PUBLIC_URL:-http://127.0.0.1:8088} PANEL_PUBLIC_URL: ${PANEL_PUBLIC_URL:-http://127.0.0.1:8088}
TRUSTED_PROXY_CIDRS: ${TRUSTED_PROXY_CIDRS:-} TRUSTED_PROXY_CIDRS: ${TRUSTED_PROXY_CIDRS:-}
SESSION_TTL: ${SESSION_TTL:-24h}
DSH_RELEASE_REPOSITORY: ${DSH_RELEASE_REPOSITORY:-https://git.misaka.ren/m1saka/dsh-web.git}
DSH_RELEASE_BRANCH: ${DSH_RELEASE_BRANCH:-dockerfile}
WEB_DIR: /app/web/dist WEB_DIR: /app/web/dist
UPLOAD_ALLOWED_EXTENSIONS: ${UPLOAD_ALLOWED_EXTENSIONS:-} UPLOAD_ALLOWED_EXTENSIONS: ${UPLOAD_ALLOWED_EXTENSIONS:-}
ADMIN_USERNAME: ${DSH_ADMIN_USERNAME:-admin} ADMIN_USERNAME: ${DSH_ADMIN_USERNAME:-admin}
-7
View File
@@ -1,7 +0,0 @@
{
admin off
auto_https off
}
:8080 {
reverse_proxy 127.0.0.1:3080
}
-30
View File
@@ -1,30 +0,0 @@
FROM caddy:2.10.2 AS proxy
FROM node:24-bookworm-slim
ARG DSH_VERSION
# Keep the historical dependency-layer cache key; the template revision is set below.
ARG RECIPE_REV=3
RUN apt-get update && apt-get install -y --no-install-recommends tini ca-certificates git procps && rm -rf /var/lib/apt/lists/*
RUN test -n "$DSH_VERSION" && npm install --global --foreground-scripts --loglevel=verbose --registry=https://registry.npmjs.org "@deepseek-ai/dsh@${DSH_VERSION}"
RUN npm install --global --ignore-scripts --registry=https://registry.npmjs.org [email protected] && test "$(pnpm --version)" = "10.11.0"
COPY install-mobile.mjs /usr/local/lib/dsh-mgr/install-mobile.mjs
RUN node /usr/local/lib/dsh-mgr/install-mobile.mjs
COPY install-free-search.mjs /usr/local/lib/dsh-mgr/install-free-search.mjs
RUN node /usr/local/lib/dsh-mgr/install-free-search.mjs
COPY install-workspace.mjs /usr/local/lib/dsh-mgr/install-workspace.mjs
RUN node /usr/local/lib/dsh-mgr/install-workspace.mjs
COPY init-profile.mjs /usr/local/lib/dsh-mgr/init-profile.mjs
COPY --from=proxy /usr/bin/caddy /usr/bin/caddy
# Strip upstream file capabilities without granting any runtime capabilities.
# A fresh file copy omits security.capability xattrs; port 8080 needs none.
RUN cp /usr/bin/caddy /usr/bin/caddy.clean && mv /usr/bin/caddy.clean /usr/bin/caddy
COPY Caddyfile /etc/caddy/Caddyfile
COPY entrypoint.sh /usr/local/bin/dsh-entrypoint
RUN chmod 755 /usr/local/bin/dsh-entrypoint && mkdir -p /data /workspace /shared && ln -s /workspace /data/dsh-home && chown -R node:node /data /workspace /shared
ARG RECIPE_REV=7
LABEL dsh-mgr.role=runtime dsh-mgr.version=$DSH_VERSION dsh-mgr.recipe=$RECIPE_REV
ENV DSH_HOME=/workspace HOME=/workspace XDG_DATA_HOME=/workspace/.local/share XDG_CONFIG_HOME=/workspace/.config
USER node
WORKDIR /workspace
EXPOSE 8080
ENTRYPOINT ["/usr/bin/tini", "-g", "--", "/usr/local/bin/dsh-entrypoint"]
CMD ["web", "--no-open", "--port", "3080"]
-6
View File
@@ -1,6 +0,0 @@
package image
import "embed"
//go:embed Dockerfile Caddyfile entrypoint.sh install-mobile.mjs install-free-search.mjs install-workspace.mjs init-profile.mjs
var Files embed.FS
-44
View File
@@ -1,44 +0,0 @@
#!/bin/bash
set -Eeuo pipefail
export npm_config_ignore_scripts=true npm_config_ignore_pnpmfile=true
node /usr/local/lib/dsh-mgr/init-profile.mjs
if [[ -f "${DSH_HOME}/profiles/web/package.json" ]]; then
echo "dsh-mgr: plugin prestart reconciliation starting" >&2
# pnpm otherwise probes registry metadata for the overlapping optional peer.
# Restrict offline mode to profiles containing only our fixed archives.
install_env=()
if node -e 'const m=require(process.env.DSH_HOME+"/profiles/web/package.json"); const pins={"dsh-mgr-workspace":"file:/usr/local/lib/dsh-mgr/dsh-mgr-workspace-1.0.0.tgz","dsh-web-mobile":"file:/usr/local/lib/dsh-mgr/dsh-web-mobile-3.0.5.tgz","dsh-free-search":"file:/usr/local/lib/dsh-mgr/dsh-free-search-0.8.5.tgz"}; process.exit(Object.entries({...m.dependencies,...m.devDependencies,...m.optionalDependencies}).every(([n,v])=>pins[n]===v)?0:1)'; then
install_env=(npm_config_offline=true)
fi
if timeout --signal=TERM --kill-after=5s 120s env CI=true "${install_env[@]}" dsh plugin --profile web install --ignore-scripts; then
echo "dsh-mgr: plugin prestart reconciliation complete" >&2
else
status=$?
echo "dsh-mgr: plugin prestart reconciliation failed (exit $status); application not started" >&2
exit "$status"
fi
else
echo "dsh-mgr: no web plugin manifest; using bundled plugins" >&2
fi
caddy run --config /etc/caddy/Caddyfile --adapter caddyfile &
proxy_pid=$!
echo
dsh "$@" &
dsh_pid=$!
cleanup() {
trap - TERM INT
kill -TERM "$proxy_pid" "$dsh_pid" 2>/dev/null || true
wait "$proxy_pid" 2>/dev/null || true
wait "$dsh_pid" 2>/dev/null || true
}
trap cleanup EXIT
trap 'exit 143' TERM
trap 'exit 130' INT
# Either child exiting terminates the container; never leave a proxy-only instance.
set +e
wait -n "$proxy_pid" "$dsh_pid"
status=$?
set -e
exit "$status"
-35
View File
@@ -1,35 +0,0 @@
import { mkdirSync, existsSync, writeFileSync, readFileSync, readdirSync, lstatSync } from 'node:fs';
import { join } from 'node:path';
const home = process.env.DSH_HOME;
if (!home) throw new Error('DSH_HOME required');
const profile = join(home, 'profiles/web');
for (const dir of [home, join(home, 'profiles'), profile]) {
try {
const stat = lstatSync(dir);
if (!stat.isDirectory() || stat.isSymbolicLink()) throw new Error('unsafe profile directory');
} catch (error) { if (error.code !== 'ENOENT') throw error; }
}
// Never rewrite an existing user manifest or patch.
if (!existsSync(profile) || readdirSync(profile).length === 0) {
mkdirSync(profile, { recursive: true });
writeFileSync(join(profile, 'package.json'), JSON.stringify({
name: 'dsh-profile-web', private: true,
dependencies: { 'dsh-mgr-workspace': 'file:/usr/local/lib/dsh-mgr/dsh-mgr-workspace-1.0.0.tgz', 'dsh-web-mobile': 'file:/usr/local/lib/dsh-mgr/dsh-web-mobile-3.0.5.tgz', 'dsh-free-search': 'file:/usr/local/lib/dsh-mgr/dsh-free-search-0.8.5.tgz' },
pnpm: { overrides: {
'dsh-free-search>@deepseek-ai/schemastery': 'file:/usr/local/lib/dsh-mgr/schemastery-3.18.4.tgz',
'@deepseek-ai/schemastery>@deepseek-ai/cosmokit': 'file:/usr/local/lib/dsh-mgr/cosmokit-1.8.5.tgz',
'@deepseek-ai/schemastery>@standard-schema/spec': 'file:/usr/local/lib/dsh-mgr/spec-1.1.0.tgz',
} },
dsh: { profile: { bundles: ['@deepseek-ai/dsh-base', '@deepseek-ai/dsh-web-app', 'dsh-web-mobile', 'dsh-free-search', 'dsh-mgr-workspace'] } },
}, null, 2), { flag: 'wx', mode: 0o600 });
console.log('dsh-mgr: new web profile includes [email protected] and [email protected]');
}
const manifestPath = join(profile, 'package.json');
const workspacePath = join(profile, 'pnpm-workspace.yaml');
// Backups omit workspace settings; restore host-provided peers before pnpm runs.
if (existsSync(manifestPath) && !existsSync(workspacePath)) {
const manifest = JSON.parse(readFileSync(manifestPath, 'utf8'));
if (manifest.dependencies?.['dsh-web-mobile'] === 'file:/usr/local/lib/dsh-mgr/dsh-web-mobile-3.0.5.tgz' || manifest.dependencies?.['dsh-free-search'] === 'file:/usr/local/lib/dsh-mgr/dsh-free-search-0.8.5.tgz') {
writeFileSync(workspacePath, 'packages:\n - .\n\nnodeLinker: hoisted\nautoInstallPeers: false\n', { flag: 'wx', mode: 0o600 });
}
}
-64
View File
@@ -1,64 +0,0 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { mkdtempSync, mkdirSync, readFileSync, writeFileSync, rmSync, symlinkSync, existsSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { execFileSync } from 'node:child_process';
test('new profile enables bundle once and preserves existing configuration', () => {
const home = mkdtempSync(join(tmpdir(), 'dsh-plugin-init-'));
try {
const run = () => execFileSync(process.execPath, [new URL('./init-profile.mjs', import.meta.url).pathname], {env:{...process.env, DSH_HOME:home}});
run();
const file=join(home,'profiles/web/package.json');
const original=readFileSync(file,'utf8');
assert.ok(JSON.parse(original).dsh.profile.bundles.includes('dsh-web-mobile'));
assert.ok(JSON.parse(original).dsh.profile.bundles.includes('dsh-free-search'));
assert.ok(JSON.parse(original).dsh.profile.bundles.includes('dsh-mgr-workspace'));
assert.equal(JSON.parse(original).dependencies['dsh-mgr-workspace'], 'file:/usr/local/lib/dsh-mgr/dsh-mgr-workspace-1.0.0.tgz');
assert.equal(JSON.parse(original).dependencies['dsh-free-search'], 'file:/usr/local/lib/dsh-mgr/dsh-free-search-0.8.5.tgz');
assert.equal(JSON.parse(original).pnpm.overrides['dsh-free-search>@deepseek-ai/schemastery'], 'file:/usr/local/lib/dsh-mgr/schemastery-3.18.4.tgz');
assert.equal(JSON.parse(original).dependencies['dsh-web-mobile'], 'file:/usr/local/lib/dsh-mgr/dsh-web-mobile-3.0.5.tgz');
const workspace = join(home, 'profiles/web/pnpm-workspace.yaml');
assert.match(readFileSync(workspace, 'utf8'), /autoInstallPeers: false/);
assert.match(readFileSync(workspace, 'utf8'), /nodeLinker: hoisted/);
run();assert.equal(readFileSync(file,'utf8'),original);
const legacy = JSON.parse(original);
delete legacy.dependencies['dsh-mgr-workspace'];
legacy.dsh.profile.bundles = legacy.dsh.profile.bundles.filter(name => name !== 'dsh-mgr-workspace');
writeFileSync(file, JSON.stringify(legacy));
run();
assert.deepEqual(JSON.parse(readFileSync(file)), legacy);
const disabled = JSON.parse(original);
disabled.dsh.profile.bundles = disabled.dsh.profile.bundles.filter(name => name !== 'dsh-web-mobile');
writeFileSync(file, JSON.stringify(disabled));
writeFileSync(workspace, 'custom workspace');
const patch = join(home, 'profiles/web/cordis.patch.yml');
writeFileSync(patch, '[]');
run();
assert.deepEqual(JSON.parse(readFileSync(file)), disabled);
assert.equal(readFileSync(workspace, 'utf8'), 'custom workspace');
assert.equal(readFileSync(patch, 'utf8'), '[]');
const custom=JSON.stringify({custom:true});
writeFileSync(file,custom);run();assert.equal(readFileSync(file,'utf8'),custom);
} finally { rmSync(home,{recursive:true,force:true}); }
});
test('nonempty legacy profiles are not automatically injected', () => {
const home = mkdtempSync(join(tmpdir(), 'dsh-plugin-legacy-'));
try {
const profile = join(home, 'profiles/web');
mkdirSync(profile, { recursive: true });
writeFileSync(join(profile, 'cordis.patch.yml'), '[]');
execFileSync(process.execPath, [new URL('./init-profile.mjs', import.meta.url).pathname], { env: { ...process.env, DSH_HOME: home } });
assert.equal(existsSync(join(profile, 'package.json')), false);
assert.equal(existsSync(join(profile, 'pnpm-workspace.yaml')), false);
} finally { rmSync(home, { recursive: true, force: true }); }
});
test('rejects symlinked profile ancestors', () => {
const home=mkdtempSync(join(tmpdir(),'dsh-plugin-init-'));
const outside=mkdtempSync(join(tmpdir(),'dsh-plugin-outside-'));
try {
symlinkSync(outside,join(home,'profiles'));
assert.throws(()=>execFileSync(process.execPath,[new URL('./init-profile.mjs',import.meta.url).pathname],{env:{...process.env,DSH_HOME:home},stdio:'pipe'}));
assert.equal(existsSync(join(outside,'web/package.json')),false);
} finally {rmSync(home,{recursive:true,force:true});rmSync(outside,{recursive:true,force:true});}
});
-40
View File
@@ -1,40 +0,0 @@
import { createHash } from 'node:crypto';
import { mkdirSync, writeFileSync, readFileSync } from 'node:fs';
import { execFileSync } from 'node:child_process';
import { createRequire } from 'node:module';
const root = '/usr/local/lib/dsh-mgr';
const packages = [
['dsh-free-search', '0.8.5', 'https://registry.npmjs.org/dsh-free-search/-/dsh-free-search-0.8.5.tgz', 'MT00Xtypw9uk9QsqaiRmtVbYwnPON+HJY1lZjYritEvuwSe9ahgG9NVO43lUI+Eje2umrqUapaeqLRYf34DrZw=='],
['@deepseek-ai/schemastery', '3.18.4', 'https://registry.npmjs.org/@deepseek-ai/schemastery/-/schemastery-3.18.4.tgz', 'SSXO6tYuyrIqKVbmOnIq0s+riUywYzouFMcnBluHF9n4KMo2G8HvEzhCYj6pj2617/glOjbJuVInJ9hfONsjkg=='],
['@deepseek-ai/cosmokit', '1.8.5', 'https://registry.npmjs.org/@deepseek-ai/cosmokit/-/cosmokit-1.8.5.tgz', 'LXsrlem9z8dq4sLflj2yuCuYX7KqhdzF5hZly3eZyuTo8d6oDSOXuEgC5DbQWKW+lksm6zmOutQLsP/ma6wR6A=='],
['@standard-schema/spec', '1.1.0', 'https://registry.npmjs.org/@standard-schema/spec/-/spec-1.1.0.tgz', 'l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w=='],
];
for (const [name, version, url, integrity] of packages) {
const response = await fetch(url, { signal: AbortSignal.timeout(120000) });
if (!response.ok) throw new Error('download failed: ' + name);
const data = Buffer.from(await response.arrayBuffer());
if (createHash('sha512').update(data).digest('base64') !== integrity) throw new Error('integrity mismatch: ' + name);
const archive = `${root}/${name.split('/').pop()}-${version}.tgz`;
writeFileSync(archive, data, { mode: 0o644 });
}
const hostManifest = '/usr/local/lib/node_modules/@deepseek-ai/dsh/package.json';
const destination = '/usr/local/lib/node_modules/@deepseek-ai/dsh/node_modules/dsh-free-search';
mkdirSync(destination, { recursive: true });
execFileSync('tar', ['-xzf', `${root}/dsh-free-search-0.8.5.tgz`, '--strip-components=1', '--no-same-owner', '-C', destination]);
execFileSync('chmod', ['-R', 'a+rX', destination]);
const pkg = JSON.parse(readFileSync(destination + '/package.json'));
if (pkg.name !== 'dsh-free-search' || pkg.version !== '0.8.5' || JSON.stringify(pkg.dependencies) !== JSON.stringify({ '@deepseek-ai/schemastery': '^3.18.4' })) throw new Error('unexpected free-search metadata');
const require = createRequire(hostManifest);
const semver = createRequire('/usr/local/lib/node_modules/npm/package.json')('semver');
for (const [name, range] of Object.entries({ ...pkg.peerDependencies, ...pkg.dependencies })) {
if (name === 'react') {
if (!['0.2.0-rc.1', '0.2.0-rc.2'].includes(require('./package.json').version)) throw new Error('unverified browser React host');
continue;
}
const peer = require(name + '/package.json');
if (!semver.satisfies(peer.version, range)) throw new Error('incompatible free-search host dependency: ' + name);
}
const host = JSON.parse(readFileSync(hostManifest));
host.dependencies = { ...host.dependencies, 'dsh-free-search': '0.8.5' };
writeFileSync(hostManifest, JSON.stringify(host, null, 2));
console.log('Installed [email protected] and pinned offline dependency archives (SHA-512 verified; no lifecycle scripts)');
-44
View File
@@ -1,44 +0,0 @@
import { createHash } from 'node:crypto';
import { mkdirSync, writeFileSync, readFileSync } from 'node:fs';
import { execFileSync } from 'node:child_process';
import { createRequire } from 'node:module';
const version = '3.0.5';
const integrity = 'N3wmegFHSd23J0+xjotKyBq0ZbLXUbkWZ65E+aSw5pzB/nudhaNHWT2V3CSNaW/LX3s6AxAWB2APpfA0dLXJZQ==';
const response = await fetch('https://registry.npmjs.org/dsh-web-mobile/-/dsh-web-mobile-3.0.5.tgz', { signal: AbortSignal.timeout(120000) });
if (!response.ok) throw new Error('mobile plugin download failed');
const data = Buffer.from(await response.arrayBuffer());
if (createHash('sha512').update(data).digest('base64') !== integrity) throw new Error('mobile plugin integrity mismatch');
const destination = '/usr/local/lib/node_modules/@deepseek-ai/dsh/node_modules/dsh-web-mobile';
mkdirSync(destination, { recursive: true });
const archive = '/usr/local/lib/dsh-mgr/dsh-web-mobile-3.0.5.tgz';
writeFileSync(archive, data, { mode: 0o644 });
execFileSync('tar', ['-xzf', archive, '--strip-components=1', '--no-same-owner', '-C', destination]);
// Published tar modes can be owner-only; runtime is UID 1000, not build root.
execFileSync('chmod', ['-R', 'a+rX', destination]);
const pkg = JSON.parse(readFileSync(destination + '/package.json'));
if (pkg.version !== version || pkg.name !== 'dsh-web-mobile' || Object.keys(pkg.dependencies || {}).length) throw new Error('unexpected mobile package metadata');
const require = createRequire('/usr/local/lib/node_modules/@deepseek-ai/dsh/package.json');
const semver = createRequire('/usr/local/lib/node_modules/npm/package.json')('semver');
for (const [name, range] of Object.entries(pkg.peerDependencies || {})) {
// React is provided by the browser shell, not a Node package in DSH.
// Limit that exception to the upstream-declared, tested RC hosts.
if (name === 'react') {
const host = require('./package.json').version;
if (!['0.2.0-rc.1', '0.2.0-rc.2'].includes(host)) throw new Error('unverified browser React host: ' + host);
continue;
}
let peer;
try { peer = require(name + '/package.json'); }
catch (error) {
if (pkg.peerDependenciesMeta?.[name]?.optional) continue;
throw new Error('missing required mobile peer: ' + name);
}
if (!semver.satisfies(peer.version, range)) throw new Error('incompatible mobile peer: ' + name + '@' + peer.version);
}
// DSH runtime resolution traverses the installation manifest, not arbitrary
// directories in node_modules. Declare the baked package in that graph.
const hostManifest = '/usr/local/lib/node_modules/@deepseek-ai/dsh/package.json';
const host = JSON.parse(readFileSync(hostManifest));
host.dependencies = { ...host.dependencies, 'dsh-web-mobile': version };
writeFileSync(hostManifest, JSON.stringify(host, null, 2));
console.log('Installed [email protected] (SHA-512 verified; no lifecycle scripts)');
-12
View File
@@ -1,12 +0,0 @@
import { mkdirSync, writeFileSync } from 'node:fs';
import { execFileSync } from 'node:child_process';
const root = '/usr/local/lib/dsh-mgr';
const destination = root + '/workspace-package/package';
mkdirSync(destination, { recursive: true });
writeFileSync(destination + '/package.json', JSON.stringify({
name: 'dsh-mgr-workspace', version: '1.0.0', type: 'module', main: 'index.js',
dsh: { bundle: { patch: 'cordis.patch.yml' } },
}));
writeFileSync(destination + '/index.js', "export const inject = ['workspaceRegistry', 'sessions'];\nexport async function apply(ctx) {\n await ctx.workspaceRegistry.initializeDefault(() => '/workspace');\n}\n");
writeFileSync(destination + '/cordis.patch.yml', "- insert:\n - id: dsh-mgr-workspace\n name: dsh-mgr-workspace\n");
execFileSync('tar', ['-czf', root + '/dsh-mgr-workspace-1.0.0.tgz', '-C', root + '/workspace-package', 'package']);
+37 -7
View File
@@ -4,19 +4,49 @@
five-minute HS256 access JWT in an additional HttpOnly cookie. JWT verification five-minute HS256 access JWT in an additional HttpOnly cookie. JWT verification
fixes the algorithm and binds subject/session/issuer/audience/expiry. Bare fixes the algorithm and binds subject/session/issuer/audience/expiry. Bare
session IDs never authenticate. Internal callers and test fixtures must also session IDs never authenticate. Internal callers and test fixtures must also
attach a signed access cookie. attach a signed access cookie. Database session expiry or revocation rejects
- Refresh is a random 256-bit HttpOnly cookie, scoped to `/api`, with only its access even if the JWT's five-minute expiry has not yet elapsed.
SHA-256 hash stored in SQLite. The session has a 24-hour absolute lifetime. - The initial refresh token is random 256-bit data in an HttpOnly cookie, scoped to `/api`, with only its
Conditional SQL update makes rotation single-use even across concurrent requests. SHA-256 hash stored in SQLite. The session has an absolute lifetime configured
by SESSION_TTL (default 24h), applied only to new logins. Values use Go duration
syntax (e.g. 168h, not 7d), must be at least one second and a whole number of
seconds; invalid values prevent startup when the store is opened.
Session, refresh and CSRF cookies persist across browser restarts with explicit
Max-Age/Expires bounded by that database expiry; rotation does not extend it.
Transactional rotation stores current and previous token hashes. A keyed,
session-bound successor derivation makes retries of the previous token return
the same current token, including after response loss or server restart. Only
one previous generation is accepted (until the next rotation or session expiry);
this intentionally permits that generation to be replayed with the required
session/Origin/CSRF checks. Older and unknown tokens are rejected. Plaintext
refresh credentials are never stored.
Logout deletes the database session and its refresh binding. Both refresh and Logout deletes the database session and its refresh binding. Both refresh and
logout retain mandatory exact Origin and session-bound CSRF checks, even when logout retain mandatory exact Origin and session-bound CSRF checks, even when
access has expired. Other operations require valid access. access has expired, but require an unexpired database session. Logout does not
require the refresh cookie. Other protected operations require valid access.
Self-service password changes and administrator password resets revoke all of
that user's sessions and refresh bindings, requiring a new login.
- Signing key is generated once under the configured data root, private mode - Signing key is generated once under the configured data root, private mode
0600, and reused across restarts. Initialization errors fail authentication closed. 0600, and reused across restarts. Initialization errors fail authentication closed.
Tokens never appear in JSON, URLs, audit detail, or application logs. Session/access/refresh credentials are issued as cookies, not JSON or URLs,
and are not included in authentication audit detail or application logs. The
CSRF token is intentionally returned as `csrf_token` by login, refresh and
`/api/me`, and is also in the JavaScript-readable `dsh_csrf` cookie. All four
cookies use SameSite=Lax; the other three are HttpOnly. Secure is enabled by
direct TLS or an HTTPS PANEL_PUBLIC_URL, not forwarded protocol headers.
Origin checks compare against the configured public origin, or direct request
scheme/Host if it is unset; forwarded Host/Proto headers are not used.
- Known accounts lock for 15 minutes after five failures; the counter/lock survive - Known accounts lock for 15 minutes after five failures; the counter/lock survive
restart. Successful login resets the counter; expired locks reset the counter. restart. Successful login resets the counter; expired locks reset the counter.
Existing global throttling remains additional protection. Current-password reauthentication failures share this account lock; successful
reauthentication alone does not reset the counter. Additional login protection
allows 30 attempts per source per minute, at most 1024 source buckets, and eight
admitted concurrent login requests (credential checks are serialized). Source
identity uses the direct peer unless it matches TRUSTED_PROXY_CIDRS, a
comma-separated CIDR list; then X-Forwarded-For is scanned right-to-left for
the first untrusted address. These in-memory limits reset on process restart.
The server runs expired-session/refresh and elapsed-lock cleanup immediately
and every ten minutes; expiry checks do not wait for cleanup.
- Tests cover missing/tampered/expired access, database binding, refresh rotation - Tests cover missing/tampered/expired access, database binding, refresh rotation
and replay, CSRF/Origin rejection, logout revocation, key reuse/permissions, and replay, CSRF/Origin rejection, logout revocation, key reuse/permissions,
account isolation, restart persistence, lock expiry, and runtime compatibility. account isolation, restart persistence, lock expiry, and runtime compatibility.
+10 -24
View File
@@ -14,7 +14,6 @@ import (
// Admission writes serialize with new runtime operations, including their Docker side effects. // Admission writes serialize with new runtime operations, including their Docker side effects.
var admissionMu sync.RWMutex var admissionMu sync.RWMutex
var errBuildUnsupported = errors.New("version is not supported for normal builds; administrator experimental build required")
func defaultAdmission(version, recipe, platform string) string { func defaultAdmission(version, recipe, platform string) string {
if recipe == "7" && platform == "linux/amd64" { if recipe == "7" && platform == "linux/amd64" {
@@ -28,31 +27,9 @@ func defaultAdmission(version, recipe, platform string) string {
return "existing_only" return "existing_only"
} }
func supportedLayout(recipe string) bool { func supportedLayout(recipe string) bool {
switch recipe { return recipe == imageRecipeRev
case "3", "4", "5", "6", "7":
return true
}
return false
}
func normalBuildAllowed(version string) bool {
return positiveAdmission(defaultAdmission(version, imageRecipeRev, imagePlatform))
} }
func positiveAdmission(state string) bool { return state == "recommended" || state == "compatible" } func positiveAdmission(state string) bool { return state == "recommended" || state == "compatible" }
func (a *API) compatibility(w http.ResponseWriter, r *http.Request) {
if _, _, err := a.user(r); err != nil {
fail(w, 401, "authentication required")
return
}
if r.Method != http.MethodGet {
fail(w, 405, "method not allowed")
return
}
entries := []map[string]any{}
for _, v := range []string{"0.2.0-rc.1", "0.2.0-rc.2"} {
entries = append(entries, map[string]any{"version": v, "recipe_rev": imageRecipeRev, "platform": imagePlatform, "state": defaultAdmission(v, imageRecipeRev, imagePlatform), "build_allowed": normalBuildAllowed(v), "reason": "only linux/amd64 recipe 7 is built-in; successful exact image verification still required"})
}
writeJSON(w, map[string]any{"recipe_rev": imageRecipeRev, "platform": imagePlatform, "entries": entries, "unknown": map[string]any{"state": "existing_only", "build_allowed": false, "reason": "not accepted; administrator experimental build and explicit admission required"}, "experimental_admin_only": true})
}
func (a *API) evaluateImage(ctx context.Context, id string) (imageRecord, error) { func (a *API) evaluateImage(ctx context.Context, id string) (imageRecord, error) {
im := imageRecord{ID: id, ImageID: id, Status: "missing", Admission: "existing_only", AdmissionReason: "local image missing"} im := imageRecord{ID: id, ImageID: id, Status: "missing", Admission: "existing_only", AdmissionReason: "local image missing"}
@@ -89,6 +66,11 @@ func (a *API) evaluateImage(ctx context.Context, id string) (imageRecord, error)
if count == 0 { if count == 0 {
return im, nil return im, nil
} }
var trustedRelease int
err = a.s.DB.QueryRowContext(ctx, "SELECT count(*) FROM image_builds WHERE image_id=? AND version=? AND recipe_rev=? AND platform=? AND status='success' AND release_id<>'' AND release_id=? AND source_repository=? AND source_commit=?", id, im.Version, im.RecipeRev, im.Platform, actual.Config.Labels["dsh-mgr.release"], actual.Config.Labels["dsh-mgr.source"], actual.Config.Labels["dsh-mgr.commit"]).Scan(&trustedRelease)
if err != nil {
return im, err
}
im.Admission = defaultAdmission(im.Version, im.RecipeRev, im.Platform) im.Admission = defaultAdmission(im.Version, im.RecipeRev, im.Platform)
im.AdmissionReason = "no accepted compatibility evidence for this tuple" im.AdmissionReason = "no accepted compatibility evidence for this tuple"
if positiveAdmission(im.Admission) { if positiveAdmission(im.Admission) {
@@ -101,6 +83,10 @@ func (a *API) evaluateImage(ctx context.Context, id string) (imageRecord, error)
} else { } else {
im.Status = "existing_only" im.Status = "existing_only"
} }
if trustedRelease > 0 {
im.Admission = "compatible"
im.AdmissionReason = "trusted pinned release passed exact image startup validation"
}
if state != "" { if state != "" {
im.Admission = state im.Admission = state
im.AdmissionReason = evidence im.AdmissionReason = evidence
+5 -6
View File
@@ -77,16 +77,15 @@ func TestAdmissionRoutesEnforceAuthorization(t *testing.T) {
a.Handler().ServeHTTP(w, r) a.Handler().ServeHTTP(w, r)
return w return w
} }
if w := request("GET", "/api/compatibility", "", false, false); w.Code != 401 { for _, path := range []string{"/api/latest", "/api/compatibility", "/api/intents", "/api/intents/old"} {
t.Fatal(w.Code, w.Body.String()) if w := request("GET", path, "", true, false); w.Code != 404 {
} t.Fatal(path, w.Code, w.Body.String())
if w := request("GET", "/api/compatibility", "", true, false); w.Code != 200 { }
t.Fatal(w.Code, w.Body.String())
} }
if w := request("POST", "/api/images/admission", body, true, true); w.Code != 403 { if w := request("POST", "/api/images/admission", body, true, true); w.Code != 403 {
t.Fatal("nonadmin admission", w.Code, w.Body.String()) t.Fatal("nonadmin admission", w.Code, w.Body.String())
} }
if w := request("POST", "/api/builds", `{"version":"9.8.7","experimental":true}`, true, true); w.Code != 403 { if w := request("POST", "/api/builds", `{"version":"9.8.7","experimental":true}`, true, true); w.Code != 405 {
t.Fatal("nonadmin experiment", w.Code, w.Body.String()) t.Fatal("nonadmin experiment", w.Code, w.Body.String())
} }
if _, err = a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='owner'"); err != nil { if _, err = a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='owner'"); err != nil {
+49 -36
View File
@@ -12,9 +12,9 @@ import (
"time" "time"
) )
func TestExperimentalBuildSuccessRequiresAcceptance(t *testing.T) { func TestHistoricalExperimentalBuildRequiresAcceptance(t *testing.T) {
id := "sha256:" + strings.Repeat("d", 64) id := "sha256:" + strings.Repeat("d", 64)
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path { switch r.URL.Path {
case "/v1.47/images/json": case "/v1.47/images/json":
writeJSON(w, []any{map[string]any{"Id": id, "Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "9.8.7", "dsh-mgr.recipe": imageRecipeRev}}}) writeJSON(w, []any{map[string]any{"Id": id, "Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "9.8.7", "dsh-mgr.recipe": imageRecipeRev}}})
@@ -35,27 +35,9 @@ func TestExperimentalBuildSuccessRequiresAcceptance(t *testing.T) {
if _, err := a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='u'"); err != nil { if _, err := a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='u'"); err != nil {
t.Fatal(err) t.Fatal(err)
} }
w := imageRequest(a.builds, sid, "POST", `{"version":"9.8.7","experimental":true}`) seedVerifiedImage(t, a, id, "9.8.7")
if w.Code != 202 { if _, err := a.s.DB.Exec("DELETE FROM image_admissions; UPDATE image_builds SET experimental=1"); err != nil {
t.Fatal(w.Code, w.Body.String()) t.Fatal(err)
}
var b buildRecord
if err := json.Unmarshal(w.Body.Bytes(), &b); err != nil || !b.Experimental {
t.Fatal(b, err)
}
deadline := time.Now().Add(5 * time.Second)
for {
var state string
if err := a.s.DB.QueryRow("SELECT status FROM image_builds WHERE id=?", b.ID).Scan(&state); err != nil {
t.Fatal(err)
}
if state == "success" {
break
}
if state == "failed" || time.Now().After(deadline) {
t.Fatal(state)
}
time.Sleep(10 * time.Millisecond)
} }
im, err := a.evaluateImage(context.Background(), id) im, err := a.evaluateImage(context.Background(), id)
if err != nil || im.Selectable || im.Status != "experimental" { if err != nil || im.Selectable || im.Status != "experimental" {
@@ -63,7 +45,7 @@ func TestExperimentalBuildSuccessRequiresAcceptance(t *testing.T) {
} }
} }
func TestQueuedUnsupportedPlatformFailsBeforeDocker(t *testing.T) { func TestLegacyQueuedBuildFailsBeforeDocker(t *testing.T) {
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("unsupported build contacted Docker") }) a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("unsupported build contacted Docker") })
buildQueue.Lock() buildQueue.Lock()
err := a.ensureBuilds() err := a.ensureBuilds()
@@ -80,7 +62,7 @@ func TestQueuedUnsupportedPlatformFailsBeforeDocker(t *testing.T) {
} }
a.runImageBuild(b) a.runImageBuild(b)
var state, detail string var state, detail string
if err := a.s.DB.QueryRow("SELECT status,error FROM image_builds WHERE id=?", b.ID).Scan(&state, &detail); err != nil || state != "failed" || !strings.Contains(detail, "experimental") { if err := a.s.DB.QueryRow("SELECT status,error FROM image_builds WHERE id=?", b.ID).Scan(&state, &detail); err != nil || state != "failed" || !strings.Contains(detail, "Git release identity required") {
t.Fatal(state, detail, err) t.Fatal(state, detail, err)
} }
} }
@@ -144,13 +126,47 @@ func TestAdmissionDowngradeWaitsForCreate(t *testing.T) {
} }
} }
func TestAdmissionHistoricalCreateAndBlock(t *testing.T) { func TestAdmissionRejectsLegacyLayouts(t *testing.T) {
for _, recipe := range []string{"", "1", "2", "3", "4", "5", "6", "8"} {
t.Run(recipe, func(t *testing.T) {
id := "sha256:" + strings.Repeat("a", 64)
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if !strings.Contains(r.URL.Path, "/images/") {
t.Error("unexpected Docker operation", r.Method, r.URL.Path)
}
writeJSON(w, map[string]any{"Id": id, "Os": "linux", "Architecture": "amd64", "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "1.2.3", "dsh-mgr.recipe": recipe}}})
})
seedVerifiedImage(t, a, id, "1.2.3")
for _, query := range []string{"UPDATE image_builds SET recipe_rev=?", "UPDATE image_admissions SET recipe_rev=?"} {
if _, err := a.s.DB.Exec(query, recipe); err != nil {
t.Fatal(err)
}
}
if _, err := a.s.DB.Exec("UPDATE users SET admin=1"); err != nil {
t.Fatal(err)
}
im, err := a.evaluateImage(context.Background(), id)
if err != nil || im.Selectable || im.Admission != "existing_only" {
t.Fatal(im, err)
}
for _, state := range []string{"compatible", "recommended"} {
w := httptest.NewRecorder()
a.imageAdmission(w, runtimeRequest("POST", "/api/images/admission", fmt.Sprintf(`{"image_id":%q,"state":%q,"evidence":"historical acceptance"}`, id, state), sid))
if w.Code != 400 {
t.Fatal(w.Code, w.Body.String())
}
}
})
}
}
func TestAdmissionCurrentCreateAndBlock(t *testing.T) {
id := "sha256:" + strings.Repeat("a", 64) id := "sha256:" + strings.Repeat("a", 64)
var body map[string]any var body map[string]any
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch { switch {
case strings.Contains(r.URL.Path, "/images/"): case strings.Contains(r.URL.Path, "/images/"):
writeJSON(w, map[string]any{"Id": id, "Os": "linux", "Architecture": "amd64", "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "1.2.3", "dsh-mgr.recipe": "4"}}}) writeJSON(w, map[string]any{"Id": id, "Os": "linux", "Architecture": "amd64", "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "1.2.3", "dsh-mgr.recipe": imageRecipeRev}}})
case strings.HasSuffix(r.URL.Path, "/containers/create"): case strings.HasSuffix(r.URL.Path, "/containers/create"):
if err := json.NewDecoder(r.Body).Decode(&body); err != nil { if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
t.Error(err) t.Error(err)
@@ -161,33 +177,33 @@ func TestAdmissionHistoricalCreateAndBlock(t *testing.T) {
} }
}) })
seedVerifiedImage(t, a, id, "1.2.3") seedVerifiedImage(t, a, id, "1.2.3")
if _, err := a.s.DB.Exec("UPDATE image_builds SET recipe_rev='4'; DELETE FROM image_admissions; UPDATE users SET admin=1"); err != nil { if _, err := a.s.DB.Exec("DELETE FROM image_admissions; UPDATE users SET admin=1"); err != nil {
t.Fatal(err) t.Fatal(err)
} }
im, err := a.evaluateImage(context.Background(), id) im, err := a.evaluateImage(context.Background(), id)
if err != nil || im.Status != "existing_only" || im.Selectable { if err != nil || im.Status != "existing_only" || im.Selectable {
t.Fatal(im, err) t.Fatal(im, err)
} }
records := []buildRecord{{ID: "unused", ImageID: id, Version: "1.2.3", RecipeRev: "4", Platform: imagePlatform, Status: "success"}} records := []buildRecord{{ID: "unused", ImageID: id, Version: "1.2.3", RecipeRev: imageRecipeRev, Platform: imagePlatform, Status: "success"}}
if err := a.reconcileBuilds(context.Background(), records); err != nil || records[0].Status != "success" { if err := a.reconcileBuilds(context.Background(), records); err != nil || records[0].Status != "success" {
t.Fatal(records, err) t.Fatal(records, err)
} }
admit := func(state string) *httptest.ResponseRecorder { admit := func(state string) *httptest.ResponseRecorder {
w := httptest.NewRecorder() w := httptest.NewRecorder()
a.imageAdmission(w, runtimeRequest("POST", "/api/images/admission", fmt.Sprintf(`{"image_id":%q,"state":%q,"evidence":"legacy layout explicitly accepted"}`, id, state), sid)) a.imageAdmission(w, runtimeRequest("POST", "/api/images/admission", fmt.Sprintf(`{"image_id":%q,"state":%q,"evidence":"current layout explicitly accepted"}`, id, state), sid))
return w return w
} }
if w := admit("compatible"); w.Code != 200 { if w := admit("compatible"); w.Code != 200 {
t.Fatal(w.Code, w.Body.String()) t.Fatal(w.Code, w.Body.String())
} }
w := httptest.NewRecorder() w := httptest.NewRecorder()
a.instances(w, runtimeRequest("POST", "/api/instances", fmt.Sprintf(`{"name":"legacy","image_id":%q}`, id), sid)) a.instances(w, runtimeRequest("POST", "/api/instances", fmt.Sprintf(`{"name":"current","image_id":%q}`, id), sid))
if w.Code != 200 { if w.Code != 200 {
t.Fatal(w.Code, w.Body.String()) t.Fatal(w.Code, w.Body.String())
} }
host := body["HostConfig"].(map[string]any) host := body["HostConfig"].(map[string]any)
mounts := host["Mounts"].([]any) mounts := host["Mounts"].([]any)
if len(mounts) != 2 || mounts[0].(map[string]any)["Target"] != "/data/dsh-home" || mounts[1].(map[string]any)["Target"] != "/workspace" || !strings.Contains(fmt.Sprint(body["Env"]), "DSH_HOME=/data/dsh-home") { if len(mounts) != 1 || mounts[0].(map[string]any)["Target"] != "/workspace" || !strings.Contains(fmt.Sprint(body["Env"]), "DSH_HOME=/workspace") {
t.Fatal(body) t.Fatal(body)
} }
var instance store.Instance var instance store.Instance
@@ -238,7 +254,7 @@ func TestAdmissionExperimentalAndPreflight(t *testing.T) {
handler http.HandlerFunc handler http.HandlerFunc
body string body string
want int want int
}{{a.builds, `{"version":"9.8.7"}`, 400}, {a.builds, `{"version":"9.8.7","experimental":true}`, 403}, {a.intents, `{"name":"unknown","version":"9.8.7"}`, 400}, {a.instances, fmt.Sprintf(`{"name":"unknown","image_id":%q}`, id), 400}} { }{{a.builds, `{"version":"9.8.7"}`, 405}, {a.builds, `{"version":"9.8.7","experimental":true}`, 405}, {a.instances, fmt.Sprintf(`{"name":"unknown","image_id":%q}`, id), 400}} {
w := httptest.NewRecorder() w := httptest.NewRecorder()
tc.handler(w, runtimeRequest("POST", "/api/test", tc.body, sid)) tc.handler(w, runtimeRequest("POST", "/api/test", tc.body, sid))
if w.Code != tc.want { if w.Code != tc.want {
@@ -249,9 +265,6 @@ func TestAdmissionExperimentalAndPreflight(t *testing.T) {
if err := a.s.DB.QueryRow("SELECT count(*) FROM image_build_requests").Scan(&n); err != nil || n != 0 { if err := a.s.DB.QueryRow("SELECT count(*) FROM image_build_requests").Scan(&n); err != nil || n != 0 {
t.Fatal(n, err) t.Fatal(n, err)
} }
if err := a.s.DB.QueryRow("SELECT count(*) FROM creation_intents").Scan(&n); err != nil || n != 0 {
t.Fatal(n, err)
}
if _, err := a.s.DB.Exec("UPDATE users SET admin=1"); err != nil { if _, err := a.s.DB.Exec("UPDATE users SET admin=1"); err != nil {
t.Fatal(err) t.Fatal(err)
} }
+7 -12
View File
@@ -34,13 +34,6 @@ type API struct {
// Protected by buildQueue; invalidates image observations across deletions. // Protected by buildQueue; invalidates image observations across deletions.
imageDeleteGeneration uint64 imageDeleteGeneration uint64
// Per-image admission evaluations cached for the intent poller; guarded
// by evalCacheMu and invalidated by imageDeleteGeneration (buildQueue)
// plus a short TTL so Docker observations stay fresh.
evalCacheMu sync.Mutex
evalCache map[string]evalCacheEntry
evalCacheGen uint64
} }
func New(s *store.Store, root string) *API { func New(s *store.Store, root string) *API {
@@ -70,12 +63,10 @@ func (a *API) Handler() http.Handler {
m.HandleFunc("/api/instances/", a.instance) m.HandleFunc("/api/instances/", a.instance)
m.HandleFunc("/api/images", a.images) m.HandleFunc("/api/images", a.images)
m.HandleFunc("/api/images/admission", a.imageAdmission) m.HandleFunc("/api/images/admission", a.imageAdmission)
m.HandleFunc("/api/compatibility", a.compatibility) m.HandleFunc("/api/releases/latest", a.latestRelease)
m.HandleFunc("/api/latest", a.latest) m.HandleFunc("/api/releases/build", a.releaseBuild)
m.HandleFunc("/api/builds", a.builds) m.HandleFunc("/api/builds", a.builds)
m.HandleFunc("/api/builds/", a.buildDetail) m.HandleFunc("/api/builds/", a.buildDetail)
m.HandleFunc("/api/intents", a.intents)
m.HandleFunc("/api/intents/", a.intent)
m.HandleFunc("/api/ws/", a.workspace) m.HandleFunc("/api/ws/", a.workspace)
m.HandleFunc("/api/users", a.users) m.HandleFunc("/api/users", a.users)
m.HandleFunc("/api/users/reset-password", a.resetPassword) m.HandleFunc("/api/users/reset-password", a.resetPassword)
@@ -235,7 +226,11 @@ func (a *API) login(w http.ResponseWriter, r *http.Request) {
fail(w, 500, "session error") fail(w, 500, "session error")
return return
} }
a.authCookies(w, r, sid, u.ID, csrf, token) if e = a.authCookies(w, r, sid, u.ID, csrf, token); e != nil {
a.s.DeleteSession(sid)
fail(w, 500, "session error")
return
}
a.s.AddAudit(u.ID, "login", u.ID, "") a.s.AddAudit(u.ID, "login", u.ID, "")
writeJSON(w, map[string]any{"user": u, "csrf_token": csrf}) writeJSON(w, map[string]any{"user": u, "csrf_token": csrf})
} }
+70 -27
View File
@@ -63,9 +63,51 @@ func (a *API) initAuth() error {
} }
_, err = a.s.DB.Exec(`CREATE TABLE IF NOT EXISTS auth_refresh(sid TEXT PRIMARY KEY REFERENCES sessions(id) ON DELETE CASCADE, token_hash TEXT NOT NULL); _, err = a.s.DB.Exec(`CREATE TABLE IF NOT EXISTS auth_refresh(sid TEXT PRIMARY KEY REFERENCES sessions(id) ON DELETE CASCADE, token_hash TEXT NOT NULL);
CREATE TABLE IF NOT EXISTS auth_failures(username TEXT PRIMARY KEY, failures INTEGER NOT NULL, locked_until INTEGER NOT NULL);`) CREATE TABLE IF NOT EXISTS auth_failures(username TEXT PRIMARY KEY, failures INTEGER NOT NULL, locked_until INTEGER NOT NULL);`)
if err == nil {
_, err = a.s.DB.Exec(`CREATE TABLE IF NOT EXISTS auth_refresh_previous(sid TEXT PRIMARY KEY REFERENCES auth_refresh(sid) ON DELETE CASCADE, token_hash TEXT NOT NULL);`)
}
return err return err
} }
// Derivation makes retries return the same successor without storing plaintext
// credentials. Domain and session binding prevent cross-session token reuse.
func (a *API) nextRefresh(sid, old string) string {
mac := hmac.New(sha256.New, a.authKey)
mac.Write([]byte("refresh-successor\x00" + sid + "\x00" + old))
return base64.RawURLEncoding.EncodeToString(mac.Sum(nil))
}
func (a *API) rotateRefresh(sid, old string) (string, error) {
tx, err := a.s.DB.Begin()
if err != nil {
return "", err
}
defer tx.Rollback()
var current, previous string
err = tx.QueryRow(`SELECT r.token_hash, COALESCE(p.token_hash, '')
FROM auth_refresh r JOIN sessions s ON s.id=r.sid
LEFT JOIN auth_refresh_previous p ON p.sid=r.sid
WHERE r.sid=? AND s.expires>?`, sid, time.Now().Unix()).Scan(&current, &previous)
if err != nil {
return "", err
}
hash := refreshHash(old)
next := a.nextRefresh(sid, old)
if hash == previous && refreshHash(next) == current {
return next, tx.Commit()
}
if hash != current {
return "", errAuthentication
}
if _, err = tx.Exec("UPDATE auth_refresh SET token_hash=? WHERE sid=?", refreshHash(next), sid); err != nil {
return "", err
}
if _, err = tx.Exec("INSERT INTO auth_refresh_previous(sid,token_hash) VALUES(?,?) ON CONFLICT(sid) DO UPDATE SET token_hash=excluded.token_hash", sid, hash); err != nil {
return "", err
}
return next, tx.Commit()
}
type accessClaims struct { type accessClaims struct {
Subject string `json:"sub"` Subject string `json:"sub"`
Session string `json:"sid"` Session string `json:"sid"`
@@ -130,18 +172,34 @@ func (a *API) secureCookies(r *http.Request) bool {
} }
return secure return secure
} }
func (a *API) authCookies(w http.ResponseWriter, r *http.Request, sid, uid, csrf, refresh string) { func (a *API) authCookies(w http.ResponseWriter, r *http.Request, sid, uid, csrf, refresh string) error {
// Persist recovery cookies across mobile browser process eviction, but never
// extend them beyond the database session's absolute expiry on refresh.
var expires int64
if err := a.s.DB.QueryRow("SELECT expires FROM sessions WHERE id=?", sid).Scan(&expires); err != nil {
return err
}
now := time.Now()
remaining := int(expires - now.Unix())
if remaining <= 0 {
return errAuthentication
}
for _, c := range []*http.Cookie{ for _, c := range []*http.Cookie{
{Name: "dsh_session", Value: sid, Path: "/", HttpOnly: true}, {Name: "dsh_session", Value: sid, Path: "/", HttpOnly: true, MaxAge: remaining},
{Name: "dsh_access", Value: a.accessJWT(sid, uid, time.Now().Add(accessLifetime)), Path: "/", HttpOnly: true, MaxAge: int(accessLifetime.Seconds())}, {Name: "dsh_access", Value: a.accessJWT(sid, uid, time.Now().Add(accessLifetime)), Path: "/", HttpOnly: true, MaxAge: int(accessLifetime.Seconds())},
{Name: "dsh_refresh", Value: refresh, Path: "/api", HttpOnly: true}, {Name: "dsh_refresh", Value: refresh, Path: "/api", HttpOnly: true, MaxAge: remaining},
{Name: "dsh_csrf", Value: csrf, Path: "/"}, {Name: "dsh_csrf", Value: csrf, Path: "/", MaxAge: remaining},
} { } {
c.Expires = time.Unix(expires, 0)
if c.Name == "dsh_access" {
c.Expires = now.Add(accessLifetime)
}
c.Secure = a.secureCookies(r) c.Secure = a.secureCookies(r)
c.SameSite = http.SameSiteLaxMode c.SameSite = http.SameSiteLaxMode
http.SetCookie(w, c) http.SetCookie(w, c)
} }
w.Header().Set("Cache-Control", "no-store") w.Header().Set("Cache-Control", "no-store")
return nil
} }
func (a *API) refresh(w http.ResponseWriter, r *http.Request) { func (a *API) refresh(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost { if r.Method != http.MethodPost {
@@ -181,34 +239,19 @@ func (a *API) refresh(w http.ResponseWriter, r *http.Request) {
fail(w, 401, "authentication required") fail(w, 401, "authentication required")
return return
} }
token, err := refreshToken() token, err := a.rotateRefresh(sid.Value, old.Value)
if err != nil { if err != nil {
fail(w, 500, "session error") if errors.Is(err, sql.ErrNoRows) || errors.Is(err, errAuthentication) {
return
}
result, err := a.s.DB.Exec("UPDATE auth_refresh SET token_hash=? WHERE sid=? AND token_hash=? AND EXISTS(SELECT 1 FROM sessions WHERE id=? AND expires>?)", refreshHash(token), sid.Value, refreshHash(old.Value), sid.Value, time.Now().Unix())
if err != nil {
fail(w, 500, "session error")
return
}
n, err := result.RowsAffected()
if err != nil {
fail(w, 500, "session error")
return
}
if n != 1 {
var active bool
err = a.s.DB.QueryRow("SELECT EXISTS(SELECT 1 FROM sessions JOIN auth_refresh ON auth_refresh.sid=sessions.id WHERE sessions.id=? AND sessions.expires>?)", sid.Value, time.Now().Unix()).Scan(&active)
if err != nil {
fail(w, 500, "session error")
} else if active {
fail(w, 409, "refresh_conflict")
} else {
fail(w, 401, "authentication required") fail(w, 401, "authentication required")
} else {
fail(w, 500, "session error")
} }
return return
} }
a.authCookies(w, r, sid.Value, u.ID, csrf, token) if err = a.authCookies(w, r, sid.Value, u.ID, csrf, token); err != nil {
fail(w, 500, "session error")
return
}
writeJSON(w, map[string]any{"user": u, "csrf_token": csrf}) writeJSON(w, map[string]any{"user": u, "csrf_token": csrf})
} }
func (a *API) accountLocked(username string) (bool, error) { func (a *API) accountLocked(username string) (bool, error) {
+71 -5
View File
@@ -107,11 +107,11 @@ func TestAccessRefreshLifecycle(t *testing.T) {
if cookieValue(fresh, "dsh_refresh") == cookieValue(cs, "dsh_refresh") { if cookieValue(fresh, "dsh_refresh") == cookieValue(cs, "dsh_refresh") {
t.Fatal("not rotated") t.Fatal("not rotated")
} }
if w = req("POST", "/api/refresh", "", csrf, cs); w.Code != 409 { if w = req("POST", "/api/refresh", "", csrf, cs); w.Code != 200 {
t.Fatal("replay accepted", w.Code) t.Fatal("retry rejected", w.Code)
} }
if len(w.Result().Cookies()) != 0 || !strings.Contains(w.Body.String(), "refresh_conflict") { if cookieValue(w.Result().Cookies(), "dsh_refresh") != cookieValue(fresh, "dsh_refresh") {
t.Fatal("conflict must not issue credentials", w.Header(), w.Body.String()) t.Fatal("retry must return the same successor")
} }
if w = req("GET", "/api/me", "", "", fresh); w.Code != 200 { if w = req("GET", "/api/me", "", "", fresh); w.Code != 200 {
t.Fatal(w.Code) t.Fatal(w.Code)
@@ -126,6 +126,72 @@ func TestAccessRefreshLifecycle(t *testing.T) {
t.Fatal("logout refresh valid") t.Fatal("logout refresh valid")
} }
} }
func TestPersistentCookiesRecoverAfterBrowserRestart(t *testing.T) {
a, req := authFixture(t)
w := req("POST", "/api/login", `{"username":"alice","password":"password-123"}`, "", nil)
if w.Code != 200 {
t.Fatal(w.Code)
}
cookies := w.Result().Cookies()
sid := cookieValue(cookies, "dsh_session")
var expires int64
if err := a.s.DB.QueryRow("SELECT expires FROM sessions WHERE id=?", sid).Scan(&expires); err != nil {
t.Fatal(err)
}
var restored []*http.Cookie
for _, c := range cookies {
if c.MaxAge <= 0 || c.Expires.IsZero() {
t.Fatalf("nonpersistent cookie: %s", c.Name)
}
if c.Name == "dsh_access" {
continue
} // Access has expired while backgrounded.
if c.Expires.Unix() != expires {
t.Fatalf("wrong expiry: %s", c.Name)
}
restored = append(restored, c)
}
if w = req("GET", "/api/me", "", "", restored); w.Code != 401 {
t.Fatal(w.Code)
}
// Simulate an older session without sleeping; rotation must not reset its TTL.
expires = time.Now().Add(time.Hour).Unix()
if _, err := a.s.DB.Exec("UPDATE sessions SET expires=? WHERE id=?", expires, sid); err != nil {
t.Fatal(err)
}
w = req("POST", "/api/refresh", "", cookieValue(restored, "dsh_csrf"), restored)
if w.Code != 200 {
t.Fatal(w.Code, w.Body.String())
}
fresh := w.Result().Cookies()
for _, c := range fresh {
if c.Name == "dsh_access" {
if c.MaxAge != int(accessLifetime.Seconds()) {
t.Fatal("access lifetime changed")
}
continue
}
if c.Expires.Unix() != expires || c.MaxAge <= 0 || c.MaxAge > 3600 {
t.Fatalf("session extended: %s", c.Name)
}
}
if w = req("GET", "/api/me", "", "", fresh); w.Code != 200 {
t.Fatal(w.Code)
}
w = req("POST", "/api/logout", "", cookieValue(fresh, "dsh_csrf"), fresh)
if w.Code != 200 {
t.Fatal(w.Code)
}
for _, c := range w.Result().Cookies() {
if c.MaxAge >= 0 {
t.Fatalf("cookie not cleared: %s", c.Name)
}
}
if w = req("GET", "/api/me", "", "", fresh); w.Code != 401 {
t.Fatal("revoked session accepted")
}
}
func TestAccountLockAndPersistentKey(t *testing.T) { func TestAccountLockAndPersistentKey(t *testing.T) {
a, req := authFixture(t) a, req := authFixture(t)
for i := 0; i < 5; i++ { for i := 0; i < 5; i++ {
@@ -198,7 +264,7 @@ func TestRefreshOriginConcurrentRotationAndJWTBinding(t *testing.T) {
for code := range codes { for code := range codes {
counts[code]++ counts[code]++
} }
if counts[200] != 1 || counts[409] != 1 { if counts[200] != 2 {
t.Fatal(counts) t.Fatal(counts)
} }
b := New(a.s, a.root) b := New(a.s, a.root)
+315
View File
@@ -0,0 +1,315 @@
package api
import (
"context"
"database/sql"
"errors"
"io/fs"
"net/http"
"os"
"regexp"
"strings"
"time"
"git.misaka.ren/m1saka/dsh-web/internal/store"
)
var snapshotIDPattern = regexp.MustCompile(`^(pre-restore-[a-f0-9]{32}|failed-restore-[a-f0-9]{32}|upgrade-[a-f0-9]{32}-snapshot|failed-home-[a-f0-9]{32})$`)
var errUnsafeManagementPath = errors.New("unsafe storage path")
func managementPathError(w http.ResponseWriter, err error) {
if errors.Is(err, errUnsafeManagementPath) {
fail(w, 400, "unsafe storage path")
return
}
fail(w, 500, "storage unavailable")
}
type snapshotInfo struct {
ID string `json:"id"`
Size int64 `json:"size"`
CreatedAt time.Time `json:"created_at"`
Protected bool `json:"protected"`
ProtectionReason string `json:"protection_reason"`
SizeError string `json:"size_error,omitempty"`
Restorable bool `json:"restorable"`
RestoreReason string `json:"restore_reason"`
SourceVersion string `json:"source_version,omitempty"`
SourceImage string `json:"source_image,omitempty"`
}
func managementRoot(base string, parts ...string) (*os.Root, error) {
root, err := os.OpenRoot(base)
if err != nil {
return nil, err
}
for _, part := range parts {
if part == "" || part == "." || part == ".." || strings.ContainsAny(part, "/\\") {
root.Close()
return nil, errUnsafeManagementPath
}
st, err := root.Lstat(part)
if err != nil {
root.Close()
return nil, err
}
if !st.IsDir() {
root.Close()
return nil, errUnsafeManagementPath
}
next, err := root.OpenRoot(part)
root.Close()
if err != nil {
return nil, err
}
actual, err := next.Stat(".")
if err != nil {
next.Close()
return nil, err
}
if !os.SameFile(st, actual) {
next.Close()
return nil, errUnsafeManagementPath
}
root = next
}
return root, nil
}
func (a *API) backupProtection(i store.Instance) string {
if i.Status == "error" {
return "instance recovery required"
}
var exists, pending int
if err := a.s.DB.QueryRow("SELECT count(*) FROM sqlite_master WHERE type='table' AND name='upgrade_journal'").Scan(&exists); err != nil {
return "recovery journal unavailable"
}
if exists != 0 {
if err := a.s.DB.QueryRow("SELECT count(*) FROM upgrade_journal WHERE instance_id=? AND (phase IS NULL OR phase NOT IN ('complete','rolled_back'))", i.ID).Scan(&pending); err != nil {
return "recovery journal unavailable"
}
if pending != 0 {
return "instance recovery required"
}
}
return ""
}
func snapshotBytes(ctx context.Context, root *os.Root, name string) (int64, error) {
var total int64
count := 0
err := fs.WalkDir(root.FS(), name, func(p string, d fs.DirEntry, err error) error {
if ctx.Err() != nil {
return ctx.Err()
}
if err != nil {
return err
}
count++
if count > 100000 || strings.Count(p, "/") > 128 {
return errors.New("tree limit")
}
st, err := d.Info()
if err != nil {
return err
}
if st.Mode().IsRegular() {
total += st.Size()
}
return nil
})
return total, err
}
func (a *API) backupManagement(w http.ResponseWriter, r *http.Request, i store.Instance, action string) {
w, flush := deferJSONResponse(w)
defer flush()
if r.Method != http.MethodDelete && !(action == "snapshots" && r.Method == http.MethodGet) {
fail(w, 405, "method not allowed")
return
}
id := r.URL.Query().Get("id")
if r.Method == http.MethodDelete && ((action == "snapshots" && !snapshotIDPattern.MatchString(id)) || (action == "backups" && !backupIDPattern.MatchString(id))) {
fail(w, 400, "invalid ID")
return
}
lock := a.instanceLocks.Acquire(i.ID)
defer lock.Release()
user := a.userLocks.Acquire(i.UID)
defer user.Release()
current, err := a.s.GetInstance(i.ID)
if err != nil || current.UID != i.UID {
fail(w, 404, "not found")
return
}
i = current
reason := a.backupProtection(i)
if r.Method == http.MethodDelete && reason != "" {
fail(w, 409, reason)
return
}
parts := []string{"workspaces", i.UID, "instances", i.ID}
if action == "backups" {
parts = []string{"backups", i.UID, i.ID}
}
root, err := managementRoot(a.root, parts...)
if errors.Is(err, os.ErrNotExist) {
if action == "backups" {
a.deleteBackup(w, i, nil, id)
return
}
if r.Method == http.MethodGet {
writeJSON(w, []snapshotInfo{})
return
}
fail(w, 404, "not found")
return
}
if err != nil {
managementPathError(w, err)
return
}
defer root.Close()
if r.Method == http.MethodGet {
f, err := root.Open(".")
if err != nil {
fail(w, 500, "snapshot listing failed")
return
}
entries, err := f.ReadDir(-1)
f.Close()
if err != nil {
fail(w, 500, "snapshot listing failed")
return
}
out := []snapshotInfo{}
for _, entry := range entries {
if !snapshotIDPattern.MatchString(entry.Name()) || !entry.IsDir() {
continue
}
st, err := entry.Info()
if err != nil {
fail(w, 500, "snapshot metadata unavailable")
return
}
s := snapshotInfo{ID: entry.Name(), CreatedAt: st.ModTime(), Protected: reason != "", ProtectionReason: reason}
s.SourceVersion, s.SourceImage, s.RestoreReason = a.snapshotCompatibility(i, s.ID)
s.Restorable = s.RestoreReason == "" || s.RestoreReason == snapshotUnknownReason
if reason != "" {
s.Restorable = false
s.RestoreReason = reason
}
s.Size, err = snapshotBytes(r.Context(), root, entry.Name())
if err != nil {
s.Size = -1
s.SizeError = "snapshot size unavailable: " + err.Error()
s.Restorable = false
s.RestoreReason = "snapshot size unavailable"
}
out = append(out, s)
}
writeJSON(w, out)
return
}
if action == "backups" {
a.deleteBackup(w, i, root, id)
return
}
st, err := root.Lstat(id)
if errors.Is(err, os.ErrNotExist) {
fail(w, 404, "snapshot not found")
return
}
if err != nil {
managementPathError(w, err)
return
}
if !st.IsDir() {
fail(w, 400, "unsafe snapshot path")
return
}
a.deleteSnapshot(w, i, id, func() error { return root.RemoveAll(id) })
}
func (a *API) deleteSnapshot(w http.ResponseWriter, i store.Instance, id string, remove func() error) {
audit := func(state string) error {
_, err := a.s.DB.Exec("INSERT INTO audit(uid,action,target,detail,at) VALUES(?,?,?,?,?)", i.UID, "snapshot.delete."+state, i.ID, id, time.Now().Unix())
return err
}
if err := audit("started"); err != nil {
fail(w, 500, "audit unavailable")
return
}
if err := remove(); err != nil {
if err := audit("failed"); err != nil {
fail(w, 500, "snapshot deletion failed; audit unavailable")
return
}
fail(w, 500, "snapshot deletion failed")
return
}
if err := audit("completed"); err != nil {
fail(w, 500, "snapshot removed; completion audit unavailable")
return
}
writeJSON(w, map[string]bool{"ok": true})
}
func (a *API) deleteBackup(w http.ResponseWriter, i store.Instance, root *os.Root, id string) {
name := id + ".dshbak"
var st os.FileInfo
statErr := error(os.ErrNotExist)
if root != nil {
st, statErr = root.Lstat(name)
}
if statErr != nil && !errors.Is(statErr, os.ErrNotExist) {
fail(w, 500, "backup unavailable")
return
}
if statErr == nil && !st.Mode().IsRegular() {
fail(w, 400, "unsafe backup path")
return
}
tx, err := a.s.DB.Begin()
if err != nil {
fail(w, 500, "backup index unavailable")
return
}
defer tx.Rollback()
var exists, indexed int
err = tx.QueryRow("SELECT count(*) FROM sqlite_master WHERE type='table' AND name='backups'").Scan(&exists)
if err == nil && exists != 0 {
var result sql.Result
result, err = tx.Exec("DELETE FROM backups WHERE id=? AND uid=? AND instance_id=?", id, i.UID, i.ID)
if err == nil {
n, e := result.RowsAffected()
err = e
indexed = int(n)
}
}
if err != nil {
fail(w, 500, "backup index unavailable")
return
}
if statErr != nil && indexed == 0 {
fail(w, 404, "backup not found")
return
}
if _, err = tx.Exec("INSERT INTO audit(uid,action,target,detail,at) VALUES(?,?,?,?,?)", i.UID, "backup.delete", i.ID, id, time.Now().Unix()); err != nil {
fail(w, 500, "audit unavailable")
return
}
if statErr == nil {
if err = root.Remove(name); err != nil {
fail(w, 500, "backup deletion failed")
return
}
}
// Filesystem and SQLite cannot commit atomically; a retry clears a stale index after a failed commit.
if err = tx.Commit(); err != nil {
fail(w, 500, "backup removed; index update failed, retry deletion")
return
}
writeJSON(w, map[string]bool{"ok": true})
}
+413
View File
@@ -0,0 +1,413 @@
package api
import (
"context"
"encoding/json"
"fmt"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
func TestSnapshotManagement(t *testing.T) {
a, i, home := configTestAPI(t, false)
a.docker = nil // These operations must never inspect or stop Docker.
parent := filepath.Dir(home)
suffix := strings.Repeat("a", 32)
names := []string{"pre-restore-" + suffix, "failed-restore-" + suffix, "upgrade-" + suffix + "-snapshot", "failed-home-" + suffix}
for _, name := range append(append([]string{}, names...), "pre-restore-no", "upgrade-"+suffix, "other") {
if err := os.Mkdir(filepath.Join(parent, name), 0700); err != nil {
t.Fatal(err)
}
configWrite(t, filepath.Join(parent, name, "data"), "abc")
}
outside := t.TempDir()
configWrite(t, filepath.Join(outside, "keep"), "outside")
if err := os.Symlink(outside, filepath.Join(parent, names[0], "link")); err != nil {
t.Fatal(err)
}
if err := os.Link(filepath.Join(parent, names[0], "data"), filepath.Join(parent, names[0], "hardlink")); err != nil {
t.Fatal(err)
}
w := configCall(a, i, "GET", "snapshots", "")
var got []snapshotInfo
if w.Code != 200 || json.Unmarshal(w.Body.Bytes(), &got) != nil || len(got) != 4 {
t.Fatal(w.Code, w.Body.String())
}
for _, s := range got {
want := int64(3)
if s.ID == names[0] {
want = 6
}
if s.Size != want || s.Protected || s.ProtectionReason != "" || s.SizeError != "" || s.CreatedAt.IsZero() {
t.Fatalf("%+v", s)
}
}
for _, name := range names {
w = configCall(a, i, "DELETE", "snapshots?id="+name, "")
if w.Code != 200 {
t.Fatal(w.Code, w.Body.String())
}
if _, err := os.Stat(filepath.Join(parent, name)); !os.IsNotExist(err) {
t.Fatal(err)
}
}
for _, p := range []string{home, filepath.Join(parent, "other"), filepath.Join(outside, "keep")} {
if _, err := os.Stat(p); err != nil {
t.Fatal(err)
}
}
var count int
if err := a.s.DB.QueryRow("SELECT count(*) FROM audit WHERE action='snapshot.delete.completed' AND uid=? AND target=?", i.UID, i.ID).Scan(&count); err != nil || count != 4 {
t.Fatal(count, err)
}
if w = configCall(a, i, "DELETE", "snapshots?id="+names[0], ""); w.Code != 404 {
t.Fatal(w.Code)
}
}
func TestManagementInvalidPaths(t *testing.T) {
a, i, home := configTestAPI(t, false)
a.docker = nil
for _, action := range []string{"snapshots", "backups"} {
for _, id := range []string{"", "home", "..", "../home", "/home", strings.Repeat("A", 32), "pre-restore-" + strings.Repeat("a", 32) + "/../home"} {
if w := configCall(a, i, "DELETE", action+"?id="+url.QueryEscape(id), ""); w.Code != 400 {
t.Fatal(action, id, w.Code)
}
}
}
id := "failed-home-" + strings.Repeat("b", 32)
outside := t.TempDir()
configWrite(t, filepath.Join(outside, "keep"), "safe")
if err := os.Symlink(outside, filepath.Join(filepath.Dir(home), id)); err != nil {
t.Fatal(err)
}
if w := configCall(a, i, "DELETE", "snapshots?id="+id, ""); w.Code != 400 {
t.Fatal(w.Code, w.Body.String())
}
if w := configCall(a, i, "GET", "snapshots", ""); w.Code != 200 || strings.TrimSpace(w.Body.String()) != "[]" {
t.Fatal(w.Code, w.Body.String())
}
if err := os.Rename(filepath.Dir(home), filepath.Dir(home)+"-saved"); err != nil {
t.Fatal(err)
}
if err := os.Symlink(outside, filepath.Dir(home)); err != nil {
t.Fatal(err)
}
for _, method := range []string{"GET", "DELETE"} {
if w := configCall(a, i, method, "snapshots?id="+id, ""); w.Code != 400 {
t.Fatal(w.Code, w.Body.String())
}
}
if _, err := os.Stat(filepath.Join(outside, "keep")); err != nil {
t.Fatal(err)
}
}
func TestManagementProtection(t *testing.T) {
for _, state := range []string{"error", "pending", "journal-error", "complete", "rolled_back"} {
t.Run(state, func(t *testing.T) {
a, i, home := configTestAPI(t, false)
id := "pre-restore-" + strings.Repeat("c", 32)
if err := os.Mkdir(filepath.Join(filepath.Dir(home), id), 0700); err != nil {
t.Fatal(err)
}
var err error
if state == "error" {
_, err = a.s.DB.Exec("UPDATE instances SET status='error' WHERE id=?", i.ID)
} else if state == "journal-error" {
_, err = a.s.DB.Exec("CREATE TABLE upgrade_journal(broken TEXT)")
} else {
_, err = a.s.DB.Exec("CREATE TABLE upgrade_journal(instance_id TEXT,phase TEXT)")
if err == nil {
_, err = a.s.DB.Exec("INSERT INTO upgrade_journal VALUES(?,?)", i.ID, state)
}
}
if err != nil {
t.Fatal(err)
}
protected := state != "complete" && state != "rolled_back"
w := configCall(a, i, "GET", "snapshots", "")
var got []snapshotInfo
if w.Code != 200 || json.Unmarshal(w.Body.Bytes(), &got) != nil || len(got) != 1 || got[0].Protected != protected || (got[0].ProtectionReason != "") != protected {
t.Fatal(w.Code, w.Body.String())
}
w = configCall(a, i, "DELETE", "snapshots?id="+id, "")
want := 200
if protected {
want = 409
}
if w.Code != want {
t.Fatal(w.Code, w.Body.String())
}
if protected {
w = configCall(a, i, "DELETE", "backups?id="+strings.Repeat("a", 32), "")
if w.Code != 409 {
t.Fatal(w.Code, w.Body.String())
}
}
})
}
}
func TestSnapshotSizeError(t *testing.T) {
a, i, home := configTestAPI(t, false)
id := "failed-restore-" + strings.Repeat("d", 32)
deep := filepath.Join(filepath.Dir(home), id)
for n := 0; n < 130; n++ {
deep = filepath.Join(deep, "d")
}
if err := os.MkdirAll(deep, 0700); err != nil {
t.Fatal(err)
}
w := configCall(a, i, "GET", "snapshots", "")
var got []snapshotInfo
if w.Code != 200 || json.Unmarshal(w.Body.Bytes(), &got) != nil || len(got) != 1 || got[0].Size != -1 || got[0].SizeError == "" {
t.Fatal(w.Code, w.Body.String())
}
}
func TestBackupDelete(t *testing.T) {
for _, mode := range []string{"indexed", "legacy", "stale", "missing-parent", "index-error", "symlink"} {
t.Run(mode, func(t *testing.T) {
a, i, _ := configTestAPI(t, false)
a.docker = nil
id := strings.Repeat("e", 32)
dir := filepath.Join(a.root, "backups", i.UID, i.ID)
if err := os.MkdirAll(dir, 0700); err != nil {
t.Fatal(err)
}
file := filepath.Join(dir, id+".dshbak")
if mode != "stale" && mode != "symlink" {
configWrite(t, file, "encrypted")
}
if mode == "symlink" {
if err := os.Symlink(t.TempDir(), file); err != nil {
t.Fatal(err)
}
}
if mode == "indexed" || mode == "stale" || mode == "missing-parent" {
if err := a.indexBackup(i, id, "manual", 2, 9); err != nil {
t.Fatal(err)
}
}
if mode == "index-error" {
if _, err := a.s.DB.Exec("CREATE TABLE backups(broken TEXT)"); err != nil {
t.Fatal(err)
}
}
if mode == "missing-parent" {
if err := os.RemoveAll(filepath.Join(a.root, "backups")); err != nil {
t.Fatal(err)
}
}
w := configCall(a, i, "DELETE", "backups?id="+id, "")
want := 200
if mode == "index-error" {
want = 500
}
if mode == "symlink" {
want = 400
}
if w.Code != want {
t.Fatal(w.Code, w.Body.String())
}
if want != 200 {
if _, err := os.Lstat(file); err != nil {
t.Fatal(err)
}
return
}
if _, err := os.Stat(file); !os.IsNotExist(err) {
t.Fatal(err)
}
if mode != "legacy" {
var count int
if err := a.s.DB.QueryRow("SELECT count(*) FROM backups WHERE id=?", id).Scan(&count); err != nil || count != 0 {
t.Fatal(count, err)
}
}
var count int
if err := a.s.DB.QueryRow("SELECT count(*) FROM audit WHERE action='backup.delete' AND target=? AND detail=?", i.ID, id).Scan(&count); err != nil || count != 1 {
t.Fatal(count, err)
}
})
}
}
func TestManagementOwnershipRecheck(t *testing.T) {
a, i, _ := configTestAPI(t, false)
if _, err := a.s.DB.Exec("UPDATE instances SET uid='someone-else' WHERE id=?", i.ID); err != nil {
t.Fatal(err)
}
for _, action := range []string{"snapshots", "snapshots?id=pre-restore-" + strings.Repeat("a", 32), "backups?id=" + strings.Repeat("a", 32)} {
method := "DELETE"
if action == "snapshots" {
method = "GET"
}
if w := configCall(a, i, method, action, ""); w.Code != 404 {
t.Fatal(w.Code, w.Body.String())
}
}
}
func TestManagementAuditFailure(t *testing.T) {
a, i, home := configTestAPI(t, false)
id := strings.Repeat("f", 32)
snapshot := filepath.Join(filepath.Dir(home), "pre-restore-"+id)
if err := os.Mkdir(snapshot, 0700); err != nil {
t.Fatal(err)
}
dir := filepath.Join(a.root, "backups", i.UID, i.ID)
if err := os.MkdirAll(dir, 0700); err != nil {
t.Fatal(err)
}
file := filepath.Join(dir, id+".dshbak")
configWrite(t, file, "encrypted")
if err := a.indexBackup(i, id, "manual", 2, 9); err != nil {
t.Fatal(err)
}
if _, err := a.s.DB.Exec("DROP TABLE audit"); err != nil {
t.Fatal(err)
}
for _, action := range []string{"snapshots?id=pre-restore-" + id, "backups?id=" + id} {
if w := configCall(a, i, "DELETE", action, ""); w.Code != 500 {
t.Fatal(w.Code, w.Body.String())
}
}
for _, p := range []string{snapshot, file} {
if _, err := os.Stat(p); err != nil {
t.Fatal(err)
}
}
var count int
if err := a.s.DB.QueryRow("SELECT count(*) FROM backups WHERE id=?", id).Scan(&count); err != nil || count != 1 {
t.Fatal(count, err)
}
}
func TestSnapshotScanCancelled(t *testing.T) {
root, err := os.OpenRoot(t.TempDir())
if err != nil {
t.Fatal(err)
}
defer root.Close()
ctx, cancel := context.WithCancel(context.Background())
cancel()
if _, err := snapshotBytes(ctx, root, "."); err != context.Canceled {
t.Fatal(err)
}
}
func TestSnapshotDeletionReleasesSQLConnection(t *testing.T) {
a, i, _ := configTestAPI(t, false)
a.s.DB.SetMaxOpenConns(1)
entered, release, done := make(chan struct{}), make(chan struct{}), make(chan struct{})
w := httptest.NewRecorder()
go func() {
defer close(done)
a.deleteSnapshot(w, i, "snapshot", func() error {
close(entered)
<-release
return nil
})
}()
defer func() { close(release); <-done }()
select {
case <-entered:
case <-time.After(5 * time.Second):
t.Fatal("removal did not start")
}
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
var count int
if err := a.s.DB.QueryRowContext(ctx, "SELECT count(*) FROM audit WHERE action='snapshot.delete.started'").Scan(&count); err != nil || count != 1 {
t.Fatalf("SQL blocked during removal or initial audit not persisted: count=%d err=%v", count, err)
}
}
func TestSnapshotDeletionAuditOutcomes(t *testing.T) {
for _, mode := range []string{"success", "remove-failed", "start-failed", "completion-audit-failed", "failure-audit-failed"} {
t.Run(mode, func(t *testing.T) {
a, i, _ := configTestAPI(t, false)
if mode == "start-failed" {
if _, err := a.s.DB.Exec("DROP TABLE audit"); err != nil {
t.Fatal(err)
}
}
w := httptest.NewRecorder()
called := false
a.deleteSnapshot(w, i, "snapshot", func() error {
called = true
if strings.Contains(mode, "audit-failed") {
if _, err := a.s.DB.Exec(`CREATE TRIGGER reject_terminal BEFORE INSERT ON audit WHEN NEW.action != 'snapshot.delete.started' BEGIN SELECT RAISE(FAIL, 'audit unavailable'); END`); err != nil {
t.Fatal(err)
}
}
if mode == "remove-failed" || mode == "failure-audit-failed" {
return os.ErrPermission
}
return nil
})
want := 500
if mode == "success" {
want = 200
}
if w.Code != want || called != (mode != "start-failed") {
t.Fatal(w.Code, w.Body.String(), called)
}
if mode == "start-failed" {
return
}
for _, state := range []string{"started", "completed", "failed"} {
wantCount := 0
if state == "started" || (state == "completed" && mode == "success") || (state == "failed" && mode == "remove-failed") {
wantCount = 1
}
var count int
if err := a.s.DB.QueryRow("SELECT count(*) FROM audit WHERE action=? AND uid=? AND target=? AND detail=?", "snapshot.delete."+state, i.UID, i.ID, "snapshot").Scan(&count); err != nil || count != wantCount {
t.Fatal(state, count, err)
}
}
})
}
}
func TestManagementPathErrorClassification(t *testing.T) {
for _, tc := range []struct {
name string
err error
want int
}{
{"unsafe", errUnsafeManagementPath, 400},
{"wrapped-unsafe", fmt.Errorf("path: %w", errUnsafeManagementPath), 400},
{"permission", &os.PathError{Op: "lstat", Path: "snapshot", Err: os.ErrPermission}, 500},
{"closed", os.ErrClosed, 500},
} {
t.Run(tc.name, func(t *testing.T) {
w := httptest.NewRecorder()
managementPathError(w, tc.err)
if w.Code != tc.want {
t.Fatal(w.Code, w.Body.String())
}
})
}
a, i, _ := configTestAPI(t, false)
base := t.TempDir()
configWrite(t, filepath.Join(base, "file"), "not a directory")
a.root = filepath.Join(base, "file", "child")
if w := configCall(a, i, "GET", "snapshots", ""); w.Code != 500 {
t.Fatal(w.Code, w.Body.String())
}
a.root = filepath.Join(base, "missing")
if w := configCall(a, i, "GET", "snapshots", ""); w.Code != 200 || strings.TrimSpace(w.Body.String()) != "[]" {
t.Fatal(w.Code, w.Body.String())
}
if w := configCall(a, i, "DELETE", "snapshots?id=pre-restore-"+strings.Repeat("a", 32), ""); w.Code != 404 {
t.Fatal(w.Code, w.Body.String())
}
}
+3 -3
View File
@@ -11,10 +11,10 @@ import (
) )
// Summary queries never load log bodies; history cannot inflate a page response. // Summary queries never load log bodies; history cannot inflate a page response.
const buildSummaryColumns = "id,version,status,error,image_id,created_at,requested_by,COALESCE(recipe_rev,''),COALESCE(platform,''),COALESCE(finished_at,''),experimental" const buildSummaryColumns = "id,version,status,error,image_id,created_at,requested_by,COALESCE(recipe_rev,''),COALESCE(platform,''),COALESCE(finished_at,''),experimental,release_id,source_repository,source_commit"
func scanBuildSummary(row interface{ Scan(...any) error }, b *buildRecord) error { func scanBuildSummary(row interface{ Scan(...any) error }, b *buildRecord) error {
return row.Scan(&b.ID, &b.Version, &b.Status, &b.Error, &b.ImageID, &b.Created, &b.RequestedBy, &b.RecipeRev, &b.Platform, &b.Finished, &b.Experimental) return row.Scan(&b.ID, &b.Version, &b.Status, &b.Error, &b.ImageID, &b.Created, &b.RequestedBy, &b.RecipeRev, &b.Platform, &b.Finished, &b.Experimental, &b.ReleaseID, &b.SourceRepository, &b.SourceCommit)
} }
func (a *API) listBuilds(w http.ResponseWriter, r *http.Request) { func (a *API) listBuilds(w http.ResponseWriter, r *http.Request) {
@@ -97,7 +97,7 @@ func (a *API) buildDetail(w http.ResponseWriter, r *http.Request) {
buildQueue.Unlock() buildQueue.Unlock()
var b buildRecord var b buildRecord
if err == nil { if err == nil {
err = a.s.DB.QueryRowContext(ctx, "SELECT "+buildSummaryColumns+",logs FROM image_builds WHERE id=?", id).Scan(&b.ID, &b.Version, &b.Status, &b.Error, &b.ImageID, &b.Created, &b.RequestedBy, &b.RecipeRev, &b.Platform, &b.Finished, &b.Experimental, &b.Logs) err = a.s.DB.QueryRowContext(ctx, "SELECT "+buildSummaryColumns+",logs FROM image_builds WHERE id=?", id).Scan(&b.ID, &b.Version, &b.Status, &b.Error, &b.ImageID, &b.Created, &b.RequestedBy, &b.RecipeRev, &b.Platform, &b.Finished, &b.Experimental, &b.ReleaseID, &b.SourceRepository, &b.SourceCommit, &b.Logs)
} }
if errors.Is(err, sql.ErrNoRows) { if errors.Is(err, sql.ErrNoRows) {
fail(w, 404, "build not found") fail(w, 404, "build not found")
+24 -1
View File
@@ -23,6 +23,18 @@ var backupIDPattern = regexp.MustCompile("^[a-f0-9]{32}$")
var configBodySlots = make(chan struct{}, 4) var configBodySlots = make(chan struct{}, 4)
func (a *API) config(w http.ResponseWriter, r *http.Request, i store.Instance, action string) { func (a *API) config(w http.ResponseWriter, r *http.Request, i store.Instance, action string) {
if action == "snapshots" && r.Method == http.MethodGet && r.URL.Query().Get("download") != "" {
a.snapshotDownload(w, r, i)
return
}
if action == "snapshots" && r.Method == http.MethodPost {
a.snapshotRestore(w, r, i)
return
}
if action == "snapshots" || (action == "backups" && r.Method == http.MethodDelete) {
a.backupManagement(w, r, i, action)
return
}
controller := http.NewResponseController(w) controller := http.NewResponseController(w)
if r.Method != http.MethodGet || action != "backups" || r.URL.Query().Get("download") == "" { if r.Method != http.MethodGet || action != "backups" || r.URL.Query().Get("download") == "" {
var flush func() var flush func()
@@ -205,6 +217,9 @@ func (a *API) config(w http.ResponseWriter, r *http.Request, i store.Instance, a
return return
} }
var document any var document any
if strings.TrimSpace(in.Content) == "" {
in.Content = "[]\n"
}
if yaml.Unmarshal([]byte(in.Content), &document) != nil { if yaml.Unmarshal([]byte(in.Content), &document) != nil {
fail(w, 400, "invalid YAML") fail(w, 400, "invalid YAML")
return return
@@ -465,6 +480,15 @@ func (a *API) config(w http.ResponseWriter, r *http.Request, i store.Instance, a
return return
} }
snapshot := "pre-restore-" + store.ID() snapshot := "pre-restore-" + store.ID()
failed := "failed-restore-" + store.ID()
if e = a.recordSnapshot(i, snapshot, i.Version, i.ImageID); e != nil {
fail(w, 500, "snapshot metadata unavailable")
return
}
if e = a.recordSnapshot(i, failed, i.Version, i.ImageID); e != nil {
fail(w, 500, "snapshot metadata unavailable")
return
}
if _, e = a.quotaSnapshot(i.UID, parent, restoredBytes); e != nil { if _, e = a.quotaSnapshot(i.UID, parent, restoredBytes); e != nil {
fail(w, 413, "quota exceeded or snapshot size unavailable") fail(w, 413, "quota exceeded or snapshot size unavailable")
return return
@@ -533,7 +557,6 @@ func (a *API) config(w http.ResponseWriter, r *http.Request, i store.Instance, a
fail(w, 500, "restore rollback requires runtime stop") fail(w, 500, "restore rollback requires runtime stop")
return return
} }
failed := "failed-restore-" + store.ID()
if parent.Rename("home", failed) != nil || parent.Rename(snapshot, "home") != nil || upgradeSync(parent) != nil { if parent.Rename("home", failed) != nil || parent.Rename(snapshot, "home") != nil || upgradeSync(parent) != nil {
a.s.SetInstance(i.ID, "error") a.s.SetInstance(i.ID, "error")
fail(w, 500, "restore rollback requires administrator recovery") fail(w, 500, "restore rollback requires administrator recovery")
+6 -1
View File
@@ -3,7 +3,9 @@ package api
import ( import (
"context" "context"
"errors" "errors"
"git.misaka.ren/m1saka/dsh-web/internal/docker"
"git.misaka.ren/m1saka/dsh-web/internal/store" "git.misaka.ren/m1saka/dsh-web/internal/store"
"net/http"
"net/url" "net/url"
"os" "os"
"path/filepath" "path/filepath"
@@ -57,7 +59,10 @@ func (a *API) configStop(ctx context.Context, id string) error {
ctx, cancel := context.WithTimeout(ctx, 30*time.Second) ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel() defer cancel()
if err := a.docker.Do(ctx, "POST", "/containers/"+url.PathEscape(id)+"/stop?t=10", nil, nil); err != nil { if err := a.docker.Do(ctx, "POST", "/containers/"+url.PathEscape(id)+"/stop?t=10", nil, nil); err != nil {
return err var apiErr *docker.APIError
if !errors.As(err, &apiErr) || apiErr.StatusCode != http.StatusNotModified {
return err
}
} }
c, err := a.docker.Inspect(ctx, id) c, err := a.docker.Inspect(ctx, id)
if err != nil { if err != nil {
+42
View File
@@ -5,6 +5,7 @@ import (
"context" "context"
"encoding/json" "encoding/json"
"fmt" "fmt"
"git.misaka.ren/m1saka/dsh-web/internal/docker"
"git.misaka.ren/m1saka/dsh-web/internal/store" "git.misaka.ren/m1saka/dsh-web/internal/store"
"golang.org/x/crypto/bcrypt" "golang.org/x/crypto/bcrypt"
"mime/multipart" "mime/multipart"
@@ -20,6 +21,47 @@ import (
"time" "time"
) )
func TestConfigStopAlreadyStopped(t *testing.T) {
for _, state := range []string{`{}`, `{"Running":true}`, `{"Paused":true}`, `{"Restarting":true}`} {
t.Run(state, func(t *testing.T) {
inspected := false
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == "POST" {
w.WriteHeader(http.StatusNotModified)
return
}
inspected = true
fmt.Fprintf(w, `{"State":%s}`, state)
}))
defer srv.Close()
a := &API{docker: docker.New(srv.URL)}
err := a.configStop(context.Background(), "test")
if !inspected || (err == nil) != (state == `{}`) {
t.Fatalf("inspected=%v error=%v", inspected, err)
}
})
}
}
func TestConfigEmptyProfileRoundTrip(t *testing.T) {
if os.Geteuid() != 0 {
t.Skip("requires chown")
}
a, i, home := configTestAPI(t, false)
w := configCall(a, i, "GET", "profile", "")
if w.Code != 200 {
t.Fatal(w.Code, w.Body.String())
}
w = configCall(a, i, "PUT", "profile", w.Body.String())
if w.Code != 200 {
t.Fatal(w.Code, w.Body.String())
}
content, err := os.ReadFile(filepath.Join(home, "profiles/web/cordis.patch.yml"))
if err != nil || string(content) != "[]\n" {
t.Fatalf("content=%q error=%v", content, err)
}
}
func TestConfigPasswordBackupImport(t *testing.T) { func TestConfigPasswordBackupImport(t *testing.T) {
if os.Geteuid() != 0 { if os.Geteuid() != 0 {
t.Skip("requires chown") t.Skip("requires chown")
+101
View File
@@ -0,0 +1,101 @@
package api
import (
"archive/zip"
"bytes"
"io"
"mime"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"testing"
"git.misaka.ren/m1saka/dsh-web/internal/store"
)
func TestWorkspaceDownloadFilename(t *testing.T) {
a, sid := runtimeTestAPI(t, func(http.ResponseWriter, *http.Request) { t.Error("unexpected Docker call") })
if err := a.s.AddInstance(store.Instance{ID: "one", UID: "owner"}); err != nil {
t.Fatal(err)
}
home := filepath.Join(a.root, "workspaces/owner/instances/one/home")
checkFilename := func(t *testing.T, w *httptest.ResponseRecorder, want string) {
t.Helper()
header := w.Header().Get("Content-Disposition")
mediaType, params, err := mime.ParseMediaType(header)
if err != nil {
t.Fatalf("parse Content-Disposition %q: %v", header, err)
}
if mediaType != "attachment" || params["filename"] != want {
t.Fatalf("Content-Disposition = %q, want attachment with filename %q", header, want)
}
}
const content = "0123456789\nunchanged 下载 content\x00\xff"
for _, name := range []string{"report.txt", "中文文件.txt", "report with spaces.txt", `report";100%.txt`, "README"} {
t.Run(name, func(t *testing.T) {
p := "parent/nested/" + name
file := filepath.Join(home, filepath.FromSlash(p))
if err := os.MkdirAll(filepath.Dir(file), 0750); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(file, []byte(content), 0600); err != nil {
t.Fatal(err)
}
for _, partial := range []bool{false, true} {
r := runtimeRequest(http.MethodGet, "/api/ws/download?instance=one&path="+url.QueryEscape(p), "", sid)
wantStatus, wantBody := http.StatusOK, content
if partial {
r.Header.Set("Range", "bytes=2-6")
wantStatus, wantBody = http.StatusPartialContent, content[2:7]
}
w := httptest.NewRecorder()
a.workspace(w, r)
if w.Code != wantStatus {
t.Fatalf("partial=%v: status = %d, want %d: %s", partial, w.Code, wantStatus, w.Body.String())
}
checkFilename(t, w, name)
if w.Body.String() != wantBody {
t.Errorf("partial=%v: body = %q, want %q", partial, w.Body.String(), wantBody)
}
}
})
}
t.Run("directory zip", func(t *testing.T) {
const name = `资料 ";100%`
p := "parent/" + name
dir := filepath.Join(home, filepath.FromSlash(p))
if err := os.MkdirAll(dir, 0750); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "entry.txt"), []byte(content), 0600); err != nil {
t.Fatal(err)
}
w := httptest.NewRecorder()
a.workspace(w, runtimeRequest(http.MethodGet, "/api/ws/download?instance=one&path="+url.QueryEscape(p), "", sid))
if w.Code != http.StatusOK {
t.Fatalf("status = %d: %s", w.Code, w.Body.String())
}
checkFilename(t, w, name+".zip")
archive, err := zip.NewReader(bytes.NewReader(w.Body.Bytes()), int64(w.Body.Len()))
if err != nil {
t.Fatal(err)
}
if len(archive.File) != 1 || archive.File[0].Name != "entry.txt" {
t.Fatalf("unexpected archive entries: %v", archive.File)
}
r, err := archive.File[0].Open()
if err != nil {
t.Fatal(err)
}
defer r.Close()
got, err := io.ReadAll(r)
if err != nil {
t.Fatal(err)
}
if string(got) != content {
t.Errorf("archive body = %q, want %q", got, content)
}
})
}
-97
View File
@@ -1,97 +0,0 @@
package api
import (
"context"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
func TestRuntimePrestart(t *testing.T) {
for _, status := range []string{"0", "23", "124", "137", "empty"} {
t.Run(status, func(t *testing.T) {
dir := t.TempDir()
if status != "empty" {
if err := os.MkdirAll(filepath.Join(dir, "profiles/web"), 0700); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "profiles/web/package.json"), []byte("{}"), 0600); err != nil {
t.Fatal(err)
}
}
scripts := map[string]string{
"node": "#!/bin/bash\nexit 0\n",
"timeout": `#!/bin/bash
printf "timeout %s\n" "$*" >> "$TRACE"
test "$1 $2 $3" = "--signal=TERM --kill-after=5s 120s" || exit 90
shift 3
exec "$@"
`,
"dsh": `#!/bin/bash
printf "dsh %s\n" "$*" >> "$TRACE"
test "$npm_config_ignore_scripts:$npm_config_ignore_pnpmfile" = true:true || exit 91
if [ "$1" = plugin ]; then
test "$*" = "plugin --profile web install --ignore-scripts" || exit 92
exit "$PLUGIN_STATUS"
fi
exit 0
`,
"caddy": "#!/bin/bash\necho caddy >> \"$TRACE\"\nsleep 0.1\n",
}
for name, script := range scripts {
if err := os.WriteFile(filepath.Join(dir, name), []byte(script), 0700); err != nil {
t.Fatal(err)
}
}
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "bash", "../../image/entrypoint.sh", "web", "--no-open", "--port", "3080")
cmd.Env = append(os.Environ(), "PATH="+dir+":"+os.Getenv("PATH"), "DSH_HOME="+dir, "TRACE="+filepath.Join(dir, "trace"), "PLUGIN_STATUS="+status)
output, err := cmd.CombinedOutput()
data, readErr := os.ReadFile(filepath.Join(dir, "trace"))
if readErr != nil {
t.Fatal(readErr)
}
trace := string(data)
if status == "empty" {
if err != nil || strings.Contains(trace, "plugin") || !strings.Contains(trace, "dsh web") {
t.Fatalf("%v: %s: %s", err, trace, output)
}
return
}
if !strings.Contains(trace, "dsh plugin --profile web install --ignore-scripts") {
t.Fatal(trace)
}
if status == "0" {
if err != nil || !strings.Contains(string(output), "reconciliation complete") {
t.Fatalf("%v: %s", err, output)
}
if !strings.Contains(trace, "dsh web") {
t.Fatal(trace)
}
} else {
if err == nil || !strings.Contains(string(output), "failed (exit "+status+")") {
t.Fatalf("%v: %s", err, output)
}
if strings.Contains(trace, "caddy") || strings.Contains(trace, "dsh web") {
t.Fatalf("application started after failure: %s", trace)
}
}
})
}
}
func TestRuntimePNPMPinned(t *testing.T) {
data, err := os.ReadFile("../../image/Dockerfile")
if err != nil {
t.Fatal(err)
}
for _, required := range []string{"[email protected]", `test "$(pnpm --version)" = "10.11.0"`, "USER node", `"/usr/bin/tini", "-g"`} {
if !strings.Contains(string(data), required) {
t.Errorf("missing %s", required)
}
}
}
+193 -30
View File
@@ -3,8 +3,10 @@ package api
import ( import (
"archive/zip" "archive/zip"
"context" "context"
"crypto/sha256"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt"
"git.misaka.ren/m1saka/dsh-web/internal/store" "git.misaka.ren/m1saka/dsh-web/internal/store"
"io" "io"
"mime" "mime"
@@ -15,6 +17,8 @@ import (
"strings" "strings"
"syscall" "syscall"
"time" "time"
"unicode"
"unicode/utf8"
"golang.org/x/sys/unix" "golang.org/x/sys/unix"
) )
@@ -80,40 +84,21 @@ func safeRel(p string) (string, error) {
return c, nil return c, nil
} }
func workspacePath(uid string) string { return path.Join("workspaces", uid, "files") } func (a *API) openWorkspace(u store.User, instanceID string) (*os.Root, error) {
if instanceID == "" {
func (a *API) openWorkspace(u store.User, instanceIDs ...string) (*os.Root, error) { return nil, errors.New("instance required")
instanceID := ""
if len(instanceIDs) > 0 {
instanceID = instanceIDs[0]
} }
base, err := os.OpenRoot(a.root) base, err := os.OpenRoot(a.root)
if err != nil { if err != nil {
return nil, err return nil, err
} }
defer base.Close() defer base.Close()
parts := []string{"workspaces", path.Join("workspaces", u.ID), workspacePath(u.ID)} rel := path.Join("workspaces", u.ID, "instances", instanceID, "home")
if instanceID != "" { if err := rootSafe(base, rel, false); err != nil {
rel := path.Join("workspaces", u.ID, "instances", instanceID, "home")
if err := rootSafe(base, rel, false); err != nil {
return nil, err
}
return base.OpenRoot(rel)
}
for _, p := range parts {
if err = rootSafe(base, p, true); err != nil {
return nil, err
}
if err = base.Mkdir(p, 0750); err != nil && !errors.Is(err, os.ErrExist) {
return nil, err
}
}
if err = base.Chown(workspacePath(u.ID), 1000, 1000); err != nil {
return nil, err return nil, err
} }
return base.OpenRoot(workspacePath(u.ID)) return base.OpenRoot(rel)
} }
func rootSafe(root *os.Root, rel string, finalMayMissing bool) error { func rootSafe(root *os.Root, rel string, finalMayMissing bool) error {
if rel == "." { if rel == "." {
return nil return nil
@@ -136,7 +121,7 @@ func rootSafe(root *os.Root, rel string, finalMayMissing bool) error {
} }
func (a *API) workspace(w http.ResponseWriter, r *http.Request) { func (a *API) workspace(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/api/ws/download" { if r.URL.Path != "/api/ws/download" && r.URL.Path != "/api/ws/image" {
var flush func() var flush func()
w, flush = deferJSONResponse(w) w, flush = deferJSONResponse(w)
defer flush() defer flush()
@@ -184,6 +169,10 @@ func (a *API) workspace(w http.ResponseWriter, r *http.Request) {
a.readersMu.Unlock() a.readersMu.Unlock()
}() }()
} }
if r.URL.Path == "/api/ws/usage" {
a.wsUsage(w, r, u)
return
}
user := a.userLocks.Acquire(u.ID) user := a.userLocks.Acquire(u.ID)
root, err := a.openWorkspace(u, instanceID) root, err := a.openWorkspace(u, instanceID)
user.Release() user.Release()
@@ -193,8 +182,6 @@ func (a *API) workspace(w http.ResponseWriter, r *http.Request) {
} }
defer root.Close() defer root.Close()
switch strings.TrimPrefix(r.URL.Path, "/api/ws/") { switch strings.TrimPrefix(r.URL.Path, "/api/ws/") {
case "usage":
a.wsUsage(w, r, root, u)
case "tree": case "tree":
a.wsTree(w, r, root, u) a.wsTree(w, r, root, u)
case "mkdir": case "mkdir":
@@ -207,11 +194,184 @@ func (a *API) workspace(w http.ResponseWriter, r *http.Request) {
a.wsUpload(w, r, root, u) a.wsUpload(w, r, root, u)
case "download": case "download":
a.wsDownload(w, r, root) a.wsDownload(w, r, root)
case "image":
a.wsImage(w, r, root)
case "text":
a.wsText(w, r, root, u)
default: default:
fail(w, 404, "not found") fail(w, 404, "not found")
} }
} }
const maxTextBytes = 1 << 20
func textStatus(content string) int {
if len(content) > maxTextBytes {
return 413
}
if !utf8.ValidString(content) {
return 415
}
for _, c := range content {
if unicode.IsControl(c) && c != 9 && c != 10 && c != 13 {
return 415
}
}
return 0
}
func textVersion(content string, st os.FileInfo) string {
s := st.Sys().(*syscall.Stat_t)
return fmt.Sprintf("%x", sha256.Sum256([]byte(fmt.Sprintf("%d:%d:%d:%d:%d:%d:%s", s.Dev, s.Ino, st.ModTime().UnixNano(), s.Mode, s.Uid, s.Gid, content))))
}
func readText(parent *os.File, name string) (string, string, os.FileInfo, int) {
f, err := openDownload(parent, name)
if err != nil {
return "", "", nil, 404
}
defer f.Close()
st, err := f.Stat()
if err != nil || !st.Mode().IsRegular() {
return "", "", nil, 415
}
if st.Size() > maxTextBytes {
return "", "", nil, 413
}
b, err := io.ReadAll(io.LimitReader(f, maxTextBytes+1))
if err != nil {
return "", "", nil, 500
}
if code := textStatus(string(b)); code != 0 {
return "", "", nil, code
}
after, err := f.Stat()
if err != nil || st.Size() != after.Size() || !st.ModTime().Equal(after.ModTime()) || st.Sys().(*syscall.Stat_t).Ctim != after.Sys().(*syscall.Stat_t).Ctim {
return "", "", nil, 409
}
return string(b), textVersion(string(b), after), after, 0
}
func (a *API) wsText(w http.ResponseWriter, r *http.Request, root *os.Root, u store.User) {
if r.Method != http.MethodGet && r.Method != http.MethodPut {
fail(w, 405, "method not allowed")
return
}
p := r.URL.Query().Get("path")
if p == "" || path.IsAbs(p) || p == "." || p == ".." || strings.HasPrefix(p, "../") || path.Clean(p) != p || strings.ContainsRune(p, 0) {
fail(w, 400, "invalid path")
return
}
var x struct {
Content *string `json:"content"`
Version string `json:"version"`
}
if r.Method == http.MethodPut {
b, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 6*maxTextBytes+4096))
if err != nil {
fail(w, 413, "text request too large")
return
}
if !utf8.Valid(b) {
fail(w, 415, "invalid UTF-8")
return
}
if json.Unmarshal(b, &x) != nil || x.Content == nil || x.Version == "" {
fail(w, 400, "content and version required")
return
}
if code := textStatus(*x.Content); code != 0 {
fail(w, code, "unsupported text or text too large")
return
}
}
user := a.userLocks.Acquire(u.ID)
defer user.Release()
anchor, err := root.Open(".")
if err != nil {
fail(w, 500, "storage unavailable")
return
}
defer anchor.Close()
fd, err := unix.Openat2(int(anchor.Fd()), path.Dir(p), &unix.OpenHow{
Flags: unix.O_RDONLY | unix.O_DIRECTORY | unix.O_CLOEXEC,
Resolve: unix.RESOLVE_BENEATH | unix.RESOLVE_NO_SYMLINKS | unix.RESOLVE_NO_XDEV,
})
if err != nil {
code := 404
if r.Method == http.MethodPut {
code = 409
}
fail(w, code, "parent unavailable")
return
}
parent := os.NewFile(uintptr(fd), path.Dir(p))
defer parent.Close()
content, version, st, code := readText(parent, path.Base(p))
if code != 0 {
if r.Method == http.MethodPut && code == 404 {
code = 409
}
fail(w, code, "text file unavailable")
return
}
if r.Method == http.MethodGet {
writeJSON(w, map[string]string{"content": content, "version": version})
return
}
if version != x.Version {
fail(w, 409, "file changed; reload before saving")
return
}
if err := a.s.DB.QueryRow("SELECT quota FROM users WHERE id=?", u.ID).Scan(&u.Quota); err != nil {
fail(w, 413, "quota unavailable")
return
}
used, err := a.userUsage(u)
if err != nil || int64(len(*x.Content)) > u.Quota-used {
fail(w, 413, "quota exceeded or usage unavailable")
return
}
tmp := ".text-" + store.ID()
tfd, err := unix.Openat2(fd, tmp, &unix.OpenHow{Flags: unix.O_WRONLY | unix.O_CREAT | unix.O_EXCL | unix.O_CLOEXEC, Mode: 0600, Resolve: unix.RESOLVE_BENEATH | unix.RESOLVE_NO_SYMLINKS})
if err != nil {
fail(w, 500, "cannot prepare text save")
return
}
f := os.NewFile(uintptr(tfd), tmp)
defer f.Close()
defer unix.Unlinkat(fd, tmp, 0)
_, err = io.WriteString(f, *x.Content)
s := st.Sys().(*syscall.Stat_t)
if err == nil {
err = f.Chown(int(s.Uid), int(s.Gid))
}
if err == nil {
err = unix.Fchmod(tfd, s.Mode&07777)
}
if err == nil {
err = f.Sync()
}
next, statErr := f.Stat()
if err != nil || statErr != nil {
fail(w, 500, "cannot prepare text save")
return
}
nextVersion := textVersion(*x.Content, next)
_, latest, _, code := readText(parent, path.Base(p))
if code != 0 || latest != version {
fail(w, 409, "file changed; reload before saving")
return
}
// External writers do not take the user lock; this is not a filesystem CAS.
if err := unix.Renameat(fd, tmp, fd, path.Base(p)); err != nil {
fail(w, 500, "text save failed")
return
}
a.s.AddAudit(u.ID, "workspace.text", p, "")
writeJSON(w, map[string]string{"version": nextVersion})
}
func entry(root *os.Root, rel string) (os.FileInfo, error) { func entry(root *os.Root, rel string) (os.FileInfo, error) {
if err := rootSafe(root, rel, false); err != nil { if err := rootSafe(root, rel, false); err != nil {
return nil, err return nil, err
@@ -219,7 +379,7 @@ func entry(root *os.Root, rel string) (os.FileInfo, error) {
return root.Lstat(rel) return root.Lstat(rel)
} }
func (a *API) wsUsage(w http.ResponseWriter, r *http.Request, root *os.Root, u store.User) { func (a *API) wsUsage(w http.ResponseWriter, r *http.Request, u store.User) {
if r.Method != "GET" { if r.Method != "GET" {
fail(w, 405, "method not allowed") fail(w, 405, "method not allowed")
return return
@@ -537,6 +697,9 @@ func (a *API) userUsage(u store.User) (int64, error) {
} }
defer base.Close() defer base.Close()
root, e := base.OpenRoot(path.Join("workspaces", u.ID)) root, e := base.OpenRoot(path.Join("workspaces", u.ID))
if errors.Is(e, os.ErrNotExist) {
return 0, nil
}
if e != nil { if e != nil {
return 0, e return 0, e
} }
@@ -618,7 +781,7 @@ func (a *API) wsDownload(w http.ResponseWriter, r *http.Request, root *os.Root)
fail(w, 400, "invalid file") fail(w, 400, "invalid file")
return return
} }
w.Header().Set("Content-Disposition", "attachment") w.Header().Set("Content-Disposition", mime.FormatMediaType("attachment", map[string]string{"filename": path.Base(p)}))
http.ServeContent(w, r, path.Base(p), st.ModTime(), f) http.ServeContent(w, r, path.Base(p), st.ModTime(), f)
} }
+188 -2
View File
@@ -4,6 +4,7 @@ import (
"archive/zip" "archive/zip"
"bytes" "bytes"
"context" "context"
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"git.misaka.ren/m1saka/dsh-web/internal/store" "git.misaka.ren/m1saka/dsh-web/internal/store"
@@ -20,6 +21,188 @@ import (
"time" "time"
) )
func TestWorkspaceText(t *testing.T) {
a, sid := runtimeTestAPI(t, func(http.ResponseWriter, *http.Request) { t.Error("unexpected Docker call") })
for _, i := range []store.Instance{{ID: "one", UID: "owner"}, {ID: "foreign", UID: "other"}} {
if err := a.s.AddInstance(i); err != nil {
t.Fatal(err)
}
}
dir := filepath.Join(a.root, "workspaces/owner/instances/one/home")
if err := os.MkdirAll(dir, 0750); err != nil {
t.Fatal(err)
}
file := filepath.Join(dir, "file")
putFile := func(content string) {
t.Helper()
if err := os.WriteFile(file, []byte(content), 0751); err != nil {
t.Fatal(err)
}
}
putFile("original")
if _, err := a.s.DB.Exec("UPDATE users SET quota=? WHERE id='owner'", 32<<20); err != nil {
t.Fatal(err)
}
request := func(method, query, body, session string, want int) map[string]string {
t.Helper()
w := httptest.NewRecorder()
a.workspace(w, runtimeRequest(method, "/api/ws/text?"+query, body, session))
if w.Code != want {
t.Fatalf("%s %s: got %d want %d: %s", method, query, w.Code, want, w.Body.String())
}
out := map[string]string{}
if want == 200 {
if err := json.Unmarshal(w.Body.Bytes(), &out); err != nil {
t.Fatal(err)
}
}
return out
}
q := "instance=one&path=file"
body := func(content, version string) string {
b, _ := json.Marshal(map[string]string{"content": content, "version": version})
return string(b)
}
request("GET", q, "", "", 401)
request("GET", "instance=foreign&path=file", "", sid, 404)
request("GET", "path=file", "", sid, 410)
for _, p := range []string{"../file", "..", "/etc/passwd", ".", "", "\x00", "./file", "a/../file", "a/./file", "a//file", "file/"} {
for _, method := range []string{"GET", "PUT"} {
request(method, "instance=one&path="+url.QueryEscape(p), "", sid, 400)
}
}
for _, tc := range []struct{ exact, counterpart string }{
{"config.yaml ", "config.yaml"},
{" config.yaml", "config.yaml"},
{`a\b`, "a/b"},
} {
for name, content := range map[string]string{tc.exact: "exact", tc.counterpart: "counterpart"} {
if err := os.MkdirAll(filepath.Dir(filepath.Join(dir, name)), 0750); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, name), []byte(content), 0600); err != nil {
t.Fatal(err)
}
}
exactQuery := "instance=one&path=" + url.QueryEscape(tc.exact)
got := request("GET", exactQuery, "", sid, 200)
if got["content"] != "exact" {
t.Fatalf("GET %q returned %q", tc.exact, got["content"])
}
request("PUT", exactQuery, body("updated", got["version"]), sid, 200)
if got := request("GET", exactQuery, "", sid, 200); got["content"] != "updated" {
t.Fatalf("GET %q after PUT returned %q", tc.exact, got["content"])
}
for name, want := range map[string]string{tc.exact: "updated", tc.counterpart: "counterpart"} {
got, err := os.ReadFile(filepath.Join(dir, name))
if err != nil || string(got) != want {
t.Fatalf("file %q: got %q, want %q: %v", name, got, want, err)
}
}
}
request("POST", q, "", sid, 405)
v := request("GET", q, "", sid, 200)["version"]
if v == "" {
t.Fatal("missing version")
}
request("PUT", q, "{}", sid, 400)
request("PUT", q, body("next", "stale"), sid, 409)
request("PUT", q, body("next", ""), sid, 400)
for _, content := range []string{"nul\x00", "del\x7f", "control\x01", "invalid\xff"} {
putFile(content)
request("GET", q, "", sid, 415)
}
putFile(strings.Repeat("x", maxTextBytes+1))
request("GET", q, "", sid, 413)
putFile(strings.Repeat("x", maxTextBytes))
request("GET", q, "", sid, 200)
putFile("original")
v = request("GET", q, "", sid, 200)["version"]
request("PUT", q, body(strings.Repeat("x", maxTextBytes+1), v), sid, 413)
request("PUT", q, body("bad\x00", v), sid, 415)
request("PUT", q, "{\"content\":\"\xff\",\"version\":\"v\"}", sid, 415)
request("PUT", q, strings.Repeat(" ", 6*maxTextBytes+4097), sid, 413)
if err := os.Symlink(file, filepath.Join(dir, "link")); err != nil {
t.Fatal(err)
}
if err := os.Symlink(dir, filepath.Join(dir, "parent")); err != nil {
t.Fatal(err)
}
for _, p := range []string{"link", "parent/file"} {
request("GET", "instance=one&path="+p, "", sid, 404)
request("PUT", "instance=one&path="+p, body("next", v), sid, 409)
}
if err := os.Link(file, filepath.Join(dir, "hard")); err != nil {
t.Fatal(err)
}
request("GET", q, "", sid, 404)
request("PUT", q, body("next", v), sid, 409)
if err := os.Remove(filepath.Join(dir, "hard")); err != nil {
t.Fatal(err)
}
if _, err := a.s.DB.Exec("UPDATE users SET quota=8 WHERE id='owner'"); err != nil {
t.Fatal(err)
}
request("PUT", q, body("next", v), sid, 413)
if _, err := a.s.DB.Exec("UPDATE users SET quota=? WHERE id='owner'", 32<<20); err != nil {
t.Fatal(err)
}
before, err := os.Stat(file)
if err != nil {
t.Fatal(err)
}
next := "你好\tline\r\n"
saved := request("PUT", q, body(next, v), sid, 200)["version"]
got := request("GET", q, "", sid, 200)
if got["content"] != next || got["version"] != saved || saved == v {
t.Fatal(got, saved)
}
after, err := os.Stat(file)
if err != nil {
t.Fatal(err)
}
bs, as := before.Sys().(*syscall.Stat_t), after.Sys().(*syscall.Stat_t)
if bs.Mode != as.Mode || bs.Uid != as.Uid || bs.Gid != as.Gid {
t.Fatal("metadata changed")
}
request("PUT", q, body("stale save", v), sid, 409)
results := make(chan int, 2)
for range 2 {
go func() {
w := httptest.NewRecorder()
a.workspace(w, runtimeRequest("PUT", "/api/ws/text?"+q, body("concurrent", saved), sid))
results <- w.Code
}()
}
first, second := <-results, <-results
if !(first == 200 && second == 409 || first == 409 && second == 200) {
t.Fatalf("concurrent saves: %d %d", first, second)
}
saved = request("GET", q, "", sid, 200)["version"]
putFile("external")
request("PUT", q, body("stale save", saved), sid, 409)
if err := os.Remove(file); err != nil {
t.Fatal(err)
}
request("PUT", q, body("recreate", saved), sid, 409)
request("GET", q, "", sid, 404)
putFile("restored")
if _, err := a.s.DB.Exec("UPDATE instances SET status='error' WHERE id='one'"); err != nil {
t.Fatal(err)
}
request("PUT", q, body("blocked", saved), sid, 409)
request("GET", q, "", sid, 200)
entries, err := os.ReadDir(dir)
if err != nil {
t.Fatal(err)
}
for _, e := range entries {
if strings.HasPrefix(e.Name(), ".text-") {
t.Fatal("temporary file leaked")
}
}
}
func TestUploadExtensionPolicy(t *testing.T) { func TestUploadExtensionPolicy(t *testing.T) {
for _, tc := range []struct { for _, tc := range []struct {
name, policy string name, policy string
@@ -449,7 +632,10 @@ func TestWorkspaceSafetyAndUpload(t *testing.T) {
if err := s.CreateUser(u, "unused"); err != nil { if err := s.CreateUser(u, "unused"); err != nil {
t.Fatal(err) t.Fatal(err)
} }
root, e := a.openWorkspace(u) if err := os.MkdirAll(filepath.Join(a.root, "workspaces/user/instances/one/home"), 0750); err != nil {
t.Fatal(err)
}
root, e := a.openWorkspace(u, "one")
if e != nil { if e != nil {
t.Fatal(e) t.Fatal(e)
} }
@@ -463,7 +649,7 @@ func TestWorkspaceSafetyAndUpload(t *testing.T) {
} }
outside := t.TempDir() outside := t.TempDir()
os.WriteFile(filepath.Join(outside, "secret"), []byte("secret"), 0600) os.WriteFile(filepath.Join(outside, "secret"), []byte("secret"), 0600)
if e = os.Symlink(outside, filepath.Join(a.root, "workspaces/user/files/link")); e != nil { if e = os.Symlink(outside, filepath.Join(a.root, "workspaces/user/instances/one/home/link")); e != nil {
t.Fatal(e) t.Fatal(e)
} }
w := httptest.NewRecorder() w := httptest.NewRecorder()
+34 -345
View File
@@ -1,14 +1,11 @@
package api package api
import ( import (
"archive/tar"
"bytes"
"context" "context"
"database/sql" "database/sql"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
imageassets "git.misaka.ren/m1saka/dsh-web/image"
"git.misaka.ren/m1saka/dsh-web/internal/store" "git.misaka.ren/m1saka/dsh-web/internal/store"
"io" "io"
"net/http" "net/http"
@@ -35,6 +32,9 @@ func (a *API) ensureBuilds() error {
"ALTER TABLE image_builds ADD COLUMN platform TEXT", "ALTER TABLE image_builds ADD COLUMN platform TEXT",
"ALTER TABLE image_builds ADD COLUMN finished_at TEXT", "ALTER TABLE image_builds ADD COLUMN finished_at TEXT",
"ALTER TABLE image_builds ADD COLUMN experimental INTEGER NOT NULL DEFAULT 0", "ALTER TABLE image_builds ADD COLUMN experimental INTEGER NOT NULL DEFAULT 0",
"ALTER TABLE image_builds ADD COLUMN release_id TEXT NOT NULL DEFAULT ''",
"ALTER TABLE image_builds ADD COLUMN source_repository TEXT NOT NULL DEFAULT ''",
"ALTER TABLE image_builds ADD COLUMN source_commit TEXT NOT NULL DEFAULT ''",
"CREATE TABLE IF NOT EXISTS image_admissions(image_id TEXT NOT NULL,version TEXT NOT NULL,recipe_rev TEXT NOT NULL,platform TEXT NOT NULL,state TEXT NOT NULL,evidence TEXT NOT NULL,admitted_by TEXT NOT NULL,admitted_at TEXT NOT NULL,PRIMARY KEY(image_id,version,recipe_rev,platform))", "CREATE TABLE IF NOT EXISTS image_admissions(image_id TEXT NOT NULL,version TEXT NOT NULL,recipe_rev TEXT NOT NULL,platform TEXT NOT NULL,state TEXT NOT NULL,evidence TEXT NOT NULL,admitted_by TEXT NOT NULL,admitted_at TEXT NOT NULL,PRIMARY KEY(image_id,version,recipe_rev,platform))",
"ALTER TABLE images ADD COLUMN recipe_rev TEXT", "ALTER TABLE images ADD COLUMN recipe_rev TEXT",
"ALTER TABLE images ADD COLUMN platform TEXT", "ALTER TABLE images ADD COLUMN platform TEXT",
@@ -44,13 +44,10 @@ func (a *API) ensureBuilds() error {
return e return e
} }
} }
_, err = a.s.DB.Exec("DROP INDEX IF EXISTS image_builds_version; CREATE UNIQUE INDEX IF NOT EXISTS image_builds_identity ON image_builds(version,recipe_rev,platform) WHERE recipe_rev IS NOT NULL AND platform IS NOT NULL AND status IN ('queued','running','validating','success')") _, err = a.s.DB.Exec("DROP INDEX IF EXISTS image_builds_version; DROP INDEX IF EXISTS image_builds_identity; CREATE UNIQUE INDEX image_builds_identity ON image_builds(version,recipe_rev,platform) WHERE release_id='' AND recipe_rev IS NOT NULL AND platform IS NOT NULL AND status IN ('queued','running','validating','success'); CREATE UNIQUE INDEX IF NOT EXISTS image_builds_release_identity ON image_builds(source_repository,source_commit,release_id,platform) WHERE release_id<>'' AND status IN ('queued','running','validating','success')")
if err != nil { if err != nil {
return err return err
} }
if _, err = a.s.DB.Exec("UPDATE image_builds SET status='failed',error='version requires administrator experimental build' WHERE status='queued' AND experimental=0 AND version NOT IN ('0.2.0-rc.1','0.2.0-rc.2')"); err != nil {
return err
}
if !buildQueue.initialized[a.s.DB] { if !buildQueue.initialized[a.s.DB] {
if _, err = a.s.DB.Exec("UPDATE image_builds SET status='failed',error='image template identity is unknown' WHERE status='queued' AND (recipe_rev IS NULL OR recipe_rev<>? OR platform IS NULL OR platform<>?)", imageRecipeRev, imagePlatform); err != nil { if _, err = a.s.DB.Exec("UPDATE image_builds SET status='failed',error='image template identity is unknown' WHERE status='queued' AND (recipe_rev IS NULL OR recipe_rev<>? OR platform IS NULL OR platform<>?)", imageRecipeRev, imagePlatform); err != nil {
return err return err
@@ -63,179 +60,20 @@ func (a *API) ensureBuilds() error {
return nil return nil
} }
func (a *API) builds(w http.ResponseWriter, r *http.Request) { func (a *API) builds(w http.ResponseWriter, r *http.Request) {
u, _, authErr := a.user(r) _, _, authErr := a.user(r)
if authErr != nil { if authErr != nil {
fail(w, 401, "authentication required") fail(w, 401, "authentication required")
return return
} }
if r.Method != http.MethodGet && r.Method != http.MethodPost { if r.Method != http.MethodGet {
fail(w, 405, "method not allowed") fail(w, 405, "method not allowed")
return return
} }
var in struct { a.listBuilds(w, r)
Version string `json:"version"`
Experimental bool `json:"experimental"`
}
if r.Method == http.MethodPost {
dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096))
dec.DisallowUnknownFields()
if dec.Decode(&in) != nil || dec.Decode(new(any)) != io.EOF || !validBuildVersion(in.Version) {
fail(w, 400, "a fixed version is required")
return
}
}
if r.Method == http.MethodGet {
a.listBuilds(w, r)
return
}
if in.Experimental && !u.Admin {
fail(w, 403, "administrator required")
return
}
b, err := a.enqueueBuild(r.Context(), u.ID, in.Version, in.Experimental)
if err != nil {
status := 502
if errors.Is(err, errBuildUnsupported) {
status = 400
}
if errors.Is(err, errBuildRateLimited) {
status = 429
w.Header().Set("Retry-After", "3600")
}
fail(w, status, sanitizeBuildLog(err.Error()))
return
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusAccepted)
writeJSON(w, b)
} }
var errBuildRateLimited = errors.New("build limit: three new tasks per user per hour") var errBuildRateLimited = errors.New("build limit: three new tasks per user per hour")
// lookupActiveBuild returns the active or successful record for a version; it
// performs no I/O and must be called with the build queue held.
func (a *API) lookupActiveBuild(version string) (buildRecord, bool, error) {
var b buildRecord
var finished, recipe, platform sql.NullString
err := a.s.DB.QueryRow("SELECT id,version,status,logs,error,image_id,created_at,requested_by,recipe_rev,platform,finished_at,experimental FROM image_builds WHERE version=? AND recipe_rev=? AND platform=? AND status IN ('queued','running','validating','success')", version, imageRecipeRev, imagePlatform).Scan(&b.ID, &b.Version, &b.Status, &b.Logs, &b.Error, &b.ImageID, &b.Created, &b.RequestedBy, &recipe, &platform, &finished, &b.Experimental)
b.RecipeRev, b.Platform, b.Finished = recipe.String, platform.String, finished.String
if errors.Is(err, sql.ErrNoRows) {
return b, false, nil
}
if err != nil {
return b, false, err
}
return b, true, nil
}
func (a *API) enqueueBuild(ctx context.Context, uid, version string, experiment ...bool) (buildRecord, error) {
experimental := len(experiment) > 0 && experiment[0]
if experimental {
u, err := a.s.UserByID(uid)
if err != nil || !u.Admin {
return buildRecord{}, errors.New("administrator required")
}
}
if !experimental && !normalBuildAllowed(version) {
return buildRecord{}, errBuildUnsupported
}
if !validBuildVersion(version) {
return buildRecord{}, errors.New("a fixed version is required")
}
buildQueue.Lock()
err := a.ensureBuilds()
var b buildRecord
var found bool
if err == nil {
b, found, err = a.lookupActiveBuild(version)
}
buildQueue.Unlock()
if err != nil {
return b, err
}
if found && !experimental && b.Experimental && b.Status != "success" {
return b, errBuildUnsupported
}
if found && b.Status != "success" {
a.startBuildWorker()
return b, nil
}
if found {
// Verifying a previously built image talks to Docker; keep that I/O
// out of the queue lock shared with instance creation and upgrades.
verify, cancel := context.WithTimeout(ctx, 15*time.Second)
id, lookupErr := a.localBuildImage(verify, b.Version, b.ImageID)
cancel()
if lookupErr != nil {
return b, lookupErr
}
if id != "" && id == b.ImageID {
if !experimental {
im, e := a.evaluateImage(ctx, id)
if e != nil {
return b, e
}
if !im.Selectable {
return b, errBuildUnsupported
}
}
a.startBuildWorker()
return b, nil
}
buildQueue.Lock()
if _, err = a.s.DB.Exec("UPDATE image_builds SET status='missing',error='image no longer available' WHERE id=? AND status='success'", b.ID); err == nil {
// Another request may have queued a replacement while the image
// was being verified; honor it instead of inserting a duplicate.
b, found, err = a.lookupActiveBuild(version)
}
if err == nil && !found {
b, err = a.insertBuild(uid, version, experimental)
}
buildQueue.Unlock()
if err != nil {
return b, err
}
if !experimental && found && b.Experimental {
return b, errBuildUnsupported
}
a.startBuildWorker()
return b, nil
}
buildQueue.Lock()
defer buildQueue.Unlock()
// Re-check after the unlocked phases: another request may have inserted.
b, found, err = a.lookupActiveBuild(version)
if err != nil {
return b, err
}
if found {
if !experimental && b.Experimental {
return b, errBuildUnsupported
}
return b, nil
}
b, err = a.insertBuild(uid, version, experimental)
if err != nil {
return b, err
}
a.startBuildWorkerLocked()
return b, nil
}
// insertBuild applies the per-user rate limit and inserts a queued record;
// call with the build queue held.
func (a *API) insertBuild(uid, version string, experiment ...bool) (buildRecord, error) {
var count int
if err := a.s.DB.QueryRow("SELECT count(*) FROM image_build_requests WHERE user_id=? AND created_at>?", uid, time.Now().Add(-time.Hour).UTC().Format(time.RFC3339Nano)).Scan(&count); err != nil {
return buildRecord{}, err
}
if count >= 3 {
return buildRecord{}, errBuildRateLimited
}
b := buildRecord{ID: store.ID(), Version: version, Status: "queued", Created: time.Now().UTC().Format(time.RFC3339Nano), RequestedBy: uid, RecipeRev: imageRecipeRev, Platform: imagePlatform}
b.Experimental = len(experiment) > 0 && experiment[0]
return b, a.insertBuildRequest(b, uid)
}
func (a *API) startBuildWorker() { func (a *API) startBuildWorker() {
buildQueue.Lock() buildQueue.Lock()
defer buildQueue.Unlock() defer buildQueue.Unlock()
@@ -280,7 +118,8 @@ func (a *API) startBuildWorkerLocked() {
} }
buildQueue.Lock() buildQueue.Lock()
var b buildRecord var b buildRecord
err := a.s.DB.QueryRow("SELECT id,version,experimental,requested_by FROM image_builds WHERE status='queued' ORDER BY created_at,id LIMIT 1").Scan(&b.ID, &b.Version, &b.Experimental, &b.RequestedBy) row := a.s.DB.QueryRow("SELECT " + buildSummaryColumns + " FROM image_builds WHERE status='queued' AND release_id<>'' ORDER BY created_at,id LIMIT 1")
err := scanBuildSummary(row, &b)
if err == nil { if err == nil {
_, err = a.s.DB.Exec("UPDATE image_builds SET status='running' WHERE id=?", b.ID) _, err = a.s.DB.Exec("UPDATE image_builds SET status='running' WHERE id=?", b.ID)
} }
@@ -303,27 +142,6 @@ func (a *API) startBuildWorkerLocked() {
} }
}() }()
} }
func imageBuildContext() (io.Reader, error) {
var buf bytes.Buffer
tw := tar.NewWriter(&buf)
for _, name := range []string{"Dockerfile", "Caddyfile", "entrypoint.sh", "install-mobile.mjs", "install-free-search.mjs", "install-workspace.mjs", "init-profile.mjs"} {
dataBytes, err := imageassets.Files.ReadFile(name)
if err != nil {
return nil, err
}
data := string(dataBytes)
if err := tw.WriteHeader(&tar.Header{Name: name, Mode: 0755, Size: int64(len(data))}); err != nil {
return nil, err
}
if _, err := tw.Write([]byte(data)); err != nil {
return nil, err
}
}
if err := tw.Close(); err != nil {
return nil, err
}
return &buf, nil
}
func (a *API) runImageBuild(b buildRecord) { func (a *API) runImageBuild(b buildRecord) {
buildWorker.Lock() buildWorker.Lock()
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute) ctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute)
@@ -344,32 +162,23 @@ func (a *API) runImageBuild(b buildRecord) {
_, logErr = a.s.DB.Exec("UPDATE image_builds SET logs=? WHERE id=?", logs, b.ID) _, logErr = a.s.DB.Exec("UPDATE image_builds SET logs=? WHERE id=?", logs, b.ID)
} }
} }
tarball, err := imageBuildContext()
if !b.Experimental && !normalBuildAllowed(b.Version) {
err = errBuildUnsupported
}
if b.Experimental {
u, e := a.s.UserByID(b.RequestedBy)
if e != nil || !u.Admin {
err = errors.New("experimental build requester is not an administrator")
}
}
var id string var id string
if err == nil { var err error
id, err = a.localBuildImage(ctx, b.Version, "") if b.ReleaseID == "" {
} err = errors.New("Git release identity required")
if err == nil && id == "" { } else {
_, err = fetchNPMVersion(ctx, npmBuildClient, b.Version) var tarball io.Reader
tarball, err = a.releaseBuildContext(ctx, b)
if err == nil { if err == nil {
err = a.checkBuildDisk(ctx) err = a.checkBuildDisk(ctx)
} }
} if err == nil {
if err == nil && id == "" { id, err = a.docker.BuildRelease(ctx, tarball, b.Version, "dsh-web:release-"+b.SourceCommit, b.ReleaseID, b.SourceRepository, b.SourceCommit, appendLog)
id, err = a.docker.Build(ctx, tarball, b.Version, "dsh-web:"+b.Version+"-"+imageRecipeRev, appendLog) }
} }
if err == nil { if err == nil {
var matches bool var matches bool
matches, err = a.imageMatches(ctx, id, b.Version) matches, err = a.buildImageMatches(ctx, id, b)
if err == nil && !matches { if err == nil && !matches {
err = errors.New("image template or platform mismatch") err = errors.New("image template or platform mismatch")
} }
@@ -517,18 +326,21 @@ type imageRecord struct {
UsageCount int `json:"usage_count"` UsageCount int `json:"usage_count"`
} }
type buildRecord struct { type buildRecord struct {
Experimental bool `json:"experimental"` ReleaseID string `json:"release_id,omitempty"`
ImageID string `json:"image_id,omitempty"` SourceRepository string `json:"source_repository,omitempty"`
ID string `json:"id"` SourceCommit string `json:"source_commit,omitempty"`
Version string `json:"version"` Experimental bool `json:"experimental"`
Status string `json:"status"` ImageID string `json:"image_id,omitempty"`
Logs string `json:"logs,omitempty"` ID string `json:"id"`
Error string `json:"error,omitempty"` Version string `json:"version"`
Created string `json:"created_at"` Status string `json:"status"`
RequestedBy string `json:"requested_by,omitempty"` Logs string `json:"logs,omitempty"`
RecipeRev string `json:"recipe_rev,omitempty"` Error string `json:"error,omitempty"`
Platform string `json:"platform,omitempty"` Created string `json:"created_at"`
Finished string `json:"finished_at,omitempty"` RequestedBy string `json:"requested_by,omitempty"`
RecipeRev string `json:"recipe_rev,omitempty"`
Platform string `json:"platform,omitempty"`
Finished string `json:"finished_at,omitempty"`
} }
func (a *API) images(w http.ResponseWriter, r *http.Request) { func (a *API) images(w http.ResponseWriter, r *http.Request) {
@@ -554,38 +366,6 @@ func (a *API) images(w http.ResponseWriter, r *http.Request) {
writeJSON(w, result) writeJSON(w, result)
} }
// verifiedImagesTTL bounds how long a cached admission evaluation is reused
// by the one-second intent poller.
var verifiedImagesTTL = 5 * time.Second
type evalCacheEntry struct {
im imageRecord
at time.Time
}
// evaluateImageCached reuses recent per-image evaluations for the intent
// poller so repeated ticks do not hammer Docker. Deletions and admission
// writes bump imageDeleteGeneration and invalidate every entry.
func (a *API) evaluateImageCached(ctx context.Context, gen uint64, id string) (imageRecord, error) {
a.evalCacheMu.Lock()
if entry, ok := a.evalCache[id]; ok && a.evalCacheGen == gen && time.Since(entry.at) < verifiedImagesTTL {
a.evalCacheMu.Unlock()
return entry.im, nil
}
a.evalCacheMu.Unlock()
im, err := a.evaluateImage(ctx, id)
if err != nil {
return im, err
}
a.evalCacheMu.Lock()
if a.evalCache == nil || a.evalCacheGen != gen {
a.evalCache, a.evalCacheGen = make(map[string]evalCacheEntry), gen
}
a.evalCache[id] = evalCacheEntry{im: im, at: time.Now()}
a.evalCacheMu.Unlock()
return im, nil
}
func (a *API) successfulImageIDs(ctx context.Context) ([]string, error) { func (a *API) successfulImageIDs(ctx context.Context) ([]string, error) {
rows, err := a.s.DB.QueryContext(ctx, "SELECT DISTINCT image_id FROM image_builds WHERE status='success' AND image_id<>''") rows, err := a.s.DB.QueryContext(ctx, "SELECT DISTINCT image_id FROM image_builds WHERE status='success' AND image_id<>''")
if err != nil { if err != nil {
@@ -632,74 +412,6 @@ func (a *API) verifiedImages(ctx context.Context) (map[string]string, error) {
return result, nil return result, nil
} }
// verifiedImagesPoll is the intent poller's view: build rows are reread every
// tick, but per-image Docker evaluation is cached briefly so a stuck waiting
// intent cannot turn the one-second loop into Docker hammering.
func (a *API) verifiedImagesPoll(ctx context.Context) (map[string]string, error) {
buildQueue.Lock()
err := a.ensureBuilds()
gen := a.imageDeleteGeneration
buildQueue.Unlock()
if err != nil {
return nil, err
}
ids, err := a.successfulImageIDs(ctx)
if err != nil {
return nil, err
}
result := map[string]string{}
for _, id := range ids {
im, err := a.evaluateImageCached(ctx, gen, id)
if err != nil {
return nil, err
}
if im.Selectable {
result[id] = im.Version
}
}
return result, nil
}
func (a *API) latest(w http.ResponseWriter, r *http.Request) {
if _, _, err := a.user(r); err != nil {
fail(w, 401, "authentication required")
return
}
if r.Method != http.MethodGet {
fail(w, 405, "method not allowed")
return
}
v, err := fetchLatest(r.Context(), npmBuildClient)
if err != nil {
fail(w, 502, "npm version lookup failed")
return
}
writeJSON(w, map[string]any{"version": v, "checked_at": time.Now().UTC().Format(time.RFC3339), "supported": normalBuildAllowed(v), "build_allowed": normalBuildAllowed(v), "admission": defaultAdmission(v, imageRecipeRev, imagePlatform), "admission_reason": "normal builds require a built-in tested tuple; unknown versions require administrator experimental validation and admission"})
}
func fetchLatest(ctx context.Context, client *http.Client) (string, error) {
return fetchNPMVersion(ctx, client, "latest")
}
func fetchNPMVersion(ctx context.Context, client *http.Client, version string) (string, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://registry.npmjs.org/@deepseek-ai%2Fdsh/"+url.PathEscape(version), nil)
if err != nil {
return "", err
}
resp, err := client.Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
return "", fmt.Errorf("npm status %d", resp.StatusCode)
}
var v struct{ Version string }
if err = json.NewDecoder(io.LimitReader(resp.Body, 1<<20)).Decode(&v); err != nil {
return "", err
}
if !validBuildVersion(v.Version) || (version != "latest" && v.Version != version) {
return "", errors.New("invalid npm version")
}
return v.Version, nil
}
func validBuildVersion(v string) bool { return len(v) <= 100 && fixedVersion.MatchString(v) } func validBuildVersion(v string) bool { return len(v) <= 100 && fixedVersion.MatchString(v) }
func sanitizeBuildLog(s string) string { func sanitizeBuildLog(s string) string {
s = buildURLCredentials.ReplaceAllString(s, "$1[REDACTED]@") s = buildURLCredentials.ReplaceAllString(s, "$1[REDACTED]@")
@@ -710,26 +422,3 @@ func sanitizeBuildLog(s string) string {
} }
return s return s
} }
func (a *API) localBuildImage(ctx context.Context, version, preferred string) (string, error) {
filters, _ := json.Marshal(map[string][]string{"label": {"dsh-mgr.role=runtime", "dsh-mgr.version=" + version}})
var images []struct {
ID string
Labels map[string]string
}
if err := a.docker.Do(ctx, http.MethodGet, "/images/json?filters="+url.QueryEscape(string(filters)), nil, &images); err != nil {
return "", err
}
for _, im := range images {
if im.ID != "" && im.Labels["dsh-mgr.role"] == "runtime" && im.Labels["dsh-mgr.version"] == version && im.Labels["dsh-mgr.recipe"] == imageRecipeRev && (preferred == "" || im.ID == preferred) {
ok, err := a.imageMatches(ctx, im.ID, version)
if err != nil {
return "", err
}
if ok {
return im.ID, nil
}
}
}
return "", nil
}
-2
View File
@@ -47,7 +47,6 @@ func (a *API) deleteImage(w http.ResponseWriter, r *http.Request) {
return return
} }
defer tx.Rollback() defer tx.Rollback()
// Take SQLite's writer lock before checking intents: reservation does not use the build queue.
if _, err = tx.ExecContext(ctx, "UPDATE images SET status=status WHERE image_id=?", id); err != nil { if _, err = tx.ExecContext(ctx, "UPDATE images SET status=status WHERE image_id=?", id); err != nil {
fail(w, 500, "image index unavailable") fail(w, 500, "image index unavailable")
return return
@@ -64,7 +63,6 @@ func (a *API) deleteImage(w http.ResponseWriter, r *http.Request) {
} }
for _, check := range []struct{ table, query string }{ for _, check := range []struct{ table, query string }{
{"image_builds", "SELECT count(*) FROM image_builds WHERE status IN ('queued','running','validating')"}, {"image_builds", "SELECT count(*) FROM image_builds WHERE status IN ('queued','running','validating')"},
{"creation_intents", "SELECT count(*) FROM creation_intents WHERE status IN ('waiting','creating')"},
{"upgrade_journal", "SELECT count(*) FROM upgrade_journal WHERE phase NOT IN ('complete','rolled_back')"}, {"upgrade_journal", "SELECT count(*) FROM upgrade_journal WHERE phase NOT IN ('complete','rolled_back')"},
} { } {
var exists, count int var exists, count int
+1 -6
View File
@@ -10,7 +10,7 @@ import (
func TestImageDelete(t *testing.T) { func TestImageDelete(t *testing.T) {
id := "sha256:" + strings.Repeat("a", 64) id := "sha256:" + strings.Repeat("a", 64)
for _, mode := range []string{"success", "anonymous", "non-admin", "invalid", "unmanaged", "role", "instance", "container-running", "container-exited", "containers-failure", "containers-null", "containers-unknown", "inspect-failure", "delete-failure", "delete-conflict", "queued", "running", "validating", "worker", "waiting", "creating", "journal", "complete", "rolled_back", "db-failure", "task-query-failure", "audit-failure"} { for _, mode := range []string{"success", "anonymous", "non-admin", "invalid", "unmanaged", "role", "instance", "container-running", "container-exited", "containers-failure", "containers-null", "containers-unknown", "inspect-failure", "delete-failure", "delete-conflict", "queued", "running", "validating", "worker", "journal", "complete", "rolled_back", "db-failure", "task-query-failure", "audit-failure"} {
t.Run(mode, func(t *testing.T) { t.Run(mode, func(t *testing.T) {
var deletes atomic.Int32 var deletes atomic.Int32
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
@@ -101,11 +101,6 @@ func TestImageDelete(t *testing.T) {
buildQueue.active[a.s.DB] = true buildQueue.active[a.s.DB] = true
buildQueue.Unlock() buildQueue.Unlock()
defer func() { buildQueue.Lock(); delete(buildQueue.active, a.s.DB); buildQueue.Unlock() }() defer func() { buildQueue.Lock(); delete(buildQueue.active, a.s.DB); buildQueue.Unlock() }()
case "waiting", "creating":
if err := a.s.EnsureIntents(); err != nil {
t.Fatal(err)
}
exec("INSERT INTO creation_intents(id,uid,name,version,status,host,created) VALUES('intent','other','i','2.0.0',?,'host',0)", mode)
case "db-failure": case "db-failure":
exec("ALTER TABLE instances RENAME COLUMN image_id TO broken") exec("ALTER TABLE instances RENAME COLUMN image_id TO broken")
want = 500 want = 500
@@ -1,14 +1,13 @@
package api package api
import ( import (
"context"
"net/http" "net/http"
"sync" "sync"
"testing" "testing"
"time" "time"
) )
func TestEnqueueBuildConcurrentMissingImageReturnsReplacement(t *testing.T) { func TestEnqueueReleaseConcurrentMissingImageReturnsReplacement(t *testing.T) {
inspections := make(chan chan struct{}, 2) inspections := make(chan chan struct{}, 2)
stop := make(chan struct{}) stop := make(chan struct{})
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
@@ -41,6 +40,10 @@ func TestEnqueueBuildConcurrentMissingImageReturnsReplacement(t *testing.T) {
} }
}) })
seedVerifiedImage(t, a, "old-image", "0.2.0-rc.1") seedVerifiedImage(t, a, "old-image", "0.2.0-rc.1")
info := releaseFixture()
if _, err := a.s.DB.Exec("UPDATE image_builds SET release_id=?,source_repository=?,source_commit=?,version=? WHERE image_id='old-image'", info.ReleaseID, info.Repository, info.Commit, info.Version); err != nil {
t.Fatal(err)
}
var oldID string var oldID string
if err := a.s.DB.QueryRow("SELECT id FROM image_builds WHERE image_id='old-image'").Scan(&oldID); err != nil { if err := a.s.DB.QueryRow("SELECT id FROM image_builds WHERE image_id='old-image'").Scan(&oldID); err != nil {
t.Fatal(err) t.Fatal(err)
@@ -49,11 +52,9 @@ func TestEnqueueBuildConcurrentMissingImageReturnsReplacement(t *testing.T) {
buildQueue.Lock() buildQueue.Lock()
buildQueue.active[a.s.DB] = true buildQueue.active[a.s.DB] = true
buildQueue.Unlock() buildQueue.Unlock()
ctx, cancel := context.WithCancel(context.Background())
var submissions sync.WaitGroup var submissions sync.WaitGroup
t.Cleanup(func() { t.Cleanup(func() {
close(stop) close(stop)
cancel()
submissions.Wait() submissions.Wait()
buildQueue.Lock() buildQueue.Lock()
delete(buildQueue.active, a.s.DB) delete(buildQueue.active, a.s.DB)
@@ -69,7 +70,7 @@ func TestEnqueueBuildConcurrentMissingImageReturnsReplacement(t *testing.T) {
submissions.Add(1) submissions.Add(1)
go func() { go func() {
defer submissions.Done() defer submissions.Done()
b, err := a.enqueueBuild(ctx, "u", "0.2.0-rc.1") b, err := a.enqueueRelease("u", info)
done <- result{b, err} done <- result{b, err}
}() }()
return done return done
@@ -93,22 +93,19 @@ func TestImageIndexDoesNotUndoConcurrentDelete(t *testing.T) {
if im.ImageID == id { if im.ImageID == id {
matches++ matches++
} }
if im.ImageID == id && im.Status != "missing" {
t.Errorf("returned deleted image as %s", im.Status)
}
} }
if matches != 1 { if matches != 0 {
t.Errorf("returned %d records for deleted image, want 1", matches) t.Errorf("returned %d records for deleted image, want 0", matches)
} }
case <-ctx.Done(): case <-ctx.Done():
t.Fatal("index did not finish") t.Fatal("index did not finish")
} }
var status string var count int
if err := a.s.DB.QueryRow("SELECT status FROM images WHERE image_id=?", id).Scan(&status); err != nil { if err := a.s.DB.QueryRow("SELECT count(*) FROM images WHERE image_id=?", id).Scan(&count); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if status != "missing" { if count != 0 {
t.Errorf("persisted deleted image as %s", status) t.Errorf("persisted %d rows for deleted image", count)
} }
}) })
} }
+23 -10
View File
@@ -16,8 +16,6 @@ import (
"time" "time"
) )
var npmBuildClient = &http.Client{Timeout: 15 * time.Second, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}
const imageRecipeRev = "7" const imageRecipeRev = "7"
var imagePlatform = runtime.GOOS + "/" + runtime.GOARCH var imagePlatform = runtime.GOOS + "/" + runtime.GOARCH
@@ -44,6 +42,12 @@ func (im inspectedImage) platform() string {
return "" return ""
} }
p := im.Os + "/" + im.Architecture p := im.Os + "/" + im.Architecture
// Docker can report the default ARM64 variant explicitly. Keep its
// identity consistent with build records and admission keys; preserve
// all other variants so incompatible images remain rejected.
if im.Architecture == "arm64" && im.Variant == "v8" {
return p
}
if im.Variant != "" { if im.Variant != "" {
p += "/" + im.Variant p += "/" + im.Variant
} }
@@ -94,7 +98,7 @@ func (a *API) imageIndexSnapshot(ctx context.Context) ([]imageRecord, error) {
err := a.ensureBuilds() err := a.ensureBuilds()
if err == nil { if err == nil {
_, err = a.s.DB.ExecContext(ctx, `INSERT OR IGNORE INTO images(image_id,version,recipe_rev,platform,tag,status,built_at) _, err = a.s.DB.ExecContext(ctx, `INSERT OR IGNORE INTO images(image_id,version,recipe_rev,platform,tag,status,built_at)
SELECT image_id,version,recipe_rev,platform,'','missing',COALESCE(finished_at,created_at) FROM image_builds WHERE status IN ('success','missing') AND image_id<>'' ORDER BY created_at DESC,id DESC`) SELECT image_id,version,recipe_rev,platform,'','missing',COALESCE(finished_at,created_at) FROM image_builds WHERE status='success' AND image_id<>'' ORDER BY created_at DESC,id DESC`)
} }
var rows *sql.Rows var rows *sql.Rows
if err == nil { if err == nil {
@@ -186,18 +190,27 @@ func (a *API) imageIndexSnapshot(ctx context.Context) ([]imageRecord, error) {
return nil, err return nil, err
} }
defer tx.Rollback() defer tx.Rollback()
present := make([]imageRecord, 0, len(result))
for _, im := range result { for _, im := range result {
if _, err = tx.ExecContext(ctx, `INSERT INTO images(image_id,version,recipe_rev,platform,tag,status,built_at,usage_count) VALUES(?,?,?,?,?,?,?,?)
ON CONFLICT(image_id) DO UPDATE SET version=excluded.version,recipe_rev=excluded.recipe_rev,platform=excluded.platform,tag=excluded.tag,status=excluded.status,built_at=excluded.built_at,usage_count=excluded.usage_count`, im.ImageID, im.Version, im.RecipeRev, im.Platform, im.Tag, im.Status, im.Created, im.UsageCount); err != nil {
return nil, err
}
if im.Status == "missing" { if im.Status == "missing" {
if _, err = tx.ExecContext(ctx, "UPDATE image_builds SET status='missing',error='image no longer available or template identity is unknown' WHERE image_id=? AND status='success'", im.ImageID); err != nil { if _, err = tx.ExecContext(ctx, "UPDATE image_builds SET status='missing',error='image no longer available or template identity is unknown' WHERE image_id=? AND status='success'", im.ImageID); err != nil {
return nil, err return nil, err
} }
if _, err = tx.ExecContext(ctx, "DELETE FROM images WHERE image_id=?", im.ImageID); err != nil {
return nil, err
}
continue
} }
if _, err = tx.ExecContext(ctx, `INSERT INTO images(image_id,version,recipe_rev,platform,tag,status,built_at,usage_count) VALUES(?,?,?,?,?,?,?,?)
ON CONFLICT(image_id) DO UPDATE SET version=excluded.version,recipe_rev=excluded.recipe_rev,platform=excluded.platform,tag=excluded.tag,status=excluded.status,built_at=excluded.built_at,usage_count=excluded.usage_count`, im.ImageID, im.Version, im.RecipeRev, im.Platform, im.Tag, im.Status, im.Created, im.UsageCount); err != nil {
return nil, err
}
present = append(present, im)
} }
return result, tx.Commit() if err = tx.Commit(); err != nil {
return nil, err
}
return present, nil
} }
func (a *API) StartImageBuilds() error { func (a *API) StartImageBuilds() error {
@@ -240,7 +253,7 @@ func (a *API) insertBuildRequest(b buildRecord, user string) error {
return err return err
} }
defer tx.Rollback() defer tx.Rollback()
if _, err = tx.Exec("INSERT INTO image_builds(id,version,status,created_at,requested_by,recipe_rev,platform,experimental) VALUES(?,?,?,?,?,?,?,?)", b.ID, b.Version, b.Status, b.Created, b.RequestedBy, b.RecipeRev, b.Platform, b.Experimental); err != nil { if _, err = tx.Exec("INSERT INTO image_builds(id,version,status,created_at,requested_by,recipe_rev,platform,experimental,release_id,source_repository,source_commit) VALUES(?,?,?,?,?,?,?,?,?,?,?)", b.ID, b.Version, b.Status, b.Created, b.RequestedBy, b.RecipeRev, b.Platform, b.Experimental, b.ReleaseID, b.SourceRepository, b.SourceCommit); err != nil {
return err return err
} }
if _, err = tx.Exec("INSERT INTO image_build_requests(user_id,build_id,created_at) VALUES(?,?,?)", user, b.ID, b.Created); err != nil { if _, err = tx.Exec("INSERT INTO image_build_requests(user_id,build_id,created_at) VALUES(?,?,?)", user, b.ID, b.Created); err != nil {
@@ -261,7 +274,7 @@ func (a *API) reconcileBuilds(parent context.Context, records []buildRecord) err
if err != nil { if err != nil {
return err return err
} }
if actual == nil || actual.ID != b.ImageID || actual.Config.Labels["dsh-mgr.role"] != "runtime" || actual.Config.Labels["dsh-mgr.version"] != b.Version || actual.Config.Labels["dsh-mgr.recipe"] != b.RecipeRev || actual.platform() != b.Platform { if actual == nil || actual.ID != b.ImageID || actual.Config.Labels["dsh-mgr.role"] != "runtime" || actual.Config.Labels["dsh-mgr.version"] != b.Version || actual.Config.Labels["dsh-mgr.recipe"] != b.RecipeRev || actual.platform() != b.Platform || !releaseLabelsMatch(actual, *b) {
b.Status, b.Error = "missing", "image no longer available" b.Status, b.Error = "missing", "image no longer available"
if _, err := a.s.DB.Exec("UPDATE image_builds SET status=?,error=? WHERE id=? AND status='success'", b.Status, b.Error, b.ID); err != nil { if _, err := a.s.DB.Exec("UPDATE image_builds SET status=?,error=? WHERE id=? AND status='success'", b.Status, b.Error, b.ID); err != nil {
return err return err
+112 -124
View File
@@ -11,9 +11,30 @@ import (
"strings" "strings"
"sync/atomic" "sync/atomic"
"testing" "testing"
"time"
) )
func TestInspectedImagePlatform(t *testing.T) {
for _, tc := range []struct{ os, arch, variant, want string }{
{"linux", "arm64", "", "linux/arm64"},
{"linux", "arm64", "v8", "linux/arm64"},
{"linux", "arm64", "v9", "linux/arm64/v9"},
{"linux", "arm64", "v8.1", "linux/arm64/v8.1"},
{"linux", "arm", "v7", "linux/arm/v7"},
{"linux", "arm", "v8", "linux/arm/v8"},
{"linux", "amd64", "", "linux/amd64"},
{"linux", "amd64", "v8", "linux/amd64/v8"},
{"", "arm64", "v8", ""},
{"linux", "", "v8", ""},
} {
t.Run(tc.os+"/"+tc.arch+"/"+tc.variant, func(t *testing.T) {
im := inspectedImage{Os: tc.os, Architecture: tc.arch, Variant: tc.variant}
if got := im.platform(); got != tc.want {
t.Fatalf("platform() = %q, want %q", got, tc.want)
}
})
}
}
func TestImageLegacyMigrationNeverCertifies(t *testing.T) { func TestImageLegacyMigrationNeverCertifies(t *testing.T) {
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/json" { if r.URL.Path == "/v1.47/images/json" {
@@ -174,7 +195,7 @@ func TestImageIndexLocalInventory(t *testing.T) {
} }
deleted.Store(true) deleted.Store(true)
w = imageRequest(a.images, sid, "GET", "") w = imageRequest(a.images, sid, "GET", "")
if err := json.Unmarshal(w.Body.Bytes(), &images); err != nil || w.Code != 200 || len(images) != 1 || images[0].Status != "missing" { if err := json.Unmarshal(w.Body.Bytes(), &images); err != nil || w.Code != 200 || len(images) != 0 {
t.Fatal(w.Body.String(), err) t.Fatal(w.Body.String(), err)
} }
}) })
@@ -227,41 +248,109 @@ func TestImageIndexUsageMissingAndDockerFailure(t *testing.T) {
t.Fatal(status, err) t.Fatal(status, err)
} }
mode.Store(2) mode.Store(2)
check("missing", 2) for refresh := 0; refresh < 3; refresh++ {
if _, err := a.s.DB.Exec("DELETE FROM instances WHERE id='two'"); err != nil { w := imageRequest(a.images, sid, "GET", "")
t.Fatal(err) if w.Code != 200 || strings.TrimSpace(w.Body.String()) != "[]" {
t.Fatal(w.Code, w.Body.String())
}
var count int
if err := a.s.DB.QueryRow("SELECT count(*) FROM images").Scan(&count); err != nil || count != 0 {
t.Fatal(count, err)
}
}
var count int
if err := a.s.DB.QueryRow("SELECT count(*) FROM instances WHERE image_id='image'").Scan(&count); err != nil || count != 2 {
t.Fatal(count, err)
}
if err := a.s.DB.QueryRow("SELECT status FROM image_builds WHERE image_id='image'").Scan(&status); err != nil || status != "missing" {
t.Fatal(status, err)
} }
check("missing", 1)
verified, err := a.verifiedImages(context.Background()) verified, err := a.verifiedImages(context.Background())
if err != nil || len(verified) != 0 { if err != nil || len(verified) != 0 {
t.Fatal(verified, err) t.Fatal(verified, err)
} }
} }
func TestBuildRateLimitDoesNotPenalizeSharedTask(t *testing.T) { func TestImageIndexFailurePreservesRows(t *testing.T) {
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { for _, mode := range []string{"list", "inspect", "rollback"} {
if r.URL.Path == "/v1.47/images/verified/json" { t.Run(mode, func(t *testing.T) {
writeImageInspect(w, "verified", "0.2.0-rc.1") a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/json" {
if mode == "list" {
w.WriteHeader(500)
} else {
writeJSON(w, []any{})
}
return
}
if mode == "inspect" && strings.Contains(r.URL.Path, "second") {
w.WriteHeader(500)
return
}
w.WriteHeader(404)
})
seedVerifiedImage(t, a, "first", "1.0.0")
seedVerifiedImage(t, a, "second", "2.0.0")
if _, err := a.s.DB.Exec("INSERT INTO images(image_id,version,status,built_at) VALUES('first','1.0.0','available','2'),('second','2.0.0','available','1')"); err != nil {
t.Fatal(err)
}
if mode == "rollback" {
if _, err := a.s.DB.Exec("CREATE TRIGGER deny_second_delete BEFORE DELETE ON images WHEN OLD.image_id='second' BEGIN SELECT RAISE(FAIL,'unavailable'); END"); err != nil {
t.Fatal(err)
}
}
if images, err := a.imageIndex(context.Background()); err == nil || images != nil {
t.Fatalf("images=%v err=%v", images, err)
}
for _, query := range []string{"SELECT count(*) FROM images WHERE status='available'", "SELECT count(*) FROM image_builds WHERE status='success'"} {
var count int
if err := a.s.DB.QueryRow(query).Scan(&count); err != nil || count != 2 {
t.Fatal(query, count, err)
}
}
})
}
}
func TestImageIndexMissingHistoryAndBlockedReimport(t *testing.T) {
var present atomic.Bool
var inspections atomic.Int32
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/json" {
if present.Load() {
writeJSON(w, []any{map[string]any{"Id": "image", "Labels": map[string]string{"dsh-mgr.role": "runtime"}}})
} else {
writeJSON(w, []any{})
}
return return
} }
io.WriteString(w, `[{"Id":"verified","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"}}]`) inspections.Add(1)
if !present.Load() {
w.WriteHeader(404)
return
}
writeImageInspect(w, "image", "0.2.0-rc.1")
}) })
seedVerifiedImage(t, a, "verified", "0.2.0-rc.1") seedVerifiedImage(t, a, "image", "0.2.0-rc.1")
for _, id := range []string{"a", "b", "c"} { if _, err := a.s.DB.Exec("UPDATE image_admissions SET state='blocked',evidence='unsafe'"); err != nil {
if _, err := a.s.DB.Exec("INSERT INTO image_build_requests VALUES('u',?,?)", id, time.Now().UTC().Format(time.RFC3339Nano)); err != nil { t.Fatal(err)
t.Fatal(err) }
for refresh := 0; refresh < 3; refresh++ {
if images, err := a.imageIndex(context.Background()); err != nil || len(images) != 0 {
t.Fatal(images, err)
} }
} }
if w := imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.1"}`); w.Code != 202 { if inspections.Load() != 1 {
t.Fatal(w.Code, w.Body.String()) t.Fatalf("missing history caused %d inspections", inspections.Load())
} }
if w := imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.2"}`); w.Code != 429 || w.Header().Get("Retry-After") == "" { present.Store(true)
t.Fatal(w.Code, w.Body.String()) images, err := a.imageIndex(context.Background())
if err != nil || len(images) != 1 || images[0].Status != "blocked" || images[0].Selectable || images[0].AdmissionReason != "unsafe" {
t.Fatal(images, err)
} }
var n int var status string
a.s.DB.QueryRow("SELECT count(*) FROM image_build_requests").Scan(&n) if err := a.s.DB.QueryRow("SELECT status FROM image_builds WHERE image_id='image'").Scan(&status); err != nil || status != "missing" {
if n != 3 { t.Fatal(status, err)
t.Fatal(n)
} }
} }
@@ -310,62 +399,6 @@ func TestBuildDiskPolicy(t *testing.T) {
} }
} }
func TestUnknownDockerDiskBlocksBuild(t *testing.T) {
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1.47/images/json" {
t.Errorf("unexpected Docker operation %s", r.URL.Path)
}
io.WriteString(w, "[]")
})
t.Setenv("DSH_DOCKER_ROOT_VISIBLE", "")
w := imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.1"}`)
if w.Code != 202 {
t.Fatal(w.Body.String())
}
deadline := time.Now().Add(3 * time.Second)
for {
var status, detail string
if err := a.s.DB.QueryRow("SELECT status,error FROM image_builds").Scan(&status, &detail); err != nil {
t.Fatal(err)
}
if status == "failed" {
if !strings.Contains(detail, "capacity unknown") {
t.Fatal(detail)
}
break
}
if time.Now().After(deadline) {
t.Fatal(status, detail)
}
time.Sleep(time.Millisecond)
}
}
func TestExactNPMMetadata(t *testing.T) {
for _, test := range []struct {
body string
status int
ok bool
}{
{`{"version":"0.2.0-rc.1"}`, 200, true},
{`{"version":"0.2.0-rc.2"}`, 200, false},
{`{"version":"latest"}`, 200, false},
{`{}`, 404, false},
{`invalid`, 200, false},
} {
client := &http.Client{Transport: imageRoundTrip(func(r *http.Request) (*http.Response, error) {
if r.URL.Host != "registry.npmjs.org" || !strings.HasSuffix(r.URL.Path, "/0.2.0-rc.1") {
t.Error(r.URL)
}
return &http.Response{StatusCode: test.status, Body: io.NopCloser(strings.NewReader(test.body)), Header: make(http.Header)}, nil
})}
_, err := fetchNPMVersion(context.Background(), client, "0.2.0-rc.1")
if (err == nil) != test.ok {
t.Fatalf("%s: %v", test.body, err)
}
}
}
func TestBuildListReconcilesMissingAndLatestTimestamp(t *testing.T) { func TestBuildListReconcilesMissingAndLatestTimestamp(t *testing.T) {
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(404) }) a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(404) })
seedVerifiedImage(t, a, "deleted", "0.2.0-rc.1") seedVerifiedImage(t, a, "deleted", "0.2.0-rc.1")
@@ -374,49 +407,4 @@ func TestBuildListReconcilesMissingAndLatestTimestamp(t *testing.T) {
if err := json.Unmarshal(w.Body.Bytes(), &records); err != nil || len(records) != 1 || records[0].Status != "missing" { if err := json.Unmarshal(w.Body.Bytes(), &records); err != nil || len(records) != 1 || records[0].Status != "missing" {
t.Fatal(w.Body.String()) t.Fatal(w.Body.String())
} }
w = imageRequest(a.latest, sid, "GET", "")
var latest struct {
Version string `json:"version"`
CheckedAt string `json:"checked_at"`
}
if err := json.Unmarshal(w.Body.Bytes(), &latest); err != nil {
t.Fatal(err)
}
if _, err := time.Parse(time.RFC3339, latest.CheckedAt); err != nil || latest.Version != "0.2.0-rc.1" {
t.Fatal(latest, err)
}
}
// The one-second intent poller must reuse per-image evaluations instead of
// re-inspecting Docker every tick, and a generation bump (deletion or
// admission change) must invalidate the cache.
func TestIntentPollerCachesImageEvaluations(t *testing.T) {
inspects := 0
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/image/json" {
inspects++
writeImageInspect(w, "image", "0.2.0-rc.1")
return
}
w.WriteHeader(404)
})
seedVerifiedImage(t, a, "image", "0.2.0-rc.1")
for n := 0; n < 3; n++ {
verified, err := a.verifiedImagesPoll(context.Background())
if err != nil || len(verified) != 1 || verified["image"] != "0.2.0-rc.1" {
t.Fatal(verified, err)
}
}
if inspects != 1 {
t.Fatalf("poller re-inspected Docker %d times for the same image", inspects)
}
buildQueue.Lock()
a.imageDeleteGeneration++
buildQueue.Unlock()
if _, err := a.verifiedImagesPoll(context.Background()); err != nil {
t.Fatal(err)
}
if inspects != 2 {
t.Fatalf("generation bump did not invalidate the cache: %d inspects", inspects)
}
} }
+24 -272
View File
@@ -1,7 +1,6 @@
package api package api
import ( import (
"archive/tar"
"context" "context"
"encoding/json" "encoding/json"
"fmt" "fmt"
@@ -10,10 +9,8 @@ import (
"io" "io"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"os"
"path/filepath" "path/filepath"
"strings" "strings"
"sync"
"sync/atomic" "sync/atomic"
"testing" "testing"
"time" "time"
@@ -24,11 +21,6 @@ func imageTestAPI(t *testing.T, h http.HandlerFunc) (*API, string) {
oldDiskAvailable := buildDiskAvailable oldDiskAvailable := buildDiskAvailable
buildDiskAvailable = func(string) (uint64, error) { return minimumBuildDiskBytes, nil } buildDiskAvailable = func(string) (uint64, error) { return minimumBuildDiskBytes, nil }
t.Cleanup(func() { buildDiskAvailable = oldDiskAvailable }) t.Cleanup(func() { buildDiskAvailable = oldDiskAvailable })
oldClient := npmBuildClient
npmBuildClient = &http.Client{Transport: imageRoundTrip(func(r *http.Request) (*http.Response, error) {
return &http.Response{StatusCode: 200, Header: make(http.Header), Body: io.NopCloser(strings.NewReader(`{"version":"0.2.0-rc.1"}`))}, nil
})}
t.Cleanup(func() { npmBuildClient = oldClient })
root := t.TempDir() root := t.TempDir()
t.Setenv("DSH_DOCKER_ROOT_VISIBLE", root) t.Setenv("DSH_DOCKER_ROOT_VISIBLE", root)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -129,34 +121,7 @@ func seedVerifiedImage(t *testing.T, a *API, id, version string) {
t.Fatal(err) t.Fatal(err)
} }
} }
func TestImageContextAndSanitizing(t *testing.T) { func TestImageSanitizing(t *testing.T) {
r, err := imageBuildContext()
if err != nil {
t.Fatal(err)
}
tr := tar.NewReader(r)
n := 0
for {
h, err := tr.Next()
if err == io.EOF {
break
}
if err != nil {
t.Fatal(err)
}
data, _ := io.ReadAll(tr)
original, err := os.ReadFile(filepath.Join("..", "..", "image", h.Name))
if err != nil || string(data) != string(original) {
t.Fatalf("recipe drift %s: %v", h.Name, err)
}
if h.Name == "Dockerfile" && (!strings.Contains(string(data), "ARG RECIPE_REV="+imageRecipeRev+"\n") || !strings.Contains(string(data), "dsh-mgr.recipe=$RECIPE_REV")) {
t.Fatal("recipe revision or label differs from image policy")
}
n++
}
if n != 7 {
t.Fatal(n)
}
s := sanitizeBuildLog("token=topsecret Authorization: Bearer hidden password=private https://user:[email protected]/") s := sanitizeBuildLog("token=topsecret Authorization: Bearer hidden password=private https://user:[email protected]/")
for _, secret := range []string{"topsecret", "hidden", "private", "user:pass"} { for _, secret := range []string{"topsecret", "hidden", "private", "user:pass"} {
if strings.Contains(s, secret) { if strings.Contains(s, secret) {
@@ -176,129 +141,6 @@ func TestImageContextAndSanitizing(t *testing.T) {
} }
} }
type imageRoundTrip func(*http.Request) (*http.Response, error)
func (f imageRoundTrip) RoundTrip(r *http.Request) (*http.Response, error) { return f(r) }
func TestLatestOfficialRegistry(t *testing.T) {
client := &http.Client{Transport: imageRoundTrip(func(r *http.Request) (*http.Response, error) {
if r.URL.Host != "registry.npmjs.org" || r.URL.Scheme != "https" {
t.Fatal(r.URL)
}
return &http.Response{StatusCode: 200, Body: io.NopCloser(strings.NewReader(`{"version":"0.2.0-rc.1"}`)), Header: make(http.Header)}, nil
})}
v, err := fetchLatest(context.Background(), client)
if err != nil || v != "0.2.0-rc.1" {
t.Fatalf("%s %v", v, err)
}
}
func TestBuildDedupAndVerification(t *testing.T) {
var calls, deleted atomic.Int32
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v1.47/images/sha256:abc/json", "/v1.47/images/sha256:reuse/json":
writeImageInspect(w, strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/v1.47/images/"), "/json"), "0.2.0-rc.1")
case "/v1.47/images/json":
if calls.Load() == 0 {
io.WriteString(w, "[]")
} else {
io.WriteString(w, `[{"Id":"sha256:abc","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"}}]`)
}
case "/v1.47/build":
calls.Add(1)
if r.URL.Query().Get("t") != "dsh-web:0.2.0-rc.1-"+imageRecipeRev {
t.Error("unexpected tag", r.URL.Query())
}
var args map[string]string
json.Unmarshal([]byte(r.URL.Query().Get("buildargs")), &args)
if args["DSH_VERSION"] != "0.2.0-rc.1" {
t.Error(args)
}
io.Copy(io.Discard, r.Body)
io.WriteString(w, `{"stream":"token=secretvalue"}`+string(rune(10))+`{"aux":{"ID":"sha256:abc"}}`)
case "/v1.47/containers/create":
var cfg struct {
Image string
HostConfig struct {
NetworkMode string
ReadonlyRootfs bool
Tmpfs map[string]string
}
}
json.NewDecoder(r.Body).Decode(&cfg)
if cfg.Image != "sha256:abc" || cfg.HostConfig.NetworkMode != "none" || !cfg.HostConfig.ReadonlyRootfs || !strings.Contains(cfg.HostConfig.Tmpfs["/tmp"], "exec") {
t.Errorf("unsafe config %+v", cfg)
}
io.WriteString(w, `{"Id":"check"}`)
case "/v1.47/containers/check/start", "/v1.47/exec/probe/start":
w.WriteHeader(204)
case "/v1.47/containers/check/exec":
var cfg struct{ Cmd []string }
if err := json.NewDecoder(r.Body).Decode(&cfg); err != nil || len(cfg.Cmd) != 3 || !strings.Contains(cfg.Cmd[2], "execFileSync('dsh',['--version']") || !strings.Contains(cfg.Cmd[2], `actual!=="0.2.0-rc.1"`) {
t.Errorf("missing version verification: %+v %v", cfg, err)
}
io.WriteString(w, `{"Id":"probe"}`)
case "/v1.47/exec/probe/json":
io.WriteString(w, `{"Running":false,"ExitCode":0}`)
case "/v1.47/containers/check/json":
io.WriteString(w, `{"State":{"Running":true}}`)
case "/v1.47/containers/check":
deleted.Add(1)
w.WriteHeader(204)
default:
t.Errorf("unexpected %s", r.URL)
w.WriteHeader(500)
}
})
responses := make(chan *httptest.ResponseRecorder, 8)
var wg sync.WaitGroup
for i := 0; i < 8; i++ {
wg.Add(1)
go func() {
defer wg.Done()
responses <- imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.1"}`)
}()
}
wg.Wait()
close(responses)
first := ""
for w := range responses {
if w.Code != 202 {
t.Fatal(w.Body.String())
}
var b buildRecord
json.Unmarshal(w.Body.Bytes(), &b)
if first == "" {
first = b.ID
}
if first != b.ID {
t.Fatal("duplicate job")
}
}
deadline := time.Now().Add(5 * time.Second)
for {
w := imageRequest(a.builds, sid, "GET", "")
var jobs []buildRecord
json.Unmarshal(w.Body.Bytes(), &jobs)
if len(jobs) == 1 && jobs[0].Status == "success" {
if strings.Contains(jobs[0].Logs, "secretvalue") {
t.Fatal(jobs)
}
break
}
if time.Now().After(deadline) {
t.Fatal(w.Body.String())
}
time.Sleep(time.Millisecond)
}
if calls.Load() != 1 || deleted.Load() != 1 {
t.Fatalf("builds %d cleanup %d", calls.Load(), deleted.Load())
}
for _, body := range []string{`{"version":"latest"}`, `{"version":"0.2.0-rc.1","url":"bad"}`, `{"version":"0.2.0-rc.1"} {}`} {
if w := imageRequest(a.builds, sid, "POST", body); w.Code != 400 {
t.Fatal(w.Code)
}
}
}
func TestVerificationFailureCleansUp(t *testing.T) { func TestVerificationFailureCleansUp(t *testing.T) {
var deleted atomic.Bool var deleted atomic.Bool
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
@@ -321,86 +163,6 @@ func TestVerificationFailureCleansUp(t *testing.T) {
} }
} }
func TestBuildReuseAndMissingImage(t *testing.T) {
var exists atomic.Bool
exists.Store(true)
var builds, checks atomic.Int32
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v1.47/images/sha256:abc/json", "/v1.47/images/sha256:reuse/json":
writeImageInspect(w, strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/v1.47/images/"), "/json"), "0.2.0-rc.1")
case "/v1.47/images/json":
if exists.Load() {
io.WriteString(w, `[{"Id":"sha256:reuse","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"}}]`)
} else {
io.WriteString(w, "[]")
}
case "/v1.47/build":
builds.Add(1)
exists.Store(true)
io.Copy(io.Discard, r.Body)
io.WriteString(w, `{"aux":{"ID":"sha256:reuse"}}`)
case "/v1.47/containers/create":
checks.Add(1)
io.WriteString(w, `{"Id":"c"}`)
case "/v1.47/containers/c/start", "/v1.47/exec/e/start", "/v1.47/containers/c":
w.WriteHeader(204)
case "/v1.47/containers/c/exec":
io.WriteString(w, `{"Id":"e"}`)
case "/v1.47/exec/e/json":
io.WriteString(w, `{"Running":false,"ExitCode":0}`)
case "/v1.47/containers/c/json":
io.WriteString(w, `{"State":{"Running":true}}`)
default:
t.Errorf("unexpected %s", r.URL)
w.WriteHeader(500)
}
})
wait := func(id string) {
t.Helper()
deadline := time.Now().Add(5 * time.Second)
for {
var status, image string
err := a.s.DB.QueryRow("SELECT status,image_id FROM image_builds WHERE id=?", id).Scan(&status, &image)
if err == nil && status == "success" {
if image != "sha256:reuse" {
t.Fatal(image)
}
return
}
if time.Now().After(deadline) {
t.Fatalf("status %s: %v", status, err)
}
time.Sleep(time.Millisecond)
}
}
post := func() buildRecord {
t.Helper()
w := imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.1"}`)
if w.Code != 202 {
t.Fatal(w.Body.String())
}
var b buildRecord
if err := json.Unmarshal(w.Body.Bytes(), &b); err != nil {
t.Fatal(err)
}
return b
}
first := post()
wait(first.ID)
if builds.Load() != 0 || checks.Load() != 1 {
t.Fatal("local image not reused and verified")
}
exists.Store(false)
second := post()
if second.ID == first.ID {
t.Fatal("missing image deduplicated")
}
wait(second.ID)
if builds.Load() != 1 || checks.Load() != 2 {
t.Fatal("missing image not rebuilt")
}
}
func TestBuildRestartRecovery(t *testing.T) { func TestBuildRestartRecovery(t *testing.T) {
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(500) }) a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(500) })
buildQueue.Lock() buildQueue.Lock()
@@ -427,6 +189,7 @@ func TestBuildRestartRecovery(t *testing.T) {
func TestFailedVerificationNeverAvailable(t *testing.T) { func TestFailedVerificationNeverAvailable(t *testing.T) {
var deleted atomic.Bool var deleted atomic.Bool
imageID := "sha256:" + strings.Repeat("c", 64) imageID := "sha256:" + strings.Repeat("c", 64)
info := releaseFixture()
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path { switch r.URL.Path {
case "/v1.47/images/sha256:abc/json", "/v1.47/images/sha256:reuse/json": case "/v1.47/images/sha256:abc/json", "/v1.47/images/sha256:reuse/json":
@@ -444,33 +207,36 @@ func TestFailedVerificationNeverAvailable(t *testing.T) {
case "/v1.47/containers/check": case "/v1.47/containers/check":
deleted.Store(true) deleted.Store(true)
w.WriteHeader(204) w.WriteHeader(204)
case "/v1.47/build":
writeJSON(w, map[string]any{"aux": map[string]string{"ID": imageID}})
case "/v1.47/images/" + imageID + "/json": case "/v1.47/images/" + imageID + "/json":
writeImageInspect(w, imageID, "0.2.0-rc.1") parts := strings.Split(imagePlatform, "/")
writeJSON(w, map[string]any{"Id": imageID, "Os": parts[0], "Architecture": parts[1], "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": info.Version, "dsh-mgr.recipe": imageRecipeRev, "dsh-mgr.release": info.ReleaseID, "dsh-mgr.source": info.Repository, "dsh-mgr.commit": info.Commit}}})
default: default:
t.Errorf("unexpected Docker call %s", r.URL) t.Errorf("unexpected Docker call %s", r.URL)
w.WriteHeader(500) w.WriteHeader(500)
} }
}) })
w := imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.1"}`) if _, err := a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='u'"); err != nil {
if w.Code != 202 { t.Fatal(err)
t.Fatal(w.Body.String())
} }
deadline := time.Now().Add(5 * time.Second) old := fetchRelease
for { t.Cleanup(func() { fetchRelease = old })
var status, id, detail string fetchRelease = func(context.Context, string, string) (releaseInfo, io.Reader, error) {
err := a.s.DB.QueryRow("SELECT status,image_id,error FROM image_builds").Scan(&status, &id, &detail) return info, strings.NewReader("context"), nil
if err == nil && status == "failed" {
if id != imageID || !strings.Contains(detail, "verification failed") || !deleted.Load() {
t.Fatalf("%s %s cleanup=%v", id, detail, deleted.Load())
}
break
}
if time.Now().After(deadline) {
t.Fatalf("status=%s error=%v", status, err)
}
time.Sleep(time.Millisecond)
} }
w = imageRequest(a.images, sid, "GET", "") b, err := a.enqueueRelease("u", info)
if err != nil {
t.Fatal(err)
}
a.runImageBuild(b)
if err := scanBuildSummary(a.s.DB.QueryRow("SELECT "+buildSummaryColumns+" FROM image_builds WHERE id=?", b.ID), &b); err != nil {
t.Fatal(err)
}
if b.Status != "failed" || b.ImageID != imageID || !strings.Contains(b.Error, "verification failed") || !deleted.Load() {
t.Fatalf("%+v cleanup=%v", b, deleted.Load())
}
w := imageRequest(a.images, sid, "GET", "")
var images []imageRecord var images []imageRecord
if err := json.Unmarshal(w.Body.Bytes(), &images); err != nil || w.Code != 200 || len(images) != 1 || images[0].Status != "unverified" { if err := json.Unmarshal(w.Body.Bytes(), &images); err != nil || w.Code != 200 || len(images) != 1 || images[0].Status != "unverified" {
t.Fatal("failed image advertised", w.Body.String()) t.Fatal("failed image advertised", w.Body.String())
@@ -481,20 +247,6 @@ func TestFailedVerificationNeverAvailable(t *testing.T) {
} }
} }
func TestLocalBuildImagePrefersPersistedID(t *testing.T) {
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/verified/json" {
writeImageInspect(w, "verified", "0.2.0-rc.1")
return
}
io.WriteString(w, `[{"Id":"newer","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"}},{"Id":"verified","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"}}]`)
})
id, err := a.localBuildImage(context.Background(), "0.2.0-rc.1", "verified")
if err != nil || id != "verified" {
t.Fatalf("id=%s err=%v", id, err)
}
}
func writeImageInspect(w http.ResponseWriter, id, version string) { func writeImageInspect(w http.ResponseWriter, id, version string) {
parts := strings.Split(imagePlatform, "/") parts := strings.Split(imagePlatform, "/")
writeJSON(w, map[string]any{"Id": id, "Os": parts[0], "Architecture": parts[1], "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": version, "dsh-mgr.recipe": imageRecipeRev}}}) writeJSON(w, map[string]any{"Id": id, "Os": parts[0], "Architecture": parts[1], "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": version, "dsh-mgr.recipe": imageRecipeRev}}})
+57
View File
@@ -0,0 +1,57 @@
package api
import (
"errors"
"os"
"strings"
"git.misaka.ren/m1saka/dsh-web/internal/store"
)
// Call only with the instance lock held, after Docker resources are removed.
func (a *API) removeInstanceData(i store.Instance) error {
for _, id := range []string{i.UID, i.ID} {
if id == "" || id == "." || id == ".." || strings.ContainsAny(id, "/\\") {
return errors.New("invalid instance storage path")
}
}
for _, parts := range [][]string{{"workspaces", i.UID, "instances", i.ID}, {"backups", i.UID, i.ID}} {
if err := a.removeDataDirectory(parts); err != nil {
return err
}
}
return nil
}
// Walk parents without following symlinks; RemoveAll unlinks child symlinks.
func (a *API) removeDataDirectory(parts []string) error {
root, err := os.OpenRoot(a.root)
if err != nil {
return err
}
defer func() { root.Close() }()
for _, part := range parts[:len(parts)-1] {
st, err := root.Lstat(part)
if errors.Is(err, os.ErrNotExist) {
return nil
}
if err != nil {
return err
}
if !st.IsDir() {
return errors.New("unsafe storage path")
}
next, err := root.OpenRoot(part)
if err != nil {
return err
}
actual, err := next.Stat(".")
if err != nil || !os.SameFile(st, actual) {
next.Close()
return errors.New("storage path changed")
}
root.Close()
root = next
}
return root.RemoveAll(parts[len(parts)-1])
}
+110
View File
@@ -0,0 +1,110 @@
package api
import (
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"git.misaka.ren/m1saka/dsh-web/internal/store"
)
func TestInstanceDeleteDataAndRetry(t *testing.T) {
for _, blocked := range []bool{false, true} {
t.Run(map[bool]string{false: "success", true: "retry"}[blocked], func(t *testing.T) {
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodDelete {
w.WriteHeader(204)
} else {
w.WriteHeader(404)
}
})
i := store.Instance{ID: "one", UID: "owner", ContainerID: "container"}
if err := a.s.AddInstance(i); err != nil {
t.Fatal(err)
}
write := func(rel string) {
p := filepath.Join(a.root, rel)
if err := os.MkdirAll(filepath.Dir(p), 0750); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, []byte("keep"), 0600); err != nil {
t.Fatal(err)
}
}
write("workspaces/owner/instances/one/home/data")
write("workspaces/owner/instances/one/pre-restore-test/data")
write("workspaces/owner/instances/two/home/data")
write("backups/owner/one/archive.dshbak")
write("backups/owner/two/archive.dshbak")
outside := t.TempDir()
sentinel := filepath.Join(outside, "sentinel")
if err := os.WriteFile(sentinel, []byte("safe"), 0600); err != nil {
t.Fatal(err)
}
if err := os.Symlink(outside, filepath.Join(a.root, "workspaces/owner/instances/one/home/link")); err != nil {
t.Fatal(err)
}
backups := filepath.Join(a.root, "backups/owner")
if blocked {
if err := os.Rename(backups, backups+"-saved"); err != nil {
t.Fatal(err)
}
if err := os.Symlink(backups+"-saved", backups); err != nil {
t.Fatal(err)
}
}
remove := func() *httptest.ResponseRecorder {
w := httptest.NewRecorder()
a.instance(w, runtimeRequest("DELETE", "/api/instances/one", "", sid))
return w
}
w := remove()
if blocked {
if w.Code != 500 {
t.Fatal(w.Code, w.Body.String())
}
current, err := a.s.GetInstance(i.ID)
if err != nil || current.Status != "error" {
t.Fatal(current, err)
}
if err := os.Remove(backups); err != nil {
t.Fatal(err)
}
if err := os.Rename(backups+"-saved", backups); err != nil {
t.Fatal(err)
}
w = remove()
}
if w.Code != 200 {
t.Fatal(w.Code, w.Body.String())
}
for _, rel := range []string{"workspaces/owner/instances/one", "backups/owner/one"} {
if _, err := os.Lstat(filepath.Join(a.root, rel)); !os.IsNotExist(err) {
t.Fatal("data retained", rel, err)
}
}
for _, p := range []string{sentinel, filepath.Join(a.root, "workspaces/owner/instances/two/home/data"), filepath.Join(a.root, "backups/owner/two/archive.dshbak")} {
if _, err := os.Stat(p); err != nil {
t.Fatal("unrelated data removed", err)
}
}
})
}
}
func TestUsageDoesNotCreateLegacyFiles(t *testing.T) {
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Fatal("unexpected Docker request") })
w := httptest.NewRecorder()
a.workspace(w, runtimeRequest("GET", "/api/ws/usage", "", sid))
if w.Code != 200 {
t.Fatal(w.Code, w.Body.String())
}
if _, err := os.Stat(filepath.Join(a.root, "workspaces/owner/files")); !os.IsNotExist(err) {
t.Fatal("legacy directory created", err)
}
if _, err := a.openWorkspace(store.User{ID: "owner"}, ""); err == nil {
t.Fatal("shared workspace accepted")
}
}
-233
View File
@@ -1,233 +0,0 @@
package api
import (
"context"
"database/sql"
"encoding/json"
"errors"
"io"
"net/http"
"strings"
"time"
"git.misaka.ren/m1saka/dsh-web/internal/store"
)
func (a *API) intents(w http.ResponseWriter, r *http.Request) {
u, _, err := a.user(r)
if err != nil {
fail(w, 401, "authentication required")
return
}
if err = a.s.EnsureIntents(); err != nil {
fail(w, 500, "database error")
return
}
switch r.Method {
case http.MethodGet:
list, err := a.s.Intents(u.ID)
if err != nil {
fail(w, 500, "database error")
return
}
writeJSON(w, list)
case http.MethodPost:
var in struct {
Name string `json:"name"`
Version string `json:"version"`
}
dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096))
dec.DisallowUnknownFields()
if dec.Decode(&in) != nil || dec.Decode(new(any)) != io.EOF || strings.TrimSpace(in.Name) == "" || len(in.Name) > 200 || !validBuildVersion(in.Version) {
fail(w, 400, "name and fixed version required")
return
}
if !normalBuildAllowed(in.Version) {
fail(w, 400, errBuildUnsupported.Error())
return
}
i := store.Intent{ID: store.ID(), UID: u.ID, Name: strings.TrimSpace(in.Name), Version: in.Version, Host: a.runtimeHost(r), Status: "waiting"}
if err = a.s.ReserveIntent(i); err != nil {
if errors.Is(err, store.ErrInstanceLimit) {
fail(w, 409, err.Error())
} else {
fail(w, 500, "database error")
}
return
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusAccepted)
writeJSON(w, i)
default:
fail(w, 405, "method not allowed")
}
}
func (a *API) intent(w http.ResponseWriter, r *http.Request) {
u, _, err := a.user(r)
if err != nil {
fail(w, 401, "authentication required")
return
}
if r.Method != http.MethodDelete {
fail(w, 405, "method not allowed")
return
}
if err = a.s.EnsureIntents(); err != nil {
fail(w, 500, "database error")
return
}
id := strings.TrimPrefix(r.URL.Path, "/api/intents/")
if err = a.s.CancelIntent(id, u.ID); err != nil {
if errors.Is(err, store.ErrIntentState) {
fail(w, 409, "intent not found or no longer waiting")
} else {
fail(w, 500, "database error")
}
return
}
writeJSON(w, map[string]string{"status": "cancelled"})
}
// Call once at startup, after the build queue has recovered interrupted jobs.
func (a *API) StartIntents(ctx context.Context) error {
if err := a.s.EnsureIntents(); err != nil {
return err
}
if err := a.recoverIntents(ctx); err != nil {
return err
}
go func() {
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
default:
}
a.processIntents(ctx)
select {
case <-ctx.Done():
return
case <-ticker.C:
}
}
}()
return nil
}
func (a *API) finishIntent(id, status, detail string) {
_, _ = a.s.DB.Exec(`UPDATE creation_intents SET status=?,error=? WHERE id=? AND status IN ('waiting','creating')`, status, detail, id)
}
func (a *API) recoverIntents(ctx context.Context) error {
list, err := a.s.Intents("")
if err != nil {
return err
}
for _, in := range list {
if in.Status != "creating" {
continue
}
i, err := a.s.GetInstance(in.InstanceID)
if errors.Is(err, sql.ErrNoRows) {
a.finishIntent(in.ID, "failed", "instance reservation missing after process restart")
continue
}
if err != nil {
return err
}
if err == nil && i.Status == "running" {
a.finishIntent(in.ID, "success", "")
continue
}
cleanup, cancel := context.WithTimeout(ctx, 30*time.Second)
if err == nil && i.ContainerID == "" {
err = a.recoverRuntimeContainer(cleanup, &i)
}
if err == nil && i.ContainerID != "" {
err = a.removeRuntimeContainer(cleanup, i.ContainerID)
}
if err == nil {
err = a.removeRuntimeNetwork(cleanup, in.InstanceID)
}
cancel()
if err != nil {
a.s.SetInstance(in.InstanceID, "error")
a.finishIntent(in.ID, "failed", "instance creation requires recovery: "+err.Error())
a.s.AddAudit(in.UID, "instance.recovery_error", in.InstanceID, err.Error())
continue
}
a.s.DeleteInstance(in.InstanceID)
_, _ = a.s.DB.Exec("DELETE FROM runtime_creates WHERE instance_id=?", in.InstanceID)
_, _ = a.s.DB.Exec("DELETE FROM runtime_networks WHERE instance_id=?", in.InstanceID)
a.finishIntent(in.ID, "failed", "instance creation interrupted by process restart")
}
return nil
}
func (a *API) processIntents(ctx context.Context) {
list, err := a.s.Intents("")
if err != nil {
return
}
for _, in := range list {
if ctx.Err() != nil {
return
}
if in.Status != "waiting" {
continue
}
if in.BuildID == "" {
b, err := a.enqueueBuild(ctx, in.UID, in.Version)
if err != nil {
a.finishIntent(in.ID, "failed", "build submission failed")
continue
}
if _, err = a.s.DB.Exec(`UPDATE creation_intents SET build_id=? WHERE id=? AND status='waiting'`, b.ID, in.ID); err != nil {
continue
}
in.BuildID = b.ID
}
var status, imageID, version string
err := a.s.DB.QueryRow(`SELECT status,image_id,version FROM image_builds WHERE id=?`, in.BuildID).Scan(&status, &imageID, &version)
if err != nil {
continue
}
if version != in.Version || status == "failed" || status == "cancelled" || status == "missing" {
a.finishIntent(in.ID, "failed", "build failed; requested version was not created")
continue
}
if status != "success" {
continue
}
work, cancel := context.WithTimeout(ctx, 90*time.Second)
verified, err := a.verifiedImagesPoll(work)
if err != nil {
cancel()
continue
}
if !runtimeImageID.MatchString(imageID) || verified[imageID] != in.Version {
cancel()
a.finishIntent(in.ID, "failed", "verified image for requested version unavailable")
continue
}
u, err := a.s.UserByID(in.UID)
if err != nil {
cancel()
a.finishIntent(in.ID, "failed", "owner unavailable")
continue
}
i := store.Instance{ID: store.ID(), UID: in.UID, Name: in.Name, Version: in.Version, ImageID: imageID, Status: "creating", Created: time.Now()}
if err = a.s.ClaimIntent(in.ID, i); err != nil {
cancel()
continue
}
_, err = a.createRuntime(work, u, i, in.Host)
cancel()
if err != nil {
a.finishIntent(in.ID, "failed", err.Error())
} else {
a.finishIntent(in.ID, "success", "")
}
}
}
-166
View File
@@ -1,166 +0,0 @@
package api
import (
"context"
"git.misaka.ren/m1saka/dsh-web/internal/store"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
)
func TestIntentsReserveCancel(t *testing.T) {
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("reservation contacted Docker") })
if err := a.initAuth(); err != nil {
t.Fatal(err)
}
for n := 0; n < 3; n++ {
w := httptest.NewRecorder()
a.intents(w, runtimeRequest("POST", "/api/intents", `{"name":"one","version":"0.2.0-rc.1"}`, sid))
want := 202
if n == 2 {
want = 409
}
if w.Code != want {
t.Fatalf("%d: %d %s", n, w.Code, w.Body.String())
}
}
list, err := a.s.Intents("owner")
if err != nil || len(list) != 2 {
t.Fatalf("%v %v", list, err)
}
w := httptest.NewRecorder()
a.intent(w, runtimeRequest("DELETE", "/api/intents/"+list[0].ID, "", sid))
if w.Code != 200 {
t.Fatal(w.Code)
}
if err := a.s.ReserveIntent(store.Intent{ID: store.ID(), UID: "owner"}); err != nil {
t.Fatal(err)
}
}
func TestIntentBuildSuccessUsesRealRuntime(t *testing.T) {
if os.Geteuid() != 0 {
t.Skip("runtime filesystem ownership requires root")
}
image := "sha256:" + strings.Repeat("a", 64)
creates := 0
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch {
case strings.Contains(r.URL.Path, "/images/"):
writeJSON(w, map[string]any{"Id": image, "Os": "linux", "Architecture": "amd64", "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "0.2.0-rc.1", "dsh-mgr.recipe": imageRecipeRev}}})
case strings.HasSuffix(r.URL.Path, "/containers/create"):
creates++
writeJSON(w, map[string]string{"Id": "runtime-created"})
default:
w.WriteHeader(204)
}
})
seedVerifiedImage(t, a, image, "0.2.0-rc.1")
if err := a.s.EnsureIntents(); err != nil {
t.Fatal(err)
}
var build string
if err := a.s.DB.QueryRow("SELECT id FROM image_builds WHERE version='0.2.0-rc.1'").Scan(&build); err != nil {
t.Fatal(err)
}
in := store.Intent{ID: store.ID(), UID: "owner", Name: "one", Version: "0.2.0-rc.1", Host: "panel.test"}
if err := a.s.ReserveIntent(in); err != nil {
t.Fatal(err)
}
if _, err := a.s.DB.Exec("UPDATE creation_intents SET build_id=? WHERE id=?", build, in.ID); err != nil {
t.Fatal(err)
}
a.processIntents(context.Background())
a.processIntents(context.Background())
list, err := a.s.Intents("owner")
if err != nil || len(list) != 1 || list[0].Status != "success" || creates != 1 {
t.Fatalf("%+v err=%v creates=%d", list, err, creates)
}
instances, err := a.s.Instances("owner")
if err != nil || len(instances) != 1 || instances[0].ImageID != image || instances[0].Version != "0.2.0-rc.1" {
t.Fatalf("%+v %v", instances, err)
}
}
func TestIntentFailedBuildReleasesAndCancelPreservesBuild(t *testing.T) {
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("failed intent contacted Docker") })
seedVerifiedImage(t, a, "sha256:"+strings.Repeat("b", 64), "0.2.0-rc.1")
if err := a.s.EnsureIntents(); err != nil {
t.Fatal(err)
}
var build string
a.s.DB.QueryRow("SELECT id FROM image_builds LIMIT 1").Scan(&build)
for _, id := range []string{"cancel", "fail"} {
if err := a.s.ReserveIntent(store.Intent{ID: id, UID: "owner", Version: "0.2.0-rc.1"}); err != nil {
t.Fatal(err)
}
a.s.DB.Exec("UPDATE creation_intents SET build_id=? WHERE id=?", build, id)
}
if err := a.s.CancelIntent("cancel", "owner"); err != nil {
t.Fatal(err)
}
var status string
a.s.DB.QueryRow("SELECT status FROM image_builds WHERE id=?", build).Scan(&status)
if status != "success" {
t.Fatal(status)
}
a.s.DB.Exec("UPDATE image_builds SET status='failed' WHERE id=?", build)
a.processIntents(context.Background())
for n := 0; n < 2; n++ {
if err := a.s.ReserveIntent(store.Intent{ID: store.ID(), UID: "owner"}); err != nil {
t.Fatal(err)
}
}
}
func TestIntentRestartCleansInterruptedCreationAndRetainsWaiting(t *testing.T) {
deletes := 0
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.Method == "GET" && r.URL.Path == "/v1.47/networks/dsh-inst-reserved" {
w.Write([]byte(`{"Id":"network","Name":"dsh-inst-reserved","Driver":"bridge","Labels":{"dsh-mgr.instance":"reserved"}}`))
return
}
if r.Method == "GET" && r.URL.Path == "/v1.47/containers/dsh-inst-reserved/json" {
w.Write([]byte(`{"Id":"recovered","Name":"/dsh-inst-reserved","Config":{"Labels":{"dsh-mgr.instance":"reserved","dsh-mgr.owner":"owner","dsh-mgr.role":"dsh"}}}`))
return
}
if r.Method == "DELETE" {
deletes++
}
w.WriteHeader(204)
})
if err := a.s.EnsureIntents(); err != nil {
t.Fatal(err)
}
for _, id := range []string{"waiting", "creating"} {
if err := a.s.ReserveIntent(store.Intent{ID: id, UID: "owner", Version: "0.2.0-rc.1"}); err != nil {
t.Fatal(err)
}
}
if err := a.s.ClaimIntent("creating", store.Instance{ID: "reserved", UID: "owner", Version: "0.2.0-rc.1", Status: "creating"}); err != nil {
t.Fatal(err)
}
if err := a.recoverIntents(context.Background()); err != nil {
t.Fatal(err)
}
if deletes != 2 {
t.Fatalf("cleanup calls=%d", deletes)
}
list, err := a.s.Intents("owner")
if err != nil {
t.Fatal(err)
}
for _, in := range list {
if in.ID == "waiting" && in.Status != "waiting" {
t.Fatal(in)
}
if in.ID == "creating" && in.Status != "failed" {
t.Fatal(in)
}
}
if err := a.s.ReserveIntent(store.Intent{ID: store.ID(), UID: "owner"}); err != nil {
t.Fatal(err)
}
}
-11
View File
@@ -62,17 +62,6 @@ func (a *API) checkOrphan(ctx context.Context, id string) error {
return errors.New("host root unavailable") return errors.New("host root unavailable")
} }
expected := map[string]string{"/workspace": filepath.Join(host, "workspaces", owner, "instances", inst, "home")} expected := map[string]string{"/workspace": filepath.Join(host, "workspaces", owner, "instances", inst, "home")}
// Accept complete historical layouts, never arbitrary bind sources.
for _, m := range c.Mounts {
if m.Destination == "/data/dsh-home" {
expected = map[string]string{"/workspace": filepath.Join(host, "workspaces", owner, "files"), "/data/dsh-home": filepath.Join(host, "workspaces", owner, "instances", inst, "home")}
}
}
for _, m := range c.Mounts {
if m.Destination == "/shared" {
expected["/shared"] = filepath.Join(host, "workspaces", owner, "files")
}
}
for _, m := range c.Mounts { for _, m := range c.Mounts {
if m.Type == "tmpfs" && m.Destination == "/tmp" { if m.Type == "tmpfs" && m.Destination == "/tmp" {
continue continue
+10 -2
View File
@@ -10,7 +10,7 @@ import (
) )
func TestOrphanReclaimGuards(t *testing.T) { func TestOrphanReclaimGuards(t *testing.T) {
for _, kind := range []string{"safe", "running", "foreign-mount", "reference", "upgrade", "confirmation", "nonadmin"} { for _, kind := range []string{"safe", "legacy", "shared", "running", "foreign-mount", "reference", "upgrade", "confirmation", "nonadmin"} {
t.Run(kind, func(t *testing.T) { t.Run(kind, func(t *testing.T) {
t.Setenv("HOST_DATA_ROOT", "/host/data") t.Setenv("HOST_DATA_ROOT", "/host/data")
id, inst, owner := strings.Repeat("a", 64), strings.Repeat("b", 32), strings.Repeat("c", 32) id, inst, owner := strings.Repeat("a", 64), strings.Repeat("b", 32), strings.Repeat("c", 32)
@@ -28,7 +28,15 @@ func TestOrphanReclaimGuards(t *testing.T) {
if kind == "foreign-mount" { if kind == "foreign-mount" {
source = "/unrelated" source = "/unrelated"
} }
writeJSON(w, map[string]any{"Id": id, "Name": "/dsh-inst-" + inst, "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "dsh", "dsh-mgr.instance": inst, "dsh-mgr.owner": owner}}, "State": map[string]any{"Status": "exited", "Running": kind == "running"}, "Mounts": []map[string]string{{"Type": "bind", "Source": source, "Destination": "/workspace"}}}) mounts := []map[string]string{{"Type": "bind", "Source": source, "Destination": "/workspace"}}
if kind == "legacy" {
mounts[0]["Destination"] = "/data/dsh-home"
mounts = append(mounts, map[string]string{"Type": "bind", "Source": "/host/data/workspaces/" + owner + "/files", "Destination": "/workspace"})
}
if kind == "shared" {
mounts = append(mounts, map[string]string{"Type": "bind", "Source": "/host/data/workspaces/" + owner + "/files", "Destination": "/shared"})
}
writeJSON(w, map[string]any{"Id": id, "Name": "/dsh-inst-" + inst, "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "dsh", "dsh-mgr.instance": inst, "dsh-mgr.owner": owner}}, "State": map[string]any{"Status": "exited", "Running": kind == "running"}, "Mounts": mounts})
}) })
if err := a.s.CreateUser(store.User{ID: owner, Username: "orphan-owner"}, "unused"); err != nil { if err := a.s.CreateUser(store.User{ID: owner, Username: "orphan-owner"}, "unused"); err != nil {
t.Fatal(err) t.Fatal(err)
+59
View File
@@ -0,0 +1,59 @@
package api
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
func TestRefreshLostResponseRecovery(t *testing.T) {
a, req := authFixture(t)
w := req("POST", "/api/login", `{"username":"alice","password":"password-123"}`, "", nil)
old := w.Result().Cookies()
csrf, sid := cookieValue(old, "dsh_csrf"), cookieValue(old, "dsh_session")
w = req("POST", "/api/refresh", "", csrf, old)
if w.Code != 200 {
t.Fatal(w.Code)
}
successor := cookieValue(w.Result().Cookies(), "dsh_refresh")
// Lose the response, expire access and restart the server: recovery is durable
// and does not depend on an active access cookie or a short grace timer.
for _, c := range old {
if c.Name == "dsh_access" {
c.Value = a.accessJWT(sid, "alice", time.Now().Add(-time.Hour))
}
}
b := New(a.s, a.root)
b.origin = a.origin
r := httptest.NewRequest("POST", "/api/refresh", strings.NewReader(""))
r.Header.Set("Origin", a.origin)
r.Header.Set("X-CSRF-Token", csrf)
for _, c := range old {
r.AddCookie(c)
}
w = httptest.NewRecorder()
b.Handler().ServeHTTP(w, r)
if w.Code != 200 || cookieValue(w.Result().Cookies(), "dsh_refresh") != successor {
t.Fatal("lost response recovery failed", w.Code)
}
current := w.Result().Cookies()
w = req("POST", "/api/refresh", "", csrf, current)
if w.Code != 200 {
t.Fatal(w.Code)
}
if w = req("POST", "/api/refresh", "", csrf, old); w.Code != 401 || len(w.Result().Cookies()) != 0 {
t.Fatal("two-generation-old token accepted", w.Code)
}
bad := []*http.Cookie{{Name: "dsh_session", Value: sid}, {Name: "dsh_refresh", Value: "invalid"}}
if w = req("POST", "/api/refresh", "", csrf, bad); w.Code != 401 {
t.Fatal(w.Code)
}
if _, err := a.s.DB.Exec("UPDATE sessions SET expires=? WHERE id=?", time.Now().Unix(), sid); err != nil {
t.Fatal(err)
}
if w = req("POST", "/api/refresh", "", csrf, current); w.Code == 200 || len(w.Result().Cookies()) != 0 {
t.Fatal("expired session recovered")
}
}
+198
View File
@@ -0,0 +1,198 @@
package api
import (
"context"
"database/sql"
"encoding/json"
"errors"
"git.misaka.ren/m1saka/dsh-web/internal/store"
"io"
"net/http"
"strconv"
"time"
)
var fetchRelease = readRelease
func (a *API) latestRelease(w http.ResponseWriter, r *http.Request) {
u, _, err := a.user(r)
if err != nil {
fail(w, 401, "authentication required")
return
}
if !u.Admin {
fail(w, 403, "administrator required")
return
}
if r.Method != http.MethodGet {
fail(w, 405, "method not allowed")
return
}
repository, branch, err := releaseSource()
if err != nil {
fail(w, 500, err.Error())
return
}
info, _, err := fetchRelease(r.Context(), repository, "refs/heads/"+branch)
if err != nil && !errors.Is(err, errReleaseIncompatible) {
fail(w, 502, sanitizeBuildLog(err.Error()))
return
}
compatible := err == nil
reason := "trusted release supports current layout and platform"
if !compatible {
reason = errReleaseIncompatible.Error()
}
buildQueue.Lock()
err = a.ensureBuilds()
buildQueue.Unlock()
if err != nil {
fail(w, 500, "database error")
return
}
var b buildRecord
err = scanBuildSummary(a.s.DB.QueryRow("SELECT "+buildSummaryColumns+" FROM image_builds WHERE source_repository=? AND source_commit=? AND release_id=? AND platform=? ORDER BY created_at DESC,id DESC LIMIT 1", repository, info.Commit, info.ReleaseID, imagePlatform), &b)
if err != nil && !errors.Is(err, sql.ErrNoRows) {
fail(w, 500, "database error")
return
}
result := map[string]any{"release_id": info.ReleaseID, "version": info.Version, "commit": info.Commit, "repository": repository, "branch": branch, "recipe_rev": strconv.Itoa(info.DeploymentSchema), "platforms": info.Platforms, "compatible": compatible, "reason": reason}
if err == nil {
result["build"] = b
}
writeJSON(w, result)
}
func (a *API) releaseBuild(w http.ResponseWriter, r *http.Request) {
u, _, err := a.user(r)
if err != nil {
fail(w, 401, "authentication required")
return
}
if !u.Admin {
fail(w, 403, "administrator required")
return
}
if r.Method != http.MethodPost {
fail(w, 405, "method not allowed")
return
}
var in struct {
Commit string `json:"commit"`
}
dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096))
dec.DisallowUnknownFields()
if dec.Decode(&in) != nil || dec.Decode(new(any)) != io.EOF || !releaseCommit.MatchString(in.Commit) {
fail(w, 400, "fixed commit required")
return
}
repository, branch, err := releaseSource()
if err != nil {
fail(w, 500, err.Error())
return
}
info, _, err := fetchRelease(r.Context(), repository, "refs/heads/"+branch)
if err != nil {
fail(w, 502, sanitizeBuildLog(err.Error()))
return
}
if info.Commit != in.Commit {
fail(w, 409, "release changed; check the latest release again")
return
}
b, err := a.enqueueRelease(u.ID, info)
if err != nil {
status := 500
if errors.Is(err, errBuildRateLimited) {
status = 429
}
fail(w, status, sanitizeBuildLog(err.Error()))
return
}
a.startBuildWorker()
w.WriteHeader(http.StatusAccepted)
writeJSON(w, b)
}
func (a *API) enqueueRelease(uid string, info releaseInfo) (buildRecord, error) {
buildQueue.Lock()
defer buildQueue.Unlock()
var b buildRecord
if err := a.ensureBuilds(); err != nil {
return b, err
}
err := scanBuildSummary(a.s.DB.QueryRow("SELECT "+buildSummaryColumns+" FROM image_builds WHERE source_repository=? AND source_commit=? AND release_id=? AND platform=? AND status IN ('queued','running','validating','success')", info.Repository, info.Commit, info.ReleaseID, imagePlatform), &b)
if err == nil {
if b.Status == "success" {
buildQueue.Unlock()
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
matches, verifyErr := a.buildImageMatches(ctx, b.ImageID, b)
cancel()
buildQueue.Lock()
if verifyErr != nil {
return b, verifyErr
}
if !matches {
if _, err = a.s.DB.Exec("UPDATE image_builds SET status='missing',error='image no longer available' WHERE id=? AND status='success'", b.ID); err != nil {
return b, err
}
buildQueue.Unlock()
b, err = a.enqueueRelease(uid, info)
buildQueue.Lock()
return b, err
}
}
return b, nil
}
if !errors.Is(err, sql.ErrNoRows) {
return b, err
}
var count int
if err = a.s.DB.QueryRow("SELECT count(*) FROM image_build_requests WHERE user_id=? AND created_at>?", uid, time.Now().Add(-time.Hour).UTC().Format(time.RFC3339Nano)).Scan(&count); err != nil {
return b, err
}
if count >= 3 {
return b, errBuildRateLimited
}
b = buildRecord{ID: store.ID(), Version: info.Version, Status: "queued", Created: time.Now().UTC().Format(time.RFC3339Nano), RequestedBy: uid, RecipeRev: imageRecipeRev, Platform: imagePlatform, ReleaseID: info.ReleaseID, SourceRepository: info.Repository, SourceCommit: info.Commit}
return b, a.insertBuildRequest(b, uid)
}
func (a *API) releaseBuildContext(ctx context.Context, b buildRecord) (io.Reader, error) {
repository, _, err := releaseSource()
if err != nil {
return nil, err
}
if repository != b.SourceRepository || !releaseCommit.MatchString(b.SourceCommit) {
return nil, errors.New("queued release source is no longer trusted")
}
u, err := a.s.UserByID(b.RequestedBy)
if err != nil || !u.Admin {
return nil, errors.New("release requester is not an administrator")
}
info, archive, err := fetchRelease(ctx, repository, b.SourceCommit)
if err != nil {
return nil, err
}
if info.Commit != b.SourceCommit || info.ReleaseID != b.ReleaseID || info.Version != b.Version {
return nil, errors.New("pinned release manifest mismatch")
}
return archive, nil
}
func releaseLabelsMatch(im *inspectedImage, b buildRecord) bool {
if b.ReleaseID == "" {
return im.Config.Labels["dsh-mgr.release"] == ""
}
return im.Config.Labels["dsh-mgr.release"] == b.ReleaseID && im.Config.Labels["dsh-mgr.source"] == b.SourceRepository && im.Config.Labels["dsh-mgr.commit"] == b.SourceCommit
}
func (a *API) buildImageMatches(ctx context.Context, id string, b buildRecord) (bool, error) {
matches, err := a.imageMatches(ctx, id, b.Version)
if err != nil || !matches {
return matches, err
}
im, err := a.inspectLocalImage(ctx, id)
if err != nil || im == nil {
return false, err
}
return releaseLabelsMatch(im, b), nil
}
+217
View File
@@ -0,0 +1,217 @@
package api
import (
"archive/tar"
"bytes"
"context"
"encoding/json"
"errors"
"io"
"net/url"
"os"
"os/exec"
"path"
"regexp"
"strings"
"syscall"
"time"
)
type releaseManifest struct {
SchemaVersion int `json:"schema_version"`
ReleaseID string `json:"release_id"`
Version string `json:"version"`
DeploymentSchema int `json:"deployment_schema"`
Platforms []string `json:"platforms"`
}
type releaseInfo struct {
releaseManifest
Repository string `json:"source_repository"`
Commit string `json:"source_commit"`
}
var errReleaseIncompatible = errors.New("release layout or platform is incompatible")
var releaseCommit = regexp.MustCompile(`^[0-9a-f]{40}$`)
var releaseID = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$`)
const releaseLimit = 32 << 20
func releaseSource() (string, string, error) {
repository := os.Getenv("DSH_RELEASE_REPOSITORY")
if repository == "" {
repository = "https://git.misaka.ren/m1saka/dsh-web.git"
}
branch := os.Getenv("DSH_RELEASE_BRANCH")
if branch == "" {
branch = "dockerfile"
}
u, err := url.Parse(repository)
if err != nil || u.Scheme != "https" || u.Hostname() == "" || u.User != nil || u.RawQuery != "" || u.Fragment != "" || strings.ContainsAny(repository, "\r\n") || !regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._/-]*$`).MatchString(branch) || strings.Contains(branch, "..") {
return "", "", errors.New("invalid trusted release source configuration")
}
return repository, branch, nil
}
type boundedReleaseBuffer struct{ bytes.Buffer }
func (b *boundedReleaseBuffer) Write(p []byte) (int, error) {
if b.Len()+len(p) > releaseLimit {
return 0, errors.New("release exceeds size limit")
}
return b.Buffer.Write(p)
}
func gitReleaseCommand(ctx context.Context, dir string, args ...string) ([]byte, error) {
gitArgs := append([]string{"-c", "credential.helper=", "-c", "http.followRedirects=false", "-c", "core.hooksPath=/dev/null", "-c", "protocol.allow=never", "-c", "protocol.https.allow=always", "-c", "fetch.unpackLimit=1"}, args...)
// The inherited file-size limit also bounds Git's on-disk pack, not just stdout.
cmd := exec.CommandContext(ctx, "/bin/sh", append([]string{"-c", "ulimit -f 32768; exec git \"$@\"", "git"}, gitArgs...)...)
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
cmd.Cancel = func() error { return syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL) }
cmd.Dir = dir
cmd.Env = []string{"PATH=" + os.Getenv("PATH"), "GIT_CONFIG_NOSYSTEM=1", "GIT_CONFIG_GLOBAL=/dev/null", "GIT_TERMINAL_PROMPT=0", "GIT_LFS_SKIP_SMUDGE=1"}
cmd.WaitDelay = time.Second
var out boundedReleaseBuffer
cmd.Stdout = &out
if err := cmd.Run(); err != nil {
return nil, errors.New("trusted release Git operation failed")
}
return out.Bytes(), nil
}
var releaseFetchSlot = make(chan struct{}, 1)
func readRelease(ctx context.Context, repository, ref string) (releaseInfo, io.Reader, error) {
var info releaseInfo
ctx, cancel := context.WithTimeout(ctx, 45*time.Second)
defer cancel()
select {
case releaseFetchSlot <- struct{}{}:
defer func() { <-releaseFetchSlot }()
case <-ctx.Done():
return info, nil, ctx.Err()
}
dir, err := os.MkdirTemp("", "dsh-release-")
if err != nil {
return info, nil, err
}
defer os.RemoveAll(dir)
if _, err = gitReleaseCommand(ctx, dir, "init", "--quiet"); err != nil {
return info, nil, err
}
if _, err = gitReleaseCommand(ctx, dir, "fetch", "--quiet", "--depth=1", "--no-tags", repository, ref); err != nil {
return info, nil, err
}
commit, err := gitReleaseCommand(ctx, dir, "rev-parse", "FETCH_HEAD^{commit}")
if err != nil {
return info, nil, err
}
info.Commit = strings.TrimSpace(string(commit))
info.Repository = repository
if !releaseCommit.MatchString(info.Commit) || (releaseCommit.MatchString(ref) && info.Commit != ref) {
return info, nil, errors.New("release commit mismatch")
}
archive, err := gitReleaseCommand(ctx, dir, "archive", "--format=tar", info.Commit, "--", "release.json", "image")
if err != nil {
return info, nil, err
}
return parseReleaseArchive(info, archive)
}
func parseReleaseArchive(info releaseInfo, archive []byte) (releaseInfo, io.Reader, error) {
var manifest []byte
tr := tar.NewReader(bytes.NewReader(archive))
for {
h, err := tr.Next()
if err == io.EOF {
break
}
if err != nil {
return info, nil, err
}
if h.Name == "release.json" && h.Typeflag == tar.TypeReg {
if h.Size > 16384 {
return info, nil, errors.New("release manifest too large")
}
manifest, err = io.ReadAll(tr)
if err != nil {
return info, nil, err
}
}
}
dec := json.NewDecoder(bytes.NewReader(manifest))
dec.DisallowUnknownFields()
if dec.Decode(&info.releaseManifest) != nil || dec.Decode(new(any)) != io.EOF {
return info, nil, errors.New("invalid release manifest")
}
m := info.releaseManifest
compatible := false
for _, p := range m.Platforms {
if p == imagePlatform {
compatible = true
}
}
if m.SchemaVersion != 1 || !releaseID.MatchString(m.ReleaseID) || !validBuildVersion(m.Version) || m.DeploymentSchema < 1 || len(m.Platforms) == 0 {
return info, nil, errors.New("unsupported release manifest or platform")
}
if m.DeploymentSchema != 7 || !compatible {
return info, nil, errReleaseIncompatible
}
var out bytes.Buffer
tw := tar.NewWriter(&out)
tr = tar.NewReader(bytes.NewReader(archive))
dockerfile := false
for {
h, err := tr.Next()
if err == io.EOF {
break
}
if err != nil {
return info, nil, err
}
name := h.Name
if !strings.HasPrefix(name, "image/") {
continue
}
name = strings.TrimPrefix(name, "image/")
if name == "" {
continue
}
if path.IsAbs(name) || path.Clean(strings.TrimSuffix(name, "/")) != strings.TrimSuffix(name, "/") || name == ".." || strings.HasPrefix(name, "../") {
return info, nil, errors.New("unsafe release context path")
}
excluded := false
for _, part := range strings.Split(name, "/") {
if part == ".git" || part == ".env" || strings.HasPrefix(part, ".env.") || part == ".netrc" || part == ".npmrc" || part == ".ssh" || part == ".git-credentials" {
excluded = true
}
}
if excluded {
continue
}
if h.Typeflag != tar.TypeReg && h.Typeflag != tar.TypeDir {
return info, nil, errors.New("release context links and special files are not supported")
}
h.Name = name
h.PAXRecords = nil
h.Uname = ""
h.Gname = ""
h.Uid = 0
h.Gid = 0
if name == "Dockerfile" && h.Typeflag == tar.TypeReg {
dockerfile = true
}
if err = tw.WriteHeader(h); err != nil {
return info, nil, err
}
if _, err = io.Copy(tw, tr); err != nil {
return info, nil, err
}
}
if !dockerfile {
return info, nil, errors.New("release context Dockerfile missing")
}
if err := tw.Close(); err != nil {
return info, nil, err
}
return info, bytes.NewReader(out.Bytes()), nil
}
+318
View File
@@ -0,0 +1,318 @@
package api
import (
"archive/tar"
"bytes"
"context"
"encoding/json"
"errors"
"io"
"net/http"
"strings"
"testing"
)
func testReleaseArchive(t *testing.T, m string, files map[string]string) []byte {
t.Helper()
var b bytes.Buffer
tw := tar.NewWriter(&b)
files["release.json"] = m
for name, data := range files {
if err := tw.WriteHeader(&tar.Header{Name: name, Size: int64(len(data)), Mode: 0644}); err != nil {
t.Fatal(err)
}
if _, err := tw.Write([]byte(data)); err != nil {
t.Fatal(err)
}
}
if err := tw.Close(); err != nil {
t.Fatal(err)
}
return b.Bytes()
}
func releaseFixture() releaseInfo {
return releaseInfo{releaseManifest: releaseManifest{SchemaVersion: 1, ReleaseID: "runtime-7.1", Version: "0.2.0-rc.3", DeploymentSchema: 7, Platforms: []string{imagePlatform}}, Repository: "https://git.misaka.ren/m1saka/dsh-web.git", Commit: strings.Repeat("a", 40)}
}
func TestReleaseManifestAndContext(t *testing.T) {
info := releaseFixture()
data, _ := json.Marshal(info.releaseManifest)
for _, tc := range []struct {
name, manifest string
files map[string]string
wantErr bool
}{
{"valid", string(data), map[string]string{"image/Dockerfile": "FROM scratch", "image/nested/helper": "complete", "image/.git/config": "secret", "image/.env": "secret"}, false},
{"malformed", "{", map[string]string{"image/Dockerfile": "FROM scratch"}, true},
{"unknown", strings.Replace(string(data), "schema_version", "unknown", 1), map[string]string{"image/Dockerfile": "FROM scratch"}, true},
{"incompatible", strings.Replace(string(data), "deployment_schema\":7", "deployment_schema\":8", 1), map[string]string{"image/Dockerfile": "FROM scratch"}, true},
{"missing Dockerfile", string(data), map[string]string{"image/helper": "x"}, true},
{"unsafe path", string(data), map[string]string{"image/Dockerfile": "FROM scratch", "image/../escape": "x"}, true},
} {
t.Run(tc.name, func(t *testing.T) {
_, r, err := parseReleaseArchive(info, testReleaseArchive(t, tc.manifest, tc.files))
if (err != nil) != tc.wantErr {
t.Fatalf("err=%v", err)
}
if err != nil {
return
}
tr := tar.NewReader(r)
seen := map[string]bool{}
for {
h, e := tr.Next()
if e == io.EOF {
break
}
if e != nil {
t.Fatal(e)
}
seen[h.Name] = true
}
if !seen["nested/helper"] || seen[".env"] || seen[".git/config"] {
t.Fatal(seen)
}
})
}
}
func TestReleaseQueueIdentityRecoveryAndFailure(t *testing.T) {
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(404) })
info := releaseFixture()
b, err := a.enqueueRelease("u", info)
if err != nil {
t.Fatal(err)
}
again, err := a.enqueueRelease("u", info)
if err != nil || again.ID != b.ID {
t.Fatalf("repeat %v %v", again, err)
}
info.Commit = strings.Repeat("b", 40)
info.ReleaseID = "runtime-7.2"
next, err := a.enqueueRelease("u", info)
if err != nil || next.ID == b.ID || next.Version != b.Version {
t.Fatalf("next %v %v", next, err)
}
buildQueue.Lock()
delete(buildQueue.initialized, a.s.DB)
err = a.ensureBuilds()
buildQueue.Unlock()
if err != nil {
t.Fatal(err)
}
var recovered buildRecord
if err = scanBuildSummary(a.s.DB.QueryRow("SELECT "+buildSummaryColumns+" FROM image_builds WHERE id=?", next.ID), &recovered); err != nil {
t.Fatal(err)
}
if recovered.Status != "queued" || recovered.SourceCommit != info.Commit {
t.Fatal(recovered)
}
if _, err = a.s.DB.Exec("UPDATE image_builds SET status='failed' WHERE id=?", next.ID); err != nil {
t.Fatal(err)
}
retry, err := a.enqueueRelease("u", info)
if err != nil || retry.ID == next.ID {
t.Fatalf("retry %v %v", retry, err)
}
}
func TestReleaseEndpointsAuthorizationFailureAndMismatch(t *testing.T) {
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(404) })
if w := imageRequest(a.latestRelease, sid, "GET", ""); w.Code != 403 {
t.Fatal(w.Code)
}
if _, err := a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='u'"); err != nil {
t.Fatal(err)
}
old := fetchRelease
t.Cleanup(func() { fetchRelease = old })
fetchRelease = func(context.Context, string, string) (releaseInfo, io.Reader, error) {
return releaseInfo{}, nil, errors.New("network failure")
}
if w := imageRequest(a.latestRelease, sid, "GET", ""); w.Code != 502 {
t.Fatalf("%d %s", w.Code, w.Body.String())
}
fetchRelease = func(context.Context, string, string) (releaseInfo, io.Reader, error) {
return releaseFixture(), nil, nil
}
if w := imageRequest(a.releaseBuild, sid, "POST", `{"commit":"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"}`); w.Code != 409 {
t.Fatalf("%d %s", w.Code, w.Body.String())
}
fetchRelease = func(context.Context, string, string) (releaseInfo, io.Reader, error) {
return releaseFixture(), nil, errReleaseIncompatible
}
if w := imageRequest(a.latestRelease, sid, "GET", ""); w.Code != 200 || !strings.Contains(w.Body.String(), `"compatible":false`) {
t.Fatalf("%d %s", w.Code, w.Body.String())
}
}
func TestReleaseGitEnvironmentAndFailure(t *testing.T) {
out, err := gitReleaseCommand(context.Background(), t.TempDir(), "--version")
if err != nil || !strings.HasPrefix(string(out), "git version") {
t.Fatalf("%q %v", out, err)
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
if _, err = gitReleaseCommand(ctx, t.TempDir(), "--version"); err == nil {
t.Fatal("cancel ignored")
}
for _, repository := range []string{"http://example.test/repo", "https://user:[email protected]/repo", "https://example.test/repo?token=secret"} {
t.Setenv("DSH_RELEASE_REPOSITORY", repository)
if _, _, err = releaseSource(); err == nil {
t.Fatal("accepted unsafe source")
}
}
}
func TestPinnedReleaseWorkerAdmissionAndBlock(t *testing.T) {
testPinnedReleaseWorkerAdmissionAndBlock(t, "")
}
func TestPinnedReleaseWorkerARM64V8AdmissionAndBlock(t *testing.T) {
previous := imagePlatform
imagePlatform = "linux/arm64"
t.Cleanup(func() { imagePlatform = previous })
testPinnedReleaseWorkerAdmissionAndBlock(t, "v8")
}
func testPinnedReleaseWorkerAdmissionAndBlock(t *testing.T, variant string) {
t.Helper()
info := releaseFixture()
labels := map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": info.Version, "dsh-mgr.recipe": "7", "dsh-mgr.release": info.ReleaseID, "dsh-mgr.source": info.Repository, "dsh-mgr.commit": info.Commit}
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v1.47/build":
var got map[string]string
if err := json.Unmarshal([]byte(r.URL.Query().Get("labels")), &got); err != nil {
t.Error(err)
}
if got["dsh-mgr.commit"] != info.Commit || r.URL.Query().Get("t") != "dsh-web:release-"+info.Commit {
t.Error("missing provenance", r.URL)
}
data, _ := io.ReadAll(r.Body)
if !bytes.Contains(data, []byte("pinned context")) {
t.Error("wrong build context")
}
io.WriteString(w, `{"aux":{"ID":"sha256:release"}}`)
case "/v1.47/images/sha256:release/json":
parts := strings.Split(imagePlatform, "/")
writeJSON(w, map[string]any{"Id": "sha256:release", "Os": parts[0], "Architecture": parts[1], "Variant": variant, "Config": map[string]any{"Labels": labels}})
case "/v1.47/containers/create":
var cfg struct {
Image, User string
HostConfig struct {
NetworkMode string
ReadonlyRootfs bool
Tmpfs map[string]string
}
}
if err := json.NewDecoder(r.Body).Decode(&cfg); err != nil || cfg.Image != "sha256:release" || cfg.User != "1000:1000" || cfg.HostConfig.NetworkMode != "none" || !cfg.HostConfig.ReadonlyRootfs || !strings.Contains(cfg.HostConfig.Tmpfs["/tmp"], "exec") {
t.Errorf("unsafe verification config %+v err=%v", cfg, err)
}
io.WriteString(w, `{"Id":"check"}`)
case "/v1.47/containers/check/start", "/v1.47/exec/probe/start", "/v1.47/containers/check":
w.WriteHeader(204)
case "/v1.47/containers/check/exec":
var cfg struct{ Cmd []string }
expected, _ := json.Marshal(info.Version)
if err := json.NewDecoder(r.Body).Decode(&cfg); err != nil || len(cfg.Cmd) != 3 || !strings.Contains(cfg.Cmd[2], "execFileSync('dsh',['--version']") || !strings.Contains(cfg.Cmd[2], "actual!=="+string(expected)) {
t.Errorf("missing exact version check %+v err=%v", cfg, err)
}
io.WriteString(w, `{"Id":"probe"}`)
case "/v1.47/exec/probe/json":
io.WriteString(w, `{"Running":false,"ExitCode":0}`)
case "/v1.47/containers/check/json":
io.WriteString(w, `{"State":{"Running":true}}`)
default:
t.Errorf("unexpected %s", r.URL)
w.WriteHeader(500)
}
})
if _, err := a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='u'"); err != nil {
t.Fatal(err)
}
old := fetchRelease
t.Cleanup(func() { fetchRelease = old })
fetchRelease = func(ctx context.Context, repository, ref string) (releaseInfo, io.Reader, error) {
if repository != info.Repository || ref != info.Commit {
t.Error("not pinned", repository, ref)
}
return info, strings.NewReader("pinned context"), nil
}
b, err := a.enqueueRelease("u", info)
if err != nil {
t.Fatal(err)
}
a.runImageBuild(b)
if err = scanBuildSummary(a.s.DB.QueryRow("SELECT "+buildSummaryColumns+" FROM image_builds WHERE id=?", b.ID), &b); err != nil {
t.Fatal(err)
}
if b.Status != "success" {
t.Fatalf("%+v", b)
}
records := []buildRecord{b}
if err = a.reconcileBuilds(context.Background(), records); err != nil || records[0].Status != "success" {
t.Fatalf("reconciliation: %+v %v", records, err)
}
im, err := a.evaluateImage(context.Background(), b.ImageID)
if err != nil || !im.Selectable {
t.Fatalf("%+v %v", im, err)
}
if _, err = a.s.DB.Exec("INSERT INTO image_admissions(image_id,version,recipe_rev,platform,state,evidence,admitted_by,admitted_at) VALUES(?,?,?,?,?,?,?,?)", b.ImageID, b.Version, "7", imagePlatform, "blocked", "explicit block", "u", "now"); err != nil {
t.Fatal(err)
}
im, err = a.evaluateImage(context.Background(), b.ImageID)
if err != nil || im.Selectable || im.Admission != "blocked" {
t.Fatalf("%+v %v", im, err)
}
}
func TestReleaseRateLimitDedup(t *testing.T) {
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("queued release contacted Docker") })
info := releaseFixture()
first, err := a.enqueueRelease("u", info)
if err != nil {
t.Fatal(err)
}
for _, commit := range []string{strings.Repeat("b", 40), strings.Repeat("c", 40)} {
next := info
next.Commit = commit
if _, err := a.enqueueRelease("u", next); err != nil {
t.Fatal(err)
}
}
again, err := a.enqueueRelease("u", info)
if err != nil || again.ID != first.ID {
t.Fatal(again, err)
}
info.Commit = strings.Repeat("d", 40)
if _, err := a.enqueueRelease("u", info); err != errBuildRateLimited {
t.Fatal(err)
}
var count int
if err := a.s.DB.QueryRow("SELECT count(*) FROM image_build_requests").Scan(&count); err != nil || count != 3 {
t.Fatal(count, err)
}
}
func TestReleaseUnknownDiskBlocksBuild(t *testing.T) {
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Errorf("unexpected Docker request %s", r.URL) })
t.Setenv("DSH_DOCKER_ROOT_VISIBLE", "")
if _, err := a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='u'"); err != nil {
t.Fatal(err)
}
info := releaseFixture()
old := fetchRelease
t.Cleanup(func() { fetchRelease = old })
fetchRelease = func(context.Context, string, string) (releaseInfo, io.Reader, error) {
return info, strings.NewReader("context"), nil
}
b, err := a.enqueueRelease("u", info)
if err != nil {
t.Fatal(err)
}
a.runImageBuild(b)
if err := scanBuildSummary(a.s.DB.QueryRow("SELECT "+buildSummaryColumns+" FROM image_builds WHERE id=?", b.ID), &b); err != nil {
t.Fatal(err)
}
if b.Status != "failed" || !strings.Contains(b.Error, "capacity unknown") {
t.Fatalf("%+v", b)
}
}
+5 -1
View File
@@ -277,7 +277,11 @@ func (a *API) persistBuildResult(b buildRecord, status, logs, detail, image, fin
} }
defer tx.Rollback() defer tx.Rollback()
if status == "success" && image != "" { if status == "success" && image != "" {
_, err = tx.Exec("INSERT INTO images(image_id,version,recipe_rev,platform,tag,status,built_at) VALUES(?,?,?,?,?,?,?) ON CONFLICT(image_id) DO UPDATE SET version=excluded.version,recipe_rev=excluded.recipe_rev,platform=excluded.platform,tag=excluded.tag,status=excluded.status,built_at=excluded.built_at", image, b.Version, imageRecipeRev, imagePlatform, "dsh-web:"+b.Version+"-"+imageRecipeRev, "unverified", finished) tag := "dsh-web:" + b.Version + "-" + imageRecipeRev
if b.ReleaseID != "" {
tag = "dsh-web:release-" + b.SourceCommit
}
_, err = tx.Exec("INSERT INTO images(image_id,version,recipe_rev,platform,tag,status,built_at) VALUES(?,?,?,?,?,?,?) ON CONFLICT(image_id) DO UPDATE SET version=excluded.version,recipe_rev=excluded.recipe_rev,platform=excluded.platform,tag=excluded.tag,status=excluded.status,built_at=excluded.built_at", image, b.Version, imageRecipeRev, imagePlatform, tag, "unverified", finished)
if err != nil { if err != nil {
return err return err
} }
+1 -64
View File
@@ -84,43 +84,6 @@ func TestRuntimeNetworkDeletionLostResponseAndReplacement(t *testing.T) {
}) })
} }
} }
func TestStartIntentsUncertainCreationKeepsRecoveryAccessible(t *testing.T) {
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/image/json" {
writeImageInspect(w, "image", "1.2.3")
return
}
if r.Method != "GET" {
t.Error("unexpected destructive call")
}
w.WriteHeader(404)
})
if err := a.s.EnsureIntents(); err != nil {
t.Fatal(err)
}
if err := a.s.ReserveIntent(store.Intent{ID: "intent", UID: "owner"}); err != nil {
t.Fatal(err)
}
if err := a.s.ClaimIntent("intent", store.Instance{ID: "pending", UID: "owner", Status: "creating"}); err != nil {
t.Fatal(err)
}
if err := a.setRuntimeCreatePending("pending", "container_pending", true); err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
if err := a.StartIntents(ctx); err != nil {
t.Fatalf("uncertain creation blocked startup: %v", err)
}
i, err := a.s.GetInstance("pending")
if err != nil || i.Status != "error" {
t.Fatalf("recovery reservation missing %+v %v", i, err)
}
list, err := a.s.Intents("owner")
if err != nil || len(list) != 1 || list[0].Status != "failed" {
t.Fatalf("intent not terminated %+v %v", list, err)
}
}
func TestNetworkCreateLostResponse(t *testing.T) { func TestNetworkCreateLostResponse(t *testing.T) {
for _, mode := range []string{"present", "late", "foreign"} { for _, mode := range []string{"present", "late", "foreign"} {
@@ -258,7 +221,7 @@ func TestBuildTerminalFailureDoesNotBlockNextTask(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
for _, id := range []string{"a", "b"} { for _, id := range []string{"a", "b"} {
if _, err = a.s.DB.Exec("INSERT INTO image_builds(id,version,status,created_at) VALUES(?,'1.2.3','queued',?)", id, id); err != nil { if _, err = a.s.DB.Exec("INSERT INTO image_builds(id,version,status,created_at,release_id) VALUES(?,'1.2.3','queued',?,'release')", id, id); err != nil {
t.Fatal(err) t.Fatal(err)
} }
} }
@@ -535,32 +498,6 @@ func TestBuildResultTransactionAndRetry(t *testing.T) {
} }
} }
func TestIntentRecoveryMissingReservationDoesNotTouchDocker(t *testing.T) {
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/image/json" {
writeImageInspect(w, "image", "1.2.3")
return
}
t.Error("missing reservation must not authorize Docker cleanup")
})
if err := a.s.EnsureIntents(); err != nil {
t.Fatal(err)
}
if err := a.s.ReserveIntent(store.Intent{ID: "missing", UID: "owner"}); err != nil {
t.Fatal(err)
}
if _, err := a.s.DB.Exec("UPDATE creation_intents SET status='creating',instance_id='gone' WHERE id='missing'"); err != nil {
t.Fatal(err)
}
if err := a.recoverIntents(context.Background()); err != nil {
t.Fatal(err)
}
list, err := a.s.Intents("owner")
if err != nil || len(list) != 1 || list[0].Status != "failed" {
t.Fatalf("intent not terminated: %+v %v", list, err)
}
}
func TestBuildTerminalRetryStopsWhenDatabaseCloses(t *testing.T) { func TestBuildTerminalRetryStopsWhenDatabaseCloses(t *testing.T) {
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("unexpected Docker call") }) a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("unexpected Docker call") })
a.s.Close() a.s.Close()
+2 -1
View File
@@ -8,6 +8,7 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
"syscall"
"testing" "testing"
) )
@@ -30,7 +31,7 @@ func TestRestoreSnapshotCopyFailureDoesNotConsumeRetention(t *testing.T) {
} }
configWrite(t, filepath.Join(base, name, "keep"), name) configWrite(t, filepath.Join(base, name, "keep"), name)
} }
if err := os.Symlink("missing-target", filepath.Join(home, "dangling")); err != nil { if err := syscall.Mkfifo(filepath.Join(home, "dangling"), 0600); err != nil {
t.Fatal(err) t.Fatal(err)
} }
body := fmt.Sprintf(`{"backup_id":%q,"password":"original-password"}`, result.ID) body := fmt.Sprintf(`{"backup_id":%q,"password":"original-password"}`, result.ID)
+34 -16
View File
@@ -110,10 +110,6 @@ func (a *API) instances(w http.ResponseWriter, r *http.Request) {
return return
} }
i := store.Instance{ID: store.ID(), UID: u.ID, Name: strings.TrimSpace(in.Name), ImageID: image.ID, Version: image.Config.Labels["dsh-mgr.version"], Status: "creating", Created: time.Now()} i := store.Instance{ID: store.ID(), UID: u.ID, Name: strings.TrimSpace(in.Name), ImageID: image.ID, Version: image.Config.Labels["dsh-mgr.version"], Status: "creating", Created: time.Now()}
if err := a.s.EnsureIntents(); err != nil {
fail(w, 500, "database error")
return
}
if err := a.s.ReserveInstance(i); err != nil { if err := a.s.ReserveInstance(i); err != nil {
if errors.Is(err, store.ErrInstanceLimit) { if errors.Is(err, store.ErrInstanceLimit) {
fail(w, 409, err.Error()) fail(w, 409, err.Error())
@@ -171,6 +167,10 @@ func (a *API) createRuntime(ctx context.Context, u store.User, i store.Instance,
return return
} }
} }
if err := a.removeInstanceData(i); err != nil {
a.s.SetInstance(i.ID, "error")
return
}
a.s.DeleteInstance(i.ID) a.s.DeleteInstance(i.ID)
_, _ = a.s.DB.Exec("DELETE FROM runtime_networks WHERE instance_id=?", i.ID) _, _ = a.s.DB.Exec("DELETE FROM runtime_networks WHERE instance_id=?", i.ID)
_, _ = a.s.DB.Exec("DELETE FROM runtime_creates WHERE instance_id=?", i.ID) _, _ = a.s.DB.Exec("DELETE FROM runtime_creates WHERE instance_id=?", i.ID)
@@ -195,7 +195,7 @@ func (a *API) createRuntime(ctx context.Context, u store.User, i store.Instance,
return i, errors.New("storage unavailable") return i, errors.New("storage unavailable")
} }
suffix := filepath.Join("workspaces", u.ID) suffix := filepath.Join("workspaces", u.ID)
for _, relative := range []string{filepath.Join(suffix, "files"), filepath.Join(suffix, "instances", i.ID, "home")} { for _, relative := range []string{filepath.Join(suffix, "instances", i.ID, "home")} {
dir := filepath.Join(root, relative) dir := filepath.Join(root, relative)
if err := os.MkdirAll(dir, 0750); err != nil { if err := os.MkdirAll(dir, 0750); err != nil {
return i, errors.New("storage unavailable") return i, errors.New("storage unavailable")
@@ -262,14 +262,6 @@ func (a *API) createRuntime(ctx context.Context, u store.User, i store.Instance,
}, },
}, },
} }
if policy.RecipeRev != "7" {
sharedSource, err := toHostPath(root, hostRoot, filepath.Join(root, suffix, "files"))
if err != nil {
return i, err
}
body["Env"] = []string{"DSH_HOME=/data/dsh-home", "HOME=/data/dsh-home", "XDG_DATA_HOME=/data/dsh-home/.local/share", "XDG_CONFIG_HOME=/data/dsh-home/.config"}
body["HostConfig"].(map[string]any)["Mounts"] = []map[string]any{{"Type": "bind", "Source": homeSource, "Target": "/data/dsh-home"}, {"Type": "bind", "Source": sharedSource, "Target": "/workspace"}}
}
var created struct { var created struct {
ID string `json:"Id"` ID string `json:"Id"`
} }
@@ -348,7 +340,7 @@ func (a *API) instance(w http.ResponseWriter, r *http.Request) {
case "profile-preview": case "profile-preview":
a.profilePreview(w, r, i) a.profilePreview(w, r, i)
return return
case "profile", "backups", "restore": case "profile", "backups", "restore", "snapshots":
a.config(w, r, i, action) a.config(w, r, i, action)
return return
case "upgrade": case "upgrade":
@@ -419,12 +411,17 @@ func (a *API) instance(w http.ResponseWriter, r *http.Request) {
fail(w, 502, "network deletion failed") fail(w, 502, "network deletion failed")
return return
} }
if err := a.removeInstanceData(i); err != nil {
a.s.SetInstance(i.ID, "error")
fail(w, 500, "instance data deletion failed; retry deletion")
return
}
if _, err := a.s.DB.ExecContext(ctx, "DELETE FROM instances WHERE id=?", i.ID); err != nil { if _, err := a.s.DB.ExecContext(ctx, "DELETE FROM instances WHERE id=?", i.ID); err != nil {
fail(w, 500, "database error") fail(w, 500, "database error")
return return
} }
_, _ = a.s.DB.Exec("DELETE FROM runtime_networks WHERE instance_id=?", i.ID) _, _ = a.s.DB.Exec("DELETE FROM runtime_networks WHERE instance_id=?", i.ID)
a.s.AddAudit(u.ID, "instance.delete", i.ID, "data retained") a.s.AddAudit(u.ID, "instance.delete", i.ID, "instance data deleted")
// The instance row is gone and its resources were just cleaned, so // The instance row is gone and its resources were just cleaned, so
// any pending flags cannot resolve later: drop the row unconditionally. // any pending flags cannot resolve later: drop the row unconditionally.
_, _ = a.s.DB.Exec("DELETE FROM runtime_creates WHERE instance_id=?", i.ID) _, _ = a.s.DB.Exec("DELETE FROM runtime_creates WHERE instance_id=?", i.ID)
@@ -538,6 +535,27 @@ func (a *API) runtimeOriginAllowed(r *http.Request) bool {
var runtimeTransport = &http.Transport{Proxy: nil, DialContext: (&net.Dialer{Timeout: 10 * time.Second, KeepAlive: 30 * time.Second}).DialContext, ResponseHeaderTimeout: 20 * time.Second, IdleConnTimeout: 90 * time.Second} var runtimeTransport = &http.Transport{Proxy: nil, DialContext: (&net.Dialer{Timeout: 10 * time.Second, KeepAlive: 30 * time.Second}).DialContext, ResponseHeaderTimeout: 20 * time.Second, IdleConnTimeout: 90 * time.Second}
// Office rendering can take longer than the normal API header deadline (the
// bundled converter defaults to 60 seconds). Reuse a separate connection pool
// without extending authentication or unrelated runtime requests.
var runtimeOfficeTransport = func() *http.Transport {
t := runtimeTransport.Clone()
t.ResponseHeaderTimeout = 120 * time.Second
return t
}()
type runtimeProxyTransport struct {
normal, office http.RoundTripper
}
func (t runtimeProxyTransport) RoundTrip(r *http.Request) (*http.Response, error) {
// ReverseProxy has already removed the instance prefix here.
if r.URL.Path == "/api/officeToPdf/render" {
return t.office.RoundTrip(r)
}
return t.normal.RoundTrip(r)
}
type runtimeAuthKey struct { type runtimeAuthKey struct {
api *API api *API
id string id string
@@ -1019,7 +1037,7 @@ func withoutRuntimeToken(rawQuery string) string {
func runtimeReverseProxy(target *url.URL, prefix, host, cookies string) *httputil.ReverseProxy { func runtimeReverseProxy(target *url.URL, prefix, host, cookies string) *httputil.ReverseProxy {
return &httputil.ReverseProxy{ return &httputil.ReverseProxy{
Transport: runtimeTransport, FlushInterval: -1, ErrorLog: log.New(io.Discard, "", 0), Transport: runtimeProxyTransport{normal: runtimeTransport, office: runtimeOfficeTransport}, FlushInterval: -1, ErrorLog: log.New(io.Discard, "", 0),
Rewrite: func(p *httputil.ProxyRequest) { Rewrite: func(p *httputil.ProxyRequest) {
p.SetURL(target) p.SetURL(target)
p.Out.Host = host p.Out.Host = host
+167
View File
@@ -0,0 +1,167 @@
package api
import (
"context"
"io"
"net/http"
"net/http/httptest"
"net/http/httputil"
"net/url"
"strings"
"sync/atomic"
"testing"
"time"
)
func officeTestProxy(t *testing.T, upstream string, officeTimeout time.Duration) *httputil.ReverseProxy {
t.Helper()
target, err := url.Parse(upstream)
if err != nil {
t.Fatal(err)
}
normal := runtimeTransport.Clone()
office := runtimeOfficeTransport.Clone()
normal.ResponseHeaderTimeout = 30 * time.Millisecond
office.ResponseHeaderTimeout = officeTimeout
t.Cleanup(normal.CloseIdleConnections)
t.Cleanup(office.CloseIdleConnections)
proxy := runtimeReverseProxy(target, "/dsh/office-test", "panel.test", "runtime=secret")
proxy.Transport = runtimeProxyTransport{normal: normal, office: office}
return proxy
}
func TestRuntimeOfficeTransportDefaults(t *testing.T) {
if got := runtimeTransport.ResponseHeaderTimeout; got != 20*time.Second {
t.Errorf("normal timeout = %s, want 20s", got)
}
if got := runtimeOfficeTransport.ResponseHeaderTimeout; got != 120*time.Second {
t.Errorf("office timeout = %s, want 120s", got)
}
if runtimeTransport == runtimeOfficeTransport {
t.Fatal("office transport must be separate")
}
target, _ := url.Parse("http://panel.test")
proxy := runtimeReverseProxy(target, "/dsh/office-test", "panel.test", "")
transport, ok := proxy.Transport.(runtimeProxyTransport)
if !ok || transport.normal != runtimeTransport || transport.office != runtimeOfficeTransport {
t.Fatalf("unexpected default transport: %#v", proxy.Transport)
}
}
func TestRuntimeOfficeProxyRouting(t *testing.T) {
for _, path := range []string{
"/api/officeToPdf/render",
"/api/status",
"/api/officeToPdf/render/",
"/api/officeToPdf/render/child",
"/api/officeToPdf/renderOther",
"/api/officetopdf/render",
"/api/officeToPdf/status",
} {
t.Run(path, func(t *testing.T) {
const body = "document payload"
const query = "file=report%20one.docx&mode=pdf&mode=preview"
var calls atomic.Int32
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls.Add(1)
data, err := io.ReadAll(r.Body)
if err != nil || string(data) != body || r.Method != http.MethodPost {
t.Errorf("upstream method/body = %s %q, error %v", r.Method, data, err)
}
if r.URL.Path != path || r.URL.RawQuery != query {
t.Errorf("upstream URL = %s", r.URL)
}
if r.Host != "panel.test" || r.Header.Get("X-Forwarded-Prefix") != "/dsh/office-test" || r.Header.Get("Cookie") != "runtime=secret" {
t.Errorf("upstream host/headers = %s %v", r.Host, r.Header)
}
time.Sleep(100 * time.Millisecond)
w.Header().Set("Content-Type", "application/pdf")
_, _ = io.WriteString(w, "rendered PDF")
}))
defer upstream.Close()
proxy := officeTestProxy(t, upstream.URL, 500*time.Millisecond)
req := httptest.NewRequest(http.MethodPost, "/dsh/office-test"+path+"?"+query+"&token=removed", strings.NewReader(body))
req.Header.Set("Cookie", "panel=private")
originalURL := req.URL.String()
w := httptest.NewRecorder()
proxy.ServeHTTP(w, req)
want := http.StatusBadGateway
if path == "/api/officeToPdf/render" {
want = http.StatusOK
if w.Body.String() != "rendered PDF" {
t.Errorf("response body = %q", w.Body.String())
}
}
if w.Code != want {
t.Errorf("status = %d, want %d: %s", w.Code, want, w.Body.String())
}
if req.URL.String() != originalURL || req.Header.Get("Cookie") != "panel=private" {
t.Error("proxy mutated incoming URL or cookies")
}
upstream.Close()
if got := calls.Load(); got != 1 {
t.Errorf("mutating request sent %d times, want exactly once", got)
}
})
}
}
func TestRuntimeOfficeProxyFiniteTimeoutAndCancellation(t *testing.T) {
for _, name := range []string{"header timeout", "caller cancellation"} {
t.Run(name, func(t *testing.T) {
started := make(chan struct{}, 2)
canceled := make(chan struct{}, 2)
var calls atomic.Int32
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls.Add(1)
_, _ = io.Copy(io.Discard, r.Body)
started <- struct{}{}
select {
case <-r.Context().Done():
canceled <- struct{}{}
case <-time.After(2 * time.Second):
}
}))
defer upstream.Close()
officeTimeout := 100 * time.Millisecond
if name == "caller cancellation" {
officeTimeout = 5 * time.Second
}
proxy := officeTestProxy(t, upstream.URL, officeTimeout)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
req := httptest.NewRequest(http.MethodPost, "/dsh/office-test/api/officeToPdf/render?mode=pdf", strings.NewReader("document")).WithContext(ctx)
w := httptest.NewRecorder()
done := make(chan struct{})
go func() {
proxy.ServeHTTP(w, req)
close(done)
}()
select {
case <-started:
case <-time.After(time.Second):
t.Fatal("request never reached upstream")
}
if name == "caller cancellation" {
cancel()
}
select {
case <-done:
case <-time.After(500 * time.Millisecond):
t.Fatal("proxy did not promptly finish")
}
if w.Code != http.StatusBadGateway {
t.Errorf("status = %d, want 502", w.Code)
}
select {
case <-canceled:
case <-time.After(500 * time.Millisecond):
t.Fatal("upstream context was not canceled")
}
upstream.Close()
if got := calls.Load(); got != 1 {
t.Errorf("mutating request sent %d times, want exactly once", got)
}
})
}
}
+2 -2
View File
@@ -278,8 +278,8 @@ func TestRuntimeLifecycleAndOwnership(t *testing.T) {
if w.Code != 200 { if w.Code != 200 {
t.Fatal(w.Code, w.Body.String()) t.Fatal(w.Code, w.Body.String())
} }
if _, err := os.Stat(dir); err != nil { if _, err := os.Stat(dir); !os.IsNotExist(err) {
t.Fatal("data removed", err) t.Fatal("instance data retained", err)
} }
if _, err := a.s.GetInstance(i.ID); err == nil { if _, err := a.s.GetInstance(i.ID); err == nil {
t.Fatal("row retained") t.Fatal("row retained")
+230
View File
@@ -0,0 +1,230 @@
package api
import (
"archive/tar"
"compress/gzip"
"context"
"errors"
"io"
"mime"
"net/http"
"os"
"path/filepath"
"syscall"
"time"
"git.misaka.ren/m1saka/dsh-web/internal/store"
)
var errSnapshotDownloadLimit = errors.New("snapshot archive exceeds size, entry or depth limit")
var errUnsafeSnapshotArchive = errors.New("unsafe snapshot link or unsupported file type")
type snapshotArchiveWriter struct {
ctx context.Context
w io.Writer
remaining int64
}
func (w *snapshotArchiveWriter) Write(p []byte) (int, error) {
if err := w.ctx.Err(); err != nil {
return 0, err
}
if int64(len(p)) > w.remaining {
return 0, errSnapshotDownloadLimit
}
n, err := w.w.Write(p)
w.remaining -= int64(n)
return n, err
}
func snapshotArchive(ctx context.Context, root *os.Root, out io.Writer, limit int64) error {
// Bound headers, padding and gzip expansion independently of file content.
gz := gzip.NewWriter(&snapshotArchiveWriter{ctx, out, limit + (256 << 20)})
tw := tar.NewWriter(&snapshotArchiveWriter{ctx, gz, limit + (128 << 20)})
remaining := limit
entries := 100000
var walk func(*os.Root, string, string, int) error
walk = func(parent *os.Root, name, rel string, depth int) error {
if err := ctx.Err(); err != nil {
return err
}
entries--
if entries < 0 || depth > 128 {
return errSnapshotDownloadLimit
}
st, err := parent.Lstat(name)
if err != nil {
return err
}
link := ""
if st.Mode()&os.ModeSymlink != 0 {
link, err = parent.Readlink(name)
if err != nil {
return err
}
// Archive link text only; do not read or traverse its target.
} else if !st.IsDir() && !st.Mode().IsRegular() {
return errUnsafeSnapshotArchive
}
if st.Mode().IsRegular() && st.Size() > remaining {
return errSnapshotDownloadLimit
}
if st.Mode().IsRegular() {
remaining -= st.Size()
}
h, err := tar.FileInfoHeader(st, link)
if err != nil {
return err
}
h.Name = filepath.ToSlash(rel)
if st.IsDir() {
h.Name += "/"
}
if err = tw.WriteHeader(h); err != nil {
return err
}
if st.Mode()&os.ModeSymlink != 0 {
return nil
}
if st.IsDir() {
child, err := parent.OpenRoot(name)
if err != nil {
return err
}
defer child.Close()
actual, err := child.Stat(".")
if err != nil || !os.SameFile(st, actual) {
return errors.New("snapshot directory changed")
}
f, err := child.Open(".")
if err != nil {
return err
}
defer f.Close()
for {
children, err := f.ReadDir(128)
if err != nil && err != io.EOF {
return err
}
for _, entry := range children {
if err := walk(child, entry.Name(), filepath.Join(rel, entry.Name()), depth+1); err != nil {
return err
}
}
if err == io.EOF {
return nil
}
}
}
f, err := parent.OpenFile(name, os.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK, 0)
if err != nil {
return err
}
defer f.Close()
actual, err := f.Stat()
if err != nil || !actual.Mode().IsRegular() || !os.SameFile(st, actual) || actual.Size() != st.Size() {
return errors.New("snapshot file changed")
}
if _, err = io.CopyN(tw, f, st.Size()); err != nil {
return err
}
after, err := f.Stat()
if err != nil || after.Size() != st.Size() || !after.ModTime().Equal(st.ModTime()) {
return errors.New("snapshot file changed")
}
return nil
}
if err := walk(root, ".", ".", 0); err != nil {
return err
}
if err := tw.Close(); err != nil {
return err
}
return gz.Close()
}
func (a *API) snapshotDownload(w http.ResponseWriter, r *http.Request, i store.Instance) {
if r.Method != http.MethodGet {
fail(w, 405, "method not allowed")
return
}
ids := r.URL.Query()["download"]
if len(ids) != 1 || !snapshotIDPattern.MatchString(ids[0]) {
fail(w, 400, "invalid snapshot ID")
return
}
id := ids[0]
select {
case zipSlots <- struct{}{}:
defer func() { <-zipSlots }()
default:
fail(w, 429, "too many archive downloads")
return
}
ctx, cancel := context.WithTimeout(r.Context(), zipTimeout)
defer cancel()
archive, err := os.CreateTemp("", "dsh-snapshot-*.tar.gz")
if err != nil {
fail(w, 500, "temporary archive unavailable")
return
}
defer os.Remove(archive.Name())
defer archive.Close()
code := 500
err = func() error {
lock := a.instanceLocks.Acquire(i.ID)
defer lock.Release()
if err := ctx.Err(); err != nil {
return err
}
current, err := a.s.GetInstance(i.ID)
if err != nil || current.UID != i.UID {
code = 404
return errors.New("not found")
}
root, err := managementRoot(a.root, "workspaces", current.UID, "instances", current.ID, id)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
code = 404
}
if errors.Is(err, errUnsafeManagementPath) {
code = 400
}
return errors.New("snapshot unavailable or unsafe path")
}
defer root.Close()
return snapshotArchive(ctx, root, archive, upgradeSnapshotLimit)
}()
if err != nil {
if errors.Is(err, errUnsafeSnapshotArchive) {
code = 400
}
if errors.Is(err, errSnapshotDownloadLimit) {
code = 413
}
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
code = 408
}
fail(w, code, err.Error())
return
}
if ctx.Err() != nil {
fail(w, 408, "snapshot download timed out or canceled")
return
}
if _, err = archive.Seek(0, io.SeekStart); err != nil {
fail(w, 500, "archive unavailable")
return
}
controller := http.NewResponseController(w)
if err = controller.SetWriteDeadline(time.Now().Add(zipTimeout)); err != nil && !errors.Is(err, http.ErrNotSupported) {
fail(w, 500, "download unavailable")
return
}
defer controller.SetWriteDeadline(time.Time{})
a.s.AddAudit(i.UID, "snapshot.download", i.ID, id)
w.Header().Set("Content-Type", "application/gzip")
w.Header().Set("Cache-Control", "no-store")
w.Header().Set("Content-Disposition", mime.FormatMediaType("attachment", map[string]string{"filename": id + ".tar.gz"}))
http.ServeContent(w, r, id+".tar.gz", time.Time{}, archive)
}
+297
View File
@@ -0,0 +1,297 @@
package api
import (
"archive/tar"
"bytes"
"compress/gzip"
"context"
"errors"
"io"
"mime"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strings"
"syscall"
"testing"
)
type snapshotTransferWriter struct {
*httptest.ResponseRecorder
check func()
}
func (w *snapshotTransferWriter) WriteHeader(code int) {
w.check()
w.ResponseRecorder.WriteHeader(code)
}
func TestSnapshotDownloadTransferUnlockedAndOversize(t *testing.T) {
a, i, home := configTestAPI(t, false)
tmp := t.TempDir()
t.Setenv("TMPDIR", tmp)
id := "failed-home-" + strings.Repeat("c", 32)
dir := filepath.Join(filepath.Dir(home), id)
if err := os.Mkdir(dir, 0700); err != nil {
t.Fatal(err)
}
configWrite(t, filepath.Join(dir, "data"), "saved")
checked := false
w := &snapshotTransferWriter{httptest.NewRecorder(), func() {
checked = true
if a.instanceLocks.held() {
t.Fatal("instance locked during transfer")
}
files, err := os.ReadDir(tmp)
if err != nil || len(files) != 1 {
t.Fatal(files, err)
}
st, err := files[0].Info()
if err != nil || st.Mode().Perm() != 0600 {
t.Fatal(st, err)
}
if err := os.RemoveAll(dir); err != nil {
t.Fatal(err)
}
}}
a.snapshotDownload(w, httptest.NewRequest(http.MethodGet, "/?download="+id, nil), i)
if !checked || w.Code != 200 {
t.Fatal(checked, w.Code)
}
gz, err := gzip.NewReader(bytes.NewReader(w.Body.Bytes()))
if err != nil {
t.Fatal(err)
}
defer gz.Close()
if _, err := io.Copy(io.Discard, gz); err != nil {
t.Fatal(err)
}
if err := os.Mkdir(dir, 0700); err != nil {
t.Fatal(err)
}
f, err := os.Create(filepath.Join(dir, "huge"))
if err != nil {
t.Fatal(err)
}
err = f.Truncate(upgradeSnapshotLimit + 1)
f.Close()
if err != nil {
t.Fatal(err)
}
rec := httptest.NewRecorder()
a.snapshotDownload(rec, httptest.NewRequest("GET", "/?download="+id, nil), i)
if rec.Code != 413 || !strings.Contains(rec.Body.String(), "limit") {
t.Fatal(rec.Code, rec.Body.String())
}
files, err := os.ReadDir(tmp)
if err != nil || len(files) != 0 {
t.Fatal(files, err)
}
}
func TestSnapshotDownloadContentsAndCleanup(t *testing.T) {
a, i, home := configTestAPI(t, true)
tmp := t.TempDir()
t.Setenv("TMPDIR", tmp)
id := "pre-restore-" + strings.Repeat("a", 32)
dir := filepath.Join(filepath.Dir(home), id)
if err := os.MkdirAll(filepath.Join(dir, "node_modules/.pnpm/pkg"), 0750); err != nil {
t.Fatal(err)
}
configWrite(t, filepath.Join(dir, "node_modules/.pnpm/pkg/bin"), "#!/bin/sh")
if err := os.Chmod(filepath.Join(dir, "node_modules/.pnpm/pkg/bin"), 0751); err != nil {
t.Fatal(err)
}
if err := os.Symlink(".pnpm/pkg", filepath.Join(dir, "node_modules/pkg")); err != nil {
t.Fatal(err)
}
configWrite(t, filepath.Join(dir, ".credentials.yaml"), "secret")
if _, err := a.s.DB.Exec("UPDATE instances SET status='error' WHERE id=?", i.ID); err != nil {
t.Fatal(err)
}
w := httptest.NewRecorder()
a.config(w, httptest.NewRequest("GET", "/?download="+id, nil), i, "snapshots")
if w.Code != 200 {
t.Fatal(w.Code, w.Body.String())
}
_, params, err := mime.ParseMediaType(w.Header().Get("Content-Disposition"))
if err != nil || params["filename"] != id+".tar.gz" {
t.Fatal(params, err)
}
gz, err := gzip.NewReader(bytes.NewReader(w.Body.Bytes()))
if err != nil {
t.Fatal(err)
}
defer gz.Close()
tr := tar.NewReader(gz)
seen := map[string]bool{}
for {
h, err := tr.Next()
if err == io.EOF {
break
}
if err != nil {
t.Fatal(err)
}
seen[h.Name] = true
b, err := io.ReadAll(tr)
if err != nil {
t.Fatal(err)
}
switch h.Name {
case "node_modules/.pnpm/pkg/bin":
if string(b) != "#!/bin/sh" || h.Mode != 0751 {
t.Fatal(h, string(b))
}
case "node_modules/pkg":
if h.Typeflag != tar.TypeSymlink || h.Linkname != ".pnpm/pkg" {
t.Fatal(h)
}
case ".credentials.yaml":
if string(b) != "secret" {
t.Fatal(string(b))
}
}
}
if len(seen) != 7 || !seen["node_modules/pkg"] || seen["node_modules/pkg/bin"] {
t.Fatal(seen)
}
files, err := os.ReadDir(tmp)
if err != nil || len(files) != 0 {
t.Fatal(files, err)
}
r := httptest.NewRequest("GET", "/?download="+id, nil)
r.Header.Set("Range", "bytes=0-9")
w = httptest.NewRecorder()
a.snapshotDownload(w, r, i)
if w.Code != 206 || w.Body.Len() != 10 {
t.Fatal(w.Code, w.Body.Len())
}
}
func TestSnapshotDownloadRejectsUnsafeAndForeign(t *testing.T) {
a, i, home := configTestAPI(t, false)
id := "pre-restore-" + strings.Repeat("b", 32)
dir := filepath.Join(filepath.Dir(home), id)
if err := os.Mkdir(dir, 0700); err != nil {
t.Fatal(err)
}
for _, query := range []string{"", "../home", "/home", id + "/../home", "pre-restore-ABC"} {
w := httptest.NewRecorder()
a.snapshotDownload(w, httptest.NewRequest("GET", "/?download="+url.QueryEscape(query), nil), i)
if w.Code != 400 {
t.Fatal(query, w.Code)
}
}
foreign := i
foreign.UID = "someone-else"
w := httptest.NewRecorder()
a.snapshotDownload(w, httptest.NewRequest("GET", "/?download="+id, nil), foreign)
if w.Code != 404 {
t.Fatal(w.Code)
}
for _, target := range []string{"../home", "/workspace/example", "../../elsewhere", "missing", "link"} {
if err := os.Symlink(target, filepath.Join(dir, "link")); err != nil {
t.Fatal(err)
}
w = httptest.NewRecorder()
a.snapshotDownload(w, httptest.NewRequest("GET", "/?download="+id, nil), i)
if w.Code != 200 || w.Header().Get("Content-Disposition") == "" {
t.Fatal(target, w.Code)
}
gz, err := gzip.NewReader(w.Body)
if err != nil {
t.Fatal(err)
}
tr := tar.NewReader(gz)
found := false
for {
h, err := tr.Next()
if err == io.EOF {
break
}
if err != nil {
t.Fatal(err)
}
if h.Name == "link" {
found = h.Typeflag == tar.TypeSymlink && h.Linkname == target && h.Size == 0
} else if h.Name != "./" {
t.Fatalf("unexpected archived entry: %s", h.Name)
}
}
gz.Close()
if !found {
t.Fatal("link text was not preserved")
}
if err := os.Remove(filepath.Join(dir, "link")); err != nil {
t.Fatal(err)
}
}
if err := syscall.Mkfifo(filepath.Join(dir, "fifo"), 0600); err != nil {
t.Fatal(err)
}
w = httptest.NewRecorder()
a.snapshotDownload(w, httptest.NewRequest("GET", "/?download="+id, nil), i)
if w.Code != 400 {
t.Fatal(w.Code)
}
if err := os.Remove(filepath.Join(dir, "fifo")); err != nil {
t.Fatal(err)
}
if err := os.Remove(dir); err != nil {
t.Fatal(err)
}
if err := os.Symlink(home, dir); err != nil {
t.Fatal(err)
}
w = httptest.NewRecorder()
a.snapshotDownload(w, httptest.NewRequest("GET", "/?download="+id, nil), i)
if w.Code != 400 {
t.Fatal(w.Code)
}
}
func TestSnapshotArchiveLimitsAndCancellation(t *testing.T) {
dir := t.TempDir()
configWrite(t, filepath.Join(dir, "file"), strings.Repeat("x", 2048))
root, err := os.OpenRoot(dir)
if err != nil {
t.Fatal(err)
}
defer root.Close()
if err := snapshotArchive(context.Background(), root, io.Discard, 1024); !errors.Is(err, errSnapshotDownloadLimit) {
t.Fatal(err)
}
if err := snapshotArchive(context.Background(), root, io.Discard, 2048); err != nil {
t.Fatal("exact content limit should allow archive overhead", err)
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
if err := snapshotArchive(ctx, root, io.Discard, upgradeSnapshotLimit); !errors.Is(err, context.Canceled) {
t.Fatal(err)
}
a, i, _ := configTestAPI(t, false)
tmp := t.TempDir()
t.Setenv("TMPDIR", tmp)
r := httptest.NewRequest("GET", "/?download=pre-restore-"+strings.Repeat("a", 32), nil).WithContext(ctx)
w := httptest.NewRecorder()
a.snapshotDownload(w, r, i)
if w.Code != 408 {
t.Fatal(w.Code)
}
files, err := os.ReadDir(tmp)
if err != nil || len(files) != 0 {
t.Fatal(files, err)
}
if len(zipSlots) != 0 || a.instanceLocks.held() {
t.Fatal("resource leak")
}
t.Setenv("TMPDIR", filepath.Join(tmp, "missing"))
w = httptest.NewRecorder()
a.snapshotDownload(w, httptest.NewRequest("GET", r.URL.String(), nil), i)
if w.Code != 500 {
t.Fatal(w.Code)
}
}
+394
View File
@@ -0,0 +1,394 @@
package api
import (
"context"
"database/sql"
"encoding/json"
"errors"
"git.misaka.ren/m1saka/dsh-web/internal/store"
"io"
"net/http"
"net/url"
"os"
"path/filepath"
"strings"
"time"
)
const snapshotUnknownReason = "snapshot source image/version unknown; explicit acknowledgement required"
func snapshotRuntimeLayout(config, host map[string]json.RawMessage, mounts []struct{ Type, Source, Destination string }, home string) bool {
workspace := 0
for _, mount := range mounts {
if mount.Type == "tmpfs" && mount.Destination == "/tmp" {
continue
}
if mount.Type != "bind" || mount.Source != home || mount.Destination != "/workspace" {
return false
}
workspace++
}
if workspace != 1 {
return false
}
var env []string
if json.Unmarshal(config["Env"], &env) != nil {
return false
}
values := map[string]string{}
for _, entry := range env {
key, value, _ := strings.Cut(entry, "=")
values[key] = value
}
for key, want := range map[string]string{"DSH_HOME": "/workspace", "HOME": "/workspace", "XDG_DATA_HOME": "/workspace/.local/share", "XDG_CONFIG_HOME": "/workspace/.config"} {
if values[key] != want {
return false
}
}
for index, raw := range []json.RawMessage{host["Tmpfs"], config["Volumes"]} {
if len(raw) == 0 {
continue
}
var mounts map[string]json.RawMessage
if json.Unmarshal(raw, &mounts) != nil {
return false
}
if index == 0 {
delete(mounts, "/tmp")
}
if len(mounts) != 0 {
return false
}
}
return true
}
func (a *API) recordSnapshot(i store.Instance, id, version, image string) error {
_, err := a.s.DB.Exec("CREATE TABLE IF NOT EXISTS snapshot_metadata (instance_id TEXT NOT NULL, id TEXT NOT NULL, source_version TEXT NOT NULL, source_image TEXT NOT NULL, PRIMARY KEY(instance_id,id))")
if err != nil {
return err
}
_, err = a.s.DB.Exec("INSERT INTO snapshot_metadata(instance_id,id,source_version,source_image) VALUES(?,?,?,?)", i.ID, id, version, image)
return err
}
func (a *API) snapshotCompatibility(i store.Instance, id string) (version, image, reason string) {
var exists int
if err := a.s.DB.QueryRow("SELECT count(*) FROM sqlite_master WHERE type='table' AND name='snapshot_copy_state'").Scan(&exists); err != nil {
return "", "", "snapshot completion metadata unavailable"
}
if exists != 0 {
var ready int
err := a.s.DB.QueryRow("SELECT ready FROM snapshot_copy_state WHERE instance_id=? AND id=?", i.ID, id).Scan(&ready)
if err != nil && !errors.Is(err, sql.ErrNoRows) {
return "", "", "snapshot completion metadata unavailable"
}
if err == nil && ready != 1 {
return "", "", "snapshot copy incomplete"
}
}
if err := a.s.DB.QueryRow("SELECT count(*) FROM sqlite_master WHERE type='table' AND name='snapshot_metadata'").Scan(&exists); err != nil {
return "", "", "snapshot metadata unavailable"
}
if exists != 0 {
err := a.s.DB.QueryRow("SELECT source_version,source_image FROM snapshot_metadata WHERE instance_id=? AND id=?", i.ID, id).Scan(&version, &image)
if err != nil && !errors.Is(err, sql.ErrNoRows) {
return "", "", "snapshot metadata unavailable"
}
}
if (version != "" && version != i.Version) || (image != "" && image != i.ImageID) {
return version, image, "snapshot source image/version is incompatible"
}
if version == "" || image == "" {
return version, image, snapshotUnknownReason
}
return version, image, ""
}
func (a *API) snapshotRestore(w http.ResponseWriter, r *http.Request, i store.Instance) {
controller := http.NewResponseController(w)
w, flush := deferJSONResponse(w)
defer flush()
select {
case configBodySlots <- struct{}{}:
defer func() { r.Body = http.NoBody; <-configBodySlots }()
default:
fail(w, 503, "configuration uploads busy; retry later")
return
}
if !configReadBody(w, r, controller, 4096) {
return
}
var in struct {
ID string
Password string
Confirm bool
AllowUnknown bool `json:"allow_unknown_version"`
}
dec := json.NewDecoder(r.Body)
if dec.Decode(&in) != nil || dec.Decode(new(any)) != io.EOF || !snapshotIDPattern.MatchString(in.ID) || !in.Confirm {
fail(w, 400, "valid snapshot ID and explicit confirmation required")
return
}
lock := a.instanceLocks.Acquire(i.ID)
defer lock.Release()
a.readersMu.Lock()
readers := a.workspaceReaders[i.ID]
a.readersMu.Unlock()
if readers > 0 {
fail(w, 409, "instance files busy; retry later")
return
}
user := a.userLocks.Acquire(i.UID)
defer user.Release()
current, err := a.s.GetInstance(i.ID)
if err != nil || current.UID != i.UID {
fail(w, 404, "not found")
return
}
i = current
if !a.reauthenticate(w, i.UID, in.Password) {
return
}
if reason := a.backupProtection(i); reason != "" {
fail(w, 409, reason)
return
}
admissionMu.RLock()
defer admissionMu.RUnlock()
buildQueue.Lock()
err = a.ensureBuilds()
buildQueue.Unlock()
if err != nil {
fail(w, 500, "database error")
return
}
var blocked int
if err = a.s.DB.QueryRowContext(r.Context(), "SELECT count(*) FROM image_admissions WHERE image_id=? AND state='blocked'", i.ImageID).Scan(&blocked); err != nil {
fail(w, 500, "database error")
return
}
if blocked > 0 {
fail(w, 409, "image is blocked; restore denied")
return
}
version, image, reason := a.snapshotCompatibility(i, in.ID)
if reason != "" && !(reason == snapshotUnknownReason && in.AllowUnknown) {
fail(w, 409, reason)
return
}
root, err := managementRoot(a.root, "workspaces", i.UID, "instances", i.ID)
if errors.Is(err, os.ErrNotExist) {
fail(w, 404, "home or snapshot not found")
return
}
if err != nil {
managementPathError(w, err)
return
}
defer root.Close()
for _, name := range []string{"home", in.ID} {
st, e := root.Lstat(name)
if errors.Is(e, os.ErrNotExist) {
fail(w, 404, "home or snapshot not found")
return
}
if e != nil {
managementPathError(w, e)
return
}
if !st.IsDir() {
fail(w, 400, "unsafe home or snapshot")
return
}
}
f, err := root.Open(".")
if err != nil {
fail(w, 500, "snapshot storage unavailable")
return
}
entries, err := f.ReadDir(-1)
f.Close()
count := 0
for _, entry := range entries {
if strings.HasPrefix(entry.Name(), "pre-restore-") || strings.HasPrefix(entry.Name(), "failed-restore-") {
count++
}
}
if err != nil {
fail(w, 500, "snapshot listing failed")
return
}
if count >= 3 {
fail(w, 409, "restore snapshot limit reached; cleanup required")
return
}
size, err := quotaTreeBytes(root, in.ID)
var pathError *os.PathError
if errors.As(err, &pathError) {
managementPathError(w, err)
return
}
if err != nil || a.quotaGrowth(i.UID, size) != nil {
fail(w, 413, "quota exceeded or snapshot size unavailable")
return
}
ctx, cancel := context.WithTimeout(r.Context(), 3*time.Minute)
defer cancel()
var container struct {
Image string
Config, HostConfig map[string]json.RawMessage
Mounts []struct{ Type, Source, Destination string }
State struct{ Running, Paused, Restarting bool }
}
err = a.docker.Do(ctx, "GET", "/containers/"+url.PathEscape(i.ContainerID)+"/json", nil, &container)
if err != nil {
fail(w, 502, "container inspection failed")
return
}
if container.State.Paused || container.State.Restarting {
fail(w, 409, "unsupported container state")
return
}
var runtimeImage struct {
ID string `json:"Id"`
Config struct{ Labels map[string]string }
}
hostRoot := os.Getenv("HOST_DATA_ROOT")
if hostRoot == "" {
hostRoot = a.root
}
homeSource, homeErr := toHostPath(a.root, hostRoot, filepath.Join(root.Name(), "home"))
if homeErr != nil || !snapshotRuntimeLayout(container.Config, container.HostConfig, container.Mounts, homeSource) {
fail(w, 409, "incompatible effective runtime workspace layout")
return
}
if i.ImageID == "" || container.Image != i.ImageID || container.Config == nil || container.HostConfig == nil || homeErr != nil || a.docker.Do(ctx, "GET", "/images/"+url.PathEscape(i.ImageID)+"/json", nil, &runtimeImage) != nil || runtimeImage.ID != i.ImageID || runtimeImage.Config.Labels["dsh-mgr.version"] != i.Version || upgradeWorkspace(container.Config, container.HostConfig, runtimeImage.Config.Labels["dsh-mgr.recipe"], homeSource) != nil {
fail(w, 409, "incompatible runtime image or workspace layout")
return
}
wasRunning := container.State.Running
op := store.ID()
staging := ".snapshot-restore-" + op
previous := "pre-restore-" + op
failed := "failed-restore-" + op
if err = upgradeCopy(ctx, root, in.ID, staging, size); err != nil {
if root.RemoveAll(staging) != nil {
a.s.SetInstance(i.ID, "error")
fail(w, 500, "staging cleanup requires administrator recovery")
return
}
if errors.As(err, &pathError) {
managementPathError(w, err)
return
}
fail(w, 400, "unsafe or oversized snapshot")
return
}
defer func() {
if root.RemoveAll(staging) != nil {
a.s.SetInstance(i.ID, "error")
}
}()
if err = a.recordSnapshot(i, previous, i.Version, i.ImageID); err != nil {
fail(w, 500, "snapshot metadata unavailable")
return
}
if err = a.recordSnapshot(i, failed, version, image); err != nil {
fail(w, 500, "snapshot metadata unavailable")
return
}
if _, err = a.s.DB.Exec("UPDATE instances SET status='error' WHERE id=?", i.ID); err != nil {
fail(w, 500, "cannot record restore state")
return
}
if _, err = a.s.DB.Exec("INSERT INTO audit(uid,action,target,detail,at) VALUES(?,?,?,?,?)", i.UID, "snapshot.restore.started", i.ID, in.ID+" "+previous+" "+failed, time.Now().Unix()); err != nil {
fail(w, 500, "restore audit unavailable; recovery required")
return
}
if err = a.configStop(ctx, i.ContainerID); err != nil {
fail(w, 502, "runtime stop failed; recovery required")
return
}
moved, installed := false, false
var owner store.User
owner.ID = i.UID
err = a.s.DB.QueryRow("SELECT quota FROM users WHERE id=?", i.UID).Scan(&owner.Quota)
if err == nil {
var used int64
used, err = a.userUsage(owner)
if err == nil && used > owner.Quota {
err = errors.New("quota exceeded after runtime stop")
}
}
if err == nil {
err = root.Rename("home", previous)
}
if err == nil {
moved = true
err = upgradeSync(root)
}
if err == nil {
err = root.Rename(staging, "home")
installed = err == nil
}
if err == nil {
err = upgradeSync(root)
}
startReady := func(c context.Context) error {
runtimeAuthCache.Delete(runtimeAuthKey{a, i.ID})
if e := a.docker.Do(c, "POST", "/containers/"+url.PathEscape(i.ContainerID)+"/start", nil, nil); e != nil {
return e
}
return a.upgradeReady(c, i.ContainerID, i.ID, a.runtimeHost(r))
}
if err == nil {
err = startReady(ctx)
}
if err == nil && !wasRunning {
err = a.configStop(ctx, i.ContainerID)
}
rejected := err != nil
if rejected {
rollback, done := context.WithTimeout(context.Background(), 90*time.Second)
defer done()
if a.configStop(rollback, i.ContainerID) != nil {
fail(w, 500, "restore rollback requires runtime stop; recovery required")
return
}
if installed {
if root.Rename("home", failed) != nil || upgradeSync(root) != nil {
fail(w, 500, "restore rollback requires administrator recovery")
return
}
}
if moved {
if root.Rename(previous, "home") != nil || upgradeSync(root) != nil {
fail(w, 500, "restore rollback requires administrator recovery")
return
}
}
if wasRunning {
if startReady(rollback) != nil {
fail(w, 500, "restore rollback readiness failed; recovery required")
return
}
}
}
status := "exited"
if wasRunning {
status = "running"
}
if _, err = a.s.DB.Exec("UPDATE instances SET status=? WHERE id=?", status, i.ID); err != nil {
fail(w, 500, "restore state requires administrator recovery")
return
}
runtimeAuthCache.Delete(runtimeAuthKey{a, i.ID})
if rejected {
a.s.AddAudit(i.UID, "snapshot.restore.rolled_back", i.ID, in.ID)
fail(w, 400, "restore rejected; previous home restored")
return
}
a.s.AddAudit(i.UID, "snapshot.restore.completed", i.ID, in.ID)
writeJSON(w, map[string]any{"ok": true, "snapshot_id": previous})
}
+267
View File
@@ -0,0 +1,267 @@
package api
import (
"encoding/json"
"fmt"
"git.misaka.ren/m1saka/dsh-web/internal/docker"
"git.misaka.ren/m1saka/dsh-web/internal/store"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"syscall"
"testing"
)
func snapshotRestoreFixture(t *testing.T, running bool, failure string) (*API, store.Instance, string, string) {
t.Helper()
a, i, home := configTestAPI(t, running, failure)
i.ImageID = "sha256:" + strings.Repeat("a", 64)
i.Version = "1.0"
if _, err := a.s.DB.Exec("UPDATE instances SET image_id=?,version=? WHERE id=?", i.ImageID, i.Version, i.ID); err != nil {
t.Fatal(err)
}
original := a.docker
proxy := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if strings.Contains(r.URL.Path, "/images/") {
writeJSON(w, map[string]any{"Id": i.ImageID, "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.recipe": "7", "dsh-mgr.version": i.Version}}})
return
}
if strings.HasSuffix(r.URL.Path, "/json") {
var data map[string]any
if err := original.Do(r.Context(), "GET", r.URL.RequestURI(), nil, &data); err != nil {
t.Error(err)
w.WriteHeader(500)
return
}
data["Image"] = i.ImageID
data["Config"] = map[string]any{"Env": []string{"DSH_HOME=/workspace", "HOME=/workspace", "XDG_DATA_HOME=/workspace/.local/share", "XDG_CONFIG_HOME=/workspace/.config"}}
data["Mounts"] = []map[string]string{{"Type": "bind", "Source": home, "Destination": "/workspace"}}
data["HostConfig"] = map[string]any{"Mounts": []map[string]string{{"Type": "bind", "Source": home, "Target": "/workspace"}}}
data["HostConfig"].(map[string]any)["Tmpfs"] = map[string]string{"/tmp": "rw,noexec,nosuid,size=64m"}
writeJSON(w, data)
return
}
resp, err := original.Stream(r.Context(), r.Method, r.URL.RequestURI(), nil, "")
if err != nil {
t.Error(err)
w.WriteHeader(500)
return
}
defer resp.Body.Close()
w.WriteHeader(resp.StatusCode)
io.Copy(w, resp.Body)
}))
t.Cleanup(proxy.Close)
a.docker = docker.New(proxy.URL)
id := "upgrade-" + strings.Repeat("b", 32) + "-snapshot"
if err := os.Mkdir(filepath.Join(filepath.Dir(home), id), 0700); err != nil {
t.Fatal(err)
}
configWrite(t, filepath.Join(filepath.Dir(home), id, "data"), "snapshot")
configWrite(t, filepath.Join(home, "data"), "current")
return a, i, home, id
}
func snapshotRestoreBody(id string, unknown bool) string {
return fmt.Sprintf(`{"id":%q,"password":"original-password","confirm":true,"allow_unknown_version":%t}`, id, unknown)
}
func TestSnapshotRuntimeLayout(t *testing.T) {
for _, test := range []string{"valid", "env", "tmpfs", "volume", "effective"} {
t.Run(test, func(t *testing.T) {
env := []string{"DSH_HOME=/workspace", "HOME=/workspace", "XDG_DATA_HOME=/workspace/.local/share", "XDG_CONFIG_HOME=/workspace/.config"}
if test == "env" {
env = append(env, "DSH_HOME=/old-shared")
}
raw, _ := json.Marshal(env)
config := map[string]json.RawMessage{"Env": raw}
host := map[string]json.RawMessage{"Tmpfs": json.RawMessage(`{"/tmp":"rw"}`)}
mounts := []struct{ Type, Source, Destination string }{{"bind", "/home", "/workspace"}}
mounts = append(mounts, struct{ Type, Source, Destination string }{"tmpfs", "", "/tmp"})
if test == "tmpfs" {
host["Tmpfs"] = json.RawMessage(`{"/workspace/data":"rw"}`)
}
if test == "volume" {
config["Volumes"] = json.RawMessage(`{"/workspace/data":{}}`)
}
if test == "effective" {
mounts = append(mounts, struct{ Type, Source, Destination string }{"volume", "/other", "/workspace/data"})
}
if snapshotRuntimeLayout(config, host, mounts, "/home") != (test == "valid") {
t.Fatal(test)
}
})
}
}
func TestSnapshotListProvenance(t *testing.T) {
a, i, _, id := snapshotRestoreFixture(t, false, "")
for _, known := range []bool{false, true} {
if known {
if err := a.recordSnapshot(i, id, "old-version", i.ImageID); err != nil {
t.Fatal(err)
}
}
w := configCall(a, i, "GET", "snapshots", "")
var list []snapshotInfo
if w.Code != 200 || json.Unmarshal(w.Body.Bytes(), &list) != nil || len(list) != 1 {
t.Fatal(w.Code, w.Body.String())
}
if list[0].Restorable == known || list[0].RestoreReason == "" {
t.Fatal(list)
}
if !known && (list[0].SourceVersion != "" || list[0].SourceImage != "") {
t.Fatal("inferred provenance", list)
}
}
}
func TestSnapshotFullRestore(t *testing.T) {
for _, running := range []bool{false, true} {
t.Run(fmt.Sprint(running), func(t *testing.T) {
a, i, home, id := snapshotRestoreFixture(t, running, "")
links := map[string]string{"absolute": "/workspace/example", "dangling": "missing", "loop": "loop", "relative": "data"}
for name, target := range links {
if err := os.Symlink(target, filepath.Join(filepath.Dir(home), id, name)); err != nil {
t.Fatal(err)
}
}
if err := a.recordSnapshot(i, id, i.Version, i.ImageID); err != nil {
t.Fatal(err)
}
w := configCall(a, i, "POST", "snapshots", snapshotRestoreBody(id, false))
if w.Code != 200 {
t.Fatal(w.Code, w.Body.String())
}
var result struct {
SnapshotID string `json:"snapshot_id"`
}
json.Unmarshal(w.Body.Bytes(), &result)
for path, want := range map[string]string{filepath.Join(home, "data"): "snapshot", filepath.Join(filepath.Dir(home), id, "data"): "snapshot", filepath.Join(filepath.Dir(home), result.SnapshotID, "data"): "current"} {
got, err := os.ReadFile(path)
if err != nil || string(got) != want {
t.Fatal(path, string(got), err)
}
}
current, _ := a.s.GetInstance(i.ID)
for name, target := range links {
got, err := os.Readlink(filepath.Join(home, name))
if err != nil || got != target {
t.Fatalf("restored link %s: %q %v", name, got, err)
}
}
want := "exited"
if running {
want = "running"
}
if current.Status != want {
t.Fatal(current.Status)
}
})
}
}
func TestSnapshotRestoreRollback(t *testing.T) {
a, i, home, id := snapshotRestoreFixture(t, false, "reconcile")
w := configCall(a, i, "POST", "snapshots", snapshotRestoreBody(id, true))
if w.Code != 400 || !strings.Contains(w.Body.String(), "previous home restored") {
t.Fatal(w.Code, w.Body.String())
}
for path, want := range map[string]string{filepath.Join(home, "data"): "current", filepath.Join(filepath.Dir(home), id, "data"): "snapshot"} {
got, err := os.ReadFile(path)
if err != nil || string(got) != want {
t.Fatal(path, string(got), err)
}
}
matches, _ := filepath.Glob(filepath.Join(filepath.Dir(home), "failed-restore-*", "data"))
if len(matches) != 1 {
t.Fatal(matches)
}
}
func TestIncompleteSnapshotNotRestorable(t *testing.T) {
a, i, _, id := snapshotRestoreFixture(t, false, "")
if _, err := a.s.DB.Exec("CREATE TABLE snapshot_copy_state (instance_id TEXT, id TEXT, ready INTEGER)"); err != nil {
t.Fatal(err)
}
if _, err := a.s.DB.Exec("INSERT INTO snapshot_copy_state VALUES(?,?,0)", i.ID, id); err != nil {
t.Fatal(err)
}
w := configCall(a, i, "GET", "snapshots", "")
var list []snapshotInfo
if w.Code != 200 || json.Unmarshal(w.Body.Bytes(), &list) != nil || len(list) != 1 || list[0].Restorable || list[0].RestoreReason != "snapshot copy incomplete" {
t.Fatal(w.Code, w.Body.String())
}
w = configCall(a, i, "POST", "snapshots", snapshotRestoreBody(id, true))
if w.Code != 409 || !strings.Contains(w.Body.String(), "snapshot copy incomplete") {
t.Fatal(w.Code, w.Body.String())
}
if err := a.recordSnapshot(i, id, i.Version, i.ImageID); err != nil {
t.Fatal(err)
}
if _, err := a.s.DB.Exec("UPDATE snapshot_copy_state SET ready=1"); err != nil {
t.Fatal(err)
}
if _, _, reason := a.snapshotCompatibility(i, id); reason != "" {
t.Fatal(reason)
}
if _, err := a.s.DB.Exec("ALTER TABLE snapshot_copy_state RENAME COLUMN ready TO unavailable"); err != nil {
t.Fatal(err)
}
if _, _, reason := a.snapshotCompatibility(i, id); reason != "snapshot completion metadata unavailable" {
t.Fatal(reason)
}
}
func TestSnapshotRestoreGuards(t *testing.T) {
for _, test := range []string{"unknown", "mismatch", "password", "confirm", "reader", "quota", "error", "unsafe", "foreign", "oversized"} {
t.Run(test, func(t *testing.T) {
a, i, home, id := snapshotRestoreFixture(t, false, "")
body := snapshotRestoreBody(id, true)
want := 409
switch test {
case "unknown":
body = snapshotRestoreBody(id, false)
case "mismatch":
if err := a.recordSnapshot(i, id, "different", i.ImageID); err != nil {
t.Fatal(err)
}
case "password":
body = strings.ReplaceAll(body, "original-password", "wrong")
want = 403
case "confirm":
body = strings.ReplaceAll(body, `"confirm":true`, `"confirm":false`)
want = 400
case "reader":
a.workspaceReaders = map[string]int{i.ID: 1}
case "quota":
a.s.DB.Exec("UPDATE users SET quota=1 WHERE id=?", i.UID)
want = 413
case "error":
a.s.SetInstance(i.ID, "error")
case "unsafe":
if err := syscall.Mkfifo(filepath.Join(filepath.Dir(home), id, "unsafe"), 0600); err != nil {
t.Fatal(err)
}
want = 400
case "foreign":
i.UID = "someone-else"
want = 404
case "oversized":
body = strings.Repeat("x", 4097)
want = 413
}
w := configCall(a, i, "POST", "snapshots", body)
if w.Code != want {
t.Fatal(w.Code, w.Body.String())
}
got, err := os.ReadFile(filepath.Join(home, "data"))
if err != nil || string(got) != "current" {
t.Fatal(string(got), err)
}
})
}
}
+55 -74
View File
@@ -4,6 +4,7 @@ import (
"context" "context"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt"
"io" "io"
"net" "net"
"net/http" "net/http"
@@ -20,7 +21,7 @@ import (
const upgradeSnapshotLimit int64 = 1 << 30 const upgradeSnapshotLimit int64 = 1 << 30
func upgradeWorkspace(config, host map[string]json.RawMessage, recipe, home, shared string) error { func upgradeWorkspace(config, host map[string]json.RawMessage, recipe, home string) error {
var mounts []map[string]json.RawMessage var mounts []map[string]json.RawMessage
if err := json.Unmarshal(host["Mounts"], &mounts); err != nil { if err := json.Unmarshal(host["Mounts"], &mounts); err != nil {
return errors.New("unsupported runtime mounts") return errors.New("unsupported runtime mounts")
@@ -30,53 +31,11 @@ func upgradeWorkspace(config, host map[string]json.RawMessage, recipe, home, sha
_ = json.Unmarshal(m[key], &value) _ = json.Unmarshal(m[key], &value)
return value return value
} }
homeIndex, sharedIndex := -1, -1
for n, m := range mounts {
target, source := get(m, "Target"), get(m, "Source")
if target != "/workspace" && target != "/data/dsh-home" && target != "/shared" {
continue
}
if get(m, "Type") != "bind" {
return errors.New("unsupported runtime mount type")
}
switch {
case source == home && (target == "/workspace" || target == "/data/dsh-home") && homeIndex == -1:
homeIndex = n
case source == shared && (target == "/workspace" || target == "/shared") && sharedIndex == -1:
sharedIndex = n
default:
return errors.New("ambiguous runtime mounts")
}
}
if homeIndex == -1 {
return errors.New("instance home mount required")
}
if !supportedLayout(recipe) { if !supportedLayout(recipe) {
return errors.New("unsupported template layout") return errors.New("unsupported template layout; template 7 required")
} }
if recipe != "7" { if len(mounts) != 1 || get(mounts[0], "Type") != "bind" || get(mounts[0], "Source") != home || get(mounts[0], "Target") != "/workspace" {
if get(mounts[homeIndex], "Target") == "/workspace" { return errors.New("independent instance home at /workspace required")
return errors.New("workspace home cannot be downgraded to legacy shared layout")
}
if sharedIndex == -1 || get(mounts[sharedIndex], "Target") != "/workspace" {
return errors.New("legacy shared workspace mount required")
}
var env []string
if raw := config["Env"]; len(raw) != 0 {
if err := json.Unmarshal(raw, &env); err != nil {
return err
}
}
kept := []string{}
for _, entry := range env {
key, _, _ := strings.Cut(entry, "=")
if key != "DSH_HOME" && key != "HOME" && key != "XDG_DATA_HOME" && key != "XDG_CONFIG_HOME" {
kept = append(kept, entry)
}
}
kept = append(kept, "DSH_HOME=/data/dsh-home", "HOME=/data/dsh-home", "XDG_DATA_HOME=/data/dsh-home/.local/share", "XDG_CONFIG_HOME=/data/dsh-home/.config")
config["Env"], _ = json.Marshal(kept)
return nil
} }
for _, key := range []string{"Binds", "VolumesFrom"} { for _, key := range []string{"Binds", "VolumesFrom"} {
var values []string var values []string
@@ -102,11 +61,6 @@ func upgradeWorkspace(config, host map[string]json.RawMessage, recipe, home, sha
} }
} }
kept = append(kept, "DSH_HOME=/workspace", "HOME=/workspace", "XDG_DATA_HOME=/workspace/.local/share", "XDG_CONFIG_HOME=/workspace/.config") kept = append(kept, "DSH_HOME=/workspace", "HOME=/workspace", "XDG_DATA_HOME=/workspace/.local/share", "XDG_CONFIG_HOME=/workspace/.config")
mounts[homeIndex]["Target"], _ = json.Marshal("/workspace")
if sharedIndex != -1 {
mounts = append(mounts[:sharedIndex], mounts[sharedIndex+1:]...)
}
host["Mounts"], _ = json.Marshal(mounts)
config["Env"], _ = json.Marshal(kept) config["Env"], _ = json.Marshal(kept)
return nil return nil
} }
@@ -147,9 +101,17 @@ func upgradeSync(r *os.Root) error {
func upgradeCopy(ctx context.Context, r *os.Root, source, target string, limit int64) error { func upgradeCopy(ctx context.Context, r *os.Root, source, target string, limit int64) error {
remaining, entries := limit, 100000 remaining, entries := limit, 100000
var links []string
var copyTree func(string, string, int) error var copyTree func(string, string, int) error
copyTree = func(src, dst string, depth int) error { copyTree = func(src, dst string, depth int) (copyErr error) {
defer func() {
if copyErr != nil {
if pathErr, ok := copyErr.(*os.PathError); ok {
copyErr = fmt.Errorf("snapshot copy: %w", &os.PathError{Op: pathErr.Op, Path: src, Err: pathErr.Err})
return
}
copyErr = fmt.Errorf("%s: %w", src, copyErr)
}
}()
if err := ctx.Err(); err != nil { if err := ctx.Err(); err != nil {
return err return err
} }
@@ -202,14 +164,11 @@ func upgradeCopy(ctx context.Context, r *os.Root, source, target string, limit i
if err != nil { if err != nil {
return err return err
} }
rel, err := filepath.Rel(source, src) // Preserve the link itself, including dangling and container-absolute
if err != nil || filepath.IsAbs(link) || !filepath.IsLocal(filepath.Join(filepath.Dir(rel), link)) { // targets. Never resolve it against the panel's filesystem.
return errors.New("unsafe snapshot link")
}
if err = r.Symlink(link, dst); err != nil { if err = r.Symlink(link, dst); err != nil {
return err return err
} }
links = append(links, rel)
return r.Lchown(dst, 1000, 1000) return r.Lchown(dst, 1000, 1000)
} }
if !st.Mode().IsRegular() || st.Size() > remaining { if !st.Mode().IsRegular() || st.Size() > remaining {
@@ -248,17 +207,6 @@ func upgradeCopy(ctx context.Context, r *os.Root, source, target string, limit i
if err := copyTree(source, target, 0); err != nil { if err := copyTree(source, target, 0); err != nil {
return err return err
} }
snapshot, err := r.OpenRoot(target)
if err != nil {
return err
}
defer snapshot.Close()
// Resolve only within the completed snapshot, including intermediate symlinks.
for _, link := range links {
if _, err := snapshot.Stat(link); err != nil {
return errors.New("unsafe or unresolved snapshot link")
}
}
return upgradeSync(r) return upgradeSync(r)
} }
@@ -373,12 +321,11 @@ func (a *API) upgrade(w http.ResponseWriter, r *http.Request, i store.Instance)
hostRoot = a.root hostRoot = a.root
} }
homeSource, homeErr := toHostPath(a.root, hostRoot, filepath.Join(root.Name(), "home")) homeSource, homeErr := toHostPath(a.root, hostRoot, filepath.Join(root.Name(), "home"))
sharedSource, sharedErr := toHostPath(a.root, hostRoot, filepath.Join(a.root, "workspaces", i.UID, "files")) if homeErr != nil {
if homeErr != nil || sharedErr != nil {
fail(w, 500, "host storage unavailable") fail(w, 500, "host storage unavailable")
return return
} }
if err = upgradeWorkspace(old.Config, old.HostConfig, image.Config.Labels["dsh-mgr.recipe"], homeSource, sharedSource); err != nil { if err = upgradeWorkspace(old.Config, old.HostConfig, image.Config.Labels["dsh-mgr.recipe"], homeSource); err != nil {
fail(w, 409, "incompatible runtime workspace layout") fail(w, 409, "incompatible runtime workspace layout")
return return
} }
@@ -388,6 +335,10 @@ func (a *API) upgrade(w http.ResponseWriter, r *http.Request, i store.Instance)
} }
op := store.ID() op := store.ID()
snapshot, failedHome := "upgrade-"+op+"-snapshot", "failed-home-"+op snapshot, failedHome := "upgrade-"+op+"-snapshot", "failed-home-"+op
if a.recordSnapshot(i, snapshot, i.Version, i.ImageID) != nil || a.recordSnapshot(i, failedHome, image.Config.Labels["dsh-mgr.version"], image.ID) != nil {
fail(w, 500, "snapshot metadata unavailable")
return
}
newName := runtimeNetwork(i.ID) + "-upgrade-" + op newName := runtimeNetwork(i.ID) + "-upgrade-" + op
_, err = a.s.DB.Exec(`CREATE TABLE IF NOT EXISTS upgrade_journal (id TEXT PRIMARY KEY, instance_id TEXT NOT NULL, old_container TEXT NOT NULL, new_container TEXT NOT NULL DEFAULT '', new_name TEXT NOT NULL, image_id TEXT NOT NULL, snapshot TEXT NOT NULL, failed_home TEXT NOT NULL, phase TEXT NOT NULL)`) _, err = a.s.DB.Exec(`CREATE TABLE IF NOT EXISTS upgrade_journal (id TEXT PRIMARY KEY, instance_id TEXT NOT NULL, old_container TEXT NOT NULL, new_container TEXT NOT NULL DEFAULT '', new_name TEXT NOT NULL, image_id TEXT NOT NULL, snapshot TEXT NOT NULL, failed_home TEXT NOT NULL, phase TEXT NOT NULL)`)
if err != nil { if err != nil {
@@ -504,8 +455,38 @@ func (a *API) upgrade(w http.ResponseWriter, r *http.Request, i store.Instance)
fail(w, 413, "quota exceeded or snapshot size unavailable") fail(w, 413, "quota exceeded or snapshot size unavailable")
return return
} }
if upgradeCopy(ctx, root, "home", snapshot, snapshotBytes) != nil { // Only publish a completed copy under a name the snapshot API recognizes.
fail(w, 500, "home snapshot failed") staging := "." + snapshot + "-partial"
if _, err := a.s.DB.Exec("CREATE TABLE IF NOT EXISTS snapshot_copy_state (instance_id TEXT NOT NULL, id TEXT NOT NULL, ready INTEGER NOT NULL, PRIMARY KEY(instance_id,id))"); err != nil {
fail(w, 500, "snapshot completion metadata unavailable")
return
}
if _, err := a.s.DB.Exec("INSERT INTO snapshot_copy_state VALUES(?,?,0)", i.ID, snapshot); err != nil {
fail(w, 500, "snapshot completion metadata unavailable")
return
}
if copyErr := upgradeCopy(ctx, root, "home", staging, snapshotBytes); copyErr != nil {
if cleanupErr := root.RemoveAll(staging); cleanupErr != nil {
a.s.AddAudit(i.UID, "upgrade.snapshot.cleanup_failed", i.ID, staging)
fail(w, 500, "home snapshot failed: "+copyErr.Error()+"; partial snapshot cleanup required")
return
}
fail(w, 500, "home snapshot failed: "+copyErr.Error())
return
}
if err := root.Rename(staging, snapshot); err != nil {
if root.RemoveAll(staging) != nil {
a.s.AddAudit(i.UID, "upgrade.snapshot.cleanup_failed", i.ID, staging)
}
fail(w, 500, "home snapshot publication failed")
return
}
if upgradeSync(root) != nil {
fail(w, 500, "home snapshot publication sync failed")
return
}
if _, err := a.s.DB.Exec("UPDATE snapshot_copy_state SET ready=1 WHERE instance_id=? AND id=?", i.ID, snapshot); err != nil {
fail(w, 500, "snapshot completion metadata unavailable")
return return
} }
snapReady = true snapReady = true
+74 -28
View File
@@ -23,10 +23,10 @@ func TestUpgradeWorkspaceCompatibility(t *testing.T) {
name, recipe, target string name, recipe, target string
shared, reject bool shared, reject bool
}{ }{
{"legacy-to-seven", "7", "/data/dsh-home", true, false}, {"legacy-to-seven", "7", "/data/dsh-home", true, true},
{"single-to-seven", "7", "/workspace", false, false}, {"single-to-seven", "7", "/workspace", false, false},
{"migrated-to-seven", "7", "/workspace", true, false}, {"migrated-to-seven", "7", "/workspace", true, true},
{"legacy-to-six", "6", "/data/dsh-home", true, false}, {"legacy-to-six", "6", "/data/dsh-home", true, true},
{"legacy-to-unknown", "", "/data/dsh-home", true, true}, {"legacy-to-unknown", "", "/data/dsh-home", true, true},
{"single-to-six", "6", "/workspace", false, true}, {"single-to-six", "6", "/workspace", false, true},
{"single-to-unknown", "", "/workspace", false, true}, {"single-to-unknown", "", "/workspace", false, true},
@@ -44,16 +44,10 @@ func TestUpgradeWorkspaceCompatibility(t *testing.T) {
host := map[string]json.RawMessage{"Mounts": raw} host := map[string]json.RawMessage{"Mounts": raw}
config := map[string]json.RawMessage{"Env": json.RawMessage(`["CUSTOM=yes","DSH_HOME=/data/dsh-home","HOME=/data/dsh-home","XDG_DATA_HOME=/old/data","XDG_CONFIG_HOME=/old/config"]`)} config := map[string]json.RawMessage{"Env": json.RawMessage(`["CUSTOM=yes","DSH_HOME=/data/dsh-home","HOME=/data/dsh-home","XDG_DATA_HOME=/old/data","XDG_CONFIG_HOME=/old/config"]`)}
before := string(config["Env"]) before := string(config["Env"])
err := upgradeWorkspace(config, host, tc.recipe, "/instance/home", "/user/files") err := upgradeWorkspace(config, host, tc.recipe, "/instance/home")
if (err != nil) != tc.reject { if (err != nil) != tc.reject {
t.Fatalf("error=%v", err) t.Fatalf("error=%v", err)
} }
if !tc.reject && tc.recipe != "7" {
if !strings.Contains(string(config["Env"]), "XDG_DATA_HOME=/data/dsh-home/.local/share") || string(host["Mounts"]) != string(raw) {
t.Fatal(config, host)
}
return
}
if tc.reject { if tc.reject {
if string(host["Mounts"]) != string(raw) || string(config["Env"]) != before { if string(host["Mounts"]) != string(raw) || string(config["Env"]) != before {
t.Fatal("legacy or rejected config changed") t.Fatal("legacy or rejected config changed")
@@ -71,7 +65,7 @@ func TestUpgradeWorkspaceCompatibility(t *testing.T) {
t.Fatal(got) t.Fatal(got)
} }
first, _ := json.Marshal([]any{config, host}) first, _ := json.Marshal([]any{config, host})
if err := upgradeWorkspace(config, host, tc.recipe, "/instance/home", "/user/files"); err != nil { if err := upgradeWorkspace(config, host, tc.recipe, "/instance/home"); err != nil {
t.Fatal(err) t.Fatal(err)
} }
second, _ := json.Marshal([]any{config, host}) second, _ := json.Marshal([]any{config, host})
@@ -90,9 +84,9 @@ func TestUpgradeWorkspaceRejectsAmbiguousMounts(t *testing.T) {
`{"Type":"bind","Source":"/instance/home","Target":"/workspace"}`, `{"Type":"bind","Source":"/instance/home","Target":"/workspace"}`,
} { } {
config := map[string]json.RawMessage{"Env": json.RawMessage(`["CUSTOM=yes"]`)} config := map[string]json.RawMessage{"Env": json.RawMessage(`["CUSTOM=yes"]`)}
host := map[string]json.RawMessage{"Mounts": json.RawMessage(`[{"Type":"bind","Source":"/instance/home","Target":"/data/dsh-home"},` + extra + `]`)} host := map[string]json.RawMessage{"Mounts": json.RawMessage(`[{"Type":"bind","Source":"/instance/home","Target":"/workspace"},` + extra + `]`)}
before, _ := json.Marshal([]any{config, host}) before, _ := json.Marshal([]any{config, host})
if err := upgradeWorkspace(config, host, "7", "/instance/home", "/user/files"); err == nil { if err := upgradeWorkspace(config, host, "7", "/instance/home"); err == nil {
t.Fatal("accepted ambiguous mounts", extra) t.Fatal("accepted ambiguous mounts", extra)
} }
after, _ := json.Marshal([]any{config, host}) after, _ := json.Marshal([]any{config, host})
@@ -106,7 +100,7 @@ func TestUpgradeLifecycle(t *testing.T) {
if os.Geteuid() != 0 { if os.Geteuid() != 0 {
t.Skip("requires chown") t.Skip("requires chown")
} }
for _, mode := range []string{"success", "start-failure", "readiness-failure", "cleanup-failure", "running", "running-failure", "restart-failure", "delete-failure", "invalid-image", "unverified-image", "failed-build", "version-mismatch", "image-id-mismatch", "db-failure"} { for _, mode := range []string{"snapshot-fifo", "snapshot-dangling-link", "success", "legacy-layout", "shared-layout", "start-failure", "readiness-failure", "cleanup-failure", "running", "running-failure", "restart-failure", "delete-failure", "invalid-image", "unverified-image", "failed-build", "version-mismatch", "image-id-mismatch", "db-failure"} {
t.Run(mode, func(t *testing.T) { t.Run(mode, func(t *testing.T) {
image := "sha256:" + strings.Repeat("b", 64) image := "sha256:" + strings.Repeat("b", 64)
var a *API var a *API
@@ -149,7 +143,15 @@ func TestUpgradeLifecycle(t *testing.T) {
if running { if running {
state = "running" state = "running"
} }
writeJSON(w, map[string]any{"Config": map[string]any{"Hostname": "old", "Cmd": []string{"obsolete-flag"}, "Image": "old-image", "Env": []string{"CUSTOM=yes"}, "User": "1000:1000"}, "HostConfig": map[string]any{"NetworkMode": "dsh-inst-one", "Memory": 9007199254740993, "Mounts": []map[string]string{{"Type": "bind", "Source": home, "Target": "/data/dsh-home"}, {"Type": "bind", "Source": shared, "Target": "/workspace"}}}, "State": map[string]any{"Status": state, "Running": running && !stopped}, "NetworkSettings": map[string]any{"Networks": map[string]any{"dsh-inst-one": map[string]any{"IPAddress": "172.18.0.3", "Aliases": []string{"old"}}}}}) mounts := []map[string]string{{"Type": "bind", "Source": home, "Target": "/workspace"}}
if mode == "legacy-layout" {
mounts[0]["Target"] = "/data/dsh-home"
mounts = append(mounts, map[string]string{"Type": "bind", "Source": shared, "Target": "/workspace"})
}
if mode == "shared-layout" {
mounts = append(mounts, map[string]string{"Type": "bind", "Source": shared, "Target": "/shared"})
}
writeJSON(w, map[string]any{"Config": map[string]any{"Hostname": "old", "Cmd": []string{"obsolete-flag"}, "Image": "old-image", "Env": []string{"CUSTOM=yes"}, "User": "1000:1000"}, "HostConfig": map[string]any{"NetworkMode": "dsh-inst-one", "Memory": 9007199254740993, "Mounts": mounts}, "State": map[string]any{"Status": state, "Running": running && !stopped}, "NetworkSettings": map[string]any{"Networks": map[string]any{"dsh-inst-one": map[string]any{"IPAddress": "172.18.0.3", "Aliases": []string{"old"}}}}})
case strings.HasPrefix(p, "/images/"): case strings.HasPrefix(p, "/images/"):
role := "runtime" role := "runtime"
if mode == "invalid-image" { if mode == "invalid-image" {
@@ -258,6 +260,16 @@ func TestUpgradeLifecycle(t *testing.T) {
if err := os.WriteFile(filepath.Join(home, ".config", "state"), []byte("original"), 0600); err != nil { if err := os.WriteFile(filepath.Join(home, ".config", "state"), []byte("original"), 0600); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if mode == "snapshot-dangling-link" {
if err := os.Symlink("missing", filepath.Join(home, "dangling")); err != nil {
t.Fatal(err)
}
}
if mode == "snapshot-fifo" {
if err := syscall.Mkfifo(filepath.Join(home, "pipe"), 0600); err != nil {
t.Fatal(err)
}
}
i := store.Instance{ID: "one", UID: "owner", Name: "one", ContainerID: "old", ImageID: "old-image", Version: "v1", Status: "exited", Created: time.Now()} i := store.Instance{ID: "one", UID: "owner", Name: "one", ContainerID: "old", ImageID: "old-image", Version: "v1", Status: "exited", Created: time.Now()}
if err := a.s.AddInstance(i); err != nil { if err := a.s.AddInstance(i); err != nil {
t.Fatal(err) t.Fatal(err)
@@ -273,8 +285,11 @@ func TestUpgradeLifecycle(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
if mode == "invalid-image" || mode == "unverified-image" || mode == "failed-build" || mode == "version-mismatch" || mode == "image-id-mismatch" { if mode == "legacy-layout" || mode == "shared-layout" || mode == "invalid-image" || mode == "unverified-image" || mode == "failed-build" || mode == "version-mismatch" || mode == "image-id-mismatch" {
want := 400 want := 400
if mode == "legacy-layout" || mode == "shared-layout" {
want = 409
}
if w.Code != want { if w.Code != want {
t.Fatalf("%d %s", w.Code, w.Body.String()) t.Fatalf("%d %s", w.Code, w.Body.String())
} }
@@ -292,6 +307,38 @@ func TestUpgradeLifecycle(t *testing.T) {
if err := a.s.DB.QueryRow("SELECT phase,snapshot,failed_home FROM upgrade_journal WHERE instance_id='one'").Scan(&phase, &snapshot, &failed); err != nil { if err := a.s.DB.QueryRow("SELECT phase,snapshot,failed_home FROM upgrade_journal WHERE instance_id='one'").Scan(&phase, &snapshot, &failed); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if mode == "snapshot-fifo" {
if w.Code != 500 || !strings.Contains(w.Body.String(), "home/pipe") || phase != "rolled_back" || got.ContainerID != "old" {
t.Fatalf("%d %s phase=%s", w.Code, w.Body.String(), phase)
}
entries, err := os.ReadDir(base)
if err != nil || len(entries) != 1 || entries[0].Name() != "home" {
t.Fatalf("partial snapshot leaked: %v %v", entries, err)
}
listed := configCall(a, got, "GET", "snapshots", "")
if listed.Code != 200 || strings.TrimSpace(listed.Body.String()) != "[]" {
t.Fatal(listed.Code, listed.Body.String())
}
return
}
if mode == "snapshot-dangling-link" {
if w.Code != 200 || phase != "complete" || got.ContainerID != "new" {
t.Fatalf("unexpected result: %d %s phase=%s instance=%+v", w.Code, w.Body.String(), phase, got)
}
saved, err := os.ReadFile(filepath.Join(base, snapshot, ".config", "state"))
if err != nil || string(saved) != "original" {
t.Fatalf("copied data missing: %q %v", saved, err)
}
link, err := os.Readlink(filepath.Join(base, snapshot, "dangling"))
if err != nil || link != "missing" {
t.Fatalf("copied link missing: %q %v", link, err)
}
listed := configCall(a, got, "GET", "snapshots", "")
if listed.Code != 200 || !strings.Contains(listed.Body.String(), snapshot) {
t.Fatalf("snapshot not listed: %d %s", listed.Code, listed.Body.String())
}
return
}
data, err := os.ReadFile(filepath.Join(home, ".config", "state")) data, err := os.ReadFile(filepath.Join(home, ".config", "state"))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -370,7 +417,7 @@ func TestUpgradeLifecycle(t *testing.T) {
} }
func TestUpgradeSnapshotRejectsUnsafeFiles(t *testing.T) { func TestUpgradeSnapshotRejectsUnsafeFiles(t *testing.T) {
for _, kind := range []string{"symlink", "fifo", "oversize", "directory-symlink"} { for _, kind := range []string{"fifo", "oversize"} {
t.Run(kind, func(t *testing.T) { t.Run(kind, func(t *testing.T) {
base := t.TempDir() base := t.TempDir()
if err := os.Mkdir(filepath.Join(base, "home"), 0700); err != nil { if err := os.Mkdir(filepath.Join(base, "home"), 0700); err != nil {
@@ -379,14 +426,10 @@ func TestUpgradeSnapshotRejectsUnsafeFiles(t *testing.T) {
p := filepath.Join(base, "home", "bad") p := filepath.Join(base, "home", "bad")
var err error var err error
switch kind { switch kind {
case "symlink":
err = os.Symlink("/etc/passwd", p)
case "fifo": case "fifo":
err = syscall.Mkfifo(p, 0600) err = syscall.Mkfifo(p, 0600)
case "oversize": case "oversize":
err = os.WriteFile(p, []byte("12345"), 0600) err = os.WriteFile(p, []byte("12345"), 0600)
case "directory-symlink":
err = os.Symlink("/tmp", p)
} }
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -407,7 +450,7 @@ func TestUpgradeSnapshotRelativeLinks(t *testing.T) {
if os.Geteuid() != 0 { if os.Geteuid() != 0 {
t.Skip("requires chown") t.Skip("requires chown")
} }
for _, mode := range []string{"pnpm", "escape", "chain-escape", "loop", "dangling"} { for _, mode := range []string{"pnpm", "absolute", "escape", "chain-escape", "loop", "dangling"} {
t.Run(mode, func(t *testing.T) { t.Run(mode, func(t *testing.T) {
base := t.TempDir() base := t.TempDir()
home := filepath.Join(base, "home") home := filepath.Join(base, "home")
@@ -420,6 +463,8 @@ func TestUpgradeSnapshotRelativeLinks(t *testing.T) {
} }
link := ".pnpm/pkg/node_modules/pkg" link := ".pnpm/pkg/node_modules/pkg"
switch mode { switch mode {
case "absolute":
link = "/workspace/example"
case "escape": case "escape":
link = "../../outside" link = "../../outside"
case "chain-escape": case "chain-escape":
@@ -441,12 +486,6 @@ func TestUpgradeSnapshotRelativeLinks(t *testing.T) {
} }
defer root.Close() defer root.Close()
err = upgradeCopy(context.Background(), root, "home", "snapshot", 1024) err = upgradeCopy(context.Background(), root, "home", "snapshot", 1024)
if mode != "pnpm" {
if err == nil {
t.Fatal("unsafe link accepted")
}
return
}
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -460,6 +499,13 @@ func TestUpgradeSnapshotRelativeLinks(t *testing.T) {
if err := root.Rename("snapshot", "home"); err != nil { if err := root.Rename("snapshot", "home"); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if mode != "pnpm" {
got, err := root.Readlink("home/node_modules/pkg")
if err != nil || got != link {
t.Fatalf("restored link changed: %q %v", got, err)
}
return
}
data, err := os.ReadFile(filepath.Join(home, "node_modules/pkg/index.js")) data, err := os.ReadFile(filepath.Join(home, "node_modules/pkg/index.js"))
if err != nil || string(data) != "original" { if err != nil || string(data) != "original" {
t.Fatalf("rollback link broken: %q %v", data, err) t.Fatalf("rollback link broken: %q %v", data, err)
+117
View File
@@ -0,0 +1,117 @@
package api
import (
"bytes"
"encoding/xml"
"io"
"mime"
"net/http"
"os"
"path"
"strconv"
"strings"
"syscall"
)
const maxImageBytes = 16 << 20
func (a *API) wsImage(w http.ResponseWriter, r *http.Request, root *os.Root) {
if r.Method != http.MethodGet {
fail(w, 405, "method not allowed")
return
}
p := r.URL.Query().Get("path")
if p == "" || path.IsAbs(p) || p == "." || p == ".." || strings.HasPrefix(p, "../") || path.Clean(p) != p || strings.ContainsRune(p, 0) {
fail(w, 400, "invalid path")
return
}
anchor, err := root.Open(".")
if err != nil {
fail(w, 500, "storage unavailable")
return
}
defer anchor.Close()
f, err := openDownload(anchor, p)
if err != nil {
fail(w, 404, "image unavailable")
return
}
defer f.Close()
st, err := f.Stat()
if err != nil || !st.Mode().IsRegular() {
fail(w, 415, "unsupported image")
return
}
if st.Size() > maxImageBytes {
fail(w, 413, "image too large")
return
}
b, err := io.ReadAll(io.LimitReader(f, maxImageBytes+1))
if err != nil {
fail(w, 500, "image read failed")
return
}
if len(b) > maxImageBytes {
fail(w, 413, "image too large")
return
}
after, err := f.Stat()
if err != nil || int64(len(b)) != st.Size() || st.Size() != after.Size() || !st.ModTime().Equal(after.ModTime()) || st.Sys().(*syscall.Stat_t).Ctim != after.Sys().(*syscall.Stat_t).Ctim {
fail(w, 409, "image changed; reload")
return
}
mimeType := workspaceImageType(p, b)
if mimeType == "" {
fail(w, 415, "unsupported image")
return
}
w.Header().Set("Content-Type", mimeType)
w.Header().Set("X-Content-Type-Options", "nosniff")
w.Header().Set("Content-Security-Policy", "sandbox; default-src 'none'; style-src 'unsafe-inline'")
w.Header().Set("Content-Disposition", mime.FormatMediaType("attachment", map[string]string{"filename": path.Base(p)}))
w.Header().Set("Cache-Control", "no-store")
w.Header().Set("Content-Length", strconv.Itoa(len(b)))
w.Write(b)
}
func workspaceImageType(name string, b []byte) string {
switch mimeType := http.DetectContentType(b); mimeType {
case "image/png", "image/jpeg", "image/gif", "image/webp":
return mimeType
}
if !strings.EqualFold(path.Ext(name), ".svg") {
return ""
}
d := xml.NewDecoder(bytes.NewReader(bytes.TrimPrefix(b, []byte{0xef, 0xbb, 0xbf})))
depth, roots := 0, 0
for {
token, err := d.Token()
if err == io.EOF {
if roots == 1 && depth == 0 {
return "image/svg+xml"
}
return ""
}
if err != nil {
return ""
}
switch t := token.(type) {
case xml.StartElement:
if depth == 0 {
roots++
if roots != 1 || t.Name.Local != "svg" || t.Name.Space != "http://www.w3.org/2000/svg" {
return ""
}
}
depth++
case xml.EndElement:
depth--
case xml.CharData:
if depth == 0 && len(bytes.TrimSpace(t)) != 0 {
return ""
}
case xml.Directive:
return ""
}
}
}
+129
View File
@@ -0,0 +1,129 @@
package api
import (
"bytes"
"mime"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strconv"
"syscall"
"testing"
"git.misaka.ren/m1saka/dsh-web/internal/store"
)
func TestWorkspaceImage(t *testing.T) {
a, sid := runtimeTestAPI(t, func(http.ResponseWriter, *http.Request) { t.Error("unexpected Docker call") })
for _, i := range []store.Instance{{ID: "one", UID: "owner"}, {ID: "foreign", UID: "other"}} {
if err := a.s.AddInstance(i); err != nil {
t.Fatal(err)
}
}
dir := filepath.Join(a.root, "workspaces/owner/instances/one/home")
if err := os.MkdirAll(dir, 0750); err != nil {
t.Fatal(err)
}
put := func(name string, b []byte) {
t.Helper()
if err := os.WriteFile(filepath.Join(dir, name), b, 0600); err != nil {
t.Fatal(err)
}
}
request := func(method, query, session string, want int) *httptest.ResponseRecorder {
t.Helper()
w := httptest.NewRecorder()
a.workspace(w, runtimeRequest(method, "/api/ws/image?"+query, "", session))
if w.Code != want {
t.Fatalf("%s %s: got %d want %d: %s", method, query, w.Code, want, w.Body.String())
}
if len(a.workspaceReaders) != 0 {
t.Fatal("workspace lease leaked")
}
return w
}
get := func(name string, want int) *httptest.ResponseRecorder {
t.Helper()
return request("GET", "instance=one&path="+url.QueryEscape(name), sid, want)
}
request("GET", "instance=one&path=file", "", 401)
request("GET", "instance=foreign&path=file", sid, 404)
request("GET", "instance=missing&path=file", sid, 404)
request("GET", "path=file", sid, 410)
for _, method := range []string{"HEAD", "POST", "PUT", "DELETE", "PATCH"} {
request(method, "instance=one&path=file", sid, 405)
}
for _, p := range []string{"", ".", "..", "../file", "/etc/passwd", "a/../file", "./file", "a//file", "file/", "a/./file", "a\x00b"} {
get(p, 400)
}
get("missing", 404)
for _, tc := range []struct{ name, content, mime string }{
{"png.txt", "\x89PNG\r\n\x1a\n", "image/png"},
{"jpeg", "\xff\xd8\xff\xe0", "image/jpeg"},
{"gif87", "GIF87a", "image/gif"},
{"gif89", "GIF89a", "image/gif"},
{"webp", "RIFF\x00\x00\x00\x00WEBPVP8 ", "image/webp"},
{"active.svg", `<svg xmlns="http://www.w3.org/2000/svg" onload="alert(1)"><style>rect{fill:red}</style><script>alert(1)</script><foreignObject><html xmlns="http://www.w3.org/1999/xhtml"/></foreignObject></svg>`, "image/svg+xml"},
{"prefix.SVG", `<?xml version="1.0"?><!--ok--><s:svg xmlns:s="http://www.w3.org/2000/svg"/>`, "image/svg+xml"},
{" space ", "GIF89a exact", "image/gif"},
{`a\b`, "GIF89a backslash", "image/gif"},
} {
t.Run(tc.name, func(t *testing.T) {
put(tc.name, []byte(tc.content))
w := get(tc.name, 200)
if w.Body.String() != tc.content {
t.Fatal("raw image bytes changed")
}
disposition, params, err := mime.ParseMediaType(w.Header().Get("Content-Disposition"))
if err != nil || disposition != "attachment" || params["filename"] != tc.name {
t.Fatalf("invalid download filename: %s", w.Header().Get("Content-Disposition"))
}
for key, want := range map[string]string{"Content-Type": tc.mime, "Content-Length": strconv.Itoa(len(tc.content)), "Cache-Control": "no-store", "X-Content-Type-Options": "nosniff", "Content-Security-Policy": "sandbox; default-src 'none'; style-src 'unsafe-inline'"} {
if got := w.Header().Get(key); got != want {
t.Errorf("%s = %q, want %q", key, got, want)
}
}
})
}
for _, content := range []string{"", "plain text", "<html><svg/></html>", `<svg/>`, `<svg xmlns="wrong"/>`, `<svg xmlns="http://www.w3.org/2000/svg">`, `<svg xmlns="http://www.w3.org/2000/svg"/><html/>`, `<svg xmlns="http://www.w3.org/2000/svg"/>junk`, `<!DOCTYPE svg [<!ENTITY x SYSTEM "file:///etc/passwd">]><svg xmlns="http://www.w3.org/2000/svg">&x;</svg>`, `<svg xmlns="http://www.w3.org/2000/svg">&unknown;</svg>`} {
put("bad.svg", []byte(content))
get("bad.svg", 415)
}
put("wrong.png", []byte(`<svg xmlns="http://www.w3.org/2000/svg"/>`))
get("wrong.png", 415)
put("fake.png", []byte("<html>not an image</html>"))
get("fake.png", 415)
if err := os.Mkdir(filepath.Join(dir, "directory"), 0700); err != nil {
t.Fatal(err)
}
get("directory", 415)
if err := os.Symlink("png.txt", filepath.Join(dir, "link")); err != nil {
t.Fatal(err)
}
get("link", 404)
if err := os.Symlink(dir, filepath.Join(dir, "parent")); err != nil {
t.Fatal(err)
}
get("parent/png.txt", 404)
if err := os.Link(filepath.Join(dir, "png.txt"), filepath.Join(dir, "hard")); err != nil {
t.Fatal(err)
}
get("hard", 404)
get("png.txt", 404)
if err := syscall.Mkfifo(filepath.Join(dir, "fifo"), 0600); err != nil {
t.Fatal(err)
}
get("fifo", 404)
large := bytes.Repeat([]byte{0}, maxImageBytes)
copy(large, "GIF89a")
put("limit", large)
get("limit", 200)
put("oversize", append(large, 0))
get("oversize", 413)
if _, err := a.s.DB.Exec("UPDATE instances SET status='error' WHERE id='one'"); err != nil {
t.Fatal(err)
}
get("gif89", 200)
}
+4 -1
View File
@@ -189,12 +189,15 @@ func (c *Client) Logs(ctx context.Context, id string) (string, error) {
return output.String(), nil return output.String(), nil
} }
func (c *Client) Build(ctx context.Context, tar io.Reader, version, tag string, onLog func(string)) (string, error) { func (c *Client) BuildRelease(ctx context.Context, tar io.Reader, version, tag, releaseID, repository, commit string, onLog func(string)) (string, error) {
labels := map[string]string{"dsh-mgr.release": releaseID, "dsh-mgr.source": repository, "dsh-mgr.commit": commit}
args, err := json.Marshal(map[string]string{"DSH_VERSION": version}) args, err := json.Marshal(map[string]string{"DSH_VERSION": version})
if err != nil { if err != nil {
return "", err return "", err
} }
query := url.Values{"t": {tag}, "buildargs": {string(args)}, "rm": {"1"}} query := url.Values{"t": {tag}, "buildargs": {string(args)}, "rm": {"1"}}
data, _ := json.Marshal(labels)
query.Set("labels", string(data))
resp, err := c.Stream(ctx, http.MethodPost, "/build?"+query.Encode(), tar, "application/x-tar") resp, err := c.Stream(ctx, http.MethodPost, "/build?"+query.Encode(), tar, "application/x-tar")
if err != nil { if err != nil {
return "", err return "", err
+3 -3
View File
@@ -201,7 +201,7 @@ func TestBuild(t *testing.T) {
} }
}) })
var logs []string var logs []string
id, err := client.Build(context.Background(), strings.NewReader("caller tar bytes"), "1.2.3", "test:1", func(s string) { logs = append(logs, s) }) id, err := client.BuildRelease(context.Background(), strings.NewReader("caller tar bytes"), "1.2.3", "test:1", "release", "repository", "commit", func(s string) { logs = append(logs, s) })
if err != nil || id != "sha256:image" || len(logs) != 1 || logs[0] != "step 1\n" { if err != nil || id != "sha256:image" || len(logs) != 1 || logs[0] != "step 1\n" {
t.Fatalf("id=%q logs=%v err=%v", id, logs, err) t.Fatalf("id=%q logs=%v err=%v", id, logs, err)
} }
@@ -219,7 +219,7 @@ func TestBuildErrors(t *testing.T) {
} { } {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
client := testClient(t, func(w http.ResponseWriter, r *http.Request) { io.WriteString(w, tc.data) }) client := testClient(t, func(w http.ResponseWriter, r *http.Request) { io.WriteString(w, tc.data) })
id, err := client.Build(context.Background(), nil, "1", "test", nil) id, err := client.BuildRelease(context.Background(), nil, "1", "test", "release", "repository", "commit", nil)
if err == nil || !strings.Contains(err.Error(), tc.want) || id != "" { if err == nil || !strings.Contains(err.Error(), tc.want) || id != "" {
t.Fatalf("id=%q err=%v", id, err) t.Fatalf("id=%q err=%v", id, err)
} }
@@ -235,7 +235,7 @@ func TestBuildCancellation(t *testing.T) {
}) })
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel() defer cancel()
_, err := client.Build(ctx, nil, "1", "test", func(string) { cancel() }) _, err := client.BuildRelease(ctx, nil, "1", "test", "release", "repository", "commit", func(string) { cancel() })
if !errors.Is(err, context.Canceled) { if !errors.Is(err, context.Canceled) {
t.Fatalf("err=%v", err) t.Fatalf("err=%v", err)
} }
+1 -95
View File
@@ -7,29 +7,10 @@ import (
) )
var ErrInstanceLimit = errors.New("instance limit reached") var ErrInstanceLimit = errors.New("instance limit reached")
var ErrIntentState = errors.New("intent is no longer waiting")
type Intent struct {
ID string `json:"id"`
UID string `json:"uid"`
Name string `json:"name"`
Version string `json:"version"`
BuildID string `json:"build_id"`
InstanceID string `json:"instance_id"`
Status string `json:"status"`
Error string `json:"error"`
Host string `json:"-"`
Created int64 `json:"created_at"`
}
func (s *Store) EnsureIntents() error {
_, err := s.DB.Exec(`CREATE TABLE IF NOT EXISTS creation_intents(id TEXT PRIMARY KEY,uid TEXT NOT NULL,name TEXT NOT NULL,version TEXT NOT NULL,build_id TEXT NOT NULL DEFAULT '',instance_id TEXT NOT NULL DEFAULT '',status TEXT NOT NULL,error TEXT NOT NULL DEFAULT '',host TEXT NOT NULL,created INTEGER NOT NULL); CREATE INDEX IF NOT EXISTS intents_owner ON creation_intents(uid,status);`)
return err
}
func reserveSlot(tx *sql.Tx, uid string) error { func reserveSlot(tx *sql.Tx, uid string) error {
var count int var count int
if err := tx.QueryRow(`SELECT (SELECT count(*) FROM instances WHERE uid=?) + (SELECT count(*) FROM creation_intents WHERE uid=? AND status='waiting')`, uid, uid).Scan(&count); err != nil { if err := tx.QueryRow(`SELECT count(*) FROM instances WHERE uid=?`, uid).Scan(&count); err != nil {
return err return err
} }
if count >= 2 { if count >= 2 {
@@ -38,22 +19,6 @@ func reserveSlot(tx *sql.Tx, uid string) error {
return nil return nil
} }
func (s *Store) ReserveIntent(i Intent) error {
tx, err := s.DB.Begin()
if err != nil {
return err
}
defer tx.Rollback()
if err = reserveSlot(tx, i.UID); err != nil {
return err
}
_, err = tx.Exec(`INSERT INTO creation_intents(id,uid,name,version,status,host,created) VALUES(?,?,?,?,'waiting',?,?)`, i.ID, i.UID, i.Name, i.Version, i.Host, time.Now().Unix())
if err != nil {
return err
}
return tx.Commit()
}
func (s *Store) ReserveInstance(i Instance) error { func (s *Store) ReserveInstance(i Instance) error {
tx, err := s.DB.Begin() tx, err := s.DB.Begin()
if err != nil { if err != nil {
@@ -72,62 +37,3 @@ func insertInstance(tx *sql.Tx, i Instance) error {
_, err := tx.Exec(`INSERT INTO instances VALUES(?,?,?,?,?,?,?,?)`, i.ID, i.UID, i.Name, i.Version, i.ImageID, i.Status, i.ContainerID, time.Now().Unix()) _, err := tx.Exec(`INSERT INTO instances VALUES(?,?,?,?,?,?,?,?)`, i.ID, i.UID, i.Name, i.Version, i.ImageID, i.Status, i.ContainerID, time.Now().Unix())
return err return err
} }
func (s *Store) ClaimIntent(id string, i Instance) error {
tx, err := s.DB.Begin()
if err != nil {
return err
}
defer tx.Rollback()
result, err := tx.Exec(`UPDATE creation_intents SET status='creating',instance_id=? WHERE id=? AND uid=? AND version=? AND status='waiting'`, i.ID, id, i.UID, i.Version)
if err != nil {
return err
}
n, err := result.RowsAffected()
if err != nil {
return err
}
if n != 1 {
return ErrIntentState
}
if err = insertInstance(tx, i); err != nil {
return err
}
return tx.Commit()
}
func (s *Store) CancelIntent(id, uid string) error {
r, err := s.DB.Exec(`UPDATE creation_intents SET status='cancelled' WHERE id=? AND uid=? AND status='waiting'`, id, uid)
if err != nil {
return err
}
n, err := r.RowsAffected()
if err != nil {
return err
}
if n != 1 {
return ErrIntentState
}
return nil
}
func (s *Store) Intents(uid string) ([]Intent, error) {
query := `SELECT id,uid,name,version,build_id,instance_id,status,error,host,created FROM creation_intents`
var args []any
if uid != "" {
query += ` WHERE uid=?`
args = append(args, uid)
}
query += ` ORDER BY created,id`
rows, err := s.DB.Query(query, args...)
if err != nil {
return nil, err
}
defer rows.Close()
out := []Intent{}
for rows.Next() {
var i Intent
if err = rows.Scan(&i.ID, &i.UID, &i.Name, &i.Version, &i.BuildID, &i.InstanceID, &i.Status, &i.Error, &i.Host, &i.Created); err != nil {
return nil, err
}
out = append(out, i)
}
return out, rows.Err()
}
+11 -39
View File
@@ -7,26 +7,19 @@ import (
"testing" "testing"
) )
func TestIntentReservationConcurrencyAndRestart(t *testing.T) { func TestInstanceReservationConcurrencyAndRestart(t *testing.T) {
path := filepath.Join(t.TempDir(), "store.db") path := filepath.Join(t.TempDir(), "store.db")
s, err := Open(path) s, err := Open(path)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
if err = s.EnsureIntents(); err != nil {
t.Fatal(err)
}
var wg sync.WaitGroup var wg sync.WaitGroup
result := make(chan error, 30) result := make(chan error, 30)
for n := 0; n < 30; n++ { for n := 0; n < 30; n++ {
wg.Add(1) wg.Add(1)
go func(n int) { go func(n int) {
defer wg.Done() defer wg.Done()
if n%2 == 0 { result <- s.ReserveInstance(Instance{ID: ID(), UID: "u"})
result <- s.ReserveIntent(Intent{ID: ID(), UID: "u", Name: "test", Version: "1.2.3"})
} else {
result <- s.ReserveInstance(Instance{ID: ID(), UID: "u"})
}
}(n) }(n)
} }
wg.Wait() wg.Wait()
@@ -48,47 +41,26 @@ func TestIntentReservationConcurrencyAndRestart(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
defer s.Close() defer s.Close()
if err = s.ReserveIntent(Intent{ID: ID(), UID: "u"}); !errors.Is(err, ErrInstanceLimit) { if err = s.ReserveInstance(Instance{ID: ID(), UID: "u"}); !errors.Is(err, ErrInstanceLimit) {
t.Fatalf("restart lost reservation: %v", err) t.Fatalf("restart lost reservation: %v", err)
} }
} }
func TestIntentClaimCancelAndVersion(t *testing.T) {
func TestRetiredIntentsDoNotReserveInstanceSlots(t *testing.T) {
s, err := Open(filepath.Join(t.TempDir(), "store.db")) s, err := Open(filepath.Join(t.TempDir(), "store.db"))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
defer s.Close() defer s.Close()
if err = s.EnsureIntents(); err != nil { if _, err := s.DB.Exec("CREATE TABLE creation_intents(uid TEXT,status TEXT); INSERT INTO creation_intents VALUES('u','waiting'),('u','creating')"); err != nil {
t.Fatal(err) t.Fatal(err)
} }
in := Intent{ID: ID(), UID: "u", Version: "1.2.3"} for n := 0; n < 2; n++ {
if err = s.ReserveIntent(in); err != nil { if err := s.ReserveInstance(Instance{ID: ID(), UID: "u"}); err != nil {
t.Fatal(err) t.Fatal(err)
}
} }
i := Instance{ID: ID(), UID: "u", Version: "1.2.4"} if err := s.ReserveInstance(Instance{ID: ID(), UID: "u"}); !errors.Is(err, ErrInstanceLimit) {
if err = s.ClaimIntent(in.ID, i); !errors.Is(err, ErrIntentState) {
t.Fatal(err)
}
if err = s.CancelIntent(in.ID, "other"); !errors.Is(err, ErrIntentState) {
t.Fatal(err)
}
i.Version = in.Version
if err = s.ClaimIntent(in.ID, i); err != nil {
t.Fatal(err)
}
if err = s.CancelIntent(in.ID, in.UID); !errors.Is(err, ErrIntentState) {
t.Fatal(err)
}
if err = s.ReserveIntent(Intent{ID: "second", UID: "u"}); err != nil {
t.Fatal(err)
}
if err = s.ReserveIntent(Intent{ID: "third", UID: "u"}); !errors.Is(err, ErrInstanceLimit) {
t.Fatal(err)
}
if err = s.CancelIntent("second", "u"); err != nil {
t.Fatal(err)
}
if err = s.ReserveIntent(Intent{ID: "third", UID: "u"}); err != nil {
t.Fatal(err) t.Fatal(err)
} }
} }
+42
View File
@@ -0,0 +1,42 @@
package store
import (
"path/filepath"
"testing"
"time"
)
func TestSessionTTL(t *testing.T) {
for _, value := range []string{"", "24h", "168h", "60s"} {
t.Run(value, func(t *testing.T) {
t.Setenv("SESSION_TTL", value)
s, err := Open(filepath.Join(t.TempDir(), "db"))
if err != nil {
t.Fatal(err)
}
defer s.Close()
before := time.Now().Unix()
sid, _, err := s.Session("user")
if err != nil {
t.Fatal(err)
}
var expiry int64
if err = s.DB.QueryRow("SELECT expires FROM sessions WHERE id=?", sid).Scan(&expiry); err != nil {
t.Fatal(err)
}
ttl, _ := sessionLifetime(value)
if expiry < before+int64(ttl.Seconds()) || expiry > time.Now().Unix()+int64(ttl.Seconds()) {
t.Fatal(expiry)
}
})
}
for _, value := range []string{"0", "-1h", "7d", "bad", "500ms", "1.5s"} {
t.Run(value, func(t *testing.T) {
t.Setenv("SESSION_TTL", value)
if s, err := Open(filepath.Join(t.TempDir(), "db")); err == nil {
s.Close()
t.Fatal("invalid TTL accepted")
}
})
}
}
+24 -6
View File
@@ -4,7 +4,9 @@ import (
"crypto/rand" "crypto/rand"
"database/sql" "database/sql"
"encoding/hex" "encoding/hex"
"fmt"
_ "modernc.org/sqlite" _ "modernc.org/sqlite"
"os"
"sync" "sync"
"time" "time"
) )
@@ -27,12 +29,17 @@ type Instance struct {
Created time.Time `json:"created_at"` Created time.Time `json:"created_at"`
} }
type Store struct { type Store struct {
DB *sql.DB DB *sql.DB
closed chan struct{} sessionTTL time.Duration
closeOnce sync.Once closed chan struct{}
closeOnce sync.Once
} }
func Open(path string) (*Store, error) { func Open(path string) (*Store, error) {
ttl, err := sessionLifetime(os.Getenv("SESSION_TTL"))
if err != nil {
return nil, err
}
db, e := sql.Open("sqlite", path) db, e := sql.Open("sqlite", path)
if e != nil { if e != nil {
return nil, e return nil, e
@@ -47,7 +54,18 @@ func Open(path string) (*Store, error) {
db.Close() db.Close()
return nil, e return nil, e
} }
return &Store{DB: db, closed: make(chan struct{})}, nil return &Store{DB: db, sessionTTL: ttl, closed: make(chan struct{})}, nil
}
func sessionLifetime(value string) (time.Duration, error) {
if value == "" {
return 24 * time.Hour, nil
}
ttl, err := time.ParseDuration(value)
if err != nil || ttl < time.Second || ttl%time.Second != 0 {
return 0, fmt.Errorf("SESSION_TTL must be a positive whole-second duration, e.g. 24h or 168h")
}
return ttl, nil
} }
func (s *Store) Close() { s.closeOnce.Do(func() { close(s.closed); s.DB.Close() }) } func (s *Store) Close() { s.closeOnce.Do(func() { close(s.closed); s.DB.Close() }) }
func (s *Store) Done() <-chan struct{} { return s.closed } func (s *Store) Done() <-chan struct{} { return s.closed }
@@ -85,14 +103,14 @@ func (s *Store) Password(name string) (string, error) {
} }
func (s *Store) Session(uid string) (string, string, error) { func (s *Store) Session(uid string) (string, string, error) {
id, csrf := ID(), ID() id, csrf := ID(), ID()
_, e := s.DB.Exec("INSERT INTO sessions VALUES(?,?,?,?)", id, uid, csrf, time.Now().Add(24*time.Hour).Unix()) _, e := s.DB.Exec("INSERT INTO sessions VALUES(?,?,?,?)", id, uid, csrf, time.Now().Add(s.sessionTTL).Unix())
return id, csrf, e return id, csrf, e
} }
func (s *Store) SessionUser(id string) (User, string, error) { func (s *Store) SessionUser(id string) (User, string, error) {
var uid, csrf string var uid, csrf string
var exp int64 var exp int64
e := s.DB.QueryRow("SELECT uid,csrf,expires FROM sessions WHERE id=?", id).Scan(&uid, &csrf, &exp) e := s.DB.QueryRow("SELECT uid,csrf,expires FROM sessions WHERE id=?", id).Scan(&uid, &csrf, &exp)
if e != nil || exp < time.Now().Unix() { if e != nil || exp <= time.Now().Unix() {
return User{}, "", sql.ErrNoRows return User{}, "", sql.ErrNoRows
} }
u, e := s.UserByID(uid) u, e := s.UserByID(uid)
-15
View File
@@ -1,15 +0,0 @@
#!/bin/bash
set -Eeuo pipefail
project_dir=$(CDPATH= cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)
version=${1:-}
if [[ -z $version ]]; then
version=$(npm view @deepseek-ai/dsh@latest version --registry=https://registry.npmjs.org)
fi
if [[ ! $version =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]]; then
echo 'Expected an exact DSH version, not a dist-tag or arbitrary build argument.' >&2
exit 2
fi
tag="dsh-web:${version}-7"
printf 'Building local image %s (no push)\n' "$tag"
docker build --build-arg "DSH_VERSION=$version" --tag "$tag" "$project_dir/image"
docker image inspect --format '{{.Id}}' "$tag"
-1
View File
@@ -23,7 +23,6 @@ run go vet ./...
run go build ./... run go build ./...
run npm --prefix web test run npm --prefix web test
run npm --prefix web run build run npm --prefix web run build
run node --test image/init-profile.test.mjs
run node --check scripts/test-panel.mjs run node --check scripts/test-panel.mjs
printf '\nAll acceptance checks passed. 所有测试与检查均通过。\n' printf '\nAll acceptance checks passed. 所有测试与检查均通过。\n'
-1
View File
@@ -12,7 +12,6 @@ const commands = [
'go build ./...', 'go build ./...',
'npm --prefix web test', 'npm --prefix web test',
'npm --prefix web run build', 'npm --prefix web run build',
'node --test image/init-profile.test.mjs',
'node --check scripts/test-panel.mjs', 'node --check scripts/test-panel.mjs',
]; ];
-33
View File
@@ -1,33 +0,0 @@
import assert from 'node:assert/strict';
import {readFileSync} from 'node:fs';
const base=process.env.TEST_PANEL_URL;
assert.ok(base && process.env.TEST_PASSWORD_FILE);
let csrf='',cookie='';
async function req(path,method='GET',body){
const r=await fetch(base+path,{method,headers:{Cookie:cookie,Origin:base,'Content-Type':'application/json','X-CSRF-Token':csrf},body:body===undefined?undefined:JSON.stringify(body)});
if(path==='/api/login')cookie=r.headers.getSetCookie().map(v=>v.split(';')[0]).join('; ');
return {status:r.status,data:await r.json()};
}
const login=await req('/api/login','POST',{username:'admin',password:readFileSync(process.env.TEST_PASSWORD_FILE,'utf8').trim()});
assert.equal(login.status,200);csrf=login.data.csrf_token;
const before=(await req('/api/instances')).data;assert.equal(before.length,1,'run with exactly one retained test instance');
const intent=await req('/api/intents','POST',{name:'Design real intent',version:'0.2.0-rc.2'});assert.equal(intent.status,202);
let created;
try {
for(let n=0;n<60;n++){
const item=(await req('/api/intents')).data.find(x=>x.id===intent.data.id);
assert.notEqual(item.status,'failed');
if(item.instance_id){created=item.instance_id;break;}
await new Promise(r=>setTimeout(r,1000));
}
assert.ok(created,'intent must create a real instance');
const list=(await req('/api/instances')).data;assert.equal(list.length,2);assert.ok(list.every(i=>i.status==='running'));
const extra=await req('/api/instances','POST',{name:'must reject third',image_id:list[0].image_id});assert.equal(extra.status,409);
assert.equal((await req('/api/intents','POST',{name:'must reject extra reservation',version:'0.2.0-rc.2'})).status,409);
console.log('PASS persisted intent -> real runtime; two running instances; third create and reservation rejected');
} finally {
if(created)assert.equal((await req('/api/instances/'+created,'DELETE')).status,200);
else await req('/api/intents/'+intent.data.id,'DELETE');
}
assert.equal((await req('/api/instances')).data.length,1);
console.log('PASS deleted only this test runtime, retained data and prior instance');
+5 -8
View File
@@ -23,17 +23,14 @@ try {
assert.equal((await request('/api/me')).data.user.username,process.env.TEST_USERNAME || 'testadmin'); assert.equal((await request('/api/me')).data.user.username,process.env.TEST_USERNAME || 'testadmin');
await request('/api/refresh','POST'); await request('/api/refresh','POST');
assert.equal((await request('/api/me')).data.user.username,process.env.TEST_USERNAME || 'testadmin'); assert.equal((await request('/api/me')).data.user.username,process.env.TEST_USERNAME || 'testadmin');
let images=(await request('/api/images')).data; const images=(await request('/api/images')).data.filter(i=>i.status==='available' && i.selectable===true);
const build=(await request('/api/builds','POST',{version:process.env.TEST_FROM_VERSION || images[0]?.version || '0.2.0-rc.2'})).data; assert.ok(images.length,'prepare a validated, admitted image through the admin Git release flow before running this test');
let buildSucceeded=false;
for(let n=0;n<120;n++){let job;try{job=(await request('/api/builds/'+encodeURIComponent(build.id))).data;}catch(e){if(!(e instanceof Error))throw e;await new Promise(r=>setTimeout(r,1000));continue;}if(job?.status==='failed')throw new Error('build/validation failed: '+job.error);if(job?.status==='success'){buildSucceeded=true;break;}await new Promise(r=>setTimeout(r,1000));}
assert.ok(buildSucceeded,'submitted build/validation job did not complete before timeout');
images=(await request('/api/images')).data.filter(i=>i.status==='available');assert.ok(images.length,'validated image required');
const source=images.find(i=>!process.env.TEST_FROM_VERSION || i.version===process.env.TEST_FROM_VERSION); const source=images.find(i=>!process.env.TEST_FROM_VERSION || i.version===process.env.TEST_FROM_VERSION);
const target=images.find(i=>!process.env.TEST_TO_VERSION || i.version===process.env.TEST_TO_VERSION); const target=images.find(i=>!process.env.TEST_TO_VERSION || i.version===process.env.TEST_TO_VERSION);
assert.ok(source,'source image required');assert.ok(target,'target image required'); assert.ok(source,'source image required');assert.ok(target,'target image required');
console.log('PASS local image reuse, persistent build task and runtime verification'); console.log('PASS validated and admitted local image selection');
instance=(await request('/api/instances','POST',{name:'Integration check',image_id:source.image_id || source.id,version:source.version})).data; instance=(await request('/api/instances','POST',{name:'Integration check',image_id:source.image_id || source.id})).data;
console.log('Disposable instance:', instance.id);
await request('/api/ws/mkdir','POST',{path:testDirectory}); await request('/api/ws/mkdir','POST',{path:testDirectory});
const form=new FormData();form.set('path',testDirectory);form.set('file',new Blob(['hello workspace']), 'hello.txt');await request('/api/ws/upload','POST',form); const form=new FormData();form.set('path',testDirectory);form.set('file',new Blob(['hello workspace']), 'hello.txt');await request('/api/ws/upload','POST',form);
assert.equal((await request('/api/ws/download?path='+testDirectory+'/hello.txt')).data,'hello workspace'); assert.equal((await request('/api/ws/download?path='+testDirectory+'/hello.txt')).data,'hello workspace');
+1
View File
@@ -1,4 +1,5 @@
node_modules/ node_modules/
.vite/
dist/ dist/
*.tsbuildinfo *.tsbuildinfo
.playwright-cli/ .playwright-cli/
+27 -6
View File
@@ -1,11 +1,32 @@
# 前后端协调 # 前后端接口对接
前端 agent 当前工具没有 actor/send,无法直接发送给 general-2。请后端在根目录 API.md 记录实际接口、鉴权/CSRF、列表 envelope、实例/profile/备份/升级、文件上传和管理员字段。前端将以 API.md 校对实现,不模拟成功。 根目录 API.md 是接口契约。公共请求封装与响应类型在 src/api.ts,实例、备份及管理员等组件也通过 api.request 发起业务请求。本文记录当前前端依赖,不将界面入口或请求成功等同于端到端验收。
Vite 开发监听 127.0.0.1,代理默认 http://127.0.0.1:8088,生产构建 web/dist,同源 /api 和 /dsh。 ## 运行与认证
已完成中文页面与首次构建,当前待确认的响应字段:images 是 id 还是 image_id;builds 日志字段 logs;ws/tree 是否直接数组及 is_dir/mod_time;usage used_bytes/quota_bytes;audit user_id/created_at;backups id/size/created_at。请明确空列表返回 [] 或 null。恢复/下载备份/升级接口尚未列出,UI 将明确提示缺口而不猜测接口。 Vite 开发监听 127.0.0.1,默认端口 5173(占用时自动尝试后续端口),代理至 http://127.0.0.1:8088;/dsh 启用 WebSocket 代理。生产构建目录为 web/dist,需由后端同源提供静态页面、/api 和 /dsh。
11:24 检查 internal/api/api.go:创建实例入参目前 struct{ Name, ImageID, Version string } 无 json 标签,image_id 不会映射到 ImageID,需后端补标签。生命周期仍只是写 start/stop 字符串而非真实 Docker 操作,前端不会把 start 字符串映射成 running。请务必真实实现或返回明确错误,避免假成功。 - 登录和 /api/me 返回 {user,csrf_token}。认证使用 HttpOnly Cookie;浏览器请求携带同源 Cookie,非 GET 请求从可读的 dsh_csrf Cookie 设置 X-CSRF-Token,而不是只使用内存中的旧 token。
- 存在 CSRF token 时,非登录/刷新请求的首次 401 会尝试恢复会话,再重试原请求;使用页内共享 Promise 和可用时的 Web Locks 协调刷新,刷新冲突后复验 /api/me。无法恢复的会话通过 session-expired 通知界面。
- 错误响应读取 {error:string};网络错误和非 JSON 成功响应均显示明确错误,不模拟成功。文件/备份下载和图片预览直接使用同源资源地址,不经过 JSON 请求封装。
验证:npm test 六项全部通过;npm run build 通过。Playwright 真浏览器已验证桌面/390px 移动登录界面、离线登录错误,移动 documentWidth=390 无横向溢出。8088 暂未启动,无法验证登录后页面;请父 agent 启动后端后通知可用临时验证账号,避免操作共享数据。 ## 响应结构与镜像构建
- 实例、镜像、构建历史、工作区目录、用户、审计、备份及孤儿容器的 GET 列表响应为顶层数组,不使用 data envelope;空列表应返回 []。其中 /api/orphans 返回容器 ID 字符串数组;详情、用量和操作结果不适用此列表约定。
- 镜像类型含 id、可选 image_id、version、status、admission、admission_reason、selectable 等字段;前端提交镜像标识时使用 image_id || id,仅允许 status=available 且 selectable=true 的镜像用于创建/升级。
- 构建历史通过 GET /api/builds?limit=...&offset=... 获取摘要;前端每页显示 20 条、多取 1 条判断下一页,不读取 X-Has-More。日志按需通过 GET /api/builds/{id} 的 logs 字段获取。
- 管理员通过 GET /api/releases/latest 检查 Git 发布,确认后 POST /api/releases/build {commit};构建来源字段包括 release_id、source_commit、source_repository。镜像准入使用 POST /api/images/admission {image_id,state,evidence}。不使用旧的 /api/latest、/api/compatibility、POST /api/builds 或创建意图接口。
- 工作区目录项读取 name/path/is_dir/size/mod_time;用量读取 used_bytes/quota_bytes;审计读取 user_id/created_at 等字段;备份读取 id/size/created_at。
## 已接入的业务操作
- 创建实例:POST /api/instances {name,image_id},当前前端不发送 version。启动/停止/重启调用实例对应子路径;删除使用 DELETE /api/instances/{id}。显示后端返回的实例状态,不把操作名称当作运行状态。
- Profile:读取 /profile 和 /profile-preview;停止实例后 PUT /profile {content,restart:true} 保存并启动验证。日志读取 /logs 的 logs 字段。上述路径均位于 /api/instances/{id} 下。
- 备份:GET /api/instances/{id}/backups 列表,POST 同路径 JSON {password} 创建,multipart file,password 导入;GET 同路径 ?download={backup_id} 下载;POST /api/instances/{id}/restore {backup_id,password} 恢复。
- 升级:POST /api/instances/{id}/upgrade {image_id,password}。创建备份和升级使用当前登录密码;导入/恢复使用原包密码,旧平台 key 包可留空。恢复和升级有二次确认。
- 工作区:/api/ws/tree、usage、mkdir、rename、delete、upload、download,并通过 instance 参数选择实例;上传使用 multipart file,path。图片预览使用 /api/ws/image;文本读取 /api/ws/text 返回 {content,version},保存 PUT 同路径 {content,version},冲突不静默覆盖。
- 账户管理:本人 POST /api/me/password;管理员通过 /api/users 创建用户、PATCH 修改配额,通过 /api/users/reset-password 重设密码,并读取 /api/audit。本人改密后重新登录,管理员重设按 reauthenticate 决定是否退出当前会话。孤儿容器回收使用 /api/orphans,删除提交 id 与 confirm。
## 验证边界
在 web/ 执行 npm test 和 npm run build 获取当前测试与构建结果,不沿用早期测试数量或临时服务状态。单元测试包含 fetch 替身,不能证明真实后端、Docker、WS/SSE 或数据恢复流程已验收;真实联调需独立测试环境和授权账号,具体后端限制与验收记录见根目录 API.md、README.md、ACCEPTANCE.md。
+9 -7
View File
@@ -1,10 +1,10 @@
# DSH 中文工作台 # DSH 中文工作台
React 19 + TypeScript + Vite;普通 CSS 响应式布局,没有运行时演示数据。 React 19 + TypeScript + Vite;使用 Tailwind CSS 4、Radix UI 和自定义 CSS 构建响应式界面,Profile 编辑器按需加载 Monaco,没有运行时演示数据。
## 开发与验证 ## 开发与验证
使用 Node.js 22+: 使用 Node.js 22+,在 web/ 目录执行:
~~~bash ~~~bash
npm ci npm ci
@@ -13,14 +13,16 @@ npm test
npm run build npm run build
~~~ ~~~
开发服务只监听 127.0.0.1:5173,/api 和 /dsh 代理至 127.0.0.1:8088。生产产物为 dist/,由后端同源提供;不能直接通过 file:// 使用。 开发服务监听 127.0.0.1,默认端口 5173(占用时 Vite 会尝试后续端口);/api 和 /dsh 代理至 127.0.0.1:8088,/dsh 支持 WebSocket 代理。生产产物为 web/dist/,需配置后端静态目录并同源提供;不能直接通过 file:// 使用。
## API 与安全 ## API 与安全
- 根目录 API.md 是接口契约;API 客户端集中在 src/api.ts,可注入 fetch 进行独立单元测试。 - 根目录 API.md 是接口契约;公共请求封装和类型位于 src/api.ts,可注入 fetch 进行独立单元测试。业务组件也通过 api.request 调用接口;当前对接说明见 API_REQUEST.md。
- HttpOnly Cookie 登录;CSRF token 仅留在内存,刷新通过 /api/me 重新获取。密码、token 不写入 localStorage。 - 使用同源 HttpOnly 认证 Cookie;浏览器写请求从可读的 dsh_csrf Cookie 获取 X-CSRF-Token。启动时通过 /api/me 恢复登录状态,符合条件的 401 会触发 /api/refresh 并重试原请求,支持并发刷新协调。密码、token 不写入 localStorage;工作台所选实例会按用户保存在 localStorage。
- 操作失败保留错误,不模拟成功;删除、重启、构建等操作二次确认。 - 操作失败保留错误,不模拟成功;删除、重启、构建等操作二次确认。
- 备份恢复和升级等待后端契约,当前明确显示不可用说明,不发送猜测接口。 - 已接入 Profile 编辑/预览、加密备份创建/导入/下载/恢复和本地镜像升级;创建备份和升级要求当前登录密码,导入/恢复使用原备份密码。
- 创建实例和升级只允许 status=available 且 selectable=true 的本地镜像;管理员通过 Git 发布检查与精确 commit 提交构建,并可管理镜像准入策略。
- 工作区支持文件管理、目录 ZIP 下载、图片预览和带版本校验的文本编辑;管理员页面提供用户、配额、密码重设、审计及孤儿容器回收,本人可修改密码。
- 单元测试里的 fetch 替身仅用于测试,不打包进生产页面。 - 单元测试里的 fetch 替身仅用于测试,不打包进生产页面。
当前后端仍在实现部分端点,页面会真实显示其 HTTP 错误。请不要把前端入口存在等同于后端能力已交付。 测试使用 Vitest,覆盖 API、认证刷新、组件和构建分块等行为;其中构建测试使用不落盘的 Vite 构建。测试或构建通过不等于真实 Docker 生命周期、代理会话及备份恢复已完成端到端验收;部署限制和后端验收范围以根目录 README.md、API.md 和 ACCEPTANCE.md 为准。
+1467 -75
View File
File diff suppressed because it is too large Load Diff
+12 -1
View File
@@ -10,9 +10,20 @@
"test": "vitest run" "test": "vitest run"
}, },
"dependencies": { "dependencies": {
"@radix-ui/react-dialog": "^1.3.1",
"@radix-ui/react-dropdown-menu": "^2.2.1",
"@radix-ui/react-label": "^2.1.17",
"@radix-ui/react-slot": "^1.5.0",
"@radix-ui/react-tabs": "^1.2.1",
"@tailwindcss/vite": "^4.3.3",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"lucide-react": "^1.57.0",
"monaco-editor": "^0.52.2", "monaco-editor": "^0.52.2",
"react": "^19.0.0", "react": "^19.0.0",
"react-dom": "^19.0.0" "react-dom": "^19.0.0",
"tailwind-merge": "^3.7.0",
"tailwindcss": "^4.3.3"
}, },
"devDependencies": { "devDependencies": {
"@types/node": "^26.6.5", "@types/node": "^26.6.5",
+26
View File
@@ -0,0 +1,26 @@
import { expect, it, vi } from 'vitest';
vi.mock('react', async original => ({
...await original<typeof import('react')>(),
useState: (value: unknown) => [value, vi.fn()],
useRef: (value: unknown) => ({current: value}),
}));
vi.mock('./ui', async original => ({
...await original<typeof import('./ui')>(),
useResource: () => ({data: [{id: 'one', username: 'alice'}, {id: 'two', username: 'bob'}], loading: false, refresh: vi.fn()}),
}));
import { Admin } from './Admin';
import { ResetPassword } from './ResetPassword';
function nodes(root: any, predicate: (node: any) => boolean): any[] {
if (Array.isArray(root)) return root.flatMap(child => nodes(child, predicate));
if (!root || typeof root !== 'object') return [];
return [...(predicate(root) ? [root] : []), ...nodes(root.props?.children, predicate)];
}
it('mounts one reset-password owner per user in a shared responsive list', () => {
const accounts = (Admin().props.children as any[])[0];
const tree = accounts.type(accounts.props);
const editors = nodes(tree, n => n.type === ResetPassword);
expect(editors.map(n => n.props.user.id)).toEqual(['one', 'two']);
for (const branch of nodes(tree, n => typeof n.props?.className === 'string' && /md:hidden|md:block/.test(n.props.className))) {
expect(nodes(branch, n => n.type === ResetPassword)).toHaveLength(0);
}
});
+96 -5
View File
@@ -5,13 +5,104 @@ import { passwordResetError } from './passwordReset';
import { Orphans } from './Orphans'; import { Orphans } from './Orphans';
import { ResetPassword } from './ResetPassword'; import { ResetPassword } from './ResetPassword';
import { api, type User, type Audit } from './api'; import { api, type User, type Audit } from './api';
import { Panel, ResourceState, Empty, bytes, date, useAction, useResource } from './ui'; import { Panel, ResourceState, Empty, bytes, date, useAction, useResource, RefreshButton, tableClass, tableHeadRowClass, thClass, tdClass, trClass } from './ui';
import { AlertBox } from './components/ui/alert';
import { buttonVariants } from './components/ui/button';
import { inputClass } from './components/ui/input';
import { cn } from './lib/utils';
export function Admin() { export function Admin() {
return <><AdminAccounts/><Orphans/></>; return <><AdminAccounts /><Orphans /></>;
} }
function AdminAccounts() { function AdminAccounts() {
const users = useResource(() => api.request<User[]>('/users')), audit = useResource(() => api.request<Audit[]>('/audit')); const action = useAction(); const users = useResource(() => api.request<User[]>('/users')), audit = useResource(() => api.request<Audit[]>('/audit'));
const action = useAction();
const [quotaUser, setQuotaUser] = useState<User | null>(null), [notice, setNotice] = useState(''); const [quotaUser, setQuotaUser] = useState<User | null>(null), [notice, setNotice] = useState('');
function create(e: FormEvent<HTMLFormElement>) {e.preventDefault(); const element = e.currentTarget, form = new FormData(element); void action.run(async () => {const password = String(form.get('password') || ''); const error = passwordResetError(password, password); if (error) throw new Error(error); const quota = quotaBytes(String(form.get('quota'))); if (quota === null) throw new Error('请输入有效配额'); await api.request('/users','POST',{username:form.get('username'),password:form.get('password'),admin:form.get('admin') === 'on',quota_bytes:quota}); element.reset(); await Promise.all([users.refresh(),audit.refresh()]);}, '账号已创建');} function create(e: FormEvent<HTMLFormElement>) {
return <>{action.feedback}{notice && <div className="notice" role="status">{notice}</div>}{quotaUser && <QuotaDialog user={quotaUser} onClose={() => setQuotaUser(null)} onSaved={() => {setNotice(quotaUser.username + ' 的配额已更新'); void users.refresh(); void audit.refresh();}}/>}<Panel title="创建账号"><form className="user-form" onSubmit={create}><label>用户名(3–40 位字母、数字、_ 或 -)<input name="username" required minLength={3} maxLength={40} pattern="[A-Za-z0-9_\-]{3,40}" disabled={action.busy} autoComplete="off"/></label><label>初始密码(12–72 UTF-8 字节)<input name="password" type="password" disabled={action.busy} required autoComplete="new-password"/></label><label>工作区配额(GiB)<input name="quota" disabled={action.busy} type="number" min="0.1" step="0.1" required defaultValue="10"/></label><label className="checkbox"><input name="admin" disabled={action.busy} type="checkbox"/>管理员(拥有平台管理权限)</label><button className="primary" disabled={action.busy}>创建账号</button></form></Panel><Panel title="用户列表" action={<button onClick={users.refresh} disabled={users.loading}>刷新</button>}><ResourceState resource={users}/>{users.data?.length === 0 && <Empty>暂无用户</Empty>}{!!users.data?.length && <div className="table-scroll"><table className="accounts-table"><thead><tr><th>用户</th><th>角色</th><th>配额</th><th>创建时间</th><th>账号安全</th></tr></thead><tbody>{users.data.map(u => <tr key={u.id}><td data-label="用户"><strong>{u.username}</strong><small className="mono">{u.id}</small></td><td data-label="角色">{u.admin ? '管理员' : '成员'}</td><td data-label="配额"><div className="quota-cell"><span>{bytes(u.quota_bytes)}</span><button className="account-action" onClick={() => {setNotice(''); setQuotaUser(u);}}>设置配额</button></div></td><td data-label="创建时间">{date(u.created_at)}</td><td data-label="账号安全"><ResetPassword user={u} onReset={audit.refresh}/></td></tr>)}</tbody></table></div>}</Panel><Panel title="操作审计" action={<button onClick={audit.refresh} disabled={audit.loading}>刷新</button>}><ResourceState resource={audit}/>{audit.data?.length === 0 && <Empty>暂无审计记录</Empty>}{!!audit.data?.length && <div className="table-scroll"><table><thead><tr><th>时间</th><th>用户</th><th>操作</th><th>目标</th><th>详情</th></tr></thead><tbody>{audit.data.map(a => <tr key={a.id}><td>{date(a.created_at)}</td><td className="mono">{a.user_id}</td><td>{a.action}</td><td className="wrap">{a.target}</td><td className="wrap">{a.detail}</td></tr>)}</tbody></table></div>}</Panel></>; e.preventDefault();
const element = e.currentTarget, form = new FormData(element);
void action.run(async () => {
const password = String(form.get('password') || '');
const error = passwordResetError(password, password);
if (error) throw new Error(error);
const quota = quotaBytes(String(form.get('quota')));
if (quota === null) throw new Error('请输入有效配额');
await api.request('/users', 'POST', {username: form.get('username'), password: form.get('password'), admin: form.get('admin') === 'on', quota_bytes: quota});
element.reset();
await Promise.all([users.refresh(), audit.refresh()]);
}, '账号已创建');
}
return <>
{action.feedback}
{notice && <AlertBox variant="info" className="mb-4">{notice}</AlertBox>}
{quotaUser && <QuotaDialog user={quotaUser} onClose={() => setQuotaUser(null)} onSaved={() => { setNotice(quotaUser.username + ' 的配额已更新'); void users.refresh(); void audit.refresh(); }} />}
<Panel title="创建账号">
<form className="grid grid-cols-1 items-end gap-4 sm:grid-cols-2 lg:grid-cols-[1fr_1fr_minmax(8rem,10rem)_auto]" onSubmit={create}>
<label className="grid gap-1.5 text-[13px] font-medium">用户名(3–40 位字母、数字、_ 或 -)
<input className={inputClass} name="username" required minLength={3} maxLength={40} pattern="[A-Za-z0-9_\-]{3,40}" disabled={action.busy} autoComplete="off" /></label>
<label className="grid gap-1.5 text-[13px] font-medium">初始密码(12–72 UTF-8 字节)
<input className={inputClass} name="password" type="password" disabled={action.busy} required autoComplete="new-password" /></label>
<label className="grid gap-1.5 text-[13px] font-medium">工作区配额(GiB)
<input className={inputClass} name="quota" disabled={action.busy} type="number" min="0.1" step="0.1" required defaultValue="10" /></label>
<label className="flex items-center gap-2 text-[13px] font-medium sm:col-span-2 lg:col-span-1 lg:row-start-2 lg:col-start-1 xl:col-auto xl:row-auto">
<input name="admin" disabled={action.busy} type="checkbox" className="size-4 accent-primary" />管理员(拥有平台管理权限)</label>
<button className={cn(buttonVariants(), 'sm:col-span-2 sm:justify-self-start lg:col-span-1 lg:col-start-4 lg:row-start-1')} disabled={action.busy}>创建账号</button>
</form>
</Panel>
<Panel title="用户列表" action={<RefreshButton loading={users.loading} onClick={users.refresh} />}>
<ResourceState resource={users} />
{users.data?.length === 0 && <Empty>暂无用户</Empty>}
{!!users.data?.length && <>
<div className="grid gap-3 md:grid-cols-2 xl:grid-cols-3">
{users.data.map(u => <div key={u.id} className="rounded-lg border border-border bg-card p-3.5">
<div className="mb-1 flex items-center justify-between gap-2">
<strong className="truncate text-[14px]">{u.username}</strong>
<span className="rounded-full bg-muted px-2.5 py-0.5 text-xs text-muted-foreground">{u.admin ? '管理员' : '成员'}</span>
</div>
<p className="m-0 font-mono text-[11px] text-muted-foreground">{u.id}</p>
<p className="mb-3 mt-1 text-xs text-muted-foreground">创建于 {date(u.created_at)}</p>
<div className="flex flex-wrap items-center gap-1.5">
<span className="mr-1 text-[13px]">配额 {bytes(u.quota_bytes)}</span>
<button className={cn(buttonVariants({variant: 'outline', size: 'sm'}))} onClick={() => { setNotice(''); setQuotaUser(u); }}>设置配额</button>
<ResetPassword user={u} onReset={audit.refresh} />
</div>
</div>)}
</div>
</>}
</Panel>
<Panel title="操作审计" action={<RefreshButton loading={audit.loading} onClick={audit.refresh} />}>
<ResourceState resource={audit} />
{audit.data?.length === 0 && <Empty>暂无审计记录</Empty>}
{!!audit.data?.length && <>
<div className="hidden overflow-x-auto md:block">
<table className={tableClass}>
<thead><tr className={tableHeadRowClass}>
<th className={cn(thClass)}>时间</th>
<th className={cn(thClass)}>用户</th>
<th className={cn(thClass)}>操作</th>
<th className={cn(thClass)}>目标</th>
<th className={cn(thClass)}>详情</th>
</tr></thead>
<tbody>{audit.data.map(a => <tr key={a.id} className={trClass}>
<td className="border-b border-border px-3 py-3 align-top whitespace-nowrap">{date(a.created_at)}</td>
<td className="border-b border-border px-3 py-3 align-top font-mono text-xs">{a.user_id}</td>
<td className={tdClass}>{a.action}</td>
<td className="max-w-52 border-b border-border px-3 py-3 align-top wrap-anywhere">{a.target}</td>
<td className="max-w-52 border-b border-border px-3 py-3 align-top wrap-anywhere">{a.detail}</td>
</tr>)}</tbody>
</table>
</div>
<div className="grid gap-2 md:hidden">
{audit.data.map(a => <div key={a.id} className="rounded-lg border border-border bg-card p-3 text-[13px]">
<div className="mb-1 flex items-center justify-between gap-2 text-xs text-muted-foreground">
<span>{date(a.created_at)}</span><span className="font-mono">{a.user_id}</span>
</div>
<p className="m-0"><strong>{a.action}</strong> · <span className="wrap-anywhere">{a.target}</span></p>
{a.detail && <p className="m-0 mt-0.5 text-muted-foreground wrap-anywhere">{a.detail}</p>}
</div>)}
</div>
</>}
</Panel>
</>;
} }
+181 -38
View File
@@ -1,6 +1,8 @@
import { useEffect, useState, type FormEvent } from 'react'; import { useEffect, useRef, useState, type FormEvent } from 'react';
import { api, ApiError, type User } from './api'; import { Boxes, Container, FolderOpen, LogOut, Package, Settings, SquareTerminal, Users } from 'lucide-react';
import { Alert, message, Panel, useAction } from './ui'; import { api, type User } from './api';
import { Alert, Panel, useAction } from './ui';
import { watchSession, type LoginState } from './sessionRecovery';
import { Instances } from './Instances'; import { Instances } from './Instances';
import { Images } from './Images'; import { Images } from './Images';
import { RuntimeSelector } from './RuntimeSelector'; import { RuntimeSelector } from './RuntimeSelector';
@@ -8,52 +10,193 @@ import { Workspace } from './Workspace';
import { Admin } from './Admin'; import { Admin } from './Admin';
import { ChangePassword } from './ChangePassword'; import { ChangePassword } from './ChangePassword';
import { RuntimeWorkspace, useRuntimeInstances } from './RuntimeWorkspace'; import { RuntimeWorkspace, useRuntimeInstances } from './RuntimeWorkspace';
const pages = [{id:'dsh',label:'DSH',icon:'◈',description:'你的 DeepSeek Harness 工作台'}, {id:'overview',label:'实例',icon:'◫',description:'管理你的 DeepSeek Harness 运行环境'}, {id:'workspace',label:'文件',icon:'▤',description:'管理当前实例的文件与目录'}, {id:'settings',label:'设置',icon:'⚙',description:'选择默认实例与管理工作台'}]; import { Button, buttonVariants } from './components/ui/button';
const secondaryPages = [{id:'images',label:'镜像与构建',description:'本地版本管理 · 手动构建 · 任务日志'}, {id:'admin',label:'用户与审计',description:'账号管理与操作追踪'}]; import { Input } from './components/ui/input';
import { cn } from './lib/utils';
import { ThemeSettings } from './ThemeSettings';
const pages = [
{id: 'dsh', label: 'DSH', icon: SquareTerminal, description: '你的 DeepSeek Harness 工作台'},
{id: 'overview', label: '实例', icon: Container, description: '管理你的 DeepSeek Harness 运行环境'},
{id: 'workspace', label: '文件', icon: FolderOpen, description: '管理当前实例的文件与目录'},
{id: 'settings', label: '设置', icon: Settings, description: '选择默认实例与管理工作台'},
];
const secondaryPages = [
{id: 'images', label: '镜像与构建', icon: Package, description: '本地版本管理 · 手动构建 · 任务日志'},
{id: 'admin', label: '用户与审计', icon: Users, description: '账号管理与操作追踪'},
];
export default function App() { export default function App() {
const [user, setUser] = useState<User | null>(null), [loading, setLoading] = useState(true), [error, setError] = useState(''); const [{user, loading, error}, setSession] = useState<LoginState>({user: null, loading: true, error: ''});
useEffect(() => { let active = true; api.me().then(s => {if(active) setUser(s.user);}).catch(e => {if(active && (!(e instanceof ApiError) || e.status !== 401)) setError(message(e));}).finally(() => {if(active) setLoading(false);}); const expired = (event: Event) => {setUser(null); setError(event instanceof CustomEvent && typeof event.detail === 'string' ? event.detail : '会话已过期,请重新登录。');}; window.addEventListener('session-expired', expired); return () => {active = false; window.removeEventListener('session-expired', expired);}; }, []); const session = useRef<ReturnType<typeof watchSession> | null>(null);
if (loading) return <div className='splash' role='status'>正在连接工作台…</div>; useEffect(() => {
if (!user) return <Login error={error} onLogin={u => {setUser(u); setError('');}} />; const watcher = watchSession(() => api.me(), setSession, window, document);
return <Console key={user.id} user={user} onLogout={() => setUser(null)} onPasswordChanged={() => {setUser(null); setError('密码已修改,请使用新密码重新登录。');}}/>; session.current = watcher;
return () => { watcher.dispose(); session.current = null; };
}, []);
if (loading) return <div className="grid min-h-dvh place-items-center text-muted-foreground" role="status">正在连接工作台…</div>;
if (!user) return <Login error={error} onLogin={u => session.current?.login(u)} />;
return <>{error && <div role="status" className="fixed inset-x-0 top-0 z-50 p-3"><Alert>{error}</Alert></div>}<Console key={user.id} user={user} onLogout={() => session.current?.stop()} onPasswordChanged={() => session.current?.stop('密码已修改,请使用新密码重新登录。')} /></>;
} }
function Console({user, onLogout, onPasswordChanged}: {user: User; onLogout: () => void; onPasswordChanged: () => void}) { function Console({user, onLogout, onPasswordChanged}: {user: User; onLogout: () => void; onPasswordChanged: () => void}) {
const [page, setPage] = useState('dsh'); const [page, setPage] = useState('dsh');
const logout = useAction(); const logout = useAction();
const runtime = useRuntimeInstances(user.id, page); const runtime = useRuntimeInstances(user.id, page);
const current = [...pages, ...secondaryPages].find(p => p.id === page)!; const current = [...pages, ...secondaryPages].find(p => p.id === page)!;
const primaryPage = secondaryPages.some(p => p.id === page) ? 'settings' : page; // Secondary pages live under 设置 on the mobile bottom nav.
const exit = () => void logout.run(async () => {await api.logout(); onLogout();}); const navPage = secondaryPages.some(p => p.id === page) ? 'settings' : page;
return <div className='shell console-shell'> const exit = () => void logout.run(async () => { await api.logout(); onLogout(); });
<aside className='sidebar'><div className='brand'><b>DSH<span>●</span></b><small>HARNESS CONSOLE</small></div><div className='nav-label'>工作台</div>
<nav aria-label='主导航'>{pages.map(p => <button key={p.id} className={primaryPage === p.id ? 'active' : ''} aria-current={primaryPage === p.id ? 'page' : undefined} onClick={() => setPage(p.id)}><span aria-hidden='true'>{p.icon}</span>{p.label}</button>)}</nav> const navButton = (p: typeof pages[number], mobile: boolean) => {
<div className='sidebar-bottom'><div className='avatar'>{user.username.slice(0,1).toUpperCase()}</div><div><strong>{user.username}</strong><small>{user.admin ? '管理员' : '成员'}</small></div><button title='退出登录' disabled={logout.busy} onClick={exit}>退出</button></div> const Icon = p.icon;
</aside> // Desktop sidebar lists secondary pages directly, so highlight by the real
<main><header className='topbar'><span>控制台 / {current.label}</span><span className='tag'>本地镜像 · 独立实例</span></header> // page; the mobile bottom bar only has the four primary entries.
{logout.feedback} const active = (mobile ? navPage : page) === p.id;
<RuntimeWorkspace runtime={runtime} visible={page === 'dsh'} manage={() => setPage('overview')} settings={() => setPage('settings')}/> return <button key={p.id} aria-current={active ? 'page' : undefined} onClick={() => setPage(p.id)}
{page !== 'dsh' && <div className='panel-scroll'><div className='content'><div className='page-heading'><div><p className='eyebrow'>DEEPSEEK HARNESS</p><h1>{current.label}</h1><p className='muted'>{current.description}</p></div></div> className={mobile
{page === 'overview' && <Instances />} ? cn('flex min-w-0 flex-1 flex-col items-center justify-center gap-0.5 py-1 text-[12px] transition-colors', active ? 'text-nav-accent' : 'text-nav-foreground hover:text-white')
{page === 'images' && <><button className='back-settings' onClick={() => setPage('settings')}>← 返回设置</button><Images admin={user.admin}/></>} : cn('flex w-full items-center gap-2.5 rounded-lg px-3 py-2.5 text-left text-[13.5px] transition-colors focus-visible:outline-2 focus-visible:outline-ring', active ? 'bg-nav-active text-nav-accent font-medium shadow-[inset_3px_0_0_0_var(--color-nav-accent)]' : 'text-nav-foreground hover:bg-nav-active/60 hover:text-white')}>
{page === 'workspace' && <Workspace runtime={runtime}/>} <Icon className={mobile ? 'size-5' : 'size-4 shrink-0'} aria-hidden />
{page === 'admin' && user.admin && <><button className='back-settings' onClick={() => setPage('settings')}>← 返回设置</button><Admin /></>} <span className="truncate">{p.label}</span>
{page === 'settings' && <> </button>;
<Panel title='DSH 实例' action={<button disabled={runtime.loading} onClick={() => void runtime.refresh()}>刷新</button>}> };
{runtime.error && <Alert>{runtime.error}<button disabled={runtime.loading} onClick={() => void runtime.refresh()}>重试</button></Alert>}
{runtime.loading && <p role='status' className='muted'>正在刷新实例…</p>} return <div className="flex h-dvh overflow-hidden bg-background">
<RuntimeSelector runtime={runtime}/> {/* Desktop sidebar */}
<p className='muted selection-note'>仅在此浏览器按账号记住实例 ID,不保存密钥。已选择的停止实例不会自动切换。</p> <aside className="hidden w-60 shrink-0 flex-col bg-nav text-nav-foreground md:flex">
<div className='actions'><button className='primary' onClick={() => setPage('dsh')}>进入 DSH</button><button onClick={() => setPage('overview')}>管理 / 创建实例</button></div> <div className="px-5 pt-7 pb-8">
</Panel> <b className="block text-[26px] font-bold leading-tight tracking-widest text-white">DSH<span className="ml-1 align-middle text-[10px] text-nav-accent">●</span></b>
<Panel title='工作台管理'><div className='actions'><button onClick={() => setPage('images')}>镜像与构建</button>{user.admin && <button onClick={() => setPage('admin')}>用户与审计</button>}</div></Panel> <small className="mt-1 block text-[9px] tracking-[0.2em] text-nav-muted">HARNESS CONSOLE</small>
<Panel title='账号'><p>{user.username} · {user.admin ? '管理员' : '成员'}</p><p className='muted'>修改密码会使所有设备退出登录。</p><ChangePassword onChanged={onPasswordChanged}/><div className='account-signout'><p className='muted'>仅退出当前浏览器会话,不影响运行中的实例。</p><button className='danger account-action' disabled={logout.busy} onClick={exit}>{logout.busy ? '正在退出…' : '退出登录'}</button></div></Panel> </div>
<nav aria-label="主导航" className="grid gap-1 px-3">
<p className="m-0 px-2 pb-1 text-[11px] text-nav-muted">工作台</p>
{pages.map(p => navButton(p, false))}
{user.admin && <>
<p className="m-0 mt-4 px-2 pb-1 text-[11px] text-nav-muted">管理</p>
{secondaryPages.map(p => {
const Icon = p.icon;
const active = page === p.id;
return <button key={p.id} aria-current={active ? 'page' : undefined} onClick={() => setPage(p.id)}
className={cn('flex w-full items-center gap-2.5 rounded-lg px-3 py-2.5 text-left text-[13.5px] transition-colors focus-visible:outline-2 focus-visible:outline-ring', active ? 'bg-nav-active text-nav-accent font-medium shadow-[inset_3px_0_0_0_var(--color-nav-accent)]' : 'text-nav-foreground hover:bg-nav-active/60 hover:text-white')}>
<Icon className="size-4 shrink-0" aria-hidden />
<span className="truncate">{p.label}</span>
</button>;
})}
</>} </>}
</div><footer>DSH 工作台 <span>数据与操作以服务器实际结果为准</span></footer></div>} </nav>
<div className="mt-auto flex items-center gap-2.5 border-t border-white/10 px-5 py-4">
<div className="grid size-8 shrink-0 place-items-center rounded-lg bg-nav-active text-[13px] font-semibold text-nav-accent">{user.username.slice(0, 1).toUpperCase()}</div>
<div className="min-w-0 flex-1 leading-tight">
<strong className="block truncate text-[13px] text-white">{user.username}</strong>
<small className="text-[11px] text-nav-muted">{user.admin ? '管理员' : '成员'}</small>
</div>
<button title="退出登录" aria-label="退出登录" disabled={logout.busy} onClick={exit} className="rounded-md p-2 text-nav-muted transition-colors hover:bg-nav-active hover:text-white">
<LogOut className="size-4" aria-hidden />
</button>
</div>
</aside>
{/* Mobile bottom navigation */}
<nav aria-label="主导航" className="fixed inset-x-0 bottom-0 z-20 flex bg-nav pb-[env(safe-area-inset-bottom,0px)] md:hidden">
{pages.map(p => navButton(p, true))}
</nav>
<main className="flex min-w-0 flex-1 flex-col pb-[calc(60px+env(safe-area-inset-bottom,0px))] md:pb-0">
<header className="flex h-12 shrink-0 items-center justify-between gap-3 border-b border-border bg-card px-4 sm:px-6">
<span className="truncate text-[13px] text-muted-foreground">控制台 <span aria-hidden>/</span> <span className="font-medium text-foreground">{current.label}</span></span>
<span className="hidden rounded-md border border-border bg-muted px-2 py-0.5 text-[11px] text-muted-foreground sm:inline">本地镜像 · 独立实例</span>
</header>
{logout.feedback}
<RuntimeWorkspace runtime={runtime} visible={page === 'dsh'} manage={() => setPage('overview')} settings={() => setPage('settings')} />
{page !== 'dsh' && <div className="min-h-0 flex-1 overflow-auto overscroll-contain">
<div className="mx-auto w-full max-w-6xl p-4 sm:p-6">
<div className="mb-5">
<p className="m-0 mb-1 text-[10px] font-bold tracking-[0.22em] text-primary">DEEPSEEK HARNESS</p>
<h1 className="mb-1 text-[22px] font-semibold leading-snug tracking-tight">{current.label}</h1>
<p className="m-0 text-[13px] text-muted-foreground">{current.description}</p>
</div>
{page === 'overview' && <Instances />}
{(page === 'images' || page === 'admin') && <button className={cn(buttonVariants({variant: 'ghost', size: 'sm'}), 'mb-4 -ml-2 text-muted-foreground md:hidden')} onClick={() => setPage('settings')}>← 返回设置</button>}
{page === 'images' && <Images admin={user.admin} />}
{page === 'workspace' && <Workspace runtime={runtime} />}
{page === 'admin' && user.admin && <Admin />}
{page === 'settings' && <>
<ThemeSettings />
<Panel title="DSH 实例" action={<Button variant="ghost" size="sm" disabled={runtime.loading} onClick={() => void runtime.refresh()}>刷新</Button>}>
{runtime.error && <Alert>{runtime.error}<Button variant="outline" size="sm" className="ml-2" disabled={runtime.loading} onClick={() => void runtime.refresh()}>重试</Button></Alert>}
{runtime.loading && <p role="status" className="m-0 mb-3 text-[13px] text-muted-foreground">正在刷新实例…</p>}
<RuntimeSelector runtime={runtime} />
<p className="mb-4 mt-3 text-[13px] text-muted-foreground">仅在此浏览器按账号记住实例 ID,不保存密钥。已选择的停止实例不会自动切换。</p>
<div className="flex flex-wrap gap-2">
<Button onClick={() => setPage('dsh')}>进入 DSH</Button>
<Button variant="outline" onClick={() => setPage('overview')}>管理 / 创建实例</Button>
</div>
</Panel>
<Panel title="工作台管理">
<div className="flex flex-wrap gap-2">
<Button variant="outline" onClick={() => setPage('images')}><Package aria-hidden />镜像与构建</Button>
{user.admin && <Button variant="outline" onClick={() => setPage('admin')}><Users aria-hidden />用户与审计</Button>}
</div>
</Panel>
<Panel title="账号">
<p className="mb-1 text-[13px]">{user.username} · {user.admin ? '管理员' : '成员'}</p>
<p className="mb-4 text-[13px] text-muted-foreground">修改密码会使所有设备退出登录。</p>
<ChangePassword onChanged={onPasswordChanged} />
<div className="mt-5 border-t border-border pt-4">
<p className="mb-3 text-[13px] text-muted-foreground">仅退出当前浏览器会话,不影响运行中的实例。</p>
<Button variant="destructive-ghost" disabled={logout.busy} onClick={exit}><LogOut aria-hidden />{logout.busy ? '正在退出…' : '退出登录'}</Button>
</div>
</Panel>
</>}
<footer className="mt-8 hidden items-center justify-between border-t border-border pt-4 text-[11px] text-muted-foreground md:flex">
<span>DSH 工作台</span><span>数据与操作以服务器实际结果为准</span>
</footer>
</div>
</div>}
</main> </main>
</div>; </div>;
} }
function Login({error, onLogin}: {error: string; onLogin: (user: User) => void}) { function Login({error, onLogin}: {error: string; onLogin: (user: User) => void}) {
const action = useAction(); const action = useAction();
function submit(e: FormEvent<HTMLFormElement>) {e.preventDefault(); const form = new FormData(e.currentTarget); void action.run(async () => {const session = await api.login(String(form.get('username')), String(form.get('password'))); onLogin(session.user);});} function submit(e: FormEvent<HTMLFormElement>) {
return <div className="login"><section className="login-intro"><div className="brand"><b>DSH<span>●</span></b><small>HARNESS CONSOLE</small></div><div><p className="eyebrow">你的 AI 开发空间</p><h1>专注创造,<br/>让环境管理更简单。</h1><p>独立实例、本地镜像、实例数据根。<br/>在一个工作台中,掌控你的 DeepSeek Harness。</p></div><small>安全会话 · 资源隔离 · 操作审计</small></section><section className="login-form"><form onSubmit={submit}><p className="eyebrow">WELCOME BACK</p><h2>登录工作台</h2><p className="muted">使用管理员分配的账号继续</p>{error && <Alert>{error}</Alert>}{action.feedback}<label>用户名<input name="username" autoComplete="username" required autoFocus /></label><label>密码<input name="password" type="password" autoComplete="current-password" required /></label><button className="primary" disabled={action.busy}>{action.busy ? '正在登录…' : '登录 →'}</button><p className="login-note">没有账号?请联系平台管理员。<br/>登录凭据不会存储在浏览器本地存储中。</p></form></section></div>; e.preventDefault();
const form = new FormData(e.currentTarget);
void action.run(async () => {
const session = await api.login(String(form.get('username')), String(form.get('password')));
onLogin(session.user);
});
}
return <div className="grid min-h-dvh bg-card md:grid-cols-2">
<section className="flex flex-col justify-between gap-10 bg-nav p-8 text-nav-foreground sm:p-12">
<div>
<b className="block text-[26px] font-bold leading-tight tracking-widest text-white">DSH<span className="ml-1 align-middle text-[10px] text-nav-accent">●</span></b>
<small className="mt-1 block text-[9px] tracking-[0.2em] text-nav-muted">HARNESS CONSOLE</small>
</div>
<div className="hidden md:block">
<p className="m-0 mb-3 text-[10px] font-bold tracking-[0.22em] text-nav-accent">你的 AI 开发空间</p>
<h1 className="mb-5 text-[34px] font-semibold leading-[1.5] text-white">专注创造,<br />让环境管理更简单。</h1>
<p className="m-0 leading-loose text-nav-muted">独立实例、本地镜像、实例数据根。<br />在一个工作台中,掌控你的 DeepSeek Harness。</p>
</div>
<small className="hidden text-[12px] text-nav-muted md:block">安全会话 · 资源隔离 · 操作审计</small>
</section>
<section className="grid place-items-center p-6 sm:p-10">
<form onSubmit={submit} className="w-full max-w-sm">
<p className="m-0 mb-2 text-[10px] font-bold tracking-[0.22em] text-primary">WELCOME BACK</p>
<h2 className="mb-1 text-[24px] font-semibold">登录工作台</h2>
<p className="mb-6 text-[13px] text-muted-foreground">使用管理员分配的账号继续</p>
<div className="grid gap-3">
{error && <Alert>{error}</Alert>}
{action.feedback}
</div>
<div className="mt-4 grid gap-4">
<label className="grid gap-1.5 text-[13px] font-medium">用户名<Input name="username" autoComplete="username" required autoFocus /></label>
<label className="grid gap-1.5 text-[13px] font-medium">密码<Input name="password" type="password" autoComplete="current-password" required /></label>
</div>
<Button size="lg" className="mt-6 w-full" disabled={action.busy}>{action.busy ? '正在登录…' : '登录 →'}</Button>
<p className="mb-0 mt-5 text-[12px] leading-relaxed text-muted-foreground">没有账号?请联系平台管理员。<br />登录凭据不会存储在浏览器本地存储中。</p>
</form>
</section>
</div>;
} }
+186
View File
@@ -0,0 +1,186 @@
import { beforeEach, expect, it, vi } from 'vitest';
const state = vi.hoisted(() => ({slots: [] as any[], cursor: 0, resourceCursor: 0, snapshots: [] as any[], refreshStorage: vi.fn(async () => {})}));
vi.mock('react', async original => ({
...await original<typeof import('react')>(),
useState(value: unknown) {const i = state.cursor++; if (!(i in state.slots)) state.slots[i] = value; return [state.slots[i], (next: unknown) => {state.slots[i] = next;}];},
useRef(value: unknown) {const i = state.cursor++; return state.slots[i] ??= {current: value};},
useEffect() {},
}));
vi.mock('./ui', async original => ({
...await original<typeof import('./ui')>(),
useResource: () => ({data: state.resourceCursor++ % 2 === 0 ? [{id: 'backup', created_at: '', size: 1}] : state.snapshots, loading: false, refresh: state.refreshStorage}),
}));
import { BackupPanel } from './BackupPanel';
import { api } from './api';
import { ConfirmDialog } from './components/ConfirmDialog';
import { DialogContent } from './components/ui/dialog';
function nodes(root: any, predicate: (node: any) => boolean): any[] {
if (Array.isArray(root)) return root.flatMap(child => nodes(child, predicate));
if (!root || typeof root !== 'object') return [];
return [...(predicate(root) ? [root] : []), ...nodes(root.props?.children, predicate)];
}
const refresh = vi.fn(async () => {});
const onRestored = vi.fn(async () => {});
function render() {state.cursor = 0; state.resourceCursor = 0; return BackupPanel({instance: {id: 'one'} as any, refresh, onRestored});}
function editor() {
const child = nodes(render(), n => typeof n.type === 'function' && n.props.backup)[0];
return child.type(child.props);
}
beforeEach(() => {state.slots = []; state.cursor = 0; state.snapshots = []; state.refreshStorage.mockClear(); vi.restoreAllMocks(); vi.unstubAllGlobals(); refresh.mockClear(); onRestored.mockClear();});
it('retains restore password and shows failure inside the owning modal after confirmation', async () => {
const request = vi.spyOn(api, 'request').mockRejectedValue(new Error('restore denied'));
nodes(render(), n => n.type === 'button' && Array.isArray(n.props.children) && n.props.children.includes('恢复'))[0].props.onClick();
nodes(editor(), n => n.type === 'input')[0].props.onChange({target: {value: 'original-password'}});
nodes(editor(), n => n.type === 'form')[0].props.onSubmit({preventDefault: vi.fn()});
nodes(render(), n => n.type === ConfirmDialog)[0].props.onClose(true);
await new Promise(resolve => setTimeout(resolve, 0));
const modal = nodes(editor(), n => n.type === DialogContent)[0];
expect(JSON.stringify(modal)).toContain('restore denied');
expect(nodes(modal, n => n.type === 'input')[0].props.value).toBe('original-password');
expect(request).toHaveBeenCalledExactlyOnceWith('/instances/one/restore', 'POST', {backup_id: 'backup', password: 'original-password'});
expect(refresh).not.toHaveBeenCalled();
expect(onRestored).not.toHaveBeenCalled();
});
function button(label: string) {
return nodes(render(), n => n.type === 'button' && (n.props.children === label || Array.isArray(n.props.children) && n.props.children.includes(label)))[0];
}
it('requires confirmation for configuration deletion and retains download', async () => {
const request = vi.spyOn(api, 'request').mockResolvedValue({});
expect(nodes(render(), n => n.type === 'a')[0].props.href).toBe('/api/instances/one/backups?download=backup');
button('删除配置包').props.onClick();
expect(request).not.toHaveBeenCalled();
nodes(render(), n => n.type === ConfirmDialog)[0].props.onClose(false);
await new Promise(resolve => setTimeout(resolve, 0));
expect(request).not.toHaveBeenCalled();
button('删除配置包').props.onClick();
nodes(render(), n => n.type === ConfirmDialog)[0].props.onClose(true);
await new Promise(resolve => setTimeout(resolve, 0));
expect(request).toHaveBeenCalledExactlyOnceWith('/instances/one/backups?id=backup', 'DELETE');
expect(state.refreshStorage).toHaveBeenCalledTimes(2);
expect(refresh).toHaveBeenCalledOnce();
expect(onRestored).not.toHaveBeenCalled();
});
it('shows snapshot bytes separately, deletes confirmed snapshots and refreshes after rejection', async () => {
state.snapshots = [{id: 'pre-restore-abc', size: 1048576, created_at: '', protected: false}];
const request = vi.spyOn(api, 'request').mockRejectedValue(new Error('instance recovery required'));
expect(JSON.stringify(render())).toContain('1.0 MiB');
button('删除快照').props.onClick();
nodes(render(), n => n.type === ConfirmDialog)[0].props.onClose(true);
await new Promise(resolve => setTimeout(resolve, 0));
expect(request).toHaveBeenCalledExactlyOnceWith('/instances/one/snapshots?id=pre-restore-abc', 'DELETE');
expect(state.refreshStorage).toHaveBeenCalledTimes(2);
expect(JSON.stringify(render())).toContain('instance recovery required');
expect(onRestored).not.toHaveBeenCalled();
});
it('disables protected snapshots and does not report failed size scans as zero', () => {
state.snapshots = [{id: 'pre-restore-abc', size: 0, created_at: '', protected: true, protection_reason: 'recovery required', size_error: 'tree limit'}];
expect(button('删除快照').props.disabled).toBe(true);
const text = JSON.stringify(render());
expect(text).toContain('recovery required');
expect(text).toContain('统计不完整');
expect(text).toContain('tree limit');
});
it('refreshes storage after successful snapshot deletion', async () => {
state.snapshots = [{id: 'pre-restore-abc', size: 1048576, created_at: '', protected: false}];
const request = vi.spyOn(api, 'request').mockResolvedValue({ok: true});
button('删除快照').props.onClick();
nodes(render(), n => n.type === ConfirmDialog)[0].props.onClose(true);
await new Promise(resolve => setTimeout(resolve, 0));
expect(request).toHaveBeenCalledExactlyOnceWith('/instances/one/snapshots?id=pre-restore-abc', 'DELETE');
expect(state.refreshStorage).toHaveBeenCalledTimes(2);
expect(refresh).toHaveBeenCalledOnce();
expect(JSON.stringify(render())).toContain('快照已删除');
expect(onRestored).not.toHaveBeenCalled();
});
it('manual storage refresh does not reset the profile', async () => {
button('刷新备份与占用').props.onClick();
await new Promise(resolve => setTimeout(resolve, 0));
expect(refresh).toHaveBeenCalledOnce();
expect(state.refreshStorage).toHaveBeenCalledTimes(2);
expect(onRestored).not.toHaveBeenCalled();
});
it('only a successful restore invokes the profile reload callback', async () => {
vi.spyOn(api, 'request').mockResolvedValue({ok: true});
button('恢复').props.onClick();
nodes(editor(), n => n.type === 'form')[0].props.onSubmit({preventDefault: vi.fn()});
nodes(render(), n => n.type === ConfirmDialog)[0].props.onClose(true);
await new Promise(resolve => setTimeout(resolve, 0));
expect(onRestored).toHaveBeenCalledOnce();
expect(refresh).not.toHaveBeenCalled();
});
const snapshot = {id: 'pre-restore/a b', size: 1024, created_at: '', protected: false, restorable: true, restore_reason: '', source_image: 'image', source_version: '1.2.3'};
function snapshotEditor() {
const child = nodes(render(), n => typeof n.type === 'function' && n.props.snapshot)[0];
return child.type(child.props);
}
const submitSnapshot = () => nodes(snapshotEditor(), n => n.type === 'form')[0].props.onSubmit({preventDefault: vi.fn()});
const snapshotPassword = () => nodes(snapshotEditor(), n => n.type === 'input' && n.props.type === 'password')[0];
it('warns about plaintext keys before download, cancels safely, and navigates only after confirmation', async () => {
state.snapshots = [snapshot];
const assign = vi.fn();
vi.stubGlobal('window', {location: {assign}});
button('下载快照').props.onClick();
const confirmation = nodes(render(), n => n.type === ConfirmDialog)[0];
expect(confirmation.props.request.options.body).toContain('未加密');
expect(confirmation.props.request.options.body).toContain('密钥');
expect(confirmation.props.request.options.body).toContain('明文');
expect(assign).not.toHaveBeenCalled();
confirmation.props.onClose(false);
await new Promise(resolve => setTimeout(resolve, 0));
expect(assign).not.toHaveBeenCalled();
button('下载快照').props.onClick();
nodes(render(), n => n.type === ConfirmDialog)[0].props.onClose(true);
await new Promise(resolve => setTimeout(resolve, 0));
expect(assign).toHaveBeenCalledExactlyOnceWith('/api/instances/one/snapshots?download=pre-restore%2Fa%20b');
});
it('restores a known snapshot with the current password and reloads the profile only on success', async () => {
state.snapshots = [snapshot];
const request = vi.spyOn(api, 'request').mockResolvedValue({ok: true});
expect(JSON.stringify(render())).toContain('1.2.3');
button('恢复快照').props.onClick();
await submitSnapshot();
expect(request).not.toHaveBeenCalled();
expect(nodes(snapshotEditor(), n => n.type === 'input' && n.props.type === 'checkbox')).toHaveLength(0);
const text = JSON.stringify(snapshotEditor());
for (const warning of ['覆盖完整 home', '失败回滚副本', '保留原始快照', '临时停止', '就绪验证', '不会自动降级']) expect(text).toContain(warning);
snapshotPassword().props.onChange({target: {value: 'current-password'}});
await submitSnapshot();
expect(request).toHaveBeenCalledExactlyOnceWith('/instances/one/snapshots', 'POST', {id: snapshot.id, password: 'current-password', confirm: true, allow_unknown_version: false});
expect(onRestored).toHaveBeenCalledOnce();
expect(refresh).toHaveBeenCalledOnce();
expect(state.refreshStorage).toHaveBeenCalledTimes(2);
expect(nodes(render(), n => typeof n.type === 'function' && n.props.snapshot)).toHaveLength(0);
});
it.each(['source_image', 'source_version'])('requires unknown compatibility acknowledgement when %s is missing, clears password and refreshes after failure', async missing => {
state.snapshots = [{...snapshot, [missing]: undefined}];
const request = vi.spyOn(api, 'request').mockRejectedValue(new Error('snapshot restore denied'));
button('恢复快照').props.onClick();
snapshotPassword().props.onChange({target: {value: 'current-password'}});
await submitSnapshot();
expect(request).not.toHaveBeenCalled();
expect(nodes(snapshotEditor(), n => n.type === 'button' && n.props.children === '确认覆盖并恢复快照')[0].props.disabled).toBe(true);
nodes(snapshotEditor(), n => n.type === 'input' && n.props.type === 'checkbox')[0].props.onChange({target: {checked: true}});
await submitSnapshot();
expect(request).toHaveBeenCalledExactlyOnceWith('/instances/one/snapshots', 'POST', {id: snapshot.id, password: 'current-password', confirm: true, allow_unknown_version: true});
expect(snapshotPassword().props.value).toBe('');
expect(JSON.stringify(nodes(snapshotEditor(), n => n.type === DialogContent)[0])).toContain('snapshot restore denied');
expect(onRestored).not.toHaveBeenCalled();
expect(refresh).toHaveBeenCalledOnce();
expect(state.refreshStorage).toHaveBeenCalledTimes(2);
});
it.each([{protected: true, restorable: true}, {protected: false, restorable: false}])('disables protected or incompatible snapshot restoration (%j)', flags => {
state.snapshots = [{...snapshot, ...flags, restore_reason: 'incompatible home', protection_reason: 'recovery required'}];
expect(button('恢复快照').props.disabled).toBe(true);
expect(JSON.stringify(render())).toContain('incompatible home');
expect(button('下载快照').props.disabled).toBe(false);
});
it('cancels snapshot restoration without requests or profile reloads', () => {
state.snapshots = [snapshot];
const request = vi.spyOn(api, 'request');
button('恢复快照').props.onClick();
nodes(snapshotEditor(), n => n.type === 'button' && n.props.children === '取消')[0].props.onClick();
expect(nodes(render(), n => typeof n.type === 'function' && n.props.snapshot)).toHaveLength(0);
expect(request).not.toHaveBeenCalled();
expect(onRestored).not.toHaveBeenCalled();
});
+204 -18
View File
@@ -1,26 +1,212 @@
import { useState } from 'react'; import { useState, type FormEvent, type ReactNode } from 'react';
import { api, instancePath, type Instance, type Backup } from './api'; import { Download, HardDriveDownload, RotateCcw, Trash2, Upload } from 'lucide-react';
import { api, instancePath, type Instance, type Backup, type Snapshot } from './api';
import { ResourceState, Empty, bytes, date, useAction, useResource } from './ui'; import { ResourceState, Empty, bytes, date, useAction, useResource } from './ui';
import { useConfirm } from './components/ConfirmDialog';
import { AlertBox } from './components/ui/alert';
import { DialogContent, DialogFooter, DialogRoot } from './components/ui/dialog';
import { buttonVariants } from './components/ui/button';
import { inputClass } from './components/ui/input';
import { cn } from './lib/utils';
export function BackupPanel({ instance, refresh }: { instance: Instance; refresh: () => Promise<void> }) { export function BackupPanel({instance, refresh, onRestored}: {instance: Instance; refresh: () => Promise<void>; onRestored: () => Promise<void>}) {
const base = instancePath(instance.id); const base = instancePath(instance.id);
const backups = useResource(() => api.request<Backup[]>(base + '/backups')); const backups = useResource(() => api.request<Backup[]>(base + '/backups'), base);
const snapshots = useResource(() => api.request<Snapshot[]>(base + '/snapshots'), base);
const action = useAction(); const action = useAction();
const [password, setPassword] = useState(''), [originalPassword, setOriginalPassword] = useState(''), [file, setFile] = useState<File>(); const confirm = useConfirm();
return <section aria-label="备份与恢复"> const [password, setPassword] = useState('');
<p className="callout">操作会按需停止实例并恢复原运行状态。备份使用创建时的登录密码加密;修改登录密码不改变旧包密码。恢复包含插件重建、重启验证和失败回滚。</p> const [importPassword, setImportPassword] = useState(''), [file, setFile] = useState<File>();
{action.feedback} const [restoreTarget, setRestoreTarget] = useState<Backup>();
<form onSubmit={e => { e.preventDefault(); void action.run(async () => { try { await api.request(base + '/backups', 'POST', { password }); await backups.refresh(); } finally { setPassword(''); } }, '加密备份已创建'); }}> const [snapshotTarget, setSnapshotTarget] = useState<Snapshot>();
<label>当前登录密码<input type="password" autoComplete="current-password" required value={password} onChange={e => setPassword(e.target.value)} /></label> async function refreshStorage() {
<button disabled={action.busy || !password}>创建加密备份</button> await Promise.all([backups.refresh(), snapshots.refresh(), refresh()]);
</form> }
<label>原备份密码(旧平台密钥包留空)<input type="password" autoComplete="off" value={originalPassword} onChange={e => setOriginalPassword(e.target.value)} /></label> async function remove(item: Backup | Snapshot, snapshot: boolean) {
<form onSubmit={e => { e.preventDefault(); if (!file) return; void action.run(async () => { try { if (file.size > 44743364) throw new Error('备份包超过大小限制'); const form = new FormData(); form.set('file', file); form.set('password', originalPassword); await api.request(base + '/backups', 'POST', form); await backups.refresh(); } finally { setOriginalPassword(''); } }, '备份已验证并导入;尚未覆盖当前配置'); }}> if (snapshot && (item as Snapshot).protected) return;
<label>加密备份文件<input type="file" accept=".dshbak" onChange={e => setFile(e.target.files?.[0])} /></label> if (!await confirm.ask({
<button disabled={action.busy || !file}>验证并导入</button> title: snapshot ? '删除完整 home 快照' : '删除配置备份', danger: true, confirmLabel: '永久删除',
</form> body: `确认永久删除 ${item.id}?此操作不可撤销,不影响当前 home。${snapshot ? '删除后将无法使用此快照回滚。' : '请先下载需要保留的配置包;删除配置包不会清理完整快照。'}`,
})) return;
await action.run(async () => {
try { await api.request(base + (snapshot ? '/snapshots?id=' : '/backups?id=') + encodeURIComponent(item.id), 'DELETE'); }
finally { await refreshStorage(); }
}, snapshot ? '快照已删除,空间占用已刷新' : '配置备份已删除');
}
async function downloadSnapshot(snapshot: Snapshot) {
if (!await confirm.ask({
title: '下载完整 home 快照', danger: true, confirmLabel: '确认下载',
body: '下载的是未加密的 tar.gz 完整 home 归档,可能包含密钥、密码及其他明文敏感数据。请妥善保管,不要公开分享。',
})) return;
window.location.assign('/api' + base + '/snapshots?download=' + encodeURIComponent(snapshot.id));
}
async function restoreSnapshot(snapshot: Snapshot, password: string, allowUnknownVersion: boolean) {
if (snapshot.protected || !snapshot.restorable || !password || ((!snapshot.source_image || !snapshot.source_version) && !allowUnknownVersion)) return false;
return action.run(async () => {
try {
await api.request(base + '/snapshots', 'POST', {id: snapshot.id, password, confirm: true, allow_unknown_version: allowUnknownVersion});
await onRestored();
} finally { await refreshStorage(); }
}, '完整 home 快照恢复及就绪验证完成,已恢复原运行状态。');
}
function createBackup(e: FormEvent) {
e.preventDefault();
void action.run(async () => {
try { await api.request(base + '/backups', 'POST', {password}); await backups.refresh(); }
finally { setPassword(''); }
}, '加密备份已创建');
}
function importBackup(e: FormEvent) {
e.preventDefault();
if (!file) return;
void action.run(async () => {
try {
if (file.size > 44743364) throw new Error('备份包超过大小限制');
const form = new FormData();
form.set('file', file);
form.set('password', importPassword);
await api.request(base + '/backups', 'POST', form);
await backups.refresh();
} finally { setImportPassword(''); }
}, '备份已验证并导入;尚未覆盖当前配置');
}
async function restore(backup: Backup, originalPassword: string) {
if (!await confirm.ask({
title: '恢复配置备份', danger: true, confirmLabel: '确认恢复',
body: '确认覆盖当前配置?服务器将先保留当前 home 快照,随后按需停止实例并恢复原运行状态。',
})) return false;
return action.run(async () => {
try { await api.request(base + '/restore', 'POST', {backup_id: backup.id, password: originalPassword}); await onRestored(); }
finally { await snapshots.refresh(); }
}, '配置恢复及插件验证完成,已恢复原运行状态。');
}
return <section aria-label="备份与恢复" className="grid gap-5">
{confirm.element}
<AlertBox variant="info">操作会按需停止实例并恢复原运行状态。备份使用创建时的登录密码加密;修改登录密码不改变旧包密码。恢复包含插件重建、重启验证和失败回滚。</AlertBox>
<div className="grid gap-5 lg:grid-cols-2">
<form onSubmit={createBackup} className="grid content-start gap-3 rounded-lg border border-border bg-muted/40 p-4">
<h3 className="m-0 flex items-center gap-1.5 text-[14px] font-semibold"><HardDriveDownload className="size-4 text-primary" aria-hidden />创建加密备份</h3>
<label className="grid gap-1.5 text-[13px] font-medium">当前登录密码
<input className={inputClass} type="password" autoComplete="current-password" required value={password} onChange={e => setPassword(e.target.value)} /></label>
<div><button className={cn(buttonVariants({size: 'sm'}))} disabled={action.busy || !password}>创建加密备份</button></div>
</form>
<form onSubmit={importBackup} className="grid content-start gap-3 rounded-lg border border-border bg-muted/40 p-4">
<h3 className="m-0 flex items-center gap-1.5 text-[14px] font-semibold"><Upload className="size-4 text-primary" aria-hidden />导入外部备份</h3>
<label className="grid gap-1.5 text-[13px] font-medium">加密备份文件
<input type="file" accept=".dshbak" onChange={e => setFile(e.target.files?.[0])}
className="w-full cursor-pointer rounded-md border border-input bg-card px-3 py-2 text-sm file:mr-3 file:rounded file:border-0 file:bg-secondary file:px-2.5 file:py-1 file:text-xs file:text-secondary-foreground" /></label>
<label className="grid gap-1.5 text-[13px] font-medium">原备份密码(旧平台密钥包留空)
<input className={inputClass} type="password" autoComplete="off" value={importPassword} onChange={e => setImportPassword(e.target.value)} /></label>
<div><button className={cn(buttonVariants({variant: 'secondary', size: 'sm'}))} disabled={action.busy || !file}>验证并导入</button></div>
</form>
</div>
{!restoreTarget && !snapshotTarget && action.feedback}
<h3 className="m-0 text-sm font-semibold">配置备份(.dshbak)</h3>
<p className="m-0 text-xs text-muted-foreground">仅包含指定配置与资源,不包含完整 home 或依赖;下方大小为加密配置包大小。</p>
<ResourceState resource={backups} /> <ResourceState resource={backups} />
{backups.data?.length === 0 && <Empty>暂无配置备份</Empty>} {backups.data?.length === 0 && <Empty>暂无配置备份</Empty>}
{backups.data?.map(b => <div className="backup-row" key={b.id}><span>{b.id}</span><span>{date(b.created_at)}</span><span>{bytes(b.size)}</span><a className="button" href={'/api' + base + '/backups?download=' + encodeURIComponent(b.id)}>下载</a><button disabled={action.busy} onClick={() => action.run(async () => { try { await api.request(base + '/restore', 'POST', { backup_id: b.id, password: originalPassword }); await refresh(); } finally { setOriginalPassword(''); } }, '配置恢复及插件验证完成,已恢复原运行状态。', '确认覆盖当前配置?服务器将先保留当前 home 快照。')}>恢复</button></div>)} <div className="grid gap-2">
{backups.data?.map(b => <div key={b.id} className="flex flex-wrap items-center gap-x-4 gap-y-2 rounded-lg border border-border bg-card px-3.5 py-3">
<div className="min-w-0 flex-1">
<p className="m-0 truncate font-mono text-xs">{b.id}</p>
<p className="m-0 mt-0.5 text-xs text-muted-foreground">{date(b.created_at)} · {bytes(b.size)}</p>
</div>
<div className="flex flex-wrap gap-1.5">
<a className={cn(buttonVariants({variant: 'outline', size: 'sm'}))} href={'/api' + base + '/backups?download=' + encodeURIComponent(b.id)}><Download aria-hidden />下载</a>
<button className={cn(buttonVariants({variant: 'secondary', size: 'sm'}))} disabled={action.busy} onClick={() => setRestoreTarget(b)}><RotateCcw aria-hidden />恢复</button>
<button className={cn(buttonVariants({variant: 'destructive', size: 'sm'}))} disabled={action.busy} onClick={() => void remove(b, false)}><Trash2 aria-hidden />删除配置包</button>
</div>
</div>)}
</div>
<section aria-label="完整 home 快照" className="grid gap-3 border-t border-border pt-5">
<div className="flex flex-wrap items-center justify-between gap-2">
<h3 className="m-0 text-sm font-semibold">完整 home 快照</h3>
<button className={cn(buttonVariants({variant: 'outline', size: 'sm'}))} disabled={action.busy || snapshots.loading || backups.loading} onClick={() => void refreshStorage()}>刷新备份与占用</button>
</div>
<AlertBox variant="info">升级或恢复前保留的完整 home 副本,包含依赖及工作文件,会计入空间配额;不包含在上面的配置包大小中。可下载未加密的 tar.gz 归档或恢复完整 home,请妥善保管其中的密钥与明文数据。删除不可撤销,待恢复实例的快照受保护。</AlertBox>
<ResourceState resource={snapshots} />
{snapshots.data && !snapshots.error && <p className="m-0 text-sm">快照占用(文件字节数):{snapshots.data.some(s => s.size_error) ? '统计不完整,请查看各项错误' : bytes(snapshots.data.reduce((total, s) => total + s.size, 0))}</p>}
{snapshots.data?.length === 0 && <Empty>暂无完整 home 快照</Empty>}
{snapshots.data?.map(s => <div key={s.id} className="flex flex-wrap items-center gap-3 rounded-lg border border-border bg-card px-3.5 py-3">
<div className="min-w-0 flex-1">
<p className="m-0 break-all font-mono text-xs">{s.id}</p>
<p className="m-0 mt-1 text-xs text-muted-foreground">{date(s.created_at)} · {s.size_error ? '大小统计失败:' + s.size_error : bytes(s.size)}</p>
{s.protected && <p className="m-0 mt-1 text-xs text-destructive">受保护:{s.protection_reason || '实例需要恢复,请联系管理员'}</p>}
<p className="m-0 mt-1 text-xs text-muted-foreground">来源版本:{s.source_version || '未知'}{!s.source_image && ' · 来源镜像未知'}</p>
{s.restore_reason && <p className="m-0 mt-1 text-xs text-muted-foreground">{s.restore_reason}</p>}
</div>
<div className="flex flex-wrap gap-1.5">
<button className={cn(buttonVariants({variant: 'outline', size: 'sm'}))} disabled={action.busy} onClick={() => void downloadSnapshot(s)}><Download aria-hidden />下载快照</button>
<button className={cn(buttonVariants({variant: 'secondary', size: 'sm'}))} disabled={action.busy || s.protected || !s.restorable} onClick={() => setSnapshotTarget(s)}><RotateCcw aria-hidden />恢复快照</button>
<button className={cn(buttonVariants({variant: 'destructive', size: 'sm'}))} disabled={action.busy || s.protected} onClick={() => void remove(s, true)}><Trash2 aria-hidden />删除快照</button>
</div>
</div>)}
</section>
{restoreTarget && <RestoreDialog backup={restoreTarget} busy={action.busy} feedback={action.feedback} onClose={() => setRestoreTarget(undefined)} onSubmit={async password => {
const ok = await restore(restoreTarget, password);
if (ok) setRestoreTarget(undefined);
}} />}
{snapshotTarget && <SnapshotRestoreDialog snapshot={snapshotTarget} busy={action.busy} feedback={action.feedback} onClose={() => setSnapshotTarget(undefined)} onSubmit={async (password, allowUnknownVersion) => {
const ok = await restoreSnapshot(snapshotTarget, password, allowUnknownVersion);
if (ok) setSnapshotTarget(undefined);
}} />}
</section>; </section>;
} }
function SnapshotRestoreDialog({snapshot, busy, feedback, onClose, onSubmit}: {snapshot: Snapshot; busy: boolean; feedback: ReactNode; onClose: () => void; onSubmit: (password: string, allowUnknownVersion: boolean) => Promise<void>}) {
const [password, setPassword] = useState('');
const [allowUnknownVersion, setAllowUnknownVersion] = useState(false);
const unknownVersion = !snapshot.source_image || !snapshot.source_version;
const disabled = busy || !password || snapshot.protected || !snapshot.restorable || (unknownVersion && !allowUnknownVersion);
async function submit(e: FormEvent) {
e.preventDefault();
if (disabled) return;
try { await onSubmit(password, unknownVersion && allowUnknownVersion); }
finally { setPassword(''); }
}
return <DialogRoot open onOpenChange={open => { if (!open && !busy) onClose(); }}>
<DialogContent title={'恢复完整 home 快照 · ' + snapshot.id}>
<form className="grid gap-4" onSubmit={submit}>
{feedback}
<AlertBox variant="destructive">此操作将覆盖完整 home,包括配置、依赖及工作文件,未保存的编辑将丢失。服务器会先保留当前 home 作为失败回滚副本,并保留原始快照;按需临时停止实例,完成就绪验证后恢复原运行状态。不会自动降级或切换当前镜像。</AlertBox>
<p className="m-0 text-xs text-muted-foreground">来源版本:{snapshot.source_version || '未知'}</p>
{unknownVersion && <label className="flex items-start gap-2 text-[13px]">
<input type="checkbox" checked={allowUnknownVersion} onChange={e => setAllowUnknownVersion(e.target.checked)} disabled={busy} />
我理解来源镜像或版本未知,无法确认兼容性,仍要恢复。
</label>}
<label className="grid gap-1.5 text-[13px] font-medium">当前登录密码
<input className={inputClass} type="password" autoComplete="current-password" required value={password} onChange={e => setPassword(e.target.value)} disabled={busy} autoFocus /></label>
<DialogFooter className="border-t-0 pt-0">
<button type="button" className={cn(buttonVariants({variant: 'outline'}))} disabled={busy} onClick={onClose}>取消</button>
<button className={cn(buttonVariants({variant: 'destructive'}))} disabled={disabled}>{busy ? '正在恢复…' : '确认覆盖并恢复快照'}</button>
</DialogFooter>
</form>
</DialogContent>
</DialogRoot>;
}
function RestoreDialog({backup, busy, feedback, onClose, onSubmit}: {backup: Backup; busy: boolean; feedback: ReactNode; onClose: () => void; onSubmit: (password: string) => Promise<void>}) {
const [password, setPassword] = useState('');
return <DialogRoot open onOpenChange={open => { if (!open && !busy) onClose(); }}>
<DialogContent title={'恢复备份 · ' + backup.id}>
<form className="grid gap-4" onSubmit={e => { e.preventDefault(); void onSubmit(password); }}>
{feedback}
<AlertBox variant="info">创建于 {date(backup.created_at)} · {bytes(backup.size)}。输入创建该备份时使用的登录密码;旧平台密钥包可留空。</AlertBox>
<label className="grid gap-1.5 text-[13px] font-medium">原备份密码
<input className={inputClass} type="password" autoComplete="off" value={password} onChange={e => setPassword(e.target.value)} disabled={busy} autoFocus /></label>
<DialogFooter className="border-t-0 pt-0">
<button type="button" className={cn(buttonVariants({variant: 'outline'}))} disabled={busy} onClick={onClose}>取消</button>
<button className={cn(buttonVariants({variant: 'destructive'}))} disabled={busy}>{busy ? '正在恢复…' : '恢复此备份'}</button>
</DialogFooter>
</form>
</DialogContent>
</DialogRoot>;
}
+10 -1
View File
@@ -1,8 +1,17 @@
import { useState } from 'react'; import { useState } from 'react';
import { KeyRound } from 'lucide-react';
import { api } from './api'; import { api } from './api';
import { PasswordDialog } from './PasswordDialog'; import { PasswordDialog } from './PasswordDialog';
import { buttonVariants } from './components/ui/button';
import { cn } from './lib/utils';
export function ChangePassword({onChanged}: {onChanged: () => void}) { export function ChangePassword({onChanged}: {onChanged: () => void}) {
const [open, setOpen] = useState(false); const [open, setOpen] = useState(false);
return <><button className="account-action" onClick={() => setOpen(true)}>修改密码</button>{open && <PasswordDialog title="修改密码" currentRequired onClose={() => setOpen(false)} onSubmit={async (password, current) => {await api.request('/me/password', 'POST', {current_password:current, password}); onChanged();}}/>}</>; return <>
<button className={cn(buttonVariants({variant: 'outline', size: 'sm'}))} onClick={() => setOpen(true)}><KeyRound aria-hidden />修改密码</button>
{open && <PasswordDialog title="修改密码" currentRequired onClose={() => setOpen(false)} onSubmit={async (password, current) => {
await api.request('/me/password', 'POST', {current_password: current, password});
onChanged();
}} />}
</>;
} }
+26
View File
@@ -0,0 +1,26 @@
import { beforeEach, expect, it, vi } from 'vitest';
const hooks = vi.hoisted(() => ({slots: [] as any[], cursor: 0, cleanup: undefined as undefined | (() => void)}));
vi.mock('react', async original => ({
...await original<typeof import('react')>(),
useState(value: unknown) { const i = hooks.cursor++; return [hooks.slots[i] ?? value, (next: unknown) => { hooks.slots[i] = next; }]; },
useRef(value: unknown) { const i = hooks.cursor++; return hooks.slots[i] ??= {current: value}; },
useEffect(effect: () => () => void) { hooks.cleanup = effect(); },
}));
import { useConfirm } from './components/ConfirmDialog';
beforeEach(() => { hooks.slots = []; hooks.cursor = 0; hooks.cleanup = undefined; });
it('cancels a superseded confirmation and only accepts the current request', async () => {
const confirm = useConfirm();
const first = confirm.ask({title: 'first'});
const second = confirm.ask({title: 'second'});
expect(await first).toBe(false);
confirm.element.props.onClose(true);
expect(await second).toBe(true);
});
it('unmount cancels a pending action instead of leaving it suspended', async () => {
const confirm = useConfirm();
const pending = confirm.ask({title: 'delete'});
hooks.cleanup?.();
expect(await pending).toBe(false);
confirm.element.props.onClose(true);
expect(await pending).toBe(false);
});
+32 -27
View File
@@ -1,45 +1,50 @@
import { useEffect, useRef, useState, type FormEvent } from 'react'; import { useEffect, useRef, useState, type FormEvent } from 'react';
import { api, normalBuildAllowed, selectableImage, type Image, type Instance } from './api'; import { api, selectableImage, type Instance } from './api';
import { message, ResourceState, useAction, useResource } from './ui'; import { ResourceState, useAction, useResource } from './ui';
import { AlertBox } from './components/ui/alert';
import { buttonVariants } from './components/ui/button';
import { inputClass, selectClass, selectStyle } from './components/ui/input';
import { cn } from './lib/utils';
export function CreateInstance({close, refresh}: {close: () => void; refresh: () => Promise<void>}) { export function CreateInstance({close, refresh}: {close: () => void; refresh: () => Promise<void>}) {
const compatibility = useResource(api.compatibility);
const images = useResource(api.images), action = useAction(); const images = useResource(api.images), action = useAction();
const [mode, setMode] = useState('local'), [progress, setProgress] = useState(''); const [progress, setProgress] = useState('');
const [version, setVersion] = useState('');
const [pollError, setPollError] = useState('');
const [intents, setIntents] = useState<{id:string; name:string; version:string; status:string; error:string}[]>([]);
useEffect(() => {let live=true; const load=()=>api.request<typeof intents>('/intents').then(v=>{if(live){setIntents(v);setPollError('');}}).catch(e=>{if(live)setPollError(message(e));}); void load(); const timer=window.setInterval(load,2000); return ()=>{live=false;window.clearInterval(timer);};}, []);
const active = useRef(true); const active = useRef(true);
useEffect(() => {active.current = true; return () => {active.current = false;};}, []); useEffect(() => { active.current = true; return () => { active.current = false; }; }, []);
const available = images.data?.filter(selectableImage) || []; const available = images.data?.filter(selectableImage) || [];
async function submit(e: FormEvent<HTMLFormElement>) { async function submit(e: FormEvent<HTMLFormElement>) {
e.preventDefault(); e.preventDefault();
const form = new FormData(e.currentTarget), name = String(form.get('name')); const form = new FormData(e.currentTarget), name = String(form.get('name'));
await action.run(async () => { await action.run(async () => {
let image: Image | undefined = available.find(i => (i.image_id || i.id) === form.get('image')); const image = available.find(i => (i.image_id || i.id) === form.get('image'));
if (mode === 'latest') {
if (!version) {
const latest = await api.request<{version:string; build_allowed:boolean}>('/latest');
if (!latest.version) throw new Error('服务器未返回精确版本。');
setVersion(latest.version);
if (!latest.build_allowed || !normalBuildAllowed(await api.compatibility(), latest.version)) throw new Error('此最新版不支持普通构建。管理员可前往设置 → 镜像与构建进行实验构建,但实验镜像不能直接用于创建。');
return;
}
if (!normalBuildAllowed(await api.compatibility(), version)) throw new Error('此版本不支持普通构建;实验镜像不能直接用于创建。');
const intent = await api.request<{id:string}>('/intents', 'POST', {name, version});
if (!intent.id) throw new Error('未收到意图确认,请刷新核实,勿重复提交。');
setIntents(await api.request<typeof intents>('/intents'));
await refresh(); return;
}
if (!active.current) return; if (!active.current) return;
if (!image) throw new Error('请选择可用的本地镜像。'); if (!image) throw new Error('请选择可用的本地镜像。');
setProgress('正在创建实例,请勿重复提交…'); setProgress('正在创建实例,请勿重复提交…');
const created = await api.request<Instance>('/instances', 'POST', {name, image_id:image.image_id || image.id, version:image.version}); const created = await api.request<Instance>('/instances', 'POST', {name, image_id: image.image_id || image.id});
if (!created?.id) throw new Error('未收到实例确认,请刷新实例列表核实,勿重复提交。'); if (!created?.id) throw new Error('未收到实例确认,请刷新实例列表核实,勿重复提交。');
if (active.current) {await refresh(); close();} if (active.current) { await refresh(); close(); }
}); });
if (active.current) setProgress(''); if (active.current) setProgress('');
} }
return <form className="inline-form" onSubmit={submit}><h3>创建独立实例</h3>{action.feedback}<p className="callout">确认后服务器持久预留一个名额(每用户最多两个)。刷新或关闭不会取消自动创建;失败释放名额,不回退其他版本。取消等待不会取消共享构建,创建开始后不可撤回。</p><fieldset disabled={action.busy}><label>实例名称<input name="name" required maxLength={64}/></label><label>创建方式<select value={mode} onChange={e => {setMode(e.target.value); setVersion('');}}><option value="local">选择本地镜像</option><option value="latest">构建最新版并创建</option></select></label>{mode === 'local' && <><ResourceState resource={images}/><label>本地镜像<select name="image" required defaultValue=""><option value="" disabled>选择已构建镜像</option>{available.map(i => <option key={i.image_id || i.id} value={i.image_id || i.id}>{i.version} · {i.tag || i.image_id || i.id}</option>)}</select></label>{!images.loading && !available.length && <p>暂无已准入的可用镜像。历史或实验镜像不可用于创建。</p>}</>}{mode === 'latest' && <ResourceState resource={compatibility}/>} {mode === 'latest' && version && compatibility.data && !normalBuildAllowed(compatibility.data, version) && <p className="alert">此最新版不支持普通构建。管理员可前往设置 → 镜像与构建实验构建;实验镜像不能直接用于创建。</p>}{mode === 'latest' && version && compatibility.data && normalBuildAllowed(compatibility.data, version) && <p>将锁定精确版本 <strong>{version}</strong>,确认后预留名额并构建创建。</p>}<button className="primary" disabled={mode === 'local' ? !available.length : !!version && (!compatibility.data || !normalBuildAllowed(compatibility.data, version))}>{mode === 'local' ? '确认创建' : version ? '确认版本并预留创建' : '查询最新版'}</button></fieldset>{progress && <p role="status">{progress}</p>}{pollError && <p className="muted" role="alert">意图刷新失败:{pollError}</p>}<ul>{intents.map(i => <li key={i.id}>{i.name} · {i.version} · {i.status} {i.error}{i.status === 'waiting' && <button type="button" disabled={action.busy} onClick={() => void action.run(async () => {await api.request('/intents/' + i.id, 'DELETE'); setIntents(await api.request<typeof intents>('/intents')); await refresh();})}>取消等待并释放名额</button>}{i.status === 'success' && <button type="button" onClick={() => void refresh()}>刷新实例列表</button>}</li>)}</ul><button type="button" disabled={action.busy} onClick={close}>关闭(保留已提交意图)</button></form>; return <form className="mb-5 grid gap-4 rounded-lg border border-border bg-muted/50 p-4" onSubmit={submit}>
<h3 className="m-0 text-[14px] font-semibold">创建独立实例</h3>
{action.feedback}
<AlertBox variant="info">请选择已验证并准入的本地镜像。创建使用所选镜像的精确 ID,不会自动构建或切换版本。</AlertBox>
<fieldset disabled={action.busy} className="m-0 grid min-w-0 gap-4 border-0 p-0">
<label className="grid gap-1.5 text-[13px] font-medium">实例名称<input className={inputClass} name="name" required maxLength={64} /></label>
<ResourceState resource={images} />
<label className="grid gap-1.5 text-[13px] font-medium">本地镜像
<select className={selectClass} style={selectStyle} name="image" required defaultValue="">
<option value="" disabled>选择已验证镜像</option>
{available.map(i => <option key={i.image_id || i.id} value={i.image_id || i.id}>{i.version} · {i.tag || i.image_id || i.id}</option>)}
</select>
</label>
{!images.loading && !available.length && <p className="m-0 text-[13px] text-muted-foreground">暂无已准入的可用镜像。请联系管理员构建并验证发布镜像。</p>}
<div className="flex flex-wrap gap-2">
<button className={cn(buttonVariants())} disabled={!available.length}>确认创建</button>
<button type="button" className={cn(buttonVariants({variant: 'ghost'}))} disabled={action.busy} onClick={close}>关闭</button>
</div>
</fieldset>
{progress && <p role="status" className="m-0 text-[13px] text-muted-foreground">{progress}</p>}
</form>;
} }
+12 -11
View File
@@ -1,14 +1,15 @@
import { useEffect, useId, useRef, type ReactNode } from 'react'; import type { ReactNode } from 'react';
import { DialogContent, DialogRoot } from './components/ui/dialog';
export function Dialog({title, busy, onClose, children}: {title: string; busy: boolean; onClose: () => void; children: ReactNode}) { export function Dialog({title, busy, onClose, children}: {title: string; busy: boolean; onClose: () => void; children: ReactNode}) {
const ref = useRef<HTMLDialogElement>(null), titleId = useId(); return <DialogRoot open onOpenChange={open => { if (!open && !busy) onClose(); }}>
useEffect(() => { <DialogContent title={title} aria-busy={busy} onOpenAutoFocus={e => {
const trigger = document.activeElement as HTMLElement | null, dialog = ref.current!; // Focus the first input when the dialog opens, mirroring the old native dialog.
dialog.showModal(); e.preventDefault();
dialog.querySelector<HTMLInputElement>('input')?.focus(); const content = e.currentTarget as HTMLElement;
return () => {dialog.close(); if (trigger?.isConnected) trigger.focus();}; content.querySelector<HTMLInputElement>('input')?.focus();
}, []); }}>
return <dialog ref={ref} className="account-dialog" aria-labelledby={titleId} aria-busy={busy} onCancel={event => {event.preventDefault(); if (!busy) onClose();}}> {children}
<div className="dialog-heading"><h2 id={titleId}>{title}</h2><button type="button" aria-label="关闭对话框" disabled={busy} onClick={onClose}>关闭</button></div>{children} </DialogContent>
</dialog>; </DialogRoot>;
} }
+169
View File
@@ -0,0 +1,169 @@
import { afterEach, beforeEach, expect, it, vi } from 'vitest';
const state = vi.hoisted(() => ({slots: [] as any[], cursor: 0, effects: [] as (() => void)[]}));
vi.mock('react', async importOriginal => ({
...await importOriginal<typeof import('react')>(),
useState(value: unknown) {
const i = state.cursor++;
if (!(i in state.slots)) state.slots[i] = value;
return [state.slots[i], (next: unknown) => { state.slots[i] = next; }];
},
useRef(value: unknown) { const i = state.cursor++; return state.slots[i] ??= {current: value}; },
useEffect(effect: () => void | (() => void), deps: unknown[]) {
const i = state.cursor++;
const previous = state.slots[i];
if (!previous || deps.some((value, index) => !Object.is(value, previous.deps[index]))) {
state.effects.push(() => {
previous?.cleanup?.();
state.slots[i] = {deps, cleanup: effect()};
});
}
},
}));
vi.mock('./api', async importOriginal => ({
...await importOriginal<typeof import('./api')>(),
api: {readText: vi.fn(), saveText: vi.fn()},
}));
import { FileEditor } from './FileEditor';
import { api, ApiError } from './api';
import { Button } from './components/ui/button';
import { DialogRoot } from './components/ui/dialog';
import { maxTextBytes } from './fileEditor';
const props = {path: 'nested/config.yaml', instanceId: 'instance-two', onClose: vi.fn(), onSaved: vi.fn()};
function render() {
state.cursor = 0;
const tree = FileEditor(props);
state.effects.splice(0).forEach(effect => effect());
return tree;
}
function nodes(root: any, predicate: (node: any) => boolean): any[] {
if (Array.isArray(root)) return root.flatMap(child => nodes(child, predicate));
if (!root || typeof root !== 'object') return [];
return [...(predicate(root) ? [root] : []), ...nodes(root.props?.children, predicate)];
}
const button = (text: string) => nodes(render(), n => n.type === Button && n.props.children === text)[0];
const editor = () => nodes(render(), n => n.props?.label === '文件内容 · ' + props.path)[0];
const flush = () => new Promise(resolve => setTimeout(resolve, 0));
async function load() { render(); await flush(); return render(); }
function unmount() { state.slots.forEach(slot => slot?.cleanup?.()); state.slots = []; }
beforeEach(() => {
state.slots = []; state.cursor = 0; state.effects = [];
vi.resetAllMocks();
vi.stubGlobal('window', {confirm: vi.fn(() => true), addEventListener: vi.fn(), removeEventListener: vi.fn()});
vi.mocked(api.readText).mockResolvedValue({content: 'name: original', version: 'version-one'});
vi.mocked(api.saveText).mockResolvedValue({version: 'version-two'});
});
afterEach(() => { unmount(); vi.unstubAllGlobals(); });
it('loads the selected path and instance, then exposes its content and language', async () => {
expect(JSON.stringify(render())).toContain('正在读取文件');
expect(button('保存').props.disabled).toBe(true);
await flush();
expect(api.readText).toHaveBeenCalledExactlyOnceWith(props.path, props.instanceId);
expect(editor().props).toMatchObject({value: 'name: original', language: 'yaml', readOnly: false});
expect(button('保存').props.disabled).toBe(true);
});
it('saves against the loaded version, path and instance and uses the returned version next time', async () => {
await load();
editor().props.onChange('name: edited');
button('保存').props.onClick();
await flush();
expect(api.saveText).toHaveBeenCalledExactlyOnceWith(props.path, props.instanceId, 'name: edited', 'version-one');
expect(props.onSaved).toHaveBeenCalledOnce();
expect(JSON.stringify(render())).toContain('已保存');
expect(button('保存').props.disabled).toBe(true);
editor().props.onChange('name: edited again');
button('保存').props.onClick();
await flush();
expect(api.saveText).toHaveBeenLastCalledWith(props.path, props.instanceId, 'name: edited again', 'version-two');
});
it('preserves the draft and version after a conflict', async () => {
await load();
vi.mocked(api.saveText).mockRejectedValueOnce(new ApiError(409, 'conflict'));
editor().props.onChange('precious draft');
button('保存').props.onClick();
await flush();
expect(editor().props.value).toBe('precious draft');
expect(JSON.stringify(render())).toContain('当前修改已保留');
expect(props.onSaved).not.toHaveBeenCalled();
expect(button('保存').props.disabled).toBe(false);
button('保存').props.onClick();
await flush();
expect(api.saveText).toHaveBeenLastCalledWith(props.path, props.instanceId, 'precious draft', 'version-one');
});
it('canceling close, dialog dismissal, reload and save retains edits without requests', async () => {
await load();
editor().props.onChange('keep me');
vi.mocked(window.confirm).mockReturnValue(false);
button('关闭').props.onClick();
nodes(render(), n => n.type === DialogRoot)[0].props.onOpenChange(false);
button('重新加载').props.onClick();
button('保存').props.onClick();
await flush();
expect(editor().props.value).toBe('keep me');
expect(props.onClose).not.toHaveBeenCalled();
expect(api.readText).toHaveBeenCalledTimes(1);
expect(api.saveText).not.toHaveBeenCalled();
});
it('registers an unload warning only while dirty and cleans it up on reset and unmount', async () => {
await load();
expect(window.addEventListener).not.toHaveBeenCalled();
editor().props.onChange('dirty');
render();
const listener = vi.mocked(window.addEventListener).mock.calls[0][1] as (event: any) => void;
const event = {preventDefault: vi.fn(), returnValue: undefined};
listener(event);
expect(event.preventDefault).toHaveBeenCalledOnce();
expect(event.returnValue).toBe('');
editor().props.onChange('name: original');
render();
expect(window.removeEventListener).toHaveBeenCalledWith('beforeunload', listener);
editor().props.onChange('dirty again');
render();
const last = vi.mocked(window.addEventListener).mock.calls.at(-1)![1];
unmount();
expect(window.removeEventListener).toHaveBeenLastCalledWith('beforeunload', last);
});
it.each([[413, '文件超过 1 MiB'], [415, '不是支持的 UTF-8 文本文件']])('shows a useful read error for status %s', async (status, text) => {
vi.mocked(api.readText).mockRejectedValueOnce(new ApiError(status as number, 'read failed'));
await load();
expect(JSON.stringify(render())).toContain(text);
expect(editor()).toBeUndefined();
expect(button('保存').props.disabled).toBe(true);
});
it('counts UTF-8 bytes and prevents oversize drafts from saving', async () => {
await load();
editor().props.onChange('界'.repeat(Math.floor(maxTextBytes / 3) + 1));
expect(button('保存').props.disabled).toBe(true);
expect(JSON.stringify(render())).toContain('内容超过 1 MiB');
button('保存').props.onClick();
expect(api.saveText).not.toHaveBeenCalled();
});
it('locks duplicate saves and close/reload actions while the write is pending', async () => {
await load();
let resolve!: (result: {version: string}) => void;
vi.mocked(api.saveText).mockReturnValueOnce(new Promise(done => { resolve = done; }));
editor().props.onChange('pending draft');
const save = button('保存').props.onClick;
save(); save();
expect(api.saveText).toHaveBeenCalledTimes(1);
expect(window.confirm).toHaveBeenCalledTimes(1);
expect(editor().props.readOnly).toBe(true);
for (const text of ['保存', '关闭', '重新加载']) expect(button(text).props.disabled).toBe(true);
button('关闭').props.onClick();
button('重新加载').props.onClick();
expect(props.onClose).not.toHaveBeenCalled();
expect(api.readText).toHaveBeenCalledTimes(1);
resolve({version: 'saved-version'});
await flush();
expect(editor().props.readOnly).toBe(false);
expect(props.onSaved).toHaveBeenCalledOnce();
});
+75
View File
@@ -0,0 +1,75 @@
import { lazy, Suspense, useEffect, useRef, useState } from 'react';
import { api, ApiError, type TextFile } from './api';
import { Alert, message } from './ui';
import { isSensitiveFile } from './fileWarnings';
import { fileLanguage, maxTextBytes } from './fileEditor';
import { DialogContent, DialogFooter, DialogRoot } from './components/ui/dialog';
import { Button } from './components/ui/button';
const Editor = lazy(() => import('./MonacoProfileEditor'));
export function FileEditor({path, instanceId, onClose, onSaved}: {path: string; instanceId: string; onClose: () => void; onSaved: () => void}) {
const [original, setOriginal] = useState<TextFile>();
const [draft, setDraft] = useState('');
const [loading, setLoading] = useState(true);
const [saving, setSaving] = useState(false);
const [error, setError] = useState('');
const [notice, setNotice] = useState('');
const [reload, setReload] = useState(0);
const lock = useRef(false);
const dirty = !!original && draft !== original.content;
const size = new TextEncoder().encode(draft).length;
useEffect(() => {
let active = true;
setLoading(true); setError(''); setNotice('');
api.readText(path, instanceId).then(file => {
if (active) { setOriginal(file); setDraft(file.content); }
}).catch(e => {
if (active) setError(e instanceof ApiError && e.status === 413 ? '文件超过 1 MiB,请下载后编辑。'
: e instanceof ApiError && e.status === 415 ? '不是支持的 UTF-8 文本文件,请下载查看。' : message(e));
}).finally(() => { if (active) setLoading(false); });
return () => { active = false; };
}, [path, instanceId, reload]);
useEffect(() => {
if (!dirty && !saving) return;
const warn = (e: BeforeUnloadEvent) => { e.preventDefault(); e.returnValue = ''; };
window.addEventListener('beforeunload', warn);
return () => window.removeEventListener('beforeunload', warn);
}, [dirty, saving]);
function close() {
if (!lock.current && (!dirty || window.confirm('有未保存的修改,确认放弃并关闭?'))) onClose();
}
function reloadFile() {
if (lock.current || (dirty && !window.confirm('重新加载会丢弃当前修改,是否继续?'))) return;
setReload(reload + 1);
}
async function save() {
if (!original || !dirty || loading || lock.current || size > maxTextBytes) return;
if (!window.confirm('确认保存「' + path + '」?将替换文件内容。文件可能包含配置或密钥,建议先停止实例并备份;这里不执行 YAML 校验,也不强制停机。')) return;
lock.current = true; setSaving(true); setError(''); setNotice('');
try {
const result = await api.saveText(path, instanceId, draft, original.version);
setOriginal({content: draft, version: result.version}); setNotice('已保存'); onSaved();
} catch (e) {
setError(e instanceof ApiError && e.status === 409
? '保存被拒绝:文件已变化、被删除,或实例需要恢复。当前修改已保留;请先复制需要的内容,再重新加载核对。'
: e instanceof ApiError && e.status === 413 ? '文件超过大小限制或存储配额不足,修改已保留。' : message(e));
} finally { lock.current = false; setSaving(false); }
}
return <DialogRoot open onOpenChange={open => { if (!open) close(); }}>
<DialogContent title={'编辑文件 · ' + path} className="w-[min(70rem,calc(100vw-1.5rem))]" aria-busy={saving} onInteractOutside={e => e.preventDefault()}>
<p className="text-sm text-muted-foreground">UTF-8 文本,最大 1 MiB。{isSensitiveFile(path) ? '此文件可能包含凭据或敏感配置,请谨慎修改。' : '文件可能包含敏感信息,请勿公开分享。'}建议修改前停机并备份。</p>
{error && <Alert>{error}</Alert>}
{loading ? <p role="status">正在读取文件…</p> : original && <Suspense fallback={<p role="status">正在加载编辑器…</p>}>
<Editor value={draft} onChange={setDraft} language={fileLanguage(path)} label={'文件内容 · ' + path} readOnly={saving} />
</Suspense>}
{size > maxTextBytes && <Alert>内容超过 1 MiB,无法保存。</Alert>}
<p role="status" className="mt-2 text-sm text-muted-foreground">{saving ? '正在保存…' : dirty ? '有未保存的修改' : notice}</p>
<DialogFooter>
<Button variant="outline" disabled={saving || loading} onClick={reloadFile}>重新加载</Button>
<Button variant="outline" disabled={saving} onClick={close}>关闭</Button>
<Button disabled={saving || loading || !dirty || size > maxTextBytes} onClick={() => void save()}>保存</Button>
</DialogFooter>
</DialogContent>
</DialogRoot>;
}
+118
View File
@@ -0,0 +1,118 @@
import { afterEach, beforeEach, expect, it, vi } from 'vitest';
const state = vi.hoisted(() => ({slots: [] as any[], cursor: 0}));
vi.mock('react', async importOriginal => ({
...await importOriginal<typeof import('react')>(),
useState(value: unknown) {
const i = state.cursor++;
if (!(i in state.slots)) state.slots[i] = value;
return [state.slots[i], (next: unknown) => { state.slots[i] = next; }];
},
}));
vi.mock('./api', () => ({api: {me: vi.fn()}}));
import { ImagePreview } from './ImagePreview';
import { api } from './api';
import { Button } from './components/ui/button';
import { DialogRoot } from './components/ui/dialog';
import { imagePreviewURL } from './imagePreview';
import { downloadWarning } from './fileWarnings';
const props = {path: 'nested/a &b.SVG', instanceId: 'instance two', onClose: vi.fn()};
function render() { state.cursor = 0; return ImagePreview(props); }
function nodes(root: any, predicate: (node: any) => boolean): any[] {
if (Array.isArray(root)) return root.flatMap(child => nodes(child, predicate));
if (!root || typeof root !== 'object') return [];
return [...(predicate(root) ? [root] : []), ...nodes(root.props?.children, predicate)];
}
const button = (text: string) => nodes(render(), n => n.type === Button && (n.props.children === text || n.props['aria-label'] === text))[0];
const image = () => nodes(render(), n => n.type === 'img')[0];
const load = () => image().props.onLoad({currentTarget: {naturalWidth: 800, naturalHeight: 600}});
const flush = () => new Promise(resolve => setTimeout(resolve, 0));
beforeEach(() => {
state.slots = []; state.cursor = 0;
vi.resetAllMocks();
vi.stubGlobal('window', {confirm: vi.fn(() => true), location: {assign: vi.fn()}});
});
afterEach(() => vi.unstubAllGlobals());
it('loads the encoded image as an img and shows natural dimensions after loading', () => {
expect(image().props).toMatchObject({src: imagePreviewURL(props.path, props.instanceId) + '&revision=0', alt: props.path, referrerPolicy: 'no-referrer'});
expect(nodes(render(), n => n.props?.role === 'status')).toHaveLength(1);
for (const label of ['原始尺寸', '缩小图片', '放大图片']) expect(button(label).props.disabled).toBe(true);
expect(nodes(render(), n => ['iframe', 'object', 'embed'].includes(n.type))).toHaveLength(0);
load();
expect(nodes(render(), n => n.props?.role === 'status')).toHaveLength(0);
expect(JSON.stringify(render())).toContain('800 × 600');
expect(button('原始尺寸').props.disabled).toBe(false);
});
it('zooms within bounds, resets to original size, and returns to fit mode', () => {
load();
expect(image().props.style).toMatchObject({maxWidth: '100%', maxHeight: '100%', width: 'auto'});
button('放大图片').props.onClick();
expect(image().props.style.width).toBe(1200);
for (let i = 0; i < 10; i++) button('放大图片').props.onClick();
expect(image().props.style.width).toBe(3200);
expect(button('放大图片').props.disabled).toBe(true);
for (let i = 0; i < 20; i++) button('缩小图片').props.onClick();
expect(image().props.style.width).toBe(80);
expect(button('缩小图片').props.disabled).toBe(true);
button('原始尺寸').props.onClick();
expect(image().props.style.width).toBe(800);
expect(JSON.stringify(render())).toContain('100%');
button('适应窗口').props.onClick();
expect(image().props.style).toMatchObject({maxWidth: '100%', maxHeight: '100%', width: 'auto'});
});
it('shows load errors and retries only after checking the session, clearing stale dimensions', async () => {
load();
image().props.onError();
expect(image()).toBeUndefined();
expect(nodes(render(), n => n.props?.role === 'alert')).toHaveLength(1);
expect(JSON.stringify(render())).toContain('16 MiB');
expect(button('原始尺寸').props.disabled).toBe(true);
let resolve!: () => void;
vi.mocked(api.me).mockReturnValueOnce(new Promise(done => { resolve = () => done({} as any); }));
button('重试').props.onClick();
expect(api.me).toHaveBeenCalledOnce();
expect(button('重试').props.disabled).toBe(true);
expect(image()).toBeUndefined();
resolve();
await flush();
expect(image().props.src).toBe(imagePreviewURL(props.path, props.instanceId) + '&revision=1');
expect(nodes(render(), n => n.props?.role === 'status')).toHaveLength(1);
expect(JSON.stringify(render())).not.toContain('800 × 600');
load();
expect(button('原始尺寸').props.disabled).toBe(false);
});
it('keeps the error visible when session recovery fails and permits another retry', async () => {
image().props.onError();
vi.mocked(api.me).mockRejectedValueOnce(new Error('expired'));
button('重试').props.onClick();
await flush();
expect(image()).toBeUndefined();
expect(nodes(render(), n => n.props?.role === 'alert')).toHaveLength(1);
expect(button('重试').props.disabled).toBe(false);
button('重试').props.onClick();
await flush();
expect(api.me).toHaveBeenCalledTimes(2);
expect(image()).toBeDefined();
});
it('requires download confirmation and preserves the selected path and instance', () => {
vi.mocked(window.confirm).mockReturnValueOnce(false);
button('下载图片').props.onClick();
expect(window.confirm).toHaveBeenCalledExactlyOnceWith(downloadWarning(props.path, false));
expect(window.location.assign).not.toHaveBeenCalled();
button('下载图片').props.onClick();
expect(window.location.assign).toHaveBeenCalledExactlyOnceWith(imagePreviewURL(props.path, props.instanceId));
});
it('closes from the footer and dialog dismissal but not an open notification', () => {
nodes(render(), n => n.type === DialogRoot)[0].props.onOpenChange(true);
expect(props.onClose).not.toHaveBeenCalled();
button('关闭').props.onClick();
nodes(render(), n => n.type === DialogRoot)[0].props.onOpenChange(false);
expect(props.onClose).toHaveBeenCalledTimes(2);
});
+43
View File
@@ -0,0 +1,43 @@
import { useState } from 'react';
import { DialogContent, DialogFooter, DialogRoot } from './components/ui/dialog';
import { Button } from './components/ui/button';
import { imagePreviewURL } from './imagePreview';
import { api } from './api';
import { downloadWarning } from './fileWarnings';
export function ImagePreview({path, instanceId, onClose}: {path: string; instanceId: string; onClose: () => void}) {
const [scale, setScale] = useState<number | null>(null);
const [size, setSize] = useState<{width: number; height: number}>();
const [error, setError] = useState(false);
const [revision, setRevision] = useState(0);
const [retrying, setRetrying] = useState(false);
async function retry() {
setRetrying(true);
try { await api.me(); setError(false); setSize(undefined); setRevision(revision + 1); }
catch { setError(true); }
finally { setRetrying(false); }
}
function download() {
if (window.confirm(downloadWarning(path, false))) window.location.assign(imagePreviewURL(path, instanceId));
}
return <DialogRoot open onOpenChange={open => { if (!open) onClose(); }}>
<DialogContent title={'图片预览 · ' + path} className="w-[min(70rem,calc(100vw-1.5rem))]">
<div className="mb-3 flex flex-wrap items-center gap-2">
<Button variant="outline" onClick={() => setScale(null)}>适应窗口</Button>
<Button variant="outline" disabled={!size || error} onClick={() => setScale(1)}>原始尺寸</Button>
<Button variant="outline" aria-label="缩小图片" disabled={!size || error || (scale !== null && scale <= 0.1)} onClick={() => setScale(Math.max(0.1, (scale ?? 1) / 1.5))}>−</Button>
<Button variant="outline" aria-label="放大图片" disabled={!size || error || (scale !== null && scale >= 4)} onClick={() => setScale(Math.min(4, (scale ?? 1) * 1.5))}>+</Button>
<span className="text-sm text-muted-foreground">{scale === null ? '适应窗口' : Math.round(scale * 100) + '%'}{size && ' · ' + size.width + ' × ' + size.height}</span>
</div>
{error ? <div role="alert" className="rounded-lg bg-destructive-muted p-4 text-destructive">图片无法预览:可能已删除、格式损坏、超过 16 MiB 或登录已失效。<Button variant="outline" className="ml-2" disabled={retrying} onClick={() => void retry()}>重试</Button></div>
: <div className="h-[55dvh] overflow-auto rounded-lg border border-border bg-muted p-2">
{!size && <p role="status">正在加载图片…</p>}
<img key={revision} src={imagePreviewURL(path, instanceId) + '&revision=' + revision} alt={path} referrerPolicy="no-referrer"
className="block" style={scale === null ? {maxWidth: '100%', maxHeight: '100%', width: 'auto', height: 'auto', margin: 'auto'} : {maxWidth: 'none', width: size ? size.width * scale : undefined, height: 'auto'}}
onLoad={e => setSize({width: e.currentTarget.naturalWidth, height: e.currentTarget.naturalHeight})} onError={() => setError(true)} />
</div>}
<p className="mt-3 text-xs text-muted-foreground">支持 PNG、JPEG、GIF、WebP、SVG,最大 16 MiB。SVG 仅以图片展示,不执行脚本。图片也可能包含敏感信息,请勿公开分享。</p>
<DialogFooter><Button variant="outline" onClick={download}>下载图片</Button><Button onClick={onClose}>关闭</Button></DialogFooter>
</DialogContent>
</DialogRoot>;
}
+65 -30
View File
@@ -1,5 +1,5 @@
import { afterEach, beforeEach, expect, it, vi } from 'vitest'; import { afterEach, beforeEach, expect, it, vi } from 'vitest';
const state = vi.hoisted(() => ({slots: [] as any[], cursor: 0})); const state = vi.hoisted(() => ({slots: [] as any[], cursor: 0, emptyImages: false}));
vi.mock('react', async original => ({ vi.mock('react', async original => ({
...await original<typeof import('react')>(), ...await original<typeof import('react')>(),
useState(value: unknown) {const i = state.cursor++; if (!(i in state.slots)) state.slots[i] = value; return [state.slots[i], (next: unknown) => {state.slots[i] = next;}];}, useState(value: unknown) {const i = state.cursor++; if (!(i in state.slots)) state.slots[i] = value; return [state.slots[i], (next: unknown) => {state.slots[i] = next;}];},
@@ -8,36 +8,50 @@ vi.mock('react', async original => ({
})); }));
vi.mock('./ui', async original => ({ vi.mock('./ui', async original => ({
...await original<typeof import('./ui')>(), ...await original<typeof import('./ui')>(),
useResource: (load: unknown) => ({data: load === api.compatibility ? policy : load === api.images ? [{id:'ok', version:'1.2.3', status:'available', selectable:true}, {id:'exp', version:'9.9.9', status:'experimental', selectable:false}] : [], loading:false, refresh:vi.fn(async () => {})}), useResource: (load: unknown) => ({data: load === api.images ? state.emptyImages ? [] : [{id:'ok', version:'1.2.3', status:'available', selectable:true}, {id:'exp', version:'9.9.9', status:'experimental', selectable:false}] : [], loading:false, refresh:vi.fn(async () => {})}),
})); }));
import { api, type Compatibility } from './api'; import { api } from './api';
import { Images } from './Images'; import { Images } from './Images';
import { ReleasePanel } from './ReleasePanel';
import { CreateInstance } from './CreateInstance'; import { CreateInstance } from './CreateInstance';
vi.mock('./ProfileEditor', () => ({ProfileEditor: () => null})); vi.mock('./ProfileEditor', () => ({ProfileEditor: () => null}));
import { Instances } from './Instances'; import { Instances } from './Instances';
const policy: Compatibility = {recipe_rev:'7', platform:'linux/amd64', entries:[], unknown:{state:'existing_only', build_allowed:false, reason:'unknown'}, experimental_admin_only:true}; import { ConfirmDialog } from './components/ConfirmDialog';
function nodes(root: any, predicate: (node: any) => boolean): any[] { function nodes(root: any, predicate: (node: any) => boolean): any[] {
if (Array.isArray(root)) return root.flatMap(child => nodes(child, predicate)); if (Array.isArray(root)) return root.flatMap(child => nodes(child, predicate));
if (!root || typeof root !== 'object') return []; if (!root || typeof root !== 'object') return [];
return [...(predicate(root) ? [root] : []), ...nodes(root.props?.children, predicate)]; return [...(predicate(root) ? [root] : []), ...nodes(root.props?.children, predicate)];
} }
function render(admin = false) {state.cursor = 0; return Images({admin});} function render(admin = false) {state.cursor = 0; return Images({admin});}
beforeEach(() => {state.slots = []; state.cursor = 0; vi.stubGlobal('window', {confirm:vi.fn(() => false)});}); // In-app confirm dialog replaced window.confirm: resolve it via onClose.
const dialog = (root: any) => nodes(root, n => n.type === ConfirmDialog)[0];
const approve = (root: any) => dialog(root).props.onClose(true);
const flush = () => new Promise(resolve => setTimeout(resolve, 0));
beforeEach(() => {state.slots = []; state.cursor = 0; state.emptyImages = false; vi.stubGlobal('window', {});});
afterEach(() => {vi.restoreAllMocks(); vi.unstubAllGlobals();}); afterEach(() => {vi.restoreAllMocks(); vi.unstubAllGlobals();});
it('hides experimental and admission controls from ordinary users', () => { it('hides experimental and admission controls from ordinary users', () => {
const tree = render(); const tree = render();
expect(nodes(tree, n => n.type === ReleasePanel)).toHaveLength(0);
expect(nodes(render(true), n => n.type === ReleasePanel)).toHaveLength(1);
expect(nodes(tree, n => n.type === 'input' && n.props.type === 'checkbox')).toHaveLength(0); expect(nodes(tree, n => n.type === 'input' && n.props.type === 'checkbox')).toHaveLength(0);
expect(nodes(tree, n => n.type === 'button' && n.props.children === '准入管理')).toHaveLength(0); expect(nodes(tree, n => n.type === 'button' && n.props.children === '准入管理')).toHaveLength(0);
expect(nodes(render(true), n => n.type === 'button' && n.props.children === '准入管理')).toHaveLength(2); // Each image row renders twice: once in the desktop table, once as a mobile card.
expect(nodes(render(true), n => n.type === 'button' && n.props.children === '准入管理')).toHaveLength(4);
}); });
it('canceled build confirmation sends zero requests', () => { it.each([false, true])('has no legacy build controls (admin=%s)', admin => {
const request = vi.spyOn(api, 'startBuild'); const tree = render(admin);
const check = vi.spyOn(api, 'compatibility'); expect(nodes(tree, n => n.type === 'form' || n.type === 'input')).toHaveLength(0);
nodes(render(true), n => n.type === 'form')[0].props.onSubmit({preventDefault:vi.fn()}); expect(JSON.stringify(tree)).not.toMatch(/查询 latest|精确 DSH 版本|提交实验构建/);
expect(window.confirm).toHaveBeenCalledOnce();
expect(request).not.toHaveBeenCalled(); expect(check).not.toHaveBeenCalled();
}); });
it('requires evidence and confirmation; cancel sends no admission mutation', () => { it('mounts only one selected admission editor outside responsive branches', () => {
nodes(render(true), n => n.type === 'button' && n.props.children === '准入管理')[0].props.onClick();
const tree = render(true);
expect(nodes(tree, n => typeof n.type === 'function' && n.props.imageId)).toHaveLength(1);
for (const branch of nodes(tree, n => typeof n.props?.className === 'string' && /md:hidden|md:block/.test(n.props.className))) {
expect(nodes(branch, n => typeof n.type === 'function' && n.props.imageId)).toHaveLength(0);
}
});
it('requires evidence and confirmation; cancel sends no admission mutation', async () => {
const request = vi.spyOn(api, 'admitImage'); const request = vi.spyOn(api, 'admitImage');
nodes(render(true), n => n.type === 'button' && n.props.children === '准入管理')[0].props.onClick(); nodes(render(true), n => n.type === 'button' && n.props.children === '准入管理')[0].props.onClick();
const child = nodes(render(true), n => typeof n.type === 'function' && n.props.imageId)[0]; const child = nodes(render(true), n => typeof n.type === 'function' && n.props.imageId)[0];
@@ -48,19 +62,34 @@ it('requires evidence and confirmation; cancel sends no admission mutation', ()
nodes(form, n => n.type === 'input')[0].props.onChange({target:{checked:true}}); nodes(form, n => n.type === 'input')[0].props.onChange({target:{checked:true}});
state.cursor = 0; form = child.type(child.props); state.cursor = 0; form = child.type(child.props);
form.props.onSubmit({preventDefault:vi.fn()}); form.props.onSubmit({preventDefault:vi.fn()});
expect(window.confirm).toHaveBeenCalledOnce(); expect(request).not.toHaveBeenCalled(); state.cursor = 0; form = child.type(child.props);
// The in-app confirm dialog opens; cancelling it sends no mutation.
expect(dialog(form).props.request.options.title).toBe('更新准入策略');
expect(request).not.toHaveBeenCalled();
dialog(form).props.onClose(false);
await flush();
expect(request).not.toHaveBeenCalled();
}); });
it('unknown latest never reserves an intent, including a second submit', async () => { it('creates with only name and exact image ID, leaving version resolution to the server', async () => {
const request = vi.spyOn(api, 'request').mockResolvedValue({version:'9.9.9', build_allowed:false}); const request = vi.spyOn(api, 'request').mockResolvedValue({id:'created'});
vi.spyOn(api, 'compatibility').mockResolvedValue(policy); vi.stubGlobal('FormData', class {get(key: string) {return key === 'image' ? 'ok' : 'name';}});
vi.stubGlobal('FormData', class {get() {return 'name';}}); const close = vi.fn(), refresh = vi.fn(async () => {});
const renderCreate = () => {state.cursor = 0; return CreateInstance({close:vi.fn(), refresh:vi.fn(async () => {})});}; const form = CreateInstance({close, refresh});
let form = renderCreate(); expect(nodes(form, n => n.type === 'select')).toHaveLength(1);
expect(nodes(form, n => n.type === 'option' && n.props.value === 'exp')).toHaveLength(0); expect(nodes(form, n => n.type === 'option' && n.props.value === 'exp')).toHaveLength(0);
nodes(form, n => n.type === 'select')[0].props.onChange({target:{value:'latest'}}); await form.props.onSubmit({preventDefault:vi.fn(), currentTarget:{}});
form = renderCreate(); await form.props.onSubmit({preventDefault:vi.fn(), currentTarget:{}}); expect(request.mock.calls).toEqual([['/instances', 'POST', {name:'name', image_id:'ok'}]]);
form = renderCreate(); await form.props.onSubmit({preventDefault:vi.fn(), currentTarget:{}}); expect(refresh).toHaveBeenCalledOnce(); expect(close).toHaveBeenCalledOnce();
expect(request.mock.calls).toEqual([['/latest']]); });
it('directs users to an administrator and disables creation when no verified image exists', async () => {
state.emptyImages = true;
const request = vi.spyOn(api, 'request');
vi.stubGlobal('FormData', class {get() {return 'exp';}});
const form = CreateInstance({close:vi.fn(), refresh:vi.fn(async () => {})});
expect(nodes(form, n => n.type === 'button' && n.props.children === '确认创建')[0].props.disabled).toBe(true);
expect(JSON.stringify(form)).toContain('请联系管理员');
await form.props.onSubmit({preventDefault:vi.fn(), currentTarget:{}});
expect(request).not.toHaveBeenCalled();
}); });
it('admission rejects blank/unconfirmed evidence, resets consent on edits, and submits once', async () => { it('admission rejects blank/unconfirmed evidence, resets consent on edits, and submits once', async () => {
const request = vi.spyOn(api, 'admitImage').mockResolvedValue({}); const request = vi.spyOn(api, 'admitImage').mockResolvedValue({});
@@ -79,9 +108,9 @@ it('admission rejects blank/unconfirmed evidence, resets consent on edits, and s
nodes(renderAdmission(), n => n.type === 'select')[0].props.onChange({target:{value:'recommended'}}); nodes(renderAdmission(), n => n.type === 'select')[0].props.onChange({target:{value:'recommended'}});
expect(nodes(renderAdmission(), n => n.type === 'input')[0].props.checked).toBe(false); expect(nodes(renderAdmission(), n => n.type === 'input')[0].props.checked).toBe(false);
nodes(renderAdmission(), n => n.type === 'input')[0].props.onChange({target:{checked:true}}); nodes(renderAdmission(), n => n.type === 'input')[0].props.onChange({target:{checked:true}});
vi.mocked(window.confirm).mockReturnValue(true);
submit(); submit(); submit(); submit();
await Promise.resolve(); approve(renderAdmission());
await flush();
expect(request).toHaveBeenCalledExactlyOnceWith('ok', 'recommended', '验收记录 #42'); expect(request).toHaveBeenCalledExactlyOnceWith('ok', 'recommended', '验收记录 #42');
}); });
it.each([ it.each([
@@ -96,7 +125,6 @@ it.each([
[' ' + 'é'.repeat(2048) + '\n', 4096, true], [' ' + 'é'.repeat(2048) + '\n', 4096, true],
] as const)('checks UTF-8 evidence bytes (case %#)', async (evidence, bytes, valid) => { ] as const)('checks UTF-8 evidence bytes (case %#)', async (evidence, bytes, valid) => {
const request = vi.spyOn(api, 'admitImage').mockResolvedValue({}); const request = vi.spyOn(api, 'admitImage').mockResolvedValue({});
vi.mocked(window.confirm).mockReturnValue(true);
nodes(render(true), n => n.type === 'button' && n.props.children === '准入管理')[0].props.onClick(); nodes(render(true), n => n.type === 'button' && n.props.children === '准入管理')[0].props.onClick();
const child = nodes(render(true), n => typeof n.type === 'function' && n.props.imageId)[0]; const child = nodes(render(true), n => typeof n.type === 'function' && n.props.imageId)[0];
state.slots = []; state.slots = [];
@@ -109,9 +137,16 @@ it.each([
expect(nodes(form, n => n.type === 'p' && Array.isArray(n.props.children) && n.props.children.includes(bytes))).toHaveLength(1); expect(nodes(form, n => n.type === 'p' && Array.isArray(n.props.children) && n.props.children.includes(bytes))).toHaveLength(1);
expect(nodes(form, n => n.type === 'button')[0].props.disabled).toBe(!valid); expect(nodes(form, n => n.type === 'button')[0].props.disabled).toBe(!valid);
form.props.onSubmit({preventDefault:vi.fn()}); form.props.onSubmit({preventDefault:vi.fn()});
await Promise.resolve(); if (valid) {
if (valid) expect(request).toHaveBeenCalledExactlyOnceWith('ok', 'compatible', evidence.trim()); // Valid evidence opens the in-app confirm dialog; approving submits once.
else {expect(request).not.toHaveBeenCalled(); expect(window.confirm).not.toHaveBeenCalled();} approve(renderAdmission());
await flush();
expect(request).toHaveBeenCalledExactlyOnceWith('ok', 'compatible', evidence.trim());
} else {
// Invalid evidence never reaches the confirmation step.
expect(dialog(renderAdmission()).props.request).toBeUndefined();
expect(request).not.toHaveBeenCalled();
}
}); });
it('upgrade selector excludes experimental and unadmitted images', () => { it('upgrade selector excludes experimental and unadmitted images', () => {
state.cursor = 0; Instances(); state.cursor = 0; Instances();
+188 -108
View File
@@ -1,123 +1,203 @@
import { useState, type FormEvent } from 'react'; import { useState } from 'react';
import { api, normalBuildAllowed, type AdmissionState, type Build } from './api'; import { ChevronLeft, ChevronRight } from 'lucide-react';
import { Panel, ResourceState, Empty, Status, date, useAction, useResource } from './ui'; import { api, type AdmissionState, type Build, type Image } from './api';
import { Panel, ResourceState, Empty, Status, date, useAction, useResource, RefreshButton, tableClass, tableHeadRowClass, thClass, tdClass, trClass } from './ui';
import { BuildProvenance, ReleasePanel } from './ReleasePanel';
import { useConfirm } from './components/ConfirmDialog';
import { AlertBox } from './components/ui/alert';
import { buttonVariants } from './components/ui/button';
import { selectClass, selectStyle, textareaClass } from './components/ui/input';
import { cn } from './lib/utils';
export const activeBuild = (build: Build) => ['queued', 'running', 'validating'].includes(build.status); export const activeBuild = (build: Build) => ['queued', 'running', 'validating'].includes(build.status);
const admissionLabels: Record<string, string> = {recommended: '推荐', compatible: '兼容', existing_only: '仅存量', blocked: '禁止使用'};
function BuildLogs({id, close}: {id: string; close: () => void}) { function BuildLogs({id, close}: {id: string; close: () => void}) {
const logs = useResource(() => api.build(id), id, 5000, activeBuild); const logs = useResource(() => api.build(id), id, 5000, activeBuild);
return <div> return <div className="mt-4 rounded-lg border border-border bg-muted/40 p-4">
<h3>构建日志</h3> <div className="mb-3 flex items-center justify-between gap-2">
<button onClick={close}>关闭日志</button> <h3 className="m-0 text-[14px] font-semibold">构建日志</h3>
<ResourceState resource={logs}/>{logs.data && <> <button className={cn(buttonVariants({variant: 'ghost', size: 'sm'}))} onClick={close}>关闭日志</button>
<p>{logs.data.experimental && <strong className="alert">实验构建 · 不自动准入</strong>}{logs.data.version} · <Status value={logs.data.status}/> </div>
</p>{logs.data.error && <p className="alert">{logs.data.error}</p>}<pre className="logs">{logs.data.logs || '任务尚未输出日志'}</pre> <ResourceState resource={logs} />
</>}</div>; {logs.data && <>
<p className="mt-0 flex flex-wrap items-center gap-2 text-[13px]">
{logs.data.experimental && <AlertBox variant="destructive" hideIcon className="inline-flex px-2 py-0.5 text-xs font-semibold">实验构建 · 不自动准入</AlertBox>}
{logs.data.version} · <Status value={logs.data.status} />
</p>
<BuildProvenance build={logs.data} />
{logs.data.error && <AlertBox variant="destructive" className="my-3">{logs.data.error}</AlertBox>}
<pre className="scrollbar-thin max-h-[26rem] min-h-32 overflow-auto whitespace-pre-wrap rounded-lg bg-nav p-4 font-mono text-xs leading-relaxed text-[#b7d8d3] wrap-anywhere">{logs.data.logs || '任务尚未输出日志'}</pre>
</>}
</div>;
} }
function Admission({imageId, refresh}: {imageId:string; refresh:() => Promise<void>}) {
function Admission({imageId, refresh}: {imageId: string; refresh: () => Promise<void>}) {
const [state, setState] = useState<AdmissionState>('compatible'), [evidence, setEvidence] = useState(''), [confirmed, setConfirmed] = useState(false); const [state, setState] = useState<AdmissionState>('compatible'), [evidence, setEvidence] = useState(''), [confirmed, setConfirmed] = useState(false);
const action = useAction(); const action = useAction();
const confirm = useConfirm();
const submittedEvidence = evidence.trim(), evidenceBytes = new TextEncoder().encode(submittedEvidence).length; const submittedEvidence = evidence.trim(), evidenceBytes = new TextEncoder().encode(submittedEvidence).length;
const evidenceInvalid = evidenceBytes === 0 || evidenceBytes > 4096; const evidenceInvalid = evidenceBytes === 0 || evidenceBytes > 4096;
return <form onSubmit={e => {e.preventDefault(); if (!confirmed || evidenceInvalid) return; void action.run(async () => {await api.admitImage(imageId, state, submittedEvidence); setEvidence(''); setConfirmed(false); await refresh();}, '镜像准入策略已更新', '确认依据实际验收结果修改此镜像准入策略?');}}>{action.feedback}<label>准入策略<select value={state} onChange={e => {setState(e.target.value as AdmissionState); setConfirmed(false);}}> return <form className="mt-3 grid gap-3 rounded-lg border border-border bg-muted/40 p-4" onSubmit={async e => {
<option value="recommended">推荐</option> e.preventDefault();
<option value="compatible">兼容</option> if (!confirmed || evidenceInvalid) return;
<option value="existing_only">仅存量</option> if (!await confirm.ask({title: '更新准入策略', confirmLabel: '确认更新', body: '确认依据实际验收结果修改此镜像准入策略?'})) return;
<option value="blocked">禁止使用</option> void action.run(async () => { await api.admitImage(imageId, state, submittedEvidence); setEvidence(''); setConfirmed(false); await refresh(); }, '镜像准入策略已更新');
}}>
{confirm.element}
{action.feedback}
<label className="grid gap-1.5 text-[13px] font-medium">准入策略
<select className={selectClass} style={selectStyle} value={state} onChange={e => { setState(e.target.value as AdmissionState); setConfirmed(false); }}>
<option value="recommended">推荐</option>
<option value="compatible">兼容</option>
<option value="existing_only">仅存量</option>
<option value="blocked">禁止使用</option>
</select> </select>
</label> </label>
<p className="callout">构建成功不等于通过验收。请记录启动、登录、终端、文件和配置等实际兼容性验收结果及可追溯依据,不可仅填写“构建成功”。单镜像准入不扩大普通构建版本白名单;未知版本仅可选择已准入的本地镜像 ID 创建或升级。</p> <AlertBox variant="info">构建成功不等于通过验收。请记录启动、登录、终端、文件和配置等实际兼容性验收结果及可追溯依据,不可仅填写“构建成功”。创建或升级须选择已准入的本地镜像 ID。</AlertBox>
<label>验收依据<textarea required aria-invalid={evidenceBytes > 4096} value={evidence} onChange={e => {setEvidence(e.target.value); setConfirmed(false);}}/> <label className="grid gap-1.5 text-[13px] font-medium">验收依据
</label> <textarea className={textareaClass} required aria-invalid={evidenceBytes > 4096} value={evidence} onChange={e => { setEvidence(e.target.value); setConfirmed(false); }} />
<p className={evidenceBytes > 4096 ? 'alert' : 'muted'} role={evidenceBytes > 4096 ? 'alert' : undefined}>验收依据:{evidenceBytes} / 4096 UTF-8 字节(去除首尾空白后);中文通常占 3 字节,表情通常占 4 字节。{evidenceBytes > 4096 && '已超出字节上限,请缩短依据。'}</p> </label>
<label> <p className={cn('m-0 text-xs', evidenceBytes > 4096 ? 'font-medium text-destructive' : 'text-muted-foreground')} role={evidenceBytes > 4096 ? 'alert' : undefined}>
<input type="checkbox" checked={confirmed} onChange={e => setConfirmed(e.target.checked)}/>我已核验依据;禁止使用还会阻止启动/重启,但不自动停机</label> 验收依据:{evidenceBytes} / 4096 UTF-8 字节(去除首尾空白后);中文通常占 3 字节,表情通常占 4 字节。{evidenceBytes > 4096 && '已超出字节上限,请缩短依据。'}
<button disabled={action.busy || !confirmed || evidenceInvalid}>确认更新准入</button> </p>
</form>; <label className="flex items-center gap-2 text-[13px] font-medium">
<input type="checkbox" className="size-4 accent-primary" checked={confirmed} onChange={e => setConfirmed(e.target.checked)} />
我已核验依据;禁止使用还会阻止启动/重启,但不自动停机
</label>
<div><button className={cn(buttonVariants({size: 'sm'}))} disabled={action.busy || !confirmed || evidenceInvalid}>确认更新准入</button></div>
</form>;
} }
export function Images({admin = false}: {admin?:boolean}) {
const [experimental, setExperimental] = useState(false), [admissionId, setAdmissionId] = useState<string>(); function ImageStatusCell({image}: {image: Image}) {
return <div className="grid gap-1 justify-items-start">
<Status value={image.status} />
<p className="m-0 text-[13px]">{admissionLabels[image.admission || ''] || '未验证'}</p>
<small className="max-w-56 text-xs text-muted-foreground wrap-anywhere">{image.admission_reason || '无准入依据'}</small>
<p className="m-0 text-xs text-muted-foreground">{image.selectable && image.status === 'available' ? '可创建 / 升级' : '不可创建 / 升级'}</p>
</div>;
}
export function Images({admin = false}: {admin?: boolean}) {
const [admissionId, setAdmissionId] = useState<string>();
const [page, setPage] = useState(0); const [page, setPage] = useState(0);
const compatibility = useResource(api.compatibility); const images = useResource(api.images), builds = useResource(() => api.builds(21, page * 20), String(page), 5000, rows => rows.slice(0, 20).some(activeBuild));
const images = useResource(api.images), builds = useResource(() => api.builds(21, page * 20), String(page), 5000, rows => rows.slice(0, 20).some(activeBuild)); const action = useAction(); const action = useAction();
const [latest, setLatest] = useState<{version:string; checked_at?:string}>(), [version,setVersion] = useState(''), [selected,setSelected] = useState<string>(); const confirm = useConfirm();
function build(e: FormEvent) {e.preventDefault(); void action.run(async () => {if (!(admin && experimental) && !normalBuildAllowed(await api.compatibility(), version)) throw new Error('此版本不支持普通构建,请管理员选择实验构建;实验构建不会自动准入。'); await api.startBuild(version, admin && experimental); if (page === 0) await builds.refresh(); else setPage(0);}, '构建任务已提交,请查看任务的实际状态。', (experimental ? '实验构建不会自动准入或用于创建。确认实验构建精确版本 ' : '确认在本机手动构建精确版本 ') + version + '?构建会消耗磁盘、网络与 CPU,不会自动升级实例。');} const [selected, setSelected] = useState<string>();
const rows = builds.data?.slice(0, 20); const rows = builds.data?.slice(0, 20);
function changePage(next: number) { setSelected(undefined); setPage(next); } function changePage(next: number) { setSelected(undefined); setPage(next); }
return <>{action.feedback}<Panel title="查询与手动构建"> const imageActions = (i: Image) => <>
<ResourceState resource={compatibility}/>{version && compatibility.data && !normalBuildAllowed(compatibility.data, version) && <p className="alert">此版本不支持普通构建。仅管理员可实验构建,不会自动准入。</p>}<div className="split"> {admin && <button className={cn(buttonVariants({variant: 'outline', size: 'sm'}))} onClick={() => setAdmissionId(admissionId === (i.image_id || i.id) ? undefined : (i.image_id || i.id))}>准入管理</button>}
<div> {admin && <button className={cn(buttonVariants({variant: 'destructive-ghost', size: 'sm'}))} disabled={action.busy || !!i.usage_count} onClick={async () => {
<p className="muted">从 npm 查询 dist-tags.latest,仅查询,不自动构建。</p> if (!await confirm.ask({title: '清理镜像', danger: true, confirmLabel: '删除镜像', body: '确认删除这个本地镜像?服务器将检查所有引用,不会强制删除。'})) return;
<button disabled={action.busy} onClick={() => action.run(async () => {const result = await api.request<{version:string; checked_at?:string}>('/latest'); setLatest(result); setVersion(result.version);}, '最新版本查询完成')}>查询 latest</button>{latest && <p> void action.run(async () => { await api.request('/images?image_id=' + encodeURIComponent(i.image_id || i.id), 'DELETE'); await images.refresh(); }, '镜像已删除');
<strong>{latest.version}</strong> <small>查询时间 {date(latest.checked_at)}</small> }}>清理镜像</button>}
</p>}</div> </>;
<form className="build-form" onSubmit={build}> return <>
<label>精确 DSH 版本<input required value={version} onChange={e => setVersion(e.target.value)} placeholder="查询 latest,或填写精确版本" pattern="[0-9]+\.[0-9]+\.[0-9]+.*" /> {confirm.element}
</label> {action.feedback}
<>{admin && <label> {admin && <ReleasePanel onBuilt={async () => { if (page === 0) await builds.refresh(); else changePage(0); }} />}
<input type="checkbox" checked={experimental} onChange={e => setExperimental(e.target.checked)}/>管理员实验构建(不自动准入)</label>}{experimental && <p className="alert" role="alert">实验镜像:构建成功后仍不能创建或升级实例,必须另行验收并显式准入。</p>}<button className="primary" disabled={action.busy || !version}>{experimental ? '提交实验构建' : '手动构建'}</button> <Panel title="本地镜像" action={<RefreshButton loading={images.loading} onClick={images.refresh} />}>
</> <AlertBox variant="info" className="mb-4">镜像仅保存在本机。仅已验证且允许选择的 available 镜像可创建或升级;existing_only 仅允许存量启停。blocked 禁止新建、升级及新的启动/重启,但不会自动停止已运行实例或断开连接。</AlertBox>
</form> <ResourceState resource={images} />
{images.data?.length === 0 && <Empty>本机暂无托管镜像。请联系管理员构建并验证发布镜像。</Empty>}
{!!images.data?.length && <>
<div className="hidden overflow-x-auto md:block">
<table className={tableClass}>
<thead><tr className={tableHeadRowClass}>
<th className={cn(thClass)}>DSH 版本</th>
<th className={cn(thClass)}>镜像标识</th>
<th className={cn(thClass)}>标签</th>
<th className={cn(thClass)}>状态</th>
<th className={cn(thClass)}>使用/操作</th>
</tr></thead>
<tbody>{images.data.map(i => <tr key={i.image_id || i.id} className={trClass}>
<td className={tdClass}><strong>{i.version}</strong></td>
<td className="max-w-64 border-b border-border px-3 py-3 align-top font-mono text-xs wrap-anywhere">{i.image_id || i.id}</td>
<td className={tdClass}>{i.tag || '—'}</td>
<td className={tdClass}><ImageStatusCell image={i} /></td>
<td className={tdClass}>
<div className="flex flex-wrap items-center gap-1.5">
<span className="mr-1 text-[13px] text-muted-foreground">{i.usage_count ?? 0} 个实例</span>
{imageActions(i)}
</div>
</td>
</tr>)}</tbody>
</table>
</div>
<div className="grid gap-3 md:hidden">
{images.data.map(i => <div key={i.image_id || i.id} className="rounded-lg border border-border bg-card p-3.5">
<div className="mb-1 flex items-center justify-between gap-2">
<strong className="text-[14px]">{i.version}</strong>
<Status value={i.status} />
</div>
<p className="m-0 font-mono text-[11px] text-muted-foreground wrap-anywhere">{i.image_id || i.id}</p>
{i.tag && <p className="m-0 mt-0.5 text-xs text-muted-foreground">标签:{i.tag}</p>}
<div className="my-2.5"><ImageStatusCell image={i} /></div>
<div className="flex flex-wrap items-center gap-1.5">
<span className="mr-1 text-[13px] text-muted-foreground">{i.usage_count ?? 0} 个实例</span>
{imageActions(i)}
</div>
</div>)}
</div>
</>}
</Panel>
{admin && admissionId && images.data?.some(i => (i.image_id || i.id) === admissionId) && <Panel title={'准入管理 · ' + admissionId}>
<Admission key={admissionId} imageId={admissionId} refresh={images.refresh} />
</Panel>}
<Panel title="构建任务" action={<div className="flex items-center gap-2">
<span className="hidden text-xs text-muted-foreground sm:inline">有活跃任务时每 5 秒刷新</span>
<RefreshButton loading={builds.loading} onClick={builds.refresh} label="刷新任务" />
</div>}>
<ResourceState resource={builds} />
{builds.data?.length === 0 && <Empty>没有构建任务记录</Empty>}
{!!builds.data?.length && <>
<div className="hidden overflow-x-auto md:block">
<table className={tableClass}>
<thead><tr className={tableHeadRowClass}>
<th className={cn(thClass)}>任务</th>
<th className={cn(thClass)}>版本</th>
<th className={cn(thClass)}>状态</th>
<th className={cn(thClass)}>提交时间</th>
<th className={cn(thClass, "text-right")}>日志</th>
</tr></thead>
<tbody>{rows!.map(b => <tr key={b.id} className={trClass}>
<td className="border-b border-border px-3 py-3 align-top font-mono text-xs">{b.id}</td>
<td className={tdClass}>
{b.version}
{b.experimental && <AlertBox variant="destructive" hideIcon className="mt-1 inline-flex px-2 py-0.5 text-xs font-semibold">实验构建 · 不自动准入</AlertBox>}
<BuildProvenance build={b} />
</td>
<td className={tdClass}><Status value={b.status} /></td>
<td className="border-b border-border px-3 py-3 align-top whitespace-nowrap">{date(b.created_at)}</td>
<td className="border-b border-border px-3 py-3 align-top text-right"><button className={cn(buttonVariants({variant: 'outline', size: 'sm'}))} onClick={() => setSelected(b.id)}>查看日志</button></td>
</tr>)}</tbody>
</table>
</div>
<div className="grid gap-2 md:hidden">
{rows!.map(b => <div key={b.id} className="rounded-lg border border-border bg-card p-3.5">
<div className="mb-1 flex items-center justify-between gap-2">
<strong className="text-[14px]">{b.version}</strong>
<Status value={b.status} />
</div>
<p className="m-0 font-mono text-[11px] text-muted-foreground">{b.id}</p>
{b.experimental && <AlertBox variant="destructive" hideIcon className="mt-1.5 inline-flex px-2 py-0.5 text-xs font-semibold">实验构建 · 不自动准入</AlertBox>}
<BuildProvenance build={b} />
<div className="mt-2.5 flex items-center justify-between gap-2">
<span className="text-xs text-muted-foreground">{date(b.created_at)}</span>
<button className={cn(buttonVariants({variant: 'outline', size: 'sm'}))} onClick={() => setSelected(b.id)}>查看日志</button>
</div>
</div>)}
</div>
</>}
<div className="mt-4 flex items-center justify-center gap-3">
<button className={cn(buttonVariants({variant: 'outline', size: 'sm'}))} disabled={page === 0 || builds.loading} onClick={() => changePage(page - 1)}><ChevronLeft aria-hidden />上一页</button>
<span className="text-[13px] text-muted-foreground">第 {page + 1} 页</span>
<button className={cn(buttonVariants({variant: 'outline', size: 'sm'}))} disabled={builds.loading || !builds.data || builds.data.length <= 20} onClick={() => changePage(page + 1)}>下一页<ChevronRight aria-hidden /></button>
</div> </div>
<p className="callout">镜像仅保存在本机。构建成功不等于通过验收或自动准入。仅 available 且允许选择的镜像可创建或升级;existing_only 仅允许存量启停。blocked 禁止新建、升级及新的启动/重启,但不会自动停止已运行实例或断开连接。</p> {selected && <BuildLogs key={selected} id={selected} close={() => setSelected(undefined)} />}
</Panel> </Panel>
<Panel title="本地镜像" action={<button disabled={images.loading} onClick={images.refresh}>刷新</button>}> </>;
<ResourceState resource={images}/>{images.data?.length === 0 && <Empty>本机暂无托管镜像,请先手动构建。</Empty>}{!!images.data?.length && <div className="table-scroll">
<table>
<thead>
<tr>
<th>DSH 版本</th>
<th>镜像标识</th>
<th>标签</th>
<th>状态</th>
<th>使用/操作</th>
</tr>
</thead>
<tbody>{images.data.map(i => <tr key={i.image_id || i.id}>
<td>
<strong>{i.version}</strong>
</td>
<td className="mono wrap">{i.image_id || i.id}</td>
<td>{i.tag || '—'}</td>
<td>
<Status value={i.status}/>
<p>{({recommended:'推荐',compatible:'兼容',existing_only:'仅存量',blocked:'禁止使用'} as Record<string,string>)[i.admission || ''] || '未验证'}</p>
<small>{i.admission_reason || '无准入依据'}</small>
<p>{i.selectable && i.status === 'available' ? '可创建 / 升级' : '不可创建 / 升级'}</p>
</td>
<td>{i.usage_count ?? 0} 个实例 {admin && <button onClick={() => setAdmissionId(admissionId === (i.image_id || i.id) ? undefined : (i.image_id || i.id))}>准入管理</button>}{admissionId === (i.image_id || i.id) && admin && <Admission key={admissionId} imageId={admissionId} refresh={images.refresh}/>} {admin && <button className="danger" disabled={action.busy || !!i.usage_count} onClick={() => action.run(async () => {await api.request('/images?image_id=' + encodeURIComponent(i.image_id || i.id), 'DELETE'); await images.refresh();}, '镜像已删除', '确认删除这个本地镜像?服务器将检查所有引用,不会强制删除。')}>清理镜像</button>}</td>
</tr>)}</tbody>
</table>
</div>}</Panel>
<Panel title="构建任务" action={<>
<span className="muted">当前页有活跃任务时每 5 秒刷新</span>
<button disabled={builds.loading} onClick={builds.refresh}>刷新任务</button>
</>}>
<ResourceState resource={builds}/>{builds.data?.length === 0 && <Empty>没有构建任务记录</Empty>}{!!builds.data?.length && <div className="table-scroll">
<table>
<thead>
<tr>
<th>任务</th>
<th>版本</th>
<th>状态</th>
<th>提交时间</th>
<th>日志</th>
</tr>
</thead>
<tbody>{rows!.map(b => <tr key={b.id}>
<td className="mono">{b.id}</td>
<td>{b.version}{b.experimental && <strong className="alert">实验构建 · 不自动准入</strong>}</td>
<td>
<Status value={b.status}/>
</td>
<td>{date(b.created_at)}</td>
<td>
<button onClick={() => setSelected(b.id)}>查看日志</button>
</td>
</tr>)}</tbody>
</table>
</div>}<div className="actions">
<button disabled={page === 0 || builds.loading} onClick={() => changePage(page - 1)}>上一页</button>
<span>第 {page + 1} 页</span>
<button disabled={builds.loading || !builds.data || builds.data.length <= 20} onClick={() => changePage(page + 1)}>下一页</button>
</div>{selected && <BuildLogs key={selected} id={selected} close={() => setSelected(undefined)}/>}</Panel>
</>;
} }

Some files were not shown because too many files have changed in this diff Show More