Harden image admission, resource recovery and session renewal

Add exact-image compatibility policy, experimental admission, paginated build history and shared reauthentication checks. Preserve uncertain Docker creation records, bound frontend resource waits and renew runtime sessions proactively.

Lazy-load and split Monaco, add regression and opt-in live Docker fault tests, and record full race and real-browser acceptance results.
This commit is contained in:
2026-10-14 13:41:54 +08:00
parent 66abb76318
commit 9a71772771
62 changed files with 4679 additions and 332 deletions
+63
View File
@@ -1,5 +1,68 @@
# DESIGN.md 验收记录 # DESIGN.md 验收记录
## 恢复宽限期与轮询缓存修复(独立复审后)
范围:本轮为对上述工作树的独立代码审计与全量重测后的修复。审计确认八项修复中六项正确,发现三处残留:创建结果不确定的挂起标志永不过期导致实例永久无法删除(高)、构建校验不确定行无限重试并持续新增审计(中)、意图轮询每秒对每个成功镜像发起 Docker inspect(中)。修复未改变凭据文件管理边界或准入策略;未部署、未操作真实 Docker 实例。
- `runtime_creates` 增加 `container_pending_at`/`network_pending_at` 置位时间(列名白名单,禁用未校验列名拼接)。确定性名称 404 在宽限窗口(默认1分钟)内仍视为不确定;窗口过后判定资源从未创建,标志落账清除,实例可正常删除,删除成功后总是清理 `runtime_creates`/`runtime_networks` 行。无时间戳的遗留行视为已过期立即解析。
- `image_build_checks` 增加 `created_at`。无已记录容器 ID 且名称 404 的不确定行在宽限窗口过后删除行停止重试;窗口内保持降频重试与 `build.cleanup_error` 审计(≤1/分钟不变)。
- 意图轮询改用 `verifiedImagesPoll`:成功构建行每秒重读,逐镜像准入评估按 `imageDeleteGeneration`+5秒TTL缓存;镜像删除或准入写入使缓存立即失效。创建/升级入口仍走未缓存的权威校验。
- 前端:无 Web Locks 浏览器的会话恢复重试窗口从约0.85秒延长至约3.85秒;`useResource` 对60秒不返回的请求自动放弃并重新武装轮询(序号守卫防止迟到结果覆盖新状态);`runtimeSession` 测试改为 TypeScript 并纳入 tsc 检查;`test-panel.mjs` 构建详情轮询对瞬时 5xx 容错续轮。
回归覆盖:宽限窗口内删除被拒、窗口过后删除成功且无残留行(TestUncertainPendingResolvesAfterGrace);校验行窗口内保持不确定、窗口过后与遗留行均解析(TestBuildCheckUncertainRowResolvesAfterGrace);轮询三次仅一次 Docker inspect、代际提升后缓存失效(TestIntentPollerCachesImageEvaluations);悬挂请求逃生阀(ui.test.ts)。Go共有192个顶层测试(另有子测试)。前端15个测试文件、95项测试通过;`@types/node` 新增为 devDependency,官方源 npm audit 0 漏洞。
## 完整home与镜像准入策略实施
本节落实用户确认的完整home所有者访问、兼容清单及管理员实验构建策略,取代下方上一轮“尚未改变这两项策略”的历史边界。未部署,未操作真实Docker实例或为实际镜像写入准入。
- 所有者可读取/打包凭据,普通文件管理不强制使用专用Profile流程;匿名、跨用户和路径穿越仍拒绝。下载及敏感修改增加确认,取消不触发操作;上传同名拒绝不覆盖。专用Profile页面仍要求停机,文案与通用文件管理明确区分。
- 默认普通构建支持linux/amd64、模板7的rc.1/rc.2;未知版本只能管理员实验构建。实验完整执行原检查且success不自动可选。准入绑定精确imageID/version/recipe/platform与成功验证记录,管理员填写依据,策略和审计同事务,审计失败回滚。
- 历史模板3–6默认仅存量,经明确验收可准入并使用旧双挂载;未知布局拒绝正向准入,模板7向旧共享布局降级拒绝。已准入未知版本只通过本地imageID使用,不自动扩大版本构建白名单。
- blocked拒绝新建、升级目标、start/restart及受控配置非GET操作,避免恢复/备份自动启动绕过;不自动停止既有容器,普通文件管理保持可用。队列恢复/执行重查版本、平台及实验请求者管理员权限。
- 独立复核修正了准入与生命周期锁顺序、受控配置启动遗漏及两处提示矛盾;最终静态复核未发现剩余阻塞项。
回归覆盖精确版本/模板/平台匹配、旧模板实际创建挂载、实验success仍不可选、未知请求零预留、普通用户和CSRF拒绝、审计回滚、并发降级等待创建完成、blocked启停/配置拒绝,以及前端准入确认/取消和选择器过滤。Go共有189个顶层测试(另有子测试);本轮Go普通全量和全量race禁缓存通过,API包分别约17秒与97秒。前端15个测试文件、94项测试通过。浏览器使用模拟API验证实验警告、准入表单、创建筛选及unknown latest禁止预留,临时服务和浏览器已关闭;非真实Docker或新DSH版本兼容验收。
完整 `scripts/check-trial.sh` 最终退出码0;`go vet`、Go构建、TypeScript与Vite生产构建、profile初始化3项Node测试、验收脚本自身11项Node测试和脚本语法检查均通过。仍有既有Vite大chunk提示,非构建失败;`git diff --check`通过。
## 2026-10-14 真实环境补充验收与构建拆包
- 全量 race:`go test -race -count=1 -timeout=180s ./...` 退出 0,全部包通过、无 race 报告;API 包 166.645 秒,总耗时约 170 秒。
- 真实 Docker:新增默认跳过、需 `DSH_DOCKER_LIVE_TEST=1` 启用的 `TestDockerLiveRecovery`。经故障代理向真实 `/var/run/docker.sock` 转发,真实 create/delete 成功后丢失响应,或断开客户端并延迟创建,先取得真实 404 再放行。六个子场景连续两轮通过,另一次 race 模式通过。覆盖 runtime/network/build-check 恢复、面板未连接/完整 ID/名称连接及删除响应丢失。仅使用专用唯一标签资源、本地 node:24-alpine;按精确 ID 清理后标签查询零残留。测试调用恢复方法,不模拟整机断电,也不重启共享 daemon。
- 真实浏览器续期:当前 Go 源码、隔离临时数据库、真实 DSH rc.2 模板7容器、Chromium 独立 Cookie 上下文,原始 5 分钟 access / 4 分钟续期间隔。2026-10-14T03:49:07.767Z 至 03:59:22.828Z 共 615.061 秒;初次、第 240.023 秒及第 480.020 秒 refresh 均 200;每 15 秒原生 fetch 检查 `/api/me` 和真实 runtime,共 41 轮全部 200(含第 301 秒和第 601 秒)。没有加速时钟、手动续期或共享 Cookie 的其他页面。证据 `output/playwright/renewal-acceptance.json`;不代表休眠/后台节流或多小时压力验收。
- Monaco 改为 React.lazy 按需加载,依模块分层拆包;主入口从约 2589 kB 降至 284.92 kB,最大 chunk 411.21 kB,worker 232.35 kB,保留默认 500 kB 阈值且最终构建无大 chunk 或循环 chunk 告警。新增构建测试约束体积、入口不静态依赖 Monaco 及 chunk 无环;16 文件/114 项前端测试通过。真实浏览器验证编辑输入、运行中保存禁用、只读预览及 worker 初始化成功,截图 `output/playwright/editor-chunks-verified.png`。
- 最终 `go vet ./...`、`go build ./...`、TypeScript/Vite 构建和 `git diff --check` 通过。专用面板、两个浏览器已关闭,验收容器/网络及临时数据和凭据已清理;原四个运行容器仍在,未部署到现有面板、未更改共享镜像。
复现真实故障测试:`DSH_DOCKER_LIVE_TEST=1 go test ./internal/api -run TestDockerLiveRecovery -v -count=2 -timeout=3m`;需可访问 Docker socket 和本地 node:24-alpine,不会自动拉取镜像。
## 再次审查修复(本地回归)
- runtime 注入脚本每 4 分钟主动续期,不再因 `/api/me` 仍有效而跳过;保留 Web Locks 串行刷新、同页请求合并及冲突复验。模拟时钟覆盖连续续期,补测锁等待期间 Cookie 更新。
- 资源加载超时会结束调用方等待链、清除 loading 并提示重试;覆盖无轮询、旧请求和后续请求均悬挂、迟到成功/失败、超时配置变化,不再只清空 in-flight 引用。
- Docker 创建结果不确定时,不凭一分钟后的名称 404 清除 pending/验证容器记录;覆盖旧/缺失/非法时间戳及迟到资源回收。网络清理只断开确实连接的面板端点,保留身份校验和真实错误处理。持续未确认的创建需按 OPERATIONS.md 人工确认。
- 准入依据前端按实际提交内容计算 UTF-8 字节数,超出 4096 字节时提示并禁止提交;后端保持字节上限及请求体限制,长度错误明确返回。覆盖中文、ASCII、多字节、表情及边界。
本轮通过:`go test -count=1 -timeout=180s ./...`、准入与资源恢复相关 API 测试的禁缓存 race 检查、`go vet ./...`、`go build ./...`;前端 15 文件/113 测试及 TypeScript/Vite 生产构建;profile 初始化与验收脚本自身共 14 项 Node 测试、集成脚本语法检查、`git diff --check`。本轮未执行全量 race、真实浏览器长时续期或真实 Docker 故障注入;未部署、未操作现有实例。Vite 仍有既有大 chunk 提示。
## 八项审查问题闭环(本地回归)
范围:普通创建响应丢失、验证容器回收、构建终态持久化、多页面续期、慢请求轮询、当前密码重认证、配额更新竞态、构建历史分页与按需日志。未改变凭据文件管理边界或历史镜像兼容策略;未部署、未操作真实 Docker 实例。
- 资源恢复:创建前保存恢复依据;校验资源身份后按持久 ID 清理。覆盖迟到资源(首次404随后出现)、网络创建/删除响应丢失、验证容器重命名、外部同名网络拒删、清理失败重试、重开数据库后无排队任务自动回收。个别预约待恢复不会阻止面板启动。
- 构建终态:任务与镜像索引事务提交,故障注入验证无半提交;独立重试不阻挡后续构建,Store关闭后退出。持续不确定创建保留恢复记录,不凭单次404释放资源依据;人工确认流程见 OPERATIONS.md。
- 认证:备份、升级、改密共用失败计数与锁定;旧备份密码不参与登录失败计数。并发refresh冲突返回409且不签发cookie,前端及runtime复验后保留可重试错误,真正撤销仍退出;覆盖慢升级不阻塞其他账号登录。
- 文件与前端:上传提交锁内读取最新额度;管理员修改额度共用用户锁。慢轮询不重叠、不废弃有效慢响应,显式刷新保证额外读取,key切换隔离旧结果。构建列表默认20条、最多100条且不读取日志正文,详情按需加载;参数、路由鉴权、整页边界和当前页镜像对账均有回归。
最终验证:
- 完整 `scripts/check-trial.sh` 通过,最终退出码0;最终Go测试通过 `GOFLAGS=-count=1` 禁用测试结果缓存。Go共有182个顶层测试(另含子测试),API普通测试约18秒,全量race中API包约110秒;无race报告。
- `go vet ./...`、`go build ./...`、前端类型检查与生产构建通过;Vite仍提示原有大chunk,不影响构建成功。
- 前端10个文件、57项测试通过;profile初始化3项Node测试、验收脚本自身11项Node测试通过;真实runtime注入脚本的401/403/409/500行为已自动测试。
- 浏览器配合模拟API验证每页20条、末页和禁用下一页、日志打开/关闭及终态停止轮询。临时浏览器和开发服务器已关闭。
- `git diff --check`通过。Docker故障场景使用httptest模拟,并非真实daemon断电/断网验收;未运行指向真实部署的test-panel脚本,仅更新其单任务查询并检查语法。
升级注意:已打开的旧页面需刷新;外部 `/api/builds` 消费者需迁移分页和单任务日志接口。持续数据库故障或身份冲突不能承诺自动成功,保留记录并按运维流程处理。
## 2026-10-12 可信团队试运行加固(本地自动化验收) ## 2026-10-12 可信团队试运行加固(本地自动化验收)
范围:本机或仅可信账号的团队,可由管理员通过公网 HTTPS 反代访问。未修改 Nginx/Caddy,未部署面板或操作现有实例。本节仅说明当前代码的本地自动化验收,不将历史部署证据当作本次公网实测。 范围:本机或仅可信账号的团队,可由管理员通过公网 HTTPS 反代访问。未修改 Nginx/Caddy,未部署面板或操作现有实例。本节仅说明当前代码的本地自动化验收,不将历史部署证据当作本次公网实测。
+17 -5
View File
@@ -25,7 +25,7 @@ GET /api/orphans 返回可回收的完整容器ID数组。DELETE /api/orphans?id
## 认证 ## 认证
- POST /api/login {username,password} → {user,csrf_token};user 字段为 id,username,admin,quota_bytes,created_at - POST /api/login {username,password} → {user,csrf_token};user 字段为 id,username,admin,quota_bytes,created_at
- POST /api/logout → {ok:true} - POST /api/logout → {ok:true}
- POST /api/refresh → {ok:true}(刷新HttpOnly cookie,需Origin和CSRF) - POST /api/refresh → {user,csrf_token}(刷新HttpOnly cookie,需Origin和CSRF;并发轮换冲突且session仍有效时返回409,不签发cookie,客户端可复验 `/api/me` 后重试原请求;会话撤销/过期返回401,存储异常不能当作注销)
- GET /api/me → {user,csrf_token} - GET /api/me → {user,csrf_token}
- POST /api/me/password {current_password,password} → {ok:true,reauthenticate:true}(普通用户及管理员仅修改本人;不接收目标 ID,未知 JSON 字段返回 400)。需有效 access/session、同源 Origin 和 X-CSRF-Token;新密码为 UTF-8 12–72 字节,非法 JSON、缺少当前密码或长度不符返回 400。错误当前密码返回 400 而非 401,客户端不应因此退出登录。未登录 401,Origin/CSRF 不符 403(失效 session 在 guard 中也可能返回 403),已认证的非 POST 请求返回 405。 - POST /api/me/password {current_password,password} → {ok:true,reauthenticate:true}(普通用户及管理员仅修改本人;不接收目标 ID,未知 JSON 字段返回 400)。需有效 access/session、同源 Origin 和 X-CSRF-Token;新密码为 UTF-8 12–72 字节,非法 JSON、缺少当前密码或长度不符返回 400。错误当前密码返回 400 而非 401,客户端不应因此退出登录。未登录 401,Origin/CSRF 不符 403(失效 session 在 guard 中也可能返回 403),已认证的非 POST 请求返回 405。
- 与登录、管理员重设共用 authLoginMu,等待后复验会话与当前密码;沿用账户登录失败计数,累计 5 次错误锁定 15 分钟,锁定期间返回 429(即使当前密码正确),已登录不能绕过。认证存储故障返回 503。 - 与登录、管理员重设共用 authLoginMu,等待后复验会话与当前密码;沿用账户登录失败计数,累计 5 次错误锁定 15 分钟,锁定期间返回 429(即使当前密码正确),已登录不能绕过。认证存储故障返回 503。
@@ -41,11 +41,18 @@ GET /api/orphans 返回可回收的完整容器ID数组。DELETE /api/orphans?id
- GET /api/audit(管理员) - GET /api/audit(管理员)
## 镜像与构建 ## 镜像与构建
- GET /api/images - GET /api/images(每项含 admission、admission_reason、selectable;只有 status=available 且 selectable=true 才可新建/升级)
- GET /api/compatibility(登录后可读;返回 recipe_rev、platform、entries[{version,recipe_rev,platform,state,build_allowed,reason}]、unknown{state,build_allowed,reason}、experimental_admin_only。此为普通版本构建清单,不等于每个实际镜像的准入结果)
- POST /api/images/admission {image_id,state,evidence}(管理员、同源和CSRF;state为recommended/compatible/existing_only/blocked,evidence必填。正向准入要求精确镜像身份、成功验证记录、受支持挂载布局与平台;策略与审计持久化。单镜像准入不自动扩大普通版本构建白名单)
- DELETE /api/images?image_id=sha256:...(管理员;保护实例、活动任务、升级及全部Docker容器引用,不force) - DELETE /api/images?image_id=sha256:...(管理员;保护实例、活动任务、升级及全部Docker容器引用,不force)
- GET /api/latest(查询 npm dist-tags.latest) - GET /api/latest(查询 npm dist-tags.latest;附带 supported、build_allowed、admission、admission_reason,最新版不一定受支持)
- GET /api/builds - GET /api/builds?limit=20&offset=0(摘要数组,不含日志正文;limit默认20、最大100,offset非负;X-Has-More响应头表示后续是否有记录)
- POST /api/builds {version}(固定版本,异步;版本/模板/平台去重,新任务每用户每小时最多3个,共享复用不扣次数) - GET /api/builds/{id}(按需获取单个任务及脱敏日志;仅已登录用户,日志最多64 KiB)
- POST /api/builds {version,experimental?}(固定版本,异步;experimental=true仅管理员可用。未知版本普通构建会预先拒绝;实验仍执行完整性、依赖和启动验证,success不自动准入。返回任务含experimental标记。版本/模板/平台去重,新任务每用户每小时最多3个,共享复用不扣次数)
镜像策略:recommended/compatible可在实际验证与身份检查通过后用于新建/升级;existing_only允许已有实例运行和启停,不能新建/作为升级目标;blocked还拒绝新的start/restart及受控配置非GET操作(含Profile保存、备份和恢复),但不会自动停止已运行容器或断开工作台,通用文件管理仍允许。experimental是成功实验镜像未准入时的展示状态,不是兼容性结论。missing/unverified不可选。
内置普通构建清单为 linux/amd64、模板7、0.2.0-rc.1(compatible)和0.2.0-rc.2(recommended)。其他平台/版本不继承这些验收结论。模板3–6的历史成功镜像默认仅存量,具有精确验证记录并经管理员验收可准入;未知模板布局不可正向准入,模板7向旧共享布局降级仍拒绝。
## 实例 ## 实例
- GET /api/instances → Instance[];字段 id,uid,name,version,image_id,status,container_id,created_at - GET /api/instances → Instance[];字段 id,uid,name,version,image_id,status,container_id,created_at
@@ -61,6 +68,9 @@ GET /api/orphans 返回可回收的完整容器ID数组。DELETE /api/orphans?id
- /dsh/{id}/...(登录和 owner 校验;需同 Docker 网络) - /dsh/{id}/...(登录和 owner 校验;需同 Docker 网络)
## 工作区 ## 工作区
指定实例时文件根为该所有者实例的完整home;允许下载凭据及目录ZIP,不承诺write-only。下载提示仅在界面防误操作,API不按敏感文件名过滤。通用文件修改不强制走专用Profile的停机/YAML校验流程;归属、路径安全、配额、恢复状态和现有文件操作并发保护仍有效。上传不覆盖同名文件(409)。
- GET /api/ws/tree?path=. - GET /api/ws/tree?path=.
- GET /api/ws/usage - GET /api/ws/usage
- POST /api/ws/mkdir {path} - POST /api/ws/mkdir {path}
@@ -85,4 +95,6 @@ GET /api/orphans 返回可回收的完整容器ID数组。DELETE /api/orphans?id
Profile写入要求实例先停止。备份/还原/升级会受控停启;恢复保留原运行状态,升级成功运行新版,失败恢复旧状态。模板2入口先准备插件再启动应用。删除实例仅删除容器与网络并移除索引,保留数据目录。升级失败恢复实例home,不撤销共享workspace或外部系统副作用。 Profile写入要求实例先停止。备份/还原/升级会受控停启;恢复保留原运行状态,升级成功运行新版,失败恢复旧状态。模板2入口先准备插件再启动应用。删除实例仅删除容器与网络并移除索引,保留数据目录。升级失败恢复实例home,不撤销共享workspace或外部系统副作用。
创建恢复:Docker 创建响应丢失时,首次 inspect 404 不代表原请求已结束。实例保留为可见的 error 状态并继续占用名额,用户可重试删除;后台不会仅凭名称删除身份不匹配资源。不确定创建记录不会自动过期:若资源始终不存在,需管理员确认 Docker 原创建请求已终止后再处理保留记录,不能仅凭等待时长判定安全。构建验证容器在创建前持久登记任务身份,启动时即使没有排队构建也会回收并重试失败清理;持续不存在的未知结果仍保留登记以捕获晚到容器。构建终态和镜像索引事务失败会独立重试并记录日志,不阻塞后续构建;数据库关闭停止重试,重启将未完成构建标记失败。
已实现并通过核心集成检查:镜像复用验证、Docker 生命周期、token/cookie 桥接、加密备份恢复、同镜像替换升级。WS/SSE 代理有单元测试,真实完整会话待验证。硬配额与跨公网租户域名隔离未实现;当前 dsh 发行版不支持 --public-url。默认仅 loopback 监听,不应直接暴露公网。完整限制见 README.md。 已实现并通过核心集成检查:镜像复用验证、Docker 生命周期、token/cookie 桥接、加密备份恢复、同镜像替换升级。WS/SSE 代理有单元测试,真实完整会话待验证。硬配额与跨公网租户域名隔离未实现;当前 dsh 发行版不支持 --public-url。默认仅 loopback 监听,不应直接暴露公网。完整限制见 README.md。
+10 -3
View File
@@ -21,7 +21,7 @@
| 端口/主机是 **CLI flag**(`--port/--host/--public-url/--trusted-host/--no-open`),**无 `DSH_PORT/DSH_HOST` 环境变量** | 同上 | 容器启动参数化靠 CMD args,不靠 env | | 端口/主机是 **CLI flag**(`--port/--host/--public-url/--trusted-host/--no-open`),**无 `DSH_PORT/DSH_HOST` 环境变量** | 同上 | 容器启动参数化靠 CMD args,不靠 env |
| 数据根 = `$DSH_HOME`(默认 `~/.dsh`) | `docs/user/guide/providers.md`、CLI README | 每实例独立 `DSH_HOME`,bind-mount 即可保留数据 | | 数据根 = `$DSH_HOME`(默认 `~/.dsh`) | `docs/user/guide/providers.md`、CLI README | 每实例独立 `DSH_HOME`,bind-mount 即可保留数据 |
| **多实例硬约束:每个容器必须独立 `DSH_HOME` + 独立端口**(task-board 单写者) | architecture.md | 一用户两容器 ⇒ 两个 `DSH_HOME`,不能共享 | | **多实例硬约束:每个容器必须独立 `DSH_HOME` + 独立端口**(task-board 单写者) | architecture.md | 一用户两容器 ⇒ 两个 `DSH_HOME`,不能共享 |
| 模型/API Key 存储:密钥在 `$DSH_HOME/.credentials.yaml`(YAML,write-only),模型/provider 配置在 `$DSH_HOME/profiles/<profile>/cordis.patch.yml`;`cordis.patch.yml` 只存 `apiKeyEnv` 引用 | `providers.md` | 备份"模型"= 备份这两个文件 | | 模型/API Key 存储:密钥在 `$DSH_HOME/.credentials.yaml`(YAML,上游配置接口的 write-only 语义不限制本面板所有者文件访问),模型/provider 配置在 `$DSH_HOME/profiles/<profile>/cordis.patch.yml`;`cordis.patch.yml` 只存 `apiKeyEnv` 引用 | `providers.md` | 备份"模型"= 备份这两个文件 |
| **"profile 文件" = `profiles/web/cordis.patch.yml`(YAML,用户可编辑的 patch 层)**;profile 目录还有 `package.json`(含插件依赖 + `dsh.profile` bundle 清单)+ `node_modules`(pnpm 装的插件,可重建) | `apps/cli/README.md` | 编辑 profile = 编辑 `cordis.patch.yml`;插件清单在 `package.json` | | **"profile 文件" = `profiles/web/cordis.patch.yml`(YAML,用户可编辑的 patch 层)**;profile 目录还有 `package.json`(含插件依赖 + `dsh.profile` bundle 清单)+ `node_modules`(pnpm 装的插件,可重建) | `apps/cli/README.md` | 编辑 profile = 编辑 `cordis.patch.yml`;插件清单在 `package.json` |
| 升级:用户数据全在 `DSH_HOME`(npm 安装树之外),升级 `@deepseek-ai/dsh` 后数据保留;`node_modules` 由 pnpm 重建 | README、CLI README | "保留数据 + 重启新版容器"可行 | | 升级:用户数据全在 `DSH_HOME`(npm 安装树之外),升级 `@deepseek-ai/dsh` 后数据保留;`node_modules` 由 pnpm 重建 | README、CLI README | "保留数据 + 重启新版容器"可行 |
| 公网部署:`--public-url <url/mount/>` + `--trusted-host`;代理必须 **剥前缀、保留 Host、把 `Path=/` 的 cookie 改写为 mount + 加 Secure** | `docs/user/guide/public-deployments.md` | 面板反代要处理 cookie/前缀,是最复杂点 | | 公网部署:`--public-url <url/mount/>` + `--trusted-host`;代理必须 **剥前缀、保留 Host、把 `Path=/` 的 cookie 改写为 mount + 加 Secure** | `docs/user/guide/public-deployments.md` | 面板反代要处理 cookie/前缀,是最复杂点 |
@@ -33,6 +33,13 @@
## 1. 目标与范围 ## 1. 目标与范围
### 已确认的访问与镜像策略(覆盖下文早期假设)
- 面向可信团队,实例所有者可通过文件管理读取、下载、修改完整 DSH_HOME,包括自己的凭据。面板不承诺凭据 write-only;跨用户隔离、路径安全、配额与恢复状态保护保持不变。下载文件/打包目录会提示泄密风险,提示属于交互防误操作,不是阻止所有者访问的安全边界。
- 专用 Profile 编辑器提供 YAML 校验与受控停启流程,但通用文件操作、DSH 终端和 agent 不强制经过该流程;修改运行中配置存在不一致风险,建议先停机并备份。
- 镜像准入分为推荐、仍兼容、仅存量运行、禁止使用。兼容依据绑定精确版本、模板修订、平台与镜像身份;不再仅因历史模板不是当前模板就一律判为不可用,也不能仅因历史构建成功就认定兼容。
- 未验证 DSH 新版仅管理员可显式实验构建;保留既有完整性、依赖与启动检查。实验构建成功不是兼容验收,未明确准入的镜像不可新建实例或作为升级目标。管理员正向准入须填写验收依据并记录审计,不绕过实际镜像身份与验证记录检查。
- 通过 docker.sock 在单机上**创建/启停/删除**多个 `dsh web` 容器,**每用户最多 2 个**。 - 通过 docker.sock 在单机上**创建/启停/删除**多个 `dsh web` 容器,**每用户最多 2 个**。
- 每用户一个**独立工作区**,支持文件**上传/下载/删除/重命名**,并有**容量配额**。 - 每用户一个**独立工作区**,支持文件**上传/下载/删除/重命名**,并有**容量配额**。
- 工作区**映射进每个 dsh 容器**(agent 在工作区内干活)。 - 工作区**映射进每个 dsh 容器**(agent 在工作区内干活)。
@@ -253,7 +260,7 @@ Go 端用 `github.com/docker/docker/client`:
- 前端用 Monaco 编辑器;后端 `GET` 返回原文、`POST` 校验(YAML 合法 + 可选 `--dump-config-schema` 比对)后原子写(临时文件 + rename,`dsh-atomic-write` 风格)。 - 前端用 Monaco 编辑器;后端 `GET` 返回原文、`POST` 校验(YAML 合法 + 可选 `--dump-config-schema` 比对)后原子写(临时文件 + rename,`dsh-atomic-write` 风格)。
- 保存后**重启容器**使生效(dsh 有 `dsh-hmr`,但配置类变更稳妥起见重启;后续可探热更)。 - 保存后**重启容器**使生效(dsh 有 `dsh-hmr`,但配置类变更稳妥起见重启;后续可探热更)。
- 只读展示 `dsh.profile`(bundle 清单)与 `--dump-config` 预览(面板 `docker exec` 跑 `dsh --profile web --dump-config` 取回渲染)。 - 只读展示 `dsh.profile`(bundle 清单)与 `--dump-config` 预览(面板 `docker exec` 跑 `dsh --profile web --dump-config` 取回渲染)。
- **不暴露** `.credentials.yaml` 原文(write-only 语义)。 - 专用配置预览保持脱敏;完整 home 文件管理允许实例所有者读取 `.credentials.yaml` 原文,不承诺 write-only。
--- ---
@@ -279,7 +286,7 @@ Go 端用 `github.com/docker/docker/client`:
- **docker.sock**:面板容器能访问 sock 即等效宿主 root;socket-proxy 白名单 + 面板非 root + 只读 rootfs 缓解。 - **docker.sock**:面板容器能访问 sock 即等效宿主 root;socket-proxy 白名单 + 面板非 root + 只读 rootfs 缓解。
- dsh 容器:**不 `--privileged`**、去 CAP_NET_RAW 等、只读 rootfs、tmpfs /tmp、mem/cpu 限制、不挂 sock。 - dsh 容器:**不 `--privileged`**、去 CAP_NET_RAW 等、只读 rootfs、tmpfs /tmp、mem/cpu 限制、不挂 sock。
- 面板账号:bcrypt + JWT(短期 access + refresh),失败锁定。 - 面板账号:bcrypt + JWT(短期 access + refresh),失败锁定。
- 备份包加密、`.credentials.yaml` 不经 API 明文返回。 - 配置备份包加密;实例所有者通过文件 API 下载的凭据或目录 ZIP 不属于加密配置备份,应按机密数据保管。
- 路径穿越校验、上传大小/类型白名单(可配)。 - 路径穿越校验、上传大小/类型白名单(可配)。
- 审计日志:容器操作/备份/还原/升级/profile 改动入库。 - 审计日志:容器操作/备份/还原/升级/profile 改动入库。
+23
View File
@@ -23,6 +23,8 @@
## 配置与全量备份 ## 配置与全量备份
实例所有者有完整 home 读写权限,包括自己的凭据。通用文件管理、DSH 终端及 agent 不受专用 Profile 编辑器的 YAML 校验或停机流程限制;这不是 write-only 凭据系统。文件下载和目录 ZIP 可包含明文密钥,不等同于下述加密配置备份。提示可帮助避免误操作,但无法阻止所有者导出凭据。共享下载文件前应自行检查,修改运行中配置前建议先停机和备份。
面板提供加密配置与资源备份:固定配置、插件清单/锁文件、skins、pets、agent-presets、.agent-presets及manifest;不包含会话、node_modules或工作区。新包用创建时的登录密码经Argon2id派生密钥、AES-GCM加密,不依赖或创建 `backup.key`。导入/恢复使用原备份密码;修改登录密码不会重加密旧包。旧版密钥包仍兼容,只有这类包依赖原 `backup.key`,应单独安全保管,不能因升级删除。所有包均可能含凭据。 面板提供加密配置与资源备份:固定配置、插件清单/锁文件、skins、pets、agent-presets、.agent-presets及manifest;不包含会话、node_modules或工作区。新包用创建时的登录密码经Argon2id派生密钥、AES-GCM加密,不依赖或创建 `backup.key`。导入/恢复使用原备份密码;修改登录密码不会重加密旧包。旧版密钥包仍兼容,只有这类包依赖原 `backup.key`,应单独安全保管,不能因升级删除。所有包均可能含凭据。
配置备份自动停止原运行实例并恢复运行;恢复先校验包,再快照、写入、启动验证,失败尝试回滚,成功保留原运行/停止状态。当前模板7启动前按清单协调插件;历史跨版本及离线失败验证范围见 ACCEPTANCE.md,不保证任意第三方插件兼容。配置内容上限32 MiB/4096文件,备份入口最多保留20个包;快照和保留上限需管理员监控清理。此流程不能代替全量备份。 配置备份自动停止原运行实例并恢复运行;恢复先校验包,再快照、写入、启动验证,失败尝试回滚,成功保留原运行/停止状态。当前模板7启动前按清单协调插件;历史跨版本及离线失败验证范围见 ACCEPTANCE.md,不保证任意第三方插件兼容。配置内容上限32 MiB/4096文件,备份入口最多保留20个包;快照和保留上限需管理员监控清理。此流程不能代替全量备份。
@@ -31,6 +33,14 @@
## 升级与故障 ## 升级与故障
### 镜像兼容验收与准入
管理员实验构建只负责生成并执行基础验证,不代表上线批准。准入前应在独立测试环境核验精确镜像的启动、认证/代理、终端与流式连接、文件读写、插件协调、配置备份恢复,以及所需升级/回滚路径;记录版本、模板、平台、镜像ID、测试范围和证据位置。不要在验收依据中粘贴凭据,也不要仅填写“构建成功”。本次实现不自动启动待验收镜像供普通用户使用,不自动运行真实部署测试。
在镜像页“准入管理”选择推荐或兼容并填写依据,后端还会检查精确镜像身份、成功验证记录和可支持的挂载布局。审批只作用于该镜像;新镜像需要自己的验证与审批。退为仅存量后不允许新建/升级目标;标为禁止使用还阻止新的启动/重启及受控配置修改、备份/恢复写操作,避免其自动重启绕过禁用。已有配置读取和通用home文件管理仍保留,不会自动停止正在运行的实例。若需要立即隔离已运行实例,管理员必须另行确认并执行停止操作。
历史home布局不同:模板7使用完整home作为 `/workspace`,受支持旧模板使用独立 `/data/dsh-home` 与用户共享 `/workspace`。不要把旧模板准入理解为数据布局可任意互换;不支持的反向迁移仍会拒绝,先做好完整备份。
运行时认证材料由面板首次成功连接实例时提取,并使用 `auth-signing.key` 派生的密钥加密保存在数据根的 `runtime-auth/`(不在用户home内)。按容器 ID 和启动时间绑定,不会将旧启动周期凭据用于新容器;不同 Host 分别缓存 cookie。全量备份应连同签名密钥和该目录保存,不要向用户公开。 运行时认证材料由面板首次成功连接实例时提取,并使用 `auth-signing.key` 派生的密钥加密保存在数据根的 `runtime-auth/`(不在用户home内)。按容器 ID 和启动时间绑定,不会将旧启动周期凭据用于新容器;不同 Host 分别缓存 cookie。全量备份应连同签名密钥和该目录保存,不要向用户公开。
运行时返回401/403时会使该Host的cookie缓存失效,下次请求重新交换。不会自动重放写请求,避免插件操作重复执行;遇到认证失败请重新加载页面后确认操作结果。若runtime自行轮转了启动token且旧token也不可交换,需要管理员处理,不能保证任意上游认证机制变更自动恢复。 运行时返回401/403时会使该Host的cookie缓存失效,下次请求重新交换。不会自动重放写请求,避免插件操作重复执行;遇到认证失败请重新加载页面后确认操作结果。若runtime自行轮转了启动token且旧token也不可交换,需要管理员处理,不能保证任意上游认证机制变更自动恢复。
@@ -43,6 +53,19 @@
- 人工恢复前停止相关容器并另做完整备份。中断点不同,不能统一用一次目录覆盖或状态修改处理;当前无自动灾难恢复命令。 - 人工恢复前停止相关容器并另做完整备份。中断点不同,不能统一用一次目录覆盖或状态修改处理;当前无自动灾难恢复命令。
- 删除实例会删除其容器与网络,但保留数据目录。升级快照暂无自动清理;不要删除仍被未完成 journal 引用的快照。 - 删除实例会删除其容器与网络,但保留数据目录。升级快照暂无自动清理;不要删除仍被未完成 journal 引用的快照。
## 创建与构建恢复记录
创建失败不代表 Docker 没有创建资源。面板在发请求前记录 `runtime_creates`(实例 ID、容器/网络结果待确认标志及其置位时间)及 `image_build_checks`(构建 ID、镜像 ID、已确认的验证容器 ID、记录创建时间),已确认网络身份保存在 `runtime_networks`。资源可见后按身份校验再清理;已记录 ID 时按 ID 查询,不能把名称消失当成资源删除。身份不一致时拒绝删除。构建验证残留在启动时及后台重试回收,无需新建构建任务触发。
创建请求失联后,确定性名称的 404 仍视为结果不确定,不会仅因经过一分钟而清除恢复依据:实例保持 `error` 并占用名额,资源迟到出现后可重试删除并按身份清理;验证容器的不确定记录保留并后台降频重试、审计记录 `build.cleanup_error`,不会每秒新增审计。即使恢复记录的时间戳缺失或损坏,也不能据此推断创建已结束。一直未出现的资源需要按下述人工确认流程处理,不能保证所有失联创建自动释放预约。删除实例成功后总是清理其 `runtime_creates`/`runtime_networks` 行。数据库暂时不可写时,构建终态事务独立重试,不阻挡后续构建;面板退出后未提交的任务按中断失败处理,不承诺恢复为成功。
人工确认流程(仅部署管理员,不能因单次404直接改库):
1. 停止面板接收新操作并停止面板进程,备份数据库(含 WAL)及相关恢复记录。确认 Docker daemon 健康,并结合 daemon 日志确认原创建请求已结束;必要的 daemon 维护会影响其他容器,应单独安排。
2. 根据精确实例/构建 ID 核查 `dsh-inst-<instance-id>` 网络与容器,或 `dsh-build-check-<build-id>` 验证容器;已记录容器 ID 时还必须按 ID 查询,不能仅按名称判断不存在。核对归属标签、镜像及名称,不运行全局 prune。
3. 资源存在且身份一致时,优先恢复面板并走正常删除/自动回收;身份冲突则保留记录,先调查冲突来源。
4. 只有确认原请求终结、对应资源确实不存在后,才可在维护备份基础上用 SQLite 事务清除该精确 ID 的待确认标志,或删除该精确 build ID 的 `image_build_checks` 行。不得批量清空恢复表,不修改实例为 running。重新启动面板后重试正常删除,并核对实例名额、构建状态与 Docker 资源。
## 容量与网络 ## 容量与网络
实例文件请求只在检查归属和登记使用状态时取得生命周期锁,上传/下载期间不持全局生命周期锁。同一实例存在文件操作时,启停、删除、配置修改、恢复及升级返回 409,等待传输结束后重试。文件树单目录最多列出 10000 项,超限返回 413;用量与快照遍历仍有总条目/深度上限。此限制不是文件系统硬配额。 实例文件请求只在检查归属和登记使用状态时取得生命周期锁,上传/下载期间不持全局生命周期锁。同一实例存在文件操作时,启停、删除、配置修改、恢复及升级返回 409,等待传输结束后重试。文件树单目录最多列出 10000 项,超限返回 413;用量与快照遍历仍有总条目/深度上限。此限制不是文件系统硬配额。
+12
View File
@@ -12,6 +12,8 @@
## 当前实现 ## 当前实现
实例所有者可管理完整 home,包括读取自己的配置和凭据;不承诺密钥 write-only,跨用户隔离保持不变。通用文件操作不受专用 Profile 编辑器的 YAML 校验和停机要求约束。下载、打包与敏感修改提示用于防误操作,不是权限限制;文件下载和目录 ZIP 不会自动加密,修改配置前建议先停止实例并备份。
- 内置账号、管理员配额、短JWT与refresh轮换、失败锁定及审计。 - 内置账号、管理员配额、短JWT与refresh轮换、失败锁定及审计。
- 精确版本查询、本地镜像异步构建/验证/清理、持久创建预约与事务名额(每用户最多两个实例)、启动对账。预约不再仅保存在当前页面。 - 精确版本查询、本地镜像异步构建/验证/清理、持久创建预约与事务名额(每用户最多两个实例)、启动对账。预约不再仅保存在当前页面。
- 实例启停删除、独立home(模板7挂载为 `/workspace`)、每实例bridge、归属鉴权及路径反代;DSH认证cookie由服务端保管。 - 实例启停删除、独立home(模板7挂载为 `/workspace`)、每实例bridge、归属鉴权及路径反代;DSH认证cookie由服务端保管。
@@ -166,6 +168,14 @@ sudo systemctl reload caddy
## 本地runtime构建 ## 本地runtime构建
### 兼容清单与实验构建
镜像的构建验证与使用准入分开管理:推荐/兼容镜像还必须通过精确身份与实际验证检查才可选择;仅存量镜像允许原实例继续运行和启停,不可新建或作为升级目标;禁止使用镜像还拒绝新的启动/重启,但不会自动停止现有容器或断开工作台。
未知版本仅管理员可在镜像页显式提交实验构建。实验保留镜像模板的完整性、依赖和启动检查,可能因不兼容而失败;不会为让新版通过而跳过检查。实验成功后仍不可创建实例,管理员需完成独立兼容验收,再在“准入管理”填写可追溯依据并确认。准入绑定精确镜像ID、版本、模板和平台,不自动信任同tag的新镜像,也不扩大普通版本构建白名单;已准入的未知版本请通过本地镜像选择创建。
历史模板按其自身验证记录与挂载布局判断,不因模板号不是当前版本就标为缺失。有成功验证记录、布局受支持的旧镜像可经管理员验收准入;无法识别的布局不会仅凭一段说明放行。界面展示实际准入原因,以后端校验为准。
当前模板为修订7,同时内置 `[email protected]` 与 `[email protected]`(MIT,https://github.com/DDDMUC/dsh-free-search)。Free Search及其普通依赖固定版本、校验SHA-512并保留镜像内原始tarball;新profile默认正式安装并启用,已有profile不自动注入。模板4的mobile恢复路径继续保留。仅包含内置固定依赖的profile在启动前离线安装,不限制运行期插件管理器联网。 当前模板为修订7,同时内置 `[email protected]` 与 `[email protected]`(MIT,https://github.com/DDDMUC/dsh-free-search)。Free Search及其普通依赖固定版本、校验SHA-512并保留镜像内原始tarball;新profile默认正式安装并启用,已有profile不自动注入。模板4的mobile恢复路径继续保留。仅包含内置固定依赖的profile在启动前离线安装,不限制运行期插件管理器联网。
Free Search默认引擎为Bing,注册的统一provider ID为ddg;未设置或等于deepseek-official的搜索provider会被它接管(包括显式选择同名官方provider),默认http抓取也会被接管,其他自定义provider保留。名称含free不等于保证零费用:自动回退可能读取已有API凭据并调用计费服务。请在插件设置中按需禁用引擎;fallback顺序不是引擎白名单。内置集成不自动测试搜索、配置密钥或执行在线更新。 Free Search默认引擎为Bing,注册的统一provider ID为ddg;未设置或等于deepseek-official的搜索provider会被它接管(包括显式选择同名官方provider),默认http抓取也会被接管,其他自定义provider保留。名称含free不等于保证零费用:自动回退可能读取已有API凭据并调用计费服务。请在插件设置中按需禁用引擎;fallback顺序不是引擎白名单。内置集成不自动测试搜索、配置密钥或执行在线更新。
@@ -191,6 +201,8 @@ bash scripts/smoke-image.sh dsh-web:0.2.0-rc.2-7
## 设计替代与边界 ## 设计替代与边界
构建历史接口现默认分页(20条,最大100条),列表不再携带日志正文;日志通过 `/api/builds/{id}` 按需读取。升级面板后请刷新已打开的旧页面;外部脚本需按 API.md 更新,不能再依赖 `/api/builds` 返回全部历史或日志。
合理替代方案为:stdlib HTTP Docker client而非官方SDK;每实例独立bridge而非共享bridge;服务端保管DSH cookie而非下发浏览器;普通React组件而非shadcn/ui(保留Monaco)。这些是选型差异,不降低功能、安全或验收标准,详见验收表。 合理替代方案为:stdlib HTTP Docker client而非官方SDK;每实例独立bridge而非共享bridge;服务端保管DSH cookie而非下发浏览器;普通React组件而非shadcn/ui(保留Monaco)。这些是选型差异,不降低功能、安全或验收标准,详见验收表。
同源路径代理仅适用于可信用户/插件,尚无独立origin及宿主/局域网出站防护。默认软配额不能阻止agent直接写满磁盘;硬quota是依赖宿主的可选增强,不是默认路线必做项。升级/恢复快照需额外容量,home快照上限1 GiB,模板7的 `/workspace` 就是实例home,会随home快照回滚;旧共享目录不随回滚恢复;中断及回滚失败仍可能需要管理员恢复。 同源路径代理仅适用于可信用户/插件,尚无独立origin及宿主/局域网出站防护。默认软配额不能阻止agent直接写满磁盘;硬quota是依赖宿主的可选增强,不是默认路线必做项。升级/恢复快照需额外容量,home快照上限1 GiB,模板7的 `/workspace` 就是实例home,会随home快照回滚;旧共享目录不随回滚恢复;中断及回滚失败仍可能需要管理员恢复。
+2
View File
@@ -91,6 +91,8 @@ func (a *API) users(w http.ResponseWriter, r *http.Request) {
fail(w, 400, "positive quota and user ID required") fail(w, 400, "positive quota and user ID required")
return return
} }
user := a.userLocks.Acquire(in.ID)
defer user.Release()
result, err := a.s.DB.Exec("UPDATE users SET quota=? WHERE id=?", in.Quota, in.ID) result, err := a.s.DB.Exec("UPDATE users SET quota=? WHERE id=?", in.Quota, in.ID)
if err != nil { if err != nil {
fail(w, 500, "quota update failed") fail(w, 500, "quota update failed")
+208
View File
@@ -0,0 +1,208 @@
package api
import (
"context"
"database/sql"
"encoding/json"
"errors"
"io"
"net/http"
"strings"
"sync"
"time"
)
// Admission writes serialize with new runtime operations, including their Docker side effects.
var admissionMu sync.RWMutex
var errBuildUnsupported = errors.New("version is not supported for normal builds; administrator experimental build required")
func defaultAdmission(version, recipe, platform string) string {
if recipe == "7" && platform == "linux/amd64" {
switch version {
case "0.2.0-rc.1":
return "compatible"
case "0.2.0-rc.2":
return "recommended"
}
}
return "existing_only"
}
func supportedLayout(recipe string) bool {
switch recipe {
case "3", "4", "5", "6", "7":
return true
}
return false
}
func normalBuildAllowed(version string) bool {
return positiveAdmission(defaultAdmission(version, imageRecipeRev, imagePlatform))
}
func positiveAdmission(state string) bool { return state == "recommended" || state == "compatible" }
func (a *API) compatibility(w http.ResponseWriter, r *http.Request) {
if _, _, err := a.user(r); err != nil {
fail(w, 401, "authentication required")
return
}
if r.Method != http.MethodGet {
fail(w, 405, "method not allowed")
return
}
entries := []map[string]any{}
for _, v := range []string{"0.2.0-rc.1", "0.2.0-rc.2"} {
entries = append(entries, map[string]any{"version": v, "recipe_rev": imageRecipeRev, "platform": imagePlatform, "state": defaultAdmission(v, imageRecipeRev, imagePlatform), "build_allowed": normalBuildAllowed(v), "reason": "only linux/amd64 recipe 7 is built-in; successful exact image verification still required"})
}
writeJSON(w, map[string]any{"recipe_rev": imageRecipeRev, "platform": imagePlatform, "entries": entries, "unknown": map[string]any{"state": "existing_only", "build_allowed": false, "reason": "not accepted; administrator experimental build and explicit admission required"}, "experimental_admin_only": true})
}
func (a *API) evaluateImage(ctx context.Context, id string) (imageRecord, error) {
im := imageRecord{ID: id, ImageID: id, Status: "missing", Admission: "existing_only", AdmissionReason: "local image missing"}
actual, err := a.inspectLocalImage(ctx, id)
if err != nil || actual == nil {
return im, err
}
im.Version = actual.Config.Labels["dsh-mgr.version"]
im.RecipeRev = actual.Config.Labels["dsh-mgr.recipe"]
im.Platform = actual.platform()
im.Created = actual.Created
im.Status = "unverified"
im.AdmissionReason = "successful exact image identity verification required"
if actual.ID != id || actual.Config.Labels["dsh-mgr.role"] != "runtime" {
im.Version = ""
return im, nil
}
var count, experimental int
err = a.s.DB.QueryRowContext(ctx, "SELECT count(*),COALESCE(min(experimental),0) FROM image_builds WHERE image_id=? AND version=? AND recipe_rev=? AND platform=? AND status='success'", id, im.Version, im.RecipeRev, im.Platform).Scan(&count, &experimental)
if err != nil {
return im, err
}
var state, evidence string
err = a.s.DB.QueryRowContext(ctx, "SELECT state,evidence FROM image_admissions WHERE image_id=? AND version=? AND recipe_rev=? AND platform=?", id, im.Version, im.RecipeRev, im.Platform).Scan(&state, &evidence)
if err != nil && !errors.Is(err, sql.ErrNoRows) {
return im, err
}
if state == "blocked" {
im.Admission = state
im.Status = state
im.AdmissionReason = evidence
return im, nil
}
if count == 0 {
return im, nil
}
im.Admission = defaultAdmission(im.Version, im.RecipeRev, im.Platform)
im.AdmissionReason = "no accepted compatibility evidence for this tuple"
if positiveAdmission(im.Admission) {
im.AdmissionReason = "built-in exact tuple compatibility"
}
if experimental != 0 {
im.Admission = "existing_only"
im.Status = "experimental"
im.AdmissionReason = "experimental validation succeeded; administrator acceptance required"
} else {
im.Status = "existing_only"
}
if state != "" {
im.Admission = state
im.AdmissionReason = evidence
im.Status = state
}
if !supportedLayout(im.RecipeRev) || im.Platform != imagePlatform {
im.Admission = "existing_only"
im.Status = "existing_only"
im.AdmissionReason = "historical validation retained; current mount layout or platform not supported"
}
im.Selectable = positiveAdmission(im.Admission)
if im.Selectable {
im.Status = "available"
}
return im, nil
}
func (a *API) imageAdmission(w http.ResponseWriter, r *http.Request) {
u, _, err := a.user(r)
if err != nil {
fail(w, 401, "authentication required")
return
}
if !u.Admin {
fail(w, 403, "administrator required")
return
}
if r.Method != http.MethodPost {
fail(w, 405, "method not allowed")
return
}
var in struct {
ImageID string `json:"image_id"`
State string `json:"state"`
Evidence string `json:"evidence"`
}
dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 8192))
dec.DisallowUnknownFields()
if dec.Decode(&in) != nil || dec.Decode(new(any)) != io.EOF || !runtimeImageID.MatchString(in.ImageID) || strings.TrimSpace(in.Evidence) == "" || (!positiveAdmission(in.State) && in.State != "existing_only" && in.State != "blocked") {
fail(w, 400, "image_id, valid state and evidence required")
return
}
if len(in.Evidence) > 4096 {
fail(w, 400, "evidence must not exceed 4096 UTF-8 bytes")
return
}
admissionMu.Lock()
defer admissionMu.Unlock()
buildQueue.Lock()
err = a.ensureBuilds()
buildQueue.Unlock()
if err != nil {
fail(w, 500, "database error")
return
}
ctx, cancel := context.WithTimeout(r.Context(), 15*time.Second)
defer cancel()
im, err := a.evaluateImage(ctx, in.ImageID)
if err != nil {
fail(w, 502, "image inspection failed")
return
}
if im.Status == "missing" || im.Version == "" || im.RecipeRev == "" || im.Platform == "" {
fail(w, 400, "real runtime image identity required")
return
}
if positiveAdmission(in.State) {
var count int
err = a.s.DB.QueryRowContext(ctx, "SELECT count(*) FROM image_builds WHERE image_id=? AND version=? AND recipe_rev=? AND platform=? AND status='success'", im.ImageID, im.Version, im.RecipeRev, im.Platform).Scan(&count)
if err != nil || count == 0 || !supportedLayout(im.RecipeRev) || im.Platform != imagePlatform {
fail(w, 400, "successful exact validation and supported mount layout/platform required")
return
}
}
buildQueue.Lock()
defer buildQueue.Unlock()
tx, err := a.s.DB.BeginTx(ctx, nil)
if err != nil {
fail(w, 500, "database error")
return
}
defer tx.Rollback()
_, err = tx.ExecContext(ctx, "INSERT INTO image_admissions(image_id,version,recipe_rev,platform,state,evidence,admitted_by,admitted_at) VALUES(?,?,?,?,?,?,?,?) ON CONFLICT(image_id,version,recipe_rev,platform) DO UPDATE SET state=excluded.state,evidence=excluded.evidence,admitted_by=excluded.admitted_by,admitted_at=excluded.admitted_at", im.ImageID, im.Version, im.RecipeRev, im.Platform, in.State, strings.TrimSpace(in.Evidence), u.ID, time.Now().UTC().Format(time.RFC3339Nano))
if err != nil {
fail(w, 500, "database error")
return
}
if _, err = tx.ExecContext(ctx, "INSERT INTO audit(uid,action,target,detail,at) VALUES(?,?,?,?,?)", u.ID, "image.admission", im.ImageID, in.State+": "+in.Evidence, time.Now().Unix()); err != nil {
fail(w, 500, "audit persistence failed")
return
}
if err = tx.Commit(); err != nil {
fail(w, 500, "database error")
return
}
a.imageDeleteGeneration++
im.Admission = in.State
im.AdmissionReason = strings.TrimSpace(in.Evidence)
im.Selectable = positiveAdmission(in.State)
im.Status = in.State
if im.Selectable {
im.Status = "available"
}
writeJSON(w, im)
}
+116
View File
@@ -0,0 +1,116 @@
package api
import (
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func TestAdmissionEvidenceUTF8Limit(t *testing.T) {
id := "sha256:" + strings.Repeat("d", 64)
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { writeImageInspect(w, id, "0.2.0-rc.2") })
seedVerifiedImage(t, a, id, "0.2.0-rc.2")
if _, err := a.s.DB.Exec("UPDATE users SET admin=1"); err != nil {
t.Fatal(err)
}
for _, tc := range []struct {
name, evidence string
want int
}{
{"ascii boundary", strings.Repeat("a", 4096), 200},
{"ascii over", strings.Repeat("a", 4097), 400},
{"Chinese regression", strings.Repeat("中", 1400), 400},
{"Chinese below", strings.Repeat("中", 1365), 200},
{"Chinese boundary", strings.Repeat("中", 1365) + "a", 200},
{"Chinese over", strings.Repeat("中", 1365) + "ab", 400},
{"emoji boundary", strings.Repeat("😀", 1024), 200},
{"emoji over", strings.Repeat("😀", 1024) + "a", 400},
{"two byte boundary", strings.Repeat("é", 2048), 200},
{"raw whitespace counts", " " + strings.Repeat("a", 4096), 400},
} {
t.Run(tc.name, func(t *testing.T) {
body, err := json.Marshal(map[string]string{"image_id": id, "state": "compatible", "evidence": tc.evidence})
if err != nil {
t.Fatal(err)
}
w := httptest.NewRecorder()
a.imageAdmission(w, runtimeRequest("POST", "/api/images/admission", string(body), sid))
if w.Code != tc.want {
t.Fatal(w.Code, w.Body.String())
}
if tc.want == 400 && !strings.Contains(w.Body.String(), "4096 UTF-8 bytes") {
t.Fatal(w.Body.String())
}
if tc.want == 200 {
var evidence string
if err := a.s.DB.QueryRow("SELECT evidence FROM image_admissions WHERE image_id=?", id).Scan(&evidence); err != nil || evidence != tc.evidence {
t.Fatal(evidence, err)
}
}
})
}
}
func TestAdmissionRoutesEnforceAuthorization(t *testing.T) {
id := "sha256:" + strings.Repeat("d", 64)
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { writeImageInspect(w, id, "0.2.0-rc.2") })
seedVerifiedImage(t, a, id, "0.2.0-rc.2")
_, csrf, err := a.s.SessionUser(sid)
if err != nil {
t.Fatal(err)
}
body := fmt.Sprintf(`{"image_id":%q,"state":"compatible","evidence":"isolated validation reference"}`, id)
request := func(method, path, payload string, auth, token bool) *httptest.ResponseRecorder {
session := ""
if auth {
session = sid
}
r := runtimeRequest(method, path, payload, session)
r.Header.Set("Origin", "http://panel.test")
if token {
r.Header.Set("X-CSRF-Token", csrf)
}
w := httptest.NewRecorder()
a.Handler().ServeHTTP(w, r)
return w
}
if w := request("GET", "/api/compatibility", "", false, false); w.Code != 401 {
t.Fatal(w.Code, w.Body.String())
}
if w := request("GET", "/api/compatibility", "", true, false); w.Code != 200 {
t.Fatal(w.Code, w.Body.String())
}
if w := request("POST", "/api/images/admission", body, true, true); w.Code != 403 {
t.Fatal("nonadmin admission", w.Code, w.Body.String())
}
if w := request("POST", "/api/builds", `{"version":"9.8.7","experimental":true}`, true, true); w.Code != 403 {
t.Fatal("nonadmin experiment", w.Code, w.Body.String())
}
if _, err = a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='owner'"); err != nil {
t.Fatal(err)
}
if w := request("POST", "/api/images/admission", body, true, false); w.Code != 403 {
t.Fatal("missing CSRF", w.Code, w.Body.String())
}
if w := request("POST", "/api/images/admission", body, true, true); w.Code != 200 {
t.Fatal("admin admission", w.Code, w.Body.String())
}
for _, payload := range []string{
body + "{}",
body + strings.Repeat(" ", 8192),
fmt.Sprintf(`{"image_id":%q,"state":"compatible","evidence":" "}`, id),
fmt.Sprintf(`{"image_id":%q,"state":"available","evidence":"invalid state"}`, id),
fmt.Sprintf(`{"image_id":%q,"state":"compatible","evidence":"test","dockerfile":"not allowed"}`, id),
} {
if w := request("POST", "/api/images/admission", payload, true, true); w.Code != 400 {
t.Fatal(w.Code, w.Body.String())
}
}
var count int
if err = a.s.DB.QueryRow("SELECT count(*) FROM audit WHERE action='image.admission'").Scan(&count); err != nil || count != 1 {
t.Fatal(count, err)
}
}
+274
View File
@@ -0,0 +1,274 @@
package api
import (
"context"
"encoding/json"
"fmt"
"git.misaka.ren/m1saka/dsh-web/internal/store"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
func TestExperimentalBuildSuccessRequiresAcceptance(t *testing.T) {
id := "sha256:" + strings.Repeat("d", 64)
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v1.47/images/json":
writeJSON(w, []any{map[string]any{"Id": id, "Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "9.8.7", "dsh-mgr.recipe": imageRecipeRev}}})
case "/v1.47/images/" + id + "/json":
writeImageInspect(w, id, "9.8.7")
case "/v1.47/containers/create":
writeJSON(w, map[string]string{"Id": "check"})
case "/v1.47/containers/check/exec":
writeJSON(w, map[string]string{"Id": "probe"})
case "/v1.47/exec/probe/json":
writeJSON(w, map[string]any{"Running": false, "ExitCode": 0})
case "/v1.47/containers/check/json":
writeJSON(w, map[string]any{"State": map[string]bool{"Running": true}})
default:
w.WriteHeader(204)
}
})
if _, err := a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='u'"); err != nil {
t.Fatal(err)
}
w := imageRequest(a.builds, sid, "POST", `{"version":"9.8.7","experimental":true}`)
if w.Code != 202 {
t.Fatal(w.Code, w.Body.String())
}
var b buildRecord
if err := json.Unmarshal(w.Body.Bytes(), &b); err != nil || !b.Experimental {
t.Fatal(b, err)
}
deadline := time.Now().Add(5 * time.Second)
for {
var state string
if err := a.s.DB.QueryRow("SELECT status FROM image_builds WHERE id=?", b.ID).Scan(&state); err != nil {
t.Fatal(err)
}
if state == "success" {
break
}
if state == "failed" || time.Now().After(deadline) {
t.Fatal(state)
}
time.Sleep(10 * time.Millisecond)
}
im, err := a.evaluateImage(context.Background(), id)
if err != nil || im.Selectable || im.Status != "experimental" {
t.Fatal(im, err)
}
}
func TestQueuedUnsupportedPlatformFailsBeforeDocker(t *testing.T) {
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("unsupported build contacted Docker") })
buildQueue.Lock()
err := a.ensureBuilds()
buildQueue.Unlock()
if err != nil {
t.Fatal(err)
}
previous := imagePlatform
imagePlatform = "linux/arm64"
defer func() { imagePlatform = previous }()
b := buildRecord{ID: "unsupported", Version: "0.2.0-rc.2", Status: "queued", RecipeRev: imageRecipeRev, Platform: imagePlatform}
if err := a.insertBuildRequest(b, "u"); err != nil {
t.Fatal(err)
}
a.runImageBuild(b)
var state, detail string
if err := a.s.DB.QueryRow("SELECT status,error FROM image_builds WHERE id=?", b.ID).Scan(&state, &detail); err != nil || state != "failed" || !strings.Contains(detail, "experimental") {
t.Fatal(state, detail, err)
}
}
func TestAdmissionDowngradeWaitsForCreate(t *testing.T) {
id := "sha256:" + strings.Repeat("e", 64)
entered, release := make(chan struct{}), make(chan struct{})
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch {
case strings.Contains(r.URL.Path, "/images/"):
writeImageInspect(w, id, "0.2.0-rc.1")
case strings.HasSuffix(r.URL.Path, "/networks/create"):
close(entered)
<-release
w.WriteHeader(204)
case strings.HasSuffix(r.URL.Path, "/containers/create"):
writeJSON(w, map[string]string{"Id": "created"})
default:
w.WriteHeader(204)
}
})
seedVerifiedImage(t, a, id, "0.2.0-rc.1")
if _, err := a.s.DB.Exec("UPDATE users SET admin=1"); err != nil {
t.Fatal(err)
}
created := make(chan int, 1)
go func() {
w := httptest.NewRecorder()
a.instances(w, runtimeRequest("POST", "/api/instances", fmt.Sprintf(`{"name":"race","image_id":%q}`, id), sid))
created <- w.Code
}()
select {
case <-entered:
case <-time.After(3 * time.Second):
close(release)
t.Fatal("create did not reach network")
}
downgraded := make(chan int, 1)
go func() {
w := httptest.NewRecorder()
a.imageAdmission(w, runtimeRequest("POST", "/api/images/admission", fmt.Sprintf(`{"image_id":%q,"state":"blocked","evidence":"downgrade test"}`, id), sid))
downgraded <- w.Code
}()
select {
case status := <-downgraded:
close(release)
t.Fatalf("admission raced active create: %d", status)
case <-time.After(30 * time.Millisecond):
}
close(release)
if status := <-created; status != 200 {
t.Fatal(status)
}
if status := <-downgraded; status != 200 {
t.Fatal(status)
}
w := httptest.NewRecorder()
a.instances(w, runtimeRequest("POST", "/api/instances", fmt.Sprintf(`{"name":"denied","image_id":%q}`, id), sid))
if w.Code != 400 {
t.Fatal(w.Code, w.Body.String())
}
}
func TestAdmissionHistoricalCreateAndBlock(t *testing.T) {
id := "sha256:" + strings.Repeat("a", 64)
var body map[string]any
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch {
case strings.Contains(r.URL.Path, "/images/"):
writeJSON(w, map[string]any{"Id": id, "Os": "linux", "Architecture": "amd64", "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "1.2.3", "dsh-mgr.recipe": "4"}}})
case strings.HasSuffix(r.URL.Path, "/containers/create"):
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
t.Error(err)
}
writeJSON(w, map[string]string{"Id": "created"})
default:
w.WriteHeader(204)
}
})
seedVerifiedImage(t, a, id, "1.2.3")
if _, err := a.s.DB.Exec("UPDATE image_builds SET recipe_rev='4'; DELETE FROM image_admissions; UPDATE users SET admin=1"); err != nil {
t.Fatal(err)
}
im, err := a.evaluateImage(context.Background(), id)
if err != nil || im.Status != "existing_only" || im.Selectable {
t.Fatal(im, err)
}
records := []buildRecord{{ID: "unused", ImageID: id, Version: "1.2.3", RecipeRev: "4", Platform: imagePlatform, Status: "success"}}
if err := a.reconcileBuilds(context.Background(), records); err != nil || records[0].Status != "success" {
t.Fatal(records, err)
}
admit := func(state string) *httptest.ResponseRecorder {
w := httptest.NewRecorder()
a.imageAdmission(w, runtimeRequest("POST", "/api/images/admission", fmt.Sprintf(`{"image_id":%q,"state":%q,"evidence":"legacy layout explicitly accepted"}`, id, state), sid))
return w
}
if w := admit("compatible"); w.Code != 200 {
t.Fatal(w.Code, w.Body.String())
}
w := httptest.NewRecorder()
a.instances(w, runtimeRequest("POST", "/api/instances", fmt.Sprintf(`{"name":"legacy","image_id":%q}`, id), sid))
if w.Code != 200 {
t.Fatal(w.Code, w.Body.String())
}
host := body["HostConfig"].(map[string]any)
mounts := host["Mounts"].([]any)
if len(mounts) != 2 || mounts[0].(map[string]any)["Target"] != "/data/dsh-home" || mounts[1].(map[string]any)["Target"] != "/workspace" || !strings.Contains(fmt.Sprint(body["Env"]), "DSH_HOME=/data/dsh-home") {
t.Fatal(body)
}
var instance store.Instance
if err := json.Unmarshal(w.Body.Bytes(), &instance); err != nil {
t.Fatal(err)
}
if w := admit("blocked"); w.Code != 200 {
t.Fatal(w.Body.String())
}
for _, action := range []string{"restore", "backups", "profile"} {
w := httptest.NewRecorder()
a.config(w, runtimeRequest("POST", "/api/instances/"+instance.ID+"/"+action, `{}`, sid), instance, action)
if w.Code != 409 {
t.Fatal(action, w.Code, w.Body.String())
}
}
for _, action := range []string{"start", "restart"} {
w := httptest.NewRecorder()
a.instance(w, runtimeRequest("POST", "/api/instances/"+instance.ID+"/"+action, "", sid))
if w.Code != 409 {
t.Fatal(action, w.Code, w.Body.String())
}
}
if _, err := a.s.DB.Exec("CREATE TRIGGER audit_fail BEFORE INSERT ON audit BEGIN SELECT RAISE(FAIL,'audit unavailable'); END"); err != nil {
t.Fatal(err)
}
if w := admit("compatible"); w.Code != 500 {
t.Fatal(w.Code, w.Body.String())
}
im, err = a.evaluateImage(context.Background(), id)
if err != nil || im.Admission != "blocked" {
t.Fatal(im, err)
}
}
func TestAdmissionExperimentalAndPreflight(t *testing.T) {
id := "sha256:" + strings.Repeat("b", 64)
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { writeImageInspect(w, id, "9.8.7") })
seedVerifiedImage(t, a, id, "9.8.7")
if _, err := a.s.DB.Exec("DELETE FROM image_admissions; UPDATE image_builds SET experimental=1"); err != nil {
t.Fatal(err)
}
im, err := a.evaluateImage(context.Background(), id)
if err != nil || im.Status != "experimental" || im.Selectable {
t.Fatal(im, err)
}
for _, tc := range []struct {
handler http.HandlerFunc
body string
want int
}{{a.builds, `{"version":"9.8.7"}`, 400}, {a.builds, `{"version":"9.8.7","experimental":true}`, 403}, {a.intents, `{"name":"unknown","version":"9.8.7"}`, 400}, {a.instances, fmt.Sprintf(`{"name":"unknown","image_id":%q}`, id), 400}} {
w := httptest.NewRecorder()
tc.handler(w, runtimeRequest("POST", "/api/test", tc.body, sid))
if w.Code != tc.want {
t.Fatal(w.Code, w.Body.String())
}
}
var n int
if err := a.s.DB.QueryRow("SELECT count(*) FROM image_build_requests").Scan(&n); err != nil || n != 0 {
t.Fatal(n, err)
}
if err := a.s.DB.QueryRow("SELECT count(*) FROM creation_intents").Scan(&n); err != nil || n != 0 {
t.Fatal(n, err)
}
if _, err := a.s.DB.Exec("UPDATE users SET admin=1"); err != nil {
t.Fatal(err)
}
w := httptest.NewRecorder()
a.imageAdmission(w, runtimeRequest("POST", "/api/images/admission", fmt.Sprintf(`{"image_id":%q,"state":"compatible","evidence":"accepted after startup validation"}`, id), sid))
if w.Code != 200 {
t.Fatal(w.Body.String())
}
im, err = a.evaluateImage(context.Background(), id)
if err != nil || !im.Selectable {
t.Fatal(im, err)
}
if _, err := a.s.DB.Exec("UPDATE image_builds SET status='failed'"); err != nil {
t.Fatal(err)
}
im, err = a.evaluateImage(context.Background(), id)
if err != nil || im.Selectable || im.Status != "unverified" {
t.Fatal(im, err)
}
}
+10
View File
@@ -34,6 +34,13 @@ type API struct {
// Protected by buildQueue; invalidates image observations across deletions. // Protected by buildQueue; invalidates image observations across deletions.
imageDeleteGeneration uint64 imageDeleteGeneration uint64
// Per-image admission evaluations cached for the intent poller; guarded
// by evalCacheMu and invalidated by imageDeleteGeneration (buildQueue)
// plus a short TTL so Docker observations stay fresh.
evalCacheMu sync.Mutex
evalCache map[string]evalCacheEntry
evalCacheGen uint64
} }
func New(s *store.Store, root string) *API { func New(s *store.Store, root string) *API {
@@ -62,8 +69,11 @@ func (a *API) Handler() http.Handler {
m.HandleFunc("/api/instances", a.instances) m.HandleFunc("/api/instances", a.instances)
m.HandleFunc("/api/instances/", a.instance) m.HandleFunc("/api/instances/", a.instance)
m.HandleFunc("/api/images", a.images) m.HandleFunc("/api/images", a.images)
m.HandleFunc("/api/images/admission", a.imageAdmission)
m.HandleFunc("/api/compatibility", a.compatibility)
m.HandleFunc("/api/latest", a.latest) m.HandleFunc("/api/latest", a.latest)
m.HandleFunc("/api/builds", a.builds) m.HandleFunc("/api/builds", a.builds)
m.HandleFunc("/api/builds/", a.buildDetail)
m.HandleFunc("/api/intents", a.intents) m.HandleFunc("/api/intents", a.intents)
m.HandleFunc("/api/intents/", a.intent) m.HandleFunc("/api/intents/", a.intent)
m.HandleFunc("/api/ws/", a.workspace) m.HandleFunc("/api/ws/", a.workspace)
+28 -2
View File
@@ -157,9 +157,23 @@ func (a *API) refresh(w http.ResponseWriter, r *http.Request) {
fail(w, 401, "authentication required") fail(w, 401, "authentication required")
return return
} }
u, csrf, err := a.s.SessionUser(sid.Value) var uid, csrf string
err = a.s.DB.QueryRow("SELECT uid,csrf FROM sessions WHERE id=? AND expires>?", sid.Value, time.Now().Unix()).Scan(&uid, &csrf)
if err != nil { if err != nil {
if errors.Is(err, sql.ErrNoRows) {
fail(w, 401, "authentication required") fail(w, 401, "authentication required")
} else {
fail(w, 500, "session error")
}
return
}
u, err := a.s.UserByID(uid)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
fail(w, 401, "authentication required")
} else {
fail(w, 500, "session error")
}
return return
} }
old, err := r.Cookie("dsh_refresh") old, err := r.Cookie("dsh_refresh")
@@ -178,8 +192,20 @@ func (a *API) refresh(w http.ResponseWriter, r *http.Request) {
return return
} }
n, err := result.RowsAffected() n, err := result.RowsAffected()
if err != nil || n != 1 { if err != nil {
fail(w, 500, "session error")
return
}
if n != 1 {
var active bool
err = a.s.DB.QueryRow("SELECT EXISTS(SELECT 1 FROM sessions JOIN auth_refresh ON auth_refresh.sid=sessions.id WHERE sessions.id=? AND sessions.expires>?)", sid.Value, time.Now().Unix()).Scan(&active)
if err != nil {
fail(w, 500, "session error")
} else if active {
fail(w, 409, "refresh_conflict")
} else {
fail(w, 401, "authentication required") fail(w, 401, "authentication required")
}
return return
} }
a.authCookies(w, r, sid.Value, u.ID, csrf, token) a.authCookies(w, r, sid.Value, u.ID, csrf, token)
+21 -3
View File
@@ -49,6 +49,21 @@ func cookieValue(cs []*http.Cookie, name string) string {
} }
return "" return ""
} }
func TestRefreshSessionLookupFailureIsServerError(t *testing.T) {
a, req := authFixture(t)
w := req("POST", "/api/login", `{"username":"alice","password":"password-123"}`, "", nil)
cookies := w.Result().Cookies()
a.s.Close()
r := httptest.NewRequest("POST", "/api/refresh", nil)
for _, c := range cookies {
r.AddCookie(c)
}
w = httptest.NewRecorder()
a.refresh(w, r)
if w.Code != 500 || len(w.Result().Cookies()) != 0 {
t.Fatal(w.Code, w.Header(), w.Body.String())
}
}
func TestAccessRefreshLifecycle(t *testing.T) { func TestAccessRefreshLifecycle(t *testing.T) {
a, req := authFixture(t) a, req := authFixture(t)
w := req("POST", "/api/login", `{"username":"alice","password":"password-123"}`, "", nil) w := req("POST", "/api/login", `{"username":"alice","password":"password-123"}`, "", nil)
@@ -92,9 +107,12 @@ func TestAccessRefreshLifecycle(t *testing.T) {
if cookieValue(fresh, "dsh_refresh") == cookieValue(cs, "dsh_refresh") { if cookieValue(fresh, "dsh_refresh") == cookieValue(cs, "dsh_refresh") {
t.Fatal("not rotated") t.Fatal("not rotated")
} }
if w = req("POST", "/api/refresh", "", csrf, cs); w.Code != 401 { if w = req("POST", "/api/refresh", "", csrf, cs); w.Code != 409 {
t.Fatal("replay accepted", w.Code) t.Fatal("replay accepted", w.Code)
} }
if len(w.Result().Cookies()) != 0 || !strings.Contains(w.Body.String(), "refresh_conflict") {
t.Fatal("conflict must not issue credentials", w.Header(), w.Body.String())
}
if w = req("GET", "/api/me", "", "", fresh); w.Code != 200 { if w = req("GET", "/api/me", "", "", fresh); w.Code != 200 {
t.Fatal(w.Code) t.Fatal(w.Code)
} }
@@ -104,7 +122,7 @@ func TestAccessRefreshLifecycle(t *testing.T) {
if w = req("GET", "/api/me", "", "", fresh); w.Code != 401 { if w = req("GET", "/api/me", "", "", fresh); w.Code != 401 {
t.Fatal("logout access valid") t.Fatal("logout access valid")
} }
if w = req("POST", "/api/refresh", "", csrf, fresh); w.Code == 200 { if w = req("POST", "/api/refresh", "", csrf, fresh); w.Code != 403 {
t.Fatal("logout refresh valid") t.Fatal("logout refresh valid")
} }
} }
@@ -180,7 +198,7 @@ func TestRefreshOriginConcurrentRotationAndJWTBinding(t *testing.T) {
for code := range codes { for code := range codes {
counts[code]++ counts[code]++
} }
if counts[200] != 1 || counts[401] != 1 { if counts[200] != 1 || counts[409] != 1 {
t.Fatal(counts) t.Fatal(counts)
} }
b := New(a.s, a.root) b := New(a.s, a.root)
+112
View File
@@ -0,0 +1,112 @@
package api
import (
"context"
"database/sql"
"errors"
"net/http"
"strconv"
"strings"
"time"
)
// Summary queries never load log bodies; history cannot inflate a page response.
const buildSummaryColumns = "id,version,status,error,image_id,created_at,requested_by,COALESCE(recipe_rev,''),COALESCE(platform,''),COALESCE(finished_at,''),experimental"
func scanBuildSummary(row interface{ Scan(...any) error }, b *buildRecord) error {
return row.Scan(&b.ID, &b.Version, &b.Status, &b.Error, &b.ImageID, &b.Created, &b.RequestedBy, &b.RecipeRev, &b.Platform, &b.Finished, &b.Experimental)
}
func (a *API) listBuilds(w http.ResponseWriter, r *http.Request) {
limit, offset := 20, 0
for key, dest := range map[string]*int{"limit": &limit, "offset": &offset} {
if values, ok := r.URL.Query()[key]; ok {
if len(values) != 1 {
fail(w, 400, "invalid pagination")
return
}
value, err := strconv.Atoi(values[0])
if err != nil || value < 0 || (key == "limit" && (value < 1 || value > 100)) {
fail(w, 400, "invalid pagination")
return
}
*dest = value
}
}
ctx, cancel := context.WithTimeout(r.Context(), 15*time.Second)
defer cancel()
buildQueue.Lock()
err := a.ensureBuilds()
buildQueue.Unlock()
if err != nil {
fail(w, 500, "database error")
return
}
rows, err := a.s.DB.QueryContext(ctx, "SELECT "+buildSummaryColumns+" FROM image_builds ORDER BY created_at DESC,id DESC LIMIT ? OFFSET ?", limit+1, offset)
if err != nil {
fail(w, 500, "database error")
return
}
result := []buildRecord{}
for rows.Next() {
var b buildRecord
if err = scanBuildSummary(rows, &b); err != nil {
break
}
result = append(result, b)
}
if err == nil {
err = rows.Err()
}
rows.Close()
if err != nil {
fail(w, 500, "database error")
return
}
hasMore := len(result) > limit
if hasMore {
result = result[:limit]
}
if err := a.reconcileBuilds(ctx, result); err != nil {
fail(w, 502, "docker image reconciliation failed")
return
}
a.startBuildWorker()
w.Header().Set("X-Has-More", strconv.FormatBool(hasMore))
writeJSON(w, result)
}
func (a *API) buildDetail(w http.ResponseWriter, r *http.Request) {
if _, _, err := a.user(r); err != nil {
fail(w, 401, "authentication required")
return
}
if r.Method != http.MethodGet {
fail(w, 405, "method not allowed")
return
}
id := strings.TrimPrefix(r.URL.Path, "/api/builds/")
if id == "" || len(id) > 128 || strings.Contains(id, "/") {
fail(w, 404, "build not found")
return
}
ctx, cancel := context.WithTimeout(r.Context(), 15*time.Second)
defer cancel()
buildQueue.Lock()
err := a.ensureBuilds()
buildQueue.Unlock()
var b buildRecord
if err == nil {
err = a.s.DB.QueryRowContext(ctx, "SELECT "+buildSummaryColumns+",logs FROM image_builds WHERE id=?", id).Scan(&b.ID, &b.Version, &b.Status, &b.Error, &b.ImageID, &b.Created, &b.RequestedBy, &b.RecipeRev, &b.Platform, &b.Finished, &b.Experimental, &b.Logs)
}
if errors.Is(err, sql.ErrNoRows) {
fail(w, 404, "build not found")
return
}
if err != nil {
fail(w, 500, "database error")
return
}
b.Logs = sanitizeBuildLog(b.Logs)
writeJSON(w, b)
}
+167
View File
@@ -0,0 +1,167 @@
package api
import (
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func TestBuildHistoryPaginationAndOnDemandLogs(t *testing.T) {
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Errorf("unexpected Docker request: %s", r.URL) })
buildQueue.Lock()
err := a.ensureBuilds()
buildQueue.Unlock()
if err != nil {
t.Fatal(err)
}
for n := 0; n < 125; n++ {
_, err := a.s.DB.Exec("INSERT INTO image_builds(id,version,status,logs,created_at) VALUES(?,?,?,?,?)", fmt.Sprintf("task-%03d", n), "1.2.3", "failed", "token=secret-value"+string(rune(10))+strings.Repeat("x", buildLogLimit), "2026-01-01")
if err != nil {
t.Fatal(err)
}
}
request := func(target, session, method string) *httptest.ResponseRecorder {
r := httptest.NewRequest(method, target, nil)
if session != "" {
r.AddCookie(&http.Cookie{Name: "dsh_session", Value: session})
addTestAccess(r, session)
}
w := httptest.NewRecorder()
if strings.HasPrefix(r.URL.Path, "/api/builds/") {
a.buildDetail(w, r)
} else {
a.builds(w, r)
}
return w
}
for _, tc := range []struct {
query string
count int
first, more string
}{
{"", 20, "task-124", "true"}, {"?limit=100", 100, "task-124", "true"},
{"?limit=20&offset=120", 5, "task-004", "false"}, {"?offset=125", 0, "", "false"},
{"?limit=20&offset=105", 20, "task-019", "false"}, {"?limit=20&offset=104", 20, "task-020", "true"},
} {
w := request("/api/builds"+tc.query, sid, "GET")
var records []buildRecord
if w.Code != 200 || json.Unmarshal(w.Body.Bytes(), &records) != nil || len(records) != tc.count {
t.Fatalf("%s: %d %s", tc.query, w.Code, w.Body.String())
}
if tc.count > 0 && records[0].ID != tc.first {
t.Fatal(records[0].ID)
}
if w.Header().Get("X-Has-More") != tc.more || strings.Contains(w.Body.String(), "logs") || strings.Contains(w.Body.String(), "secret-value") {
t.Fatal("unbounded or invalid summary response", w.Body.String())
}
}
for _, query := range []string{"?limit=0", "?limit=101", "?limit=-1", "?offset=-1", "?offset=x", "?limit=1&limit=2", "?offset=9999999999999999999999999"} {
if w := request("/api/builds"+query, sid, "GET"); w.Code != 400 {
t.Fatal(query, w.Code)
}
}
w := request("/api/builds/task-000", sid, "GET")
var b buildRecord
if w.Code != 200 || json.Unmarshal(w.Body.Bytes(), &b) != nil || b.ID != "task-000" || b.Logs == "" || len(b.Logs) > buildLogLimit || strings.Contains(b.Logs, "secret-value") {
t.Fatal(w.Code, w.Body.String())
}
for _, path := range []string{"/api/builds/missing", "/api/builds/", "/api/builds/task-000/extra"} {
if w := request(path, sid, "GET"); w.Code != 404 {
t.Fatal(path, w.Code)
}
}
if w := request("/api/builds/task-000", "", "GET"); w.Code != 401 {
t.Fatal(w.Code)
}
if w := request("/api/builds", "", "GET"); w.Code != 401 {
t.Fatal(w.Code)
}
if w := request("/api/builds/task-000", sid, "POST"); w.Code != 405 {
t.Fatal(w.Code)
}
}
func TestBuildDetailRouteAndAccess(t *testing.T) {
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("unexpected Docker call") })
buildQueue.Lock()
err := a.ensureBuilds()
buildQueue.Unlock()
if err != nil {
t.Fatal(err)
}
if _, err = a.s.DB.Exec("INSERT INTO image_builds(id,version,status,created_at) VALUES('task','1.2.3','failed','now')"); err != nil {
t.Fatal(err)
}
_, csrf, err := a.s.SessionUser(sid)
if err != nil {
t.Fatal(err)
}
for _, tc := range []struct {
path, method string
access bool
status int
}{
{"/api/builds/task", "GET", true, 200}, {"/api/builds/task", "GET", false, 401},
{"/api/builds/task", "POST", true, 405}, {"/api/builds/", "GET", true, 404},
{"/api/builds/task%2Fextra", "GET", true, 404}, {"/api/builds/task/extra", "GET", true, 404},
} {
r := httptest.NewRequest(tc.method, tc.path, nil)
r.AddCookie(&http.Cookie{Name: "dsh_session", Value: sid})
if tc.access {
addTestAccess(r, sid)
}
if tc.method != "GET" {
r.Header.Set("Origin", "http://example.com")
r.Header.Set("X-CSRF-Token", csrf)
}
w := httptest.NewRecorder()
a.Handler().ServeHTTP(w, r)
if w.Code != tc.status {
t.Fatalf("%s %s: %d %s", tc.method, tc.path, w.Code, w.Body.String())
}
}
}
func TestBuildPageReconcilesOnlySelectedRecords(t *testing.T) {
unavailable := false
calls := 0
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
calls++
if unavailable {
w.WriteHeader(500)
return
}
if !strings.HasPrefix(r.URL.Path, "/v1.47/images/") {
t.Errorf("unexpected request %s", r.URL)
}
w.WriteHeader(404)
})
seedVerifiedImage(t, a, "first-image", "1.2.3")
seedVerifiedImage(t, a, "second-image", "1.2.4")
if _, err := a.s.DB.Exec("UPDATE image_builds SET created_at=CASE version WHEN '1.2.3' THEN 'z' ELSE 'a' END"); err != nil {
t.Fatal(err)
}
request := func(offset int) *httptest.ResponseRecorder {
r := httptest.NewRequest("GET", fmt.Sprintf("/api/builds?limit=1&offset=%d", offset), nil)
r.AddCookie(&http.Cookie{Name: "dsh_session", Value: sid})
addTestAccess(r, sid)
w := httptest.NewRecorder()
a.builds(w, r)
return w
}
w := request(0)
if w.Code != 200 || calls != 1 || !strings.Contains(w.Body.String(), "missing") {
t.Fatal(w.Code, calls, w.Body.String())
}
var state string
if err := a.s.DB.QueryRow("SELECT status FROM image_builds WHERE version='1.2.4'").Scan(&state); err != nil || state != "success" {
t.Fatal(state, err)
}
unavailable = true
if w = request(1); w.Code != 502 {
t.Fatal(w.Code, w.Body.String())
}
}
+21 -4
View File
@@ -7,7 +7,6 @@ import (
"errors" "errors"
"git.misaka.ren/m1saka/dsh-web/internal/backup" "git.misaka.ren/m1saka/dsh-web/internal/backup"
"git.misaka.ren/m1saka/dsh-web/internal/store" "git.misaka.ren/m1saka/dsh-web/internal/store"
"golang.org/x/crypto/bcrypt"
"gopkg.in/yaml.v3" "gopkg.in/yaml.v3"
"io" "io"
"net/http" "net/http"
@@ -87,6 +86,26 @@ func (a *API) config(w http.ResponseWriter, r *http.Request, i store.Instance, a
return return
} }
i = current i = current
if r.Method != http.MethodGet {
admissionMu.RLock()
defer admissionMu.RUnlock()
buildQueue.Lock()
err := a.ensureBuilds()
buildQueue.Unlock()
if err != nil {
fail(w, 500, "database error")
return
}
var blocked int
if err = a.s.DB.QueryRowContext(r.Context(), "SELECT count(*) FROM image_admissions WHERE image_id=? AND state='blocked'", i.ImageID).Scan(&blocked); err != nil {
fail(w, 500, "database error")
return
}
if blocked > 0 {
fail(w, 409, "image is blocked; configuration mutations and automatic restarts denied")
return
}
}
if i.Status == "error" || a.upgradeRecoveryRequired(i) { if i.Status == "error" || a.upgradeRecoveryRequired(i) {
fail(w, 409, "instance recovery required") fail(w, 409, "instance recovery required")
return return
@@ -345,9 +364,7 @@ func (a *API) config(w http.ResponseWriter, r *http.Request, i store.Instance, a
fail(w, 400, "login password required") fail(w, 400, "login password required")
return return
} }
var hash string if !a.reauthenticate(w, i.UID, in.Password) {
if a.s.DB.QueryRow("SELECT pass_hash FROM users WHERE id=?", i.UID).Scan(&hash) != nil || bcrypt.CompareHashAndPassword([]byte(hash), []byte(in.Password)) != nil {
fail(w, 403, "incorrect login password")
return return
} }
if wasRunning { if wasRunning {
+4
View File
@@ -470,6 +470,10 @@ func (a *API) wsUpload(w http.ResponseWriter, r *http.Request, root *os.Root, u
fail(w, 409, "target exists") fail(w, 409, "target exists")
return return
} }
if err := a.s.DB.QueryRow("SELECT quota FROM users WHERE id=?", u.ID).Scan(&u.Quota); err != nil {
fail(w, 413, "quota unavailable")
return
}
used, e := a.userUsage(u) used, e := a.userUsage(u)
if e != nil || h.Size > maxUpload || used+h.Size > u.Quota { if e != nil || h.Size > maxUpload || used+h.Size > u.Quota {
fail(w, 413, "quota or upload limit exceeded") fail(w, 413, "quota or upload limit exceeded")
+3
View File
@@ -446,6 +446,9 @@ func TestWorkspaceSafetyAndUpload(t *testing.T) {
defer s.Close() defer s.Close()
a := &API{s: s, root: t.TempDir()} a := &API{s: s, root: t.TempDir()}
u := store.User{ID: "user", Quota: 8} u := store.User{ID: "user", Quota: 8}
if err := s.CreateUser(u, "unused"); err != nil {
t.Fatal(err)
}
root, e := a.openWorkspace(u) root, e := a.openWorkspace(u)
if e != nil { if e != nil {
t.Fatal(e) t.Fatal(e)
+86
View File
@@ -0,0 +1,86 @@
package api
import (
"archive/zip"
"bytes"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"git.misaka.ren/m1saka/dsh-web/internal/store"
)
// Full-home access is an explicit owner capability, not a credential write-only
// boundary. Keep ownership, authentication and traversal checks independent.
func TestFullHomeOwnerPolicy(t *testing.T) {
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("unexpected Docker operation") })
for _, i := range []store.Instance{{ID: "owned", UID: "owner", Status: "running"}, {ID: "foreign", UID: "other", Status: "running"}} {
if err := a.s.AddInstance(i); err != nil {
t.Fatal(err)
}
home := filepath.Join(a.root, "workspaces", i.UID, "instances", i.ID, "home")
if err := os.MkdirAll(filepath.Join(home, "profiles", "web"), 0700); err != nil {
t.Fatal(err)
}
for path, contents := range map[string]string{".credentials.yaml": "synthetic-test-credential", "dsh-ssh.json": "synthetic-test-ssh", "profiles/web/cordis.patch.yml": "old: value"} {
if err := os.WriteFile(filepath.Join(home, path), []byte(contents), 0600); err != nil {
t.Fatal(err)
}
}
}
request := func(method, path, body, session string) *httptest.ResponseRecorder {
w := httptest.NewRecorder()
a.workspace(w, runtimeRequest(method, path, body, session))
return w
}
for _, name := range []string{".credentials.yaml", "dsh-ssh.json"} {
w := request("GET", "/api/ws/download?instance=owned&path="+name, "", sid)
if w.Code != 200 || w.Body.Len() == 0 {
t.Fatal(name, w.Code, w.Body.String())
}
if w = request("GET", "/api/ws/download?instance=foreign&path="+name, "", sid); w.Code != 404 {
t.Fatal("cross-owner read", w.Code)
}
if w = request("GET", "/api/ws/download?instance=owned&path="+name, "", ""); w.Code != 401 {
t.Fatal("anonymous read", w.Code)
}
}
w := request("GET", "/api/ws/download?instance=owned&path=.", "", sid)
if w.Code != 200 {
t.Fatal(w.Code, w.Body.String())
}
archive, err := zip.NewReader(bytes.NewReader(w.Body.Bytes()), int64(w.Body.Len()))
if err != nil {
t.Fatal(err)
}
found := false
for _, f := range archive.File {
if f.Name == ".credentials.yaml" {
r, e := f.Open()
if e != nil {
t.Fatal(e)
}
data, e := io.ReadAll(r)
r.Close()
if e != nil || string(data) != "synthetic-test-credential" {
t.Fatal("wrong credential archive entry", e)
}
found = true
}
}
if !found {
t.Fatal("owner archive unexpectedly filters credentials")
}
// Running state intentionally does not force the controlled profile workflow.
w = request("POST", "/api/ws/rename?instance=owned", `{"path":"profiles/web/cordis.patch.yml","new_path":"profiles/web/saved.yml"}`, sid)
if w.Code != 200 {
t.Fatal("owner profile rename blocked", w.Code, w.Body.String())
}
w = request("GET", "/api/ws/download?instance=owned&path=../../other/instances/foreign/home/.credentials.yaml", "", sid)
if w.Code != 400 && w.Code != 404 {
t.Fatal("traversal allowed", w.Code)
}
}
+235 -72
View File
@@ -29,10 +29,13 @@ func (a *API) ensureBuilds() error {
return err return err
} }
for _, q := range []string{ for _, q := range []string{
"CREATE INDEX IF NOT EXISTS image_builds_history ON image_builds(created_at DESC,id DESC)",
"ALTER TABLE image_builds ADD COLUMN requested_by TEXT NOT NULL DEFAULT ''", "ALTER TABLE image_builds ADD COLUMN requested_by TEXT NOT NULL DEFAULT ''",
"ALTER TABLE image_builds ADD COLUMN recipe_rev TEXT", "ALTER TABLE image_builds ADD COLUMN recipe_rev TEXT",
"ALTER TABLE image_builds ADD COLUMN platform TEXT", "ALTER TABLE image_builds ADD COLUMN platform TEXT",
"ALTER TABLE image_builds ADD COLUMN finished_at TEXT", "ALTER TABLE image_builds ADD COLUMN finished_at TEXT",
"ALTER TABLE image_builds ADD COLUMN experimental INTEGER NOT NULL DEFAULT 0",
"CREATE TABLE IF NOT EXISTS image_admissions(image_id TEXT NOT NULL,version TEXT NOT NULL,recipe_rev TEXT NOT NULL,platform TEXT NOT NULL,state TEXT NOT NULL,evidence TEXT NOT NULL,admitted_by TEXT NOT NULL,admitted_at TEXT NOT NULL,PRIMARY KEY(image_id,version,recipe_rev,platform))",
"ALTER TABLE images ADD COLUMN recipe_rev TEXT", "ALTER TABLE images ADD COLUMN recipe_rev TEXT",
"ALTER TABLE images ADD COLUMN platform TEXT", "ALTER TABLE images ADD COLUMN platform TEXT",
"ALTER TABLE images ADD COLUMN usage_count INTEGER NOT NULL DEFAULT 0", "ALTER TABLE images ADD COLUMN usage_count INTEGER NOT NULL DEFAULT 0",
@@ -45,6 +48,9 @@ func (a *API) ensureBuilds() error {
if err != nil { if err != nil {
return err return err
} }
if _, err = a.s.DB.Exec("UPDATE image_builds SET status='failed',error='version requires administrator experimental build' WHERE status='queued' AND experimental=0 AND version NOT IN ('0.2.0-rc.1','0.2.0-rc.2')"); err != nil {
return err
}
if !buildQueue.initialized[a.s.DB] { if !buildQueue.initialized[a.s.DB] {
if _, err = a.s.DB.Exec("UPDATE image_builds SET status='failed',error='image template identity is unknown' WHERE status='queued' AND (recipe_rev IS NULL OR recipe_rev<>? OR platform IS NULL OR platform<>?)", imageRecipeRev, imagePlatform); err != nil { if _, err = a.s.DB.Exec("UPDATE image_builds SET status='failed',error='image template identity is unknown' WHERE status='queued' AND (recipe_rev IS NULL OR recipe_rev<>? OR platform IS NULL OR platform<>?)", imageRecipeRev, imagePlatform); err != nil {
return err return err
@@ -66,7 +72,10 @@ func (a *API) builds(w http.ResponseWriter, r *http.Request) {
fail(w, 405, "method not allowed") fail(w, 405, "method not allowed")
return return
} }
var in struct{ Version string } var in struct {
Version string `json:"version"`
Experimental bool `json:"experimental"`
}
if r.Method == http.MethodPost { if r.Method == http.MethodPost {
dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)) dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096))
dec.DisallowUnknownFields() dec.DisallowUnknownFields()
@@ -76,48 +85,19 @@ func (a *API) builds(w http.ResponseWriter, r *http.Request) {
} }
} }
if r.Method == http.MethodGet { if r.Method == http.MethodGet {
buildQueue.Lock() a.listBuilds(w, r)
err := a.ensureBuilds()
var rows *sql.Rows
if err == nil {
rows, err = a.s.DB.Query("SELECT id,version,status,logs,error,image_id,created_at,requested_by,recipe_rev,platform,finished_at FROM image_builds ORDER BY created_at DESC,id DESC")
}
result := []buildRecord{}
if err == nil {
for rows.Next() {
var b buildRecord
var finished, recipe, platform sql.NullString
if err = rows.Scan(&b.ID, &b.Version, &b.Status, &b.Logs, &b.Error, &b.ImageID, &b.Created, &b.RequestedBy, &recipe, &platform, &finished); err != nil {
break
}
b.RecipeRev, b.Platform, b.Finished = recipe.String, platform.String, finished.String
result = append(result, b)
}
scanErr := rows.Err()
rows.Close()
if err == nil {
err = scanErr
}
}
buildQueue.Unlock()
if err != nil {
fail(w, 500, "database error")
return return
} }
// Docker image reconciliation is I/O-bound and updates rows with if in.Experimental && !u.Admin {
// compare-and-set, so it must not hold the queue lock that instance fail(w, 403, "administrator required")
// creation and upgrades also need.
if err := a.reconcileBuilds(r.Context(), result); err != nil {
fail(w, 502, "docker image reconciliation failed")
return return
} }
a.startBuildWorker() b, err := a.enqueueBuild(r.Context(), u.ID, in.Version, in.Experimental)
writeJSON(w, result)
return
}
b, err := a.enqueueBuild(r.Context(), u.ID, in.Version)
if err != nil { if err != nil {
status := 502 status := 502
if errors.Is(err, errBuildUnsupported) {
status = 400
}
if errors.Is(err, errBuildRateLimited) { if errors.Is(err, errBuildRateLimited) {
status = 429 status = 429
w.Header().Set("Retry-After", "3600") w.Header().Set("Retry-After", "3600")
@@ -137,7 +117,7 @@ var errBuildRateLimited = errors.New("build limit: three new tasks per user per
func (a *API) lookupActiveBuild(version string) (buildRecord, bool, error) { func (a *API) lookupActiveBuild(version string) (buildRecord, bool, error) {
var b buildRecord var b buildRecord
var finished, recipe, platform sql.NullString var finished, recipe, platform sql.NullString
err := a.s.DB.QueryRow("SELECT id,version,status,logs,error,image_id,created_at,requested_by,recipe_rev,platform,finished_at FROM image_builds WHERE version=? AND recipe_rev=? AND platform=? AND status IN ('queued','running','validating','success')", version, imageRecipeRev, imagePlatform).Scan(&b.ID, &b.Version, &b.Status, &b.Logs, &b.Error, &b.ImageID, &b.Created, &b.RequestedBy, &recipe, &platform, &finished) err := a.s.DB.QueryRow("SELECT id,version,status,logs,error,image_id,created_at,requested_by,recipe_rev,platform,finished_at,experimental FROM image_builds WHERE version=? AND recipe_rev=? AND platform=? AND status IN ('queued','running','validating','success')", version, imageRecipeRev, imagePlatform).Scan(&b.ID, &b.Version, &b.Status, &b.Logs, &b.Error, &b.ImageID, &b.Created, &b.RequestedBy, &recipe, &platform, &finished, &b.Experimental)
b.RecipeRev, b.Platform, b.Finished = recipe.String, platform.String, finished.String b.RecipeRev, b.Platform, b.Finished = recipe.String, platform.String, finished.String
if errors.Is(err, sql.ErrNoRows) { if errors.Is(err, sql.ErrNoRows) {
return b, false, nil return b, false, nil
@@ -148,7 +128,17 @@ func (a *API) lookupActiveBuild(version string) (buildRecord, bool, error) {
return b, true, nil return b, true, nil
} }
func (a *API) enqueueBuild(ctx context.Context, uid, version string) (buildRecord, error) { func (a *API) enqueueBuild(ctx context.Context, uid, version string, experiment ...bool) (buildRecord, error) {
experimental := len(experiment) > 0 && experiment[0]
if experimental {
u, err := a.s.UserByID(uid)
if err != nil || !u.Admin {
return buildRecord{}, errors.New("administrator required")
}
}
if !experimental && !normalBuildAllowed(version) {
return buildRecord{}, errBuildUnsupported
}
if !validBuildVersion(version) { if !validBuildVersion(version) {
return buildRecord{}, errors.New("a fixed version is required") return buildRecord{}, errors.New("a fixed version is required")
} }
@@ -163,6 +153,9 @@ func (a *API) enqueueBuild(ctx context.Context, uid, version string) (buildRecor
if err != nil { if err != nil {
return b, err return b, err
} }
if found && !experimental && b.Experimental && b.Status != "success" {
return b, errBuildUnsupported
}
if found && b.Status != "success" { if found && b.Status != "success" {
a.startBuildWorker() a.startBuildWorker()
return b, nil return b, nil
@@ -177,6 +170,15 @@ func (a *API) enqueueBuild(ctx context.Context, uid, version string) (buildRecor
return b, lookupErr return b, lookupErr
} }
if id != "" && id == b.ImageID { if id != "" && id == b.ImageID {
if !experimental {
im, e := a.evaluateImage(ctx, id)
if e != nil {
return b, e
}
if !im.Selectable {
return b, errBuildUnsupported
}
}
a.startBuildWorker() a.startBuildWorker()
return b, nil return b, nil
} }
@@ -187,12 +189,15 @@ func (a *API) enqueueBuild(ctx context.Context, uid, version string) (buildRecor
b, found, err = a.lookupActiveBuild(version) b, found, err = a.lookupActiveBuild(version)
} }
if err == nil && !found { if err == nil && !found {
b, err = a.insertBuild(uid, version) b, err = a.insertBuild(uid, version, experimental)
} }
buildQueue.Unlock() buildQueue.Unlock()
if err != nil { if err != nil {
return b, err return b, err
} }
if !experimental && found && b.Experimental {
return b, errBuildUnsupported
}
a.startBuildWorker() a.startBuildWorker()
return b, nil return b, nil
} }
@@ -204,9 +209,12 @@ func (a *API) enqueueBuild(ctx context.Context, uid, version string) (buildRecor
return b, err return b, err
} }
if found { if found {
if !experimental && b.Experimental {
return b, errBuildUnsupported
}
return b, nil return b, nil
} }
b, err = a.insertBuild(uid, version) b, err = a.insertBuild(uid, version, experimental)
if err != nil { if err != nil {
return b, err return b, err
} }
@@ -216,7 +224,7 @@ func (a *API) enqueueBuild(ctx context.Context, uid, version string) (buildRecor
// insertBuild applies the per-user rate limit and inserts a queued record; // insertBuild applies the per-user rate limit and inserts a queued record;
// call with the build queue held. // call with the build queue held.
func (a *API) insertBuild(uid, version string) (buildRecord, error) { func (a *API) insertBuild(uid, version string, experiment ...bool) (buildRecord, error) {
var count int var count int
if err := a.s.DB.QueryRow("SELECT count(*) FROM image_build_requests WHERE user_id=? AND created_at>?", uid, time.Now().Add(-time.Hour).UTC().Format(time.RFC3339Nano)).Scan(&count); err != nil { if err := a.s.DB.QueryRow("SELECT count(*) FROM image_build_requests WHERE user_id=? AND created_at>?", uid, time.Now().Add(-time.Hour).UTC().Format(time.RFC3339Nano)).Scan(&count); err != nil {
return buildRecord{}, err return buildRecord{}, err
@@ -225,6 +233,7 @@ func (a *API) insertBuild(uid, version string) (buildRecord, error) {
return buildRecord{}, errBuildRateLimited return buildRecord{}, errBuildRateLimited
} }
b := buildRecord{ID: store.ID(), Version: version, Status: "queued", Created: time.Now().UTC().Format(time.RFC3339Nano), RequestedBy: uid, RecipeRev: imageRecipeRev, Platform: imagePlatform} b := buildRecord{ID: store.ID(), Version: version, Status: "queued", Created: time.Now().UTC().Format(time.RFC3339Nano), RequestedBy: uid, RecipeRev: imageRecipeRev, Platform: imagePlatform}
b.Experimental = len(experiment) > 0 && experiment[0]
return b, a.insertBuildRequest(b, uid) return b, a.insertBuildRequest(b, uid)
} }
func (a *API) startBuildWorker() { func (a *API) startBuildWorker() {
@@ -241,22 +250,56 @@ func (a *API) startBuildWorkerLocked() {
} }
buildQueue.active[a.s.DB] = true buildQueue.active[a.s.DB] = true
go func() { go func() {
buildWorker.Lock() var lastCleanupAudit time.Time
defer buildWorker.Unlock() var nextCleanup time.Time
var recoveryErr error
for { for {
select {
case <-a.s.Done():
buildQueue.Lock()
delete(buildQueue.active, a.s.DB)
buildQueue.Unlock()
return
default:
}
if time.Now().After(nextCleanup) {
buildWorker.Lock()
recovery, cancel := context.WithTimeout(context.Background(), 20*time.Second)
recoveryErr = a.recoverBuildChecks(recovery)
cancel()
buildWorker.Unlock()
if errors.Is(recoveryErr, errBuildCheckUncertain) {
nextCleanup = time.Now().Add(time.Minute)
}
}
if recoveryErr != nil {
if time.Since(lastCleanupAudit) >= time.Minute {
a.s.AddAudit("", "build.cleanup_error", "", sanitizeBuildLog(recoveryErr.Error()))
lastCleanupAudit = time.Now()
}
}
buildQueue.Lock() buildQueue.Lock()
var b buildRecord var b buildRecord
err := a.s.DB.QueryRow("SELECT id,version FROM image_builds WHERE status='queued' ORDER BY created_at,id LIMIT 1").Scan(&b.ID, &b.Version) err := a.s.DB.QueryRow("SELECT id,version,experimental,requested_by FROM image_builds WHERE status='queued' ORDER BY created_at,id LIMIT 1").Scan(&b.ID, &b.Version, &b.Experimental, &b.RequestedBy)
if err == nil { if err == nil {
_, err = a.s.DB.Exec("UPDATE image_builds SET status='running' WHERE id=?", b.ID) _, err = a.s.DB.Exec("UPDATE image_builds SET status='running' WHERE id=?", b.ID)
} }
if err != nil { if err != nil {
if !errors.Is(err, sql.ErrNoRows) || recoveryErr != nil {
buildQueue.Unlock()
select {
case <-a.s.Done():
case <-time.After(time.Second):
}
continue
}
delete(buildQueue.active, a.s.DB) delete(buildQueue.active, a.s.DB)
buildQueue.Unlock() buildQueue.Unlock()
return return
} }
buildQueue.Unlock() buildQueue.Unlock()
a.runImageBuild(b) a.runImageBuild(b)
nextCleanup = time.Time{}
} }
}() }()
} }
@@ -282,6 +325,7 @@ func imageBuildContext() (io.Reader, error) {
return &buf, nil return &buf, nil
} }
func (a *API) runImageBuild(b buildRecord) { func (a *API) runImageBuild(b buildRecord) {
buildWorker.Lock()
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute) ctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute)
defer cancel() defer cancel()
logs := "" logs := ""
@@ -301,6 +345,15 @@ func (a *API) runImageBuild(b buildRecord) {
} }
} }
tarball, err := imageBuildContext() tarball, err := imageBuildContext()
if !b.Experimental && !normalBuildAllowed(b.Version) {
err = errBuildUnsupported
}
if b.Experimental {
u, e := a.s.UserByID(b.RequestedBy)
if e != nil || !u.Admin {
err = errors.New("experimental build requester is not an administrator")
}
}
var id string var id string
if err == nil { if err == nil {
id, err = a.localBuildImage(ctx, b.Version, "") id, err = a.localBuildImage(ctx, b.Version, "")
@@ -325,7 +378,7 @@ func (a *API) runImageBuild(b buildRecord) {
_, err = a.s.DB.Exec("UPDATE image_builds SET status='validating',image_id=? WHERE id=?", id, b.ID) _, err = a.s.DB.Exec("UPDATE image_builds SET status='validating',image_id=? WHERE id=?", id, b.ID)
} }
if err == nil { if err == nil {
err = a.verifyBuiltImage(ctx, id, b.Version) err = a.verifyBuiltImage(ctx, id, b.Version, b.ID)
} }
if err == nil { if err == nil {
err = logErr err = logErr
@@ -337,31 +390,53 @@ func (a *API) runImageBuild(b buildRecord) {
appendLog(string(rune(10)) + detail) appendLog(string(rune(10)) + detail)
} }
finished := time.Now().UTC().Format(time.RFC3339Nano) finished := time.Now().UTC().Format(time.RFC3339Nano)
_, _ = a.s.DB.Exec("UPDATE image_builds SET status=?,logs=?,error=?,image_id=?,finished_at=? WHERE id=?", status, logs, detail, id, finished, b.ID) buildWorker.Unlock()
if status == "success" && id != "" { if err := a.persistBuildResult(b, status, logs, detail, id, finished); err != nil {
_, _ = a.s.DB.Exec("INSERT INTO images(image_id,version,recipe_rev,platform,tag,status,built_at) VALUES(?,?,?,?,?,?,?) ON CONFLICT(image_id) DO UPDATE SET version=excluded.version,recipe_rev=excluded.recipe_rev,platform=excluded.platform,tag=excluded.tag,status=excluded.status,built_at=excluded.built_at", id, b.Version, imageRecipeRev, imagePlatform, "dsh-web:"+b.Version+"-"+imageRecipeRev, "available", finished) go a.finishBuild(b, status, logs, detail, id, finished)
} }
} }
func (a *API) verifyBuiltImage(parent context.Context, image, version string) (result error) { func (a *API) verifyBuiltImage(parent context.Context, image, version string, buildIDs ...string) (result error) {
ctx, cancel := context.WithTimeout(parent, 90*time.Second) ctx, cancel := context.WithTimeout(parent, 90*time.Second)
defer cancel() defer cancel()
id := store.ID()
if len(buildIDs) != 0 {
id = buildIDs[0]
}
if err := a.ensureBuildChecks(); err != nil {
return err
}
if _, err := a.s.DB.Exec("INSERT INTO image_build_checks(build_id,image_id,created_at) VALUES(?,?,?)", id, image, time.Now().UTC().Format(time.RFC3339Nano)); err != nil {
return err
}
var created struct { var created struct {
ID string `json:"Id"` ID string `json:"Id"`
} }
config := map[string]any{"Image": image, "User": "1000:1000", "Labels": map[string]string{"dsh-mgr.role": "build-check"}, "HostConfig": map[string]any{"NetworkMode": "none", "ReadonlyRootfs": true, "CapDrop": []string{"ALL"}, "SecurityOpt": []string{"no-new-privileges:true"}, "Memory": int64(1 << 30), "NanoCpus": int64(1e9), "PidsLimit": 256, "Tmpfs": map[string]string{"/tmp": "rw,exec,nosuid,nodev,size=256m,mode=1777", "/workspace": "rw,nosuid,nodev,size=256m,uid=1000,gid=1000"}}} rejected := false
if err := a.docker.Do(ctx, http.MethodPost, "/containers/create", config, &created); err != nil { defer func() {
cleanup, c := context.WithTimeout(context.Background(), 10*time.Second)
defer c()
var err error
if rejected {
_, err = a.s.DB.Exec("DELETE FROM image_build_checks WHERE build_id=?", id)
} else if created.ID != "" {
err = a.deleteBuildCheck(cleanup, id, created.ID)
} else {
err = a.cleanupBuildCheck(cleanup, id, image)
}
result = errors.Join(result, err)
}()
config := map[string]any{"Image": image, "User": "1000:1000", "Labels": map[string]string{"dsh-mgr.role": "build-check", "dsh-mgr.build": id}, "HostConfig": map[string]any{"NetworkMode": "none", "ReadonlyRootfs": true, "CapDrop": []string{"ALL"}, "SecurityOpt": []string{"no-new-privileges:true"}, "Memory": int64(1 << 30), "NanoCpus": int64(1e9), "PidsLimit": 256, "Tmpfs": map[string]string{"/tmp": "rw,exec,nosuid,nodev,size=256m,mode=1777", "/workspace": "rw,nosuid,nodev,size=256m,uid=1000,gid=1000"}}}
if err := a.docker.Do(ctx, http.MethodPost, "/containers/create?name="+url.QueryEscape(buildCheckName(id)), config, &created); err != nil {
rejected = dockerCreateRejected(err)
return err return err
} }
if created.ID == "" { if created.ID == "" {
return errors.New("verification container ID missing") return errors.New("verification container ID missing")
} }
if _, err := a.s.DB.Exec("UPDATE image_build_checks SET container_id=? WHERE build_id=?", created.ID, id); err != nil {
return err
}
path := "/containers/" + url.PathEscape(created.ID) path := "/containers/" + url.PathEscape(created.ID)
defer func() {
cleanup, c := context.WithTimeout(context.Background(), 10*time.Second)
defer c()
err := a.docker.Do(cleanup, http.MethodDelete, path+"?force=1&v=1", nil, nil)
result = errors.Join(result, err)
}()
if err := a.docker.Do(ctx, http.MethodPost, path+"/start", nil, nil); err != nil { if err := a.docker.Do(ctx, http.MethodPost, path+"/start", nil, nil); err != nil {
return err return err
} }
@@ -428,6 +503,9 @@ var buildQueue = struct {
var buildWorker sync.Mutex var buildWorker sync.Mutex
type imageRecord struct { type imageRecord struct {
Admission string `json:"admission"`
AdmissionReason string `json:"admission_reason"`
Selectable bool `json:"selectable"`
ID string `json:"id"` ID string `json:"id"`
ImageID string `json:"image_id"` ImageID string `json:"image_id"`
Version string `json:"version"` Version string `json:"version"`
@@ -439,11 +517,12 @@ type imageRecord struct {
UsageCount int `json:"usage_count"` UsageCount int `json:"usage_count"`
} }
type buildRecord struct { type buildRecord struct {
Experimental bool `json:"experimental"`
ImageID string `json:"image_id,omitempty"` ImageID string `json:"image_id,omitempty"`
ID string `json:"id"` ID string `json:"id"`
Version string `json:"version"` Version string `json:"version"`
Status string `json:"status"` Status string `json:"status"`
Logs string `json:"logs"` Logs string `json:"logs,omitempty"`
Error string `json:"error,omitempty"` Error string `json:"error,omitempty"`
Created string `json:"created_at"` Created string `json:"created_at"`
RequestedBy string `json:"requested_by,omitempty"` RequestedBy string `json:"requested_by,omitempty"`
@@ -475,26 +554,110 @@ func (a *API) images(w http.ResponseWriter, r *http.Request) {
writeJSON(w, result) writeJSON(w, result)
} }
func (a *API) verifiedImages(ctx context.Context) (map[string]string, error) { // verifiedImagesTTL bounds how long a cached admission evaluation is reused
buildQueue.Lock() // by the one-second intent poller.
defer buildQueue.Unlock() var verifiedImagesTTL = 5 * time.Second
if err := a.ensureBuilds(); err != nil {
return nil, err type evalCacheEntry struct {
im imageRecord
at time.Time
}
// evaluateImageCached reuses recent per-image evaluations for the intent
// poller so repeated ticks do not hammer Docker. Deletions and admission
// writes bump imageDeleteGeneration and invalidate every entry.
func (a *API) evaluateImageCached(ctx context.Context, gen uint64, id string) (imageRecord, error) {
a.evalCacheMu.Lock()
if entry, ok := a.evalCache[id]; ok && a.evalCacheGen == gen && time.Since(entry.at) < verifiedImagesTTL {
a.evalCacheMu.Unlock()
return entry.im, nil
} }
rows, err := a.s.DB.QueryContext(ctx, "SELECT image_id,version FROM image_builds WHERE status='success' AND image_id<>'' AND recipe_rev=? AND platform=?", imageRecipeRev, imagePlatform) a.evalCacheMu.Unlock()
im, err := a.evaluateImage(ctx, id)
if err != nil {
return im, err
}
a.evalCacheMu.Lock()
if a.evalCache == nil || a.evalCacheGen != gen {
a.evalCache, a.evalCacheGen = make(map[string]evalCacheEntry), gen
}
a.evalCache[id] = evalCacheEntry{im: im, at: time.Now()}
a.evalCacheMu.Unlock()
return im, nil
}
func (a *API) successfulImageIDs(ctx context.Context) ([]string, error) {
rows, err := a.s.DB.QueryContext(ctx, "SELECT DISTINCT image_id FROM image_builds WHERE status='success' AND image_id<>''")
if err != nil { if err != nil {
return nil, err return nil, err
} }
defer rows.Close() ids := []string{}
images := make(map[string]string)
for rows.Next() { for rows.Next() {
var id, version string var id string
if err := rows.Scan(&id, &version); err != nil { if err = rows.Scan(&id); err != nil {
rows.Close()
return nil, err return nil, err
} }
images[id] = version ids = append(ids, id)
} }
return images, rows.Err() err = rows.Err()
rows.Close()
if err != nil {
return nil, err
}
return ids, nil
}
func (a *API) verifiedImages(ctx context.Context) (map[string]string, error) {
buildQueue.Lock()
err := a.ensureBuilds()
buildQueue.Unlock()
if err != nil {
return nil, err
}
ids, err := a.successfulImageIDs(ctx)
if err != nil {
return nil, err
}
result := map[string]string{}
for _, id := range ids {
im, err := a.evaluateImage(ctx, id)
if err != nil {
return nil, err
}
if im.Selectable {
result[id] = im.Version
}
}
return result, nil
}
// verifiedImagesPoll is the intent poller's view: build rows are reread every
// tick, but per-image Docker evaluation is cached briefly so a stuck waiting
// intent cannot turn the one-second loop into Docker hammering.
func (a *API) verifiedImagesPoll(ctx context.Context) (map[string]string, error) {
buildQueue.Lock()
err := a.ensureBuilds()
gen := a.imageDeleteGeneration
buildQueue.Unlock()
if err != nil {
return nil, err
}
ids, err := a.successfulImageIDs(ctx)
if err != nil {
return nil, err
}
result := map[string]string{}
for _, id := range ids {
im, err := a.evaluateImageCached(ctx, gen, id)
if err != nil {
return nil, err
}
if im.Selectable {
result[id] = im.Version
}
}
return result, nil
} }
func (a *API) latest(w http.ResponseWriter, r *http.Request) { func (a *API) latest(w http.ResponseWriter, r *http.Request) {
if _, _, err := a.user(r); err != nil { if _, _, err := a.user(r); err != nil {
@@ -510,7 +673,7 @@ func (a *API) latest(w http.ResponseWriter, r *http.Request) {
fail(w, 502, "npm version lookup failed") fail(w, 502, "npm version lookup failed")
return return
} }
writeJSON(w, map[string]string{"version": v, "checked_at": time.Now().UTC().Format(time.RFC3339)}) writeJSON(w, map[string]any{"version": v, "checked_at": time.Now().UTC().Format(time.RFC3339), "supported": normalBuildAllowed(v), "build_allowed": normalBuildAllowed(v), "admission": defaultAdmission(v, imageRecipeRev, imagePlatform), "admission_reason": "normal builds require a built-in tested tuple; unknown versions require administrator experimental validation and admission"})
} }
func fetchLatest(ctx context.Context, client *http.Client) (string, error) { func fetchLatest(ctx context.Context, client *http.Client) (string, error) {
return fetchNPMVersion(ctx, client, "latest") return fetchNPMVersion(ctx, client, "latest")
+2
View File
@@ -31,6 +31,8 @@ func (a *API) deleteImage(w http.ResponseWriter, r *http.Request) {
// entry before creating containers; the transaction below re-checks that // entry before creating containers; the transaction below re-checks that
// journal, and an upgrade racing an administrator deletion fails safely // journal, and an upgrade racing an administrator deletion fails safely
// with its own rollback path. // with its own rollback path.
admissionMu.Lock()
defer admissionMu.Unlock()
buildQueue.Lock() buildQueue.Lock()
defer buildQueue.Unlock() defer buildQueue.Unlock()
if buildQueue.active[a.s.DB] { if buildQueue.active[a.s.DB] {
@@ -20,7 +20,7 @@ func TestEnqueueBuildConcurrentMissingImageReturnsReplacement(t *testing.T) {
switch r.URL.Path { switch r.URL.Path {
case "/v1.47/images/json": case "/v1.47/images/json":
writeJSON(w, []map[string]any{{"Id": "old-image", "Labels": map[string]string{ writeJSON(w, []map[string]any{{"Id": "old-image", "Labels": map[string]string{
"dsh-mgr.role": "runtime", "dsh-mgr.version": "1.2.3", "dsh-mgr.recipe": imageRecipeRev, "dsh-mgr.role": "runtime", "dsh-mgr.version": "0.2.0-rc.1", "dsh-mgr.recipe": imageRecipeRev,
}}}) }}})
case "/v1.47/images/old-image/json": case "/v1.47/images/old-image/json":
release := make(chan struct{}) release := make(chan struct{})
@@ -40,7 +40,7 @@ func TestEnqueueBuildConcurrentMissingImageReturnsReplacement(t *testing.T) {
w.WriteHeader(http.StatusInternalServerError) w.WriteHeader(http.StatusInternalServerError)
} }
}) })
seedVerifiedImage(t, a, "old-image", "1.2.3") seedVerifiedImage(t, a, "old-image", "0.2.0-rc.1")
var oldID string var oldID string
if err := a.s.DB.QueryRow("SELECT id FROM image_builds WHERE image_id='old-image'").Scan(&oldID); err != nil { if err := a.s.DB.QueryRow("SELECT id FROM image_builds WHERE image_id='old-image'").Scan(&oldID); err != nil {
t.Fatal(err) t.Fatal(err)
@@ -69,7 +69,7 @@ func TestEnqueueBuildConcurrentMissingImageReturnsReplacement(t *testing.T) {
submissions.Add(1) submissions.Add(1)
go func() { go func() {
defer submissions.Done() defer submissions.Done()
b, err := a.enqueueBuild(ctx, "u", "1.2.3") b, err := a.enqueueBuild(ctx, "u", "0.2.0-rc.1")
done <- result{b, err} done <- result{b, err}
}() }()
return done return done
+8 -15
View File
@@ -146,6 +146,8 @@ func (a *API) imageIndexSnapshot(ctx context.Context) ([]imageRecord, error) {
for i := range result { for i := range result {
im := &result[i] im := &result[i]
im.Status = "missing" im.Status = "missing"
im.Admission = "existing_only"
im.AdmissionReason = "local image missing"
actual, err := a.inspectLocalImage(ctx, im.ImageID) actual, err := a.inspectLocalImage(ctx, im.ImageID)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -167,13 +169,11 @@ func (a *API) imageIndexSnapshot(ctx context.Context) ([]imageRecord, error) {
if err := a.s.DB.QueryRowContext(ctx, "SELECT count(*) FROM instances WHERE image_id=?", im.ImageID).Scan(&im.UsageCount); err != nil { if err := a.s.DB.QueryRowContext(ctx, "SELECT count(*) FROM instances WHERE image_id=?", im.ImageID).Scan(&im.UsageCount); err != nil {
return nil, err return nil, err
} }
var verified int policy, err := a.evaluateImage(ctx, im.ImageID)
if err := a.s.DB.QueryRowContext(ctx, "SELECT count(*) FROM image_builds WHERE image_id=? AND version=? AND recipe_rev=? AND platform=? AND status='success'", im.ImageID, im.Version, imageRecipeRev, imagePlatform).Scan(&verified); err != nil { if err != nil {
return nil, err return nil, err
} }
if verified > 0 && actual.ID == im.ImageID && actual.Config.Labels["dsh-mgr.role"] == "runtime" && im.RecipeRev == imageRecipeRev && im.Platform == imagePlatform { im.Status, im.Admission, im.AdmissionReason, im.Selectable = policy.Status, policy.Admission, policy.AdmissionReason, policy.Selectable
im.Status = "available"
}
} }
// Phase 3: write observed state back under the queue lock (DB only). // Phase 3: write observed state back under the queue lock (DB only).
buildQueue.Lock() buildQueue.Lock()
@@ -240,7 +240,7 @@ func (a *API) insertBuildRequest(b buildRecord, user string) error {
return err return err
} }
defer tx.Rollback() defer tx.Rollback()
if _, err = tx.Exec("INSERT INTO image_builds(id,version,status,created_at,requested_by,recipe_rev,platform) VALUES(?,?,?,?,?,?,?)", b.ID, b.Version, b.Status, b.Created, b.RequestedBy, b.RecipeRev, b.Platform); err != nil { if _, err = tx.Exec("INSERT INTO image_builds(id,version,status,created_at,requested_by,recipe_rev,platform,experimental) VALUES(?,?,?,?,?,?,?,?)", b.ID, b.Version, b.Status, b.Created, b.RequestedBy, b.RecipeRev, b.Platform, b.Experimental); err != nil {
return err return err
} }
if _, err = tx.Exec("INSERT INTO image_build_requests(user_id,build_id,created_at) VALUES(?,?,?)", user, b.ID, b.Created); err != nil { if _, err = tx.Exec("INSERT INTO image_build_requests(user_id,build_id,created_at) VALUES(?,?,?)", user, b.ID, b.Created); err != nil {
@@ -257,18 +257,11 @@ func (a *API) reconcileBuilds(parent context.Context, records []buildRecord) err
if b.Status != "success" { if b.Status != "success" {
continue continue
} }
if b.RecipeRev != imageRecipeRev || b.Platform != imagePlatform { actual, err := a.inspectLocalImage(ctx, b.ImageID)
b.Status, b.Error = "missing", "image template identity is unknown"
if _, err := a.s.DB.Exec("UPDATE image_builds SET status=?,error=? WHERE id=? AND status='success'", b.Status, b.Error, b.ID); err != nil {
return err
}
continue
}
id, err := a.localBuildImage(ctx, b.Version, b.ImageID)
if err != nil { if err != nil {
return err return err
} }
if id == "" || id != b.ImageID { if actual == nil || actual.ID != b.ImageID || actual.Config.Labels["dsh-mgr.role"] != "runtime" || actual.Config.Labels["dsh-mgr.version"] != b.Version || actual.Config.Labels["dsh-mgr.recipe"] != b.RecipeRev || actual.platform() != b.Platform {
b.Status, b.Error = "missing", "image no longer available" b.Status, b.Error = "missing", "image no longer available"
if _, err := a.s.DB.Exec("UPDATE image_builds SET status=?,error=? WHERE id=? AND status='success'", b.Status, b.Error, b.ID); err != nil { if _, err := a.s.DB.Exec("UPDATE image_builds SET status=?,error=? WHERE id=? AND status='success'", b.Status, b.Error, b.ID); err != nil {
return err return err
+68 -27
View File
@@ -20,13 +20,13 @@ func TestImageLegacyMigrationNeverCertifies(t *testing.T) {
io.WriteString(w, "[]") io.WriteString(w, "[]")
return return
} }
writeImageInspect(w, "legacy-image", "1.2.3") writeImageInspect(w, strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/v1.47/images/"), "/json"), "0.2.0-rc.1")
}) })
_, err := a.s.DB.Exec(`CREATE TABLE image_builds(id TEXT PRIMARY KEY,version TEXT NOT NULL,status TEXT NOT NULL,logs TEXT NOT NULL DEFAULT '',error TEXT NOT NULL DEFAULT '',image_id TEXT NOT NULL DEFAULT '',created_at TEXT NOT NULL); _, err := a.s.DB.Exec(`CREATE TABLE image_builds(id TEXT PRIMARY KEY,version TEXT NOT NULL,status TEXT NOT NULL,logs TEXT NOT NULL DEFAULT '',error TEXT NOT NULL DEFAULT '',image_id TEXT NOT NULL DEFAULT '',created_at TEXT NOT NULL);
CREATE UNIQUE INDEX image_builds_version ON image_builds(version) WHERE status IN ('queued','running','validating','success'); CREATE UNIQUE INDEX image_builds_version ON image_builds(version) WHERE status IN ('queued','running','validating','success');
INSERT INTO image_builds(id,version,status,image_id,created_at) VALUES('legacy','1.2.3','success','legacy-image','now'),('queued','1.2.4','queued','','now'); INSERT INTO image_builds(id,version,status,image_id,created_at) VALUES('legacy','0.2.0-rc.1','success','legacy-image','now'),('queued','0.2.0-rc.2','queued','','now');
CREATE TABLE images(image_id TEXT PRIMARY KEY,version TEXT NOT NULL,tag TEXT NOT NULL DEFAULT '',status TEXT NOT NULL,built_at TEXT NOT NULL); CREATE TABLE images(image_id TEXT PRIMARY KEY,version TEXT NOT NULL,tag TEXT NOT NULL DEFAULT '',status TEXT NOT NULL,built_at TEXT NOT NULL);
INSERT INTO images VALUES('legacy-image','1.2.3','','available','now');`) INSERT INTO images VALUES('legacy-image','0.2.0-rc.1','','available','now');`)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -50,9 +50,9 @@ func TestImageLegacyMigrationNeverCertifies(t *testing.T) {
if err = a.s.DB.QueryRow("SELECT status FROM image_builds WHERE id='queued'").Scan(&status); err != nil || status != "failed" { if err = a.s.DB.QueryRow("SELECT status FROM image_builds WHERE id='queued'").Scan(&status); err != nil || status != "failed" {
t.Fatal(status, err) t.Fatal(status, err)
} }
seedVerifiedImage(t, a, "new-image", "1.2.3") seedVerifiedImage(t, a, "new-image", "0.2.0-rc.1")
verified, err = a.verifiedImages(context.Background()) verified, err = a.verifiedImages(context.Background())
if err != nil || len(verified) != 1 || verified["new-image"] != "1.2.3" { if err != nil || len(verified) != 1 || verified["new-image"] != "0.2.0-rc.1" {
t.Fatal(verified, err) t.Fatal(verified, err)
} }
} }
@@ -73,7 +73,7 @@ func TestImageInspectIdentity(t *testing.T) {
return return
} }
parts := strings.Split(imagePlatform, "/") parts := strings.Split(imagePlatform, "/")
labels := map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "1.2.3", "dsh-mgr.recipe": imageRecipeRev} labels := map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "0.2.0-rc.1", "dsh-mgr.recipe": imageRecipeRev}
im := map[string]any{"Id": "image", "Os": parts[0], "Architecture": parts[1], "Config": map[string]any{"Labels": labels}} im := map[string]any{"Id": "image", "Os": parts[0], "Architecture": parts[1], "Config": map[string]any{"Labels": labels}}
switch mode { switch mode {
case "recipe": case "recipe":
@@ -96,7 +96,7 @@ func TestImageInspectIdentity(t *testing.T) {
} }
writeJSON(w, im) writeJSON(w, im)
}) })
ok, err := a.imageMatches(context.Background(), "image", "1.2.3") ok, err := a.imageMatches(context.Background(), "image", "0.2.0-rc.1")
if ok != (mode == "valid") || (err != nil) != (mode == "unavailable") { if ok != (mode == "valid") || (err != nil) != (mode == "unavailable") {
t.Fatal(ok, err) t.Fatal(ok, err)
} }
@@ -129,8 +129,8 @@ func TestImageIndexLocalInventory(t *testing.T) {
return return
} }
parts := strings.Split(imagePlatform, "/") parts := strings.Split(imagePlatform, "/")
im := inspectedImage{ID: "image", Os: parts[0], Architecture: parts[1], Created: "2026-01-01T00:00:00Z", RepoTags: []string{"dsh-web:1.2.3"}} im := inspectedImage{ID: "image", Os: parts[0], Architecture: parts[1], Created: "2026-01-01T00:00:00Z", RepoTags: []string{"dsh-web:0.2.0-rc.1"}}
im.Config.Labels = map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "1.2.3", "dsh-mgr.recipe": imageRecipeRev} im.Config.Labels = map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "0.2.0-rc.1", "dsh-mgr.recipe": imageRecipeRev}
if mode == "old-recipe" { if mode == "old-recipe" {
im.Config.Labels["dsh-mgr.recipe"] = "1" im.Config.Labels["dsh-mgr.recipe"] = "1"
} }
@@ -140,7 +140,7 @@ func TestImageIndexLocalInventory(t *testing.T) {
writeJSON(w, im) writeJSON(w, im)
}) })
if mode != "external" { if mode != "external" {
seedVerifiedImage(t, a, "image", "1.2.3") seedVerifiedImage(t, a, "image", "0.2.0-rc.1")
} }
if mode == "old-recipe" { if mode == "old-recipe" {
if _, err := a.s.DB.Exec("UPDATE image_builds SET recipe_rev='1'"); err != nil { if _, err := a.s.DB.Exec("UPDATE image_builds SET recipe_rev='1'"); err != nil {
@@ -155,8 +155,12 @@ func TestImageIndexLocalInventory(t *testing.T) {
} }
return return
} }
wantStatus := "unverified"
if mode == "old-recipe" {
wantStatus = "existing_only"
}
var images []imageRecord var images []imageRecord
if err := json.Unmarshal(w.Body.Bytes(), &images); err != nil || w.Code != 200 || len(images) != 1 || images[0].Status != "unverified" || images[0].Version != "1.2.3" || images[0].Created != "2026-01-01T00:00:00Z" || images[0].Tag != "dsh-web:1.2.3" { if err := json.Unmarshal(w.Body.Bytes(), &images); err != nil || w.Code != 200 || len(images) != 1 || images[0].Status != wantStatus || images[0].Version != "0.2.0-rc.1" || images[0].Created != "2026-01-01T00:00:00Z" || images[0].Tag != "dsh-web:0.2.0-rc.1" {
t.Fatal(w.Body.String(), err) t.Fatal(w.Body.String(), err)
} }
if mode == "other-platform" && images[0].Platform != strings.Split(imagePlatform, "/")[0]+"/arm/v7" { if mode == "other-platform" && images[0].Platform != strings.Split(imagePlatform, "/")[0]+"/arm/v7" {
@@ -193,10 +197,10 @@ func TestImageIndexUsageMissingAndDockerFailure(t *testing.T) {
case 2: case 2:
w.WriteHeader(404) w.WriteHeader(404)
default: default:
writeImageInspect(w, "image", "1.2.3") writeImageInspect(w, "image", "0.2.0-rc.1")
} }
}) })
seedVerifiedImage(t, a, "image", "1.2.3") seedVerifiedImage(t, a, "image", "0.2.0-rc.1")
if _, err := a.s.DB.Exec(`INSERT INTO instances(id,uid,name,image_id,status,created) VALUES('one','u','one','image','running',0),('two','other-user','two','image','stopped',0),('three','u','three','other','running',0)`); err != nil { if _, err := a.s.DB.Exec(`INSERT INTO instances(id,uid,name,image_id,status,created) VALUES('one','u','one','image','running',0),('two','other-user','two','image','stopped',0),('three','u','three','other','running',0)`); err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -237,21 +241,21 @@ func TestImageIndexUsageMissingAndDockerFailure(t *testing.T) {
func TestBuildRateLimitDoesNotPenalizeSharedTask(t *testing.T) { func TestBuildRateLimitDoesNotPenalizeSharedTask(t *testing.T) {
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/verified/json" { if r.URL.Path == "/v1.47/images/verified/json" {
writeImageInspect(w, "verified", "1.2.3") writeImageInspect(w, "verified", "0.2.0-rc.1")
return return
} }
io.WriteString(w, `[{"Id":"verified","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"1.2.3","dsh-mgr.recipe":"`+imageRecipeRev+`"}}]`) io.WriteString(w, `[{"Id":"verified","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"}}]`)
}) })
seedVerifiedImage(t, a, "verified", "1.2.3") seedVerifiedImage(t, a, "verified", "0.2.0-rc.1")
for _, id := range []string{"a", "b", "c"} { for _, id := range []string{"a", "b", "c"} {
if _, err := a.s.DB.Exec("INSERT INTO image_build_requests VALUES('u',?,?)", id, time.Now().UTC().Format(time.RFC3339Nano)); err != nil { if _, err := a.s.DB.Exec("INSERT INTO image_build_requests VALUES('u',?,?)", id, time.Now().UTC().Format(time.RFC3339Nano)); err != nil {
t.Fatal(err) t.Fatal(err)
} }
} }
if w := imageRequest(a.builds, sid, "POST", `{"version":"1.2.3"}`); w.Code != 202 { if w := imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.1"}`); w.Code != 202 {
t.Fatal(w.Code, w.Body.String()) t.Fatal(w.Code, w.Body.String())
} }
if w := imageRequest(a.builds, sid, "POST", `{"version":"1.2.4"}`); w.Code != 429 || w.Header().Get("Retry-After") == "" { if w := imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.2"}`); w.Code != 429 || w.Header().Get("Retry-After") == "" {
t.Fatal(w.Code, w.Body.String()) t.Fatal(w.Code, w.Body.String())
} }
var n int var n int
@@ -314,7 +318,7 @@ func TestUnknownDockerDiskBlocksBuild(t *testing.T) {
io.WriteString(w, "[]") io.WriteString(w, "[]")
}) })
t.Setenv("DSH_DOCKER_ROOT_VISIBLE", "") t.Setenv("DSH_DOCKER_ROOT_VISIBLE", "")
w := imageRequest(a.builds, sid, "POST", `{"version":"1.2.3"}`) w := imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.1"}`)
if w.Code != 202 { if w.Code != 202 {
t.Fatal(w.Body.String()) t.Fatal(w.Body.String())
} }
@@ -343,19 +347,19 @@ func TestExactNPMMetadata(t *testing.T) {
status int status int
ok bool ok bool
}{ }{
{`{"version":"1.2.3"}`, 200, true}, {`{"version":"0.2.0-rc.1"}`, 200, true},
{`{"version":"1.2.4"}`, 200, false}, {`{"version":"0.2.0-rc.2"}`, 200, false},
{`{"version":"latest"}`, 200, false}, {`{"version":"latest"}`, 200, false},
{`{}`, 404, false}, {`{}`, 404, false},
{`invalid`, 200, false}, {`invalid`, 200, false},
} { } {
client := &http.Client{Transport: imageRoundTrip(func(r *http.Request) (*http.Response, error) { client := &http.Client{Transport: imageRoundTrip(func(r *http.Request) (*http.Response, error) {
if r.URL.Host != "registry.npmjs.org" || !strings.HasSuffix(r.URL.Path, "/1.2.3") { if r.URL.Host != "registry.npmjs.org" || !strings.HasSuffix(r.URL.Path, "/0.2.0-rc.1") {
t.Error(r.URL) t.Error(r.URL)
} }
return &http.Response{StatusCode: test.status, Body: io.NopCloser(strings.NewReader(test.body)), Header: make(http.Header)}, nil return &http.Response{StatusCode: test.status, Body: io.NopCloser(strings.NewReader(test.body)), Header: make(http.Header)}, nil
})} })}
_, err := fetchNPMVersion(context.Background(), client, "1.2.3") _, err := fetchNPMVersion(context.Background(), client, "0.2.0-rc.1")
if (err == nil) != test.ok { if (err == nil) != test.ok {
t.Fatalf("%s: %v", test.body, err) t.Fatalf("%s: %v", test.body, err)
} }
@@ -363,19 +367,56 @@ func TestExactNPMMetadata(t *testing.T) {
} }
func TestBuildListReconcilesMissingAndLatestTimestamp(t *testing.T) { func TestBuildListReconcilesMissingAndLatestTimestamp(t *testing.T) {
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { io.WriteString(w, "[]") }) a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(404) })
seedVerifiedImage(t, a, "deleted", "1.2.3") seedVerifiedImage(t, a, "deleted", "0.2.0-rc.1")
w := imageRequest(a.builds, sid, "GET", "") w := imageRequest(a.builds, sid, "GET", "")
var records []buildRecord var records []buildRecord
if err := json.Unmarshal(w.Body.Bytes(), &records); err != nil || len(records) != 1 || records[0].Status != "missing" { if err := json.Unmarshal(w.Body.Bytes(), &records); err != nil || len(records) != 1 || records[0].Status != "missing" {
t.Fatal(w.Body.String()) t.Fatal(w.Body.String())
} }
w = imageRequest(a.latest, sid, "GET", "") w = imageRequest(a.latest, sid, "GET", "")
var latest map[string]string var latest struct {
Version string `json:"version"`
CheckedAt string `json:"checked_at"`
}
if err := json.Unmarshal(w.Body.Bytes(), &latest); err != nil { if err := json.Unmarshal(w.Body.Bytes(), &latest); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if _, err := time.Parse(time.RFC3339, latest["checked_at"]); err != nil || latest["version"] != "1.2.3" { if _, err := time.Parse(time.RFC3339, latest.CheckedAt); err != nil || latest.Version != "0.2.0-rc.1" {
t.Fatal(latest, err) t.Fatal(latest, err)
} }
} }
// The one-second intent poller must reuse per-image evaluations instead of
// re-inspecting Docker every tick, and a generation bump (deletion or
// admission change) must invalidate the cache.
func TestIntentPollerCachesImageEvaluations(t *testing.T) {
inspects := 0
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/image/json" {
inspects++
writeImageInspect(w, "image", "0.2.0-rc.1")
return
}
w.WriteHeader(404)
})
seedVerifiedImage(t, a, "image", "0.2.0-rc.1")
for n := 0; n < 3; n++ {
verified, err := a.verifiedImagesPoll(context.Background())
if err != nil || len(verified) != 1 || verified["image"] != "0.2.0-rc.1" {
t.Fatal(verified, err)
}
}
if inspects != 1 {
t.Fatalf("poller re-inspected Docker %d times for the same image", inspects)
}
buildQueue.Lock()
a.imageDeleteGeneration++
buildQueue.Unlock()
if _, err := a.verifiedImagesPoll(context.Background()); err != nil {
t.Fatal(err)
}
if inspects != 2 {
t.Fatalf("generation bump did not invalidate the cache: %d inspects", inspects)
}
}
+31 -28
View File
@@ -26,7 +26,7 @@ func imageTestAPI(t *testing.T, h http.HandlerFunc) (*API, string) {
t.Cleanup(func() { buildDiskAvailable = oldDiskAvailable }) t.Cleanup(func() { buildDiskAvailable = oldDiskAvailable })
oldClient := npmBuildClient oldClient := npmBuildClient
npmBuildClient = &http.Client{Transport: imageRoundTrip(func(r *http.Request) (*http.Response, error) { npmBuildClient = &http.Client{Transport: imageRoundTrip(func(r *http.Request) (*http.Response, error) {
return &http.Response{StatusCode: 200, Header: make(http.Header), Body: io.NopCloser(strings.NewReader(`{"version":"1.2.3"}`))}, nil return &http.Response{StatusCode: 200, Header: make(http.Header), Body: io.NopCloser(strings.NewReader(`{"version":"0.2.0-rc.1"}`))}, nil
})} })}
t.Cleanup(func() { npmBuildClient = oldClient }) t.Cleanup(func() { npmBuildClient = oldClient })
root := t.TempDir() root := t.TempDir()
@@ -86,19 +86,19 @@ func imageRequest(h http.HandlerFunc, sid, method, body string) *httptest.Respon
func TestImageListContract(t *testing.T) { func TestImageListContract(t *testing.T) {
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/unverified/json" { if r.URL.Path == "/v1.47/images/unverified/json" {
writeImageInspect(w, "unverified", "1.2.3") writeImageInspect(w, "unverified", "0.2.0-rc.1")
return return
} }
if r.URL.Path == "/v1.47/images/sha256:abc/json" { if r.URL.Path == "/v1.47/images/sha256:abc/json" {
writeImageInspect(w, "sha256:abc", "1.2.3") writeImageInspect(w, "sha256:abc", "0.2.0-rc.1")
return return
} }
if !strings.Contains(r.URL.Query().Get("filters"), "dsh-mgr.role=runtime") { if !strings.Contains(r.URL.Query().Get("filters"), "dsh-mgr.role=runtime") {
t.Error("missing label filter") t.Error("missing label filter")
} }
io.WriteString(w, `[{"Id":"sha256:abc","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"1.2.3","dsh-mgr.recipe":"`+imageRecipeRev+`"},"RepoTags":["runtime:1.2.3"]},{"Id":"other","Labels":{}},{"Id":"unverified","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"1.2.3","dsh-mgr.recipe":"`+imageRecipeRev+`"}},{"Id":"sha256:abc","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"9.9.9"}}]`) io.WriteString(w, `[{"Id":"sha256:abc","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"},"RepoTags":["runtime:0.2.0-rc.1"]},{"Id":"other","Labels":{}},{"Id":"unverified","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"}},{"Id":"sha256:abc","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"9.9.9"}}]`)
}) })
seedVerifiedImage(t, a, "sha256:abc", "1.2.3") seedVerifiedImage(t, a, "sha256:abc", "0.2.0-rc.1")
if w := imageRequest(a.images, "", "GET", ""); w.Code != 401 { if w := imageRequest(a.images, "", "GET", ""); w.Code != 401 {
t.Fatal(w.Code) t.Fatal(w.Code)
} }
@@ -125,6 +125,9 @@ func seedVerifiedImage(t *testing.T, a *API, id, version string) {
if _, err := a.s.DB.Exec("INSERT INTO image_builds(id,version,status,image_id,created_at,recipe_rev,platform) VALUES(?,?,'success',?,'now',?,?)", store.ID(), version, id, imageRecipeRev, imagePlatform); err != nil { if _, err := a.s.DB.Exec("INSERT INTO image_builds(id,version,status,image_id,created_at,recipe_rev,platform) VALUES(?,?,'success',?,'now',?,?)", store.ID(), version, id, imageRecipeRev, imagePlatform); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if _, err := a.s.DB.Exec("INSERT OR REPLACE INTO image_admissions(image_id,version,recipe_rev,platform,state,evidence,admitted_by,admitted_at) VALUES(?,?,?,?,'compatible','test fixture explicit acceptance','test','now')", id, version, imageRecipeRev, imagePlatform); err != nil {
t.Fatal(err)
}
} }
func TestImageContextAndSanitizing(t *testing.T) { func TestImageContextAndSanitizing(t *testing.T) {
r, err := imageBuildContext() r, err := imageBuildContext()
@@ -163,12 +166,12 @@ func TestImageContextAndSanitizing(t *testing.T) {
if len(sanitizeBuildLog(strings.Repeat("x", 2*buildLogLimit))) > buildLogLimit { if len(sanitizeBuildLog(strings.Repeat("x", 2*buildLogLimit))) > buildLogLimit {
t.Fatal("unbounded") t.Fatal("unbounded")
} }
for _, v := range []string{"latest", "*", "^1.2.3", "1.2", "1.2.3;id", "https://example.com", ""} { for _, v := range []string{"latest", "*", "^0.2.0-rc.1", "1.2", "0.2.0-rc.1;id", "https://example.com", ""} {
if validBuildVersion(v) { if validBuildVersion(v) {
t.Fatal(v) t.Fatal(v)
} }
} }
if !validBuildVersion("1.2.3-beta.1") { if !validBuildVersion("0.2.0-rc.1-beta.1") {
t.Fatal("fixed prerelease rejected") t.Fatal("fixed prerelease rejected")
} }
} }
@@ -181,10 +184,10 @@ func TestLatestOfficialRegistry(t *testing.T) {
if r.URL.Host != "registry.npmjs.org" || r.URL.Scheme != "https" { if r.URL.Host != "registry.npmjs.org" || r.URL.Scheme != "https" {
t.Fatal(r.URL) t.Fatal(r.URL)
} }
return &http.Response{StatusCode: 200, Body: io.NopCloser(strings.NewReader(`{"version":"1.2.3"}`)), Header: make(http.Header)}, nil return &http.Response{StatusCode: 200, Body: io.NopCloser(strings.NewReader(`{"version":"0.2.0-rc.1"}`)), Header: make(http.Header)}, nil
})} })}
v, err := fetchLatest(context.Background(), client) v, err := fetchLatest(context.Background(), client)
if err != nil || v != "1.2.3" { if err != nil || v != "0.2.0-rc.1" {
t.Fatalf("%s %v", v, err) t.Fatalf("%s %v", v, err)
} }
} }
@@ -193,21 +196,21 @@ func TestBuildDedupAndVerification(t *testing.T) {
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path { switch r.URL.Path {
case "/v1.47/images/sha256:abc/json", "/v1.47/images/sha256:reuse/json": case "/v1.47/images/sha256:abc/json", "/v1.47/images/sha256:reuse/json":
writeImageInspect(w, strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/v1.47/images/"), "/json"), "1.2.3") writeImageInspect(w, strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/v1.47/images/"), "/json"), "0.2.0-rc.1")
case "/v1.47/images/json": case "/v1.47/images/json":
if calls.Load() == 0 { if calls.Load() == 0 {
io.WriteString(w, "[]") io.WriteString(w, "[]")
} else { } else {
io.WriteString(w, `[{"Id":"sha256:abc","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"1.2.3","dsh-mgr.recipe":"`+imageRecipeRev+`"}}]`) io.WriteString(w, `[{"Id":"sha256:abc","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"}}]`)
} }
case "/v1.47/build": case "/v1.47/build":
calls.Add(1) calls.Add(1)
if r.URL.Query().Get("t") != "dsh-web:1.2.3-"+imageRecipeRev { if r.URL.Query().Get("t") != "dsh-web:0.2.0-rc.1-"+imageRecipeRev {
t.Error("unexpected tag", r.URL.Query()) t.Error("unexpected tag", r.URL.Query())
} }
var args map[string]string var args map[string]string
json.Unmarshal([]byte(r.URL.Query().Get("buildargs")), &args) json.Unmarshal([]byte(r.URL.Query().Get("buildargs")), &args)
if args["DSH_VERSION"] != "1.2.3" { if args["DSH_VERSION"] != "0.2.0-rc.1" {
t.Error(args) t.Error(args)
} }
io.Copy(io.Discard, r.Body) io.Copy(io.Discard, r.Body)
@@ -230,7 +233,7 @@ func TestBuildDedupAndVerification(t *testing.T) {
w.WriteHeader(204) w.WriteHeader(204)
case "/v1.47/containers/check/exec": case "/v1.47/containers/check/exec":
var cfg struct{ Cmd []string } var cfg struct{ Cmd []string }
if err := json.NewDecoder(r.Body).Decode(&cfg); err != nil || len(cfg.Cmd) != 3 || !strings.Contains(cfg.Cmd[2], "execFileSync('dsh',['--version']") || !strings.Contains(cfg.Cmd[2], `actual!=="1.2.3"`) { if err := json.NewDecoder(r.Body).Decode(&cfg); err != nil || len(cfg.Cmd) != 3 || !strings.Contains(cfg.Cmd[2], "execFileSync('dsh',['--version']") || !strings.Contains(cfg.Cmd[2], `actual!=="0.2.0-rc.1"`) {
t.Errorf("missing version verification: %+v %v", cfg, err) t.Errorf("missing version verification: %+v %v", cfg, err)
} }
io.WriteString(w, `{"Id":"probe"}`) io.WriteString(w, `{"Id":"probe"}`)
@@ -252,7 +255,7 @@ func TestBuildDedupAndVerification(t *testing.T) {
wg.Add(1) wg.Add(1)
go func() { go func() {
defer wg.Done() defer wg.Done()
responses <- imageRequest(a.builds, sid, "POST", `{"version":"1.2.3"}`) responses <- imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.1"}`)
}() }()
} }
wg.Wait() wg.Wait()
@@ -290,7 +293,7 @@ func TestBuildDedupAndVerification(t *testing.T) {
if calls.Load() != 1 || deleted.Load() != 1 { if calls.Load() != 1 || deleted.Load() != 1 {
t.Fatalf("builds %d cleanup %d", calls.Load(), deleted.Load()) t.Fatalf("builds %d cleanup %d", calls.Load(), deleted.Load())
} }
for _, body := range []string{`{"version":"latest"}`, `{"version":"1.2.3","url":"bad"}`, `{"version":"1.2.3"} {}`} { for _, body := range []string{`{"version":"latest"}`, `{"version":"0.2.0-rc.1","url":"bad"}`, `{"version":"0.2.0-rc.1"} {}`} {
if w := imageRequest(a.builds, sid, "POST", body); w.Code != 400 { if w := imageRequest(a.builds, sid, "POST", body); w.Code != 400 {
t.Fatal(w.Code) t.Fatal(w.Code)
} }
@@ -310,7 +313,7 @@ func TestVerificationFailureCleansUp(t *testing.T) {
io.WriteString(w, `{"message":"cannot start"}`) io.WriteString(w, `{"message":"cannot start"}`)
} }
}) })
if err := a.verifyBuiltImage(context.Background(), "bad", "1.2.3"); err == nil { if err := a.verifyBuiltImage(context.Background(), "bad", "0.2.0-rc.1"); err == nil {
t.Fatal("accepted failed start") t.Fatal("accepted failed start")
} }
if !deleted.Load() { if !deleted.Load() {
@@ -325,10 +328,10 @@ func TestBuildReuseAndMissingImage(t *testing.T) {
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path { switch r.URL.Path {
case "/v1.47/images/sha256:abc/json", "/v1.47/images/sha256:reuse/json": case "/v1.47/images/sha256:abc/json", "/v1.47/images/sha256:reuse/json":
writeImageInspect(w, strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/v1.47/images/"), "/json"), "1.2.3") writeImageInspect(w, strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/v1.47/images/"), "/json"), "0.2.0-rc.1")
case "/v1.47/images/json": case "/v1.47/images/json":
if exists.Load() { if exists.Load() {
io.WriteString(w, `[{"Id":"sha256:reuse","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"1.2.3","dsh-mgr.recipe":"`+imageRecipeRev+`"}}]`) io.WriteString(w, `[{"Id":"sha256:reuse","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"}}]`)
} else { } else {
io.WriteString(w, "[]") io.WriteString(w, "[]")
} }
@@ -373,7 +376,7 @@ func TestBuildReuseAndMissingImage(t *testing.T) {
} }
post := func() buildRecord { post := func() buildRecord {
t.Helper() t.Helper()
w := imageRequest(a.builds, sid, "POST", `{"version":"1.2.3"}`) w := imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.1"}`)
if w.Code != 202 { if w.Code != 202 {
t.Fatal(w.Body.String()) t.Fatal(w.Body.String())
} }
@@ -405,7 +408,7 @@ func TestBuildRestartRecovery(t *testing.T) {
if err := a.ensureBuilds(); err != nil { if err := a.ensureBuilds(); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if _, err := a.s.DB.Exec("INSERT INTO image_builds(id,version,status,created_at) VALUES('old','1.2.3','running','now')"); err != nil { if _, err := a.s.DB.Exec("INSERT INTO image_builds(id,version,status,created_at) VALUES('old','0.2.0-rc.1','running','now')"); err != nil {
t.Fatal(err) t.Fatal(err)
} }
delete(buildQueue.initialized, a.s.DB) delete(buildQueue.initialized, a.s.DB)
@@ -427,9 +430,9 @@ func TestFailedVerificationNeverAvailable(t *testing.T) {
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path { switch r.URL.Path {
case "/v1.47/images/sha256:abc/json", "/v1.47/images/sha256:reuse/json": case "/v1.47/images/sha256:abc/json", "/v1.47/images/sha256:reuse/json":
writeImageInspect(w, strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/v1.47/images/"), "/json"), "1.2.3") writeImageInspect(w, strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/v1.47/images/"), "/json"), "0.2.0-rc.1")
case "/v1.47/images/json": case "/v1.47/images/json":
writeJSON(w, []any{map[string]any{"Id": imageID, "Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "1.2.3", "dsh-mgr.recipe": imageRecipeRev}}}) writeJSON(w, []any{map[string]any{"Id": imageID, "Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "0.2.0-rc.1", "dsh-mgr.recipe": imageRecipeRev}}})
case "/v1.47/containers/create": case "/v1.47/containers/create":
io.WriteString(w, `{"Id":"check"}`) io.WriteString(w, `{"Id":"check"}`)
case "/v1.47/containers/check/start", "/v1.47/exec/probe/start": case "/v1.47/containers/check/start", "/v1.47/exec/probe/start":
@@ -442,13 +445,13 @@ func TestFailedVerificationNeverAvailable(t *testing.T) {
deleted.Store(true) deleted.Store(true)
w.WriteHeader(204) w.WriteHeader(204)
case "/v1.47/images/" + imageID + "/json": case "/v1.47/images/" + imageID + "/json":
writeImageInspect(w, imageID, "1.2.3") writeImageInspect(w, imageID, "0.2.0-rc.1")
default: default:
t.Errorf("unexpected Docker call %s", r.URL) t.Errorf("unexpected Docker call %s", r.URL)
w.WriteHeader(500) w.WriteHeader(500)
} }
}) })
w := imageRequest(a.builds, sid, "POST", `{"version":"1.2.3"}`) w := imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.1"}`)
if w.Code != 202 { if w.Code != 202 {
t.Fatal(w.Body.String()) t.Fatal(w.Body.String())
} }
@@ -481,12 +484,12 @@ func TestFailedVerificationNeverAvailable(t *testing.T) {
func TestLocalBuildImagePrefersPersistedID(t *testing.T) { func TestLocalBuildImagePrefersPersistedID(t *testing.T) {
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/verified/json" { if r.URL.Path == "/v1.47/images/verified/json" {
writeImageInspect(w, "verified", "1.2.3") writeImageInspect(w, "verified", "0.2.0-rc.1")
return return
} }
io.WriteString(w, `[{"Id":"newer","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"1.2.3","dsh-mgr.recipe":"`+imageRecipeRev+`"}},{"Id":"verified","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"1.2.3","dsh-mgr.recipe":"`+imageRecipeRev+`"}}]`) io.WriteString(w, `[{"Id":"newer","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"}},{"Id":"verified","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"}}]`)
}) })
id, err := a.localBuildImage(context.Background(), "1.2.3", "verified") id, err := a.localBuildImage(context.Background(), "0.2.0-rc.1", "verified")
if err != nil || id != "verified" { if err != nil || id != "verified" {
t.Fatalf("id=%s err=%v", id, err) t.Fatalf("id=%s err=%v", id, err)
} }
+25 -4
View File
@@ -2,6 +2,7 @@ package api
import ( import (
"context" "context"
"database/sql"
"encoding/json" "encoding/json"
"errors" "errors"
"io" "io"
@@ -41,6 +42,10 @@ func (a *API) intents(w http.ResponseWriter, r *http.Request) {
fail(w, 400, "name and fixed version required") fail(w, 400, "name and fixed version required")
return return
} }
if !normalBuildAllowed(in.Version) {
fail(w, 400, errBuildUnsupported.Error())
return
}
i := store.Intent{ID: store.ID(), UID: u.ID, Name: strings.TrimSpace(in.Name), Version: in.Version, Host: a.runtimeHost(r), Status: "waiting"} i := store.Intent{ID: store.ID(), UID: u.ID, Name: strings.TrimSpace(in.Name), Version: in.Version, Host: a.runtimeHost(r), Status: "waiting"}
if err = a.s.ReserveIntent(i); err != nil { if err = a.s.ReserveIntent(i); err != nil {
if errors.Is(err, store.ErrInstanceLimit) { if errors.Is(err, store.ErrInstanceLimit) {
@@ -124,21 +129,37 @@ func (a *API) recoverIntents(ctx context.Context) error {
continue continue
} }
i, err := a.s.GetInstance(in.InstanceID) i, err := a.s.GetInstance(in.InstanceID)
if errors.Is(err, sql.ErrNoRows) {
a.finishIntent(in.ID, "failed", "instance reservation missing after process restart")
continue
}
if err != nil {
return err
}
if err == nil && i.Status == "running" { if err == nil && i.Status == "running" {
a.finishIntent(in.ID, "success", "") a.finishIntent(in.ID, "success", "")
continue continue
} }
cleanup, cancel := context.WithTimeout(ctx, 30*time.Second) cleanup, cancel := context.WithTimeout(ctx, 30*time.Second)
// Docker may have created the container before its ID reached SQLite. if err == nil && i.ContainerID == "" {
err = a.removeRuntimeContainer(cleanup, runtimeNetwork(in.InstanceID)) err = a.recoverRuntimeContainer(cleanup, &i)
}
if err == nil && i.ContainerID != "" {
err = a.removeRuntimeContainer(cleanup, i.ContainerID)
}
if err == nil { if err == nil {
err = a.removeRuntimeNetwork(cleanup, in.InstanceID) err = a.removeRuntimeNetwork(cleanup, in.InstanceID)
} }
cancel() cancel()
if err != nil { if err != nil {
return err a.s.SetInstance(in.InstanceID, "error")
a.finishIntent(in.ID, "failed", "instance creation requires recovery: "+err.Error())
a.s.AddAudit(in.UID, "instance.recovery_error", in.InstanceID, err.Error())
continue
} }
a.s.DeleteInstance(in.InstanceID) a.s.DeleteInstance(in.InstanceID)
_, _ = a.s.DB.Exec("DELETE FROM runtime_creates WHERE instance_id=?", in.InstanceID)
_, _ = a.s.DB.Exec("DELETE FROM runtime_networks WHERE instance_id=?", in.InstanceID)
a.finishIntent(in.ID, "failed", "instance creation interrupted by process restart") a.finishIntent(in.ID, "failed", "instance creation interrupted by process restart")
} }
return nil return nil
@@ -180,7 +201,7 @@ func (a *API) processIntents(ctx context.Context) {
continue continue
} }
work, cancel := context.WithTimeout(ctx, 90*time.Second) work, cancel := context.WithTimeout(ctx, 90*time.Second)
verified, err := a.verifiedImages(work) verified, err := a.verifiedImagesPoll(work)
if err != nil { if err != nil {
cancel() cancel()
continue continue
+18 -10
View File
@@ -17,7 +17,7 @@ func TestIntentsReserveCancel(t *testing.T) {
} }
for n := 0; n < 3; n++ { for n := 0; n < 3; n++ {
w := httptest.NewRecorder() w := httptest.NewRecorder()
a.intents(w, runtimeRequest("POST", "/api/intents", `{"name":"one","version":"1.2.3"}`, sid)) a.intents(w, runtimeRequest("POST", "/api/intents", `{"name":"one","version":"0.2.0-rc.1"}`, sid))
want := 202 want := 202
if n == 2 { if n == 2 {
want = 409 want = 409
@@ -49,7 +49,7 @@ func TestIntentBuildSuccessUsesRealRuntime(t *testing.T) {
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch { switch {
case strings.Contains(r.URL.Path, "/images/"): case strings.Contains(r.URL.Path, "/images/"):
writeJSON(w, map[string]any{"Id": image, "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "1.2.3"}}}) writeJSON(w, map[string]any{"Id": image, "Os": "linux", "Architecture": "amd64", "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "0.2.0-rc.1", "dsh-mgr.recipe": imageRecipeRev}}})
case strings.HasSuffix(r.URL.Path, "/containers/create"): case strings.HasSuffix(r.URL.Path, "/containers/create"):
creates++ creates++
writeJSON(w, map[string]string{"Id": "runtime-created"}) writeJSON(w, map[string]string{"Id": "runtime-created"})
@@ -57,15 +57,15 @@ func TestIntentBuildSuccessUsesRealRuntime(t *testing.T) {
w.WriteHeader(204) w.WriteHeader(204)
} }
}) })
seedVerifiedImage(t, a, image, "1.2.3") seedVerifiedImage(t, a, image, "0.2.0-rc.1")
if err := a.s.EnsureIntents(); err != nil { if err := a.s.EnsureIntents(); err != nil {
t.Fatal(err) t.Fatal(err)
} }
var build string var build string
if err := a.s.DB.QueryRow("SELECT id FROM image_builds WHERE version='1.2.3'").Scan(&build); err != nil { if err := a.s.DB.QueryRow("SELECT id FROM image_builds WHERE version='0.2.0-rc.1'").Scan(&build); err != nil {
t.Fatal(err) t.Fatal(err)
} }
in := store.Intent{ID: store.ID(), UID: "owner", Name: "one", Version: "1.2.3", Host: "panel.test"} in := store.Intent{ID: store.ID(), UID: "owner", Name: "one", Version: "0.2.0-rc.1", Host: "panel.test"}
if err := a.s.ReserveIntent(in); err != nil { if err := a.s.ReserveIntent(in); err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -79,21 +79,21 @@ func TestIntentBuildSuccessUsesRealRuntime(t *testing.T) {
t.Fatalf("%+v err=%v creates=%d", list, err, creates) t.Fatalf("%+v err=%v creates=%d", list, err, creates)
} }
instances, err := a.s.Instances("owner") instances, err := a.s.Instances("owner")
if err != nil || len(instances) != 1 || instances[0].ImageID != image || instances[0].Version != "1.2.3" { if err != nil || len(instances) != 1 || instances[0].ImageID != image || instances[0].Version != "0.2.0-rc.1" {
t.Fatalf("%+v %v", instances, err) t.Fatalf("%+v %v", instances, err)
} }
} }
func TestIntentFailedBuildReleasesAndCancelPreservesBuild(t *testing.T) { func TestIntentFailedBuildReleasesAndCancelPreservesBuild(t *testing.T) {
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("failed intent contacted Docker") }) a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("failed intent contacted Docker") })
seedVerifiedImage(t, a, "sha256:"+strings.Repeat("b", 64), "1.2.3") seedVerifiedImage(t, a, "sha256:"+strings.Repeat("b", 64), "0.2.0-rc.1")
if err := a.s.EnsureIntents(); err != nil { if err := a.s.EnsureIntents(); err != nil {
t.Fatal(err) t.Fatal(err)
} }
var build string var build string
a.s.DB.QueryRow("SELECT id FROM image_builds LIMIT 1").Scan(&build) a.s.DB.QueryRow("SELECT id FROM image_builds LIMIT 1").Scan(&build)
for _, id := range []string{"cancel", "fail"} { for _, id := range []string{"cancel", "fail"} {
if err := a.s.ReserveIntent(store.Intent{ID: id, UID: "owner", Version: "1.2.3"}); err != nil { if err := a.s.ReserveIntent(store.Intent{ID: id, UID: "owner", Version: "0.2.0-rc.1"}); err != nil {
t.Fatal(err) t.Fatal(err)
} }
a.s.DB.Exec("UPDATE creation_intents SET build_id=? WHERE id=?", build, id) a.s.DB.Exec("UPDATE creation_intents SET build_id=? WHERE id=?", build, id)
@@ -118,6 +118,14 @@ func TestIntentFailedBuildReleasesAndCancelPreservesBuild(t *testing.T) {
func TestIntentRestartCleansInterruptedCreationAndRetainsWaiting(t *testing.T) { func TestIntentRestartCleansInterruptedCreationAndRetainsWaiting(t *testing.T) {
deletes := 0 deletes := 0
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.Method == "GET" && r.URL.Path == "/v1.47/networks/dsh-inst-reserved" {
w.Write([]byte(`{"Id":"network","Name":"dsh-inst-reserved","Driver":"bridge","Labels":{"dsh-mgr.instance":"reserved"}}`))
return
}
if r.Method == "GET" && r.URL.Path == "/v1.47/containers/dsh-inst-reserved/json" {
w.Write([]byte(`{"Id":"recovered","Name":"/dsh-inst-reserved","Config":{"Labels":{"dsh-mgr.instance":"reserved","dsh-mgr.owner":"owner","dsh-mgr.role":"dsh"}}}`))
return
}
if r.Method == "DELETE" { if r.Method == "DELETE" {
deletes++ deletes++
} }
@@ -127,11 +135,11 @@ func TestIntentRestartCleansInterruptedCreationAndRetainsWaiting(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
for _, id := range []string{"waiting", "creating"} { for _, id := range []string{"waiting", "creating"} {
if err := a.s.ReserveIntent(store.Intent{ID: id, UID: "owner", Version: "1.2.3"}); err != nil { if err := a.s.ReserveIntent(store.Intent{ID: id, UID: "owner", Version: "0.2.0-rc.1"}); err != nil {
t.Fatal(err) t.Fatal(err)
} }
} }
if err := a.s.ClaimIntent("creating", store.Instance{ID: "reserved", UID: "owner", Version: "1.2.3", Status: "creating"}); err != nil { if err := a.s.ClaimIntent("creating", store.Instance{ID: "reserved", UID: "owner", Version: "0.2.0-rc.1", Status: "creating"}); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if err := a.recoverIntents(context.Background()); err != nil { if err := a.recoverIntents(context.Background()); err != nil {
+1 -20
View File
@@ -37,26 +37,7 @@ func (a *API) changePassword(w http.ResponseWriter, r *http.Request) {
fail(w, 401, "authentication required") fail(w, 401, "authentication required")
return return
} }
locked, err := a.accountLocked(u.Username) if !a.reauthenticateLocked(w, u.ID, in.CurrentPassword, http.StatusBadRequest) {
if err != nil {
fail(w, 503, "authentication unavailable")
return
}
if locked {
fail(w, 429, "account temporarily locked; retry later")
return
}
currentHash, err := a.s.Password(u.Username)
if err != nil {
fail(w, 503, "authentication unavailable")
return
}
if len(in.CurrentPassword) > 72 || bcrypt.CompareHashAndPassword([]byte(currentHash), []byte(in.CurrentPassword)) != nil {
if err = a.failedLogin(u.Username); err != nil {
fail(w, 503, "authentication unavailable")
return
}
fail(w, 400, "invalid current password")
return return
} }
hash, err := bcrypt.GenerateFromPassword([]byte(in.Password), bcrypt.DefaultCost) hash, err := bcrypt.GenerateFromPassword([]byte(in.Password), bcrypt.DefaultCost)
+45
View File
@@ -0,0 +1,45 @@
package api
import (
"net/http"
"golang.org/x/crypto/bcrypt"
)
func (a *API) reauthenticate(w http.ResponseWriter, uid, password string) bool {
a.authLoginMu.Lock()
defer a.authLoginMu.Unlock()
return a.reauthenticateLocked(w, uid, password, http.StatusForbidden)
}
// The caller holds authLoginMu; never acquire instance or user locks while holding it.
func (a *API) reauthenticateLocked(w http.ResponseWriter, uid, password string, invalidStatus int) bool {
u, err := a.s.UserByID(uid)
if err != nil {
fail(w, 503, "authentication unavailable")
return false
}
locked, err := a.accountLocked(u.Username)
if err != nil {
fail(w, 503, "authentication unavailable")
return false
}
if locked {
fail(w, 429, "account temporarily locked; retry later")
return false
}
hash, err := a.s.Password(u.Username)
if err != nil {
fail(w, 503, "authentication unavailable")
return false
}
if len(password) > 72 || bcrypt.CompareHashAndPassword([]byte(hash), []byte(password)) != nil {
if err = a.failedLogin(u.Username); err != nil {
fail(w, 503, "authentication unavailable")
return false
}
fail(w, invalidStatus, "invalid current password")
return false
}
return true
}
+335
View File
@@ -0,0 +1,335 @@
package api
import (
"bytes"
"encoding/json"
"io"
"mime/multipart"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
"time"
"git.misaka.ren/m1saka/dsh-web/internal/store"
"golang.org/x/crypto/bcrypt"
)
func TestReauthSlowUpgradeDoesNotBlockOtherLogin(t *testing.T) {
inspecting, release := make(chan struct{}), make(chan struct{})
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet || !strings.HasSuffix(r.URL.Path, "/containers/old/json") {
t.Errorf("unexpected Docker request: %s %s", r.Method, r.URL.Path)
w.WriteHeader(500)
return
}
close(inspecting)
<-release
w.WriteHeader(503)
})
hash, err := bcrypt.GenerateFromPassword([]byte("original-password"), bcrypt.MinCost)
if err != nil {
t.Fatal(err)
}
if _, err := a.s.DB.Exec("UPDATE users SET pass_hash=? WHERE id='owner'", string(hash)); err != nil {
t.Fatal(err)
}
if err := a.s.CreateUser(store.User{ID: "other", Username: "other"}, string(hash)); err != nil {
t.Fatal(err)
}
i := store.Instance{ID: "one", UID: "owner", ContainerID: "old", Status: "exited", Created: time.Now()}
if err := a.s.AddInstance(i); err != nil {
t.Fatal(err)
}
upgradeDone := make(chan struct{})
go func() {
defer close(upgradeDone)
a.upgrade(httptest.NewRecorder(), runtimeRequest("POST", "/api/instances/one/upgrade", `{"image_id":"sha256:`+strings.Repeat("a", 64)+`","password":"original-password"}`, sid), i)
}()
defer func() {
close(release)
select {
case <-upgradeDone:
case <-time.After(5 * time.Second):
t.Error("upgrade did not finish after releasing Docker")
}
}()
select {
case <-inspecting:
case <-upgradeDone:
t.Fatal("upgrade did not reach Docker after reauthentication")
case <-time.After(5 * time.Second):
t.Fatal("upgrade did not reach inspection")
}
loginDone := make(chan *httptest.ResponseRecorder, 1)
go func() {
w := httptest.NewRecorder()
r := runtimeRequest("POST", "/api/login", resetLogin("other", "original-password"), "")
r.Header.Set("Origin", a.origin)
a.login(w, r)
loginDone <- w
}()
select {
case w := <-loginDone:
if w.Code != 200 {
t.Fatal(w.Code, w.Body.String())
}
case <-time.After(5 * time.Second):
t.Fatal("slow upgrade held authentication lock and blocked another user's login")
}
select {
case <-upgradeDone:
t.Fatal("upgrade finished before Docker was released")
default:
}
}
func TestReauthConcurrentFailures(t *testing.T) {
a, i, _ := configTestAPI(t, false)
start := make(chan struct{})
done := make(chan int, 10)
for n := 0; n < 10; n++ {
go func() {
<-start
w := httptest.NewRecorder()
a.reauthenticate(w, i.UID, "wrong")
done <- w.Code
}()
}
close(start)
counts := map[int]int{}
for n := 0; n < 10; n++ {
select {
case code := <-done:
counts[code]++
case <-time.After(5 * time.Second):
t.Fatal("reauthentication deadlocked")
}
}
if counts[403] != 5 || counts[429] != 5 {
t.Fatal(counts)
}
}
func TestReauthOldBackupPasswordIndependent(t *testing.T) {
if os.Geteuid() != 0 {
t.Skip("requires chown")
}
a, i, home := configTestAPI(t, false)
configWrite(t, filepath.Join(home, ".credentials.yaml"), "original")
w := configCall(a, i, "POST", "backups", `{"password":"original-password"}`)
var result struct {
ID string `json:"id"`
}
if w.Code != 200 || json.Unmarshal(w.Body.Bytes(), &result) != nil || result.ID == "" {
t.Fatal(w.Code, w.Body.String())
}
if _, err := a.s.DB.Exec("UPDATE users SET pass_hash='changed' WHERE id=?", i.UID); err != nil {
t.Fatal(err)
}
if _, err := a.s.DB.Exec("INSERT INTO auth_failures VALUES('owner',5,?)", time.Now().Add(time.Hour).Unix()); err != nil {
t.Fatal(err)
}
configWrite(t, filepath.Join(home, ".credentials.yaml"), "changed")
w = configCall(a, i, "POST", "restore", `{"backup_id":"`+result.ID+`","password":"original-password"}`)
if w.Code != 200 {
t.Fatal(w.Code, w.Body.String())
}
content, err := os.ReadFile(filepath.Join(home, ".credentials.yaml"))
if err != nil || string(content) != "original" {
t.Fatal(string(content), err)
}
var failures int
if err := a.s.DB.QueryRow("SELECT failures FROM auth_failures WHERE username='owner'").Scan(&failures); err != nil || failures != 5 {
t.Fatal(failures, err)
}
}
func TestReauthSharedFailureLock(t *testing.T) {
a, i, _ := configTestAPI(t, false)
sid, _, err := a.s.Session(i.UID)
if err != nil {
t.Fatal(err)
}
registerTestAccess(t, a, sid)
backup := func(password string) *httptest.ResponseRecorder {
return configCall(a, i, "POST", "backups", `{"password":"`+password+`"}`)
}
upgrade := func(password string) *httptest.ResponseRecorder {
w := httptest.NewRecorder()
a.upgrade(w, runtimeRequest("POST", "/api/instances/one/upgrade", `{"image_id":"sha256:`+strings.Repeat("a", 64)+`","password":"`+password+`"}`, sid), i)
return w
}
for _, call := range []func(string) *httptest.ResponseRecorder{backup, upgrade, backup, upgrade} {
if w := call("wrong"); w.Code != 403 {
t.Fatal(w.Code, w.Body.String())
}
}
w := httptest.NewRecorder()
a.changePassword(w, runtimeRequest("POST", changePath, changeBody("wrong", resetSecret), sid))
if w.Code != 400 {
t.Fatal(w.Code, w.Body.String())
}
for _, call := range []func(string) *httptest.ResponseRecorder{backup, upgrade} {
if w := call("original-password"); w.Code != 429 {
t.Fatal(w.Code, w.Body.String())
}
}
w = httptest.NewRecorder()
r := runtimeRequest("POST", "/api/login", resetLogin("owner", "original-password"), "")
r.Header.Set("Origin", a.origin)
a.login(w, r)
if w.Code != 429 {
t.Fatal(w.Code, w.Body.String())
}
var failures int
if err := a.s.DB.QueryRow("SELECT failures FROM auth_failures WHERE username='owner'").Scan(&failures); err != nil || failures != 5 {
t.Fatal(failures, err)
}
if _, err := a.s.DB.Exec("UPDATE auth_failures SET locked_until=?", time.Now().Add(-time.Minute).Unix()); err != nil {
t.Fatal(err)
}
if w := upgrade("wrong"); w.Code != 403 {
t.Fatal(w.Code, w.Body.String())
}
if err := a.s.DB.QueryRow("SELECT failures FROM auth_failures WHERE username='owner'").Scan(&failures); err != nil || failures != 1 {
t.Fatal(failures, err)
}
}
func TestReauthFailureStorageUnavailable(t *testing.T) {
a, i, _ := configTestAPI(t, false)
if _, err := a.s.DB.Exec("DROP TABLE auth_failures"); err != nil {
t.Fatal(err)
}
w := configCall(a, i, "POST", "backups", `{"password":"original-password"}`)
if w.Code != 503 {
t.Fatal(w.Code, w.Body.String())
}
}
type quotaPausedBody struct {
io.Reader
started chan struct{}
resume chan struct{}
}
func (b *quotaPausedBody) Read(p []byte) (int, error) {
if b.started != nil {
close(b.started)
b.started = nil
<-b.resume
}
return b.Reader.Read(p)
}
func TestReauthQuotaInFlightUpload(t *testing.T) {
a, sid := runtimeTestAPI(t, func(http.ResponseWriter, *http.Request) {})
if _, err := a.s.DB.Exec("UPDATE users SET admin=1,quota=1048576 WHERE id='owner'"); err != nil {
t.Fatal(err)
}
u, err := a.s.UserByID("owner")
if err != nil {
t.Fatal(err)
}
dir := filepath.Join(a.root, "workspaces", u.ID)
if err := os.MkdirAll(dir, 0700); err != nil {
t.Fatal(err)
}
root, err := os.OpenRoot(dir)
if err != nil {
t.Fatal(err)
}
defer root.Close()
var body bytes.Buffer
form := multipart.NewWriter(&body)
part, err := form.CreateFormFile("file", "large.txt")
if err != nil {
t.Fatal(err)
}
part.Write(bytes.Repeat([]byte("x"), 4096))
form.Close()
started, resume := make(chan struct{}), make(chan struct{})
r := httptest.NewRequest("POST", "/api/ws/upload", &quotaPausedBody{Reader: &body, started: started, resume: resume})
r.Header.Set("Content-Type", form.FormDataContentType())
done := make(chan *httptest.ResponseRecorder, 1)
go func() {
w := httptest.NewRecorder()
a.wsUpload(w, r, root, u)
done <- w
}()
select {
case <-started:
case <-time.After(5 * time.Second):
close(resume)
t.Fatal("upload did not start reading")
}
w := httptest.NewRecorder()
a.users(w, runtimeRequest("PATCH", "/api/users", `{"id":"owner","quota_bytes":1024}`, sid))
close(resume)
if w.Code != 200 {
t.Fatal(w.Code, w.Body.String())
}
select {
case w := <-done:
if w.Code != 413 {
t.Fatal(w.Code, w.Body.String())
}
case <-time.After(5 * time.Second):
t.Fatal("upload deadlocked")
}
entries, err := os.ReadDir(dir)
if err != nil || len(entries) != 0 {
t.Fatal("rejected upload left files", entries, err)
}
}
func TestReauthQuotaUpdateUsesUserLock(t *testing.T) {
a, sid := runtimeTestAPI(t, func(http.ResponseWriter, *http.Request) {})
if _, err := a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='owner'"); err != nil {
t.Fatal(err)
}
lock := a.userLocks.Acquire("owner")
defer func() {
if lock != nil {
lock.Release()
}
}()
done := make(chan *httptest.ResponseRecorder, 1)
go func() {
w := httptest.NewRecorder()
a.users(w, runtimeRequest("PATCH", "/api/users", `{"id":"owner","quota_bytes":1024}`, sid))
done <- w
}()
deadline := time.Now().Add(5 * time.Second)
for {
a.userLocks.mu.Lock()
refs := lock.refs
a.userLocks.mu.Unlock()
if refs == 2 {
break
}
select {
case w := <-done:
t.Fatal("quota update bypassed user lock", w.Code)
default:
}
if time.Now().After(deadline) {
t.Fatal("quota update did not wait on user lock")
}
runtime.Gosched()
}
lock.Release()
lock = nil
select {
case w := <-done:
if w.Code != 200 {
t.Fatal(w.Code, w.Body.String())
}
case <-time.After(5 * time.Second):
t.Fatal("quota update deadlocked")
}
}
+14
View File
@@ -44,6 +44,20 @@ func (a *API) Reconcile(ctx context.Context) error {
} }
known := make(map[string]record) known := make(map[string]record)
for _, r := range records { for _, r := range records {
if r.container == "" && (r.status == "creating" || r.status == "error") {
i, loadErr := a.s.GetInstance(r.id)
if loadErr != nil {
return loadErr
}
if recoveryErr := a.recoverRuntimeContainer(ctx, &i); recoveryErr != nil {
a.s.AddAudit(r.uid, "reconcile.recovery_error", r.id, recoveryErr.Error())
}
r.container = i.ContainerID
r.status = "error"
if _, err = a.s.DB.ExecContext(ctx, "UPDATE instances SET status='error' WHERE id=?", r.id); err != nil {
return err
}
}
known[r.id] = r known[r.id] = r
} }
matched := make(map[string]bool) matched := make(map[string]bool)
+305
View File
@@ -0,0 +1,305 @@
package api
import (
"context"
"database/sql"
"encoding/json"
"errors"
"git.misaka.ren/m1saka/dsh-web/internal/store"
"log"
"net/http"
"net/url"
"os"
"strings"
"time"
)
// pendingColumns whitelists the runtime_creates flag columns so no caller can
// concatenate an unvalidated column name into SQL.
type pendingColumn struct{ flag, stamp string }
var pendingColumns = map[string]pendingColumn{
"container_pending": {flag: "container_pending", stamp: "container_pending_at"},
"network_pending": {flag: "network_pending", stamp: "network_pending_at"},
}
// ensureColumn adds a TEXT column, tolerating databases that already have it.
func (a *API) ensureColumn(table, column string) error {
_, err := a.s.DB.Exec("ALTER TABLE " + table + " ADD COLUMN " + column + " TEXT NOT NULL DEFAULT ''")
if err != nil && strings.Contains(strings.ToLower(err.Error()), "duplicate column") {
return nil
}
return err
}
func (a *API) recoverRuntimeContainer(ctx context.Context, i *store.Instance) error {
c, err := a.docker.Inspect(ctx, runtimeNetwork(i.ID))
if dockerNotFound(err) {
_, pending, checkErr := a.pendingSince(i.ID, "container_pending")
if checkErr != nil {
return checkErr
}
if pending {
return errors.New("container creation outcome remains uncertain")
}
return nil
}
if err != nil {
return err
}
if c.ID == "" || strings.TrimPrefix(c.Name, "/") != runtimeNetwork(i.ID) || c.Config.Labels["dsh-mgr.instance"] != i.ID || c.Config.Labels["dsh-mgr.owner"] != i.UID || c.Config.Labels["dsh-mgr.role"] != "dsh" || c.Image != i.ImageID {
return errors.New("runtime container identity mismatch")
}
if _, err = a.s.DB.ExecContext(ctx, "UPDATE instances SET container_id=? WHERE id=? AND container_id=''", c.ID, i.ID); err != nil {
return err
}
i.ContainerID = c.ID
if err = a.setRuntimeCreatePending(i.ID, "container_pending", false); err != nil {
return err
}
return nil
}
func (a *API) ensureRuntimeCreates() error {
_, err := a.s.DB.Exec("CREATE TABLE IF NOT EXISTS runtime_creates(instance_id TEXT PRIMARY KEY,container_pending INTEGER NOT NULL DEFAULT 0,network_pending INTEGER NOT NULL DEFAULT 0,container_pending_at TEXT NOT NULL DEFAULT '',network_pending_at TEXT NOT NULL DEFAULT '')")
if err != nil {
return err
}
if err = a.ensureColumn("runtime_creates", "container_pending_at"); err != nil {
return err
}
if err = a.ensureColumn("runtime_creates", "network_pending_at"); err != nil {
return err
}
_, err = a.s.DB.Exec("CREATE TABLE IF NOT EXISTS runtime_networks(instance_id TEXT PRIMARY KEY,network_id TEXT NOT NULL)")
return err
}
func (a *API) setRuntimeCreatePending(id, column string, pending bool) error {
if err := a.ensureRuntimeCreates(); err != nil {
return err
}
c, ok := pendingColumns[column]
if !ok {
return errors.New("unknown pending column")
}
stamp := ""
if pending {
stamp = time.Now().UTC().Format(time.RFC3339Nano)
}
_, err := a.s.DB.Exec("INSERT INTO runtime_creates(instance_id,"+c.flag+","+c.stamp+") VALUES(?,?,?) ON CONFLICT(instance_id) DO UPDATE SET "+c.flag+"=excluded."+c.flag+","+c.stamp+"=excluded."+c.stamp, id, pending, stamp)
return err
}
// pendingSince reports when a pending flag was raised. A legacy row without a
// timestamp reports a zero time without resolving the uncertainty.
func (a *API) pendingSince(id, column string) (time.Time, bool, error) {
if err := a.ensureRuntimeCreates(); err != nil {
return time.Time{}, false, err
}
c, ok := pendingColumns[column]
if !ok {
return time.Time{}, false, errors.New("unknown pending column")
}
var pending bool
var stamp string
err := a.s.DB.QueryRow("SELECT "+c.flag+",COALESCE("+c.stamp+",'') FROM runtime_creates WHERE instance_id=?", id).Scan(&pending, &stamp)
if errors.Is(err, sql.ErrNoRows) {
return time.Time{}, false, nil
}
if err != nil {
return time.Time{}, false, err
}
if !pending {
return time.Time{}, false, nil
}
if stamp == "" {
return time.Time{}, true, nil
}
since, perr := time.Parse(time.RFC3339Nano, stamp)
if perr != nil {
return time.Time{}, true, nil
}
return since, true, nil
}
func (a *API) recoverRuntimeNetwork(ctx context.Context, id string) error {
_, pending, err := a.pendingSince(id, "network_pending")
if err != nil {
return err
}
var recorded string
err = a.s.DB.QueryRowContext(ctx, "SELECT network_id FROM runtime_networks WHERE instance_id=?", id).Scan(&recorded)
if err != nil && !errors.Is(err, sql.ErrNoRows) {
return err
}
lookup := runtimeNetwork(id)
if recorded != "" {
lookup = recorded
}
var network struct {
ID, Name, Driver string
Labels map[string]string
Containers map[string]json.RawMessage
}
err = a.docker.Do(ctx, http.MethodGet, "/networks/"+url.PathEscape(lookup), nil, &network)
if dockerNotFound(err) {
if pending && recorded == "" {
return errors.New("network creation outcome remains uncertain")
}
return a.setRuntimeCreatePending(id, "network_pending", false)
}
if err != nil {
return err
}
if network.ID == "" || (recorded != "" && recorded != network.ID) || network.Name != runtimeNetwork(id) || network.Driver != "bridge" || network.Labels["dsh-mgr.instance"] != id {
return errors.New("runtime network identity mismatch")
}
if _, err = a.s.DB.ExecContext(ctx, "INSERT INTO runtime_networks(instance_id,network_id) VALUES(?,?) ON CONFLICT(instance_id) DO UPDATE SET network_id=excluded.network_id", id, network.ID); err != nil {
return err
}
if panel := os.Getenv("PANEL_CONTAINER_ID"); panel != "" && len(network.Containers) != 0 {
if _, connected := network.Containers[panel]; !connected {
c, err := a.docker.Inspect(ctx, panel)
if err != nil {
return err
}
panel = c.ID
}
if _, connected := network.Containers[panel]; connected {
err = a.docker.Do(ctx, http.MethodPost, "/networks/"+url.PathEscape(network.ID)+"/disconnect", map[string]any{"Container": panel, "Force": true}, nil)
if err != nil && !dockerNotFound(err) {
return err
}
}
}
err = a.docker.Do(ctx, http.MethodDelete, "/networks/"+url.PathEscape(network.ID), nil, nil)
if err != nil && !dockerNotFound(err) {
return err
}
return a.setRuntimeCreatePending(id, "network_pending", false)
}
func (a *API) ensureBuildChecks() error {
_, err := a.s.DB.Exec("CREATE TABLE IF NOT EXISTS image_build_checks(build_id TEXT PRIMARY KEY,image_id TEXT NOT NULL)")
if err != nil {
return err
}
if err = a.ensureColumn("image_build_checks", "container_id"); err != nil {
return err
}
return a.ensureColumn("image_build_checks", "created_at")
}
func buildCheckName(id string) string { return "dsh-build-check-" + id }
var errBuildCheckUncertain = errors.New("verification container creation outcome remains uncertain")
func (a *API) cleanupBuildCheck(ctx context.Context, id, image string) error {
var recorded string
if err := a.s.DB.QueryRowContext(ctx, "SELECT container_id FROM image_build_checks WHERE build_id=?", id).Scan(&recorded); err != nil {
return err
}
lookup := buildCheckName(id)
if recorded != "" {
lookup = recorded
}
c, err := a.docker.Inspect(ctx, lookup)
if dockerNotFound(err) {
if recorded != "" {
_, err := a.s.DB.ExecContext(ctx, "DELETE FROM image_build_checks WHERE build_id=?", id)
return err
}
// A name lookup cannot rule out a create still running in Docker.
return errBuildCheckUncertain
}
if err != nil {
return err
}
if c.ID == "" || (recorded != "" && c.ID != recorded) || strings.TrimPrefix(c.Name, "/") != buildCheckName(id) || c.Image != image || c.Config.Labels["dsh-mgr.role"] != "build-check" || c.Config.Labels["dsh-mgr.build"] != id {
return errors.New("build verification container identity mismatch")
}
return a.deleteBuildCheck(ctx, id, c.ID)
}
func (a *API) deleteBuildCheck(ctx context.Context, id, container string) error {
if _, err := a.s.DB.ExecContext(ctx, "UPDATE image_build_checks SET container_id=? WHERE build_id=?", container, id); err != nil {
return err
}
err := a.docker.Do(ctx, http.MethodDelete, "/containers/"+url.PathEscape(container)+"?force=1&v=1", nil, nil)
if err != nil && !dockerNotFound(err) {
return err
}
_, err = a.s.DB.ExecContext(ctx, "DELETE FROM image_build_checks WHERE build_id=?", id)
return err
}
func (a *API) recoverBuildChecks(ctx context.Context) error {
if err := a.ensureBuildChecks(); err != nil {
return err
}
rows, err := a.s.DB.QueryContext(ctx, "SELECT build_id,image_id,COALESCE(created_at,'') FROM image_build_checks")
if err != nil {
return err
}
type check struct{ id, image, createdAt string }
var checks []check
for rows.Next() {
var c check
if err = rows.Scan(&c.id, &c.image, &c.createdAt); err != nil {
break
}
checks = append(checks, c)
}
if err == nil {
err = rows.Err()
}
rows.Close()
if err != nil {
return err
}
var failures []error
missing := false
for _, c := range checks {
err := a.cleanupBuildCheck(ctx, c.id, c.image)
if errors.Is(err, errBuildCheckUncertain) {
missing = true
} else {
failures = append(failures, err)
}
}
if err := errors.Join(failures...); err != nil {
return err
}
if missing {
return errBuildCheckUncertain
}
return nil
}
func (a *API) persistBuildResult(b buildRecord, status, logs, detail, image, finished string) error {
tx, err := a.s.DB.Begin()
if err != nil {
return err
}
defer tx.Rollback()
if status == "success" && image != "" {
_, err = tx.Exec("INSERT INTO images(image_id,version,recipe_rev,platform,tag,status,built_at) VALUES(?,?,?,?,?,?,?) ON CONFLICT(image_id) DO UPDATE SET version=excluded.version,recipe_rev=excluded.recipe_rev,platform=excluded.platform,tag=excluded.tag,status=excluded.status,built_at=excluded.built_at", image, b.Version, imageRecipeRev, imagePlatform, "dsh-web:"+b.Version+"-"+imageRecipeRev, "unverified", finished)
if err != nil {
return err
}
}
if _, err = tx.Exec("UPDATE image_builds SET status=?,logs=?,error=?,image_id=?,finished_at=? WHERE id=?", status, logs, detail, image, finished, b.ID); err != nil {
return err
}
return tx.Commit()
}
func (a *API) finishBuild(b buildRecord, status, logs, detail, image, finished string) {
for attempt := 0; ; attempt++ {
err := a.persistBuildResult(b, status, logs, detail, image, finished)
if err == nil {
return
}
if attempt%60 == 0 {
log.Printf("build %s: terminal transaction failed; retrying: %s", b.ID, sanitizeBuildLog(err.Error()))
}
select {
case <-a.s.Done():
return
case <-time.After(time.Second):
}
}
}
+407
View File
@@ -0,0 +1,407 @@
package api
import (
"bytes"
"context"
"crypto/rand"
"encoding/hex"
"encoding/json"
"errors"
"io"
"net"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
"git.misaka.ren/m1saka/dsh-web/internal/docker"
"git.misaka.ren/m1saka/dsh-web/internal/store"
)
const liveRecoveryLabel = "dsh-test.recovery-run"
type liveRecoveryResource struct{ kind, id string }
type liveRecoveryFault struct {
method, path string
release chan struct{}
done chan struct{}
once sync.Once
}
func (f *liveRecoveryFault) unblock() { f.once.Do(func() { close(f.release) }) }
type liveRecoveryProxy struct {
t *testing.T
run string
transport *http.Transport
mu sync.Mutex
handlers sync.WaitGroup
resources []liveRecoveryResource
fault *liveRecoveryFault
}
func liveRequire(t *testing.T, err error) {
t.Helper()
if err != nil {
t.Fatal(err)
}
}
func (p *liveRecoveryProxy) arm(method, path string, delayed bool) *liveRecoveryFault {
f := &liveRecoveryFault{method: method, path: path, release: make(chan struct{}), done: make(chan struct{})}
if !delayed {
f.unblock()
}
p.mu.Lock()
p.fault = f
p.mu.Unlock()
p.t.Cleanup(f.unblock)
return f
}
func (p *liveRecoveryProxy) ServeHTTP(w http.ResponseWriter, r *http.Request) {
p.handlers.Add(1)
defer p.handlers.Done()
body, err := io.ReadAll(r.Body)
if err != nil {
p.t.Error(err)
panic(http.ErrAbortHandler)
}
path := strings.TrimPrefix(r.URL.Path, "/v1.47")
create := r.Method == "POST" && (path == "/containers/create" || path == "/networks/create")
p.mu.Lock()
allowed := r.Method == "GET"
if create {
var spec struct {
Name string
Labels map[string]string
}
if json.Unmarshal(body, &spec) == nil {
name := spec.Name
if path == "/containers/create" {
name = r.URL.Query().Get("name")
}
allowed = spec.Labels[liveRecoveryLabel] == p.run && strings.Contains(name, p.run)
}
} else if !allowed {
for _, resource := range p.resources {
base := "/" + resource.kind + "/" + resource.id
if path == base || strings.HasPrefix(path, base+"/") {
allowed = true
}
}
if strings.HasSuffix(path, "/connect") || strings.HasSuffix(path, "/disconnect") {
var spec struct{ Container string }
_ = json.Unmarshal(body, &spec)
owned := false
for _, resource := range p.resources {
if resource.kind == "containers" && spec.Container == resource.id {
owned = true
}
}
allowed = allowed && owned
}
}
var fault *liveRecoveryFault
if p.fault != nil && p.fault.method == r.Method && p.fault.path == path {
fault = p.fault
p.fault = nil
}
p.mu.Unlock()
if !allowed {
p.t.Errorf("blocked out-of-scope request: %s %s", r.Method, path)
panic(http.ErrAbortHandler)
}
if fault != nil {
defer close(fault.done)
select {
case <-fault.release:
default:
conn, _, hijackErr := w.(http.Hijacker).Hijack()
if hijackErr != nil {
p.t.Error(hijackErr)
return
}
_ = conn.Close()
<-fault.release
}
}
// A disconnected test client must not cancel the deliberately late daemon operation.
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
req := r.Clone(ctx)
req.URL.Scheme, req.URL.Host, req.RequestURI = "http", "docker", ""
req.Body = io.NopCloser(bytes.NewReader(body))
if len(body) == 0 {
req.Body = http.NoBody
}
resp, err := p.transport.RoundTrip(req)
if err != nil {
p.t.Error(err)
panic(http.ErrAbortHandler)
}
defer resp.Body.Close()
data, err := io.ReadAll(resp.Body)
if err != nil {
p.t.Error(err)
panic(http.ErrAbortHandler)
}
if create && resp.StatusCode >= 200 && resp.StatusCode < 300 {
var result struct{ ID string }
if err := json.Unmarshal(data, &result); err != nil || result.ID == "" {
p.t.Errorf("missing real create ID: %s", data)
} else {
kind := "containers"
if path == "/networks/create" {
kind = "networks"
}
p.mu.Lock()
p.resources = append(p.resources, liveRecoveryResource{kind, result.ID})
p.mu.Unlock()
p.t.Logf("created %s ID=%s run=%s", kind, result.ID, p.run)
}
}
if fault != nil {
p.t.Logf("dropped real response: %s %s status=%d", r.Method, path, resp.StatusCode)
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
p.t.Errorf("fault did not follow successful daemon operation: %s", data)
}
panic(http.ErrAbortHandler)
}
for key, values := range resp.Header {
for _, value := range values {
w.Header().Add(key, value)
}
}
w.WriteHeader(resp.StatusCode)
_, _ = w.Write(data)
}
func newLiveRecovery(t *testing.T) (*API, *liveRecoveryProxy, string) {
t.Helper()
if os.Getenv("DSH_DOCKER_LIVE_TEST") != "1" {
t.Skip("opt in with DSH_DOCKER_LIVE_TEST=1; uses /var/run/docker.sock and local node:24-alpine only")
}
t.Setenv("PANEL_CONTAINER_ID", "")
var nonce [12]byte
_, err := rand.Read(nonce[:])
liveRequire(t, err)
p := &liveRecoveryProxy{t: t, run: "live-" + hex.EncodeToString(nonce[:]), transport: &http.Transport{DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) {
return (&net.Dialer{}).DialContext(ctx, "unix", "/var/run/docker.sock")
}}}
real := docker.New("unix:///var/run/docker.sock")
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
var image struct{ ID string }
liveRequire(t, real.Do(ctx, "GET", "/images/node:24-alpine/json", nil, &image))
s, err := store.Open(filepath.Join(t.TempDir(), "live.db"))
liveRequire(t, err)
t.Cleanup(s.Close)
server := httptest.NewServer(p)
t.Cleanup(func() {
server.Close()
p.handlers.Wait()
p.transport.CloseIdleConnections()
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
p.mu.Lock()
resources := append([]liveRecoveryResource(nil), p.resources...)
p.mu.Unlock()
for _, kind := range []string{"containers", "networks"} {
for _, resource := range resources {
if resource.kind != kind {
continue
}
path := "/" + kind + "/" + resource.id
err := real.Do(ctx, "DELETE", path+"?force=1", nil, nil)
if err != nil && !dockerNotFound(err) {
t.Errorf("cleanup %s: %v", path, err)
}
if kind == "containers" {
path += "/json"
}
if err := real.Do(ctx, "GET", path, nil, nil); !dockerNotFound(err) {
t.Errorf("residual %s: %v", path, err)
} else {
t.Logf("cleanup confirmed absent %s %s", kind, resource.id)
}
}
}
filters, _ := json.Marshal(map[string][]string{"label": {liveRecoveryLabel + "=" + p.run}})
for _, path := range []string{"/containers/json?all=1&", "/networks?"} {
var residual []json.RawMessage
if err := real.Do(ctx, "GET", path+"filters="+url.QueryEscape(string(filters)), nil, &residual); err != nil || len(residual) != 0 {
t.Errorf("label residual check: %s count=%d err=%v", p.run, len(residual), err)
} else {
t.Logf("label residual check empty: %s %s", p.run, path)
}
}
})
return &API{s: s, root: t.TempDir(), docker: docker.New(server.URL)}, p, image.ID
}
func liveContainerSpec(run, image string, labels map[string]string) map[string]any {
labels[liveRecoveryLabel] = run
return map[string]any{"Image": image, "Cmd": []string{"sleep", "600"}, "Labels": labels, "HostConfig": map[string]any{"NetworkMode": "none", "Memory": 64 << 20, "PidsLimit": 32}}
}
func TestDockerLiveRecoveryNetwork(t *testing.T) {
for _, panelMode := range []string{"unset", "unconnected", "connected-id", "connected-name"} {
t.Run(panelMode, func(t *testing.T) {
a, p, image := newLiveRecovery(t)
ctx, cancel := context.WithTimeout(context.Background(), 45*time.Second)
defer cancel()
id := p.run
liveRequire(t, a.setRuntimeCreatePending(id, "network_pending", true))
p.arm("POST", "/networks/create", false)
err := a.docker.Do(ctx, "POST", "/networks/create", map[string]any{"Name": runtimeNetwork(id), "Driver": "bridge", "Labels": map[string]string{"dsh-mgr.instance": id, liveRecoveryLabel: p.run}}, nil)
if err == nil {
t.Fatal("create response was not lost")
}
var network struct{ ID string }
liveRequire(t, a.docker.Do(ctx, "GET", "/networks/"+runtimeNetwork(id), nil, &network))
if panelMode != "unset" {
name := p.run + "-panel"
var panel struct{ ID string }
spec := liveContainerSpec(p.run, image, map[string]string{})
if strings.HasPrefix(panelMode, "connected") {
spec["HostConfig"].(map[string]any)["NetworkMode"] = network.ID
}
liveRequire(t, a.docker.Do(ctx, "POST", "/containers/create?name="+name, spec, &panel))
liveRequire(t, a.docker.Do(ctx, "POST", "/containers/"+panel.ID+"/start", nil, nil))
t.Setenv("PANEL_CONTAINER_ID", panel.ID)
if panelMode == "connected-name" {
t.Setenv("PANEL_CONTAINER_ID", name)
}
var endpoints struct{ Containers map[string]json.RawMessage }
liveRequire(t, a.docker.Do(ctx, "GET", "/networks/"+network.ID, nil, &endpoints))
_, connected := endpoints.Containers[panel.ID]
if connected != strings.HasPrefix(panelMode, "connected") {
t.Fatalf("unexpected real panel connection: %v", connected)
}
}
p.arm("DELETE", "/networks/"+network.ID, false)
if err := a.removeRuntimeNetwork(ctx, id); err == nil {
t.Fatal("delete response was not lost")
}
var recorded string
liveRequire(t, a.s.DB.QueryRow("SELECT network_id FROM runtime_networks WHERE instance_id=?", id).Scan(&recorded))
if recorded != network.ID {
t.Fatalf("lost recorded identity: %s", recorded)
}
liveRequire(t, a.removeRuntimeNetwork(ctx, id))
_, pending, err := a.pendingSince(id, "network_pending")
liveRequire(t, err)
if pending {
t.Fatal("pending did not converge")
}
if err := a.docker.Do(ctx, "GET", "/networks/"+network.ID, nil, nil); !dockerNotFound(err) {
t.Fatalf("network still exists: %v", err)
}
})
}
}
func TestDockerLiveRecoveryLateContainer(t *testing.T) {
for _, build := range []bool{false, true} {
name := "runtime"
if build {
name = "build-check"
}
t.Run(name, func(t *testing.T) {
a, p, image := newLiveRecovery(t)
ctx, cancel := context.WithTimeout(context.Background(), 45*time.Second)
defer cancel()
i := store.Instance{ID: p.run, UID: p.run, ImageID: image, Status: "error", Created: time.Now()}
name := runtimeNetwork(i.ID)
labels := map[string]string{"dsh-mgr.instance": i.ID, "dsh-mgr.owner": i.UID, "dsh-mgr.role": "dsh"}
if build {
name = buildCheckName(i.ID)
labels = map[string]string{"dsh-mgr.build": i.ID, "dsh-mgr.role": "build-check"}
liveRequire(t, a.ensureBuildChecks())
_, err := a.s.DB.Exec("INSERT INTO image_build_checks(build_id,image_id,created_at) VALUES(?,?,?)", i.ID, image, time.Now().Add(-time.Hour).Format(time.RFC3339Nano))
liveRequire(t, err)
} else {
liveRequire(t, a.s.AddInstance(i))
liveRequire(t, a.setRuntimeCreatePending(i.ID, "container_pending", true))
}
fault := p.arm("POST", "/containers/create", true)
if err := a.docker.Do(ctx, "POST", "/containers/create?name="+name, liveContainerSpec(p.run, image, labels), nil); err == nil {
t.Fatal("expected disconnected create")
}
if _, err := a.docker.Inspect(ctx, name); !dockerNotFound(err) {
t.Fatalf("expected real initial 404: %v", err)
}
for attempt := 0; attempt < 2; attempt++ {
if build {
if err := a.recoverBuildChecks(ctx); !errors.Is(err, errBuildCheckUncertain) {
t.Fatalf("uncertainty lost: %v", err)
}
var count int
liveRequire(t, a.s.DB.QueryRow("SELECT count(*) FROM image_build_checks WHERE build_id=?", i.ID).Scan(&count))
if count != 1 {
t.Fatal("build record lost")
}
} else {
if err := a.recoverRuntimeContainer(ctx, &i); err == nil {
t.Fatal("uncertainty lost")
}
_, pending, err := a.pendingSince(i.ID, "container_pending")
liveRequire(t, err)
if !pending {
t.Fatal("pending lost")
}
}
}
fault.unblock()
select {
case <-fault.done:
case <-ctx.Done():
t.Fatal(ctx.Err())
}
container, err := a.docker.Inspect(ctx, name)
liveRequire(t, err)
p.arm("DELETE", "/containers/"+container.ID, false)
if build {
if err := a.recoverBuildChecks(ctx); err == nil {
t.Fatal("delete response was not lost")
}
var recorded string
liveRequire(t, a.s.DB.QueryRow("SELECT container_id FROM image_build_checks WHERE build_id=?", i.ID).Scan(&recorded))
if recorded != container.ID {
t.Fatal("build identity lost")
}
liveRequire(t, a.recoverBuildChecks(ctx))
var count int
liveRequire(t, a.s.DB.QueryRow("SELECT count(*) FROM image_build_checks").Scan(&count))
if count != 0 {
t.Fatal("build record retained")
}
} else {
liveRequire(t, a.recoverRuntimeContainer(ctx, &i))
_, pending, err := a.pendingSince(i.ID, "container_pending")
liveRequire(t, err)
if pending {
t.Fatal("runtime pending did not converge")
}
stored, err := a.s.GetInstance(i.ID)
liveRequire(t, err)
if stored.ContainerID != container.ID {
t.Fatal("runtime identity lost")
}
if err := a.removeRuntimeContainer(ctx, i.ContainerID); err == nil {
t.Fatal("delete response was not lost")
}
liveRequire(t, a.removeRuntimeContainer(ctx, i.ContainerID))
}
if _, err := a.docker.Inspect(ctx, container.ID); !dockerNotFound(err) {
t.Fatalf("late resource not reclaimed: %v", err)
}
})
}
}
+828
View File
@@ -0,0 +1,828 @@
package api
import (
"context"
"encoding/json"
"errors"
"fmt"
"git.misaka.ren/m1saka/dsh-web/internal/docker"
"git.misaka.ren/m1saka/dsh-web/internal/store"
"net/http"
"net/http/httptest"
"path/filepath"
"sync/atomic"
"testing"
"time"
)
func TestBuildCheckRenamedContainerRetainsIdentity(t *testing.T) {
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.Method != "GET" || r.URL.Path != "/v1.47/containers/known/json" {
t.Errorf("unsafe lookup %s %s", r.Method, r.URL.Path)
w.WriteHeader(404)
return
}
fmt.Fprint(w, `{"Id":"known","Name":"/renamed","Image":"image","Config":{"Labels":{"dsh-mgr.role":"build-check","dsh-mgr.build":"task"}}}`)
})
if err := a.ensureBuildChecks(); err != nil {
t.Fatal(err)
}
if _, err := a.s.DB.Exec("INSERT INTO image_build_checks(build_id,image_id,container_id) VALUES('task','image','known')"); err != nil {
t.Fatal(err)
}
if err := a.recoverBuildChecks(context.Background()); err == nil {
t.Fatal("renamed identity accepted")
}
var n int
if err := a.s.DB.QueryRow("SELECT count(*) FROM image_build_checks").Scan(&n); err != nil || n != 1 {
t.Fatalf("record lost %d %v", n, err)
}
}
func TestRuntimeNetworkDeletionLostResponseAndReplacement(t *testing.T) {
for _, foreign := range []bool{false, true} {
t.Run(fmt.Sprint(foreign), func(t *testing.T) {
t.Setenv("PANEL_CONTAINER_ID", "")
deleted := false
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/image/json" {
writeImageInspect(w, "image", "1.2.3")
return
}
if r.Method == "GET" {
if deleted {
if r.URL.Path != "/v1.47/networks/known" {
t.Error("did not use persisted ID")
}
w.WriteHeader(404)
return
}
owner := "net"
if foreign {
owner = "foreign"
}
fmt.Fprintf(w, `{"Id":"known","Name":"dsh-inst-net","Driver":"bridge","Labels":{"dsh-mgr.instance":%q}}`, owner)
return
}
if foreign {
t.Error("deleted foreign network")
}
deleted = true
w.WriteHeader(500)
})
if err := a.setRuntimeCreatePending("net", "network_pending", false); err != nil {
t.Fatal(err)
}
if err := a.removeRuntimeNetwork(context.Background(), "net"); err == nil {
t.Fatal("first error missing")
}
if foreign {
return
}
if err := a.removeRuntimeNetwork(context.Background(), "net"); err != nil {
t.Fatalf("confirmed deletion did not converge: %v", err)
}
})
}
}
func TestStartIntentsUncertainCreationKeepsRecoveryAccessible(t *testing.T) {
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/image/json" {
writeImageInspect(w, "image", "1.2.3")
return
}
if r.Method != "GET" {
t.Error("unexpected destructive call")
}
w.WriteHeader(404)
})
if err := a.s.EnsureIntents(); err != nil {
t.Fatal(err)
}
if err := a.s.ReserveIntent(store.Intent{ID: "intent", UID: "owner"}); err != nil {
t.Fatal(err)
}
if err := a.s.ClaimIntent("intent", store.Instance{ID: "pending", UID: "owner", Status: "creating"}); err != nil {
t.Fatal(err)
}
if err := a.setRuntimeCreatePending("pending", "container_pending", true); err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
if err := a.StartIntents(ctx); err != nil {
t.Fatalf("uncertain creation blocked startup: %v", err)
}
i, err := a.s.GetInstance("pending")
if err != nil || i.Status != "error" {
t.Fatalf("recovery reservation missing %+v %v", i, err)
}
list, err := a.s.Intents("owner")
if err != nil || len(list) != 1 || list[0].Status != "failed" {
t.Fatalf("intent not terminated %+v %v", list, err)
}
}
func TestNetworkCreateLostResponse(t *testing.T) {
for _, mode := range []string{"present", "late", "foreign"} {
t.Run(mode, func(t *testing.T) {
t.Setenv("PANEL_CONTAINER_ID", "")
late := false
deletes := 0
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/image/json" {
writeImageInspect(w, "image", "1.2.3")
return
}
switch r.Method + " " + r.URL.Path {
case "POST /v1.47/networks/create":
w.WriteHeader(500)
case "GET /v1.47/containers/dsh-inst-net/json":
w.WriteHeader(404)
case "GET /v1.47/networks/dsh-inst-net":
if mode == "late" && !late {
w.WriteHeader(404)
return
}
id := "net"
if mode == "foreign" {
id = "other"
}
fmt.Fprintf(w, `{"Id":"network-id","Name":"dsh-inst-net","Driver":"bridge","Labels":{"dsh-mgr.instance":%q}}`, id)
case "DELETE /v1.47/networks/network-id":
deletes++
w.WriteHeader(204)
default:
t.Errorf("unexpected %s %s", r.Method, r.URL.Path)
w.WriteHeader(500)
}
})
seedVerifiedImage(t, a, "image", "1.2.3")
i := store.Instance{ID: "net", UID: "owner", Version: "1.2.3", ImageID: "image", Status: "creating"}
if err := a.s.AddInstance(i); err != nil {
t.Fatal(err)
}
if _, err := a.createRuntime(context.Background(), store.User{ID: "owner"}, i, "panel.test"); err == nil {
t.Fatal("lost response accepted")
}
if mode == "present" {
if deletes != 1 {
t.Fatal("network leaked")
}
return
}
saved, err := a.s.GetInstance("net")
if err != nil || saved.Status != "error" {
t.Fatalf("lost reservation %+v %v", saved, err)
}
late = true
w := httptest.NewRecorder()
a.instance(w, runtimeRequest("DELETE", "/api/instances/net", "", sid))
if mode == "foreign" {
if deletes != 0 || w.Code != 502 {
t.Fatal("foreign network touched")
}
return
}
if w.Code != 200 || deletes != 1 {
t.Fatalf("late network retry %d %d", w.Code, deletes)
}
})
}
}
func TestBuildCheckReopenWorkerWithoutQueuedBuilds(t *testing.T) {
path := filepath.Join(t.TempDir(), "recovery.db")
s, err := store.Open(path)
if err != nil {
t.Fatal(err)
}
a := &API{s: s}
if err = a.ensureBuildChecks(); err != nil {
t.Fatal(err)
}
if _, err = s.DB.Exec("INSERT INTO image_build_checks(build_id,image_id) VALUES('task','image')"); err != nil {
t.Fatal(err)
}
s.Close()
s, err = store.Open(path)
if err != nil {
t.Fatal(err)
}
defer s.Close()
var attempts atomic.Int32
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == "GET" {
fmt.Fprint(w, `{"Id":"check","Name":"/dsh-build-check-task","Image":"image","Config":{"Labels":{"dsh-mgr.role":"build-check","dsh-mgr.build":"task"}}}`)
return
}
if r.Method == "DELETE" {
if attempts.Add(1) == 1 {
w.WriteHeader(500)
} else {
w.WriteHeader(204)
}
return
}
t.Errorf("unexpected %s", r.Method)
}))
defer srv.Close()
a = &API{s: s, docker: docker.New(srv.URL)}
if err = a.StartImageBuilds(); err != nil {
t.Fatal(err)
}
deadline := time.Now().Add(4 * time.Second)
for {
var n int
if err = s.DB.QueryRow("SELECT count(*) FROM image_build_checks").Scan(&n); err != nil {
t.Fatal(err)
}
if n == 0 {
break
}
if time.Now().After(deadline) {
t.Fatal("reopen worker did not retry")
}
time.Sleep(10 * time.Millisecond)
}
if attempts.Load() != 2 {
t.Fatalf("attempts %d", attempts.Load())
}
}
func TestBuildTerminalFailureDoesNotBlockNextTask(t *testing.T) {
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(500) })
buildQueue.Lock()
err := a.ensureBuilds()
buildQueue.Unlock()
if err != nil {
t.Fatal(err)
}
for _, id := range []string{"a", "b"} {
if _, err = a.s.DB.Exec("INSERT INTO image_builds(id,version,status,created_at) VALUES(?,'1.2.3','queued',?)", id, id); err != nil {
t.Fatal(err)
}
}
if _, err = a.s.DB.Exec("CREATE TRIGGER fail_a BEFORE UPDATE ON image_builds WHEN NEW.id='a' AND NEW.status='failed' BEGIN SELECT RAISE(FAIL,'injected'); END"); err != nil {
t.Fatal(err)
}
a.startBuildWorker()
deadline := time.Now().Add(4 * time.Second)
for {
var status string
if err = a.s.DB.QueryRow("SELECT status FROM image_builds WHERE id='b'").Scan(&status); err != nil {
t.Fatal(err)
}
if status == "failed" {
break
}
if time.Now().After(deadline) {
t.Fatal("A terminal retry blocked B")
}
time.Sleep(10 * time.Millisecond)
}
if _, err = a.s.DB.Exec("DROP TRIGGER fail_a"); err != nil {
t.Fatal(err)
}
for {
var status string
if err = a.s.DB.QueryRow("SELECT status FROM image_builds WHERE id='a'").Scan(&status); err != nil {
t.Fatal(err)
}
if status == "failed" {
break
}
if time.Now().After(deadline) {
t.Fatal("A never converged")
}
time.Sleep(10 * time.Millisecond)
}
}
func TestCreateLostResponseRecovery(t *testing.T) {
for _, mode := range []string{"cleanup", "retry", "foreign", "inspect-failure", "late"} {
t.Run(mode, func(t *testing.T) {
t.Setenv("PANEL_CONTAINER_ID", "")
deleted := 0
retry := false
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/image/json" {
writeImageInspect(w, "image", "1.2.3")
return
}
switch r.Method + " " + r.URL.Path {
case "POST /v1.47/networks/create":
fmt.Fprint(w, `{"Id":"dsh-inst-lost"}`)
case "GET /v1.47/networks/dsh-inst-lost":
fmt.Fprint(w, `{"Id":"dsh-inst-lost","Name":"dsh-inst-lost","Driver":"bridge","Labels":{"dsh-mgr.instance":"lost"}}`)
case "POST /v1.47/containers/create":
fmt.Fprint(w, "{")
case "GET /v1.47/containers/dsh-inst-lost/json":
if mode == "late" && !retry {
w.WriteHeader(404)
return
}
if mode == "inspect-failure" && !retry {
w.WriteHeader(500)
return
}
owner := "owner"
if mode == "foreign" {
owner = "another-panel"
}
fmt.Fprintf(w, `{"Id":"created","Name":"/dsh-inst-lost","Image":"image","Config":{"Labels":{"dsh-mgr.instance":"lost","dsh-mgr.owner":%q,"dsh-mgr.role":"dsh"}}}`, owner)
case "DELETE /v1.47/containers/created":
if mode == "retry" && !retry {
w.WriteHeader(500)
return
}
deleted++
w.WriteHeader(204)
case "DELETE /v1.47/networks/dsh-inst-lost":
w.WriteHeader(204)
default:
t.Errorf("unexpected %s %s", r.Method, r.URL.Path)
w.WriteHeader(500)
}
})
seedVerifiedImage(t, a, "image", "1.2.3")
i := store.Instance{ID: "lost", UID: "owner", Version: "1.2.3", ImageID: "image", Status: "creating"}
if err := a.s.AddInstance(i); err != nil {
t.Fatal(err)
}
if _, err := a.createRuntime(context.Background(), store.User{ID: "owner"}, i, "panel.test"); err == nil {
t.Fatal("accepted lost response")
}
if mode == "cleanup" {
if _, err := a.s.GetInstance(i.ID); err == nil || deleted != 1 {
t.Fatal("cleanup incomplete")
}
return
}
saved, err := a.s.GetInstance(i.ID)
if err != nil || saved.Status != "error" || deleted != 0 {
t.Fatalf("lost recovery record: %+v %v", saved, err)
}
retry = true
w := httptest.NewRecorder()
a.instance(w, runtimeRequest("DELETE", "/api/instances/lost", "", sid))
if mode == "foreign" {
if w.Code != 502 || deleted != 0 {
t.Fatal("foreign resource touched")
}
return
}
if w.Code != 200 || deleted != 1 {
t.Fatalf("retry failed %d %s", w.Code, w.Body.String())
}
})
}
}
func TestBuildCheckRecoveryIdentityAndRetry(t *testing.T) {
for _, mode := range []string{"lost", "delete-failure", "foreign", "absent"} {
t.Run(mode, func(t *testing.T) {
deleted := 0
retry := false
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch {
case r.Method == "POST" && r.URL.Path == "/v1.47/containers/create":
if r.URL.Query().Get("name") != buildCheckName("task") {
t.Error("missing task name")
}
if mode == "delete-failure" {
fmt.Fprint(w, `{"Id":"check"}`)
} else {
fmt.Fprint(w, "{")
}
case r.Method == "GET":
if mode == "absent" && !retry {
w.WriteHeader(404)
return
}
if !retry && mode == "lost" {
w.WriteHeader(500)
return
}
id := "task"
if mode == "foreign" {
id = "another-panel"
}
fmt.Fprintf(w, `{"Id":"check","Name":"/dsh-build-check-task","Image":"image","Config":{"Labels":{"dsh-mgr.role":"build-check","dsh-mgr.build":%q}}}`, id)
case r.Method == "DELETE":
if !retry {
w.WriteHeader(500)
return
}
deleted++
w.WriteHeader(204)
default:
w.WriteHeader(500)
}
})
if err := a.verifyBuiltImage(context.Background(), "image", "1.2.3", "task"); err == nil {
t.Fatal("accepted failed validation")
}
var count int
if err := a.s.DB.QueryRow("SELECT count(*) FROM image_build_checks").Scan(&count); err != nil {
t.Fatal(err)
}
if count != 1 {
t.Fatal("lost cleanup record")
}
retry = true
err := a.recoverBuildChecks(context.Background())
if mode == "foreign" {
if err == nil || deleted != 0 {
t.Fatal("foreign container deleted")
}
return
}
if err != nil {
t.Fatal(err)
}
if deleted != 1 {
t.Fatal("not recovered")
}
if err := a.s.DB.QueryRow("SELECT count(*) FROM image_build_checks").Scan(&count); err != nil || count != 0 {
t.Fatalf("record retained %d %v", count, err)
}
})
}
}
func TestReconcileRecoversLostRuntimeIdentity(t *testing.T) {
t.Setenv("PANEL_CONTAINER_ID", "")
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/image/json" {
writeImageInspect(w, "image", "1.2.3")
return
}
if r.Method != "GET" {
t.Fatalf("destructive recovery: %s", r.Method)
}
if r.URL.Path == "/v1.47/containers/json" {
fmt.Fprint(w, `[{"Id":"created","State":"running","Labels":{"dsh-mgr.instance":"lost","dsh-mgr.owner":"owner"}}]`)
return
}
fmt.Fprint(w, `{"Id":"created","Name":"/dsh-inst-lost","Image":"image","Config":{"Labels":{"dsh-mgr.instance":"lost","dsh-mgr.owner":"owner","dsh-mgr.role":"dsh"}}}`)
})
if err := a.s.AddInstance(store.Instance{ID: "lost", UID: "owner", ImageID: "image", Status: "creating"}); err != nil {
t.Fatal(err)
}
if err := a.Reconcile(context.Background()); err != nil {
t.Fatal(err)
}
i, err := a.s.GetInstance("lost")
if err != nil || i.ContainerID != "created" || i.Status != "error" {
t.Fatalf("not recovered: %+v %v", i, err)
}
}
func TestBuildResultTransactionAndRetry(t *testing.T) {
for _, table := range []string{"images", "image_builds"} {
t.Run(table, func(t *testing.T) {
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("unexpected Docker request") })
buildQueue.Lock()
err := a.ensureBuilds()
buildQueue.Unlock()
if err != nil {
t.Fatal(err)
}
b := buildRecord{ID: "task", Version: "1.2.3"}
if _, err = a.s.DB.Exec("INSERT INTO image_builds(id,version,status,created_at) VALUES('task','1.2.3','validating','now')"); err != nil {
t.Fatal(err)
}
event := "INSERT"
if table == "image_builds" {
event = "UPDATE"
}
if _, err = a.s.DB.Exec("CREATE TRIGGER fail_result BEFORE " + event + " ON " + table + " BEGIN SELECT RAISE(FAIL,'injected'); END"); err != nil {
t.Fatal(err)
}
if err = a.persistBuildResult(b, "success", "logs", "", "image", "now"); err == nil {
t.Fatal("injection ignored")
}
var status string
var count int
if err = a.s.DB.QueryRow("SELECT status FROM image_builds WHERE id='task'").Scan(&status); err != nil || status != "validating" {
t.Fatalf("partial terminal write %s %v", status, err)
}
if err = a.s.DB.QueryRow("SELECT count(*) FROM images").Scan(&count); err != nil || count != 0 {
t.Fatalf("partial index write %d %v", count, err)
}
done := make(chan struct{})
go func() { a.finishBuild(b, "success", "logs", "", "image", "now"); close(done) }()
select {
case <-done:
t.Fatal("retry abandoned")
case <-time.After(30 * time.Millisecond):
}
if _, err = a.s.DB.Exec("DROP TRIGGER fail_result"); err != nil {
t.Fatal(err)
}
select {
case <-done:
case <-time.After(4 * time.Second):
t.Fatal("terminal write not retried")
}
if err = a.s.DB.QueryRow("SELECT status FROM image_builds WHERE id='task'").Scan(&status); err != nil || status != "success" {
t.Fatalf("terminal %s %v", status, err)
}
if err = a.s.DB.QueryRow("SELECT count(*) FROM images").Scan(&count); err != nil || count != 1 {
t.Fatalf("index %d %v", count, err)
}
})
}
}
func TestIntentRecoveryMissingReservationDoesNotTouchDocker(t *testing.T) {
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/image/json" {
writeImageInspect(w, "image", "1.2.3")
return
}
t.Error("missing reservation must not authorize Docker cleanup")
})
if err := a.s.EnsureIntents(); err != nil {
t.Fatal(err)
}
if err := a.s.ReserveIntent(store.Intent{ID: "missing", UID: "owner"}); err != nil {
t.Fatal(err)
}
if _, err := a.s.DB.Exec("UPDATE creation_intents SET status='creating',instance_id='gone' WHERE id='missing'"); err != nil {
t.Fatal(err)
}
if err := a.recoverIntents(context.Background()); err != nil {
t.Fatal(err)
}
list, err := a.s.Intents("owner")
if err != nil || len(list) != 1 || list[0].Status != "failed" {
t.Fatalf("intent not terminated: %+v %v", list, err)
}
}
func TestBuildTerminalRetryStopsWhenDatabaseCloses(t *testing.T) {
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("unexpected Docker call") })
a.s.Close()
done := make(chan struct{})
go func() { a.finishBuild(buildRecord{ID: "closed"}, "failed", "", "", "", "now"); close(done) }()
select {
case <-done:
case <-time.After(time.Second):
t.Fatal("closed database retry did not stop")
}
}
func TestBuildCheckDefiniteRejectionAndExternalDeletion(t *testing.T) {
for _, rejected := range []bool{true, false} {
t.Run(fmt.Sprint(rejected), func(t *testing.T) {
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.Method == "POST" && r.URL.Path == "/v1.47/containers/create" {
w.WriteHeader(400)
return
}
w.WriteHeader(404)
})
if rejected {
if err := a.verifyBuiltImage(context.Background(), "image", "1.2.3", "task"); err == nil {
t.Fatal("rejection accepted")
}
} else {
if err := a.ensureBuildChecks(); err != nil {
t.Fatal(err)
}
if _, err := a.s.DB.Exec("INSERT INTO image_build_checks(build_id,image_id,container_id) VALUES('task','image','known')"); err != nil {
t.Fatal(err)
}
if err := a.recoverBuildChecks(context.Background()); err != nil {
t.Fatal(err)
}
}
var n int
if err := a.s.DB.QueryRow("SELECT count(*) FROM image_build_checks").Scan(&n); err != nil || n != 0 {
t.Fatalf("definite outcome retained: %d %v", n, err)
}
})
}
}
func TestUncertainPendingRetainsLateResources(t *testing.T) {
for _, column := range []string{"container_pending", "network_pending"} {
for _, stamp := range []string{time.Now().Add(-time.Hour).UTC().Format(time.RFC3339Nano), "", "invalid"} {
t.Run(column+"/"+stamp, func(t *testing.T) {
t.Setenv("PANEL_CONTAINER_ID", "")
late := false
deletes := 0
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/image/json" {
writeImageInspect(w, "image", "1.2.3")
return
}
switch r.Method + " " + r.URL.Path {
case "GET /v1.47/containers/dsh-inst-ghost/json":
if !late || column != "container_pending" {
w.WriteHeader(404)
return
}
fmt.Fprint(w, `{"Id":"container-id","Name":"/dsh-inst-ghost","Image":"image","Config":{"Labels":{"dsh-mgr.instance":"ghost","dsh-mgr.owner":"owner","dsh-mgr.role":"dsh"}}}`)
case "GET /v1.47/networks/dsh-inst-ghost":
if !late || column != "network_pending" {
w.WriteHeader(404)
return
}
fmt.Fprint(w, `{"Id":"network-id","Name":"dsh-inst-ghost","Driver":"bridge","Labels":{"dsh-mgr.instance":"ghost"}}`)
case "DELETE /v1.47/containers/container-id", "DELETE /v1.47/networks/network-id":
deletes++
w.WriteHeader(204)
default:
t.Errorf("unexpected %s %s", r.Method, r.URL.Path)
w.WriteHeader(500)
}
})
i := store.Instance{ID: "ghost", UID: "owner", Version: "1.2.3", ImageID: "image", Status: "error"}
if err := a.s.AddInstance(i); err != nil {
t.Fatal(err)
}
if err := a.setRuntimeCreatePending(i.ID, column, true); err != nil {
t.Fatal(err)
}
if _, err := a.s.DB.Exec("UPDATE runtime_creates SET "+pendingColumns[column].stamp+"=? WHERE instance_id=?", stamp, i.ID); err != nil {
t.Fatal(err)
}
for attempt := 0; attempt < 2; attempt++ {
w := httptest.NewRecorder()
a.instance(w, runtimeRequest("DELETE", "/api/instances/ghost", "", sid))
if w.Code != 502 {
t.Fatalf("uncertainty discarded: %d %s", w.Code, w.Body.String())
}
if _, pending, err := a.pendingSince(i.ID, column); err != nil || !pending {
t.Fatalf("pending lost: %v %v", pending, err)
}
if _, err := a.s.GetInstance(i.ID); err != nil {
t.Fatal(err)
}
}
late = true
w := httptest.NewRecorder()
a.instance(w, runtimeRequest("DELETE", "/api/instances/ghost", "", sid))
if w.Code != 200 || deletes != 1 {
t.Fatalf("late cleanup: %d %s deletes=%d", w.Code, w.Body.String(), deletes)
}
if _, err := a.s.GetInstance(i.ID); err == nil {
t.Fatal("instance retained")
}
})
}
}
}
func TestBuildCheckUncertainRowRetainsLateContainer(t *testing.T) {
for _, stamp := range []string{time.Now().Add(-time.Hour).UTC().Format(time.RFC3339Nano), "", "invalid"} {
t.Run(stamp, func(t *testing.T) {
late := false
deletes := 0
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
switch r.Method + " " + r.URL.Path {
case "GET /v1.47/containers/dsh-build-check-task/json":
if !late {
w.WriteHeader(404)
return
}
fmt.Fprint(w, `{"Id":"check","Name":"/dsh-build-check-task","Image":"image","Config":{"Labels":{"dsh-mgr.role":"build-check","dsh-mgr.build":"task"}}}`)
case "DELETE /v1.47/containers/check":
deletes++
w.WriteHeader(204)
default:
t.Errorf("unexpected %s %s", r.Method, r.URL.Path)
w.WriteHeader(500)
}
})
if err := a.ensureBuildChecks(); err != nil {
t.Fatal(err)
}
if _, err := a.s.DB.Exec("INSERT INTO image_build_checks(build_id,image_id,created_at) VALUES('task','image',?)", stamp); err != nil {
t.Fatal(err)
}
for attempt := 0; attempt < 2; attempt++ {
if err := a.recoverBuildChecks(context.Background()); !errors.Is(err, errBuildCheckUncertain) {
t.Fatalf("uncertainty discarded: %v", err)
}
var n int
if err := a.s.DB.QueryRow("SELECT count(*) FROM image_build_checks").Scan(&n); err != nil || n != 1 {
t.Fatalf("record lost: %d %v", n, err)
}
}
late = true
if err := a.recoverBuildChecks(context.Background()); err != nil {
t.Fatal(err)
}
var n int
if err := a.s.DB.QueryRow("SELECT count(*) FROM image_build_checks").Scan(&n); err != nil || n != 0 || deletes != 1 {
t.Fatalf("late cleanup: rows=%d deletes=%d err=%v", n, deletes, err)
}
})
}
}
func TestRuntimeNetworkPanelDisconnect(t *testing.T) {
for _, mode := range []string{"not-connected", "connected", "panel-name", "other-connected", "lost-disconnect", "forbidden", "inspect-forbidden", "foreign"} {
t.Run(mode, func(t *testing.T) {
panel := "panel-id"
if mode == "panel-name" || mode == "inspect-forbidden" {
panel = "panel"
}
t.Setenv("PANEL_CONTAINER_ID", panel)
connected := mode != "not-connected"
disconnects, deletes := 0, 0
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v1.47/images/image/json" {
writeImageInspect(w, "image", "1.2.3")
return
}
switch r.Method + " " + r.URL.Path {
case "GET /v1.47/networks/dsh-inst-net", "GET /v1.47/networks/network-id":
owner := "net"
if mode == "foreign" {
owner = "other"
}
containers := "{}"
if connected {
containers = `{"panel-id":{}}`
}
if mode == "other-connected" {
containers = `{"other-id":{}}`
}
fmt.Fprintf(w, `{"Id":"network-id","Name":"dsh-inst-net","Driver":"bridge","Labels":{"dsh-mgr.instance":%q},"Containers":%s}`, owner, containers)
case "GET /v1.47/containers/panel/json", "GET /v1.47/containers/panel-id/json":
if mode == "inspect-forbidden" {
w.WriteHeader(403)
return
}
fmt.Fprint(w, `{"Id":"panel-id"}`)
case "POST /v1.47/networks/network-id/disconnect":
disconnects++
var body struct {
Container string
Force bool
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil || body.Container != "panel-id" || !body.Force {
t.Errorf("disconnect identity: %+v %v", body, err)
}
if mode == "not-connected" || mode == "other-connected" || mode == "foreign" {
t.Error("unsafe disconnect")
w.WriteHeader(500)
return
}
if mode == "forbidden" {
w.WriteHeader(403)
return
}
connected = false
if mode == "lost-disconnect" {
w.WriteHeader(500)
return
}
w.WriteHeader(204)
case "DELETE /v1.47/networks/network-id":
deletes++
w.WriteHeader(204)
default:
t.Errorf("unexpected %s %s", r.Method, r.URL.Path)
w.WriteHeader(500)
}
})
if err := a.setRuntimeCreatePending("net", "network_pending", true); err != nil {
t.Fatal(err)
}
err := a.recoverRuntimeNetwork(context.Background(), "net")
wantError := mode == "lost-disconnect" || mode == "forbidden" || mode == "inspect-forbidden" || mode == "foreign"
if (err != nil) != wantError {
t.Fatalf("recovery error=%v wantError=%v", err, wantError)
}
if wantError {
if deletes != 0 {
t.Fatal("deleted after failure")
}
if _, pending, err := a.pendingSince("net", "network_pending"); err != nil || !pending {
t.Fatalf("pending lost: %v %v", pending, err)
}
if mode != "lost-disconnect" {
return
}
if err := a.recoverRuntimeNetwork(context.Background(), "net"); err != nil {
t.Fatalf("disconnect retry: %v", err)
}
}
wantDisconnects := 1
if mode == "not-connected" || mode == "other-connected" {
wantDisconnects = 0
}
if deletes != 1 || disconnects != wantDisconnects {
t.Fatalf("deletes=%d disconnects=%d", deletes, disconnects)
}
if _, pending, err := a.pendingSince("net", "network_pending"); err != nil || pending {
t.Fatalf("pending retained: %v %v", pending, err)
}
})
}
}
+109 -14
View File
@@ -81,6 +81,7 @@ func (a *API) instances(w http.ResponseWriter, r *http.Request) {
ImageID string `json:"image_id"` ImageID string `json:"image_id"`
} }
decoder := json.NewDecoder(http.MaxBytesReader(w, r.Body, 64<<10)) decoder := json.NewDecoder(http.MaxBytesReader(w, r.Body, 64<<10))
decoder.DisallowUnknownFields()
if decoder.Decode(&in) != nil || decoder.Decode(new(any)) != io.EOF || strings.TrimSpace(in.Name) == "" || len(in.Name) > 200 || !runtimeImageID.MatchString(in.ImageID) { if decoder.Decode(&in) != nil || decoder.Decode(new(any)) != io.EOF || strings.TrimSpace(in.Name) == "" || len(in.Name) > 200 || !runtimeImageID.MatchString(in.ImageID) {
fail(w, 400, "invalid instance") fail(w, 400, "invalid instance")
return return
@@ -139,13 +140,25 @@ func (a *API) createRuntime(ctx context.Context, u store.User, i store.Instance,
if _, err := a.s.GetInstance(i.ID); err != nil { if _, err := a.s.GetInstance(i.ID); err != nil {
return i, errors.New("instance reservation unavailable") return i, errors.New("instance reservation unavailable")
} }
networkCreated, complete := false, false networkCreated, networkAttempted, containerAttempted, complete := false, false, false, false
defer func() { defer func() {
if complete { if complete {
return return
} }
cleanup, done := context.WithTimeout(context.Background(), 20*time.Second) cleanup, done := context.WithTimeout(context.Background(), 20*time.Second)
defer done() defer done()
if networkAttempted && !networkCreated {
if err := a.recoverRuntimeNetwork(cleanup, i.ID); err != nil {
a.s.SetInstance(i.ID, "error")
return
}
}
if containerAttempted && i.ContainerID == "" {
if err := a.recoverRuntimeContainer(cleanup, &i); err != nil {
a.s.SetInstance(i.ID, "error")
return
}
}
if i.ContainerID != "" { if i.ContainerID != "" {
if err := a.removeRuntimeContainer(cleanup, i.ContainerID); err != nil { if err := a.removeRuntimeContainer(cleanup, i.ContainerID); err != nil {
a.s.SetInstance(i.ID, "error") a.s.SetInstance(i.ID, "error")
@@ -159,7 +172,24 @@ func (a *API) createRuntime(ctx context.Context, u store.User, i store.Instance,
} }
} }
a.s.DeleteInstance(i.ID) a.s.DeleteInstance(i.ID)
_, _ = a.s.DB.Exec("DELETE FROM runtime_networks WHERE instance_id=?", i.ID)
_, _ = a.s.DB.Exec("DELETE FROM runtime_creates WHERE instance_id=?", i.ID)
}() }()
admissionMu.RLock()
defer admissionMu.RUnlock()
buildQueue.Lock()
policyErr := a.ensureBuilds()
buildQueue.Unlock()
if policyErr != nil {
return i, policyErr
}
policy, policyErr := a.evaluateImage(ctx, i.ImageID)
if policyErr != nil {
return i, policyErr
}
if !policy.Selectable || policy.Version != i.Version {
return i, errors.New("image is not admitted for creation")
}
root, err := filepath.Abs(a.root) root, err := filepath.Abs(a.root)
if err != nil { if err != nil {
return i, errors.New("storage unavailable") return i, errors.New("storage unavailable")
@@ -186,10 +216,30 @@ func (a *API) createRuntime(ctx context.Context, u store.User, i store.Instance,
return i, err return i, err
} }
network := runtimeNetwork(i.ID) network := runtimeNetwork(i.ID)
if err := a.docker.Do(ctx, "POST", "/networks/create", map[string]any{"Name": network, "Driver": "bridge", "CheckDuplicate": true, "Labels": map[string]string{"dsh-mgr.instance": i.ID}}, nil); err != nil { if err := a.setRuntimeCreatePending(i.ID, "network_pending", true); err != nil {
return i, err
}
networkAttempted = true
var createdNetwork struct {
ID string `json:"Id"`
}
if err := a.docker.Do(ctx, "POST", "/networks/create", map[string]any{"Name": network, "Driver": "bridge", "CheckDuplicate": true, "Labels": map[string]string{"dsh-mgr.instance": i.ID}}, &createdNetwork); err != nil {
if dockerCreateRejected(err) {
if clearErr := a.setRuntimeCreatePending(i.ID, "network_pending", false); clearErr == nil {
networkAttempted = false
}
}
return i, errors.New("network creation failed") return i, errors.New("network creation failed")
} }
networkCreated = true networkCreated = true
if createdNetwork.ID != "" {
if _, err := a.s.DB.ExecContext(ctx, "INSERT INTO runtime_networks(instance_id,network_id) VALUES(?,?)", i.ID, createdNetwork.ID); err != nil {
return i, err
}
}
if err := a.setRuntimeCreatePending(i.ID, "network_pending", false); err != nil {
return i, err
}
if panel := os.Getenv("PANEL_CONTAINER_ID"); panel != "" { if panel := os.Getenv("PANEL_CONTAINER_ID"); panel != "" {
if err := a.docker.Do(ctx, "POST", "/networks/"+network+"/connect", map[string]string{"Container": panel}, nil); err != nil { if err := a.docker.Do(ctx, "POST", "/networks/"+network+"/connect", map[string]string{"Container": panel}, nil); err != nil {
return i, errors.New("panel network connection failed") return i, errors.New("panel network connection failed")
@@ -212,13 +262,33 @@ func (a *API) createRuntime(ctx context.Context, u store.User, i store.Instance,
}, },
}, },
} }
if policy.RecipeRev != "7" {
sharedSource, err := toHostPath(root, hostRoot, filepath.Join(root, suffix, "files"))
if err != nil {
return i, err
}
body["Env"] = []string{"DSH_HOME=/data/dsh-home", "HOME=/data/dsh-home", "XDG_DATA_HOME=/data/dsh-home/.local/share", "XDG_CONFIG_HOME=/data/dsh-home/.config"}
body["HostConfig"].(map[string]any)["Mounts"] = []map[string]any{{"Type": "bind", "Source": homeSource, "Target": "/data/dsh-home"}, {"Type": "bind", "Source": sharedSource, "Target": "/workspace"}}
}
var created struct { var created struct {
ID string `json:"Id"` ID string `json:"Id"`
} }
if err := a.setRuntimeCreatePending(i.ID, "container_pending", true); err != nil {
return i, err
}
containerAttempted = true
if err := a.docker.Do(ctx, "POST", "/containers/create?name="+network, body, &created); err != nil || created.ID == "" { if err := a.docker.Do(ctx, "POST", "/containers/create?name="+network, body, &created); err != nil || created.ID == "" {
if dockerCreateRejected(err) {
if clearErr := a.setRuntimeCreatePending(i.ID, "container_pending", false); clearErr == nil {
containerAttempted = false
}
}
return i, errors.New("container creation failed") return i, errors.New("container creation failed")
} }
i.ContainerID = created.ID i.ContainerID = created.ID
if err := a.setRuntimeCreatePending(i.ID, "container_pending", false); err != nil {
return i, err
}
if _, err := a.s.DB.ExecContext(ctx, "UPDATE instances SET container_id=? WHERE id=?", i.ContainerID, i.ID); err != nil { if _, err := a.s.DB.ExecContext(ctx, "UPDATE instances SET container_id=? WHERE id=?", i.ContainerID, i.ID); err != nil {
return i, errors.New("database error") return i, errors.New("database error")
} }
@@ -230,6 +300,7 @@ func (a *API) createRuntime(ctx context.Context, u store.User, i store.Instance,
return i, errors.New("database error") return i, errors.New("database error")
} }
complete = true complete = true
_, _ = a.s.DB.Exec("DELETE FROM runtime_creates WHERE instance_id=? AND container_pending=0 AND network_pending=0", i.ID)
a.s.AddAudit(u.ID, "instance.create", i.ID, i.ImageID) a.s.AddAudit(u.ID, "instance.create", i.ID, i.ImageID)
return i, nil return i, nil
} }
@@ -238,6 +309,10 @@ func dockerNotFound(err error) bool {
var e *docker.APIError var e *docker.APIError
return errors.As(err, &e) && e.StatusCode == 404 return errors.As(err, &e) && e.StatusCode == 404
} }
func dockerCreateRejected(err error) bool {
var e *docker.APIError
return errors.As(err, &e) && e.StatusCode >= 400 && e.StatusCode < 500 && e.StatusCode != 408 && e.StatusCode != 409
}
func (a *API) removeRuntimeContainer(ctx context.Context, id string) error { func (a *API) removeRuntimeContainer(ctx context.Context, id string) error {
err := a.docker.Do(ctx, "DELETE", "/containers/"+url.PathEscape(id)+"?force=1", nil, nil) err := a.docker.Do(ctx, "DELETE", "/containers/"+url.PathEscape(id)+"?force=1", nil, nil)
if dockerNotFound(err) { if dockerNotFound(err) {
@@ -246,18 +321,7 @@ func (a *API) removeRuntimeContainer(ctx context.Context, id string) error {
return err return err
} }
func (a *API) removeRuntimeNetwork(ctx context.Context, id string) error { func (a *API) removeRuntimeNetwork(ctx context.Context, id string) error {
path := "/networks/" + runtimeNetwork(id) return a.recoverRuntimeNetwork(ctx, id)
if panel := os.Getenv("PANEL_CONTAINER_ID"); panel != "" {
err := a.docker.Do(ctx, "POST", path+"/disconnect", map[string]any{"Container": panel, "Force": true}, nil)
if err != nil && !dockerNotFound(err) {
return err
}
}
err := a.docker.Do(ctx, "DELETE", path, nil, nil)
if dockerNotFound(err) {
return nil
}
return err
} }
func (a *API) instance(w http.ResponseWriter, r *http.Request) { func (a *API) instance(w http.ResponseWriter, r *http.Request) {
@@ -339,6 +403,12 @@ func (a *API) instance(w http.ResponseWriter, r *http.Request) {
fail(w, 409, "instance files busy; retry later") fail(w, 409, "instance files busy; retry later")
return return
} }
if i.ContainerID == "" {
if err := a.recoverRuntimeContainer(ctx, &i); err != nil {
fail(w, 502, "container recovery failed")
return
}
}
if i.ContainerID != "" { if i.ContainerID != "" {
if err := a.removeRuntimeContainer(ctx, i.ContainerID); err != nil { if err := a.removeRuntimeContainer(ctx, i.ContainerID); err != nil {
fail(w, 502, "container deletion failed") fail(w, 502, "container deletion failed")
@@ -353,7 +423,11 @@ func (a *API) instance(w http.ResponseWriter, r *http.Request) {
fail(w, 500, "database error") fail(w, 500, "database error")
return return
} }
_, _ = a.s.DB.Exec("DELETE FROM runtime_networks WHERE instance_id=?", i.ID)
a.s.AddAudit(u.ID, "instance.delete", i.ID, "data retained") a.s.AddAudit(u.ID, "instance.delete", i.ID, "data retained")
// The instance row is gone and its resources were just cleaned, so
// any pending flags cannot resolve later: drop the row unconditionally.
_, _ = a.s.DB.Exec("DELETE FROM runtime_creates WHERE instance_id=?", i.ID)
runtimeAuthCache.Delete(runtimeAuthKey{a, i.ID}) runtimeAuthCache.Delete(runtimeAuthKey{a, i.ID})
writeJSON(w, map[string]bool{"ok": true}) writeJSON(w, map[string]bool{"ok": true})
return return
@@ -384,6 +458,27 @@ func (a *API) instance(w http.ResponseWriter, r *http.Request) {
fail(w, 409, "instance files busy; retry later") fail(w, 409, "instance files busy; retry later")
return return
} }
admissionMu.RLock()
defer admissionMu.RUnlock()
if action != "stop" {
buildQueue.Lock()
e := a.ensureBuilds()
buildQueue.Unlock()
if e != nil {
fail(w, 500, "database error")
return
}
var blocked int
e = a.s.DB.QueryRowContext(ctx, "SELECT count(*) FROM image_admissions WHERE image_id=? AND state='blocked'", i.ImageID).Scan(&blocked)
if e != nil {
fail(w, 500, "database error")
return
}
if blocked > 0 {
fail(w, 409, "image is blocked; start/restart denied")
return
}
}
if err := a.docker.Do(ctx, "POST", "/containers/"+url.PathEscape(i.ContainerID)+"/"+action+"?t=10", nil, nil); err != nil { if err := a.docker.Do(ctx, "POST", "/containers/"+url.PathEscape(i.ContainerID)+"/"+action+"?t=10", nil, nil); err != nil {
fail(w, 502, "container action failed") fail(w, 502, "container action failed")
return return
+12 -4
View File
@@ -34,15 +34,23 @@ const runtimeSessionJS = `(function () {
let pending; let pending;
async function renew() { async function renew() {
if (pending) return pending; if (pending) return pending;
pending = (async function () { const run = async function () {
// Renew proactively: a successful /me does not extend the five-minute
// access lifetime. Web Locks serialize cookie rotation across pages.
const csrf = document.cookie.split("; ").find(c => c.startsWith("dsh_csrf=")); const csrf = document.cookie.split("; ").find(c => c.startsWith("dsh_csrf="));
if (!csrf) return; if (!csrf) return;
const response = await fetch("/api/refresh", {method:"POST", credentials:"same-origin", headers:{"X-CSRF-Token":csrf.slice(9)}}); const response = await fetch("/api/refresh", {method:"POST", credentials:"same-origin", headers:{"X-CSRF-Token":csrf.slice(9)}});
if (response.status === 401 || response.status === 403) { if (response.status === 401 || response.status === 403 || response.status === 409) {
for (const delay of [0, 100, 250, 500]) {
if (delay) await new Promise(resolve => setTimeout(resolve, delay));
const check = await fetch("/api/me", {credentials:"same-origin"}); const check = await fetch("/api/me", {credentials:"same-origin"});
if (check.status === 401) location.reload(); if (check.status !== 401) return;
} }
})().catch(function () {}).finally(function () { pending = undefined; }); if (response.status === 401 || response.status === 403) location.reload();
}
};
pending = (navigator.locks ? navigator.locks.request("dsh-session-refresh", run) : run())
.catch(function () {}).finally(function () { pending = undefined; });
return pending; return pending;
} }
setInterval(renew, 240000); setInterval(renew, 240000);
+18 -3
View File
@@ -71,6 +71,14 @@ func TestRuntimeProxyRejectsErrorState(t *testing.T) {
// must wait instead of removing the row while the container is still starting. // must wait instead of removing the row while the container is still starting.
func TestInstanceDeleteWaitsForInFlightCreate(t *testing.T) { func TestInstanceDeleteWaitsForInFlightCreate(t *testing.T) {
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
if r.Method == "GET" && strings.Contains(r.URL.Path, "/networks/") {
w.WriteHeader(404)
return
}
if r.Method == http.MethodGet && r.URL.Path == "/v1.47/containers/dsh-inst-creating/json" {
w.WriteHeader(http.StatusNotFound)
return
}
if r.Method == http.MethodDelete && strings.Contains(r.URL.Path, "/networks/") { if r.Method == http.MethodDelete && strings.Contains(r.URL.Path, "/networks/") {
w.WriteHeader(http.StatusNoContent) w.WriteHeader(http.StatusNoContent)
return return
@@ -122,7 +130,7 @@ func TestRuntimeCreateLimitAndPolicy(t *testing.T) {
calls = append(calls, r.Method+" "+r.URL.Path) calls = append(calls, r.Method+" "+r.URL.Path)
switch { switch {
case strings.Contains(r.URL.Path, "/images/"): case strings.Contains(r.URL.Path, "/images/"):
writeJSON(w, map[string]any{"Id": imageID, "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "1.2.3"}}}) writeJSON(w, map[string]any{"Id": imageID, "Os": "linux", "Architecture": "amd64", "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "1.2.3", "dsh-mgr.recipe": imageRecipeRev}}})
case strings.HasSuffix(r.URL.Path, "/containers/create"): case strings.HasSuffix(r.URL.Path, "/containers/create"):
var body map[string]any var body map[string]any
if err := json.NewDecoder(r.Body).Decode(&body); err != nil { if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
@@ -134,7 +142,7 @@ func TestRuntimeCreateLimitAndPolicy(t *testing.T) {
w.WriteHeader(204) w.WriteHeader(204)
} }
}) })
body := fmt.Sprintf("{\"name\":\"one\",\"image_id\":%q,\"version\":\"untrusted\"}", imageID) body := fmt.Sprintf("{\"name\":\"one\",\"image_id\":%q}", imageID)
seedVerifiedImage(t, a, imageID, "1.2.3") seedVerifiedImage(t, a, imageID, "1.2.3")
var wg sync.WaitGroup var wg sync.WaitGroup
statuses := make(chan int, 4) statuses := make(chan int, 4)
@@ -213,6 +221,10 @@ func TestRuntimeLifecycleAndOwnership(t *testing.T) {
var calls []string var calls []string
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
calls = append(calls, r.Method+" "+r.URL.Path) calls = append(calls, r.Method+" "+r.URL.Path)
if r.Method == "GET" && strings.Contains(r.URL.Path, "/networks/") {
w.WriteHeader(404)
return
}
if strings.HasSuffix(r.URL.Path, "/json") { if strings.HasSuffix(r.URL.Path, "/json") {
io.WriteString(w, "{\"State\":{\"Status\":\"running\"}}") io.WriteString(w, "{\"State\":{\"Status\":\"running\"}}")
} else if strings.HasSuffix(r.URL.Path, "/logs") { } else if strings.HasSuffix(r.URL.Path, "/logs") {
@@ -447,8 +459,11 @@ func TestRuntimeCreateRollbackAndImageRejection(t *testing.T) {
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
calls = append(calls, r.Method+" "+r.URL.Path) calls = append(calls, r.Method+" "+r.URL.Path)
switch { switch {
case r.Method == "GET" && strings.Contains(r.URL.Path, "/networks/"):
name := strings.TrimPrefix(r.URL.Path, "/v1.47/networks/")
writeJSON(w, map[string]any{"Id": name, "Name": name, "Driver": "bridge", "Labels": map[string]string{"dsh-mgr.instance": strings.TrimPrefix(name, "dsh-inst-")}})
case strings.Contains(r.URL.Path, "/images/"): case strings.Contains(r.URL.Path, "/images/"):
writeJSON(w, map[string]any{"Id": imageID, "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": role, "dsh-mgr.version": "1"}}}) writeJSON(w, map[string]any{"Id": imageID, "Os": "linux", "Architecture": "amd64", "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": role, "dsh-mgr.version": "1", "dsh-mgr.recipe": imageRecipeRev}}})
case strings.HasSuffix(r.URL.Path, "/containers/create"): case strings.HasSuffix(r.URL.Path, "/containers/create"):
writeJSON(w, map[string]string{"Id": "container"}) writeJSON(w, map[string]string{"Id": "container"})
case strings.HasSuffix(r.URL.Path, "/start"): case strings.HasSuffix(r.URL.Path, "/start"):
+27 -6
View File
@@ -16,7 +16,6 @@ import (
"git.misaka.ren/m1saka/dsh-web/internal/backup" "git.misaka.ren/m1saka/dsh-web/internal/backup"
"git.misaka.ren/m1saka/dsh-web/internal/store" "git.misaka.ren/m1saka/dsh-web/internal/store"
"golang.org/x/crypto/bcrypt"
) )
const upgradeSnapshotLimit int64 = 1 << 30 const upgradeSnapshotLimit int64 = 1 << 30
@@ -52,10 +51,31 @@ func upgradeWorkspace(config, host map[string]json.RawMessage, recipe, home, sha
if homeIndex == -1 { if homeIndex == -1 {
return errors.New("instance home mount required") return errors.New("instance home mount required")
} }
if !supportedLayout(recipe) {
return errors.New("unsupported template layout")
}
if recipe != "7" { if recipe != "7" {
if get(mounts[homeIndex], "Target") == "/workspace" { if get(mounts[homeIndex], "Target") == "/workspace" {
return errors.New("workspace home requires template 7") return errors.New("workspace home cannot be downgraded to legacy shared layout")
} }
if sharedIndex == -1 || get(mounts[sharedIndex], "Target") != "/workspace" {
return errors.New("legacy shared workspace mount required")
}
var env []string
if raw := config["Env"]; len(raw) != 0 {
if err := json.Unmarshal(raw, &env); err != nil {
return err
}
}
kept := []string{}
for _, entry := range env {
key, _, _ := strings.Cut(entry, "=")
if key != "DSH_HOME" && key != "HOME" && key != "XDG_DATA_HOME" && key != "XDG_CONFIG_HOME" {
kept = append(kept, entry)
}
}
kept = append(kept, "DSH_HOME=/data/dsh-home", "HOME=/data/dsh-home", "XDG_DATA_HOME=/data/dsh-home/.local/share", "XDG_CONFIG_HOME=/data/dsh-home/.config")
config["Env"], _ = json.Marshal(kept)
return nil return nil
} }
for _, key := range []string{"Binds", "VolumesFrom"} { for _, key := range []string{"Binds", "VolumesFrom"} {
@@ -271,6 +291,7 @@ func (a *API) upgrade(w http.ResponseWriter, r *http.Request, i store.Instance)
Password string `json:"password"` Password string `json:"password"`
} }
dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 64<<10)) dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 64<<10))
dec.DisallowUnknownFields()
if dec.Decode(&in) != nil || !runtimeImageID.MatchString(in.ImageID) || dec.Decode(new(any)) != io.EOF { if dec.Decode(&in) != nil || !runtimeImageID.MatchString(in.ImageID) || dec.Decode(new(any)) != io.EOF {
fail(w, 400, "invalid image_id") fail(w, 400, "invalid image_id")
return return
@@ -286,6 +307,8 @@ func (a *API) upgrade(w http.ResponseWriter, r *http.Request, i store.Instance)
} }
user := a.userLocks.Acquire(i.UID) user := a.userLocks.Acquire(i.UID)
defer user.Release() defer user.Release()
admissionMu.RLock()
defer admissionMu.RUnlock()
current, err := a.s.GetInstance(i.ID) current, err := a.s.GetInstance(i.ID)
if err != nil || current.UID != i.UID { if err != nil || current.UID != i.UID {
fail(w, 404, "not found") fail(w, 404, "not found")
@@ -296,9 +319,7 @@ func (a *API) upgrade(w http.ResponseWriter, r *http.Request, i store.Instance)
fail(w, 400, "login password required for pre-upgrade backup") fail(w, 400, "login password required for pre-upgrade backup")
return return
} }
var passwordHash string if !a.reauthenticate(w, i.UID, in.Password) {
if a.s.DB.QueryRow("SELECT pass_hash FROM users WHERE id=?", i.UID).Scan(&passwordHash) != nil || bcrypt.CompareHashAndPassword([]byte(passwordHash), []byte(in.Password)) != nil {
fail(w, 403, "incorrect login password")
return return
} }
if a.upgradeRecoveryRequired(i) || i.Status == "error" { if a.upgradeRecoveryRequired(i) || i.Status == "error" {
@@ -337,7 +358,7 @@ func (a *API) upgrade(w http.ResponseWriter, r *http.Request, i store.Instance)
ID string `json:"Id"` ID string `json:"Id"`
Config struct{ Labels map[string]string } Config struct{ Labels map[string]string }
} }
if a.docker.Do(ctx, "GET", "/images/"+url.PathEscape(in.ImageID)+"/json", nil, &image) != nil || image.ID != in.ImageID || image.Config.Labels["dsh-mgr.role"] != "runtime" || image.Config.Labels["dsh-mgr.recipe"] != imageRecipeRev || verified[image.ID] == "" || verified[image.ID] != image.Config.Labels["dsh-mgr.version"] { if a.docker.Do(ctx, "GET", "/images/"+url.PathEscape(in.ImageID)+"/json", nil, &image) != nil || image.ID != in.ImageID || image.Config.Labels["dsh-mgr.role"] != "runtime" || !supportedLayout(image.Config.Labels["dsh-mgr.recipe"]) || verified[image.ID] == "" || verified[image.ID] != image.Config.Labels["dsh-mgr.version"] {
fail(w, 400, "local runtime image required") fail(w, 400, "local runtime image required")
return return
} }
+12 -3
View File
@@ -27,7 +27,7 @@ func TestUpgradeWorkspaceCompatibility(t *testing.T) {
{"single-to-seven", "7", "/workspace", false, false}, {"single-to-seven", "7", "/workspace", false, false},
{"migrated-to-seven", "7", "/workspace", true, false}, {"migrated-to-seven", "7", "/workspace", true, false},
{"legacy-to-six", "6", "/data/dsh-home", true, false}, {"legacy-to-six", "6", "/data/dsh-home", true, false},
{"legacy-to-unknown", "", "/data/dsh-home", true, false}, {"legacy-to-unknown", "", "/data/dsh-home", true, true},
{"single-to-six", "6", "/workspace", false, true}, {"single-to-six", "6", "/workspace", false, true},
{"single-to-unknown", "", "/workspace", false, true}, {"single-to-unknown", "", "/workspace", false, true},
} { } {
@@ -48,7 +48,13 @@ func TestUpgradeWorkspaceCompatibility(t *testing.T) {
if (err != nil) != tc.reject { if (err != nil) != tc.reject {
t.Fatalf("error=%v", err) t.Fatalf("error=%v", err)
} }
if tc.reject || tc.recipe != "7" { if !tc.reject && tc.recipe != "7" {
if !strings.Contains(string(config["Env"]), "XDG_DATA_HOME=/data/dsh-home/.local/share") || string(host["Mounts"]) != string(raw) {
t.Fatal(config, host)
}
return
}
if tc.reject {
if string(host["Mounts"]) != string(raw) || string(config["Env"]) != before { if string(host["Mounts"]) != string(raw) || string(config["Env"]) != before {
t.Fatal("legacy or rejected config changed") t.Fatal("legacy or rejected config changed")
} }
@@ -149,7 +155,7 @@ func TestUpgradeLifecycle(t *testing.T) {
if mode == "invalid-image" { if mode == "invalid-image" {
role = "builder" role = "builder"
} }
writeJSON(w, map[string]any{"Id": image, "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": role, "dsh-mgr.version": "v2", "dsh-mgr.recipe": imageRecipeRev}}}) writeJSON(w, map[string]any{"Id": image, "Os": "linux", "Architecture": "amd64", "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": role, "dsh-mgr.version": "v2", "dsh-mgr.recipe": imageRecipeRev}}})
case p == "/containers/create": case p == "/containers/create":
if !strings.HasPrefix(r.URL.Query().Get("name"), "dsh-inst-one-upgrade-") { if !strings.HasPrefix(r.URL.Query().Get("name"), "dsh-inst-one-upgrade-") {
t.Error("missing unique name") t.Error("missing unique name")
@@ -238,6 +244,9 @@ func TestUpgradeLifecycle(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
} }
if _, err := a.s.DB.Exec("INSERT INTO image_admissions(image_id,version,recipe_rev,platform,state,evidence,admitted_by,admitted_at) VALUES(?,'v2',?,?,'compatible','test acceptance','test','now')", image, imageRecipeRev, imagePlatform); err != nil {
t.Fatal(err)
}
base := filepath.Join(a.root, "workspaces", "owner", "instances", "one") base := filepath.Join(a.root, "workspaces", "owner", "instances", "one")
home = filepath.Join(base, "home") home = filepath.Join(base, "home")
shared = filepath.Join(a.root, "workspaces", "owner", "files") shared = filepath.Join(a.root, "workspaces", "owner", "files")
+9 -3
View File
@@ -5,6 +5,7 @@ import (
"database/sql" "database/sql"
"encoding/hex" "encoding/hex"
_ "modernc.org/sqlite" _ "modernc.org/sqlite"
"sync"
"time" "time"
) )
@@ -25,7 +26,11 @@ type Instance struct {
ContainerID string `json:"container_id"` ContainerID string `json:"container_id"`
Created time.Time `json:"created_at"` Created time.Time `json:"created_at"`
} }
type Store struct{ DB *sql.DB } type Store struct {
DB *sql.DB
closed chan struct{}
closeOnce sync.Once
}
func Open(path string) (*Store, error) { func Open(path string) (*Store, error) {
db, e := sql.Open("sqlite", path) db, e := sql.Open("sqlite", path)
@@ -42,9 +47,10 @@ func Open(path string) (*Store, error) {
db.Close() db.Close()
return nil, e return nil, e
} }
return &Store{db}, nil return &Store{DB: db, closed: make(chan struct{})}, nil
} }
func (s *Store) Close() { s.DB.Close() } func (s *Store) Close() { s.closeOnce.Do(func() { close(s.closed); s.DB.Close() }) }
func (s *Store) Done() <-chan struct{} { return s.closed }
func ID() string { b := make([]byte, 16); _, _ = rand.Read(b); return hex.EncodeToString(b) } func ID() string { b := make([]byte, 16); _, _ = rand.Read(b); return hex.EncodeToString(b) }
func (s *Store) UserByName(n string) (User, error) { func (s *Store) UserByName(n string) (User, error) {
var u User var u User
+1 -1
View File
@@ -26,7 +26,7 @@ try {
let images=(await request('/api/images')).data; let images=(await request('/api/images')).data;
const build=(await request('/api/builds','POST',{version:process.env.TEST_FROM_VERSION || images[0]?.version || '0.2.0-rc.2'})).data; const build=(await request('/api/builds','POST',{version:process.env.TEST_FROM_VERSION || images[0]?.version || '0.2.0-rc.2'})).data;
let buildSucceeded=false; let buildSucceeded=false;
for(let n=0;n<120;n++){const jobs=(await request('/api/builds')).data;const job=jobs.find(j=>j.id===build.id);if(job?.status==='failed')throw new Error('build/validation failed: '+job.error);if(job?.status==='success'){buildSucceeded=true;break;}await new Promise(r=>setTimeout(r,1000));} for(let n=0;n<120;n++){let job;try{job=(await request('/api/builds/'+encodeURIComponent(build.id))).data;}catch(e){if(!(e instanceof Error))throw e;await new Promise(r=>setTimeout(r,1000));continue;}if(job?.status==='failed')throw new Error('build/validation failed: '+job.error);if(job?.status==='success'){buildSucceeded=true;break;}await new Promise(r=>setTimeout(r,1000));}
assert.ok(buildSucceeded,'submitted build/validation job did not complete before timeout'); assert.ok(buildSucceeded,'submitted build/validation job did not complete before timeout');
images=(await request('/api/images')).data.filter(i=>i.status==='available');assert.ok(images.length,'validated image required'); images=(await request('/api/images')).data.filter(i=>i.status==='available');assert.ok(images.length,'validated image required');
const source=images.find(i=>!process.env.TEST_FROM_VERSION || i.version===process.env.TEST_FROM_VERSION); const source=images.find(i=>!process.env.TEST_FROM_VERSION || i.version===process.env.TEST_FROM_VERSION);
+18
View File
@@ -13,6 +13,7 @@
"react-dom": "^19.0.0" "react-dom": "^19.0.0"
}, },
"devDependencies": { "devDependencies": {
"@types/node": "^26.6.5",
"@types/react": "^19.0.0", "@types/react": "^19.0.0",
"@types/react-dom": "^19.0.0", "@types/react-dom": "^19.0.0",
"@vitejs/plugin-react": "^4.3.4", "@vitejs/plugin-react": "^4.3.4",
@@ -1281,6 +1282,16 @@
"dev": true, "dev": true,
"license": "MIT" "license": "MIT"
}, },
"node_modules/@types/node": {
"version": "26.6.5",
"resolved": "https://registry.npmmirror.com/@types/node/-/node-26.6.5.tgz",
"integrity": "sha512-uMzsv9stDdofIZBx4UGh4Z9ZgXnGC9seRDscilBcTHAeOFk67MZg/k9bWVfO+bk4qZmG+l52hRnLwUiH2LnJTg==",
"dev": true,
"license": "MIT",
"dependencies": {
"undici-types": "~8.9.0"
}
},
"node_modules/@types/react": { "node_modules/@types/react": {
"version": "19.3.0", "version": "19.3.0",
"resolved": "https://registry.npmmirror.com/@types/react/-/react-19.3.0.tgz", "resolved": "https://registry.npmmirror.com/@types/react/-/react-19.3.0.tgz",
@@ -1900,6 +1911,13 @@
"node": ">=14.17" "node": ">=14.17"
} }
}, },
"node_modules/undici-types": {
"version": "8.9.0",
"resolved": "https://registry.npmmirror.com/undici-types/-/undici-types-8.9.0.tgz",
"integrity": "sha512-KTDyRTYX8sWmKXAikPHHSyc63CRPETMctyjKFupcC6OBLXT3xsN0e9aF7m+mIXutFWpUXuedtowG7iLOzp0kQg==",
"dev": true,
"license": "MIT"
},
"node_modules/update-browserslist-db": { "node_modules/update-browserslist-db": {
"version": "1.3.4", "version": "1.3.4",
"resolved": "https://registry.npmmirror.com/update-browserslist-db/-/update-browserslist-db-1.3.4.tgz", "resolved": "https://registry.npmmirror.com/update-browserslist-db/-/update-browserslist-db-1.3.4.tgz",
+1
View File
@@ -15,6 +15,7 @@
"react-dom": "^19.0.0" "react-dom": "^19.0.0"
}, },
"devDependencies": { "devDependencies": {
"@types/node": "^26.6.5",
"@types/react": "^19.0.0", "@types/react": "^19.0.0",
"@types/react-dom": "^19.0.0", "@types/react-dom": "^19.0.0",
"@vitejs/plugin-react": "^4.3.4", "@vitejs/plugin-react": "^4.3.4",
+9 -5
View File
@@ -1,8 +1,9 @@
import { useEffect, useRef, useState, type FormEvent } from 'react'; import { useEffect, useRef, useState, type FormEvent } from 'react';
import { api, type Image, type Instance } from './api'; import { api, normalBuildAllowed, selectableImage, type Image, type Instance } from './api';
import { message, ResourceState, useAction, useResource } from './ui'; import { message, ResourceState, useAction, useResource } from './ui';
export function CreateInstance({close, refresh}: {close: () => void; refresh: () => Promise<void>}) { export function CreateInstance({close, refresh}: {close: () => void; refresh: () => Promise<void>}) {
const compatibility = useResource(api.compatibility);
const images = useResource(api.images), action = useAction(); const images = useResource(api.images), action = useAction();
const [mode, setMode] = useState('local'), [progress, setProgress] = useState(''); const [mode, setMode] = useState('local'), [progress, setProgress] = useState('');
const [version, setVersion] = useState(''); const [version, setVersion] = useState('');
@@ -11,7 +12,7 @@ export function CreateInstance({close, refresh}: {close: () => void; refresh: ()
useEffect(() => {let live=true; const load=()=>api.request<typeof intents>('/intents').then(v=>{if(live){setIntents(v);setPollError('');}}).catch(e=>{if(live)setPollError(message(e));}); void load(); const timer=window.setInterval(load,2000); return ()=>{live=false;window.clearInterval(timer);};}, []); useEffect(() => {let live=true; const load=()=>api.request<typeof intents>('/intents').then(v=>{if(live){setIntents(v);setPollError('');}}).catch(e=>{if(live)setPollError(message(e));}); void load(); const timer=window.setInterval(load,2000); return ()=>{live=false;window.clearInterval(timer);};}, []);
const active = useRef(true); const active = useRef(true);
useEffect(() => {active.current = true; return () => {active.current = false;};}, []); useEffect(() => {active.current = true; return () => {active.current = false;};}, []);
const available = images.data?.filter(i => !i.status || i.status === 'available') || []; const available = images.data?.filter(selectableImage) || [];
async function submit(e: FormEvent<HTMLFormElement>) { async function submit(e: FormEvent<HTMLFormElement>) {
e.preventDefault(); e.preventDefault();
const form = new FormData(e.currentTarget), name = String(form.get('name')); const form = new FormData(e.currentTarget), name = String(form.get('name'));
@@ -19,10 +20,13 @@ export function CreateInstance({close, refresh}: {close: () => void; refresh: ()
let image: Image | undefined = available.find(i => (i.image_id || i.id) === form.get('image')); let image: Image | undefined = available.find(i => (i.image_id || i.id) === form.get('image'));
if (mode === 'latest') { if (mode === 'latest') {
if (!version) { if (!version) {
const latest = await api.request<{version:string}>('/latest'); const latest = await api.request<{version:string; build_allowed:boolean}>('/latest');
if (!latest.version) throw new Error('服务器未返回精确版本。'); if (!latest.version) throw new Error('服务器未返回精确版本。');
setVersion(latest.version); return; setVersion(latest.version);
if (!latest.build_allowed || !normalBuildAllowed(await api.compatibility(), latest.version)) throw new Error('此最新版不支持普通构建。管理员可前往设置 → 镜像与构建进行实验构建,但实验镜像不能直接用于创建。');
return;
} }
if (!normalBuildAllowed(await api.compatibility(), version)) throw new Error('此版本不支持普通构建;实验镜像不能直接用于创建。');
const intent = await api.request<{id:string}>('/intents', 'POST', {name, version}); const intent = await api.request<{id:string}>('/intents', 'POST', {name, version});
if (!intent.id) throw new Error('未收到意图确认,请刷新核实,勿重复提交。'); if (!intent.id) throw new Error('未收到意图确认,请刷新核实,勿重复提交。');
setIntents(await api.request<typeof intents>('/intents')); setIntents(await api.request<typeof intents>('/intents'));
@@ -37,5 +41,5 @@ export function CreateInstance({close, refresh}: {close: () => void; refresh: ()
}); });
if (active.current) setProgress(''); if (active.current) setProgress('');
} }
return <form className="inline-form" onSubmit={submit}><h3>创建独立实例</h3>{action.feedback}<p className="callout">确认后服务器持久预留一个名额(每用户最多两个)。刷新或关闭不会取消自动创建;失败释放名额,不回退其他版本。取消等待不会取消共享构建,创建开始后不可撤回。</p><fieldset disabled={action.busy}><label>实例名称<input name="name" required maxLength={64}/></label><label>创建方式<select value={mode} onChange={e => {setMode(e.target.value); setVersion('');}}><option value="local">选择本地镜像</option><option value="latest">构建最新版并创建</option></select></label>{mode === 'local' && <><ResourceState resource={images}/><label>本地镜像<select name="image" required defaultValue=""><option value="" disabled>选择已构建镜像</option>{available.map(i => <option key={i.image_id || i.id} value={i.image_id || i.id}>{i.version} · {i.tag || i.image_id || i.id}</option>)}</select></label>{!images.loading && !available.length && <p>暂无可用镜像,可选择构建最新版并创建。</p>}</>}{mode === 'latest' && version && <p>将锁定精确版本 <strong>{version}</strong>,确认后预留名额并构建创建。</p>}<button className="primary" disabled={mode === 'local' && !available.length}>{mode === 'local' ? '确认创建' : version ? '确认版本并预留创建' : '查询最新版'}</button></fieldset>{progress && <p role="status">{progress}</p>}{pollError && <p className="muted" role="alert">意图刷新失败:{pollError}</p>}<ul>{intents.map(i => <li key={i.id}>{i.name} · {i.version} · {i.status} {i.error}{i.status === 'waiting' && <button type="button" disabled={action.busy} onClick={() => void action.run(async () => {await api.request('/intents/' + i.id, 'DELETE'); setIntents(await api.request<typeof intents>('/intents')); await refresh();})}>取消等待并释放名额</button>}{i.status === 'success' && <button type="button" onClick={() => void refresh()}>刷新实例列表</button>}</li>)}</ul><button type="button" disabled={action.busy} onClick={close}>关闭(保留已提交意图)</button></form>; return <form className="inline-form" onSubmit={submit}><h3>创建独立实例</h3>{action.feedback}<p className="callout">确认后服务器持久预留一个名额(每用户最多两个)。刷新或关闭不会取消自动创建;失败释放名额,不回退其他版本。取消等待不会取消共享构建,创建开始后不可撤回。</p><fieldset disabled={action.busy}><label>实例名称<input name="name" required maxLength={64}/></label><label>创建方式<select value={mode} onChange={e => {setMode(e.target.value); setVersion('');}}><option value="local">选择本地镜像</option><option value="latest">构建最新版并创建</option></select></label>{mode === 'local' && <><ResourceState resource={images}/><label>本地镜像<select name="image" required defaultValue=""><option value="" disabled>选择已构建镜像</option>{available.map(i => <option key={i.image_id || i.id} value={i.image_id || i.id}>{i.version} · {i.tag || i.image_id || i.id}</option>)}</select></label>{!images.loading && !available.length && <p>暂无已准入的可用镜像。历史或实验镜像不可用于创建。</p>}</>}{mode === 'latest' && <ResourceState resource={compatibility}/>} {mode === 'latest' && version && compatibility.data && !normalBuildAllowed(compatibility.data, version) && <p className="alert">此最新版不支持普通构建。管理员可前往设置 → 镜像与构建实验构建;实验镜像不能直接用于创建。</p>}{mode === 'latest' && version && compatibility.data && normalBuildAllowed(compatibility.data, version) && <p>将锁定精确版本 <strong>{version}</strong>,确认后预留名额并构建创建。</p>}<button className="primary" disabled={mode === 'local' ? !available.length : !!version && (!compatibility.data || !normalBuildAllowed(compatibility.data, version))}>{mode === 'local' ? '确认创建' : version ? '确认版本并预留创建' : '查询最新版'}</button></fieldset>{progress && <p role="status">{progress}</p>}{pollError && <p className="muted" role="alert">意图刷新失败:{pollError}</p>}<ul>{intents.map(i => <li key={i.id}>{i.name} · {i.version} · {i.status} {i.error}{i.status === 'waiting' && <button type="button" disabled={action.busy} onClick={() => void action.run(async () => {await api.request('/intents/' + i.id, 'DELETE'); setIntents(await api.request<typeof intents>('/intents')); await refresh();})}>取消等待并释放名额</button>}{i.status === 'success' && <button type="button" onClick={() => void refresh()}>刷新实例列表</button>}</li>)}</ul><button type="button" disabled={action.busy} onClick={close}>关闭(保留已提交意图)</button></form>;
} }
+127
View File
@@ -0,0 +1,127 @@
import { afterEach, beforeEach, expect, it, vi } from 'vitest';
const state = vi.hoisted(() => ({slots: [] as any[], cursor: 0}));
vi.mock('react', async original => ({
...await original<typeof import('react')>(),
useState(value: unknown) {const i = state.cursor++; if (!(i in state.slots)) state.slots[i] = value; return [state.slots[i], (next: unknown) => {state.slots[i] = next;}];},
useRef(value: unknown) {const i = state.cursor++; return state.slots[i] ??= {current:value};},
useEffect() {},
}));
vi.mock('./ui', async original => ({
...await original<typeof import('./ui')>(),
useResource: (load: unknown) => ({data: load === api.compatibility ? policy : load === api.images ? [{id:'ok', version:'1.2.3', status:'available', selectable:true}, {id:'exp', version:'9.9.9', status:'experimental', selectable:false}] : [], loading:false, refresh:vi.fn(async () => {})}),
}));
import { api, type Compatibility } from './api';
import { Images } from './Images';
import { CreateInstance } from './CreateInstance';
vi.mock('./ProfileEditor', () => ({ProfileEditor: () => null}));
import { Instances } from './Instances';
const policy: Compatibility = {recipe_rev:'7', platform:'linux/amd64', entries:[], unknown:{state:'existing_only', build_allowed:false, reason:'unknown'}, experimental_admin_only:true};
function nodes(root: any, predicate: (node: any) => boolean): any[] {
if (Array.isArray(root)) return root.flatMap(child => nodes(child, predicate));
if (!root || typeof root !== 'object') return [];
return [...(predicate(root) ? [root] : []), ...nodes(root.props?.children, predicate)];
}
function render(admin = false) {state.cursor = 0; return Images({admin});}
beforeEach(() => {state.slots = []; state.cursor = 0; vi.stubGlobal('window', {confirm:vi.fn(() => false)});});
afterEach(() => {vi.restoreAllMocks(); vi.unstubAllGlobals();});
it('hides experimental and admission controls from ordinary users', () => {
const tree = render();
expect(nodes(tree, n => n.type === 'input' && n.props.type === 'checkbox')).toHaveLength(0);
expect(nodes(tree, n => n.type === 'button' && n.props.children === '准入管理')).toHaveLength(0);
expect(nodes(render(true), n => n.type === 'button' && n.props.children === '准入管理')).toHaveLength(2);
});
it('canceled build confirmation sends zero requests', () => {
const request = vi.spyOn(api, 'startBuild');
const check = vi.spyOn(api, 'compatibility');
nodes(render(true), n => n.type === 'form')[0].props.onSubmit({preventDefault:vi.fn()});
expect(window.confirm).toHaveBeenCalledOnce();
expect(request).not.toHaveBeenCalled(); expect(check).not.toHaveBeenCalled();
});
it('requires evidence and confirmation; cancel sends no admission mutation', () => {
const request = vi.spyOn(api, 'admitImage');
nodes(render(true), n => n.type === 'button' && n.props.children === '准入管理')[0].props.onClick();
const child = nodes(render(true), n => typeof n.type === 'function' && n.props.imageId)[0];
state.slots = []; state.cursor = 0;
let form = child.type(child.props);
expect(nodes(form, n => n.type === 'button')[0].props.disabled).toBe(true);
nodes(form, n => n.type === 'textarea')[0].props.onChange({target:{value:'实际验收记录'}});
nodes(form, n => n.type === 'input')[0].props.onChange({target:{checked:true}});
state.cursor = 0; form = child.type(child.props);
form.props.onSubmit({preventDefault:vi.fn()});
expect(window.confirm).toHaveBeenCalledOnce(); expect(request).not.toHaveBeenCalled();
});
it('unknown latest never reserves an intent, including a second submit', async () => {
const request = vi.spyOn(api, 'request').mockResolvedValue({version:'9.9.9', build_allowed:false});
vi.spyOn(api, 'compatibility').mockResolvedValue(policy);
vi.stubGlobal('FormData', class {get() {return 'name';}});
const renderCreate = () => {state.cursor = 0; return CreateInstance({close:vi.fn(), refresh:vi.fn(async () => {})});};
let form = renderCreate();
expect(nodes(form, n => n.type === 'option' && n.props.value === 'exp')).toHaveLength(0);
nodes(form, n => n.type === 'select')[0].props.onChange({target:{value:'latest'}});
form = renderCreate(); await form.props.onSubmit({preventDefault:vi.fn(), currentTarget:{}});
form = renderCreate(); await form.props.onSubmit({preventDefault:vi.fn(), currentTarget:{}});
expect(request.mock.calls).toEqual([['/latest']]);
});
it('admission rejects blank/unconfirmed evidence, resets consent on edits, and submits once', async () => {
const request = vi.spyOn(api, 'admitImage').mockResolvedValue({});
nodes(render(true), n => n.type === 'button' && n.props.children === '准入管理')[0].props.onClick();
const child = nodes(render(true), n => typeof n.type === 'function' && n.props.imageId)[0];
state.slots = [];
const renderAdmission = () => {state.cursor = 0; return child.type(child.props);};
const submit = () => renderAdmission().props.onSubmit({preventDefault:vi.fn()});
submit(); expect(request).not.toHaveBeenCalled();
nodes(renderAdmission(), n => n.type === 'input')[0].props.onChange({target:{checked:true}});
submit(); expect(request).not.toHaveBeenCalled();
nodes(renderAdmission(), n => n.type === 'textarea')[0].props.onChange({target:{value:'验收记录 #42'}});
expect(nodes(renderAdmission(), n => n.type === 'input')[0].props.checked).toBe(false);
submit(); expect(request).not.toHaveBeenCalled();
nodes(renderAdmission(), n => n.type === 'input')[0].props.onChange({target:{checked:true}});
nodes(renderAdmission(), n => n.type === 'select')[0].props.onChange({target:{value:'recommended'}});
expect(nodes(renderAdmission(), n => n.type === 'input')[0].props.checked).toBe(false);
nodes(renderAdmission(), n => n.type === 'input')[0].props.onChange({target:{checked:true}});
vi.mocked(window.confirm).mockReturnValue(true);
submit(); submit();
await Promise.resolve();
expect(request).toHaveBeenCalledExactlyOnceWith('ok', 'recommended', '验收记录 #42');
});
it.each([
['a'.repeat(4096), 4096, true],
['a'.repeat(4097), 4097, false],
['中'.repeat(1400), 4200, false],
['中'.repeat(1365), 4095, true],
['中'.repeat(1365) + 'a', 4096, true],
['中'.repeat(1365) + 'ab', 4097, false],
['😀'.repeat(1024), 4096, true],
['😀'.repeat(1024) + 'a', 4097, false],
[' ' + 'é'.repeat(2048) + '\n', 4096, true],
] as const)('checks UTF-8 evidence bytes (case %#)', async (evidence, bytes, valid) => {
const request = vi.spyOn(api, 'admitImage').mockResolvedValue({});
vi.mocked(window.confirm).mockReturnValue(true);
nodes(render(true), n => n.type === 'button' && n.props.children === '准入管理')[0].props.onClick();
const child = nodes(render(true), n => typeof n.type === 'function' && n.props.imageId)[0];
state.slots = [];
const renderAdmission = () => {state.cursor = 0; return child.type(child.props);};
nodes(renderAdmission(), n => n.type === 'textarea')[0].props.onChange({target:{value:evidence}});
nodes(renderAdmission(), n => n.type === 'input')[0].props.onChange({target:{checked:true}});
const form = renderAdmission();
expect(nodes(form, n => n.type === 'textarea')[0].props.maxLength).toBeUndefined();
expect(nodes(form, n => n.type === 'textarea')[0].props['aria-invalid']).toBe(!valid);
expect(nodes(form, n => n.type === 'p' && Array.isArray(n.props.children) && n.props.children.includes(bytes))).toHaveLength(1);
expect(nodes(form, n => n.type === 'button')[0].props.disabled).toBe(!valid);
form.props.onSubmit({preventDefault:vi.fn()});
await Promise.resolve();
if (valid) expect(request).toHaveBeenCalledExactlyOnceWith('ok', 'compatible', evidence.trim());
else {expect(request).not.toHaveBeenCalled(); expect(window.confirm).not.toHaveBeenCalled();}
});
it('upgrade selector excludes experimental and unadmitted images', () => {
state.cursor = 0; Instances();
state.slots = [];
// Instances keeps the selected detail after its action state and creating flag.
state.slots[5] = {id:'instance', name:'test', version:'1', status:'stopped'};
state.cursor = 0;
const detail = nodes(Instances(), n => typeof n.type === 'function' && n.props.instance)[0];
state.slots = ['upgrade']; state.cursor = 0;
const tree = detail.type(detail.props);
expect(nodes(tree, n => n.type === 'option' && n.props.value === 'ok')).toHaveLength(1);
expect(nodes(tree, n => n.type === 'option' && n.props.value === 'exp')).toHaveLength(0);
});
+118 -5
View File
@@ -1,10 +1,123 @@
import { useState, type FormEvent } from 'react'; import { useState, type FormEvent } from 'react';
import { api } from './api'; import { api, normalBuildAllowed, type AdmissionState, type Build } from './api';
import { Panel, ResourceState, Empty, Status, date, useAction, useResource } from './ui'; import { Panel, ResourceState, Empty, Status, date, useAction, useResource } from './ui';
export const activeBuild = (build: Build) => ['queued', 'running', 'validating'].includes(build.status);
function BuildLogs({id, close}: {id: string; close: () => void}) {
const logs = useResource(() => api.build(id), id, 5000, activeBuild);
return <div>
<h3>构建日志</h3>
<button onClick={close}>关闭日志</button>
<ResourceState resource={logs}/>{logs.data && <>
<p>{logs.data.experimental && <strong className="alert">实验构建 · 不自动准入</strong>}{logs.data.version} · <Status value={logs.data.status}/>
</p>{logs.data.error && <p className="alert">{logs.data.error}</p>}<pre className="logs">{logs.data.logs || '任务尚未输出日志'}</pre>
</>}</div>;
}
function Admission({imageId, refresh}: {imageId:string; refresh:() => Promise<void>}) {
const [state, setState] = useState<AdmissionState>('compatible'), [evidence, setEvidence] = useState(''), [confirmed, setConfirmed] = useState(false);
const action = useAction();
const submittedEvidence = evidence.trim(), evidenceBytes = new TextEncoder().encode(submittedEvidence).length;
const evidenceInvalid = evidenceBytes === 0 || evidenceBytes > 4096;
return <form onSubmit={e => {e.preventDefault(); if (!confirmed || evidenceInvalid) return; void action.run(async () => {await api.admitImage(imageId, state, submittedEvidence); setEvidence(''); setConfirmed(false); await refresh();}, '镜像准入策略已更新', '确认依据实际验收结果修改此镜像准入策略?');}}>{action.feedback}<label>准入策略<select value={state} onChange={e => {setState(e.target.value as AdmissionState); setConfirmed(false);}}>
<option value="recommended">推荐</option>
<option value="compatible">兼容</option>
<option value="existing_only">仅存量</option>
<option value="blocked">禁止使用</option>
</select>
</label>
<p className="callout">构建成功不等于通过验收。请记录启动、登录、终端、文件和配置等实际兼容性验收结果及可追溯依据,不可仅填写“构建成功”。单镜像准入不扩大普通构建版本白名单;未知版本仅可选择已准入的本地镜像 ID 创建或升级。</p>
<label>验收依据<textarea required aria-invalid={evidenceBytes > 4096} value={evidence} onChange={e => {setEvidence(e.target.value); setConfirmed(false);}}/>
</label>
<p className={evidenceBytes > 4096 ? 'alert' : 'muted'} role={evidenceBytes > 4096 ? 'alert' : undefined}>验收依据:{evidenceBytes} / 4096 UTF-8 字节(去除首尾空白后);中文通常占 3 字节,表情通常占 4 字节。{evidenceBytes > 4096 && '已超出字节上限,请缩短依据。'}</p>
<label>
<input type="checkbox" checked={confirmed} onChange={e => setConfirmed(e.target.checked)}/>我已核验依据;禁止使用还会阻止启动/重启,但不自动停机</label>
<button disabled={action.busy || !confirmed || evidenceInvalid}>确认更新准入</button>
</form>;
}
export function Images({admin = false}: {admin?:boolean}) { export function Images({admin = false}: {admin?:boolean}) {
const images = useResource(api.images), builds = useResource(api.builds, '', 5000); const action = useAction(); const [experimental, setExperimental] = useState(false), [admissionId, setAdmissionId] = useState<string>();
const [page, setPage] = useState(0);
const compatibility = useResource(api.compatibility);
const images = useResource(api.images), builds = useResource(() => api.builds(21, page * 20), String(page), 5000, rows => rows.slice(0, 20).some(activeBuild)); const action = useAction();
const [latest, setLatest] = useState<{version:string; checked_at?:string}>(), [version,setVersion] = useState(''), [selected,setSelected] = useState<string>(); const [latest, setLatest] = useState<{version:string; checked_at?:string}>(), [version,setVersion] = useState(''), [selected,setSelected] = useState<string>();
function build(e: FormEvent) {e.preventDefault(); void action.run(async () => {await api.request('/builds','POST',{version}); await builds.refresh();}, '构建任务已提交,请查看任务的实际状态。', '确认在本机手动构建精确版本 ' + version + '?构建会消耗磁盘、网络与 CPU,不会自动升级实例。');} function build(e: FormEvent) {e.preventDefault(); void action.run(async () => {if (!(admin && experimental) && !normalBuildAllowed(await api.compatibility(), version)) throw new Error('此版本不支持普通构建,请管理员选择实验构建;实验构建不会自动准入。'); await api.startBuild(version, admin && experimental); if (page === 0) await builds.refresh(); else setPage(0);}, '构建任务已提交,请查看任务的实际状态。', (experimental ? '实验构建不会自动准入或用于创建。确认实验构建精确版本 ' : '确认在本机手动构建精确版本 ') + version + '?构建会消耗磁盘、网络与 CPU,不会自动升级实例。');}
const current = builds.data?.find(b => b.id === selected); const rows = builds.data?.slice(0, 20);
return <>{action.feedback}<Panel title="查询与手动构建"><div className="split"><div><p className="muted">从 npm 查询 dist-tags.latest,仅查询,不自动构建。</p><button disabled={action.busy} onClick={() => action.run(async () => {const result = await api.request<{version:string; checked_at?:string}>('/latest'); setLatest(result); setVersion(result.version);}, '最新版本查询完成')}>查询 latest</button>{latest && <p><strong>{latest.version}</strong> <small>查询时间 {date(latest.checked_at)}</small></p>}</div><form className="build-form" onSubmit={build}><label>精确 DSH 版本<input required value={version} onChange={e => setVersion(e.target.value)} placeholder="查询 latest,或填写精确版本" pattern="[0-9]+\.[0-9]+\.[0-9]+.*" /></label><button className="primary" disabled={action.busy || !version}>手动构建</button></form></div><p className="callout">镜像仅保存在本机。构建成功不等于实例升级;创建或升级前请检查可用状态。</p></Panel><Panel title="本地镜像" action={<button disabled={images.loading} onClick={images.refresh}>刷新</button>}><ResourceState resource={images}/>{images.data?.length === 0 && <Empty>本机暂无托管镜像,请先手动构建。</Empty>}{!!images.data?.length && <div className="table-scroll"><table><thead><tr><th>DSH 版本</th><th>镜像标识</th><th>标签</th><th>状态</th><th>使用/操作</th></tr></thead><tbody>{images.data.map(i => <tr key={i.image_id || i.id}><td><strong>{i.version}</strong></td><td className="mono wrap">{i.image_id || i.id}</td><td>{i.tag || '—'}</td><td><Status value={i.status}/></td><td>{i.usage_count ?? 0} 个实例 {admin && <button className="danger" disabled={action.busy || !!i.usage_count} onClick={() => action.run(async () => {await api.request('/images?image_id=' + encodeURIComponent(i.image_id || i.id), 'DELETE'); await images.refresh();}, '镜像已删除', '确认删除这个本地镜像?服务器将检查所有引用,不会强制删除。')}>清理镜像</button>}</td></tr>)}</tbody></table></div>}</Panel><Panel title="构建任务" action={<span className="muted">每 5 秒自动刷新</span>}><ResourceState resource={builds}/>{builds.data?.length === 0 && <Empty>没有构建任务记录</Empty>}{!!builds.data?.length && <div className="table-scroll"><table><thead><tr><th>任务</th><th>版本</th><th>状态</th><th>提交时间</th><th>日志</th></tr></thead><tbody>{builds.data.map(b => <tr key={b.id}><td className="mono">{b.id}</td><td>{b.version}</td><td><Status value={b.status}/></td><td>{date(b.created_at)}</td><td><button onClick={() => setSelected(b.id)}>查看日志</button></td></tr>)}</tbody></table></div>}{current && <div><h3>{current.version} · 构建日志</h3>{current.error && <p className="alert">{current.error}</p>}<pre className="logs">{current.logs || '任务尚未输出日志'}</pre></div>}</Panel></>; function changePage(next: number) { setSelected(undefined); setPage(next); }
return <>{action.feedback}<Panel title="查询与手动构建">
<ResourceState resource={compatibility}/>{version && compatibility.data && !normalBuildAllowed(compatibility.data, version) && <p className="alert">此版本不支持普通构建。仅管理员可实验构建,不会自动准入。</p>}<div className="split">
<div>
<p className="muted">从 npm 查询 dist-tags.latest,仅查询,不自动构建。</p>
<button disabled={action.busy} onClick={() => action.run(async () => {const result = await api.request<{version:string; checked_at?:string}>('/latest'); setLatest(result); setVersion(result.version);}, '最新版本查询完成')}>查询 latest</button>{latest && <p>
<strong>{latest.version}</strong> <small>查询时间 {date(latest.checked_at)}</small>
</p>}</div>
<form className="build-form" onSubmit={build}>
<label>精确 DSH 版本<input required value={version} onChange={e => setVersion(e.target.value)} placeholder="查询 latest,或填写精确版本" pattern="[0-9]+\.[0-9]+\.[0-9]+.*" />
</label>
<>{admin && <label>
<input type="checkbox" checked={experimental} onChange={e => setExperimental(e.target.checked)}/>管理员实验构建(不自动准入)</label>}{experimental && <p className="alert" role="alert">实验镜像:构建成功后仍不能创建或升级实例,必须另行验收并显式准入。</p>}<button className="primary" disabled={action.busy || !version}>{experimental ? '提交实验构建' : '手动构建'}</button>
</>
</form>
</div>
<p className="callout">镜像仅保存在本机。构建成功不等于通过验收或自动准入。仅 available 且允许选择的镜像可创建或升级;existing_only 仅允许存量启停。blocked 禁止新建、升级及新的启动/重启,但不会自动停止已运行实例或断开连接。</p>
</Panel>
<Panel title="本地镜像" action={<button disabled={images.loading} onClick={images.refresh}>刷新</button>}>
<ResourceState resource={images}/>{images.data?.length === 0 && <Empty>本机暂无托管镜像,请先手动构建。</Empty>}{!!images.data?.length && <div className="table-scroll">
<table>
<thead>
<tr>
<th>DSH 版本</th>
<th>镜像标识</th>
<th>标签</th>
<th>状态</th>
<th>使用/操作</th>
</tr>
</thead>
<tbody>{images.data.map(i => <tr key={i.image_id || i.id}>
<td>
<strong>{i.version}</strong>
</td>
<td className="mono wrap">{i.image_id || i.id}</td>
<td>{i.tag || '—'}</td>
<td>
<Status value={i.status}/>
<p>{({recommended:'推荐',compatible:'兼容',existing_only:'仅存量',blocked:'禁止使用'} as Record<string,string>)[i.admission || ''] || '未验证'}</p>
<small>{i.admission_reason || '无准入依据'}</small>
<p>{i.selectable && i.status === 'available' ? '可创建 / 升级' : '不可创建 / 升级'}</p>
</td>
<td>{i.usage_count ?? 0} 个实例 {admin && <button onClick={() => setAdmissionId(admissionId === (i.image_id || i.id) ? undefined : (i.image_id || i.id))}>准入管理</button>}{admissionId === (i.image_id || i.id) && admin && <Admission key={admissionId} imageId={admissionId} refresh={images.refresh}/>} {admin && <button className="danger" disabled={action.busy || !!i.usage_count} onClick={() => action.run(async () => {await api.request('/images?image_id=' + encodeURIComponent(i.image_id || i.id), 'DELETE'); await images.refresh();}, '镜像已删除', '确认删除这个本地镜像?服务器将检查所有引用,不会强制删除。')}>清理镜像</button>}</td>
</tr>)}</tbody>
</table>
</div>}</Panel>
<Panel title="构建任务" action={<>
<span className="muted">当前页有活跃任务时每 5 秒刷新</span>
<button disabled={builds.loading} onClick={builds.refresh}>刷新任务</button>
</>}>
<ResourceState resource={builds}/>{builds.data?.length === 0 && <Empty>没有构建任务记录</Empty>}{!!builds.data?.length && <div className="table-scroll">
<table>
<thead>
<tr>
<th>任务</th>
<th>版本</th>
<th>状态</th>
<th>提交时间</th>
<th>日志</th>
</tr>
</thead>
<tbody>{rows!.map(b => <tr key={b.id}>
<td className="mono">{b.id}</td>
<td>{b.version}{b.experimental && <strong className="alert">实验构建 · 不自动准入</strong>}</td>
<td>
<Status value={b.status}/>
</td>
<td>{date(b.created_at)}</td>
<td>
<button onClick={() => setSelected(b.id)}>查看日志</button>
</td>
</tr>)}</tbody>
</table>
</div>}<div className="actions">
<button disabled={page === 0 || builds.loading} onClick={() => changePage(page - 1)}>上一页</button>
<span>第 {page + 1} 页</span>
<button disabled={builds.loading || !builds.data || builds.data.length <= 20} onClick={() => changePage(page + 1)}>下一页</button>
</div>{selected && <BuildLogs key={selected} id={selected} close={() => setSelected(undefined)}/>}</Panel>
</>;
} }
+3 -3
View File
@@ -2,7 +2,7 @@ import { useState } from 'react';
import { ProfileEditor } from './ProfileEditor'; import { ProfileEditor } from './ProfileEditor';
import { BackupPanel } from './BackupPanel'; import { BackupPanel } from './BackupPanel';
import { CreateInstance } from './CreateInstance'; import { CreateInstance } from './CreateInstance';
import { api, instancePath, type Instance } from './api'; import { api, selectableImage, instancePath, type Instance } from './api';
import { Panel, ResourceState, Empty, Status, bytes, date, useAction, useResource } from './ui'; import { Panel, ResourceState, Empty, Status, bytes, date, useAction, useResource } from './ui';
export function Instances() { export function Instances() {
const instances = useResource(api.instances, '', 10000), usage = useResource(api.usage); const instances = useResource(api.instances, '', 10000), usage = useResource(api.usage);
@@ -15,11 +15,11 @@ function InstanceDetail({instance, close, refresh}: {instance: Instance; close:
const base = instancePath(instance.id); const base = instancePath(instance.id);
const stopped = ['stopped', 'exited', 'created'].includes(instance.status); const stopped = ['stopped', 'exited', 'created'].includes(instance.status);
const images = useResource(api.images); const images = useResource(api.images);
const available = images.data?.filter(i => !i.status || i.status === 'available') || []; const available = images.data?.filter(selectableImage) || [];
const [upgradeImage, setUpgradeImage] = useState(''); const [upgradeImage, setUpgradeImage] = useState('');
const [upgradePassword, setUpgradePassword] = useState(''); const [upgradePassword, setUpgradePassword] = useState('');
const profile = useResource(() => api.request<{content:string}>(base + '/profile')); const profile = useResource(() => api.request<{content:string}>(base + '/profile'));
const preview = useResource(() => api.request<{bundles:unknown;config:string;preview_error?:string}>(base + '/profile-preview')); const preview = useResource(() => api.request<{bundles:unknown;config:string;preview_error?:string}>(base + '/profile-preview'));
const logs = useResource(() => api.request<{logs:string}>(base + '/logs')); const logs = useResource(() => api.request<{logs:string}>(base + '/logs'));
return <Panel title={instance.name + ' · 实例管理'} action={<button onClick={close}>关闭</button>}><div className="tabs">{[['profile','Profile 配置'],['preview','配置预览'],['backups','备份与恢复'],['upgrade','版本升级'],['logs','运行日志']].map(([id,label]) => <button className={tab === id ? 'active' : ''} onClick={() => setTab(id)} key={id}>{label}</button>)}</div>{action.feedback}{tab === 'profile' && <div className="callout">Profile 写入前需先停止实例。当前状态:<Status value={instance.status}/>{!stopped && <button disabled={action.busy} onClick={() => action.run(async () => {await api.request(base + '/stop','POST'); await refresh();}, '停止请求已完成,请确认实例状态。', '停止将中断当前连接,是否继续?')}>停止实例</button>}</div>}{tab === 'profile' && <><p className="muted">profiles/web/cordis.patch.yml · 停止实例后保存,保存后自动启动验证。密钥原文不在此展示。</p><ResourceState resource={profile} />{profile.data && <><label>YAML 配置<ProfileEditor value={content ?? profile.data.content} onChange={setContent} readOnly={action.busy}/></label><div className="actions"><button className="primary" disabled={action.busy || !stopped || content === undefined} onClick={() => action.run(async () => {await api.request(base + '/profile','PUT',{content,restart:true}); await profile.refresh(); setContent(undefined); await refresh();}, '配置已保存并启动验证。', '确认覆盖当前 Profile 配置?')}>保存并启动</button><button disabled={action.busy} onClick={() => action.run(async () => {await api.request(base + '/restart','POST'); await refresh();}, '实例已重启', '重启将中断当前连接,是否继续?')}>重启实例</button></div></>}</>}{tab === 'preview' && <><button onClick={preview.refresh}>刷新预览</button><ResourceState resource={preview}/>{preview.data && <><h3>Bundle 清单</h3><pre>{JSON.stringify(preview.data.bundles,null,2)}</pre>{preview.data.preview_error && <p role="status">{preview.data.preview_error}</p>}<ProfileEditor value={preview.data.config} readOnly label="配置预览(字符串已脱敏)"/></>}</>}{tab === 'backups' && <BackupPanel instance={instance} refresh={async () => {await profile.refresh(); setContent(undefined); await refresh();}}/>}{tab === 'upgrade' && <form className="inline-form" onSubmit={e => {e.preventDefault(); if (!available.some(i => (i.image_id || i.id) === upgradeImage)) return; void action.run(async () => {try {await api.request(base + '/upgrade','POST',{image_id:upgradeImage,password:upgradePassword}); await refresh();} finally {setUpgradePassword('');}}, '实例升级请求已完成。', '确认使用所选本地镜像升级?建议先备份配置。');}}><p>当前版本:{instance.version}</p><label>当前登录密码(加密升级前备份)<input type="password" required autoComplete="current-password" value={upgradePassword} onChange={e => setUpgradePassword(e.target.value)}/></label><ResourceState resource={images}/><label>本地镜像<select required value={upgradeImage} onChange={e => setUpgradeImage(e.target.value)}><option value="" disabled>选择可用镜像</option>{available.map(i => <option key={i.image_id || i.id} value={i.image_id || i.id}>{i.version} · {i.tag || i.image_id || i.id}</option>)}</select></label><button className="primary" disabled={action.busy || !upgradeImage}>升级实例</button></form>}{tab === 'logs' && <><button onClick={logs.refresh} disabled={logs.loading}>刷新日志</button><ResourceState resource={logs}/><pre className="logs">{logs.data?.logs || '暂无日志'}</pre></>}</Panel>; return <Panel title={instance.name + ' · 实例管理'} action={<button onClick={close}>关闭</button>}><div className="tabs">{[['profile','Profile 配置'],['preview','配置预览'],['backups','备份与恢复'],['upgrade','版本升级'],['logs','运行日志']].map(([id,label]) => <button className={tab === id ? 'active' : ''} onClick={() => setTab(id)} key={id}>{label}</button>)}</div>{action.feedback}{tab === 'profile' && <div className="callout">Profile 写入前需先停止实例。当前状态:<Status value={instance.status}/>{!stopped && <button disabled={action.busy} onClick={() => action.run(async () => {await api.request(base + '/stop','POST'); await refresh();}, '停止请求已完成,请确认实例状态。', '停止将中断当前连接,是否继续?')}>停止实例</button>}</div>}{tab === 'profile' && <><p className="muted">profiles/web/cordis.patch.yml · 停止实例后保存,保存后自动启动验证。密钥原文不在此展示。</p><ResourceState resource={profile} />{profile.data && <><label>YAML 配置<ProfileEditor value={content ?? profile.data.content} onChange={setContent} readOnly={action.busy}/></label><div className="actions"><button className="primary" disabled={action.busy || !stopped || content === undefined} onClick={() => action.run(async () => {await api.request(base + '/profile','PUT',{content,restart:true}); await profile.refresh(); setContent(undefined); await refresh();}, '配置已保存并启动验证。', '确认覆盖当前 Profile 配置?')}>保存并启动</button><button disabled={action.busy} onClick={() => action.run(async () => {await api.request(base + '/restart','POST'); await refresh();}, '实例已重启', '重启将中断当前连接,是否继续?')}>重启实例</button></div></>}</>}{tab === 'preview' && <><button onClick={preview.refresh}>刷新预览</button><ResourceState resource={preview}/>{preview.data && <><h3>Bundle 清单</h3><pre>{JSON.stringify(preview.data.bundles,null,2)}</pre>{preview.data.preview_error && <p role="status">{preview.data.preview_error}</p>}<ProfileEditor value={preview.data.config} readOnly label="配置预览(字符串已脱敏)"/></>}</>}{tab === 'backups' && <BackupPanel instance={instance} refresh={async () => {await profile.refresh(); setContent(undefined); await refresh();}}/>}{tab === 'upgrade' && <form className="inline-form" onSubmit={e => {e.preventDefault(); if (!available.some(i => (i.image_id || i.id) === upgradeImage)) return; void action.run(async () => {try {await api.request(base + '/upgrade','POST',{image_id:upgradeImage,password:upgradePassword}); await refresh();} finally {setUpgradePassword('');}}, '实例升级请求已完成。', '确认使用所选本地镜像升级?建议先备份配置。');}}><p>当前版本:{instance.version}</p><p className="callout">仅已准入且本地可用的镜像可用于升级。历史镜像仅允许存量实例运行,不可用于新建或升级。</p><label>当前登录密码(加密升级前备份)<input type="password" required autoComplete="current-password" value={upgradePassword} onChange={e => setUpgradePassword(e.target.value)}/></label><ResourceState resource={images}/><label>本地镜像<select required value={upgradeImage} onChange={e => setUpgradeImage(e.target.value)}><option value="" disabled>选择可用镜像</option>{available.map(i => <option key={i.image_id || i.id} value={i.image_id || i.id}>{i.version} · {i.tag || i.image_id || i.id}</option>)}</select></label><button className="primary" disabled={action.busy || !upgradeImage}>升级实例</button></form>}{tab === 'logs' && <><button onClick={logs.refresh} disabled={logs.loading}>刷新日志</button><ResourceState resource={logs}/><pre className="logs">{logs.data?.logs || '暂无日志'}</pre></>}</Panel>;
} }
+48
View File
@@ -0,0 +1,48 @@
/// <reference types="vite/client" />
import { useEffect, useRef } from 'react';
import * as monaco from 'monaco-editor/esm/vs/editor/editor.api';
import 'monaco-editor/esm/vs/basic-languages/yaml/yaml.contribution';
import EditorWorker from 'monaco-editor/esm/vs/editor/editor.worker?worker';
// Vite emits this worker locally; no AMD loader or CDN is involved.
self.MonacoEnvironment = { getWorker: () => new EditorWorker() };
export interface ProfileEditorProps {
value: string;
onChange?: (value: string) => void;
readOnly?: boolean;
label?: string;
}
export default function MonacoProfileEditor({ value, onChange, readOnly = false, label = 'YAML 配置' }: ProfileEditorProps) {
const host = useRef<HTMLDivElement>(null);
const editor = useRef<monaco.editor.IStandaloneCodeEditor | null>(null);
const change = useRef(onChange);
change.current = onChange;
const initialValue = useRef(value);
const updating = useRef(false);
useEffect(() => {
if (!host.current) return;
const model = monaco.editor.createModel(initialValue.current, 'yaml');
const instance = monaco.editor.create(host.current, {
model, automaticLayout: true, minimap: { enabled: false },
scrollBeyondLastLine: false, tabSize: 2, insertSpaces: true,
wordWrap: 'on', ariaLabel: label, readOnly,
});
editor.current = instance;
const listener = instance.onDidChangeModelContent(() => {
if (!updating.current) change.current?.(instance.getValue());
});
return () => { listener.dispose(); instance.dispose(); model.dispose(); editor.current = null; };
}, []);
useEffect(() => {
const instance = editor.current;
if (instance && instance.getValue() !== value) {
updating.current = true;
instance.setValue(value);
updating.current = false;
}
}, [value]);
useEffect(() => { editor.current?.updateOptions({ readOnly, ariaLabel: label }); }, [readOnly, label]);
return <div role="group" aria-label={label} ref={host} style={{ height: 420, minWidth: 0, border: '1px solid var(--border, #d5dbe5)', borderRadius: 8, overflow: 'hidden' }} />;
}
+17
View File
@@ -0,0 +1,17 @@
import { afterEach, expect, it, vi } from 'vitest';
import { renderToStaticMarkup } from 'react-dom/server';
vi.mock('monaco-editor/esm/vs/editor/editor.api', () => ({editor: {}}));
vi.mock('monaco-editor/esm/vs/basic-languages/yaml/yaml.contribution', () => ({}));
vi.mock('monaco-editor/esm/vs/editor/editor.worker?worker', () => ({default: class {}}));
afterEach(() => vi.unstubAllGlobals());
it('distinguishes mandatory profile-page stopping from unrestricted general file management', async () => {
vi.stubGlobal('self', {});
const { ProfileEditor } = await import('./ProfileEditor');
const markup = renderToStaticMarkup(<ProfileEditor value="name: test" label="测试配置" />);
expect(markup).toContain('本页面保存必须先停止实例');
expect(markup).toContain('通用文件管理不受此流程强制约束');
expect(markup).toContain('不承诺凭据仅写入');
expect(markup).not.toContain('这里不强制停机');
expect(markup).toContain('role="group" aria-label="测试配置"');
});
+7 -45
View File
@@ -1,48 +1,10 @@
/// <reference types="vite/client" /> import { lazy, Suspense } from 'react';
import { useEffect, useRef } from 'react'; import { homeAccessNotice } from './fileWarnings';
import * as monaco from 'monaco-editor/esm/vs/editor/editor.api'; import type { ProfileEditorProps } from './MonacoProfileEditor';
import 'monaco-editor/esm/vs/basic-languages/yaml/yaml.contribution';
import EditorWorker from 'monaco-editor/esm/vs/editor/editor.worker?worker';
// Vite emits this worker locally; no AMD loader or CDN is involved. export type { ProfileEditorProps } from './MonacoProfileEditor';
self.MonacoEnvironment = { getWorker: () => new EditorWorker() }; const MonacoProfileEditor = lazy(() => import('./MonacoProfileEditor'));
export interface ProfileEditorProps { export function ProfileEditor(props: ProfileEditorProps) {
value: string; return <><p className="muted">{homeAccessNotice} 本页面保存必须先停止实例,建议修改前备份;通用文件管理不受此流程强制约束。</p><Suspense fallback={<div role="group" aria-label={props.label ?? 'YAML 配置'} style={{ height: 420 }}><p role="status">正在加载编辑器…</p></div>}><MonacoProfileEditor {...props}/></Suspense></>;
onChange?: (value: string) => void;
readOnly?: boolean;
label?: string;
}
export function ProfileEditor({ value, onChange, readOnly = false, label = 'YAML 配置' }: ProfileEditorProps) {
const host = useRef<HTMLDivElement>(null);
const editor = useRef<monaco.editor.IStandaloneCodeEditor | null>(null);
const change = useRef(onChange);
change.current = onChange;
const initialValue = useRef(value);
const updating = useRef(false);
useEffect(() => {
if (!host.current) return;
const model = monaco.editor.createModel(initialValue.current, 'yaml');
const instance = monaco.editor.create(host.current, {
model, automaticLayout: true, minimap: { enabled: false },
scrollBeyondLastLine: false, tabSize: 2, insertSpaces: true,
wordWrap: 'on', ariaLabel: label, readOnly,
});
editor.current = instance;
const listener = instance.onDidChangeModelContent(() => {
if (!updating.current) change.current?.(instance.getValue());
});
return () => { listener.dispose(); instance.dispose(); model.dispose(); editor.current = null; };
}, []);
useEffect(() => {
const instance = editor.current;
if (instance && instance.getValue() !== value) {
updating.current = true;
instance.setValue(value);
updating.current = false;
}
}, [value]);
useEffect(() => { editor.current?.updateOptions({ readOnly, ariaLabel: label }); }, [readOnly, label]);
return <div role="group" aria-label={label} ref={host} style={{ height: 420, minWidth: 0, border: '1px solid var(--border, #d5dbe5)', borderRadius: 8, overflow: 'hidden' }} />;
} }
+84
View File
@@ -0,0 +1,84 @@
import { afterEach, beforeEach, expect, it, vi } from 'vitest';
const state = vi.hoisted(() => ({slots: [] as any[], cursor: 0, entries: [] as any[]}));
vi.mock('react', async importOriginal => ({
...await importOriginal<typeof import('react')>(),
useState(value: unknown) { const i = state.cursor++; if (!(i in state.slots)) state.slots[i] = value; return [state.slots[i], (next: unknown) => {state.slots[i] = next;}]; },
useRef(value: unknown) {const i = state.cursor++; return state.slots[i] ??= {current: value};},
}));
vi.mock('./ui', async importOriginal => ({
...await importOriginal<typeof import('./ui')>(),
useResource: () => ({data: state.entries, refresh: vi.fn(async () => {}), loading: false, error: ''}),
}));
vi.mock('./api', () => ({api: {download: vi.fn(() => '/download-test'), upload: vi.fn(async () => {}), rename: vi.fn(async () => {}), delete: vi.fn(async () => {})}}));
import { Workspace } from './Workspace';
import { api } from './api';
function render() {
state.cursor = 0;
const scope = Workspace({runtime: {instances: [{id: 'one', name: 'Running instance', status: 'running'}], selectedId: 'one'} as any}) as any;
const files = scope.type(scope.props);
return files.type(files.props);
}
function nodes(root: any, predicate: (node: any) => boolean): any[] {
if (Array.isArray(root)) return root.flatMap(child => nodes(child, predicate));
if (!root || typeof root !== 'object') return [];
return [...(predicate(root) ? [root] : []), ...nodes(root.props?.children, predicate)];
}
beforeEach(() => {
state.slots = []; state.cursor = 0; state.entries = [{name: '.credentials.yaml', is_dir: false}];
vi.clearAllMocks(); vi.stubGlobal('window', {confirm: vi.fn(() => false)});
});
afterEach(() => vi.unstubAllGlobals());
it.each([false, true])('cancel blocks download navigation and acceptance preserves owner access (directory=%s)', is_dir => {
state.entries = [{name: is_dir ? 'photos' : 'renamed.txt', is_dir}];
const link = nodes(render(), n => n.type === 'a')[0];
for (const handler of ['onClick', 'onAuxClick', 'onContextMenu']) {
const event = {preventDefault: vi.fn(), button: 1};
link.props[handler](event);
expect(event.preventDefault).toHaveBeenCalledOnce();
vi.mocked(window.confirm).mockReturnValueOnce(true);
const accepted = {preventDefault: vi.fn(), button: 1};
link.props[handler](accepted);
expect(accepted.preventDefault).not.toHaveBeenCalled();
}
expect(link.props.href).toBe('/download-test');
expect(window.confirm).toHaveBeenCalledTimes(6);
expect(window.confirm).toHaveBeenCalledWith(expect.stringContaining(is_dir ? 'ZIP' : '凭据'));
});
it('does not double-prompt auxiliary right-click after the context menu', () => {
nodes(render(), n => n.type === 'a')[0].props.onAuxClick({button: 2, preventDefault: vi.fn()});
expect(window.confirm).not.toHaveBeenCalled();
});
it('canceled upload sends no request, resets input, and accepted upload needs no stop', async () => {
const input = nodes(render(), n => n.type === 'input' && n.props.type === 'file')[0];
const file = {name: '.credentials.yaml'};
const event = {target: {files: [file], value: 'chosen'}};
input.props.onChange(event);
expect(api.upload).not.toHaveBeenCalled();
expect(event.target.value).toBe('');
vi.mocked(window.confirm).mockReturnValue(true);
input.props.onChange(event);
await Promise.resolve();
expect(api.upload).toHaveBeenCalledExactlyOnceWith('.', file, 'one');
expect(window.confirm).toHaveBeenCalledWith(expect.stringContaining('备份'));
});
it('canceled delete sends no request; accepted delete retains full-home access', async () => {
const button = nodes(render(), n => n.type === 'button' && n.props.children === '删除')[0];
await button.props.onClick();
expect(api.delete).not.toHaveBeenCalled();
vi.mocked(window.confirm).mockReturnValue(true);
await button.props.onClick();
expect(api.delete).toHaveBeenCalledExactlyOnceWith('.credentials.yaml', 'one');
});
it('rename cancellation preserves the draft without writing; accepting writes once', async () => {
nodes(render(), n => n.type === 'button' && n.props.children === '重命名')[0].props.onClick();
const form = nodes(render(), n => n.type === 'form')[1];
form.props.onSubmit({preventDefault: vi.fn()});
expect(api.rename).not.toHaveBeenCalled();
expect(nodes(render(), n => n.type === 'form')).toHaveLength(2);
vi.mocked(window.confirm).mockReturnValue(true);
form.props.onSubmit({preventDefault: vi.fn()});
await Promise.resolve();
expect(api.rename).toHaveBeenCalledExactlyOnceWith('.credentials.yaml', '.credentials.yaml', 'one');
});
+2 -1
View File
@@ -1,5 +1,6 @@
import { useState, type FormEvent, type ReactNode } from 'react'; import { useState, type FormEvent, type ReactNode } from 'react';
import { api } from './api'; import { api } from './api';
import { confirmDownload, downloadWarning, homeAccessNotice, mutationWarning } from './fileWarnings';
import { Alert, Panel, ResourceState, Empty, bytes, date, useAction, useResource } from './ui'; import { Alert, Panel, ResourceState, Empty, bytes, date, useAction, useResource } from './ui';
import { RuntimeSelector } from './RuntimeSelector'; import { RuntimeSelector } from './RuntimeSelector';
import type { useRuntimeInstances } from './RuntimeWorkspace'; import type { useRuntimeInstances } from './RuntimeWorkspace';
@@ -29,5 +30,5 @@ function WorkspaceFiles({instanceId, instanceName, controls, path, setPath}: Fil
const join = (name: string) => joinFilePath(path, name); const join = (name: string) => joinFilePath(path, name);
const refresh = async () => {await Promise.all([tree.refresh(), usage.refresh()]);}; const refresh = async () => {await Promise.all([tree.refresh(), usage.refresh()]);};
function mkdir(e: FormEvent) {e.preventDefault(); void action.run(async () => {await api.mkdir(join(newName), instanceId); setNewName(''); await refresh();}, '目录已创建');} function mkdir(e: FormEvent) {e.preventDefault(); void action.run(async () => {await api.mkdir(join(newName), instanceId); setNewName(''); await refresh();}, '目录已创建');}
return <>{controls(action.busy)}<div className="workspace-banner"><div><p className="eyebrow">INSTANCE FILES</p><h2>实例文件</h2><p>{'当前实例:' + instanceName}</p><p>实例数据目录为 /workspace(模板 7);历史镜像的路径可能不同。</p><p>这里管理文件,不会切换 DSH 会话的工作目录。包含配置和密钥,请谨慎操作。</p></div><div><p>账号总用量</p><strong>{bytes(usage.data?.used_bytes)}</strong><p>账号配额:{bytes(usage.data?.quota_bytes)}</p></div></div><ResourceState resource={usage}/>{action.feedback}<Panel title="文件浏览器" action={<button onClick={refresh} disabled={tree.loading || action.busy}>刷新</button>}><p className="muted">当前完整路径:<code>{displayFilePath('instance', path)}</code></p><div className="file-toolbar"><div className="breadcrumbs"><button onClick={() => go('.')} disabled={path === '.' || action.busy}>{instanceId ? '实例文件' : '历史共享目录'}</button>{path !== '.' && path.split('/').map((part,index,parts) => <span key={index}> / <button disabled={action.busy} onClick={() => go(parts.slice(0,index+1).join('/'))}>{part}</button></span>)}</div><label className={'button upload ' + (action.busy ? 'disabled' : '')}>↑ 上传文件<input type="file" disabled={action.busy} onChange={e => {const file = e.target.files?.[0]; if(file) void action.run(async () => {await api.upload(path,file,instanceId); await refresh();}, '文件已上传'); e.target.value = '';}}/></label></div><form className="compact-form" onSubmit={mkdir}><label className="sr-only" htmlFor="new-folder">新目录名称</label><input disabled={action.busy} id="new-folder" value={newName} onChange={e => setNewName(e.target.value)} placeholder="新目录名称" required pattern="[^/\\]+"/><button disabled={action.busy}>创建目录</button></form>{rename && <form className="compact-form" onSubmit={e => {e.preventDefault(); void action.run(async () => {await api.rename(rename.path,join(rename.name),instanceId); setRename(undefined); await refresh();}, '重命名已完成', '确认重命名此文件或目录?');}}><label>新名称<input disabled={action.busy} required value={rename.name} pattern="[^/\\]+" onChange={e => setRename({...rename,name:e.target.value})}/></label><button disabled={action.busy}>确认重命名</button><button type="button" disabled={action.busy} onClick={() => setRename(undefined)}>取消</button></form>}<ResourceState resource={tree}/>{tree.data?.length === 0 && <Empty>这个目录还是空的。上传文件或创建目录开始使用。</Empty>}{!!tree.data?.length && <div className="table-scroll"><table><thead><tr><th>名称</th><th>大小</th><th>修改时间</th><th>操作</th></tr></thead><tbody>{tree.data.map(entry => {const target = entry.path || join(entry.name); return <tr key={entry.name}><td>{entry.is_dir ? <button className="file-link" disabled={action.busy} onClick={() => go(target)}>▰ {entry.name}</button> : <span>▤ {entry.name}</span>}</td><td>{entry.is_dir ? '目录' : bytes(entry.size)}</td><td>{date(entry.mod_time)}</td><td><div className="actions"><a className="button" href={api.download(target,instanceId)}>下载</a><button disabled={action.busy} onClick={() => setRename({path:target,name:entry.name})}>重命名</button><button className="danger" disabled={action.busy} onClick={() => action.run(async () => {await api.delete(target,instanceId); await refresh();}, '已删除', '确认永久删除「' + entry.name + '」' + (entry.is_dir ? '(仅允许空目录)' : '') + '?可能包含敏感配置,删除无法撤销。')}>删除</button></div></td></tr>;})}</tbody></table></div>}</Panel></>; return <>{controls(action.busy)}<div className="workspace-banner"><div><p className="eyebrow">INSTANCE FILES</p><h2>实例文件</h2><p>{'当前实例:' + instanceName}</p><p>实例数据目录为 /workspace(模板 7);历史镜像的路径可能不同。</p><p>这里管理文件,不会切换 DSH 会话的工作目录。包含配置和密钥,请谨慎操作。</p><p>{homeAccessNotice}</p><p className="muted">下载确认及文件名识别仅为操作提醒,不是权限边界;右键打开下载链接等方式可能绕过提示。</p></div><div><p>账号总用量</p><strong>{bytes(usage.data?.used_bytes)}</strong><p>账号配额:{bytes(usage.data?.quota_bytes)}</p></div></div><ResourceState resource={usage}/>{action.feedback}<Panel title="文件浏览器" action={<button onClick={refresh} disabled={tree.loading || action.busy}>刷新</button>}><p className="muted">当前完整路径:<code>{displayFilePath('instance', path)}</code></p><div className="file-toolbar"><div className="breadcrumbs"><button onClick={() => go('.')} disabled={path === '.' || action.busy}>{instanceId ? '实例文件' : '历史共享目录'}</button>{path !== '.' && path.split('/').map((part,index,parts) => <span key={index}> / <button disabled={action.busy} onClick={() => go(parts.slice(0,index+1).join('/'))}>{part}</button></span>)}</div><label className={'button upload ' + (action.busy ? 'disabled' : '')}>↑ 上传文件<input type="file" disabled={action.busy} onChange={e => {const file = e.target.files?.[0]; if(file) void action.run(async () => {await api.upload(path,file,instanceId); await refresh();}, '文件已上传', mutationWarning('upload', join(file.name))); e.target.value = '';}}/></label></div><form className="compact-form" onSubmit={mkdir}><label className="sr-only" htmlFor="new-folder">新目录名称</label><input disabled={action.busy} id="new-folder" value={newName} onChange={e => setNewName(e.target.value)} placeholder="新目录名称" required pattern="[^/\\]+"/><button disabled={action.busy}>创建目录</button></form>{rename && <form className="compact-form" onSubmit={e => {e.preventDefault(); void action.run(async () => {await api.rename(rename.path,join(rename.name),instanceId); setRename(undefined); await refresh();}, '重命名已完成', mutationWarning('rename', rename.path, join(rename.name)));}}><label>新名称<input disabled={action.busy} required value={rename.name} pattern="[^/\\]+" onChange={e => setRename({...rename,name:e.target.value})}/></label><button disabled={action.busy}>确认重命名</button><button type="button" disabled={action.busy} onClick={() => setRename(undefined)}>取消</button></form>}<ResourceState resource={tree}/>{tree.data?.length === 0 && <Empty>这个目录还是空的。上传文件或创建目录开始使用。</Empty>}{!!tree.data?.length && <div className="table-scroll"><table><thead><tr><th>名称</th><th>大小</th><th>修改时间</th><th>操作</th></tr></thead><tbody>{tree.data.map(entry => {const target = entry.path || join(entry.name); return <tr key={entry.name}><td>{entry.is_dir ? <button className="file-link" disabled={action.busy} onClick={() => go(target)}>▰ {entry.name}</button> : <span>▤ {entry.name}</span>}</td><td>{entry.is_dir ? '目录' : bytes(entry.size)}</td><td>{date(entry.mod_time)}</td><td><div className="actions"><a className="button" href={api.download(target,instanceId)} title={downloadWarning(target, entry.is_dir)} onClick={e => confirmDownload(e, target, entry.is_dir)} onAuxClick={e => {if (e.button === 1) confirmDownload(e, target, entry.is_dir);}} onContextMenu={e => confirmDownload(e, target, entry.is_dir)}>下载</a><button disabled={action.busy} onClick={() => setRename({path:target,name:entry.name})}>重命名</button><button className="danger" disabled={action.busy} onClick={() => action.run(async () => {await api.delete(target,instanceId); await refresh();}, '已删除', mutationWarning('delete', target))}>删除</button></div></td></tr>;})}</tbody></table></div>}</Panel></>;
} }
+48 -7
View File
@@ -4,8 +4,13 @@ export class ApiError extends Error {
export interface User { id: string; username: string; admin: boolean; quota_bytes: number; created_at?: string } export interface User { id: string; username: string; admin: boolean; quota_bytes: number; created_at?: string }
export interface Session { user: User; csrf_token: string } export interface Session { user: User; csrf_token: string }
export interface Instance { id: string; name: string; status: string; image_id: string; version: string; created_at: string } export interface Instance { id: string; name: string; status: string; image_id: string; version: string; created_at: string }
export interface Image { id: string; image_id?: string; version: string; tag?: string; status?: string; usage_count?: number; recipe_rev?: string; platform?: string; created_at?: string } export type AdmissionState = 'recommended' | 'compatible' | 'existing_only' | 'blocked';
export interface Build { id: string; version: string; status: string; image_id?: string; logs?: string; error?: string; created_at: string } export interface Image { id: string; image_id?: string; version: string; tag?: string; status?: string; admission?: string; admission_reason?: string; selectable?: boolean; usage_count?: number; recipe_rev?: string; platform?: string; created_at?: string }
export interface Build { id: string; version: string; status: string; experimental?: boolean; image_id?: string; logs?: string; error?: string; created_at: string }
export const selectableImage = (image: Image) => image.status === 'available' && image.selectable === true;
export interface CompatibilityRule { state: string; build_allowed: boolean; reason: string }
export interface Compatibility { recipe_rev: string; platform: string; entries: (CompatibilityRule & {version:string; recipe_rev:string; platform:string})[]; unknown: CompatibilityRule; experimental_admin_only: boolean }
export const normalBuildAllowed = (policy: Compatibility, version: string) => policy.entries.some(entry => entry.version === version && entry.recipe_rev === policy.recipe_rev && entry.platform === policy.platform && entry.build_allowed === true);
export interface Entry { name: string; path?: string; is_dir: boolean; size: number; mod_time: string } export interface Entry { name: string; path?: string; is_dir: boolean; size: number; mod_time: string }
export interface Usage { used_bytes: number; quota_bytes: number } export interface Usage { used_bytes: number; quota_bytes: number }
export interface Backup { id: string; created_at: string; size: number } export interface Backup { id: string; created_at: string; size: number }
@@ -14,20 +19,52 @@ export function createApi(fetcher: typeof fetch = fetch) {
let csrf = ''; let csrf = '';
let refreshing: Promise<Session> | undefined; let refreshing: Promise<Session> | undefined;
const csrfCookie = () => typeof document === 'undefined' ? '' : document.cookie.split('; ').find(c => c.startsWith('dsh_csrf='))?.slice(9) || ''; const csrfCookie = () => typeof document === 'undefined' ? '' : document.cookie.split('; ').find(c => c.startsWith('dsh_csrf='))?.slice(9) || '';
const currentCSRF = () => typeof document === 'undefined' ? csrf : csrfCookie();
async function recoverSession() {
const recover = async (checkFirst: boolean) => {
if (checkFirst) {
try { return await request<Session>('/me', 'GET', undefined, true); }
catch (e) { if (!(e instanceof ApiError) || e.status !== 401) throw e; }
}
try { return await request<Session>('/refresh', 'POST', undefined, true); }
catch (failure) {
// Another page may have rotated the shared cookies while this request
// was in flight; keep rechecking while a slow sibling tab settles.
const retries = [0, 100, 250, 500, 1000, 2000];
for (const delay of retries) {
if (delay) await new Promise(resolve => setTimeout(resolve, delay));
try { return await request<Session>('/me', 'GET', undefined, true); }
catch (e) {
if (!(e instanceof ApiError) || e.status !== 401) throw e;
if (delay === retries[retries.length - 1]) throw failure instanceof ApiError && failure.status === 403 ? e : failure;
}
}
throw new ApiError(401, '登录已失效,请重新登录。');
}
};
return typeof navigator !== 'undefined' && navigator.locks
? navigator.locks.request('dsh-session-refresh', () => recover(true))
: recover(false);
}
async function request<T>(path: string, method = 'GET', data?: unknown, retried = false): Promise<T> { async function request<T>(path: string, method = 'GET', data?: unknown, retried = false): Promise<T> {
const headers = new Headers(); const headers = new Headers();
if (method !== 'GET' && (csrf || csrfCookie())) headers.set('X-CSRF-Token', csrf || csrfCookie()); const token = currentCSRF();
if (method !== 'GET' && token) headers.set('X-CSRF-Token', token);
const multipart = data instanceof FormData; const multipart = data instanceof FormData;
if (data !== undefined && !multipart) headers.set('Content-Type', 'application/json'); if (data !== undefined && !multipart) headers.set('Content-Type', 'application/json');
let response: Response; let response: Response;
try { response = await fetcher('/api' + path, { method, credentials: 'same-origin', headers, body: data === undefined ? undefined : multipart ? data : JSON.stringify(data) }); } try { response = await fetcher('/api' + path, { method, credentials: 'same-origin', headers, body: data === undefined ? undefined : multipart ? data : JSON.stringify(data) }); }
catch { throw new ApiError(0, '无法连接服务器,请检查网络或后端服务。'); } catch { throw new ApiError(0, '无法连接服务器,请检查网络或后端服务。'); }
if (response.status === 401 && !retried && path !== '/login' && path !== '/refresh' && (csrf || csrfCookie())) { if (response.status === 401 && !retried && path !== '/login' && path !== '/refresh' && currentCSRF()) {
try { try {
refreshing ??= request<Session>('/refresh', 'POST', undefined, true).finally(() => { refreshing = undefined; }); refreshing ??= recoverSession().finally(() => { refreshing = undefined; });
const session = await refreshing; csrf = session.csrf_token; const session = await refreshing; csrf = session.csrf_token;
} catch (e) {
if (!(e instanceof ApiError) || e.status !== 401) throw e;
if (path !== '/me' && typeof window !== 'undefined') window.dispatchEvent(new Event('session-expired'));
throw e;
}
return request<T>(path, method, data, true); return request<T>(path, method, data, true);
} catch { /* The original response determines session expiration handling. */ }
} }
const text = await response.text(); const text = await response.text();
let payload: unknown; let payload: unknown;
@@ -46,7 +83,11 @@ export function createApi(fetcher: typeof fetch = fetch) {
async logout() { await request('/logout', 'POST'); csrf = ''; }, async logout() { await request('/logout', 'POST'); csrf = ''; },
instances: () => request<Instance[]>('/instances'), instances: () => request<Instance[]>('/instances'),
images: () => request<Image[]>('/images'), images: () => request<Image[]>('/images'),
builds: () => request<Build[]>('/builds'), compatibility: () => request<Compatibility>('/compatibility'),
startBuild: (version: string, experimental = false) => request<Build>('/builds', 'POST', {version, experimental}),
admitImage: (image_id: string, state: AdmissionState, evidence: string) => request('/images/admission', 'POST', {image_id, state, evidence}),
builds: (limit = 20, offset = 0) => request<Build[]>(`/builds?limit=${limit}&offset=${offset}`),
build: (id: string) => request<Build>('/builds/' + encodeURIComponent(id)),
usage: (instance?: string) => request<Usage>('/ws/usage' + (instance ? '?instance=' + encodeURIComponent(instance) : '')), usage: (instance?: string) => request<Usage>('/ws/usage' + (instance ? '?instance=' + encodeURIComponent(instance) : '')),
tree: (path: string, instance?: string) => request<Entry[]>('/ws/tree?path=' + encodeURIComponent(path) + (instance ? '&instance=' + encodeURIComponent(instance) : '')), tree: (path: string, instance?: string) => request<Entry[]>('/ws/tree?path=' + encodeURIComponent(path) + (instance ? '&instance=' + encodeURIComponent(instance) : '')),
download: (path: string, instance?: string) => '/api/ws/download?path=' + encodeURIComponent(path) + (instance ? '&instance=' + encodeURIComponent(instance) : ''), download: (path: string, instance?: string) => '/api/ws/download?path=' + encodeURIComponent(path) + (instance ? '&instance=' + encodeURIComponent(instance) : ''),
+81 -1
View File
@@ -1,6 +1,8 @@
import { it, expect, vi } from 'vitest'; import { it, expect, vi, beforeEach, afterEach } from 'vitest';
import { createApi } from './api'; import { createApi } from './api';
const json = (body: unknown, status = 200) => new Response(JSON.stringify(body), {status}); const json = (body: unknown, status = 200) => new Response(JSON.stringify(body), {status});
beforeEach(() => vi.stubGlobal('navigator', {}));
afterEach(() => vi.unstubAllGlobals());
it('refreshes once after access expiry, retaining CSRF and retrying the request', async () => { it('refreshes once after access expiry, retaining CSRF and retrying the request', async () => {
const fetcher = vi.fn<typeof fetch>() const fetcher = vi.fn<typeof fetch>()
.mockResolvedValueOnce(json({user:{id:'u'},csrf_token:'csrf'})) .mockResolvedValueOnce(json({user:{id:'u'},csrf_token:'csrf'}))
@@ -40,3 +42,81 @@ it('shares one refresh across concurrent expired requests', async () => {
expect(await Promise.all([first,second])).toEqual([[],[]]); expect(await Promise.all([first,second])).toEqual([[],[]]);
expect(fetcher.mock.calls.filter(c=>c[0]==='/api/refresh')).toHaveLength(1); expect(fetcher.mock.calls.filter(c=>c[0]==='/api/refresh')).toHaveLength(1);
}); });
it.each([401, 403])('rechecks the shared session after refresh fails with %s', async status => {
const dispatchEvent = vi.fn(); vi.stubGlobal('window', {dispatchEvent});
const fetcher = vi.fn<typeof fetch>()
.mockResolvedValueOnce(json({csrf_token:'old'}))
.mockResolvedValueOnce(json({},401))
.mockResolvedValueOnce(json({},status))
.mockResolvedValueOnce(json({csrf_token:'new'}))
.mockResolvedValueOnce(json([]));
const api = createApi(fetcher); await api.me();
expect(await api.instances()).toEqual([]);
expect(fetcher.mock.calls.map(c => c[0])).toEqual(['/api/me','/api/instances','/api/refresh','/api/me','/api/instances']);
expect(dispatchEvent).not.toHaveBeenCalled();
});
it.each([401,403])('does not restore a revoked session after logout/password change (%s)', async status => {
const dispatchEvent = vi.fn(); vi.stubGlobal('window', {dispatchEvent});
const fetcher = vi.fn<typeof fetch>().mockResolvedValueOnce(json({csrf_token:'old'})).mockImplementation(async path => json({},path === '/api/refresh' ? status : 401));
const api = createApi(fetcher); await api.me();
await expect(api.instances()).rejects.toMatchObject({status:401});
expect(dispatchEvent).toHaveBeenCalledOnce();
expect(fetcher).toHaveBeenCalledTimes(9);
});
it('does not mistake a failed session recheck for logout', async () => {
const dispatchEvent = vi.fn(); vi.stubGlobal('window', {dispatchEvent});
const fetcher = vi.fn<typeof fetch>().mockResolvedValueOnce(json({csrf_token:'old'})).mockResolvedValueOnce(json({},401)).mockResolvedValueOnce(json({},401)).mockRejectedValue(new Error('offline'));
const api = createApi(fetcher); await api.me();
await expect(api.instances()).rejects.toMatchObject({status:0});
expect(dispatchEvent).not.toHaveBeenCalled();
});
it('uses current cookie rather than a cached CSRF after another tab logs in or out', async () => {
const document = {cookie:'dsh_csrf=first'}; vi.stubGlobal('document', document);
const fetcher = vi.fn<typeof fetch>().mockImplementation(async () => json({csrf_token:'cached'}));
const api = createApi(fetcher); await api.me();
document.cookie = 'dsh_csrf=second'; await api.login('user','password');
expect(new Headers(fetcher.mock.calls[1][1]?.headers).get('X-CSRF-Token')).toBe('second');
document.cookie = ''; await api.login('user','password');
expect(new Headers(fetcher.mock.calls[2][1]?.headers).has('X-CSRF-Token')).toBe(false);
});
it('coordinates independent clients with the origin-wide refresh lock', async () => {
let queue = Promise.resolve<unknown>(undefined);
const lock = vi.fn((_name: string, run: () => Promise<unknown>) => { const pending = queue.then(run); queue = pending.catch(() => {}); return pending; });
vi.stubGlobal('navigator', {locks:{request:lock}});
vi.stubGlobal('document', {cookie:'dsh_csrf=shared'});
let valid = false;
const fetcher = vi.fn<typeof fetch>().mockImplementation(async path => {
if (path === '/api/refresh') { valid = true; return json({csrf_token:'shared'}); }
if (path === '/api/me') return valid ? json({csrf_token:'shared'}) : json({},401);
return valid ? json([]) : json({},401);
});
expect(await Promise.all([createApi(fetcher).instances(), createApi(fetcher).images()])).toEqual([[],[]]);
expect(fetcher.mock.calls.filter(c => c[0] === '/api/refresh')).toHaveLength(1);
expect(lock.mock.calls.every(c => c[0] === 'dsh-session-refresh')).toBe(true);
});
it('waits for delayed winning cookies without Web Locks and never refreshes twice', async () => {
vi.useFakeTimers();
try {
const fetcher = vi.fn<typeof fetch>().mockResolvedValueOnce(json({csrf_token:'old'})).mockResolvedValueOnce(json({},401)).mockResolvedValueOnce(json({},401)).mockResolvedValueOnce(json({},401)).mockResolvedValueOnce(json({csrf_token:'winner'})).mockResolvedValueOnce(json([]));
const api = createApi(fetcher); await api.me();
const pending = api.instances(); await vi.advanceTimersByTimeAsync(100);
expect(await pending).toEqual([]);
expect(fetcher.mock.calls.filter(c => c[0] === '/api/refresh')).toHaveLength(1);
} finally { vi.useRealTimers(); }
});
it.each([409, 500, 0])('preserves refresh failure %s when access remains expired beyond the recheck window', async status => {
vi.useFakeTimers();
const dispatchEvent = vi.fn(); vi.stubGlobal('window', {dispatchEvent});
try {
const fetcher = vi.fn<typeof fetch>().mockImplementation(async path => {
if (path === '/api/login') return json({csrf_token:'old'});
if (path === '/api/refresh') { if (!status) throw new Error('offline'); return json({error:'refresh_conflict'},status); }
return json({},401);
});
const api = createApi(fetcher); await api.login('u','password');
const pending = expect(api.instances()).rejects.toMatchObject({status});
await vi.advanceTimersByTimeAsync(4000); await pending;
expect(dispatchEvent).not.toHaveBeenCalled();
expect(fetcher.mock.calls.filter(c => c[0] === '/api/refresh')).toHaveLength(1);
} finally { vi.useRealTimers(); }
});
+13
View File
@@ -0,0 +1,13 @@
import { expect, it, vi } from 'vitest';
import { createApi } from './api';
import { activeBuild } from './Images';
it('requests explicit pages independently of on-demand encoded log details', async () => {
const fetcher = vi.fn<typeof fetch>().mockImplementation(async () => new Response('[]'));
const api = createApi(fetcher);
await api.builds(21, 0); await api.builds(21, 20);
expect(fetcher.mock.calls.map(c => c[0])).toEqual(['/api/builds?limit=21&offset=0','/api/builds?limit=21&offset=20']);
await api.build('a/b'); expect(fetcher.mock.calls[2][0]).toBe('/api/builds/a%2Fb');
});
it.each(['queued','running','validating','success','failed'])('polls only active build status %s', status => {
expect(activeBuild({id:'b', version:'1', created_at:'', status})).toBe(['queued','running','validating'].includes(status));
});
+30
View File
@@ -0,0 +1,30 @@
import { expect, it, vi } from 'vitest';
import { build } from 'vite';
it('keeps Monaco lazy, production chunks below 500 kB, and static imports acyclic', async () => {
vi.stubEnv('NODE_ENV', 'production');
const result = await build({ build: { write: false }, logLevel: 'silent' }).finally(() => vi.unstubAllEnvs());
if (!('output' in result)) throw new Error('Expected a single application build');
const chunks = result.output.filter(asset => asset.type === 'chunk');
const byName = new Map(chunks.map(chunk => [chunk.fileName, chunk]));
const visited = new Set<string>();
function visit(name: string, ancestors: string[] = []) {
expect(ancestors, `Static chunk cycle: ${[...ancestors, name].join(' -> ')}`).not.toContain(name);
if (visited.has(name)) return;
const chunk = byName.get(name);
if (!chunk) return;
chunk.imports.forEach(dependency => visit(dependency, [...ancestors, name]));
visited.add(name);
}
for (const chunk of chunks) {
expect(Buffer.byteLength(chunk.code), chunk.fileName).toBeLessThanOrEqual(500_000);
visit(chunk.fileName);
}
const entry = chunks.find(chunk => chunk.isEntry)!;
visited.clear();
visit(entry.fileName);
for (const name of visited) {
expect(Object.keys(byName.get(name)!.modules).some(id => id.includes('/monaco-editor/'))).toBe(false);
}
expect(chunks.some(chunk => chunk.isDynamicEntry && chunk.name === 'MonacoProfileEditor')).toBe(true);
}, 60_000);
+26
View File
@@ -0,0 +1,26 @@
import { expect, it } from 'vitest';
import { downloadWarning, isSensitiveFile, mutationWarning } from './fileWarnings';
it('accurately explains that upload rejects existing names rather than overwriting', () => {
const warning = mutationWarning('upload', '.credentials.yaml');
expect(warning).toContain('同名文件不会覆盖,服务器将拒绝上传');
expect(warning).not.toContain('同名文件将被覆盖');
});
it.each(['.credentials.yaml', 'dsh-ssh.json', 'profiles/default.yaml', 'profiles', 'home/cordis.yml', 'cordis.config.ts', 'patch.yaml', 'package.json', 'pnpm-workspace.yaml', 'workspace.json', '.ssh/id_ed25519', '.env.local'])('recognizes common credential/config hint %s', path => {
expect(isSensitiveFile(path)).toBe(true);
});
it('warns on renamed files and arbitrary directory ZIPs without treating recognition as security', () => {
expect(isSensitiveFile('notes.txt')).toBe(false);
expect(downloadWarning('notes.txt', false)).toContain('凭据');
expect(downloadWarning('photos', true)).toContain('任意目录');
expect(downloadWarning('photos', true)).toContain('ZIP');
});
it('checks both rename paths and recommends stopping without requiring it', () => {
for (const paths of [['notes', '.credentials.yaml'], ['profiles/default.yaml', 'notes']]) {
const warning = mutationWarning('rename', paths[0], paths[1]);
expect(warning).toContain('此路径可能涉及凭据');
expect(warning).toContain('备份');
expect(warning).toContain('不强制停机');
}
});
+28
View File
@@ -0,0 +1,28 @@
export const homeAccessNotice = '所有者可完整管理实例 home,包括读取、下载凭据;不承诺凭据仅写入(write-only)。专用 profile 页面提供受控编辑流程,不是强制安全边界;通用文件上传、重命名或删除可绕过该流程的 YAML 校验和停机要求。';
const configMutationWarning = '修改配置可能影响实例运行。建议先停止实例并备份,再继续;这里不强制停机。';
// Filename hints are UX only: renamed secrets cannot be reliably identified.
export function isSensitiveFile(path: string): boolean {
const parts = path.toLowerCase().split('/');
const name = parts.at(-1) || '';
return parts.some(part => ['profiles', '.ssh', '.config', '.cordis'].includes(part))
|| /^(?:\.?credentials(?:\..+)?|dsh-ssh\.json|\.env(?:\..+)?|id_(?:rsa|ed25519|ecdsa)(?:\.pub)?)$/.test(name)
|| /^(?:\.?cordis|patch|package|(?:pnpm-)?workspace)(?:[.-].+)?$/.test(name)
|| /\.(?:pem|key|p12|pfx)$/.test(name);
}
export function downloadWarning(path: string, isDirectory: boolean): string {
if (isDirectory) return '确认下载目录「' + path + '」的 ZIP?任意目录都可能包含凭据和敏感配置,改名后也无法可靠识别。请妥善保管,勿公开分享。';
return '确认下载「' + path + '」?' + (isSensitiveFile(path) ? '此文件可能包含凭据或运行配置,请妥善保管,勿公开分享。' : '文件可能含凭据(改名无法可靠识别),请妥善保管。');
}
export function confirmDownload(event: {preventDefault: () => void}, path: string, isDirectory: boolean): void {
if (!window.confirm(downloadWarning(path, isDirectory))) event.preventDefault();
}
export function mutationWarning(operation: 'upload' | 'rename' | 'delete', path: string, destination?: string): string {
const question = operation === 'upload' ? '确认上传「' + path + '」?同名文件不会覆盖,服务器将拒绝上传。'
: operation === 'rename' ? '确认将「' + path + '」重命名为「' + destination + '」?'
: '确认永久删除「' + path + '」?目录仅允许删除空目录,删除无法撤销。';
return question + (isSensitiveFile(path) || (destination && isSensitiveFile(destination)) ? '此路径可能涉及凭据或敏感配置。' : '文件或目录也可能包含改名后的敏感配置。') + configMutationWarning;
}
+40
View File
@@ -0,0 +1,40 @@
import { expect, it, vi } from 'vitest';
import { createApi, normalBuildAllowed, selectableImage, type AdmissionState, type Compatibility } from './api';
it('reads the compatibility contract and accepts only explicit current tuples', async () => {
const policy: Compatibility = {recipe_rev:'7', platform:'linux/amd64', entries:[{version:'1.2.3', recipe_rev:'7', platform:'linux/amd64', state:'recommended', build_allowed:true, reason:'tested'}], unknown:{state:'existing_only', build_allowed:false, reason:'unknown'}, experimental_admin_only:true};
const fetcher = vi.fn<typeof fetch>().mockResolvedValue(new Response(JSON.stringify(policy)));
const result = await createApi(fetcher).compatibility();
expect(fetcher.mock.calls[0][0]).toBe('/api/compatibility');
expect(result).toEqual(policy);
expect(normalBuildAllowed(result, '1.2.3')).toBe(true);
expect(normalBuildAllowed(result, '9.9.9')).toBe(false);
expect(normalBuildAllowed({...result, recipe_rev:'8'}, '1.2.3')).toBe(false);
expect(normalBuildAllowed({...result, platform:'linux/arm64'}, '1.2.3')).toBe(false);
expect(normalBuildAllowed({...result, entries:result.entries.map(entry => ({...entry, build_allowed:false}))}, '1.2.3')).toBe(false);
});
it('sends exact experimental build and evidence-bearing admission contracts', async () => {
const fetcher = vi.fn<typeof fetch>().mockImplementation(async () => new Response('{}'));
const api = createApi(fetcher);
await api.startBuild('1.2.3');
await api.startBuild('9.9.9', true);
for (const state of ['recommended', 'compatible', 'existing_only', 'blocked'] as AdmissionState[]) {
await api.admitImage('sha256:a/b', state, '终端、文件与配置验收记录 #42');
}
expect(fetcher.mock.calls.map(([url, options]) => [url, options?.method, JSON.parse(options?.body as string)])).toEqual([
['/api/builds', 'POST', {version:'1.2.3', experimental:false}],
['/api/builds', 'POST', {version:'9.9.9', experimental:true}],
...['recommended', 'compatible', 'existing_only', 'blocked'].map(state => ['/api/images/admission', 'POST', {image_id:'sha256:a/b', state, evidence:'终端、文件与配置验收记录 #42'}]),
]);
});
it.each([undefined, 'experimental', 'existing_only', 'blocked', 'unverified', 'missing'])('never selects status %s even with selectable=true', status => {
expect(selectableImage({id:'i', version:'1', status, selectable:true})).toBe(false);
});
it('requires explicit availability and selection permission rather than inferring old-image compatibility', () => {
expect(selectableImage({id:'old', version:'1'})).toBe(false);
expect(selectableImage({id:'i', version:'1', status:'available'})).toBe(false);
expect(selectableImage({id:'i', version:'1', status:'available', selectable:false})).toBe(false);
expect(selectableImage({id:'i', version:'1', status:'available', selectable:true})).toBe(true);
});
+75
View File
@@ -0,0 +1,75 @@
import { readFileSync } from 'node:fs';
import { runInNewContext } from 'node:vm';
import { expect, it, vi } from 'vitest';
const source = readFileSync(new URL('../../internal/api/runtime_session.go', import.meta.url), 'utf8').split('const runtimeSessionJS = `')[1].split('`')[0];
it.each([401, 403, 409, 500])('runtime only reloads a confirmed revoked session (%s)', async status => {
const reload = vi.fn();
let renew: (() => void) | undefined;
const fetch = vi.fn(async (path: string) => new Response('{}', {status: path === '/api/refresh' ? status : 401}));
runInNewContext(source, {
location:{pathname:'/',reload}, navigator:{}, fetch,
document:{cookie:'dsh_csrf=csrf',addEventListener() {}},
window:{addEventListener(_name: string, handler: () => void) {renew = handler;}},
setInterval() {}, setTimeout(resolve: () => void) {resolve();},
});
await renew?.();
expect(reload).toHaveBeenCalledTimes(status === 401 || status === 403 ? 1 : 0);
expect(fetch.mock.calls.filter(c => c[0] === '/api/refresh')).toHaveLength(1);
expect(fetch.mock.calls.filter(c => c[0] === '/api/me')).toHaveLength(status === 500 ? 0 : 4);
});
it.each([false, true])('renews before access expiry, with Web Locks=%s', async withLocks => {
let now = 0, expires = 300_000;
let tick!: () => Promise<void>;
const locks = {request: vi.fn(async (_name: string, run: () => Promise<void>) => run())};
const fetch = vi.fn(async (path: string) => {
if (path === '/api/refresh') expires = now + 300_000;
return new Response('{}', {status: now < expires ? 200 : 401});
});
runInNewContext(source, {
location: {pathname: '/', reload: vi.fn()}, navigator: withLocks ? {locks} : {}, fetch,
document: {cookie: 'dsh_csrf=csrf', addEventListener() {}},
window: {addEventListener() {}},
setInterval(fn: typeof tick, interval: number) {tick = fn; expect(interval).toBe(240_000);},
setTimeout,
});
for (now = 240_000; now <= 960_000; now += 240_000) {
expect(now).toBeLessThan(expires);
await tick();
expect(expires).toBe(now + 300_000);
}
expect(fetch.mock.calls.map(c => c[0])).toEqual(Array(4).fill('/api/refresh'));
expect(locks.request).toHaveBeenCalledTimes(withLocks ? 4 : 0);
});
it('coalesces renewal events and keeps a session recovered by a sibling page', async () => {
let tick!: () => Promise<void>, finish!: (response: Response) => void;
const reload = vi.fn();
const fetch = vi.fn((path: string) => path === '/api/refresh'
? new Promise<Response>(resolve => {finish = resolve;})
: Promise.resolve(new Response('{}', {status: 200})));
runInNewContext(source, {
location: {pathname: '/', reload}, navigator: {}, fetch,
document: {cookie: 'dsh_csrf=csrf', addEventListener() {}},
window: {addEventListener() {}}, setInterval(fn: typeof tick) {tick = fn;}, setTimeout,
});
const first = tick(), second = tick();
expect(fetch).toHaveBeenCalledOnce();
finish(new Response('{}', {status: 409}));
await Promise.all([first, second]);
expect(fetch.mock.calls.map(c => c[0])).toEqual(['/api/refresh', '/api/me']);
expect(reload).not.toHaveBeenCalled();
});
it('reads the latest CSRF cookie after waiting for the refresh lock', async () => {
let tick!: () => Promise<void>, enter!: () => void;
const document = {cookie: 'dsh_csrf=old', addEventListener() {}};
const fetch = vi.fn(async () => new Response('{}'));
const locks = {request: (_name: string, run: () => Promise<void>) => new Promise<void>((resolve, reject) => {enter = () => {run().then(resolve, reject);};})};
runInNewContext(source, {
location: {pathname: '/', reload: vi.fn()}, navigator: {locks}, document, fetch,
window: {addEventListener() {}}, setInterval(fn: typeof tick) {tick = fn;}, setTimeout,
});
const pending = tick();
expect(fetch).not.toHaveBeenCalled();
document.cookie = 'dsh_csrf=new';
enter(); await pending;
expect(fetch).toHaveBeenCalledWith('/api/refresh', expect.objectContaining({headers: {'X-CSRF-Token': 'new'}}));
});
+116
View File
@@ -0,0 +1,116 @@
import { afterEach, beforeEach, expect, it, vi } from 'vitest';
const hooks = vi.hoisted(() => ({slots: [] as any[], cursor: 0, effects: [] as (() => void)[]}));
vi.mock('react', () => ({
useRef(value: unknown) { const i = hooks.cursor++; return hooks.slots[i] ??= {current:value}; },
useState(value: unknown) { const i = hooks.cursor++; if (!(i in hooks.slots)) hooks.slots[i] = value; return [hooks.slots[i], (v: unknown) => { hooks.slots[i] = v; }]; },
useCallback(fn: unknown, deps: unknown[]) {
const i = hooks.cursor++, previous = hooks.slots[i];
if (!previous || deps.some((v, n) => v !== previous.deps[n])) hooks.slots[i] = {deps, fn};
return hooks.slots[i].fn;
},
useEffect(fn: () => (() => void) | undefined, deps: unknown[]) {
const i = hooks.cursor++, previous = hooks.slots[i];
if (!previous || deps.some((v, n) => v !== previous.deps[n])) hooks.effects.push(() => { previous?.cleanup?.(); hooks.slots[i] = {deps, cleanup:fn()}; });
},
}));
import { useResource } from './ui';
const render = <T,>(load: () => Promise<T>, key = '', poll?: (data: T) => boolean, interval = 10, timeout = 60_000) => {
hooks.cursor = 0; const result = useResource(load, key, interval, poll, timeout);
hooks.effects.splice(0).forEach(effect => effect()); return result;
};
beforeEach(() => { hooks.slots = []; hooks.cursor = 0; hooks.effects = []; vi.useFakeTimers(); vi.stubGlobal('window', {setInterval, clearInterval}); });
afterEach(() => { hooks.slots.forEach(s => s?.cleanup?.()); vi.useRealTimers(); vi.unstubAllGlobals(); });
it('accepts slow results without overlapping polling', async () => {
let resolve!: (value: string) => void;
const load = vi.fn(() => new Promise<string>(r => { resolve = r; }));
render(load);
await vi.advanceTimersByTimeAsync(50);
expect(load).toHaveBeenCalledOnce();
resolve('slow result'); await Promise.resolve(); await Promise.resolve();
expect(render(load).data).toBe('slow result');
await vi.advanceTimersByTimeAsync(10); expect(load).toHaveBeenCalledTimes(2);
});
it('ignores an old key result while allowing the new key to load', async () => {
let resolve!: (value: string) => void;
const old = () => new Promise<string>(r => {resolve = r;});
const first = render(old, 'old'); const pending = first.refresh();
const next = vi.fn(async () => 'new result'); render(next, 'new');
await Promise.resolve(); await Promise.resolve();
resolve('stale'); await pending;
expect(render(next, 'new').data).toBe('new result');
});
it('stops polling terminal data but still allows manual refresh', async () => {
const load = vi.fn(async () => 'done'); const poll = () => false;
const resource = render(load, '', poll); await Promise.resolve(); await Promise.resolve();
await vi.advanceTimersByTimeAsync(50); expect(load).toHaveBeenCalledOnce();
await resource.refresh(); expect(load).toHaveBeenCalledTimes(2);
});
it('queues a post-mutation refresh after an old in-flight empty list', async () => {
let resolve!: (value: string[]) => void;
const load = vi.fn<() => Promise<string[]>>().mockImplementationOnce(() => new Promise(r => {resolve = r;})).mockResolvedValue(['new build']);
const resource = render(load, '', () => false);
const refreshed = resource.refresh(); expect(load).toHaveBeenCalledOnce();
resolve([]); await refreshed;
expect(load).toHaveBeenCalledTimes(2);
expect(render(load).data).toEqual(['new build']);
});
it('escapes a hung request so polling continues', async () => {
let hung = true;
const load = vi.fn(() => hung ? new Promise<string>(() => {}) : Promise.resolve('recovered'));
render(load, '', () => false);
expect(load).toHaveBeenCalledOnce();
await vi.advanceTimersByTimeAsync(20);
expect(load).toHaveBeenCalledOnce(); // no overlapping polls while hung
hung = false;
await vi.advanceTimersByTimeAsync(120_000);
expect(load).toHaveBeenCalledTimes(2); // escape re-armed the poller once
expect(render(load).data).toBe('recovered');
});
it('settles a refresh waiting on a hung request and performs the post-mutation read', async () => {
const load = vi.fn<() => Promise<string>>().mockImplementationOnce(() => new Promise(() => {})).mockResolvedValue('updated');
const resource = render(load, '', undefined, 0);
const finished = vi.fn();
const pending = resource.refresh().then(finished);
await vi.advanceTimersByTimeAsync(60_000);
await pending;
expect(finished).toHaveBeenCalledOnce();
expect(load).toHaveBeenCalledTimes(2);
expect(render(load, '', undefined, 0)).toMatchObject({data: 'updated', loading: false, error: ''});
});
it('releases refresh waiters even when the follow-up request also hangs', async () => {
const load = vi.fn(() => new Promise<string>(() => {}));
const resource = render(load, '', undefined, 0);
const finished = vi.fn();
const pending = resource.refresh().then(finished);
await vi.advanceTimersByTimeAsync(120_000);
await pending;
expect(finished).toHaveBeenCalledOnce();
expect(load).toHaveBeenCalledTimes(2);
expect(render(load, '', undefined, 0)).toMatchObject({loading: false, error: '请求超时,请重试。'});
});
it.each(['resolve', 'reject'])('shows a timeout without polling and ignores late %s', async outcome => {
let resolve!: (value: string) => void, reject!: (error: Error) => void;
const load = vi.fn<() => Promise<string>>().mockImplementationOnce(() => new Promise((yes, no) => {resolve = yes; reject = no;})).mockResolvedValue('fresh');
render(load, '', undefined, 0);
await vi.advanceTimersByTimeAsync(60_000);
let resource = render(load, '', undefined, 0);
expect(resource).toMatchObject({loading: false, error: '请求超时,请重试。', data: undefined});
await resource.refresh();
if (outcome === 'resolve') resolve('stale'); else reject(new Error('late failure'));
await vi.advanceTimersByTimeAsync(0);
resource = render(load, '', undefined, 0);
expect(resource).toMatchObject({loading: false, error: '', data: 'fresh'});
});
it('uses a changed timeout and isolates the previous load', async () => {
let resolve!: (value: string) => void;
const load = vi.fn<() => Promise<string>>().mockImplementationOnce(() => new Promise(r => {resolve = r;})).mockImplementationOnce(() => new Promise(() => {})).mockResolvedValue('fresh');
render(load, '', undefined, 0);
render(load, '', undefined, 0, 100);
await vi.advanceTimersByTimeAsync(100);
const resource = render(load, '', undefined, 0, 100);
expect(resource).toMatchObject({loading: false, error: '请求超时,请重试。'});
resolve('stale');
await resource.refresh();
expect(load).toHaveBeenCalledTimes(3);
expect(render(load, '', undefined, 0, 100)).toMatchObject({data: 'fresh', error: '', loading: false});
});
+45 -8
View File
@@ -2,20 +2,57 @@ import { useCallback, useEffect, useRef, useState, type ReactNode } from 'react'
export const message = (error: unknown) => error instanceof Error ? error.message : '操作失败,请重试。'; export const message = (error: unknown) => error instanceof Error ? error.message : '操作失败,请重试。';
export const bytes = (n?: number) => n === undefined ? '—' : n < 1024 ? n + ' B' : n < 1048576 ? (n / 1024).toFixed(1) + ' KiB' : n < 1073741824 ? (n / 1048576).toFixed(1) + ' MiB' : (n / 1073741824).toFixed(2) + ' GiB'; export const bytes = (n?: number) => n === undefined ? '—' : n < 1024 ? n + ' B' : n < 1048576 ? (n / 1024).toFixed(1) + ' KiB' : n < 1073741824 ? (n / 1048576).toFixed(1) + ' MiB' : (n / 1073741824).toFixed(2) + ' GiB';
export const date = (s?: string) => s ? new Date(s).toLocaleString('zh-CN') : '—'; export const date = (s?: string) => s ? new Date(s).toLocaleString('zh-CN') : '—';
export function useResource<T>(load: () => Promise<T>, key = '', interval = 0) { export function useResource<T>(load: () => Promise<T>, key = '', interval = 0, shouldPoll: (data: T) => boolean = () => true, timeoutMs = 60_000) {
const loader = useRef(load); loader.current = load; const loader = useRef(load); loader.current = load;
const polling = useRef(shouldPoll); polling.current = shouldPoll;
const latest = useRef<T>(undefined);
const [data, setData] = useState<T>(); const [data, setData] = useState<T>();
const [error, setError] = useState(''); const [error, setError] = useState('');
const [loading, setLoading] = useState(true); const [loading, setLoading] = useState(true);
const generation = useRef(0); const generation = useRef(0);
const refresh = useCallback(async () => { const inFlight = useRef<Promise<void> | undefined>(undefined);
const current = ++generation.current; const loadSeq = useRef(0);
const loadNow = useCallback(() => {
if (inFlight.current) return inFlight.current;
const current = generation.current;
const seq = ++loadSeq.current;
setLoading(true); setError(''); setLoading(true); setError('');
try { const value = await loader.current(); if (current === generation.current) setData(value); } // Bound the promise that refresh callers await, not just its reference.
catch (e) { if (current === generation.current) setError(message(e)); } // Only the winning result may update state; a timed-out loader can still
finally { if (current === generation.current) setLoading(false); } // settle later, but cannot overwrite data or errors from a newer load.
}, []); const record: {promise?: Promise<void>, escape?: ReturnType<typeof setTimeout>} = {};
useEffect(() => { setData(undefined); void refresh(); const timer = interval ? window.setInterval(refresh, interval) : undefined; return () => { generation.current++; window.clearInterval(timer); }; }, [refresh, key, interval]); const pending = (async () => {
try {
const timeout = new Promise<never>((_, reject) => {
record.escape = setTimeout(() => reject(new Error('请求超时,请重试。')), timeoutMs);
});
const value = await Promise.race([loader.current(), timeout]);
if (current === generation.current && seq === loadSeq.current) { latest.current = value; setData(value); }
}
catch (e) { if (current === generation.current && seq === loadSeq.current) setError(message(e)); }
finally { if (record.escape !== undefined) clearTimeout(record.escape); if (current === generation.current && seq === loadSeq.current) setLoading(false); if (inFlight.current === record.promise) inFlight.current = undefined; }
})();
record.promise = pending;
inFlight.current = pending;
void pending.then(() => { if (inFlight.current === pending) inFlight.current = undefined; });
return pending;
}, [timeoutMs]);
const refresh = useCallback(async () => {
const current = generation.current;
if (inFlight.current) await inFlight.current;
if (current === generation.current) await loadNow();
}, [loadNow]);
useEffect(() => {
setData(undefined); latest.current = undefined; void loadNow();
return () => { generation.current++; inFlight.current = undefined; };
}, [loadNow, key]);
useEffect(() => {
if (!interval) return;
const timer = window.setInterval(() => {
if (latest.current === undefined || polling.current(latest.current)) void loadNow();
}, interval);
return () => window.clearInterval(timer);
}, [loadNow, key, interval]);
return {data, error, loading, refresh}; return {data, error, loading, refresh};
} }
export function Alert({children}: {children: ReactNode}) { return <div className="alert" role="alert">{children}</div>; } export function Alert({children}: {children: ReactNode}) { return <div className="alert" role="alert">{children}</div>; }
+27 -1
View File
@@ -1,3 +1,29 @@
import { defineConfig } from "vite"; import { defineConfig } from "vite";
import react from "@vitejs/plugin-react"; import react from "@vitejs/plugin-react";
export default defineConfig({ plugins: [react()], server: { host: "127.0.0.1", proxy: { "/api": "http://127.0.0.1:8088", "/dsh": { target: "http://127.0.0.1:8088", ws: true } } } }); export default defineConfig({
plugins: [react()],
build: {
rollupOptions: {
output: {
// Keep shared Vite helpers out of Monaco chunks so the entry stays lazy.
onlyExplicitManualChunks: true,
manualChunks(id) {
const path = id.split("/monaco-editor/esm/vs/")[1];
if (!path || path.includes("?worker")) return;
const parts = path.split("/");
if (parts[0] === "base") return `monaco-base-${parts[1]}`;
if (path.startsWith("editor/common/core/")) return "monaco-editor-core";
if (path.startsWith("editor/common/diff/")) return "monaco-editor-diff";
if (path.startsWith("editor/common/config/")) return "monaco-editor-config";
if (path.startsWith("editor/common/viewLayout/") || path.startsWith("editor/common/viewModel/")) return "monaco-editor-view-model";
if (path.startsWith("editor/browser/config/")) return "monaco-editor-browser-config";
// Multi-diff imports diff widgets; splitting them creates a chunk cycle.
if (path.startsWith("editor/browser/widget/diffEditor/") || path.startsWith("editor/browser/widget/multiDiffEditor/")) return "monaco-editor-diff-widget";
if (parts[0] === "editor") return `monaco-editor-${parts[1]}`;
return `monaco-${parts[0]}`;
},
},
},
},
server: { host: "127.0.0.1", proxy: { "/api": "http://127.0.0.1:8088", "/dsh": { target: "http://127.0.0.1:8088", ws: true } } },
});