Compare commits
2
Commits
9a71772771
...
a2dd6ee27d
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a2dd6ee27d | ||
|
|
20635ffaec |
@@ -10,4 +10,3 @@ output
|
||||
**/*.tsbuildinfo
|
||||
.playwright-cli
|
||||
**/*.sqlite*
|
||||
image/*.test.mjs
|
||||
|
||||
@@ -19,3 +19,6 @@ UPLOAD_ALLOWED_EXTENSIONS=
|
||||
PANEL_PUBLIC_URL=http://127.0.0.1:8088
|
||||
# Optional proxy networks. X-Forwarded-For is ignored unless the peer is in these CIDRs.
|
||||
TRUSTED_PROXY_CIDRS=
|
||||
# Trusted executable runtime build source. Do not embed credentials in this URL.
|
||||
DSH_RELEASE_REPOSITORY=https://git.misaka.ren/m1saka/dsh-web.git
|
||||
DSH_RELEASE_BRANCH=dockerfile
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# DESIGN.md 验收记录
|
||||
|
||||
> 发布架构变更说明:下文为各阶段的历史验收证据,保留当时命令、测试数量及结论,不作为当前操作指南。main 已移除内置 runtime 构建资产;`release.json`、完整 `image/` 和初始化测试仅在 `dockerfile` 发布分支维护,新构建仅允许管理员通过 Git 发布检查并确认精确 commit。旧 npm/实验构建及构建后创建预约流程已被取代,这是 breaking change:移除 `/api/latest`、`/api/compatibility`、`POST /api/builds` 及 `/api/intents` 的全部查询、创建和取消接口,不提供 410 兼容入口。仅保留 `GET /api/builds` 和 `GET /api/builds/{id}` 构建历史读取。不迁移或恢复旧构建及旧创建意图,不在后台续跑或完成旧预约;这不意味着删除已有实例或数据。旧 `scripts/build-image.sh` 与 `scripts/test-intent-live.mjs` 已删除;`scripts/test-panel.mjs` 现在复用预先验证且准入的本地镜像。历史通过记录不等于新 Git 发布架构已通过真实部署验收,当前使用方式见 README.md、API.md、OPERATIONS.md。
|
||||
|
||||
## 恢复宽限期与轮询缓存修复(独立复审后)
|
||||
|
||||
范围:本轮为对上述工作树的独立代码审计与全量重测后的修复。审计确认八项修复中六项正确,发现三处残留:创建结果不确定的挂起标志永不过期导致实例永久无法删除(高)、构建校验不确定行无限重试并持续新增审计(中)、意图轮询每秒对每个成功镜像发起 Docker inspect(中)。修复未改变凭据文件管理边界或准入策略;未部署、未操作真实 Docker 实例。
|
||||
|
||||
@@ -41,27 +41,29 @@ GET /api/orphans 返回可回收的完整容器ID数组。DELETE /api/orphans?id
|
||||
- GET /api/audit(管理员)
|
||||
|
||||
## 镜像与构建
|
||||
|
||||
这是 breaking change:`/api/latest`、`/api/compatibility`、`POST /api/builds` 及 `/api/intents`(包括子路径的查询、创建、取消)已移除,不提供 410 兼容入口。`GET /api/builds` 与 `GET /api/builds/{id}` 仅用于构建历史读取。旧构建及旧创建意图不迁移、不恢复执行,不在后台续跑或完成旧预约;不因此删除已有实例或数据。管理员改用 Git 发布接口构建,用户以已验证且准入的精确本地镜像 ID 创建实例。
|
||||
|
||||
已认证请求中,`POST /api/builds` 返回 405;`/api/latest`、`/api/compatibility` 和 `/api/intents`(含子路径)返回 404。历史意图记录留在数据库中,但不再提供存储操作、轮询或恢复流程,也不计入配额或作为镜像删除引用。
|
||||
|
||||
- GET /api/releases/latest(仅管理员;获取并校验配置的 Git 发布分支,返回 release_id、version、commit、repository、branch、recipe_rev、platforms、compatible、reason,以及可选的 build。检查会下载构建文件,但不执行 Dockerfile;未知部署协议或平台不允许构建)
|
||||
- POST /api/releases/build {commit}(仅管理员、同源和 CSRF;提交检查时看到的精确 commit,分支已变化返回 409,需重新检查。返回 202 构建任务;不自动升级实例)
|
||||
- Git 构建任务附带 release_id、source_commit、source_repository 来源信息;发布身份独立于 DSH 版本及部署布局,允许同一 DSH 版本构建新的脚本修订。可信发布通过现有镜像身份和启动验证后可准入,显式禁用仍优先。新构建仅允许管理员通过 Git 发布接口提交;main 不再包含内置 runtime 构建资产。
|
||||
- GET /api/images(每项含 admission、admission_reason、selectable;只有 status=available 且 selectable=true 才可新建/升级)
|
||||
- GET /api/compatibility(登录后可读;返回 recipe_rev、platform、entries[{version,recipe_rev,platform,state,build_allowed,reason}]、unknown{state,build_allowed,reason}、experimental_admin_only。此为普通版本构建清单,不等于每个实际镜像的准入结果)
|
||||
- POST /api/images/admission {image_id,state,evidence}(管理员、同源和CSRF;state为recommended/compatible/existing_only/blocked,evidence必填。正向准入要求精确镜像身份、成功验证记录、受支持挂载布局与平台;策略与审计持久化。单镜像准入不自动扩大普通版本构建白名单)
|
||||
- POST /api/images/admission {image_id,state,evidence}(管理员、同源和CSRF;state为recommended/compatible/existing_only/blocked,evidence必填。正向准入要求精确镜像身份、成功验证记录、受支持挂载布局与平台;策略与审计持久化。单镜像准入不授权新构建)
|
||||
- DELETE /api/images?image_id=sha256:...(管理员;保护实例、活动任务、升级及全部Docker容器引用,不force)
|
||||
- GET /api/latest(查询 npm dist-tags.latest;附带 supported、build_allowed、admission、admission_reason,最新版不一定受支持)
|
||||
- GET /api/builds?limit=20&offset=0(摘要数组,不含日志正文;limit默认20、最大100,offset非负;X-Has-More响应头表示后续是否有记录)
|
||||
- GET /api/builds/{id}(按需获取单个任务及脱敏日志;仅已登录用户,日志最多64 KiB)
|
||||
- POST /api/builds {version,experimental?}(固定版本,异步;experimental=true仅管理员可用。未知版本普通构建会预先拒绝;实验仍执行完整性、依赖和启动验证,success不自动准入。返回任务含experimental标记。版本/模板/平台去重,新任务每用户每小时最多3个,共享复用不扣次数)
|
||||
|
||||
镜像策略:recommended/compatible可在实际验证与身份检查通过后用于新建/升级;existing_only允许已有实例运行和启停,不能新建/作为升级目标;blocked还拒绝新的start/restart及受控配置非GET操作(含Profile保存、备份和恢复),但不会自动停止已运行容器或断开工作台,通用文件管理仍允许。experimental是成功实验镜像未准入时的展示状态,不是兼容性结论。missing/unverified不可选。
|
||||
|
||||
内置普通构建清单为 linux/amd64、模板7、0.2.0-rc.1(compatible)和0.2.0-rc.2(recommended)。其他平台/版本不继承这些验收结论。模板3–6的历史成功镜像默认仅存量,具有精确验证记录并经管理员验收可准入;未知模板布局不可正向准入,模板7向旧共享布局降级仍拒绝。
|
||||
历史兼容基线为 linux/amd64、模板7、0.2.0-rc.1(compatible)和0.2.0-rc.2(recommended)。其他平台/版本不继承这些验收结论。模板3–6的历史成功镜像默认仅存量,具有精确验证记录并经管理员验收可准入;未知模板布局不可正向准入,模板7向旧共享布局降级仍拒绝。
|
||||
|
||||
## 实例
|
||||
- GET /api/instances → Instance[];字段 id,uid,name,version,image_id,status,container_id,created_at
|
||||
- POST /api/instances {name,image_id,version} → Instance(每用户最多 2 个)
|
||||
- POST /api/instances {name,image_id,version} → Instance(每用户最多 2 个;image_id 必须指向已验证且准入的精确本地镜像,不接受构建预约)
|
||||
- POST /api/instances/{id}/start|stop|restart
|
||||
- DELETE /api/instances/{id} → {ok:true}
|
||||
- POST /api/intents {name,version}(固定精确版本,持久预留构建后创建名额)
|
||||
- GET /api/intents(当前用户意图)
|
||||
- DELETE /api/intents/{id}(仅取消等待,不取消共享构建)
|
||||
|
||||
所有成功 JSON 列表均为顶层数组(不是 data 包装);错误均为 {error:string}。实现与验收仍在进行,不应仅凭接口列在此处认定已验收。
|
||||
- GET /api/instances/{id}/logs
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# DeepSeek Harness 多租户容器管理平台 — 设计文档
|
||||
|
||||
> 发布架构变更(取代下文旧构建方案):新运行镜像仅由管理员通过 Git 发布检查与精确 commit 确认构建。`release.json`、完整 `image/` 及 runtime 初始化测试仅在 `dockerfile` 发布分支维护,main 不再内置或打包运行镜像资产。下文 npm latest 查询、内置构建上下文、实验构建、构建后创建预约及目录示意中的 `image/` 均为历史设计,不再是操作指南。这是 breaking change:移除 `/api/latest`、`/api/compatibility`、`POST /api/builds` 及 `/api/intents` 的全部查询、创建和取消接口,不提供 410 兼容入口。仅保留 `GET /api/builds` 和 `GET /api/builds/{id}` 构建历史读取。不迁移或恢复旧构建及旧创建意图,不在后台续跑或完成旧预约;这不意味着删除已有实例或数据。当前流程以 README.md、API.md、OPERATIONS.md 为准;以下设计与历史证据保留,不表示新架构已完成相同真实环境验收。
|
||||
|
||||
> 状态:本机管理面板核心流程已实现并通过真实 Docker 集成检查;生产隔离、硬配额等仍有差距,以 README.md 的实现边界与验证范围为准。下文是目标设计,不代表所有条目已实现。
|
||||
> 目标项目:`deepseek-ai/deepseek-harness`(npm CLI `@deepseek-ai/dsh`,命令 `dsh web`)
|
||||
> 控制面技术栈:Go + 前端 SPA
|
||||
|
||||
+1
-3
@@ -11,15 +11,13 @@ COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
COPY cmd/ ./cmd/
|
||||
COPY internal/ ./internal/
|
||||
COPY image/ ./image/
|
||||
RUN CGO_ENABLED=0 go build -trimpath -o /out/dsh-mgr ./cmd/server
|
||||
|
||||
FROM debian:bookworm-slim
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates && rm -rf /var/lib/apt/lists/*
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates git && rm -rf /var/lib/apt/lists/*
|
||||
WORKDIR /app
|
||||
COPY --from=backend /out/dsh-mgr /usr/local/bin/dsh-mgr
|
||||
COPY --from=web /src/web/dist ./web/dist
|
||||
COPY image/ ./image/
|
||||
USER 1000:1000
|
||||
EXPOSE 8088
|
||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 CMD ["/usr/local/bin/dsh-mgr", "-healthcheck"]
|
||||
|
||||
+34
-4
@@ -6,7 +6,7 @@
|
||||
|
||||
1. 复制 `.env.example` 为 `.env`,填写专用绝对目录 `DSH_HOST_DATA_ROOT` 和 12–72 字节的随机管理员密码。限制 `.env` 的读取权限,不要提交。管理员账号创建后该密码不再被读取,应从 `.env` 中移除,再运行 `docker compose up -d --force-recreate panel` 重建面板容器,清除容器环境变量中保留的明文密码(`docker inspect` 可见)。仅修改 `.env` 或重启原容器不会更新其环境变量;重建面板会短暂中断面板访问。
|
||||
2. 面板以 UID/GID 1000 运行。首次部署前由管理员创建专用数据目录并赋予 1000:1000 所有权、0700 权限;不要对已有未知目录递归改权限。将 `stat -c %g /var/run/docker.sock` 的结果填入 `.env` 的 `DOCKER_GID`,以附加组访问 socket。然后运行 `docker compose up --build -d`。默认访问 `http://127.0.0.1:8088`;远程管理优先使用 SSH 本地端口转发,保留此访问地址。Compose 定义了 `dsh-mgr -healthcheck` 健康检查(探测本机 `/healthz`);进程无响应时容器会被标记为 unhealthy,配合 `restart: unless-stopped` 在退出时自动拉起。
|
||||
3. 登录后在镜像页提交精确 DSH 版本或查询最新版再手动构建。已有本地镜像会复用,但必须通过启动验证才允许创建实例。镜像不推送至仓库。
|
||||
3. 管理员登录后在镜像页检查 Git 发布并确认精确 commit 后构建;普通用户选择已验证且准入的本地镜像创建实例。没有可选镜像时等待管理员构建,不再按 npm 版本或预约构建后创建。镜像不推送至仓库。
|
||||
4. 创建实例后从面板进入 DSH。不要给 runtime 发布宿主端口或挂载 Docker socket。
|
||||
|
||||
首次启动创建管理员。修改环境变量不会重设已有账号密码。容器内 `/data` 必须对应 `HOST_DATA_ROOT` 指向的同一宿主目录,否则 sibling 容器会挂载错误数据。
|
||||
@@ -35,7 +35,7 @@
|
||||
|
||||
### 镜像兼容验收与准入
|
||||
|
||||
管理员实验构建只负责生成并执行基础验证,不代表上线批准。准入前应在独立测试环境核验精确镜像的启动、认证/代理、终端与流式连接、文件读写、插件协调、配置备份恢复,以及所需升级/回滚路径;记录版本、模板、平台、镜像ID、测试范围和证据位置。不要在验收依据中粘贴凭据,也不要仅填写“构建成功”。本次实现不自动启动待验收镜像供普通用户使用,不自动运行真实部署测试。
|
||||
管理员 Git 发布构建会执行基础验证,但不能代替完整上线验收。准入前应在独立测试环境核验精确镜像的启动、认证/代理、终端与流式连接、文件读写、插件协调、配置备份恢复,以及所需升级/回滚路径;记录版本、模板、平台、镜像ID、测试范围和证据位置。不要在验收依据中粘贴凭据,也不要仅填写“构建成功”。本次实现不自动启动待验收镜像供普通用户使用,不自动运行真实部署测试。
|
||||
|
||||
在镜像页“准入管理”选择推荐或兼容并填写依据,后端还会检查精确镜像身份、成功验证记录和可支持的挂载布局。审批只作用于该镜像;新镜像需要自己的验证与审批。退为仅存量后不允许新建/升级目标;标为禁止使用还阻止新的启动/重启及受控配置修改、备份/恢复写操作,避免其自动重启绕过禁用。已有配置读取和通用home文件管理仍保留,不会自动停止正在运行的实例。若需要立即隔离已运行实例,管理员必须另行确认并执行停止操作。
|
||||
|
||||
@@ -53,6 +53,36 @@
|
||||
- 人工恢复前停止相关容器并另做完整备份。中断点不同,不能统一用一次目录覆盖或状态修改处理;当前无自动灾难恢复命令。
|
||||
- 删除实例会删除其容器与网络,但保留数据目录。升级快照暂无自动清理;不要删除仍被未完成 journal 引用的快照。
|
||||
|
||||
## Git 分支发布运行镜像构建配置
|
||||
|
||||
发布源默认为本仓库 `https://git.misaka.ren/m1saka/dsh-web.git` 的 `dockerfile` 分支,通过 `DSH_RELEASE_REPOSITORY` / `DSH_RELEASE_BRANCH` 配置。它发布的是构建文件,不是预构建镜像,也不是面板程序自身的更新。修改配置后需重新创建面板容器。
|
||||
|
||||
直接运行面板二进制时需要安装 `git`;面板 Dockerfile 已包含该依赖。当前通道仅支持无需认证的 HTTPS 仓库,不读取宿主 Git 凭据、不跟随重定向,也不支持 SSH 或 URL 内嵌令牌。私有仓库认证需另行扩展,不能把访问令牌写入仓库地址。
|
||||
|
||||
`release.json` 和完整的 `image/` 目录仅在 `dockerfile` 发布分支维护,main 不再保留或打包这些文件。发布分支根目录放 `release.json`;发布时一起提交 Dockerfile、Caddyfile、entrypoint 和全部安装/适配脚本。只把 Dockerfile 单独放到分支上是不完整的发布。不要把 `.env`、私钥、凭据、用户数据或 `.git` 放进构建目录;`.dockerignore` 只是辅助措施,发布者仍须审查内容。
|
||||
|
||||
Git 通道直接向 Docker API 发送构建上下文,不执行 Docker CLI 的 `.dockerignore` 过滤。面板会排除 `.git`、`.env` 系列和常见凭据文件,并拒绝链接、特殊文件及越界路径,但不能识别任意文件名下的秘密。构建归档最大 32 MiB;Git 获取有 45 秒超时及文件大小限制,建议发布分支仅保留发布所需文件。
|
||||
|
||||
`release.json` 示例:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"release_id": "runtime-7.1",
|
||||
"version": "0.2.0-rc.2",
|
||||
"deployment_schema": 7,
|
||||
"platforms": ["linux/amd64"]
|
||||
}
|
||||
```
|
||||
|
||||
`version` 是精确 DSH 版本;`release_id` 是构建配置发布编号,修改脚本或 Dockerfile 后即使 DSH 版本不变,也应使用新的编号。`deployment_schema` 声明运行布局兼容协议,当前 Git 发布通道支持协议 7:实例 home 挂载 `/workspace`、共享目录 `/shared`、代理端口 8080,并保留现有认证与启动约定。它不应随普通构建修订递增;不兼容的运行约定需要面板新增支持。`platforms` 只能声明发布者实际验证的平台。
|
||||
|
||||
首次发布时,在独立 checkout 或 worktree 准备 `dockerfile` 分支,将 `release.json` 和完整 `image/` 目录提交并推送到同一远程仓库。无需切换正在开发的主工作目录,也无需 CI。此功能的代码安装本身不会替你创建或推送远程分支;分支未发布前,检查失败是预期状态,旧实例不受影响,但不会回退到内置构建。这是 breaking change:移除 `/api/latest`、`/api/compatibility`、`POST /api/builds` 及 `/api/intents` 的全部查询、创建和取消接口,不提供 410 兼容入口。仅保留 `GET /api/builds` 和 `GET /api/builds/{id}` 构建历史读取。不迁移或恢复旧构建及旧创建意图,不在后台续跑或完成旧预约;这不意味着删除已有实例或数据。
|
||||
|
||||
管理员打开镜像页会检查 Git 发布,页面保持打开时每五分钟复查,也可手动刷新。确认构建后面板以固定 commit 获取文件;确认期间分支发生变化会拒绝并要求重新检查。构建任务记录发布编号、仓库和 commit,同一 DSH 版本的新构建发布不会误用旧版本构建任务。构建与验证成功后才可选择镜像;不会自动升级任何已有实例,升级仍走密码确认、备份、快照和失败回滚流程。
|
||||
|
||||
该分支是可执行代码信任边界:限制写权限,发布前审查 Dockerfile 与脚本,不向不可信仓库开放构建。可信发布只替代内置版本名单,不替代完整的上线验收;应在发布前验证启动、代理/认证、文件访问、插件以及升级恢复。管理员对精确镜像设置的禁用策略仍然有效。上游依赖、基础镜像和网络变化可能影响重建结果,commit 固定并不意味着所有外部依赖都已固定。Git 故障不会停止已有实例。
|
||||
|
||||
## 创建与构建恢复记录
|
||||
|
||||
创建失败不代表 Docker 没有创建资源。面板在发请求前记录 `runtime_creates`(实例 ID、容器/网络结果待确认标志及其置位时间)及 `image_build_checks`(构建 ID、镜像 ID、已确认的验证容器 ID、记录创建时间),已确认网络身份保存在 `runtime_networks`。资源可见后按身份校验再清理;已记录 ID 时按 ID 查询,不能把名称消失当成资源删除。身份不一致时拒绝删除。构建验证残留在启动时及后台重试回收,无需新建构建任务触发。
|
||||
@@ -82,6 +112,6 @@
|
||||
bash scripts/check-trial.sh
|
||||
```
|
||||
|
||||
该脚本执行本地自动化测试、race、静态检查及前后端构建,不启动容器、不安装依赖、不部署。需提前准备 Go/Node/npm 和前端依赖。可通过 `TEST_TMPDIR` 指定已存在的临时目录。镜像构建单测已隔离真实磁盘空闲量;生产磁盘检查仍保留。
|
||||
该脚本执行本地自动化测试、race、静态检查及前后端构建,不启动容器、不安装依赖、不部署。需提前准备 Go/Node/npm 和前端依赖。可通过 `TEST_TMPDIR` 指定已存在的临时目录。面板构建单测已隔离真实磁盘空闲量;生产磁盘检查仍保留。`image/init-profile.test.mjs` 属于发布分支的 runtime 测试,不在 main 的检查脚本中运行。
|
||||
|
||||
`scripts/test-panel.mjs` 必须指向独立测试部署,通过 `TEST_PANEL_URL`、`TEST_USERNAME`、`TEST_PASSWORD` 指定访问参数;`TEST_UPGRADE=1` 包含同镜像替换升级。它会创建及删除测试实例、留下保留策略要求的数据,并可能构建或复用本地 runtime 镜像。不要对生产账号运行。
|
||||
`scripts/test-panel.mjs` 必须指向独立测试部署,通过 `TEST_PANEL_URL`、`TEST_USERNAME`、`TEST_PASSWORD` 指定访问参数;`TEST_UPGRADE=1` 包含同镜像替换升级。运行前需由管理员通过 Git 发布构建准备已验证且准入的本地 runtime 镜像;可用 `TEST_FROM_VERSION` / `TEST_TO_VERSION` 选择源和目标版本。脚本不提交构建,只复用这些镜像,会创建及删除测试实例并留下保留策略要求的数据。不要对生产账号运行。`scripts/smoke-image.sh <image-id>` 和 `scripts/test-mobile-image.mjs <image-id>` 仅测试已有镜像的运行行为,不读取 main 的构建源;仅在隔离测试环境使用。旧构建脚本和构建后创建意图的 live 测试已删除。
|
||||
|
||||
@@ -12,10 +12,12 @@
|
||||
|
||||
## 当前实现
|
||||
|
||||
DSH 新运行镜像仅由管理员从本仓库 `dockerfile` 分支获取独立发布的构建配置,无需 CI 预构建镜像。`release.json` 和完整的 `image/` 目录仅维护在发布分支,main 不再内置运行镜像构建资产;面板锁定 Git commit,在本地构建并验证。管理员在镜像页检查并确认构建,实例升级仍需另行确认,不会因分支更新自动停机。发布约定和首次启用步骤见 [OPERATIONS.md](OPERATIONS.md)。
|
||||
|
||||
实例所有者可管理完整 home,包括读取自己的配置和凭据;不承诺密钥 write-only,跨用户隔离保持不变。通用文件操作不受专用 Profile 编辑器的 YAML 校验和停机要求约束。下载、打包与敏感修改提示用于防误操作,不是权限限制;文件下载和目录 ZIP 不会自动加密,修改配置前建议先停止实例并备份。
|
||||
|
||||
- 内置账号、管理员配额、短JWT与refresh轮换、失败锁定及审计。
|
||||
- 精确版本查询、本地镜像异步构建/验证/清理、持久创建预约与事务名额(每用户最多两个实例)、启动对账。预约不再仅保存在当前页面。
|
||||
- 管理员 Git 发布检查、本地镜像异步构建/验证/清理、使用已验证且准入的本地镜像创建实例与事务名额(每用户最多两个实例)、启动对账。
|
||||
- 实例启停删除、独立home(模板7挂载为 `/workspace`)、每实例bridge、归属鉴权及路径反代;DSH认证cookie由服务端保管。
|
||||
- 文件上传/下载/重命名/删除及目录ZIP;软配额写前检查、写入串行化与周期巡检。上传最大32 MiB且不覆盖已有文件,删除目录须先清空;删除实例保留数据。
|
||||
- 密码加密配置备份、资源目录与manifest、外部加密包导入、恢复健康检查及回滚。备份不再限于六个固定文件;不包含会话、工作区及node_modules。
|
||||
@@ -168,11 +170,11 @@ sudo systemctl reload caddy
|
||||
|
||||
## 本地runtime构建
|
||||
|
||||
### 兼容清单与实验构建
|
||||
### Git 发布与镜像准入
|
||||
|
||||
镜像的构建验证与使用准入分开管理:推荐/兼容镜像还必须通过精确身份与实际验证检查才可选择;仅存量镜像允许原实例继续运行和启停,不可新建或作为升级目标;禁止使用镜像还拒绝新的启动/重启,但不会自动停止现有容器或断开工作台。
|
||||
|
||||
未知版本仅管理员可在镜像页显式提交实验构建。实验保留镜像模板的完整性、依赖和启动检查,可能因不兼容而失败;不会为让新版通过而跳过检查。实验成功后仍不可创建实例,管理员需完成独立兼容验收,再在“准入管理”填写可追溯依据并确认。准入绑定精确镜像ID、版本、模板和平台,不自动信任同tag的新镜像,也不扩大普通版本构建白名单;已准入的未知版本请通过本地镜像选择创建。
|
||||
管理员在镜像页检查 Git 发布并确认精确 commit 后构建。普通用户只能选择已验证且准入的本地镜像创建实例;没有可选镜像时需等待管理员发布与构建,不再提供 npm 版本、实验构建或构建后自动创建入口。这是 breaking change:移除 `/api/latest`、`/api/compatibility`、`POST /api/builds` 及 `/api/intents` 的全部查询、创建和取消接口,不提供 410 兼容入口。仅保留 `GET /api/builds` 和 `GET /api/builds/{id}` 构建历史读取。不迁移或恢复旧构建及旧创建意图,不在后台续跑或完成旧预约;这不意味着删除已有实例或数据。准入绑定精确镜像ID、版本、模板和平台,不自动信任同tag的新镜像。
|
||||
|
||||
历史模板按其自身验证记录与挂载布局判断,不因模板号不是当前版本就标为缺失。有成功验证记录、布局受支持的旧镜像可经管理员验收准入;无法识别的布局不会仅凭一段说明放行。界面展示实际准入原因,以后端校验为准。
|
||||
|
||||
@@ -186,18 +188,16 @@ Free Search默认引擎为Bing,注册的统一provider ID为ddg;未设置或
|
||||
|
||||
依赖固定引用 `file:/usr/local/lib/dsh-mgr/dsh-web-mobile-3.0.5.tgz`,首次安装与备份恢复无需下载该包。备份包含依赖和bundle清单,恢复目标必须提供相同tarball路径(模板4及兼容后续镜像);初始化会在此依赖存在且workspace缺失时补建官方no-auto-peers配置,保留已有workspace及禁用状态。
|
||||
|
||||
需要Docker、Bash、npm及基础镜像/npm网络访问;不推送镜像。
|
||||
运行镜像的构建源与初始化单测在 `dockerfile` 分支维护;main 只构建面板,不再提供 `scripts/build-image.sh`。镜像构建仍需要基础镜像与依赖网络访问,不推送镜像。
|
||||
|
||||
注意:dsh-web-mobile还包含响应压缩、移动会话删除入口和自定义模型推理档位补全,并非纯CSS。内置版本随镜像固定,不跟踪GitHub main或npm latest。
|
||||
模板4目前限定DSH 0.2.0-rc.1/rc.2,并校验已安装宿主的Cordis/DSH peer版本;其他版本需先更新兼容验证,不会自动安装第二套宿主依赖。插件没有独立参数配置页;它修改DSH原有移动界面与设置页。
|
||||
|
||||
```sh
|
||||
bash scripts/build-image.sh # 查询latest并锁定精确版本
|
||||
bash scripts/build-image.sh 0.2.0-rc.2 # 或指定精确版本
|
||||
bash scripts/smoke-image.sh dsh-web:0.2.0-rc.2-7
|
||||
bash scripts/smoke-image.sh "$RUNTIME_IMAGE_ID" # 已由管理员构建的本地镜像 ID
|
||||
```
|
||||
|
||||
当前tag为 `dsh-web:<version>-7`(模板修订7)。runtime以UID/GID 1000运行,home和workspace需允许该身份读写。DSH监听loopback:3080,Caddy监听容器内8080;不要发布runtime端口或挂载socket。原生模块需要可执行的 `/tmp`,保留nosuid/nodev和容量限制。存在插件清单时先协调插件,失败不启动应用;真实验证范围见 ACCEPTANCE.md。
|
||||
`scripts/smoke-image.sh` 只检查已有镜像运行,不读取源码、不构建镜像;需 Docker 和 Bash,会创建并清理临时容器。请从镜像页取得精确镜像 ID,不依赖旧版本 tag 推断 Git 发布身份。当前部署协议为7。runtime以UID/GID 1000运行,home和workspace需允许该身份读写。DSH监听loopback:3080,Caddy监听容器内8080;不要发布runtime端口或挂载socket。原生模块需要可执行的 `/tmp`,保留nosuid/nodev和容量限制。存在插件清单时先协调插件,失败不启动应用;真实验证范围见 ACCEPTANCE.md。
|
||||
|
||||
## 设计替代与边界
|
||||
|
||||
|
||||
@@ -125,9 +125,6 @@ func run() error {
|
||||
srv := &http.Server{Addr: getenv("LISTEN_ADDR", "127.0.0.1:8088"), Handler: handler, ReadHeaderTimeout: 10 * time.Second, IdleTimeout: 60 * time.Second, MaxHeaderBytes: 1 << 20}
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
if err := panel.StartIntents(ctx); err != nil {
|
||||
return fmt.Errorf("creation intent recovery: %w", err)
|
||||
}
|
||||
go panel.RunQuotaMonitor(ctx)
|
||||
go panel.RunSessionCleanup(ctx)
|
||||
return serve(ctx, srv)
|
||||
|
||||
@@ -17,6 +17,8 @@ services:
|
||||
HOST_DATA_ROOT: ${DSH_HOST_DATA_ROOT:?Set an absolute host data directory}
|
||||
PANEL_PUBLIC_URL: ${PANEL_PUBLIC_URL:-http://127.0.0.1:8088}
|
||||
TRUSTED_PROXY_CIDRS: ${TRUSTED_PROXY_CIDRS:-}
|
||||
DSH_RELEASE_REPOSITORY: ${DSH_RELEASE_REPOSITORY:-https://git.misaka.ren/m1saka/dsh-web.git}
|
||||
DSH_RELEASE_BRANCH: ${DSH_RELEASE_BRANCH:-dockerfile}
|
||||
WEB_DIR: /app/web/dist
|
||||
UPLOAD_ALLOWED_EXTENSIONS: ${UPLOAD_ALLOWED_EXTENSIONS:-}
|
||||
ADMIN_USERNAME: ${DSH_ADMIN_USERNAME:-admin}
|
||||
|
||||
@@ -1,7 +0,0 @@
|
||||
{
|
||||
admin off
|
||||
auto_https off
|
||||
}
|
||||
:8080 {
|
||||
reverse_proxy 127.0.0.1:3080
|
||||
}
|
||||
@@ -1,30 +0,0 @@
|
||||
FROM caddy:2.10.2 AS proxy
|
||||
FROM node:24-bookworm-slim
|
||||
ARG DSH_VERSION
|
||||
# Keep the historical dependency-layer cache key; the template revision is set below.
|
||||
ARG RECIPE_REV=3
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends tini ca-certificates git procps && rm -rf /var/lib/apt/lists/*
|
||||
RUN test -n "$DSH_VERSION" && npm install --global --foreground-scripts --loglevel=verbose --registry=https://registry.npmjs.org "@deepseek-ai/dsh@${DSH_VERSION}"
|
||||
RUN npm install --global --ignore-scripts --registry=https://registry.npmjs.org [email protected] && test "$(pnpm --version)" = "10.11.0"
|
||||
COPY install-mobile.mjs /usr/local/lib/dsh-mgr/install-mobile.mjs
|
||||
RUN node /usr/local/lib/dsh-mgr/install-mobile.mjs
|
||||
COPY install-free-search.mjs /usr/local/lib/dsh-mgr/install-free-search.mjs
|
||||
RUN node /usr/local/lib/dsh-mgr/install-free-search.mjs
|
||||
COPY install-workspace.mjs /usr/local/lib/dsh-mgr/install-workspace.mjs
|
||||
RUN node /usr/local/lib/dsh-mgr/install-workspace.mjs
|
||||
COPY init-profile.mjs /usr/local/lib/dsh-mgr/init-profile.mjs
|
||||
COPY --from=proxy /usr/bin/caddy /usr/bin/caddy
|
||||
# Strip upstream file capabilities without granting any runtime capabilities.
|
||||
# A fresh file copy omits security.capability xattrs; port 8080 needs none.
|
||||
RUN cp /usr/bin/caddy /usr/bin/caddy.clean && mv /usr/bin/caddy.clean /usr/bin/caddy
|
||||
COPY Caddyfile /etc/caddy/Caddyfile
|
||||
COPY entrypoint.sh /usr/local/bin/dsh-entrypoint
|
||||
RUN chmod 755 /usr/local/bin/dsh-entrypoint && mkdir -p /data /workspace /shared && ln -s /workspace /data/dsh-home && chown -R node:node /data /workspace /shared
|
||||
ARG RECIPE_REV=7
|
||||
LABEL dsh-mgr.role=runtime dsh-mgr.version=$DSH_VERSION dsh-mgr.recipe=$RECIPE_REV
|
||||
ENV DSH_HOME=/workspace HOME=/workspace XDG_DATA_HOME=/workspace/.local/share XDG_CONFIG_HOME=/workspace/.config
|
||||
USER node
|
||||
WORKDIR /workspace
|
||||
EXPOSE 8080
|
||||
ENTRYPOINT ["/usr/bin/tini", "-g", "--", "/usr/local/bin/dsh-entrypoint"]
|
||||
CMD ["web", "--no-open", "--port", "3080"]
|
||||
@@ -1,6 +0,0 @@
|
||||
package image
|
||||
|
||||
import "embed"
|
||||
|
||||
//go:embed Dockerfile Caddyfile entrypoint.sh install-mobile.mjs install-free-search.mjs install-workspace.mjs init-profile.mjs
|
||||
var Files embed.FS
|
||||
@@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
set -Eeuo pipefail
|
||||
|
||||
export npm_config_ignore_scripts=true npm_config_ignore_pnpmfile=true
|
||||
node /usr/local/lib/dsh-mgr/init-profile.mjs
|
||||
if [[ -f "${DSH_HOME}/profiles/web/package.json" ]]; then
|
||||
echo "dsh-mgr: plugin prestart reconciliation starting" >&2
|
||||
# pnpm otherwise probes registry metadata for the overlapping optional peer.
|
||||
# Restrict offline mode to profiles containing only our fixed archives.
|
||||
install_env=()
|
||||
if node -e 'const m=require(process.env.DSH_HOME+"/profiles/web/package.json"); const pins={"dsh-mgr-workspace":"file:/usr/local/lib/dsh-mgr/dsh-mgr-workspace-1.0.0.tgz","dsh-web-mobile":"file:/usr/local/lib/dsh-mgr/dsh-web-mobile-3.0.5.tgz","dsh-free-search":"file:/usr/local/lib/dsh-mgr/dsh-free-search-0.8.5.tgz"}; process.exit(Object.entries({...m.dependencies,...m.devDependencies,...m.optionalDependencies}).every(([n,v])=>pins[n]===v)?0:1)'; then
|
||||
install_env=(npm_config_offline=true)
|
||||
fi
|
||||
if timeout --signal=TERM --kill-after=5s 120s env CI=true "${install_env[@]}" dsh plugin --profile web install --ignore-scripts; then
|
||||
echo "dsh-mgr: plugin prestart reconciliation complete" >&2
|
||||
else
|
||||
status=$?
|
||||
echo "dsh-mgr: plugin prestart reconciliation failed (exit $status); application not started" >&2
|
||||
exit "$status"
|
||||
fi
|
||||
else
|
||||
echo "dsh-mgr: no web plugin manifest; using bundled plugins" >&2
|
||||
fi
|
||||
|
||||
caddy run --config /etc/caddy/Caddyfile --adapter caddyfile &
|
||||
proxy_pid=$!
|
||||
echo
|
||||
dsh "$@" &
|
||||
dsh_pid=$!
|
||||
cleanup() {
|
||||
trap - TERM INT
|
||||
kill -TERM "$proxy_pid" "$dsh_pid" 2>/dev/null || true
|
||||
wait "$proxy_pid" 2>/dev/null || true
|
||||
wait "$dsh_pid" 2>/dev/null || true
|
||||
}
|
||||
trap cleanup EXIT
|
||||
trap 'exit 143' TERM
|
||||
trap 'exit 130' INT
|
||||
# Either child exiting terminates the container; never leave a proxy-only instance.
|
||||
set +e
|
||||
wait -n "$proxy_pid" "$dsh_pid"
|
||||
status=$?
|
||||
set -e
|
||||
exit "$status"
|
||||
@@ -1,35 +0,0 @@
|
||||
import { mkdirSync, existsSync, writeFileSync, readFileSync, readdirSync, lstatSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
const home = process.env.DSH_HOME;
|
||||
if (!home) throw new Error('DSH_HOME required');
|
||||
const profile = join(home, 'profiles/web');
|
||||
for (const dir of [home, join(home, 'profiles'), profile]) {
|
||||
try {
|
||||
const stat = lstatSync(dir);
|
||||
if (!stat.isDirectory() || stat.isSymbolicLink()) throw new Error('unsafe profile directory');
|
||||
} catch (error) { if (error.code !== 'ENOENT') throw error; }
|
||||
}
|
||||
// Never rewrite an existing user manifest or patch.
|
||||
if (!existsSync(profile) || readdirSync(profile).length === 0) {
|
||||
mkdirSync(profile, { recursive: true });
|
||||
writeFileSync(join(profile, 'package.json'), JSON.stringify({
|
||||
name: 'dsh-profile-web', private: true,
|
||||
dependencies: { 'dsh-mgr-workspace': 'file:/usr/local/lib/dsh-mgr/dsh-mgr-workspace-1.0.0.tgz', 'dsh-web-mobile': 'file:/usr/local/lib/dsh-mgr/dsh-web-mobile-3.0.5.tgz', 'dsh-free-search': 'file:/usr/local/lib/dsh-mgr/dsh-free-search-0.8.5.tgz' },
|
||||
pnpm: { overrides: {
|
||||
'dsh-free-search>@deepseek-ai/schemastery': 'file:/usr/local/lib/dsh-mgr/schemastery-3.18.4.tgz',
|
||||
'@deepseek-ai/schemastery>@deepseek-ai/cosmokit': 'file:/usr/local/lib/dsh-mgr/cosmokit-1.8.5.tgz',
|
||||
'@deepseek-ai/schemastery>@standard-schema/spec': 'file:/usr/local/lib/dsh-mgr/spec-1.1.0.tgz',
|
||||
} },
|
||||
dsh: { profile: { bundles: ['@deepseek-ai/dsh-base', '@deepseek-ai/dsh-web-app', 'dsh-web-mobile', 'dsh-free-search', 'dsh-mgr-workspace'] } },
|
||||
}, null, 2), { flag: 'wx', mode: 0o600 });
|
||||
console.log('dsh-mgr: new web profile includes [email protected] and [email protected]');
|
||||
}
|
||||
const manifestPath = join(profile, 'package.json');
|
||||
const workspacePath = join(profile, 'pnpm-workspace.yaml');
|
||||
// Backups omit workspace settings; restore host-provided peers before pnpm runs.
|
||||
if (existsSync(manifestPath) && !existsSync(workspacePath)) {
|
||||
const manifest = JSON.parse(readFileSync(manifestPath, 'utf8'));
|
||||
if (manifest.dependencies?.['dsh-web-mobile'] === 'file:/usr/local/lib/dsh-mgr/dsh-web-mobile-3.0.5.tgz' || manifest.dependencies?.['dsh-free-search'] === 'file:/usr/local/lib/dsh-mgr/dsh-free-search-0.8.5.tgz') {
|
||||
writeFileSync(workspacePath, 'packages:\n - .\n\nnodeLinker: hoisted\nautoInstallPeers: false\n', { flag: 'wx', mode: 0o600 });
|
||||
}
|
||||
}
|
||||
@@ -1,64 +0,0 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { mkdtempSync, mkdirSync, readFileSync, writeFileSync, rmSync, symlinkSync, existsSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
test('new profile enables bundle once and preserves existing configuration', () => {
|
||||
const home = mkdtempSync(join(tmpdir(), 'dsh-plugin-init-'));
|
||||
try {
|
||||
const run = () => execFileSync(process.execPath, [new URL('./init-profile.mjs', import.meta.url).pathname], {env:{...process.env, DSH_HOME:home}});
|
||||
run();
|
||||
const file=join(home,'profiles/web/package.json');
|
||||
const original=readFileSync(file,'utf8');
|
||||
assert.ok(JSON.parse(original).dsh.profile.bundles.includes('dsh-web-mobile'));
|
||||
assert.ok(JSON.parse(original).dsh.profile.bundles.includes('dsh-free-search'));
|
||||
assert.ok(JSON.parse(original).dsh.profile.bundles.includes('dsh-mgr-workspace'));
|
||||
assert.equal(JSON.parse(original).dependencies['dsh-mgr-workspace'], 'file:/usr/local/lib/dsh-mgr/dsh-mgr-workspace-1.0.0.tgz');
|
||||
assert.equal(JSON.parse(original).dependencies['dsh-free-search'], 'file:/usr/local/lib/dsh-mgr/dsh-free-search-0.8.5.tgz');
|
||||
assert.equal(JSON.parse(original).pnpm.overrides['dsh-free-search>@deepseek-ai/schemastery'], 'file:/usr/local/lib/dsh-mgr/schemastery-3.18.4.tgz');
|
||||
assert.equal(JSON.parse(original).dependencies['dsh-web-mobile'], 'file:/usr/local/lib/dsh-mgr/dsh-web-mobile-3.0.5.tgz');
|
||||
const workspace = join(home, 'profiles/web/pnpm-workspace.yaml');
|
||||
assert.match(readFileSync(workspace, 'utf8'), /autoInstallPeers: false/);
|
||||
assert.match(readFileSync(workspace, 'utf8'), /nodeLinker: hoisted/);
|
||||
run();assert.equal(readFileSync(file,'utf8'),original);
|
||||
const legacy = JSON.parse(original);
|
||||
delete legacy.dependencies['dsh-mgr-workspace'];
|
||||
legacy.dsh.profile.bundles = legacy.dsh.profile.bundles.filter(name => name !== 'dsh-mgr-workspace');
|
||||
writeFileSync(file, JSON.stringify(legacy));
|
||||
run();
|
||||
assert.deepEqual(JSON.parse(readFileSync(file)), legacy);
|
||||
const disabled = JSON.parse(original);
|
||||
disabled.dsh.profile.bundles = disabled.dsh.profile.bundles.filter(name => name !== 'dsh-web-mobile');
|
||||
writeFileSync(file, JSON.stringify(disabled));
|
||||
writeFileSync(workspace, 'custom workspace');
|
||||
const patch = join(home, 'profiles/web/cordis.patch.yml');
|
||||
writeFileSync(patch, '[]');
|
||||
run();
|
||||
assert.deepEqual(JSON.parse(readFileSync(file)), disabled);
|
||||
assert.equal(readFileSync(workspace, 'utf8'), 'custom workspace');
|
||||
assert.equal(readFileSync(patch, 'utf8'), '[]');
|
||||
const custom=JSON.stringify({custom:true});
|
||||
writeFileSync(file,custom);run();assert.equal(readFileSync(file,'utf8'),custom);
|
||||
} finally { rmSync(home,{recursive:true,force:true}); }
|
||||
});
|
||||
test('nonempty legacy profiles are not automatically injected', () => {
|
||||
const home = mkdtempSync(join(tmpdir(), 'dsh-plugin-legacy-'));
|
||||
try {
|
||||
const profile = join(home, 'profiles/web');
|
||||
mkdirSync(profile, { recursive: true });
|
||||
writeFileSync(join(profile, 'cordis.patch.yml'), '[]');
|
||||
execFileSync(process.execPath, [new URL('./init-profile.mjs', import.meta.url).pathname], { env: { ...process.env, DSH_HOME: home } });
|
||||
assert.equal(existsSync(join(profile, 'package.json')), false);
|
||||
assert.equal(existsSync(join(profile, 'pnpm-workspace.yaml')), false);
|
||||
} finally { rmSync(home, { recursive: true, force: true }); }
|
||||
});
|
||||
test('rejects symlinked profile ancestors', () => {
|
||||
const home=mkdtempSync(join(tmpdir(),'dsh-plugin-init-'));
|
||||
const outside=mkdtempSync(join(tmpdir(),'dsh-plugin-outside-'));
|
||||
try {
|
||||
symlinkSync(outside,join(home,'profiles'));
|
||||
assert.throws(()=>execFileSync(process.execPath,[new URL('./init-profile.mjs',import.meta.url).pathname],{env:{...process.env,DSH_HOME:home},stdio:'pipe'}));
|
||||
assert.equal(existsSync(join(outside,'web/package.json')),false);
|
||||
} finally {rmSync(home,{recursive:true,force:true});rmSync(outside,{recursive:true,force:true});}
|
||||
});
|
||||
@@ -1,40 +0,0 @@
|
||||
import { createHash } from 'node:crypto';
|
||||
import { mkdirSync, writeFileSync, readFileSync } from 'node:fs';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { createRequire } from 'node:module';
|
||||
const root = '/usr/local/lib/dsh-mgr';
|
||||
const packages = [
|
||||
['dsh-free-search', '0.8.5', 'https://registry.npmjs.org/dsh-free-search/-/dsh-free-search-0.8.5.tgz', 'MT00Xtypw9uk9QsqaiRmtVbYwnPON+HJY1lZjYritEvuwSe9ahgG9NVO43lUI+Eje2umrqUapaeqLRYf34DrZw=='],
|
||||
['@deepseek-ai/schemastery', '3.18.4', 'https://registry.npmjs.org/@deepseek-ai/schemastery/-/schemastery-3.18.4.tgz', 'SSXO6tYuyrIqKVbmOnIq0s+riUywYzouFMcnBluHF9n4KMo2G8HvEzhCYj6pj2617/glOjbJuVInJ9hfONsjkg=='],
|
||||
['@deepseek-ai/cosmokit', '1.8.5', 'https://registry.npmjs.org/@deepseek-ai/cosmokit/-/cosmokit-1.8.5.tgz', 'LXsrlem9z8dq4sLflj2yuCuYX7KqhdzF5hZly3eZyuTo8d6oDSOXuEgC5DbQWKW+lksm6zmOutQLsP/ma6wR6A=='],
|
||||
['@standard-schema/spec', '1.1.0', 'https://registry.npmjs.org/@standard-schema/spec/-/spec-1.1.0.tgz', 'l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w=='],
|
||||
];
|
||||
for (const [name, version, url, integrity] of packages) {
|
||||
const response = await fetch(url, { signal: AbortSignal.timeout(120000) });
|
||||
if (!response.ok) throw new Error('download failed: ' + name);
|
||||
const data = Buffer.from(await response.arrayBuffer());
|
||||
if (createHash('sha512').update(data).digest('base64') !== integrity) throw new Error('integrity mismatch: ' + name);
|
||||
const archive = `${root}/${name.split('/').pop()}-${version}.tgz`;
|
||||
writeFileSync(archive, data, { mode: 0o644 });
|
||||
}
|
||||
const hostManifest = '/usr/local/lib/node_modules/@deepseek-ai/dsh/package.json';
|
||||
const destination = '/usr/local/lib/node_modules/@deepseek-ai/dsh/node_modules/dsh-free-search';
|
||||
mkdirSync(destination, { recursive: true });
|
||||
execFileSync('tar', ['-xzf', `${root}/dsh-free-search-0.8.5.tgz`, '--strip-components=1', '--no-same-owner', '-C', destination]);
|
||||
execFileSync('chmod', ['-R', 'a+rX', destination]);
|
||||
const pkg = JSON.parse(readFileSync(destination + '/package.json'));
|
||||
if (pkg.name !== 'dsh-free-search' || pkg.version !== '0.8.5' || JSON.stringify(pkg.dependencies) !== JSON.stringify({ '@deepseek-ai/schemastery': '^3.18.4' })) throw new Error('unexpected free-search metadata');
|
||||
const require = createRequire(hostManifest);
|
||||
const semver = createRequire('/usr/local/lib/node_modules/npm/package.json')('semver');
|
||||
for (const [name, range] of Object.entries({ ...pkg.peerDependencies, ...pkg.dependencies })) {
|
||||
if (name === 'react') {
|
||||
if (!['0.2.0-rc.1', '0.2.0-rc.2'].includes(require('./package.json').version)) throw new Error('unverified browser React host');
|
||||
continue;
|
||||
}
|
||||
const peer = require(name + '/package.json');
|
||||
if (!semver.satisfies(peer.version, range)) throw new Error('incompatible free-search host dependency: ' + name);
|
||||
}
|
||||
const host = JSON.parse(readFileSync(hostManifest));
|
||||
host.dependencies = { ...host.dependencies, 'dsh-free-search': '0.8.5' };
|
||||
writeFileSync(hostManifest, JSON.stringify(host, null, 2));
|
||||
console.log('Installed [email protected] and pinned offline dependency archives (SHA-512 verified; no lifecycle scripts)');
|
||||
@@ -1,44 +0,0 @@
|
||||
import { createHash } from 'node:crypto';
|
||||
import { mkdirSync, writeFileSync, readFileSync } from 'node:fs';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { createRequire } from 'node:module';
|
||||
const version = '3.0.5';
|
||||
const integrity = 'N3wmegFHSd23J0+xjotKyBq0ZbLXUbkWZ65E+aSw5pzB/nudhaNHWT2V3CSNaW/LX3s6AxAWB2APpfA0dLXJZQ==';
|
||||
const response = await fetch('https://registry.npmjs.org/dsh-web-mobile/-/dsh-web-mobile-3.0.5.tgz', { signal: AbortSignal.timeout(120000) });
|
||||
if (!response.ok) throw new Error('mobile plugin download failed');
|
||||
const data = Buffer.from(await response.arrayBuffer());
|
||||
if (createHash('sha512').update(data).digest('base64') !== integrity) throw new Error('mobile plugin integrity mismatch');
|
||||
const destination = '/usr/local/lib/node_modules/@deepseek-ai/dsh/node_modules/dsh-web-mobile';
|
||||
mkdirSync(destination, { recursive: true });
|
||||
const archive = '/usr/local/lib/dsh-mgr/dsh-web-mobile-3.0.5.tgz';
|
||||
writeFileSync(archive, data, { mode: 0o644 });
|
||||
execFileSync('tar', ['-xzf', archive, '--strip-components=1', '--no-same-owner', '-C', destination]);
|
||||
// Published tar modes can be owner-only; runtime is UID 1000, not build root.
|
||||
execFileSync('chmod', ['-R', 'a+rX', destination]);
|
||||
const pkg = JSON.parse(readFileSync(destination + '/package.json'));
|
||||
if (pkg.version !== version || pkg.name !== 'dsh-web-mobile' || Object.keys(pkg.dependencies || {}).length) throw new Error('unexpected mobile package metadata');
|
||||
const require = createRequire('/usr/local/lib/node_modules/@deepseek-ai/dsh/package.json');
|
||||
const semver = createRequire('/usr/local/lib/node_modules/npm/package.json')('semver');
|
||||
for (const [name, range] of Object.entries(pkg.peerDependencies || {})) {
|
||||
// React is provided by the browser shell, not a Node package in DSH.
|
||||
// Limit that exception to the upstream-declared, tested RC hosts.
|
||||
if (name === 'react') {
|
||||
const host = require('./package.json').version;
|
||||
if (!['0.2.0-rc.1', '0.2.0-rc.2'].includes(host)) throw new Error('unverified browser React host: ' + host);
|
||||
continue;
|
||||
}
|
||||
let peer;
|
||||
try { peer = require(name + '/package.json'); }
|
||||
catch (error) {
|
||||
if (pkg.peerDependenciesMeta?.[name]?.optional) continue;
|
||||
throw new Error('missing required mobile peer: ' + name);
|
||||
}
|
||||
if (!semver.satisfies(peer.version, range)) throw new Error('incompatible mobile peer: ' + name + '@' + peer.version);
|
||||
}
|
||||
// DSH runtime resolution traverses the installation manifest, not arbitrary
|
||||
// directories in node_modules. Declare the baked package in that graph.
|
||||
const hostManifest = '/usr/local/lib/node_modules/@deepseek-ai/dsh/package.json';
|
||||
const host = JSON.parse(readFileSync(hostManifest));
|
||||
host.dependencies = { ...host.dependencies, 'dsh-web-mobile': version };
|
||||
writeFileSync(hostManifest, JSON.stringify(host, null, 2));
|
||||
console.log('Installed [email protected] (SHA-512 verified; no lifecycle scripts)');
|
||||
@@ -1,12 +0,0 @@
|
||||
import { mkdirSync, writeFileSync } from 'node:fs';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
const root = '/usr/local/lib/dsh-mgr';
|
||||
const destination = root + '/workspace-package/package';
|
||||
mkdirSync(destination, { recursive: true });
|
||||
writeFileSync(destination + '/package.json', JSON.stringify({
|
||||
name: 'dsh-mgr-workspace', version: '1.0.0', type: 'module', main: 'index.js',
|
||||
dsh: { bundle: { patch: 'cordis.patch.yml' } },
|
||||
}));
|
||||
writeFileSync(destination + '/index.js', "export const inject = ['workspaceRegistry', 'sessions'];\nexport async function apply(ctx) {\n await ctx.workspaceRegistry.initializeDefault(() => '/workspace');\n}\n");
|
||||
writeFileSync(destination + '/cordis.patch.yml', "- insert:\n - id: dsh-mgr-workspace\n name: dsh-mgr-workspace\n");
|
||||
execFileSync('tar', ['-czf', root + '/dsh-mgr-workspace-1.0.0.tgz', '-C', root + '/workspace-package', 'package']);
|
||||
@@ -14,7 +14,6 @@ import (
|
||||
|
||||
// Admission writes serialize with new runtime operations, including their Docker side effects.
|
||||
var admissionMu sync.RWMutex
|
||||
var errBuildUnsupported = errors.New("version is not supported for normal builds; administrator experimental build required")
|
||||
|
||||
func defaultAdmission(version, recipe, platform string) string {
|
||||
if recipe == "7" && platform == "linux/amd64" {
|
||||
@@ -34,25 +33,7 @@ func supportedLayout(recipe string) bool {
|
||||
}
|
||||
return false
|
||||
}
|
||||
func normalBuildAllowed(version string) bool {
|
||||
return positiveAdmission(defaultAdmission(version, imageRecipeRev, imagePlatform))
|
||||
}
|
||||
func positiveAdmission(state string) bool { return state == "recommended" || state == "compatible" }
|
||||
func (a *API) compatibility(w http.ResponseWriter, r *http.Request) {
|
||||
if _, _, err := a.user(r); err != nil {
|
||||
fail(w, 401, "authentication required")
|
||||
return
|
||||
}
|
||||
if r.Method != http.MethodGet {
|
||||
fail(w, 405, "method not allowed")
|
||||
return
|
||||
}
|
||||
entries := []map[string]any{}
|
||||
for _, v := range []string{"0.2.0-rc.1", "0.2.0-rc.2"} {
|
||||
entries = append(entries, map[string]any{"version": v, "recipe_rev": imageRecipeRev, "platform": imagePlatform, "state": defaultAdmission(v, imageRecipeRev, imagePlatform), "build_allowed": normalBuildAllowed(v), "reason": "only linux/amd64 recipe 7 is built-in; successful exact image verification still required"})
|
||||
}
|
||||
writeJSON(w, map[string]any{"recipe_rev": imageRecipeRev, "platform": imagePlatform, "entries": entries, "unknown": map[string]any{"state": "existing_only", "build_allowed": false, "reason": "not accepted; administrator experimental build and explicit admission required"}, "experimental_admin_only": true})
|
||||
}
|
||||
|
||||
func (a *API) evaluateImage(ctx context.Context, id string) (imageRecord, error) {
|
||||
im := imageRecord{ID: id, ImageID: id, Status: "missing", Admission: "existing_only", AdmissionReason: "local image missing"}
|
||||
@@ -89,6 +70,11 @@ func (a *API) evaluateImage(ctx context.Context, id string) (imageRecord, error)
|
||||
if count == 0 {
|
||||
return im, nil
|
||||
}
|
||||
var trustedRelease int
|
||||
err = a.s.DB.QueryRowContext(ctx, "SELECT count(*) FROM image_builds WHERE image_id=? AND version=? AND recipe_rev=? AND platform=? AND status='success' AND release_id<>'' AND release_id=? AND source_repository=? AND source_commit=?", id, im.Version, im.RecipeRev, im.Platform, actual.Config.Labels["dsh-mgr.release"], actual.Config.Labels["dsh-mgr.source"], actual.Config.Labels["dsh-mgr.commit"]).Scan(&trustedRelease)
|
||||
if err != nil {
|
||||
return im, err
|
||||
}
|
||||
im.Admission = defaultAdmission(im.Version, im.RecipeRev, im.Platform)
|
||||
im.AdmissionReason = "no accepted compatibility evidence for this tuple"
|
||||
if positiveAdmission(im.Admission) {
|
||||
@@ -101,6 +87,10 @@ func (a *API) evaluateImage(ctx context.Context, id string) (imageRecord, error)
|
||||
} else {
|
||||
im.Status = "existing_only"
|
||||
}
|
||||
if trustedRelease > 0 {
|
||||
im.Admission = "compatible"
|
||||
im.AdmissionReason = "trusted pinned release passed exact image startup validation"
|
||||
}
|
||||
if state != "" {
|
||||
im.Admission = state
|
||||
im.AdmissionReason = evidence
|
||||
|
||||
@@ -77,16 +77,15 @@ func TestAdmissionRoutesEnforceAuthorization(t *testing.T) {
|
||||
a.Handler().ServeHTTP(w, r)
|
||||
return w
|
||||
}
|
||||
if w := request("GET", "/api/compatibility", "", false, false); w.Code != 401 {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
for _, path := range []string{"/api/latest", "/api/compatibility", "/api/intents", "/api/intents/old"} {
|
||||
if w := request("GET", path, "", true, false); w.Code != 404 {
|
||||
t.Fatal(path, w.Code, w.Body.String())
|
||||
}
|
||||
if w := request("GET", "/api/compatibility", "", true, false); w.Code != 200 {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
}
|
||||
if w := request("POST", "/api/images/admission", body, true, true); w.Code != 403 {
|
||||
t.Fatal("nonadmin admission", w.Code, w.Body.String())
|
||||
}
|
||||
if w := request("POST", "/api/builds", `{"version":"9.8.7","experimental":true}`, true, true); w.Code != 403 {
|
||||
if w := request("POST", "/api/builds", `{"version":"9.8.7","experimental":true}`, true, true); w.Code != 405 {
|
||||
t.Fatal("nonadmin experiment", w.Code, w.Body.String())
|
||||
}
|
||||
if _, err = a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='owner'"); err != nil {
|
||||
|
||||
@@ -12,9 +12,9 @@ import (
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestExperimentalBuildSuccessRequiresAcceptance(t *testing.T) {
|
||||
func TestHistoricalExperimentalBuildRequiresAcceptance(t *testing.T) {
|
||||
id := "sha256:" + strings.Repeat("d", 64)
|
||||
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
switch r.URL.Path {
|
||||
case "/v1.47/images/json":
|
||||
writeJSON(w, []any{map[string]any{"Id": id, "Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "9.8.7", "dsh-mgr.recipe": imageRecipeRev}}})
|
||||
@@ -35,35 +35,17 @@ func TestExperimentalBuildSuccessRequiresAcceptance(t *testing.T) {
|
||||
if _, err := a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='u'"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
w := imageRequest(a.builds, sid, "POST", `{"version":"9.8.7","experimental":true}`)
|
||||
if w.Code != 202 {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
}
|
||||
var b buildRecord
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &b); err != nil || !b.Experimental {
|
||||
t.Fatal(b, err)
|
||||
}
|
||||
deadline := time.Now().Add(5 * time.Second)
|
||||
for {
|
||||
var state string
|
||||
if err := a.s.DB.QueryRow("SELECT status FROM image_builds WHERE id=?", b.ID).Scan(&state); err != nil {
|
||||
seedVerifiedImage(t, a, id, "9.8.7")
|
||||
if _, err := a.s.DB.Exec("DELETE FROM image_admissions; UPDATE image_builds SET experimental=1"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if state == "success" {
|
||||
break
|
||||
}
|
||||
if state == "failed" || time.Now().After(deadline) {
|
||||
t.Fatal(state)
|
||||
}
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
im, err := a.evaluateImage(context.Background(), id)
|
||||
if err != nil || im.Selectable || im.Status != "experimental" {
|
||||
t.Fatal(im, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestQueuedUnsupportedPlatformFailsBeforeDocker(t *testing.T) {
|
||||
func TestLegacyQueuedBuildFailsBeforeDocker(t *testing.T) {
|
||||
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("unsupported build contacted Docker") })
|
||||
buildQueue.Lock()
|
||||
err := a.ensureBuilds()
|
||||
@@ -80,7 +62,7 @@ func TestQueuedUnsupportedPlatformFailsBeforeDocker(t *testing.T) {
|
||||
}
|
||||
a.runImageBuild(b)
|
||||
var state, detail string
|
||||
if err := a.s.DB.QueryRow("SELECT status,error FROM image_builds WHERE id=?", b.ID).Scan(&state, &detail); err != nil || state != "failed" || !strings.Contains(detail, "experimental") {
|
||||
if err := a.s.DB.QueryRow("SELECT status,error FROM image_builds WHERE id=?", b.ID).Scan(&state, &detail); err != nil || state != "failed" || !strings.Contains(detail, "Git release identity required") {
|
||||
t.Fatal(state, detail, err)
|
||||
}
|
||||
}
|
||||
@@ -238,7 +220,7 @@ func TestAdmissionExperimentalAndPreflight(t *testing.T) {
|
||||
handler http.HandlerFunc
|
||||
body string
|
||||
want int
|
||||
}{{a.builds, `{"version":"9.8.7"}`, 400}, {a.builds, `{"version":"9.8.7","experimental":true}`, 403}, {a.intents, `{"name":"unknown","version":"9.8.7"}`, 400}, {a.instances, fmt.Sprintf(`{"name":"unknown","image_id":%q}`, id), 400}} {
|
||||
}{{a.builds, `{"version":"9.8.7"}`, 405}, {a.builds, `{"version":"9.8.7","experimental":true}`, 405}, {a.instances, fmt.Sprintf(`{"name":"unknown","image_id":%q}`, id), 400}} {
|
||||
w := httptest.NewRecorder()
|
||||
tc.handler(w, runtimeRequest("POST", "/api/test", tc.body, sid))
|
||||
if w.Code != tc.want {
|
||||
@@ -249,9 +231,6 @@ func TestAdmissionExperimentalAndPreflight(t *testing.T) {
|
||||
if err := a.s.DB.QueryRow("SELECT count(*) FROM image_build_requests").Scan(&n); err != nil || n != 0 {
|
||||
t.Fatal(n, err)
|
||||
}
|
||||
if err := a.s.DB.QueryRow("SELECT count(*) FROM creation_intents").Scan(&n); err != nil || n != 0 {
|
||||
t.Fatal(n, err)
|
||||
}
|
||||
if _, err := a.s.DB.Exec("UPDATE users SET admin=1"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
+2
-11
@@ -34,13 +34,6 @@ type API struct {
|
||||
|
||||
// Protected by buildQueue; invalidates image observations across deletions.
|
||||
imageDeleteGeneration uint64
|
||||
|
||||
// Per-image admission evaluations cached for the intent poller; guarded
|
||||
// by evalCacheMu and invalidated by imageDeleteGeneration (buildQueue)
|
||||
// plus a short TTL so Docker observations stay fresh.
|
||||
evalCacheMu sync.Mutex
|
||||
evalCache map[string]evalCacheEntry
|
||||
evalCacheGen uint64
|
||||
}
|
||||
|
||||
func New(s *store.Store, root string) *API {
|
||||
@@ -70,12 +63,10 @@ func (a *API) Handler() http.Handler {
|
||||
m.HandleFunc("/api/instances/", a.instance)
|
||||
m.HandleFunc("/api/images", a.images)
|
||||
m.HandleFunc("/api/images/admission", a.imageAdmission)
|
||||
m.HandleFunc("/api/compatibility", a.compatibility)
|
||||
m.HandleFunc("/api/latest", a.latest)
|
||||
m.HandleFunc("/api/releases/latest", a.latestRelease)
|
||||
m.HandleFunc("/api/releases/build", a.releaseBuild)
|
||||
m.HandleFunc("/api/builds", a.builds)
|
||||
m.HandleFunc("/api/builds/", a.buildDetail)
|
||||
m.HandleFunc("/api/intents", a.intents)
|
||||
m.HandleFunc("/api/intents/", a.intent)
|
||||
m.HandleFunc("/api/ws/", a.workspace)
|
||||
m.HandleFunc("/api/users", a.users)
|
||||
m.HandleFunc("/api/users/reset-password", a.resetPassword)
|
||||
|
||||
@@ -11,10 +11,10 @@ import (
|
||||
)
|
||||
|
||||
// Summary queries never load log bodies; history cannot inflate a page response.
|
||||
const buildSummaryColumns = "id,version,status,error,image_id,created_at,requested_by,COALESCE(recipe_rev,''),COALESCE(platform,''),COALESCE(finished_at,''),experimental"
|
||||
const buildSummaryColumns = "id,version,status,error,image_id,created_at,requested_by,COALESCE(recipe_rev,''),COALESCE(platform,''),COALESCE(finished_at,''),experimental,release_id,source_repository,source_commit"
|
||||
|
||||
func scanBuildSummary(row interface{ Scan(...any) error }, b *buildRecord) error {
|
||||
return row.Scan(&b.ID, &b.Version, &b.Status, &b.Error, &b.ImageID, &b.Created, &b.RequestedBy, &b.RecipeRev, &b.Platform, &b.Finished, &b.Experimental)
|
||||
return row.Scan(&b.ID, &b.Version, &b.Status, &b.Error, &b.ImageID, &b.Created, &b.RequestedBy, &b.RecipeRev, &b.Platform, &b.Finished, &b.Experimental, &b.ReleaseID, &b.SourceRepository, &b.SourceCommit)
|
||||
}
|
||||
|
||||
func (a *API) listBuilds(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -97,7 +97,7 @@ func (a *API) buildDetail(w http.ResponseWriter, r *http.Request) {
|
||||
buildQueue.Unlock()
|
||||
var b buildRecord
|
||||
if err == nil {
|
||||
err = a.s.DB.QueryRowContext(ctx, "SELECT "+buildSummaryColumns+",logs FROM image_builds WHERE id=?", id).Scan(&b.ID, &b.Version, &b.Status, &b.Error, &b.ImageID, &b.Created, &b.RequestedBy, &b.RecipeRev, &b.Platform, &b.Finished, &b.Experimental, &b.Logs)
|
||||
err = a.s.DB.QueryRowContext(ctx, "SELECT "+buildSummaryColumns+",logs FROM image_builds WHERE id=?", id).Scan(&b.ID, &b.Version, &b.Status, &b.Error, &b.ImageID, &b.Created, &b.RequestedBy, &b.RecipeRev, &b.Platform, &b.Finished, &b.Experimental, &b.ReleaseID, &b.SourceRepository, &b.SourceCommit, &b.Logs)
|
||||
}
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
fail(w, 404, "build not found")
|
||||
|
||||
@@ -1,97 +0,0 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestRuntimePrestart(t *testing.T) {
|
||||
for _, status := range []string{"0", "23", "124", "137", "empty"} {
|
||||
t.Run(status, func(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
if status != "empty" {
|
||||
if err := os.MkdirAll(filepath.Join(dir, "profiles/web"), 0700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, "profiles/web/package.json"), []byte("{}"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
scripts := map[string]string{
|
||||
"node": "#!/bin/bash\nexit 0\n",
|
||||
"timeout": `#!/bin/bash
|
||||
printf "timeout %s\n" "$*" >> "$TRACE"
|
||||
test "$1 $2 $3" = "--signal=TERM --kill-after=5s 120s" || exit 90
|
||||
shift 3
|
||||
exec "$@"
|
||||
`,
|
||||
"dsh": `#!/bin/bash
|
||||
printf "dsh %s\n" "$*" >> "$TRACE"
|
||||
test "$npm_config_ignore_scripts:$npm_config_ignore_pnpmfile" = true:true || exit 91
|
||||
if [ "$1" = plugin ]; then
|
||||
test "$*" = "plugin --profile web install --ignore-scripts" || exit 92
|
||||
exit "$PLUGIN_STATUS"
|
||||
fi
|
||||
exit 0
|
||||
`,
|
||||
"caddy": "#!/bin/bash\necho caddy >> \"$TRACE\"\nsleep 0.1\n",
|
||||
}
|
||||
for name, script := range scripts {
|
||||
if err := os.WriteFile(filepath.Join(dir, name), []byte(script), 0700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "bash", "../../image/entrypoint.sh", "web", "--no-open", "--port", "3080")
|
||||
cmd.Env = append(os.Environ(), "PATH="+dir+":"+os.Getenv("PATH"), "DSH_HOME="+dir, "TRACE="+filepath.Join(dir, "trace"), "PLUGIN_STATUS="+status)
|
||||
output, err := cmd.CombinedOutput()
|
||||
data, readErr := os.ReadFile(filepath.Join(dir, "trace"))
|
||||
if readErr != nil {
|
||||
t.Fatal(readErr)
|
||||
}
|
||||
trace := string(data)
|
||||
if status == "empty" {
|
||||
if err != nil || strings.Contains(trace, "plugin") || !strings.Contains(trace, "dsh web") {
|
||||
t.Fatalf("%v: %s: %s", err, trace, output)
|
||||
}
|
||||
return
|
||||
}
|
||||
if !strings.Contains(trace, "dsh plugin --profile web install --ignore-scripts") {
|
||||
t.Fatal(trace)
|
||||
}
|
||||
if status == "0" {
|
||||
if err != nil || !strings.Contains(string(output), "reconciliation complete") {
|
||||
t.Fatalf("%v: %s", err, output)
|
||||
}
|
||||
if !strings.Contains(trace, "dsh web") {
|
||||
t.Fatal(trace)
|
||||
}
|
||||
} else {
|
||||
if err == nil || !strings.Contains(string(output), "failed (exit "+status+")") {
|
||||
t.Fatalf("%v: %s", err, output)
|
||||
}
|
||||
if strings.Contains(trace, "caddy") || strings.Contains(trace, "dsh web") {
|
||||
t.Fatalf("application started after failure: %s", trace)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRuntimePNPMPinned(t *testing.T) {
|
||||
data, err := os.ReadFile("../../image/Dockerfile")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, required := range []string{"[email protected]", `test "$(pnpm --version)" = "10.11.0"`, "USER node", `"/usr/bin/tini", "-g"`} {
|
||||
if !strings.Contains(string(data), required) {
|
||||
t.Errorf("missing %s", required)
|
||||
}
|
||||
}
|
||||
}
|
||||
+20
-331
@@ -1,14 +1,11 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"bytes"
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
imageassets "git.misaka.ren/m1saka/dsh-web/image"
|
||||
"git.misaka.ren/m1saka/dsh-web/internal/store"
|
||||
"io"
|
||||
"net/http"
|
||||
@@ -35,6 +32,9 @@ func (a *API) ensureBuilds() error {
|
||||
"ALTER TABLE image_builds ADD COLUMN platform TEXT",
|
||||
"ALTER TABLE image_builds ADD COLUMN finished_at TEXT",
|
||||
"ALTER TABLE image_builds ADD COLUMN experimental INTEGER NOT NULL DEFAULT 0",
|
||||
"ALTER TABLE image_builds ADD COLUMN release_id TEXT NOT NULL DEFAULT ''",
|
||||
"ALTER TABLE image_builds ADD COLUMN source_repository TEXT NOT NULL DEFAULT ''",
|
||||
"ALTER TABLE image_builds ADD COLUMN source_commit TEXT NOT NULL DEFAULT ''",
|
||||
"CREATE TABLE IF NOT EXISTS image_admissions(image_id TEXT NOT NULL,version TEXT NOT NULL,recipe_rev TEXT NOT NULL,platform TEXT NOT NULL,state TEXT NOT NULL,evidence TEXT NOT NULL,admitted_by TEXT NOT NULL,admitted_at TEXT NOT NULL,PRIMARY KEY(image_id,version,recipe_rev,platform))",
|
||||
"ALTER TABLE images ADD COLUMN recipe_rev TEXT",
|
||||
"ALTER TABLE images ADD COLUMN platform TEXT",
|
||||
@@ -44,13 +44,10 @@ func (a *API) ensureBuilds() error {
|
||||
return e
|
||||
}
|
||||
}
|
||||
_, err = a.s.DB.Exec("DROP INDEX IF EXISTS image_builds_version; CREATE UNIQUE INDEX IF NOT EXISTS image_builds_identity ON image_builds(version,recipe_rev,platform) WHERE recipe_rev IS NOT NULL AND platform IS NOT NULL AND status IN ('queued','running','validating','success')")
|
||||
_, err = a.s.DB.Exec("DROP INDEX IF EXISTS image_builds_version; DROP INDEX IF EXISTS image_builds_identity; CREATE UNIQUE INDEX image_builds_identity ON image_builds(version,recipe_rev,platform) WHERE release_id='' AND recipe_rev IS NOT NULL AND platform IS NOT NULL AND status IN ('queued','running','validating','success'); CREATE UNIQUE INDEX IF NOT EXISTS image_builds_release_identity ON image_builds(source_repository,source_commit,release_id,platform) WHERE release_id<>'' AND status IN ('queued','running','validating','success')")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err = a.s.DB.Exec("UPDATE image_builds SET status='failed',error='version requires administrator experimental build' WHERE status='queued' AND experimental=0 AND version NOT IN ('0.2.0-rc.1','0.2.0-rc.2')"); err != nil {
|
||||
return err
|
||||
}
|
||||
if !buildQueue.initialized[a.s.DB] {
|
||||
if _, err = a.s.DB.Exec("UPDATE image_builds SET status='failed',error='image template identity is unknown' WHERE status='queued' AND (recipe_rev IS NULL OR recipe_rev<>? OR platform IS NULL OR platform<>?)", imageRecipeRev, imagePlatform); err != nil {
|
||||
return err
|
||||
@@ -63,179 +60,20 @@ func (a *API) ensureBuilds() error {
|
||||
return nil
|
||||
}
|
||||
func (a *API) builds(w http.ResponseWriter, r *http.Request) {
|
||||
u, _, authErr := a.user(r)
|
||||
_, _, authErr := a.user(r)
|
||||
if authErr != nil {
|
||||
fail(w, 401, "authentication required")
|
||||
return
|
||||
}
|
||||
if r.Method != http.MethodGet && r.Method != http.MethodPost {
|
||||
if r.Method != http.MethodGet {
|
||||
fail(w, 405, "method not allowed")
|
||||
return
|
||||
}
|
||||
var in struct {
|
||||
Version string `json:"version"`
|
||||
Experimental bool `json:"experimental"`
|
||||
}
|
||||
if r.Method == http.MethodPost {
|
||||
dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096))
|
||||
dec.DisallowUnknownFields()
|
||||
if dec.Decode(&in) != nil || dec.Decode(new(any)) != io.EOF || !validBuildVersion(in.Version) {
|
||||
fail(w, 400, "a fixed version is required")
|
||||
return
|
||||
}
|
||||
}
|
||||
if r.Method == http.MethodGet {
|
||||
a.listBuilds(w, r)
|
||||
return
|
||||
}
|
||||
if in.Experimental && !u.Admin {
|
||||
fail(w, 403, "administrator required")
|
||||
return
|
||||
}
|
||||
b, err := a.enqueueBuild(r.Context(), u.ID, in.Version, in.Experimental)
|
||||
if err != nil {
|
||||
status := 502
|
||||
if errors.Is(err, errBuildUnsupported) {
|
||||
status = 400
|
||||
}
|
||||
if errors.Is(err, errBuildRateLimited) {
|
||||
status = 429
|
||||
w.Header().Set("Retry-After", "3600")
|
||||
}
|
||||
fail(w, status, sanitizeBuildLog(err.Error()))
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(http.StatusAccepted)
|
||||
writeJSON(w, b)
|
||||
}
|
||||
|
||||
var errBuildRateLimited = errors.New("build limit: three new tasks per user per hour")
|
||||
|
||||
// lookupActiveBuild returns the active or successful record for a version; it
|
||||
// performs no I/O and must be called with the build queue held.
|
||||
func (a *API) lookupActiveBuild(version string) (buildRecord, bool, error) {
|
||||
var b buildRecord
|
||||
var finished, recipe, platform sql.NullString
|
||||
err := a.s.DB.QueryRow("SELECT id,version,status,logs,error,image_id,created_at,requested_by,recipe_rev,platform,finished_at,experimental FROM image_builds WHERE version=? AND recipe_rev=? AND platform=? AND status IN ('queued','running','validating','success')", version, imageRecipeRev, imagePlatform).Scan(&b.ID, &b.Version, &b.Status, &b.Logs, &b.Error, &b.ImageID, &b.Created, &b.RequestedBy, &recipe, &platform, &finished, &b.Experimental)
|
||||
b.RecipeRev, b.Platform, b.Finished = recipe.String, platform.String, finished.String
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return b, false, nil
|
||||
}
|
||||
if err != nil {
|
||||
return b, false, err
|
||||
}
|
||||
return b, true, nil
|
||||
}
|
||||
|
||||
func (a *API) enqueueBuild(ctx context.Context, uid, version string, experiment ...bool) (buildRecord, error) {
|
||||
experimental := len(experiment) > 0 && experiment[0]
|
||||
if experimental {
|
||||
u, err := a.s.UserByID(uid)
|
||||
if err != nil || !u.Admin {
|
||||
return buildRecord{}, errors.New("administrator required")
|
||||
}
|
||||
}
|
||||
if !experimental && !normalBuildAllowed(version) {
|
||||
return buildRecord{}, errBuildUnsupported
|
||||
}
|
||||
if !validBuildVersion(version) {
|
||||
return buildRecord{}, errors.New("a fixed version is required")
|
||||
}
|
||||
buildQueue.Lock()
|
||||
err := a.ensureBuilds()
|
||||
var b buildRecord
|
||||
var found bool
|
||||
if err == nil {
|
||||
b, found, err = a.lookupActiveBuild(version)
|
||||
}
|
||||
buildQueue.Unlock()
|
||||
if err != nil {
|
||||
return b, err
|
||||
}
|
||||
if found && !experimental && b.Experimental && b.Status != "success" {
|
||||
return b, errBuildUnsupported
|
||||
}
|
||||
if found && b.Status != "success" {
|
||||
a.startBuildWorker()
|
||||
return b, nil
|
||||
}
|
||||
if found {
|
||||
// Verifying a previously built image talks to Docker; keep that I/O
|
||||
// out of the queue lock shared with instance creation and upgrades.
|
||||
verify, cancel := context.WithTimeout(ctx, 15*time.Second)
|
||||
id, lookupErr := a.localBuildImage(verify, b.Version, b.ImageID)
|
||||
cancel()
|
||||
if lookupErr != nil {
|
||||
return b, lookupErr
|
||||
}
|
||||
if id != "" && id == b.ImageID {
|
||||
if !experimental {
|
||||
im, e := a.evaluateImage(ctx, id)
|
||||
if e != nil {
|
||||
return b, e
|
||||
}
|
||||
if !im.Selectable {
|
||||
return b, errBuildUnsupported
|
||||
}
|
||||
}
|
||||
a.startBuildWorker()
|
||||
return b, nil
|
||||
}
|
||||
buildQueue.Lock()
|
||||
if _, err = a.s.DB.Exec("UPDATE image_builds SET status='missing',error='image no longer available' WHERE id=? AND status='success'", b.ID); err == nil {
|
||||
// Another request may have queued a replacement while the image
|
||||
// was being verified; honor it instead of inserting a duplicate.
|
||||
b, found, err = a.lookupActiveBuild(version)
|
||||
}
|
||||
if err == nil && !found {
|
||||
b, err = a.insertBuild(uid, version, experimental)
|
||||
}
|
||||
buildQueue.Unlock()
|
||||
if err != nil {
|
||||
return b, err
|
||||
}
|
||||
if !experimental && found && b.Experimental {
|
||||
return b, errBuildUnsupported
|
||||
}
|
||||
a.startBuildWorker()
|
||||
return b, nil
|
||||
}
|
||||
buildQueue.Lock()
|
||||
defer buildQueue.Unlock()
|
||||
// Re-check after the unlocked phases: another request may have inserted.
|
||||
b, found, err = a.lookupActiveBuild(version)
|
||||
if err != nil {
|
||||
return b, err
|
||||
}
|
||||
if found {
|
||||
if !experimental && b.Experimental {
|
||||
return b, errBuildUnsupported
|
||||
}
|
||||
return b, nil
|
||||
}
|
||||
b, err = a.insertBuild(uid, version, experimental)
|
||||
if err != nil {
|
||||
return b, err
|
||||
}
|
||||
a.startBuildWorkerLocked()
|
||||
return b, nil
|
||||
}
|
||||
|
||||
// insertBuild applies the per-user rate limit and inserts a queued record;
|
||||
// call with the build queue held.
|
||||
func (a *API) insertBuild(uid, version string, experiment ...bool) (buildRecord, error) {
|
||||
var count int
|
||||
if err := a.s.DB.QueryRow("SELECT count(*) FROM image_build_requests WHERE user_id=? AND created_at>?", uid, time.Now().Add(-time.Hour).UTC().Format(time.RFC3339Nano)).Scan(&count); err != nil {
|
||||
return buildRecord{}, err
|
||||
}
|
||||
if count >= 3 {
|
||||
return buildRecord{}, errBuildRateLimited
|
||||
}
|
||||
b := buildRecord{ID: store.ID(), Version: version, Status: "queued", Created: time.Now().UTC().Format(time.RFC3339Nano), RequestedBy: uid, RecipeRev: imageRecipeRev, Platform: imagePlatform}
|
||||
b.Experimental = len(experiment) > 0 && experiment[0]
|
||||
return b, a.insertBuildRequest(b, uid)
|
||||
}
|
||||
func (a *API) startBuildWorker() {
|
||||
buildQueue.Lock()
|
||||
defer buildQueue.Unlock()
|
||||
@@ -280,7 +118,8 @@ func (a *API) startBuildWorkerLocked() {
|
||||
}
|
||||
buildQueue.Lock()
|
||||
var b buildRecord
|
||||
err := a.s.DB.QueryRow("SELECT id,version,experimental,requested_by FROM image_builds WHERE status='queued' ORDER BY created_at,id LIMIT 1").Scan(&b.ID, &b.Version, &b.Experimental, &b.RequestedBy)
|
||||
row := a.s.DB.QueryRow("SELECT " + buildSummaryColumns + " FROM image_builds WHERE status='queued' AND release_id<>'' ORDER BY created_at,id LIMIT 1")
|
||||
err := scanBuildSummary(row, &b)
|
||||
if err == nil {
|
||||
_, err = a.s.DB.Exec("UPDATE image_builds SET status='running' WHERE id=?", b.ID)
|
||||
}
|
||||
@@ -303,27 +142,6 @@ func (a *API) startBuildWorkerLocked() {
|
||||
}
|
||||
}()
|
||||
}
|
||||
func imageBuildContext() (io.Reader, error) {
|
||||
var buf bytes.Buffer
|
||||
tw := tar.NewWriter(&buf)
|
||||
for _, name := range []string{"Dockerfile", "Caddyfile", "entrypoint.sh", "install-mobile.mjs", "install-free-search.mjs", "install-workspace.mjs", "init-profile.mjs"} {
|
||||
dataBytes, err := imageassets.Files.ReadFile(name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
data := string(dataBytes)
|
||||
if err := tw.WriteHeader(&tar.Header{Name: name, Mode: 0755, Size: int64(len(data))}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, err := tw.Write([]byte(data)); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
if err := tw.Close(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &buf, nil
|
||||
}
|
||||
func (a *API) runImageBuild(b buildRecord) {
|
||||
buildWorker.Lock()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute)
|
||||
@@ -344,32 +162,23 @@ func (a *API) runImageBuild(b buildRecord) {
|
||||
_, logErr = a.s.DB.Exec("UPDATE image_builds SET logs=? WHERE id=?", logs, b.ID)
|
||||
}
|
||||
}
|
||||
tarball, err := imageBuildContext()
|
||||
if !b.Experimental && !normalBuildAllowed(b.Version) {
|
||||
err = errBuildUnsupported
|
||||
}
|
||||
if b.Experimental {
|
||||
u, e := a.s.UserByID(b.RequestedBy)
|
||||
if e != nil || !u.Admin {
|
||||
err = errors.New("experimental build requester is not an administrator")
|
||||
}
|
||||
}
|
||||
var id string
|
||||
if err == nil {
|
||||
id, err = a.localBuildImage(ctx, b.Version, "")
|
||||
}
|
||||
if err == nil && id == "" {
|
||||
_, err = fetchNPMVersion(ctx, npmBuildClient, b.Version)
|
||||
var err error
|
||||
if b.ReleaseID == "" {
|
||||
err = errors.New("Git release identity required")
|
||||
} else {
|
||||
var tarball io.Reader
|
||||
tarball, err = a.releaseBuildContext(ctx, b)
|
||||
if err == nil {
|
||||
err = a.checkBuildDisk(ctx)
|
||||
}
|
||||
if err == nil {
|
||||
id, err = a.docker.BuildRelease(ctx, tarball, b.Version, "dsh-web:release-"+b.SourceCommit, b.ReleaseID, b.SourceRepository, b.SourceCommit, appendLog)
|
||||
}
|
||||
if err == nil && id == "" {
|
||||
id, err = a.docker.Build(ctx, tarball, b.Version, "dsh-web:"+b.Version+"-"+imageRecipeRev, appendLog)
|
||||
}
|
||||
if err == nil {
|
||||
var matches bool
|
||||
matches, err = a.imageMatches(ctx, id, b.Version)
|
||||
matches, err = a.buildImageMatches(ctx, id, b)
|
||||
if err == nil && !matches {
|
||||
err = errors.New("image template or platform mismatch")
|
||||
}
|
||||
@@ -517,6 +326,9 @@ type imageRecord struct {
|
||||
UsageCount int `json:"usage_count"`
|
||||
}
|
||||
type buildRecord struct {
|
||||
ReleaseID string `json:"release_id,omitempty"`
|
||||
SourceRepository string `json:"source_repository,omitempty"`
|
||||
SourceCommit string `json:"source_commit,omitempty"`
|
||||
Experimental bool `json:"experimental"`
|
||||
ImageID string `json:"image_id,omitempty"`
|
||||
ID string `json:"id"`
|
||||
@@ -554,38 +366,6 @@ func (a *API) images(w http.ResponseWriter, r *http.Request) {
|
||||
writeJSON(w, result)
|
||||
}
|
||||
|
||||
// verifiedImagesTTL bounds how long a cached admission evaluation is reused
|
||||
// by the one-second intent poller.
|
||||
var verifiedImagesTTL = 5 * time.Second
|
||||
|
||||
type evalCacheEntry struct {
|
||||
im imageRecord
|
||||
at time.Time
|
||||
}
|
||||
|
||||
// evaluateImageCached reuses recent per-image evaluations for the intent
|
||||
// poller so repeated ticks do not hammer Docker. Deletions and admission
|
||||
// writes bump imageDeleteGeneration and invalidate every entry.
|
||||
func (a *API) evaluateImageCached(ctx context.Context, gen uint64, id string) (imageRecord, error) {
|
||||
a.evalCacheMu.Lock()
|
||||
if entry, ok := a.evalCache[id]; ok && a.evalCacheGen == gen && time.Since(entry.at) < verifiedImagesTTL {
|
||||
a.evalCacheMu.Unlock()
|
||||
return entry.im, nil
|
||||
}
|
||||
a.evalCacheMu.Unlock()
|
||||
im, err := a.evaluateImage(ctx, id)
|
||||
if err != nil {
|
||||
return im, err
|
||||
}
|
||||
a.evalCacheMu.Lock()
|
||||
if a.evalCache == nil || a.evalCacheGen != gen {
|
||||
a.evalCache, a.evalCacheGen = make(map[string]evalCacheEntry), gen
|
||||
}
|
||||
a.evalCache[id] = evalCacheEntry{im: im, at: time.Now()}
|
||||
a.evalCacheMu.Unlock()
|
||||
return im, nil
|
||||
}
|
||||
|
||||
func (a *API) successfulImageIDs(ctx context.Context) ([]string, error) {
|
||||
rows, err := a.s.DB.QueryContext(ctx, "SELECT DISTINCT image_id FROM image_builds WHERE status='success' AND image_id<>''")
|
||||
if err != nil {
|
||||
@@ -632,74 +412,6 @@ func (a *API) verifiedImages(ctx context.Context) (map[string]string, error) {
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// verifiedImagesPoll is the intent poller's view: build rows are reread every
|
||||
// tick, but per-image Docker evaluation is cached briefly so a stuck waiting
|
||||
// intent cannot turn the one-second loop into Docker hammering.
|
||||
func (a *API) verifiedImagesPoll(ctx context.Context) (map[string]string, error) {
|
||||
buildQueue.Lock()
|
||||
err := a.ensureBuilds()
|
||||
gen := a.imageDeleteGeneration
|
||||
buildQueue.Unlock()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ids, err := a.successfulImageIDs(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result := map[string]string{}
|
||||
for _, id := range ids {
|
||||
im, err := a.evaluateImageCached(ctx, gen, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if im.Selectable {
|
||||
result[id] = im.Version
|
||||
}
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
func (a *API) latest(w http.ResponseWriter, r *http.Request) {
|
||||
if _, _, err := a.user(r); err != nil {
|
||||
fail(w, 401, "authentication required")
|
||||
return
|
||||
}
|
||||
if r.Method != http.MethodGet {
|
||||
fail(w, 405, "method not allowed")
|
||||
return
|
||||
}
|
||||
v, err := fetchLatest(r.Context(), npmBuildClient)
|
||||
if err != nil {
|
||||
fail(w, 502, "npm version lookup failed")
|
||||
return
|
||||
}
|
||||
writeJSON(w, map[string]any{"version": v, "checked_at": time.Now().UTC().Format(time.RFC3339), "supported": normalBuildAllowed(v), "build_allowed": normalBuildAllowed(v), "admission": defaultAdmission(v, imageRecipeRev, imagePlatform), "admission_reason": "normal builds require a built-in tested tuple; unknown versions require administrator experimental validation and admission"})
|
||||
}
|
||||
func fetchLatest(ctx context.Context, client *http.Client) (string, error) {
|
||||
return fetchNPMVersion(ctx, client, "latest")
|
||||
}
|
||||
func fetchNPMVersion(ctx context.Context, client *http.Client, version string) (string, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://registry.npmjs.org/@deepseek-ai%2Fdsh/"+url.PathEscape(version), nil)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != 200 {
|
||||
return "", fmt.Errorf("npm status %d", resp.StatusCode)
|
||||
}
|
||||
var v struct{ Version string }
|
||||
if err = json.NewDecoder(io.LimitReader(resp.Body, 1<<20)).Decode(&v); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if !validBuildVersion(v.Version) || (version != "latest" && v.Version != version) {
|
||||
return "", errors.New("invalid npm version")
|
||||
}
|
||||
return v.Version, nil
|
||||
}
|
||||
func validBuildVersion(v string) bool { return len(v) <= 100 && fixedVersion.MatchString(v) }
|
||||
func sanitizeBuildLog(s string) string {
|
||||
s = buildURLCredentials.ReplaceAllString(s, "$1[REDACTED]@")
|
||||
@@ -710,26 +422,3 @@ func sanitizeBuildLog(s string) string {
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
func (a *API) localBuildImage(ctx context.Context, version, preferred string) (string, error) {
|
||||
filters, _ := json.Marshal(map[string][]string{"label": {"dsh-mgr.role=runtime", "dsh-mgr.version=" + version}})
|
||||
var images []struct {
|
||||
ID string
|
||||
Labels map[string]string
|
||||
}
|
||||
if err := a.docker.Do(ctx, http.MethodGet, "/images/json?filters="+url.QueryEscape(string(filters)), nil, &images); err != nil {
|
||||
return "", err
|
||||
}
|
||||
for _, im := range images {
|
||||
if im.ID != "" && im.Labels["dsh-mgr.role"] == "runtime" && im.Labels["dsh-mgr.version"] == version && im.Labels["dsh-mgr.recipe"] == imageRecipeRev && (preferred == "" || im.ID == preferred) {
|
||||
ok, err := a.imageMatches(ctx, im.ID, version)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if ok {
|
||||
return im.ID, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
return "", nil
|
||||
}
|
||||
|
||||
@@ -47,7 +47,6 @@ func (a *API) deleteImage(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
defer tx.Rollback()
|
||||
// Take SQLite's writer lock before checking intents: reservation does not use the build queue.
|
||||
if _, err = tx.ExecContext(ctx, "UPDATE images SET status=status WHERE image_id=?", id); err != nil {
|
||||
fail(w, 500, "image index unavailable")
|
||||
return
|
||||
@@ -64,7 +63,6 @@ func (a *API) deleteImage(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
for _, check := range []struct{ table, query string }{
|
||||
{"image_builds", "SELECT count(*) FROM image_builds WHERE status IN ('queued','running','validating')"},
|
||||
{"creation_intents", "SELECT count(*) FROM creation_intents WHERE status IN ('waiting','creating')"},
|
||||
{"upgrade_journal", "SELECT count(*) FROM upgrade_journal WHERE phase NOT IN ('complete','rolled_back')"},
|
||||
} {
|
||||
var exists, count int
|
||||
|
||||
@@ -10,7 +10,7 @@ import (
|
||||
|
||||
func TestImageDelete(t *testing.T) {
|
||||
id := "sha256:" + strings.Repeat("a", 64)
|
||||
for _, mode := range []string{"success", "anonymous", "non-admin", "invalid", "unmanaged", "role", "instance", "container-running", "container-exited", "containers-failure", "containers-null", "containers-unknown", "inspect-failure", "delete-failure", "delete-conflict", "queued", "running", "validating", "worker", "waiting", "creating", "journal", "complete", "rolled_back", "db-failure", "task-query-failure", "audit-failure"} {
|
||||
for _, mode := range []string{"success", "anonymous", "non-admin", "invalid", "unmanaged", "role", "instance", "container-running", "container-exited", "containers-failure", "containers-null", "containers-unknown", "inspect-failure", "delete-failure", "delete-conflict", "queued", "running", "validating", "worker", "journal", "complete", "rolled_back", "db-failure", "task-query-failure", "audit-failure"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
var deletes atomic.Int32
|
||||
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -101,11 +101,6 @@ func TestImageDelete(t *testing.T) {
|
||||
buildQueue.active[a.s.DB] = true
|
||||
buildQueue.Unlock()
|
||||
defer func() { buildQueue.Lock(); delete(buildQueue.active, a.s.DB); buildQueue.Unlock() }()
|
||||
case "waiting", "creating":
|
||||
if err := a.s.EnsureIntents(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
exec("INSERT INTO creation_intents(id,uid,name,version,status,host,created) VALUES('intent','other','i','2.0.0',?,'host',0)", mode)
|
||||
case "db-failure":
|
||||
exec("ALTER TABLE instances RENAME COLUMN image_id TO broken")
|
||||
want = 500
|
||||
|
||||
@@ -1,14 +1,13 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestEnqueueBuildConcurrentMissingImageReturnsReplacement(t *testing.T) {
|
||||
func TestEnqueueReleaseConcurrentMissingImageReturnsReplacement(t *testing.T) {
|
||||
inspections := make(chan chan struct{}, 2)
|
||||
stop := make(chan struct{})
|
||||
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -41,6 +40,10 @@ func TestEnqueueBuildConcurrentMissingImageReturnsReplacement(t *testing.T) {
|
||||
}
|
||||
})
|
||||
seedVerifiedImage(t, a, "old-image", "0.2.0-rc.1")
|
||||
info := releaseFixture()
|
||||
if _, err := a.s.DB.Exec("UPDATE image_builds SET release_id=?,source_repository=?,source_commit=?,version=? WHERE image_id='old-image'", info.ReleaseID, info.Repository, info.Commit, info.Version); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var oldID string
|
||||
if err := a.s.DB.QueryRow("SELECT id FROM image_builds WHERE image_id='old-image'").Scan(&oldID); err != nil {
|
||||
t.Fatal(err)
|
||||
@@ -49,11 +52,9 @@ func TestEnqueueBuildConcurrentMissingImageReturnsReplacement(t *testing.T) {
|
||||
buildQueue.Lock()
|
||||
buildQueue.active[a.s.DB] = true
|
||||
buildQueue.Unlock()
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
var submissions sync.WaitGroup
|
||||
t.Cleanup(func() {
|
||||
close(stop)
|
||||
cancel()
|
||||
submissions.Wait()
|
||||
buildQueue.Lock()
|
||||
delete(buildQueue.active, a.s.DB)
|
||||
@@ -69,7 +70,7 @@ func TestEnqueueBuildConcurrentMissingImageReturnsReplacement(t *testing.T) {
|
||||
submissions.Add(1)
|
||||
go func() {
|
||||
defer submissions.Done()
|
||||
b, err := a.enqueueBuild(ctx, "u", "0.2.0-rc.1")
|
||||
b, err := a.enqueueRelease("u", info)
|
||||
done <- result{b, err}
|
||||
}()
|
||||
return done
|
||||
|
||||
@@ -16,8 +16,6 @@ import (
|
||||
"time"
|
||||
)
|
||||
|
||||
var npmBuildClient = &http.Client{Timeout: 15 * time.Second, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}
|
||||
|
||||
const imageRecipeRev = "7"
|
||||
|
||||
var imagePlatform = runtime.GOOS + "/" + runtime.GOARCH
|
||||
@@ -240,7 +238,7 @@ func (a *API) insertBuildRequest(b buildRecord, user string) error {
|
||||
return err
|
||||
}
|
||||
defer tx.Rollback()
|
||||
if _, err = tx.Exec("INSERT INTO image_builds(id,version,status,created_at,requested_by,recipe_rev,platform,experimental) VALUES(?,?,?,?,?,?,?,?)", b.ID, b.Version, b.Status, b.Created, b.RequestedBy, b.RecipeRev, b.Platform, b.Experimental); err != nil {
|
||||
if _, err = tx.Exec("INSERT INTO image_builds(id,version,status,created_at,requested_by,recipe_rev,platform,experimental,release_id,source_repository,source_commit) VALUES(?,?,?,?,?,?,?,?,?,?,?)", b.ID, b.Version, b.Status, b.Created, b.RequestedBy, b.RecipeRev, b.Platform, b.Experimental, b.ReleaseID, b.SourceRepository, b.SourceCommit); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err = tx.Exec("INSERT INTO image_build_requests(user_id,build_id,created_at) VALUES(?,?,?)", user, b.ID, b.Created); err != nil {
|
||||
@@ -261,7 +259,7 @@ func (a *API) reconcileBuilds(parent context.Context, records []buildRecord) err
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if actual == nil || actual.ID != b.ImageID || actual.Config.Labels["dsh-mgr.role"] != "runtime" || actual.Config.Labels["dsh-mgr.version"] != b.Version || actual.Config.Labels["dsh-mgr.recipe"] != b.RecipeRev || actual.platform() != b.Platform {
|
||||
if actual == nil || actual.ID != b.ImageID || actual.Config.Labels["dsh-mgr.role"] != "runtime" || actual.Config.Labels["dsh-mgr.version"] != b.Version || actual.Config.Labels["dsh-mgr.recipe"] != b.RecipeRev || actual.platform() != b.Platform || !releaseLabelsMatch(actual, *b) {
|
||||
b.Status, b.Error = "missing", "image no longer available"
|
||||
if _, err := a.s.DB.Exec("UPDATE image_builds SET status=?,error=? WHERE id=? AND status='success'", b.Status, b.Error, b.ID); err != nil {
|
||||
return err
|
||||
|
||||
@@ -11,7 +11,6 @@ import (
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestImageLegacyMigrationNeverCertifies(t *testing.T) {
|
||||
@@ -238,33 +237,6 @@ func TestImageIndexUsageMissingAndDockerFailure(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildRateLimitDoesNotPenalizeSharedTask(t *testing.T) {
|
||||
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path == "/v1.47/images/verified/json" {
|
||||
writeImageInspect(w, "verified", "0.2.0-rc.1")
|
||||
return
|
||||
}
|
||||
io.WriteString(w, `[{"Id":"verified","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"}}]`)
|
||||
})
|
||||
seedVerifiedImage(t, a, "verified", "0.2.0-rc.1")
|
||||
for _, id := range []string{"a", "b", "c"} {
|
||||
if _, err := a.s.DB.Exec("INSERT INTO image_build_requests VALUES('u',?,?)", id, time.Now().UTC().Format(time.RFC3339Nano)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if w := imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.1"}`); w.Code != 202 {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
}
|
||||
if w := imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.2"}`); w.Code != 429 || w.Header().Get("Retry-After") == "" {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
}
|
||||
var n int
|
||||
a.s.DB.QueryRow("SELECT count(*) FROM image_build_requests").Scan(&n)
|
||||
if n != 3 {
|
||||
t.Fatal(n)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildDiskPolicy(t *testing.T) {
|
||||
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("unexpected build") })
|
||||
if err := a.checkBuildDisk(context.Background()); err != nil {
|
||||
@@ -310,62 +282,6 @@ func TestBuildDiskPolicy(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestUnknownDockerDiskBlocksBuild(t *testing.T) {
|
||||
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path != "/v1.47/images/json" {
|
||||
t.Errorf("unexpected Docker operation %s", r.URL.Path)
|
||||
}
|
||||
io.WriteString(w, "[]")
|
||||
})
|
||||
t.Setenv("DSH_DOCKER_ROOT_VISIBLE", "")
|
||||
w := imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.1"}`)
|
||||
if w.Code != 202 {
|
||||
t.Fatal(w.Body.String())
|
||||
}
|
||||
deadline := time.Now().Add(3 * time.Second)
|
||||
for {
|
||||
var status, detail string
|
||||
if err := a.s.DB.QueryRow("SELECT status,error FROM image_builds").Scan(&status, &detail); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if status == "failed" {
|
||||
if !strings.Contains(detail, "capacity unknown") {
|
||||
t.Fatal(detail)
|
||||
}
|
||||
break
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatal(status, detail)
|
||||
}
|
||||
time.Sleep(time.Millisecond)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExactNPMMetadata(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
body string
|
||||
status int
|
||||
ok bool
|
||||
}{
|
||||
{`{"version":"0.2.0-rc.1"}`, 200, true},
|
||||
{`{"version":"0.2.0-rc.2"}`, 200, false},
|
||||
{`{"version":"latest"}`, 200, false},
|
||||
{`{}`, 404, false},
|
||||
{`invalid`, 200, false},
|
||||
} {
|
||||
client := &http.Client{Transport: imageRoundTrip(func(r *http.Request) (*http.Response, error) {
|
||||
if r.URL.Host != "registry.npmjs.org" || !strings.HasSuffix(r.URL.Path, "/0.2.0-rc.1") {
|
||||
t.Error(r.URL)
|
||||
}
|
||||
return &http.Response{StatusCode: test.status, Body: io.NopCloser(strings.NewReader(test.body)), Header: make(http.Header)}, nil
|
||||
})}
|
||||
_, err := fetchNPMVersion(context.Background(), client, "0.2.0-rc.1")
|
||||
if (err == nil) != test.ok {
|
||||
t.Fatalf("%s: %v", test.body, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildListReconcilesMissingAndLatestTimestamp(t *testing.T) {
|
||||
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(404) })
|
||||
seedVerifiedImage(t, a, "deleted", "0.2.0-rc.1")
|
||||
@@ -374,49 +290,4 @@ func TestBuildListReconcilesMissingAndLatestTimestamp(t *testing.T) {
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &records); err != nil || len(records) != 1 || records[0].Status != "missing" {
|
||||
t.Fatal(w.Body.String())
|
||||
}
|
||||
w = imageRequest(a.latest, sid, "GET", "")
|
||||
var latest struct {
|
||||
Version string `json:"version"`
|
||||
CheckedAt string `json:"checked_at"`
|
||||
}
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &latest); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := time.Parse(time.RFC3339, latest.CheckedAt); err != nil || latest.Version != "0.2.0-rc.1" {
|
||||
t.Fatal(latest, err)
|
||||
}
|
||||
}
|
||||
|
||||
// The one-second intent poller must reuse per-image evaluations instead of
|
||||
// re-inspecting Docker every tick, and a generation bump (deletion or
|
||||
// admission change) must invalidate the cache.
|
||||
func TestIntentPollerCachesImageEvaluations(t *testing.T) {
|
||||
inspects := 0
|
||||
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path == "/v1.47/images/image/json" {
|
||||
inspects++
|
||||
writeImageInspect(w, "image", "0.2.0-rc.1")
|
||||
return
|
||||
}
|
||||
w.WriteHeader(404)
|
||||
})
|
||||
seedVerifiedImage(t, a, "image", "0.2.0-rc.1")
|
||||
for n := 0; n < 3; n++ {
|
||||
verified, err := a.verifiedImagesPoll(context.Background())
|
||||
if err != nil || len(verified) != 1 || verified["image"] != "0.2.0-rc.1" {
|
||||
t.Fatal(verified, err)
|
||||
}
|
||||
}
|
||||
if inspects != 1 {
|
||||
t.Fatalf("poller re-inspected Docker %d times for the same image", inspects)
|
||||
}
|
||||
buildQueue.Lock()
|
||||
a.imageDeleteGeneration++
|
||||
buildQueue.Unlock()
|
||||
if _, err := a.verifiedImagesPoll(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if inspects != 2 {
|
||||
t.Fatalf("generation bump did not invalidate the cache: %d inspects", inspects)
|
||||
}
|
||||
}
|
||||
|
||||
+21
-269
@@ -1,7 +1,6 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
@@ -10,10 +9,8 @@ import (
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
@@ -24,11 +21,6 @@ func imageTestAPI(t *testing.T, h http.HandlerFunc) (*API, string) {
|
||||
oldDiskAvailable := buildDiskAvailable
|
||||
buildDiskAvailable = func(string) (uint64, error) { return minimumBuildDiskBytes, nil }
|
||||
t.Cleanup(func() { buildDiskAvailable = oldDiskAvailable })
|
||||
oldClient := npmBuildClient
|
||||
npmBuildClient = &http.Client{Transport: imageRoundTrip(func(r *http.Request) (*http.Response, error) {
|
||||
return &http.Response{StatusCode: 200, Header: make(http.Header), Body: io.NopCloser(strings.NewReader(`{"version":"0.2.0-rc.1"}`))}, nil
|
||||
})}
|
||||
t.Cleanup(func() { npmBuildClient = oldClient })
|
||||
root := t.TempDir()
|
||||
t.Setenv("DSH_DOCKER_ROOT_VISIBLE", root)
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -129,34 +121,7 @@ func seedVerifiedImage(t *testing.T, a *API, id, version string) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
func TestImageContextAndSanitizing(t *testing.T) {
|
||||
r, err := imageBuildContext()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
tr := tar.NewReader(r)
|
||||
n := 0
|
||||
for {
|
||||
h, err := tr.Next()
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
data, _ := io.ReadAll(tr)
|
||||
original, err := os.ReadFile(filepath.Join("..", "..", "image", h.Name))
|
||||
if err != nil || string(data) != string(original) {
|
||||
t.Fatalf("recipe drift %s: %v", h.Name, err)
|
||||
}
|
||||
if h.Name == "Dockerfile" && (!strings.Contains(string(data), "ARG RECIPE_REV="+imageRecipeRev+"\n") || !strings.Contains(string(data), "dsh-mgr.recipe=$RECIPE_REV")) {
|
||||
t.Fatal("recipe revision or label differs from image policy")
|
||||
}
|
||||
n++
|
||||
}
|
||||
if n != 7 {
|
||||
t.Fatal(n)
|
||||
}
|
||||
func TestImageSanitizing(t *testing.T) {
|
||||
s := sanitizeBuildLog("token=topsecret Authorization: Bearer hidden password=private https://user:[email protected]/")
|
||||
for _, secret := range []string{"topsecret", "hidden", "private", "user:pass"} {
|
||||
if strings.Contains(s, secret) {
|
||||
@@ -176,129 +141,6 @@ func TestImageContextAndSanitizing(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
type imageRoundTrip func(*http.Request) (*http.Response, error)
|
||||
|
||||
func (f imageRoundTrip) RoundTrip(r *http.Request) (*http.Response, error) { return f(r) }
|
||||
func TestLatestOfficialRegistry(t *testing.T) {
|
||||
client := &http.Client{Transport: imageRoundTrip(func(r *http.Request) (*http.Response, error) {
|
||||
if r.URL.Host != "registry.npmjs.org" || r.URL.Scheme != "https" {
|
||||
t.Fatal(r.URL)
|
||||
}
|
||||
return &http.Response{StatusCode: 200, Body: io.NopCloser(strings.NewReader(`{"version":"0.2.0-rc.1"}`)), Header: make(http.Header)}, nil
|
||||
})}
|
||||
v, err := fetchLatest(context.Background(), client)
|
||||
if err != nil || v != "0.2.0-rc.1" {
|
||||
t.Fatalf("%s %v", v, err)
|
||||
}
|
||||
}
|
||||
func TestBuildDedupAndVerification(t *testing.T) {
|
||||
var calls, deleted atomic.Int32
|
||||
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
switch r.URL.Path {
|
||||
case "/v1.47/images/sha256:abc/json", "/v1.47/images/sha256:reuse/json":
|
||||
writeImageInspect(w, strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/v1.47/images/"), "/json"), "0.2.0-rc.1")
|
||||
case "/v1.47/images/json":
|
||||
if calls.Load() == 0 {
|
||||
io.WriteString(w, "[]")
|
||||
} else {
|
||||
io.WriteString(w, `[{"Id":"sha256:abc","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"}}]`)
|
||||
}
|
||||
case "/v1.47/build":
|
||||
calls.Add(1)
|
||||
if r.URL.Query().Get("t") != "dsh-web:0.2.0-rc.1-"+imageRecipeRev {
|
||||
t.Error("unexpected tag", r.URL.Query())
|
||||
}
|
||||
var args map[string]string
|
||||
json.Unmarshal([]byte(r.URL.Query().Get("buildargs")), &args)
|
||||
if args["DSH_VERSION"] != "0.2.0-rc.1" {
|
||||
t.Error(args)
|
||||
}
|
||||
io.Copy(io.Discard, r.Body)
|
||||
io.WriteString(w, `{"stream":"token=secretvalue"}`+string(rune(10))+`{"aux":{"ID":"sha256:abc"}}`)
|
||||
case "/v1.47/containers/create":
|
||||
var cfg struct {
|
||||
Image string
|
||||
HostConfig struct {
|
||||
NetworkMode string
|
||||
ReadonlyRootfs bool
|
||||
Tmpfs map[string]string
|
||||
}
|
||||
}
|
||||
json.NewDecoder(r.Body).Decode(&cfg)
|
||||
if cfg.Image != "sha256:abc" || cfg.HostConfig.NetworkMode != "none" || !cfg.HostConfig.ReadonlyRootfs || !strings.Contains(cfg.HostConfig.Tmpfs["/tmp"], "exec") {
|
||||
t.Errorf("unsafe config %+v", cfg)
|
||||
}
|
||||
io.WriteString(w, `{"Id":"check"}`)
|
||||
case "/v1.47/containers/check/start", "/v1.47/exec/probe/start":
|
||||
w.WriteHeader(204)
|
||||
case "/v1.47/containers/check/exec":
|
||||
var cfg struct{ Cmd []string }
|
||||
if err := json.NewDecoder(r.Body).Decode(&cfg); err != nil || len(cfg.Cmd) != 3 || !strings.Contains(cfg.Cmd[2], "execFileSync('dsh',['--version']") || !strings.Contains(cfg.Cmd[2], `actual!=="0.2.0-rc.1"`) {
|
||||
t.Errorf("missing version verification: %+v %v", cfg, err)
|
||||
}
|
||||
io.WriteString(w, `{"Id":"probe"}`)
|
||||
case "/v1.47/exec/probe/json":
|
||||
io.WriteString(w, `{"Running":false,"ExitCode":0}`)
|
||||
case "/v1.47/containers/check/json":
|
||||
io.WriteString(w, `{"State":{"Running":true}}`)
|
||||
case "/v1.47/containers/check":
|
||||
deleted.Add(1)
|
||||
w.WriteHeader(204)
|
||||
default:
|
||||
t.Errorf("unexpected %s", r.URL)
|
||||
w.WriteHeader(500)
|
||||
}
|
||||
})
|
||||
responses := make(chan *httptest.ResponseRecorder, 8)
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 8; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
responses <- imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.1"}`)
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
close(responses)
|
||||
first := ""
|
||||
for w := range responses {
|
||||
if w.Code != 202 {
|
||||
t.Fatal(w.Body.String())
|
||||
}
|
||||
var b buildRecord
|
||||
json.Unmarshal(w.Body.Bytes(), &b)
|
||||
if first == "" {
|
||||
first = b.ID
|
||||
}
|
||||
if first != b.ID {
|
||||
t.Fatal("duplicate job")
|
||||
}
|
||||
}
|
||||
deadline := time.Now().Add(5 * time.Second)
|
||||
for {
|
||||
w := imageRequest(a.builds, sid, "GET", "")
|
||||
var jobs []buildRecord
|
||||
json.Unmarshal(w.Body.Bytes(), &jobs)
|
||||
if len(jobs) == 1 && jobs[0].Status == "success" {
|
||||
if strings.Contains(jobs[0].Logs, "secretvalue") {
|
||||
t.Fatal(jobs)
|
||||
}
|
||||
break
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatal(w.Body.String())
|
||||
}
|
||||
time.Sleep(time.Millisecond)
|
||||
}
|
||||
if calls.Load() != 1 || deleted.Load() != 1 {
|
||||
t.Fatalf("builds %d cleanup %d", calls.Load(), deleted.Load())
|
||||
}
|
||||
for _, body := range []string{`{"version":"latest"}`, `{"version":"0.2.0-rc.1","url":"bad"}`, `{"version":"0.2.0-rc.1"} {}`} {
|
||||
if w := imageRequest(a.builds, sid, "POST", body); w.Code != 400 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestVerificationFailureCleansUp(t *testing.T) {
|
||||
var deleted atomic.Bool
|
||||
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -321,86 +163,6 @@ func TestVerificationFailureCleansUp(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildReuseAndMissingImage(t *testing.T) {
|
||||
var exists atomic.Bool
|
||||
exists.Store(true)
|
||||
var builds, checks atomic.Int32
|
||||
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
switch r.URL.Path {
|
||||
case "/v1.47/images/sha256:abc/json", "/v1.47/images/sha256:reuse/json":
|
||||
writeImageInspect(w, strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/v1.47/images/"), "/json"), "0.2.0-rc.1")
|
||||
case "/v1.47/images/json":
|
||||
if exists.Load() {
|
||||
io.WriteString(w, `[{"Id":"sha256:reuse","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"}}]`)
|
||||
} else {
|
||||
io.WriteString(w, "[]")
|
||||
}
|
||||
case "/v1.47/build":
|
||||
builds.Add(1)
|
||||
exists.Store(true)
|
||||
io.Copy(io.Discard, r.Body)
|
||||
io.WriteString(w, `{"aux":{"ID":"sha256:reuse"}}`)
|
||||
case "/v1.47/containers/create":
|
||||
checks.Add(1)
|
||||
io.WriteString(w, `{"Id":"c"}`)
|
||||
case "/v1.47/containers/c/start", "/v1.47/exec/e/start", "/v1.47/containers/c":
|
||||
w.WriteHeader(204)
|
||||
case "/v1.47/containers/c/exec":
|
||||
io.WriteString(w, `{"Id":"e"}`)
|
||||
case "/v1.47/exec/e/json":
|
||||
io.WriteString(w, `{"Running":false,"ExitCode":0}`)
|
||||
case "/v1.47/containers/c/json":
|
||||
io.WriteString(w, `{"State":{"Running":true}}`)
|
||||
default:
|
||||
t.Errorf("unexpected %s", r.URL)
|
||||
w.WriteHeader(500)
|
||||
}
|
||||
})
|
||||
wait := func(id string) {
|
||||
t.Helper()
|
||||
deadline := time.Now().Add(5 * time.Second)
|
||||
for {
|
||||
var status, image string
|
||||
err := a.s.DB.QueryRow("SELECT status,image_id FROM image_builds WHERE id=?", id).Scan(&status, &image)
|
||||
if err == nil && status == "success" {
|
||||
if image != "sha256:reuse" {
|
||||
t.Fatal(image)
|
||||
}
|
||||
return
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatalf("status %s: %v", status, err)
|
||||
}
|
||||
time.Sleep(time.Millisecond)
|
||||
}
|
||||
}
|
||||
post := func() buildRecord {
|
||||
t.Helper()
|
||||
w := imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.1"}`)
|
||||
if w.Code != 202 {
|
||||
t.Fatal(w.Body.String())
|
||||
}
|
||||
var b buildRecord
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &b); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return b
|
||||
}
|
||||
first := post()
|
||||
wait(first.ID)
|
||||
if builds.Load() != 0 || checks.Load() != 1 {
|
||||
t.Fatal("local image not reused and verified")
|
||||
}
|
||||
exists.Store(false)
|
||||
second := post()
|
||||
if second.ID == first.ID {
|
||||
t.Fatal("missing image deduplicated")
|
||||
}
|
||||
wait(second.ID)
|
||||
if builds.Load() != 1 || checks.Load() != 2 {
|
||||
t.Fatal("missing image not rebuilt")
|
||||
}
|
||||
}
|
||||
func TestBuildRestartRecovery(t *testing.T) {
|
||||
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(500) })
|
||||
buildQueue.Lock()
|
||||
@@ -427,6 +189,7 @@ func TestBuildRestartRecovery(t *testing.T) {
|
||||
func TestFailedVerificationNeverAvailable(t *testing.T) {
|
||||
var deleted atomic.Bool
|
||||
imageID := "sha256:" + strings.Repeat("c", 64)
|
||||
info := releaseFixture()
|
||||
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
switch r.URL.Path {
|
||||
case "/v1.47/images/sha256:abc/json", "/v1.47/images/sha256:reuse/json":
|
||||
@@ -444,33 +207,36 @@ func TestFailedVerificationNeverAvailable(t *testing.T) {
|
||||
case "/v1.47/containers/check":
|
||||
deleted.Store(true)
|
||||
w.WriteHeader(204)
|
||||
case "/v1.47/build":
|
||||
writeJSON(w, map[string]any{"aux": map[string]string{"ID": imageID}})
|
||||
case "/v1.47/images/" + imageID + "/json":
|
||||
writeImageInspect(w, imageID, "0.2.0-rc.1")
|
||||
parts := strings.Split(imagePlatform, "/")
|
||||
writeJSON(w, map[string]any{"Id": imageID, "Os": parts[0], "Architecture": parts[1], "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": info.Version, "dsh-mgr.recipe": imageRecipeRev, "dsh-mgr.release": info.ReleaseID, "dsh-mgr.source": info.Repository, "dsh-mgr.commit": info.Commit}}})
|
||||
default:
|
||||
t.Errorf("unexpected Docker call %s", r.URL)
|
||||
w.WriteHeader(500)
|
||||
}
|
||||
})
|
||||
w := imageRequest(a.builds, sid, "POST", `{"version":"0.2.0-rc.1"}`)
|
||||
if w.Code != 202 {
|
||||
t.Fatal(w.Body.String())
|
||||
if _, err := a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='u'"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
deadline := time.Now().Add(5 * time.Second)
|
||||
for {
|
||||
var status, id, detail string
|
||||
err := a.s.DB.QueryRow("SELECT status,image_id,error FROM image_builds").Scan(&status, &id, &detail)
|
||||
if err == nil && status == "failed" {
|
||||
if id != imageID || !strings.Contains(detail, "verification failed") || !deleted.Load() {
|
||||
t.Fatalf("%s %s cleanup=%v", id, detail, deleted.Load())
|
||||
old := fetchRelease
|
||||
t.Cleanup(func() { fetchRelease = old })
|
||||
fetchRelease = func(context.Context, string, string) (releaseInfo, io.Reader, error) {
|
||||
return info, strings.NewReader("context"), nil
|
||||
}
|
||||
break
|
||||
b, err := a.enqueueRelease("u", info)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatalf("status=%s error=%v", status, err)
|
||||
a.runImageBuild(b)
|
||||
if err := scanBuildSummary(a.s.DB.QueryRow("SELECT "+buildSummaryColumns+" FROM image_builds WHERE id=?", b.ID), &b); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
time.Sleep(time.Millisecond)
|
||||
if b.Status != "failed" || b.ImageID != imageID || !strings.Contains(b.Error, "verification failed") || !deleted.Load() {
|
||||
t.Fatalf("%+v cleanup=%v", b, deleted.Load())
|
||||
}
|
||||
w = imageRequest(a.images, sid, "GET", "")
|
||||
w := imageRequest(a.images, sid, "GET", "")
|
||||
var images []imageRecord
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &images); err != nil || w.Code != 200 || len(images) != 1 || images[0].Status != "unverified" {
|
||||
t.Fatal("failed image advertised", w.Body.String())
|
||||
@@ -481,20 +247,6 @@ func TestFailedVerificationNeverAvailable(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestLocalBuildImagePrefersPersistedID(t *testing.T) {
|
||||
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path == "/v1.47/images/verified/json" {
|
||||
writeImageInspect(w, "verified", "0.2.0-rc.1")
|
||||
return
|
||||
}
|
||||
io.WriteString(w, `[{"Id":"newer","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"}},{"Id":"verified","Labels":{"dsh-mgr.role":"runtime","dsh-mgr.version":"0.2.0-rc.1","dsh-mgr.recipe":"`+imageRecipeRev+`"}}]`)
|
||||
})
|
||||
id, err := a.localBuildImage(context.Background(), "0.2.0-rc.1", "verified")
|
||||
if err != nil || id != "verified" {
|
||||
t.Fatalf("id=%s err=%v", id, err)
|
||||
}
|
||||
}
|
||||
|
||||
func writeImageInspect(w http.ResponseWriter, id, version string) {
|
||||
parts := strings.Split(imagePlatform, "/")
|
||||
writeJSON(w, map[string]any{"Id": id, "Os": parts[0], "Architecture": parts[1], "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": version, "dsh-mgr.recipe": imageRecipeRev}}})
|
||||
|
||||
@@ -1,233 +0,0 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.misaka.ren/m1saka/dsh-web/internal/store"
|
||||
)
|
||||
|
||||
func (a *API) intents(w http.ResponseWriter, r *http.Request) {
|
||||
u, _, err := a.user(r)
|
||||
if err != nil {
|
||||
fail(w, 401, "authentication required")
|
||||
return
|
||||
}
|
||||
if err = a.s.EnsureIntents(); err != nil {
|
||||
fail(w, 500, "database error")
|
||||
return
|
||||
}
|
||||
switch r.Method {
|
||||
case http.MethodGet:
|
||||
list, err := a.s.Intents(u.ID)
|
||||
if err != nil {
|
||||
fail(w, 500, "database error")
|
||||
return
|
||||
}
|
||||
writeJSON(w, list)
|
||||
case http.MethodPost:
|
||||
var in struct {
|
||||
Name string `json:"name"`
|
||||
Version string `json:"version"`
|
||||
}
|
||||
dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096))
|
||||
dec.DisallowUnknownFields()
|
||||
if dec.Decode(&in) != nil || dec.Decode(new(any)) != io.EOF || strings.TrimSpace(in.Name) == "" || len(in.Name) > 200 || !validBuildVersion(in.Version) {
|
||||
fail(w, 400, "name and fixed version required")
|
||||
return
|
||||
}
|
||||
if !normalBuildAllowed(in.Version) {
|
||||
fail(w, 400, errBuildUnsupported.Error())
|
||||
return
|
||||
}
|
||||
i := store.Intent{ID: store.ID(), UID: u.ID, Name: strings.TrimSpace(in.Name), Version: in.Version, Host: a.runtimeHost(r), Status: "waiting"}
|
||||
if err = a.s.ReserveIntent(i); err != nil {
|
||||
if errors.Is(err, store.ErrInstanceLimit) {
|
||||
fail(w, 409, err.Error())
|
||||
} else {
|
||||
fail(w, 500, "database error")
|
||||
}
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(http.StatusAccepted)
|
||||
writeJSON(w, i)
|
||||
default:
|
||||
fail(w, 405, "method not allowed")
|
||||
}
|
||||
}
|
||||
func (a *API) intent(w http.ResponseWriter, r *http.Request) {
|
||||
u, _, err := a.user(r)
|
||||
if err != nil {
|
||||
fail(w, 401, "authentication required")
|
||||
return
|
||||
}
|
||||
if r.Method != http.MethodDelete {
|
||||
fail(w, 405, "method not allowed")
|
||||
return
|
||||
}
|
||||
if err = a.s.EnsureIntents(); err != nil {
|
||||
fail(w, 500, "database error")
|
||||
return
|
||||
}
|
||||
id := strings.TrimPrefix(r.URL.Path, "/api/intents/")
|
||||
if err = a.s.CancelIntent(id, u.ID); err != nil {
|
||||
if errors.Is(err, store.ErrIntentState) {
|
||||
fail(w, 409, "intent not found or no longer waiting")
|
||||
} else {
|
||||
fail(w, 500, "database error")
|
||||
}
|
||||
return
|
||||
}
|
||||
writeJSON(w, map[string]string{"status": "cancelled"})
|
||||
}
|
||||
|
||||
// Call once at startup, after the build queue has recovered interrupted jobs.
|
||||
func (a *API) StartIntents(ctx context.Context) error {
|
||||
if err := a.s.EnsureIntents(); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := a.recoverIntents(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
go func() {
|
||||
ticker := time.NewTicker(time.Second)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
default:
|
||||
}
|
||||
a.processIntents(ctx)
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
}
|
||||
}
|
||||
}()
|
||||
return nil
|
||||
}
|
||||
|
||||
func (a *API) finishIntent(id, status, detail string) {
|
||||
_, _ = a.s.DB.Exec(`UPDATE creation_intents SET status=?,error=? WHERE id=? AND status IN ('waiting','creating')`, status, detail, id)
|
||||
}
|
||||
func (a *API) recoverIntents(ctx context.Context) error {
|
||||
list, err := a.s.Intents("")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, in := range list {
|
||||
if in.Status != "creating" {
|
||||
continue
|
||||
}
|
||||
i, err := a.s.GetInstance(in.InstanceID)
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
a.finishIntent(in.ID, "failed", "instance reservation missing after process restart")
|
||||
continue
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err == nil && i.Status == "running" {
|
||||
a.finishIntent(in.ID, "success", "")
|
||||
continue
|
||||
}
|
||||
cleanup, cancel := context.WithTimeout(ctx, 30*time.Second)
|
||||
if err == nil && i.ContainerID == "" {
|
||||
err = a.recoverRuntimeContainer(cleanup, &i)
|
||||
}
|
||||
if err == nil && i.ContainerID != "" {
|
||||
err = a.removeRuntimeContainer(cleanup, i.ContainerID)
|
||||
}
|
||||
if err == nil {
|
||||
err = a.removeRuntimeNetwork(cleanup, in.InstanceID)
|
||||
}
|
||||
cancel()
|
||||
if err != nil {
|
||||
a.s.SetInstance(in.InstanceID, "error")
|
||||
a.finishIntent(in.ID, "failed", "instance creation requires recovery: "+err.Error())
|
||||
a.s.AddAudit(in.UID, "instance.recovery_error", in.InstanceID, err.Error())
|
||||
continue
|
||||
}
|
||||
a.s.DeleteInstance(in.InstanceID)
|
||||
_, _ = a.s.DB.Exec("DELETE FROM runtime_creates WHERE instance_id=?", in.InstanceID)
|
||||
_, _ = a.s.DB.Exec("DELETE FROM runtime_networks WHERE instance_id=?", in.InstanceID)
|
||||
a.finishIntent(in.ID, "failed", "instance creation interrupted by process restart")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (a *API) processIntents(ctx context.Context) {
|
||||
list, err := a.s.Intents("")
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
for _, in := range list {
|
||||
if ctx.Err() != nil {
|
||||
return
|
||||
}
|
||||
if in.Status != "waiting" {
|
||||
continue
|
||||
}
|
||||
if in.BuildID == "" {
|
||||
b, err := a.enqueueBuild(ctx, in.UID, in.Version)
|
||||
if err != nil {
|
||||
a.finishIntent(in.ID, "failed", "build submission failed")
|
||||
continue
|
||||
}
|
||||
if _, err = a.s.DB.Exec(`UPDATE creation_intents SET build_id=? WHERE id=? AND status='waiting'`, b.ID, in.ID); err != nil {
|
||||
continue
|
||||
}
|
||||
in.BuildID = b.ID
|
||||
}
|
||||
var status, imageID, version string
|
||||
err := a.s.DB.QueryRow(`SELECT status,image_id,version FROM image_builds WHERE id=?`, in.BuildID).Scan(&status, &imageID, &version)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if version != in.Version || status == "failed" || status == "cancelled" || status == "missing" {
|
||||
a.finishIntent(in.ID, "failed", "build failed; requested version was not created")
|
||||
continue
|
||||
}
|
||||
if status != "success" {
|
||||
continue
|
||||
}
|
||||
work, cancel := context.WithTimeout(ctx, 90*time.Second)
|
||||
verified, err := a.verifiedImagesPoll(work)
|
||||
if err != nil {
|
||||
cancel()
|
||||
continue
|
||||
}
|
||||
if !runtimeImageID.MatchString(imageID) || verified[imageID] != in.Version {
|
||||
cancel()
|
||||
a.finishIntent(in.ID, "failed", "verified image for requested version unavailable")
|
||||
continue
|
||||
}
|
||||
u, err := a.s.UserByID(in.UID)
|
||||
if err != nil {
|
||||
cancel()
|
||||
a.finishIntent(in.ID, "failed", "owner unavailable")
|
||||
continue
|
||||
}
|
||||
i := store.Instance{ID: store.ID(), UID: in.UID, Name: in.Name, Version: in.Version, ImageID: imageID, Status: "creating", Created: time.Now()}
|
||||
if err = a.s.ClaimIntent(in.ID, i); err != nil {
|
||||
cancel()
|
||||
continue
|
||||
}
|
||||
_, err = a.createRuntime(work, u, i, in.Host)
|
||||
cancel()
|
||||
if err != nil {
|
||||
a.finishIntent(in.ID, "failed", err.Error())
|
||||
} else {
|
||||
a.finishIntent(in.ID, "success", "")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,166 +0,0 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"git.misaka.ren/m1saka/dsh-web/internal/store"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestIntentsReserveCancel(t *testing.T) {
|
||||
a, sid := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("reservation contacted Docker") })
|
||||
if err := a.initAuth(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for n := 0; n < 3; n++ {
|
||||
w := httptest.NewRecorder()
|
||||
a.intents(w, runtimeRequest("POST", "/api/intents", `{"name":"one","version":"0.2.0-rc.1"}`, sid))
|
||||
want := 202
|
||||
if n == 2 {
|
||||
want = 409
|
||||
}
|
||||
if w.Code != want {
|
||||
t.Fatalf("%d: %d %s", n, w.Code, w.Body.String())
|
||||
}
|
||||
}
|
||||
list, err := a.s.Intents("owner")
|
||||
if err != nil || len(list) != 2 {
|
||||
t.Fatalf("%v %v", list, err)
|
||||
}
|
||||
w := httptest.NewRecorder()
|
||||
a.intent(w, runtimeRequest("DELETE", "/api/intents/"+list[0].ID, "", sid))
|
||||
if w.Code != 200 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
if err := a.s.ReserveIntent(store.Intent{ID: store.ID(), UID: "owner"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIntentBuildSuccessUsesRealRuntime(t *testing.T) {
|
||||
if os.Geteuid() != 0 {
|
||||
t.Skip("runtime filesystem ownership requires root")
|
||||
}
|
||||
image := "sha256:" + strings.Repeat("a", 64)
|
||||
creates := 0
|
||||
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
switch {
|
||||
case strings.Contains(r.URL.Path, "/images/"):
|
||||
writeJSON(w, map[string]any{"Id": image, "Os": "linux", "Architecture": "amd64", "Config": map[string]any{"Labels": map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": "0.2.0-rc.1", "dsh-mgr.recipe": imageRecipeRev}}})
|
||||
case strings.HasSuffix(r.URL.Path, "/containers/create"):
|
||||
creates++
|
||||
writeJSON(w, map[string]string{"Id": "runtime-created"})
|
||||
default:
|
||||
w.WriteHeader(204)
|
||||
}
|
||||
})
|
||||
seedVerifiedImage(t, a, image, "0.2.0-rc.1")
|
||||
if err := a.s.EnsureIntents(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var build string
|
||||
if err := a.s.DB.QueryRow("SELECT id FROM image_builds WHERE version='0.2.0-rc.1'").Scan(&build); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
in := store.Intent{ID: store.ID(), UID: "owner", Name: "one", Version: "0.2.0-rc.1", Host: "panel.test"}
|
||||
if err := a.s.ReserveIntent(in); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := a.s.DB.Exec("UPDATE creation_intents SET build_id=? WHERE id=?", build, in.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
a.processIntents(context.Background())
|
||||
a.processIntents(context.Background())
|
||||
list, err := a.s.Intents("owner")
|
||||
if err != nil || len(list) != 1 || list[0].Status != "success" || creates != 1 {
|
||||
t.Fatalf("%+v err=%v creates=%d", list, err, creates)
|
||||
}
|
||||
instances, err := a.s.Instances("owner")
|
||||
if err != nil || len(instances) != 1 || instances[0].ImageID != image || instances[0].Version != "0.2.0-rc.1" {
|
||||
t.Fatalf("%+v %v", instances, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIntentFailedBuildReleasesAndCancelPreservesBuild(t *testing.T) {
|
||||
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("failed intent contacted Docker") })
|
||||
seedVerifiedImage(t, a, "sha256:"+strings.Repeat("b", 64), "0.2.0-rc.1")
|
||||
if err := a.s.EnsureIntents(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var build string
|
||||
a.s.DB.QueryRow("SELECT id FROM image_builds LIMIT 1").Scan(&build)
|
||||
for _, id := range []string{"cancel", "fail"} {
|
||||
if err := a.s.ReserveIntent(store.Intent{ID: id, UID: "owner", Version: "0.2.0-rc.1"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
a.s.DB.Exec("UPDATE creation_intents SET build_id=? WHERE id=?", build, id)
|
||||
}
|
||||
if err := a.s.CancelIntent("cancel", "owner"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var status string
|
||||
a.s.DB.QueryRow("SELECT status FROM image_builds WHERE id=?", build).Scan(&status)
|
||||
if status != "success" {
|
||||
t.Fatal(status)
|
||||
}
|
||||
a.s.DB.Exec("UPDATE image_builds SET status='failed' WHERE id=?", build)
|
||||
a.processIntents(context.Background())
|
||||
for n := 0; n < 2; n++ {
|
||||
if err := a.s.ReserveIntent(store.Intent{ID: store.ID(), UID: "owner"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestIntentRestartCleansInterruptedCreationAndRetainsWaiting(t *testing.T) {
|
||||
deletes := 0
|
||||
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method == "GET" && r.URL.Path == "/v1.47/networks/dsh-inst-reserved" {
|
||||
w.Write([]byte(`{"Id":"network","Name":"dsh-inst-reserved","Driver":"bridge","Labels":{"dsh-mgr.instance":"reserved"}}`))
|
||||
return
|
||||
}
|
||||
if r.Method == "GET" && r.URL.Path == "/v1.47/containers/dsh-inst-reserved/json" {
|
||||
w.Write([]byte(`{"Id":"recovered","Name":"/dsh-inst-reserved","Config":{"Labels":{"dsh-mgr.instance":"reserved","dsh-mgr.owner":"owner","dsh-mgr.role":"dsh"}}}`))
|
||||
return
|
||||
}
|
||||
if r.Method == "DELETE" {
|
||||
deletes++
|
||||
}
|
||||
w.WriteHeader(204)
|
||||
})
|
||||
if err := a.s.EnsureIntents(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, id := range []string{"waiting", "creating"} {
|
||||
if err := a.s.ReserveIntent(store.Intent{ID: id, UID: "owner", Version: "0.2.0-rc.1"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := a.s.ClaimIntent("creating", store.Instance{ID: "reserved", UID: "owner", Version: "0.2.0-rc.1", Status: "creating"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := a.recoverIntents(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if deletes != 2 {
|
||||
t.Fatalf("cleanup calls=%d", deletes)
|
||||
}
|
||||
list, err := a.s.Intents("owner")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, in := range list {
|
||||
if in.ID == "waiting" && in.Status != "waiting" {
|
||||
t.Fatal(in)
|
||||
}
|
||||
if in.ID == "creating" && in.Status != "failed" {
|
||||
t.Fatal(in)
|
||||
}
|
||||
}
|
||||
if err := a.s.ReserveIntent(store.Intent{ID: store.ID(), UID: "owner"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,198 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"git.misaka.ren/m1saka/dsh-web/internal/store"
|
||||
"io"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"time"
|
||||
)
|
||||
|
||||
var fetchRelease = readRelease
|
||||
|
||||
func (a *API) latestRelease(w http.ResponseWriter, r *http.Request) {
|
||||
u, _, err := a.user(r)
|
||||
if err != nil {
|
||||
fail(w, 401, "authentication required")
|
||||
return
|
||||
}
|
||||
if !u.Admin {
|
||||
fail(w, 403, "administrator required")
|
||||
return
|
||||
}
|
||||
if r.Method != http.MethodGet {
|
||||
fail(w, 405, "method not allowed")
|
||||
return
|
||||
}
|
||||
repository, branch, err := releaseSource()
|
||||
if err != nil {
|
||||
fail(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
info, _, err := fetchRelease(r.Context(), repository, "refs/heads/"+branch)
|
||||
if err != nil && !errors.Is(err, errReleaseIncompatible) {
|
||||
fail(w, 502, sanitizeBuildLog(err.Error()))
|
||||
return
|
||||
}
|
||||
compatible := err == nil
|
||||
reason := "trusted release supports current layout and platform"
|
||||
if !compatible {
|
||||
reason = errReleaseIncompatible.Error()
|
||||
}
|
||||
buildQueue.Lock()
|
||||
err = a.ensureBuilds()
|
||||
buildQueue.Unlock()
|
||||
if err != nil {
|
||||
fail(w, 500, "database error")
|
||||
return
|
||||
}
|
||||
var b buildRecord
|
||||
err = scanBuildSummary(a.s.DB.QueryRow("SELECT "+buildSummaryColumns+" FROM image_builds WHERE source_repository=? AND source_commit=? AND release_id=? AND platform=? ORDER BY created_at DESC,id DESC LIMIT 1", repository, info.Commit, info.ReleaseID, imagePlatform), &b)
|
||||
if err != nil && !errors.Is(err, sql.ErrNoRows) {
|
||||
fail(w, 500, "database error")
|
||||
return
|
||||
}
|
||||
result := map[string]any{"release_id": info.ReleaseID, "version": info.Version, "commit": info.Commit, "repository": repository, "branch": branch, "recipe_rev": strconv.Itoa(info.DeploymentSchema), "platforms": info.Platforms, "compatible": compatible, "reason": reason}
|
||||
if err == nil {
|
||||
result["build"] = b
|
||||
}
|
||||
writeJSON(w, result)
|
||||
}
|
||||
|
||||
func (a *API) releaseBuild(w http.ResponseWriter, r *http.Request) {
|
||||
u, _, err := a.user(r)
|
||||
if err != nil {
|
||||
fail(w, 401, "authentication required")
|
||||
return
|
||||
}
|
||||
if !u.Admin {
|
||||
fail(w, 403, "administrator required")
|
||||
return
|
||||
}
|
||||
if r.Method != http.MethodPost {
|
||||
fail(w, 405, "method not allowed")
|
||||
return
|
||||
}
|
||||
var in struct {
|
||||
Commit string `json:"commit"`
|
||||
}
|
||||
dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096))
|
||||
dec.DisallowUnknownFields()
|
||||
if dec.Decode(&in) != nil || dec.Decode(new(any)) != io.EOF || !releaseCommit.MatchString(in.Commit) {
|
||||
fail(w, 400, "fixed commit required")
|
||||
return
|
||||
}
|
||||
repository, branch, err := releaseSource()
|
||||
if err != nil {
|
||||
fail(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
info, _, err := fetchRelease(r.Context(), repository, "refs/heads/"+branch)
|
||||
if err != nil {
|
||||
fail(w, 502, sanitizeBuildLog(err.Error()))
|
||||
return
|
||||
}
|
||||
if info.Commit != in.Commit {
|
||||
fail(w, 409, "release changed; check the latest release again")
|
||||
return
|
||||
}
|
||||
b, err := a.enqueueRelease(u.ID, info)
|
||||
if err != nil {
|
||||
status := 500
|
||||
if errors.Is(err, errBuildRateLimited) {
|
||||
status = 429
|
||||
}
|
||||
fail(w, status, sanitizeBuildLog(err.Error()))
|
||||
return
|
||||
}
|
||||
a.startBuildWorker()
|
||||
w.WriteHeader(http.StatusAccepted)
|
||||
writeJSON(w, b)
|
||||
}
|
||||
|
||||
func (a *API) enqueueRelease(uid string, info releaseInfo) (buildRecord, error) {
|
||||
buildQueue.Lock()
|
||||
defer buildQueue.Unlock()
|
||||
var b buildRecord
|
||||
if err := a.ensureBuilds(); err != nil {
|
||||
return b, err
|
||||
}
|
||||
err := scanBuildSummary(a.s.DB.QueryRow("SELECT "+buildSummaryColumns+" FROM image_builds WHERE source_repository=? AND source_commit=? AND release_id=? AND platform=? AND status IN ('queued','running','validating','success')", info.Repository, info.Commit, info.ReleaseID, imagePlatform), &b)
|
||||
if err == nil {
|
||||
if b.Status == "success" {
|
||||
buildQueue.Unlock()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
|
||||
matches, verifyErr := a.buildImageMatches(ctx, b.ImageID, b)
|
||||
cancel()
|
||||
buildQueue.Lock()
|
||||
if verifyErr != nil {
|
||||
return b, verifyErr
|
||||
}
|
||||
if !matches {
|
||||
if _, err = a.s.DB.Exec("UPDATE image_builds SET status='missing',error='image no longer available' WHERE id=? AND status='success'", b.ID); err != nil {
|
||||
return b, err
|
||||
}
|
||||
buildQueue.Unlock()
|
||||
b, err = a.enqueueRelease(uid, info)
|
||||
buildQueue.Lock()
|
||||
return b, err
|
||||
}
|
||||
}
|
||||
return b, nil
|
||||
}
|
||||
if !errors.Is(err, sql.ErrNoRows) {
|
||||
return b, err
|
||||
}
|
||||
var count int
|
||||
if err = a.s.DB.QueryRow("SELECT count(*) FROM image_build_requests WHERE user_id=? AND created_at>?", uid, time.Now().Add(-time.Hour).UTC().Format(time.RFC3339Nano)).Scan(&count); err != nil {
|
||||
return b, err
|
||||
}
|
||||
if count >= 3 {
|
||||
return b, errBuildRateLimited
|
||||
}
|
||||
b = buildRecord{ID: store.ID(), Version: info.Version, Status: "queued", Created: time.Now().UTC().Format(time.RFC3339Nano), RequestedBy: uid, RecipeRev: imageRecipeRev, Platform: imagePlatform, ReleaseID: info.ReleaseID, SourceRepository: info.Repository, SourceCommit: info.Commit}
|
||||
return b, a.insertBuildRequest(b, uid)
|
||||
}
|
||||
|
||||
func (a *API) releaseBuildContext(ctx context.Context, b buildRecord) (io.Reader, error) {
|
||||
repository, _, err := releaseSource()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if repository != b.SourceRepository || !releaseCommit.MatchString(b.SourceCommit) {
|
||||
return nil, errors.New("queued release source is no longer trusted")
|
||||
}
|
||||
u, err := a.s.UserByID(b.RequestedBy)
|
||||
if err != nil || !u.Admin {
|
||||
return nil, errors.New("release requester is not an administrator")
|
||||
}
|
||||
info, archive, err := fetchRelease(ctx, repository, b.SourceCommit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if info.Commit != b.SourceCommit || info.ReleaseID != b.ReleaseID || info.Version != b.Version {
|
||||
return nil, errors.New("pinned release manifest mismatch")
|
||||
}
|
||||
return archive, nil
|
||||
}
|
||||
func releaseLabelsMatch(im *inspectedImage, b buildRecord) bool {
|
||||
if b.ReleaseID == "" {
|
||||
return im.Config.Labels["dsh-mgr.release"] == ""
|
||||
}
|
||||
return im.Config.Labels["dsh-mgr.release"] == b.ReleaseID && im.Config.Labels["dsh-mgr.source"] == b.SourceRepository && im.Config.Labels["dsh-mgr.commit"] == b.SourceCommit
|
||||
}
|
||||
func (a *API) buildImageMatches(ctx context.Context, id string, b buildRecord) (bool, error) {
|
||||
matches, err := a.imageMatches(ctx, id, b.Version)
|
||||
if err != nil || !matches {
|
||||
return matches, err
|
||||
}
|
||||
im, err := a.inspectLocalImage(ctx, id)
|
||||
if err != nil || im == nil {
|
||||
return false, err
|
||||
}
|
||||
return releaseLabelsMatch(im, b), nil
|
||||
}
|
||||
@@ -0,0 +1,217 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"net/url"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path"
|
||||
"regexp"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
type releaseManifest struct {
|
||||
SchemaVersion int `json:"schema_version"`
|
||||
ReleaseID string `json:"release_id"`
|
||||
Version string `json:"version"`
|
||||
DeploymentSchema int `json:"deployment_schema"`
|
||||
Platforms []string `json:"platforms"`
|
||||
}
|
||||
type releaseInfo struct {
|
||||
releaseManifest
|
||||
Repository string `json:"source_repository"`
|
||||
Commit string `json:"source_commit"`
|
||||
}
|
||||
|
||||
var errReleaseIncompatible = errors.New("release layout or platform is incompatible")
|
||||
|
||||
var releaseCommit = regexp.MustCompile(`^[0-9a-f]{40}$`)
|
||||
var releaseID = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$`)
|
||||
|
||||
const releaseLimit = 32 << 20
|
||||
|
||||
func releaseSource() (string, string, error) {
|
||||
repository := os.Getenv("DSH_RELEASE_REPOSITORY")
|
||||
if repository == "" {
|
||||
repository = "https://git.misaka.ren/m1saka/dsh-web.git"
|
||||
}
|
||||
branch := os.Getenv("DSH_RELEASE_BRANCH")
|
||||
if branch == "" {
|
||||
branch = "dockerfile"
|
||||
}
|
||||
u, err := url.Parse(repository)
|
||||
if err != nil || u.Scheme != "https" || u.Hostname() == "" || u.User != nil || u.RawQuery != "" || u.Fragment != "" || strings.ContainsAny(repository, "\r\n") || !regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._/-]*$`).MatchString(branch) || strings.Contains(branch, "..") {
|
||||
return "", "", errors.New("invalid trusted release source configuration")
|
||||
}
|
||||
return repository, branch, nil
|
||||
}
|
||||
|
||||
type boundedReleaseBuffer struct{ bytes.Buffer }
|
||||
|
||||
func (b *boundedReleaseBuffer) Write(p []byte) (int, error) {
|
||||
if b.Len()+len(p) > releaseLimit {
|
||||
return 0, errors.New("release exceeds size limit")
|
||||
}
|
||||
return b.Buffer.Write(p)
|
||||
}
|
||||
func gitReleaseCommand(ctx context.Context, dir string, args ...string) ([]byte, error) {
|
||||
gitArgs := append([]string{"-c", "credential.helper=", "-c", "http.followRedirects=false", "-c", "core.hooksPath=/dev/null", "-c", "protocol.allow=never", "-c", "protocol.https.allow=always", "-c", "fetch.unpackLimit=1"}, args...)
|
||||
// The inherited file-size limit also bounds Git's on-disk pack, not just stdout.
|
||||
cmd := exec.CommandContext(ctx, "/bin/sh", append([]string{"-c", "ulimit -f 32768; exec git \"$@\"", "git"}, gitArgs...)...)
|
||||
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
|
||||
cmd.Cancel = func() error { return syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL) }
|
||||
cmd.Dir = dir
|
||||
cmd.Env = []string{"PATH=" + os.Getenv("PATH"), "GIT_CONFIG_NOSYSTEM=1", "GIT_CONFIG_GLOBAL=/dev/null", "GIT_TERMINAL_PROMPT=0", "GIT_LFS_SKIP_SMUDGE=1"}
|
||||
cmd.WaitDelay = time.Second
|
||||
var out boundedReleaseBuffer
|
||||
cmd.Stdout = &out
|
||||
if err := cmd.Run(); err != nil {
|
||||
return nil, errors.New("trusted release Git operation failed")
|
||||
}
|
||||
return out.Bytes(), nil
|
||||
}
|
||||
|
||||
var releaseFetchSlot = make(chan struct{}, 1)
|
||||
|
||||
func readRelease(ctx context.Context, repository, ref string) (releaseInfo, io.Reader, error) {
|
||||
var info releaseInfo
|
||||
ctx, cancel := context.WithTimeout(ctx, 45*time.Second)
|
||||
defer cancel()
|
||||
select {
|
||||
case releaseFetchSlot <- struct{}{}:
|
||||
defer func() { <-releaseFetchSlot }()
|
||||
case <-ctx.Done():
|
||||
return info, nil, ctx.Err()
|
||||
}
|
||||
dir, err := os.MkdirTemp("", "dsh-release-")
|
||||
if err != nil {
|
||||
return info, nil, err
|
||||
}
|
||||
defer os.RemoveAll(dir)
|
||||
if _, err = gitReleaseCommand(ctx, dir, "init", "--quiet"); err != nil {
|
||||
return info, nil, err
|
||||
}
|
||||
if _, err = gitReleaseCommand(ctx, dir, "fetch", "--quiet", "--depth=1", "--no-tags", repository, ref); err != nil {
|
||||
return info, nil, err
|
||||
}
|
||||
commit, err := gitReleaseCommand(ctx, dir, "rev-parse", "FETCH_HEAD^{commit}")
|
||||
if err != nil {
|
||||
return info, nil, err
|
||||
}
|
||||
info.Commit = strings.TrimSpace(string(commit))
|
||||
info.Repository = repository
|
||||
if !releaseCommit.MatchString(info.Commit) || (releaseCommit.MatchString(ref) && info.Commit != ref) {
|
||||
return info, nil, errors.New("release commit mismatch")
|
||||
}
|
||||
archive, err := gitReleaseCommand(ctx, dir, "archive", "--format=tar", info.Commit, "--", "release.json", "image")
|
||||
if err != nil {
|
||||
return info, nil, err
|
||||
}
|
||||
return parseReleaseArchive(info, archive)
|
||||
}
|
||||
|
||||
func parseReleaseArchive(info releaseInfo, archive []byte) (releaseInfo, io.Reader, error) {
|
||||
var manifest []byte
|
||||
tr := tar.NewReader(bytes.NewReader(archive))
|
||||
for {
|
||||
h, err := tr.Next()
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return info, nil, err
|
||||
}
|
||||
if h.Name == "release.json" && h.Typeflag == tar.TypeReg {
|
||||
if h.Size > 16384 {
|
||||
return info, nil, errors.New("release manifest too large")
|
||||
}
|
||||
manifest, err = io.ReadAll(tr)
|
||||
if err != nil {
|
||||
return info, nil, err
|
||||
}
|
||||
}
|
||||
}
|
||||
dec := json.NewDecoder(bytes.NewReader(manifest))
|
||||
dec.DisallowUnknownFields()
|
||||
if dec.Decode(&info.releaseManifest) != nil || dec.Decode(new(any)) != io.EOF {
|
||||
return info, nil, errors.New("invalid release manifest")
|
||||
}
|
||||
m := info.releaseManifest
|
||||
compatible := false
|
||||
for _, p := range m.Platforms {
|
||||
if p == imagePlatform {
|
||||
compatible = true
|
||||
}
|
||||
}
|
||||
if m.SchemaVersion != 1 || !releaseID.MatchString(m.ReleaseID) || !validBuildVersion(m.Version) || m.DeploymentSchema < 1 || len(m.Platforms) == 0 {
|
||||
return info, nil, errors.New("unsupported release manifest or platform")
|
||||
}
|
||||
if m.DeploymentSchema != 7 || !compatible {
|
||||
return info, nil, errReleaseIncompatible
|
||||
}
|
||||
var out bytes.Buffer
|
||||
tw := tar.NewWriter(&out)
|
||||
tr = tar.NewReader(bytes.NewReader(archive))
|
||||
dockerfile := false
|
||||
for {
|
||||
h, err := tr.Next()
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return info, nil, err
|
||||
}
|
||||
name := h.Name
|
||||
if !strings.HasPrefix(name, "image/") {
|
||||
continue
|
||||
}
|
||||
name = strings.TrimPrefix(name, "image/")
|
||||
if name == "" {
|
||||
continue
|
||||
}
|
||||
if path.IsAbs(name) || path.Clean(strings.TrimSuffix(name, "/")) != strings.TrimSuffix(name, "/") || name == ".." || strings.HasPrefix(name, "../") {
|
||||
return info, nil, errors.New("unsafe release context path")
|
||||
}
|
||||
excluded := false
|
||||
for _, part := range strings.Split(name, "/") {
|
||||
if part == ".git" || part == ".env" || strings.HasPrefix(part, ".env.") || part == ".netrc" || part == ".npmrc" || part == ".ssh" || part == ".git-credentials" {
|
||||
excluded = true
|
||||
}
|
||||
}
|
||||
if excluded {
|
||||
continue
|
||||
}
|
||||
if h.Typeflag != tar.TypeReg && h.Typeflag != tar.TypeDir {
|
||||
return info, nil, errors.New("release context links and special files are not supported")
|
||||
}
|
||||
h.Name = name
|
||||
h.PAXRecords = nil
|
||||
h.Uname = ""
|
||||
h.Gname = ""
|
||||
h.Uid = 0
|
||||
h.Gid = 0
|
||||
if name == "Dockerfile" && h.Typeflag == tar.TypeReg {
|
||||
dockerfile = true
|
||||
}
|
||||
if err = tw.WriteHeader(h); err != nil {
|
||||
return info, nil, err
|
||||
}
|
||||
if _, err = io.Copy(tw, tr); err != nil {
|
||||
return info, nil, err
|
||||
}
|
||||
}
|
||||
if !dockerfile {
|
||||
return info, nil, errors.New("release context Dockerfile missing")
|
||||
}
|
||||
if err := tw.Close(); err != nil {
|
||||
return info, nil, err
|
||||
}
|
||||
return info, bytes.NewReader(out.Bytes()), nil
|
||||
}
|
||||
@@ -0,0 +1,302 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func testReleaseArchive(t *testing.T, m string, files map[string]string) []byte {
|
||||
t.Helper()
|
||||
var b bytes.Buffer
|
||||
tw := tar.NewWriter(&b)
|
||||
files["release.json"] = m
|
||||
for name, data := range files {
|
||||
if err := tw.WriteHeader(&tar.Header{Name: name, Size: int64(len(data)), Mode: 0644}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := tw.Write([]byte(data)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := tw.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return b.Bytes()
|
||||
}
|
||||
func releaseFixture() releaseInfo {
|
||||
return releaseInfo{releaseManifest: releaseManifest{SchemaVersion: 1, ReleaseID: "runtime-7.1", Version: "0.2.0-rc.3", DeploymentSchema: 7, Platforms: []string{imagePlatform}}, Repository: "https://git.misaka.ren/m1saka/dsh-web.git", Commit: strings.Repeat("a", 40)}
|
||||
}
|
||||
func TestReleaseManifestAndContext(t *testing.T) {
|
||||
info := releaseFixture()
|
||||
data, _ := json.Marshal(info.releaseManifest)
|
||||
for _, tc := range []struct {
|
||||
name, manifest string
|
||||
files map[string]string
|
||||
wantErr bool
|
||||
}{
|
||||
{"valid", string(data), map[string]string{"image/Dockerfile": "FROM scratch", "image/nested/helper": "complete", "image/.git/config": "secret", "image/.env": "secret"}, false},
|
||||
{"malformed", "{", map[string]string{"image/Dockerfile": "FROM scratch"}, true},
|
||||
{"unknown", strings.Replace(string(data), "schema_version", "unknown", 1), map[string]string{"image/Dockerfile": "FROM scratch"}, true},
|
||||
{"incompatible", strings.Replace(string(data), "deployment_schema\":7", "deployment_schema\":8", 1), map[string]string{"image/Dockerfile": "FROM scratch"}, true},
|
||||
{"missing Dockerfile", string(data), map[string]string{"image/helper": "x"}, true},
|
||||
{"unsafe path", string(data), map[string]string{"image/Dockerfile": "FROM scratch", "image/../escape": "x"}, true},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
_, r, err := parseReleaseArchive(info, testReleaseArchive(t, tc.manifest, tc.files))
|
||||
if (err != nil) != tc.wantErr {
|
||||
t.Fatalf("err=%v", err)
|
||||
}
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
tr := tar.NewReader(r)
|
||||
seen := map[string]bool{}
|
||||
for {
|
||||
h, e := tr.Next()
|
||||
if e == io.EOF {
|
||||
break
|
||||
}
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
seen[h.Name] = true
|
||||
}
|
||||
if !seen["nested/helper"] || seen[".env"] || seen[".git/config"] {
|
||||
t.Fatal(seen)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
func TestReleaseQueueIdentityRecoveryAndFailure(t *testing.T) {
|
||||
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(404) })
|
||||
info := releaseFixture()
|
||||
b, err := a.enqueueRelease("u", info)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
again, err := a.enqueueRelease("u", info)
|
||||
if err != nil || again.ID != b.ID {
|
||||
t.Fatalf("repeat %v %v", again, err)
|
||||
}
|
||||
info.Commit = strings.Repeat("b", 40)
|
||||
info.ReleaseID = "runtime-7.2"
|
||||
next, err := a.enqueueRelease("u", info)
|
||||
if err != nil || next.ID == b.ID || next.Version != b.Version {
|
||||
t.Fatalf("next %v %v", next, err)
|
||||
}
|
||||
buildQueue.Lock()
|
||||
delete(buildQueue.initialized, a.s.DB)
|
||||
err = a.ensureBuilds()
|
||||
buildQueue.Unlock()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var recovered buildRecord
|
||||
if err = scanBuildSummary(a.s.DB.QueryRow("SELECT "+buildSummaryColumns+" FROM image_builds WHERE id=?", next.ID), &recovered); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if recovered.Status != "queued" || recovered.SourceCommit != info.Commit {
|
||||
t.Fatal(recovered)
|
||||
}
|
||||
if _, err = a.s.DB.Exec("UPDATE image_builds SET status='failed' WHERE id=?", next.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
retry, err := a.enqueueRelease("u", info)
|
||||
if err != nil || retry.ID == next.ID {
|
||||
t.Fatalf("retry %v %v", retry, err)
|
||||
}
|
||||
}
|
||||
func TestReleaseEndpointsAuthorizationFailureAndMismatch(t *testing.T) {
|
||||
a, sid := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(404) })
|
||||
if w := imageRequest(a.latestRelease, sid, "GET", ""); w.Code != 403 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
if _, err := a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='u'"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
old := fetchRelease
|
||||
t.Cleanup(func() { fetchRelease = old })
|
||||
fetchRelease = func(context.Context, string, string) (releaseInfo, io.Reader, error) {
|
||||
return releaseInfo{}, nil, errors.New("network failure")
|
||||
}
|
||||
if w := imageRequest(a.latestRelease, sid, "GET", ""); w.Code != 502 {
|
||||
t.Fatalf("%d %s", w.Code, w.Body.String())
|
||||
}
|
||||
fetchRelease = func(context.Context, string, string) (releaseInfo, io.Reader, error) {
|
||||
return releaseFixture(), nil, nil
|
||||
}
|
||||
if w := imageRequest(a.releaseBuild, sid, "POST", `{"commit":"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"}`); w.Code != 409 {
|
||||
t.Fatalf("%d %s", w.Code, w.Body.String())
|
||||
}
|
||||
fetchRelease = func(context.Context, string, string) (releaseInfo, io.Reader, error) {
|
||||
return releaseFixture(), nil, errReleaseIncompatible
|
||||
}
|
||||
if w := imageRequest(a.latestRelease, sid, "GET", ""); w.Code != 200 || !strings.Contains(w.Body.String(), `"compatible":false`) {
|
||||
t.Fatalf("%d %s", w.Code, w.Body.String())
|
||||
}
|
||||
}
|
||||
func TestReleaseGitEnvironmentAndFailure(t *testing.T) {
|
||||
out, err := gitReleaseCommand(context.Background(), t.TempDir(), "--version")
|
||||
if err != nil || !strings.HasPrefix(string(out), "git version") {
|
||||
t.Fatalf("%q %v", out, err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
if _, err = gitReleaseCommand(ctx, t.TempDir(), "--version"); err == nil {
|
||||
t.Fatal("cancel ignored")
|
||||
}
|
||||
for _, repository := range []string{"http://example.test/repo", "https://user:[email protected]/repo", "https://example.test/repo?token=secret"} {
|
||||
t.Setenv("DSH_RELEASE_REPOSITORY", repository)
|
||||
if _, _, err = releaseSource(); err == nil {
|
||||
t.Fatal("accepted unsafe source")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPinnedReleaseWorkerAdmissionAndBlock(t *testing.T) {
|
||||
info := releaseFixture()
|
||||
labels := map[string]string{"dsh-mgr.role": "runtime", "dsh-mgr.version": info.Version, "dsh-mgr.recipe": "7", "dsh-mgr.release": info.ReleaseID, "dsh-mgr.source": info.Repository, "dsh-mgr.commit": info.Commit}
|
||||
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
switch r.URL.Path {
|
||||
case "/v1.47/build":
|
||||
var got map[string]string
|
||||
if err := json.Unmarshal([]byte(r.URL.Query().Get("labels")), &got); err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
if got["dsh-mgr.commit"] != info.Commit || r.URL.Query().Get("t") != "dsh-web:release-"+info.Commit {
|
||||
t.Error("missing provenance", r.URL)
|
||||
}
|
||||
data, _ := io.ReadAll(r.Body)
|
||||
if !bytes.Contains(data, []byte("pinned context")) {
|
||||
t.Error("wrong build context")
|
||||
}
|
||||
io.WriteString(w, `{"aux":{"ID":"sha256:release"}}`)
|
||||
case "/v1.47/images/sha256:release/json":
|
||||
parts := strings.Split(imagePlatform, "/")
|
||||
writeJSON(w, map[string]any{"Id": "sha256:release", "Os": parts[0], "Architecture": parts[1], "Config": map[string]any{"Labels": labels}})
|
||||
case "/v1.47/containers/create":
|
||||
var cfg struct {
|
||||
Image, User string
|
||||
HostConfig struct {
|
||||
NetworkMode string
|
||||
ReadonlyRootfs bool
|
||||
Tmpfs map[string]string
|
||||
}
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&cfg); err != nil || cfg.Image != "sha256:release" || cfg.User != "1000:1000" || cfg.HostConfig.NetworkMode != "none" || !cfg.HostConfig.ReadonlyRootfs || !strings.Contains(cfg.HostConfig.Tmpfs["/tmp"], "exec") {
|
||||
t.Errorf("unsafe verification config %+v err=%v", cfg, err)
|
||||
}
|
||||
io.WriteString(w, `{"Id":"check"}`)
|
||||
case "/v1.47/containers/check/start", "/v1.47/exec/probe/start", "/v1.47/containers/check":
|
||||
w.WriteHeader(204)
|
||||
case "/v1.47/containers/check/exec":
|
||||
var cfg struct{ Cmd []string }
|
||||
expected, _ := json.Marshal(info.Version)
|
||||
if err := json.NewDecoder(r.Body).Decode(&cfg); err != nil || len(cfg.Cmd) != 3 || !strings.Contains(cfg.Cmd[2], "execFileSync('dsh',['--version']") || !strings.Contains(cfg.Cmd[2], "actual!=="+string(expected)) {
|
||||
t.Errorf("missing exact version check %+v err=%v", cfg, err)
|
||||
}
|
||||
io.WriteString(w, `{"Id":"probe"}`)
|
||||
case "/v1.47/exec/probe/json":
|
||||
io.WriteString(w, `{"Running":false,"ExitCode":0}`)
|
||||
case "/v1.47/containers/check/json":
|
||||
io.WriteString(w, `{"State":{"Running":true}}`)
|
||||
default:
|
||||
t.Errorf("unexpected %s", r.URL)
|
||||
w.WriteHeader(500)
|
||||
}
|
||||
})
|
||||
if _, err := a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='u'"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
old := fetchRelease
|
||||
t.Cleanup(func() { fetchRelease = old })
|
||||
fetchRelease = func(ctx context.Context, repository, ref string) (releaseInfo, io.Reader, error) {
|
||||
if repository != info.Repository || ref != info.Commit {
|
||||
t.Error("not pinned", repository, ref)
|
||||
}
|
||||
return info, strings.NewReader("pinned context"), nil
|
||||
}
|
||||
b, err := a.enqueueRelease("u", info)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
a.runImageBuild(b)
|
||||
if err = scanBuildSummary(a.s.DB.QueryRow("SELECT "+buildSummaryColumns+" FROM image_builds WHERE id=?", b.ID), &b); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if b.Status != "success" {
|
||||
t.Fatalf("%+v", b)
|
||||
}
|
||||
im, err := a.evaluateImage(context.Background(), b.ImageID)
|
||||
if err != nil || !im.Selectable {
|
||||
t.Fatalf("%+v %v", im, err)
|
||||
}
|
||||
if _, err = a.s.DB.Exec("INSERT INTO image_admissions(image_id,version,recipe_rev,platform,state,evidence,admitted_by,admitted_at) VALUES(?,?,?,?,?,?,?,?)", b.ImageID, b.Version, "7", imagePlatform, "blocked", "explicit block", "u", "now"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
im, err = a.evaluateImage(context.Background(), b.ImageID)
|
||||
if err != nil || im.Selectable || im.Admission != "blocked" {
|
||||
t.Fatalf("%+v %v", im, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReleaseRateLimitDedup(t *testing.T) {
|
||||
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("queued release contacted Docker") })
|
||||
info := releaseFixture()
|
||||
first, err := a.enqueueRelease("u", info)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, commit := range []string{strings.Repeat("b", 40), strings.Repeat("c", 40)} {
|
||||
next := info
|
||||
next.Commit = commit
|
||||
if _, err := a.enqueueRelease("u", next); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
again, err := a.enqueueRelease("u", info)
|
||||
if err != nil || again.ID != first.ID {
|
||||
t.Fatal(again, err)
|
||||
}
|
||||
info.Commit = strings.Repeat("d", 40)
|
||||
if _, err := a.enqueueRelease("u", info); err != errBuildRateLimited {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var count int
|
||||
if err := a.s.DB.QueryRow("SELECT count(*) FROM image_build_requests").Scan(&count); err != nil || count != 3 {
|
||||
t.Fatal(count, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReleaseUnknownDiskBlocksBuild(t *testing.T) {
|
||||
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Errorf("unexpected Docker request %s", r.URL) })
|
||||
t.Setenv("DSH_DOCKER_ROOT_VISIBLE", "")
|
||||
if _, err := a.s.DB.Exec("UPDATE users SET admin=1 WHERE id='u'"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
info := releaseFixture()
|
||||
old := fetchRelease
|
||||
t.Cleanup(func() { fetchRelease = old })
|
||||
fetchRelease = func(context.Context, string, string) (releaseInfo, io.Reader, error) {
|
||||
return info, strings.NewReader("context"), nil
|
||||
}
|
||||
b, err := a.enqueueRelease("u", info)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
a.runImageBuild(b)
|
||||
if err := scanBuildSummary(a.s.DB.QueryRow("SELECT "+buildSummaryColumns+" FROM image_builds WHERE id=?", b.ID), &b); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if b.Status != "failed" || !strings.Contains(b.Error, "capacity unknown") {
|
||||
t.Fatalf("%+v", b)
|
||||
}
|
||||
}
|
||||
@@ -277,7 +277,11 @@ func (a *API) persistBuildResult(b buildRecord, status, logs, detail, image, fin
|
||||
}
|
||||
defer tx.Rollback()
|
||||
if status == "success" && image != "" {
|
||||
_, err = tx.Exec("INSERT INTO images(image_id,version,recipe_rev,platform,tag,status,built_at) VALUES(?,?,?,?,?,?,?) ON CONFLICT(image_id) DO UPDATE SET version=excluded.version,recipe_rev=excluded.recipe_rev,platform=excluded.platform,tag=excluded.tag,status=excluded.status,built_at=excluded.built_at", image, b.Version, imageRecipeRev, imagePlatform, "dsh-web:"+b.Version+"-"+imageRecipeRev, "unverified", finished)
|
||||
tag := "dsh-web:" + b.Version + "-" + imageRecipeRev
|
||||
if b.ReleaseID != "" {
|
||||
tag = "dsh-web:release-" + b.SourceCommit
|
||||
}
|
||||
_, err = tx.Exec("INSERT INTO images(image_id,version,recipe_rev,platform,tag,status,built_at) VALUES(?,?,?,?,?,?,?) ON CONFLICT(image_id) DO UPDATE SET version=excluded.version,recipe_rev=excluded.recipe_rev,platform=excluded.platform,tag=excluded.tag,status=excluded.status,built_at=excluded.built_at", image, b.Version, imageRecipeRev, imagePlatform, tag, "unverified", finished)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -84,43 +84,6 @@ func TestRuntimeNetworkDeletionLostResponseAndReplacement(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
func TestStartIntentsUncertainCreationKeepsRecoveryAccessible(t *testing.T) {
|
||||
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path == "/v1.47/images/image/json" {
|
||||
writeImageInspect(w, "image", "1.2.3")
|
||||
return
|
||||
}
|
||||
if r.Method != "GET" {
|
||||
t.Error("unexpected destructive call")
|
||||
}
|
||||
w.WriteHeader(404)
|
||||
})
|
||||
if err := a.s.EnsureIntents(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := a.s.ReserveIntent(store.Intent{ID: "intent", UID: "owner"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := a.s.ClaimIntent("intent", store.Instance{ID: "pending", UID: "owner", Status: "creating"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := a.setRuntimeCreatePending("pending", "container_pending", true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
if err := a.StartIntents(ctx); err != nil {
|
||||
t.Fatalf("uncertain creation blocked startup: %v", err)
|
||||
}
|
||||
i, err := a.s.GetInstance("pending")
|
||||
if err != nil || i.Status != "error" {
|
||||
t.Fatalf("recovery reservation missing %+v %v", i, err)
|
||||
}
|
||||
list, err := a.s.Intents("owner")
|
||||
if err != nil || len(list) != 1 || list[0].Status != "failed" {
|
||||
t.Fatalf("intent not terminated %+v %v", list, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNetworkCreateLostResponse(t *testing.T) {
|
||||
for _, mode := range []string{"present", "late", "foreign"} {
|
||||
@@ -258,7 +221,7 @@ func TestBuildTerminalFailureDoesNotBlockNextTask(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, id := range []string{"a", "b"} {
|
||||
if _, err = a.s.DB.Exec("INSERT INTO image_builds(id,version,status,created_at) VALUES(?,'1.2.3','queued',?)", id, id); err != nil {
|
||||
if _, err = a.s.DB.Exec("INSERT INTO image_builds(id,version,status,created_at,release_id) VALUES(?,'1.2.3','queued',?,'release')", id, id); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -535,32 +498,6 @@ func TestBuildResultTransactionAndRetry(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestIntentRecoveryMissingReservationDoesNotTouchDocker(t *testing.T) {
|
||||
a, _ := runtimeTestAPI(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path == "/v1.47/images/image/json" {
|
||||
writeImageInspect(w, "image", "1.2.3")
|
||||
return
|
||||
}
|
||||
t.Error("missing reservation must not authorize Docker cleanup")
|
||||
})
|
||||
if err := a.s.EnsureIntents(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := a.s.ReserveIntent(store.Intent{ID: "missing", UID: "owner"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := a.s.DB.Exec("UPDATE creation_intents SET status='creating',instance_id='gone' WHERE id='missing'"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := a.recoverIntents(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
list, err := a.s.Intents("owner")
|
||||
if err != nil || len(list) != 1 || list[0].Status != "failed" {
|
||||
t.Fatalf("intent not terminated: %+v %v", list, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildTerminalRetryStopsWhenDatabaseCloses(t *testing.T) {
|
||||
a, _ := imageTestAPI(t, func(w http.ResponseWriter, r *http.Request) { t.Error("unexpected Docker call") })
|
||||
a.s.Close()
|
||||
|
||||
@@ -110,10 +110,6 @@ func (a *API) instances(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
i := store.Instance{ID: store.ID(), UID: u.ID, Name: strings.TrimSpace(in.Name), ImageID: image.ID, Version: image.Config.Labels["dsh-mgr.version"], Status: "creating", Created: time.Now()}
|
||||
if err := a.s.EnsureIntents(); err != nil {
|
||||
fail(w, 500, "database error")
|
||||
return
|
||||
}
|
||||
if err := a.s.ReserveInstance(i); err != nil {
|
||||
if errors.Is(err, store.ErrInstanceLimit) {
|
||||
fail(w, 409, err.Error())
|
||||
|
||||
@@ -189,12 +189,15 @@ func (c *Client) Logs(ctx context.Context, id string) (string, error) {
|
||||
return output.String(), nil
|
||||
}
|
||||
|
||||
func (c *Client) Build(ctx context.Context, tar io.Reader, version, tag string, onLog func(string)) (string, error) {
|
||||
func (c *Client) BuildRelease(ctx context.Context, tar io.Reader, version, tag, releaseID, repository, commit string, onLog func(string)) (string, error) {
|
||||
labels := map[string]string{"dsh-mgr.release": releaseID, "dsh-mgr.source": repository, "dsh-mgr.commit": commit}
|
||||
args, err := json.Marshal(map[string]string{"DSH_VERSION": version})
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
query := url.Values{"t": {tag}, "buildargs": {string(args)}, "rm": {"1"}}
|
||||
data, _ := json.Marshal(labels)
|
||||
query.Set("labels", string(data))
|
||||
resp, err := c.Stream(ctx, http.MethodPost, "/build?"+query.Encode(), tar, "application/x-tar")
|
||||
if err != nil {
|
||||
return "", err
|
||||
|
||||
@@ -201,7 +201,7 @@ func TestBuild(t *testing.T) {
|
||||
}
|
||||
})
|
||||
var logs []string
|
||||
id, err := client.Build(context.Background(), strings.NewReader("caller tar bytes"), "1.2.3", "test:1", func(s string) { logs = append(logs, s) })
|
||||
id, err := client.BuildRelease(context.Background(), strings.NewReader("caller tar bytes"), "1.2.3", "test:1", "release", "repository", "commit", func(s string) { logs = append(logs, s) })
|
||||
if err != nil || id != "sha256:image" || len(logs) != 1 || logs[0] != "step 1\n" {
|
||||
t.Fatalf("id=%q logs=%v err=%v", id, logs, err)
|
||||
}
|
||||
@@ -219,7 +219,7 @@ func TestBuildErrors(t *testing.T) {
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
client := testClient(t, func(w http.ResponseWriter, r *http.Request) { io.WriteString(w, tc.data) })
|
||||
id, err := client.Build(context.Background(), nil, "1", "test", nil)
|
||||
id, err := client.BuildRelease(context.Background(), nil, "1", "test", "release", "repository", "commit", nil)
|
||||
if err == nil || !strings.Contains(err.Error(), tc.want) || id != "" {
|
||||
t.Fatalf("id=%q err=%v", id, err)
|
||||
}
|
||||
@@ -235,7 +235,7 @@ func TestBuildCancellation(t *testing.T) {
|
||||
})
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
|
||||
defer cancel()
|
||||
_, err := client.Build(ctx, nil, "1", "test", func(string) { cancel() })
|
||||
_, err := client.BuildRelease(ctx, nil, "1", "test", "release", "repository", "commit", func(string) { cancel() })
|
||||
if !errors.Is(err, context.Canceled) {
|
||||
t.Fatalf("err=%v", err)
|
||||
}
|
||||
|
||||
@@ -7,29 +7,10 @@ import (
|
||||
)
|
||||
|
||||
var ErrInstanceLimit = errors.New("instance limit reached")
|
||||
var ErrIntentState = errors.New("intent is no longer waiting")
|
||||
|
||||
type Intent struct {
|
||||
ID string `json:"id"`
|
||||
UID string `json:"uid"`
|
||||
Name string `json:"name"`
|
||||
Version string `json:"version"`
|
||||
BuildID string `json:"build_id"`
|
||||
InstanceID string `json:"instance_id"`
|
||||
Status string `json:"status"`
|
||||
Error string `json:"error"`
|
||||
Host string `json:"-"`
|
||||
Created int64 `json:"created_at"`
|
||||
}
|
||||
|
||||
func (s *Store) EnsureIntents() error {
|
||||
_, err := s.DB.Exec(`CREATE TABLE IF NOT EXISTS creation_intents(id TEXT PRIMARY KEY,uid TEXT NOT NULL,name TEXT NOT NULL,version TEXT NOT NULL,build_id TEXT NOT NULL DEFAULT '',instance_id TEXT NOT NULL DEFAULT '',status TEXT NOT NULL,error TEXT NOT NULL DEFAULT '',host TEXT NOT NULL,created INTEGER NOT NULL); CREATE INDEX IF NOT EXISTS intents_owner ON creation_intents(uid,status);`)
|
||||
return err
|
||||
}
|
||||
|
||||
func reserveSlot(tx *sql.Tx, uid string) error {
|
||||
var count int
|
||||
if err := tx.QueryRow(`SELECT (SELECT count(*) FROM instances WHERE uid=?) + (SELECT count(*) FROM creation_intents WHERE uid=? AND status='waiting')`, uid, uid).Scan(&count); err != nil {
|
||||
if err := tx.QueryRow(`SELECT count(*) FROM instances WHERE uid=?`, uid).Scan(&count); err != nil {
|
||||
return err
|
||||
}
|
||||
if count >= 2 {
|
||||
@@ -38,22 +19,6 @@ func reserveSlot(tx *sql.Tx, uid string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) ReserveIntent(i Intent) error {
|
||||
tx, err := s.DB.Begin()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer tx.Rollback()
|
||||
if err = reserveSlot(tx, i.UID); err != nil {
|
||||
return err
|
||||
}
|
||||
_, err = tx.Exec(`INSERT INTO creation_intents(id,uid,name,version,status,host,created) VALUES(?,?,?,?,'waiting',?,?)`, i.ID, i.UID, i.Name, i.Version, i.Host, time.Now().Unix())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Commit()
|
||||
}
|
||||
|
||||
func (s *Store) ReserveInstance(i Instance) error {
|
||||
tx, err := s.DB.Begin()
|
||||
if err != nil {
|
||||
@@ -72,62 +37,3 @@ func insertInstance(tx *sql.Tx, i Instance) error {
|
||||
_, err := tx.Exec(`INSERT INTO instances VALUES(?,?,?,?,?,?,?,?)`, i.ID, i.UID, i.Name, i.Version, i.ImageID, i.Status, i.ContainerID, time.Now().Unix())
|
||||
return err
|
||||
}
|
||||
func (s *Store) ClaimIntent(id string, i Instance) error {
|
||||
tx, err := s.DB.Begin()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer tx.Rollback()
|
||||
result, err := tx.Exec(`UPDATE creation_intents SET status='creating',instance_id=? WHERE id=? AND uid=? AND version=? AND status='waiting'`, i.ID, id, i.UID, i.Version)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
n, err := result.RowsAffected()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n != 1 {
|
||||
return ErrIntentState
|
||||
}
|
||||
if err = insertInstance(tx, i); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Commit()
|
||||
}
|
||||
func (s *Store) CancelIntent(id, uid string) error {
|
||||
r, err := s.DB.Exec(`UPDATE creation_intents SET status='cancelled' WHERE id=? AND uid=? AND status='waiting'`, id, uid)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
n, err := r.RowsAffected()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n != 1 {
|
||||
return ErrIntentState
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func (s *Store) Intents(uid string) ([]Intent, error) {
|
||||
query := `SELECT id,uid,name,version,build_id,instance_id,status,error,host,created FROM creation_intents`
|
||||
var args []any
|
||||
if uid != "" {
|
||||
query += ` WHERE uid=?`
|
||||
args = append(args, uid)
|
||||
}
|
||||
query += ` ORDER BY created,id`
|
||||
rows, err := s.DB.Query(query, args...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []Intent{}
|
||||
for rows.Next() {
|
||||
var i Intent
|
||||
if err = rows.Scan(&i.ID, &i.UID, &i.Name, &i.Version, &i.BuildID, &i.InstanceID, &i.Status, &i.Error, &i.Host, &i.Created); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, i)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
@@ -7,26 +7,19 @@ import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestIntentReservationConcurrencyAndRestart(t *testing.T) {
|
||||
func TestInstanceReservationConcurrencyAndRestart(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "store.db")
|
||||
s, err := Open(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = s.EnsureIntents(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var wg sync.WaitGroup
|
||||
result := make(chan error, 30)
|
||||
for n := 0; n < 30; n++ {
|
||||
wg.Add(1)
|
||||
go func(n int) {
|
||||
defer wg.Done()
|
||||
if n%2 == 0 {
|
||||
result <- s.ReserveIntent(Intent{ID: ID(), UID: "u", Name: "test", Version: "1.2.3"})
|
||||
} else {
|
||||
result <- s.ReserveInstance(Instance{ID: ID(), UID: "u"})
|
||||
}
|
||||
}(n)
|
||||
}
|
||||
wg.Wait()
|
||||
@@ -48,47 +41,26 @@ func TestIntentReservationConcurrencyAndRestart(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer s.Close()
|
||||
if err = s.ReserveIntent(Intent{ID: ID(), UID: "u"}); !errors.Is(err, ErrInstanceLimit) {
|
||||
if err = s.ReserveInstance(Instance{ID: ID(), UID: "u"}); !errors.Is(err, ErrInstanceLimit) {
|
||||
t.Fatalf("restart lost reservation: %v", err)
|
||||
}
|
||||
}
|
||||
func TestIntentClaimCancelAndVersion(t *testing.T) {
|
||||
|
||||
func TestRetiredIntentsDoNotReserveInstanceSlots(t *testing.T) {
|
||||
s, err := Open(filepath.Join(t.TempDir(), "store.db"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer s.Close()
|
||||
if err = s.EnsureIntents(); err != nil {
|
||||
if _, err := s.DB.Exec("CREATE TABLE creation_intents(uid TEXT,status TEXT); INSERT INTO creation_intents VALUES('u','waiting'),('u','creating')"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
in := Intent{ID: ID(), UID: "u", Version: "1.2.3"}
|
||||
if err = s.ReserveIntent(in); err != nil {
|
||||
for n := 0; n < 2; n++ {
|
||||
if err := s.ReserveInstance(Instance{ID: ID(), UID: "u"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
i := Instance{ID: ID(), UID: "u", Version: "1.2.4"}
|
||||
if err = s.ClaimIntent(in.ID, i); !errors.Is(err, ErrIntentState) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = s.CancelIntent(in.ID, "other"); !errors.Is(err, ErrIntentState) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
i.Version = in.Version
|
||||
if err = s.ClaimIntent(in.ID, i); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = s.CancelIntent(in.ID, in.UID); !errors.Is(err, ErrIntentState) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = s.ReserveIntent(Intent{ID: "second", UID: "u"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = s.ReserveIntent(Intent{ID: "third", UID: "u"}); !errors.Is(err, ErrInstanceLimit) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = s.CancelIntent("second", "u"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = s.ReserveIntent(Intent{ID: "third", UID: "u"}); err != nil {
|
||||
if err := s.ReserveInstance(Instance{ID: ID(), UID: "u"}); !errors.Is(err, ErrInstanceLimit) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,15 +0,0 @@
|
||||
#!/bin/bash
|
||||
set -Eeuo pipefail
|
||||
project_dir=$(CDPATH= cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)
|
||||
version=${1:-}
|
||||
if [[ -z $version ]]; then
|
||||
version=$(npm view @deepseek-ai/dsh@latest version --registry=https://registry.npmjs.org)
|
||||
fi
|
||||
if [[ ! $version =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]]; then
|
||||
echo 'Expected an exact DSH version, not a dist-tag or arbitrary build argument.' >&2
|
||||
exit 2
|
||||
fi
|
||||
tag="dsh-web:${version}-7"
|
||||
printf 'Building local image %s (no push)\n' "$tag"
|
||||
docker build --build-arg "DSH_VERSION=$version" --tag "$tag" "$project_dir/image"
|
||||
docker image inspect --format '{{.Id}}' "$tag"
|
||||
@@ -23,7 +23,6 @@ run go vet ./...
|
||||
run go build ./...
|
||||
run npm --prefix web test
|
||||
run npm --prefix web run build
|
||||
run node --test image/init-profile.test.mjs
|
||||
run node --check scripts/test-panel.mjs
|
||||
|
||||
printf '\nAll acceptance checks passed. 所有测试与检查均通过。\n'
|
||||
|
||||
@@ -12,7 +12,6 @@ const commands = [
|
||||
'go build ./...',
|
||||
'npm --prefix web test',
|
||||
'npm --prefix web run build',
|
||||
'node --test image/init-profile.test.mjs',
|
||||
'node --check scripts/test-panel.mjs',
|
||||
];
|
||||
|
||||
|
||||
@@ -1,33 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import {readFileSync} from 'node:fs';
|
||||
const base=process.env.TEST_PANEL_URL;
|
||||
assert.ok(base && process.env.TEST_PASSWORD_FILE);
|
||||
let csrf='',cookie='';
|
||||
async function req(path,method='GET',body){
|
||||
const r=await fetch(base+path,{method,headers:{Cookie:cookie,Origin:base,'Content-Type':'application/json','X-CSRF-Token':csrf},body:body===undefined?undefined:JSON.stringify(body)});
|
||||
if(path==='/api/login')cookie=r.headers.getSetCookie().map(v=>v.split(';')[0]).join('; ');
|
||||
return {status:r.status,data:await r.json()};
|
||||
}
|
||||
const login=await req('/api/login','POST',{username:'admin',password:readFileSync(process.env.TEST_PASSWORD_FILE,'utf8').trim()});
|
||||
assert.equal(login.status,200);csrf=login.data.csrf_token;
|
||||
const before=(await req('/api/instances')).data;assert.equal(before.length,1,'run with exactly one retained test instance');
|
||||
const intent=await req('/api/intents','POST',{name:'Design real intent',version:'0.2.0-rc.2'});assert.equal(intent.status,202);
|
||||
let created;
|
||||
try {
|
||||
for(let n=0;n<60;n++){
|
||||
const item=(await req('/api/intents')).data.find(x=>x.id===intent.data.id);
|
||||
assert.notEqual(item.status,'failed');
|
||||
if(item.instance_id){created=item.instance_id;break;}
|
||||
await new Promise(r=>setTimeout(r,1000));
|
||||
}
|
||||
assert.ok(created,'intent must create a real instance');
|
||||
const list=(await req('/api/instances')).data;assert.equal(list.length,2);assert.ok(list.every(i=>i.status==='running'));
|
||||
const extra=await req('/api/instances','POST',{name:'must reject third',image_id:list[0].image_id});assert.equal(extra.status,409);
|
||||
assert.equal((await req('/api/intents','POST',{name:'must reject extra reservation',version:'0.2.0-rc.2'})).status,409);
|
||||
console.log('PASS persisted intent -> real runtime; two running instances; third create and reservation rejected');
|
||||
} finally {
|
||||
if(created)assert.equal((await req('/api/instances/'+created,'DELETE')).status,200);
|
||||
else await req('/api/intents/'+intent.data.id,'DELETE');
|
||||
}
|
||||
assert.equal((await req('/api/instances')).data.length,1);
|
||||
console.log('PASS deleted only this test runtime, retained data and prior instance');
|
||||
@@ -23,16 +23,12 @@ try {
|
||||
assert.equal((await request('/api/me')).data.user.username,process.env.TEST_USERNAME || 'testadmin');
|
||||
await request('/api/refresh','POST');
|
||||
assert.equal((await request('/api/me')).data.user.username,process.env.TEST_USERNAME || 'testadmin');
|
||||
let images=(await request('/api/images')).data;
|
||||
const build=(await request('/api/builds','POST',{version:process.env.TEST_FROM_VERSION || images[0]?.version || '0.2.0-rc.2'})).data;
|
||||
let buildSucceeded=false;
|
||||
for(let n=0;n<120;n++){let job;try{job=(await request('/api/builds/'+encodeURIComponent(build.id))).data;}catch(e){if(!(e instanceof Error))throw e;await new Promise(r=>setTimeout(r,1000));continue;}if(job?.status==='failed')throw new Error('build/validation failed: '+job.error);if(job?.status==='success'){buildSucceeded=true;break;}await new Promise(r=>setTimeout(r,1000));}
|
||||
assert.ok(buildSucceeded,'submitted build/validation job did not complete before timeout');
|
||||
images=(await request('/api/images')).data.filter(i=>i.status==='available');assert.ok(images.length,'validated image required');
|
||||
const images=(await request('/api/images')).data.filter(i=>i.status==='available' && i.selectable===true);
|
||||
assert.ok(images.length,'prepare a validated, admitted image through the admin Git release flow before running this test');
|
||||
const source=images.find(i=>!process.env.TEST_FROM_VERSION || i.version===process.env.TEST_FROM_VERSION);
|
||||
const target=images.find(i=>!process.env.TEST_TO_VERSION || i.version===process.env.TEST_TO_VERSION);
|
||||
assert.ok(source,'source image required');assert.ok(target,'target image required');
|
||||
console.log('PASS local image reuse, persistent build task and runtime verification');
|
||||
console.log('PASS validated and admitted local image selection');
|
||||
instance=(await request('/api/instances','POST',{name:'Integration check',image_id:source.image_id || source.id,version:source.version})).data;
|
||||
await request('/api/ws/mkdir','POST',{path:testDirectory});
|
||||
const form=new FormData();form.set('path',testDirectory);form.set('file',new Blob(['hello workspace']), 'hello.txt');await request('/api/ws/upload','POST',form);
|
||||
|
||||
+12
-24
@@ -1,15 +1,10 @@
|
||||
import { useEffect, useRef, useState, type FormEvent } from 'react';
|
||||
import { api, normalBuildAllowed, selectableImage, type Image, type Instance } from './api';
|
||||
import { message, ResourceState, useAction, useResource } from './ui';
|
||||
import { api, selectableImage, type Instance } from './api';
|
||||
import { ResourceState, useAction, useResource } from './ui';
|
||||
|
||||
export function CreateInstance({close, refresh}: {close: () => void; refresh: () => Promise<void>}) {
|
||||
const compatibility = useResource(api.compatibility);
|
||||
const images = useResource(api.images), action = useAction();
|
||||
const [mode, setMode] = useState('local'), [progress, setProgress] = useState('');
|
||||
const [version, setVersion] = useState('');
|
||||
const [pollError, setPollError] = useState('');
|
||||
const [intents, setIntents] = useState<{id:string; name:string; version:string; status:string; error:string}[]>([]);
|
||||
useEffect(() => {let live=true; const load=()=>api.request<typeof intents>('/intents').then(v=>{if(live){setIntents(v);setPollError('');}}).catch(e=>{if(live)setPollError(message(e));}); void load(); const timer=window.setInterval(load,2000); return ()=>{live=false;window.clearInterval(timer);};}, []);
|
||||
const [progress, setProgress] = useState('');
|
||||
const active = useRef(true);
|
||||
useEffect(() => {active.current = true; return () => {active.current = false;};}, []);
|
||||
const available = images.data?.filter(selectableImage) || [];
|
||||
@@ -17,21 +12,7 @@ export function CreateInstance({close, refresh}: {close: () => void; refresh: ()
|
||||
e.preventDefault();
|
||||
const form = new FormData(e.currentTarget), name = String(form.get('name'));
|
||||
await action.run(async () => {
|
||||
let image: Image | undefined = available.find(i => (i.image_id || i.id) === form.get('image'));
|
||||
if (mode === 'latest') {
|
||||
if (!version) {
|
||||
const latest = await api.request<{version:string; build_allowed:boolean}>('/latest');
|
||||
if (!latest.version) throw new Error('服务器未返回精确版本。');
|
||||
setVersion(latest.version);
|
||||
if (!latest.build_allowed || !normalBuildAllowed(await api.compatibility(), latest.version)) throw new Error('此最新版不支持普通构建。管理员可前往设置 → 镜像与构建进行实验构建,但实验镜像不能直接用于创建。');
|
||||
return;
|
||||
}
|
||||
if (!normalBuildAllowed(await api.compatibility(), version)) throw new Error('此版本不支持普通构建;实验镜像不能直接用于创建。');
|
||||
const intent = await api.request<{id:string}>('/intents', 'POST', {name, version});
|
||||
if (!intent.id) throw new Error('未收到意图确认,请刷新核实,勿重复提交。');
|
||||
setIntents(await api.request<typeof intents>('/intents'));
|
||||
await refresh(); return;
|
||||
}
|
||||
const image = available.find(i => (i.image_id || i.id) === form.get('image'));
|
||||
if (!active.current) return;
|
||||
if (!image) throw new Error('请选择可用的本地镜像。');
|
||||
setProgress('正在创建实例,请勿重复提交…');
|
||||
@@ -41,5 +22,12 @@ export function CreateInstance({close, refresh}: {close: () => void; refresh: ()
|
||||
});
|
||||
if (active.current) setProgress('');
|
||||
}
|
||||
return <form className="inline-form" onSubmit={submit}><h3>创建独立实例</h3>{action.feedback}<p className="callout">确认后服务器持久预留一个名额(每用户最多两个)。刷新或关闭不会取消自动创建;失败释放名额,不回退其他版本。取消等待不会取消共享构建,创建开始后不可撤回。</p><fieldset disabled={action.busy}><label>实例名称<input name="name" required maxLength={64}/></label><label>创建方式<select value={mode} onChange={e => {setMode(e.target.value); setVersion('');}}><option value="local">选择本地镜像</option><option value="latest">构建最新版并创建</option></select></label>{mode === 'local' && <><ResourceState resource={images}/><label>本地镜像<select name="image" required defaultValue=""><option value="" disabled>选择已构建镜像</option>{available.map(i => <option key={i.image_id || i.id} value={i.image_id || i.id}>{i.version} · {i.tag || i.image_id || i.id}</option>)}</select></label>{!images.loading && !available.length && <p>暂无已准入的可用镜像。历史或实验镜像不可用于创建。</p>}</>}{mode === 'latest' && <ResourceState resource={compatibility}/>} {mode === 'latest' && version && compatibility.data && !normalBuildAllowed(compatibility.data, version) && <p className="alert">此最新版不支持普通构建。管理员可前往设置 → 镜像与构建实验构建;实验镜像不能直接用于创建。</p>}{mode === 'latest' && version && compatibility.data && normalBuildAllowed(compatibility.data, version) && <p>将锁定精确版本 <strong>{version}</strong>,确认后预留名额并构建创建。</p>}<button className="primary" disabled={mode === 'local' ? !available.length : !!version && (!compatibility.data || !normalBuildAllowed(compatibility.data, version))}>{mode === 'local' ? '确认创建' : version ? '确认版本并预留创建' : '查询最新版'}</button></fieldset>{progress && <p role="status">{progress}</p>}{pollError && <p className="muted" role="alert">意图刷新失败:{pollError}</p>}<ul>{intents.map(i => <li key={i.id}>{i.name} · {i.version} · {i.status} {i.error}{i.status === 'waiting' && <button type="button" disabled={action.busy} onClick={() => void action.run(async () => {await api.request('/intents/' + i.id, 'DELETE'); setIntents(await api.request<typeof intents>('/intents')); await refresh();})}>取消等待并释放名额</button>}{i.status === 'success' && <button type="button" onClick={() => void refresh()}>刷新实例列表</button>}</li>)}</ul><button type="button" disabled={action.busy} onClick={close}>关闭(保留已提交意图)</button></form>;
|
||||
return <form className="inline-form" onSubmit={submit}><h3>创建独立实例</h3>{action.feedback}
|
||||
<p className="callout">请选择已验证并准入的本地镜像。创建使用所选镜像的精确 ID,不会自动构建或切换版本。</p>
|
||||
<fieldset disabled={action.busy}><label>实例名称<input name="name" required maxLength={64}/></label>
|
||||
<ResourceState resource={images}/><label>本地镜像<select name="image" required defaultValue=""><option value="" disabled>选择已验证镜像</option>{available.map(i => <option key={i.image_id || i.id} value={i.image_id || i.id}>{i.version} · {i.tag || i.image_id || i.id}</option>)}</select></label>
|
||||
{!images.loading && !available.length && <p>暂无已准入的可用镜像。请联系管理员构建并验证发布镜像。</p>}
|
||||
<button className="primary" disabled={!available.length}>确认创建</button>
|
||||
</fieldset>{progress && <p role="status">{progress}</p>}<button type="button" disabled={action.busy} onClick={close}>关闭</button>
|
||||
</form>;
|
||||
}
|
||||
|
||||
+30
-21
@@ -1,5 +1,5 @@
|
||||
import { afterEach, beforeEach, expect, it, vi } from 'vitest';
|
||||
const state = vi.hoisted(() => ({slots: [] as any[], cursor: 0}));
|
||||
const state = vi.hoisted(() => ({slots: [] as any[], cursor: 0, emptyImages: false}));
|
||||
vi.mock('react', async original => ({
|
||||
...await original<typeof import('react')>(),
|
||||
useState(value: unknown) {const i = state.cursor++; if (!(i in state.slots)) state.slots[i] = value; return [state.slots[i], (next: unknown) => {state.slots[i] = next;}];},
|
||||
@@ -8,34 +8,34 @@ vi.mock('react', async original => ({
|
||||
}));
|
||||
vi.mock('./ui', async original => ({
|
||||
...await original<typeof import('./ui')>(),
|
||||
useResource: (load: unknown) => ({data: load === api.compatibility ? policy : load === api.images ? [{id:'ok', version:'1.2.3', status:'available', selectable:true}, {id:'exp', version:'9.9.9', status:'experimental', selectable:false}] : [], loading:false, refresh:vi.fn(async () => {})}),
|
||||
useResource: (load: unknown) => ({data: load === api.images ? state.emptyImages ? [] : [{id:'ok', version:'1.2.3', status:'available', selectable:true}, {id:'exp', version:'9.9.9', status:'experimental', selectable:false}] : [], loading:false, refresh:vi.fn(async () => {})}),
|
||||
}));
|
||||
import { api, type Compatibility } from './api';
|
||||
import { api } from './api';
|
||||
import { Images } from './Images';
|
||||
import { ReleasePanel } from './ReleasePanel';
|
||||
import { CreateInstance } from './CreateInstance';
|
||||
vi.mock('./ProfileEditor', () => ({ProfileEditor: () => null}));
|
||||
import { Instances } from './Instances';
|
||||
const policy: Compatibility = {recipe_rev:'7', platform:'linux/amd64', entries:[], unknown:{state:'existing_only', build_allowed:false, reason:'unknown'}, experimental_admin_only:true};
|
||||
function nodes(root: any, predicate: (node: any) => boolean): any[] {
|
||||
if (Array.isArray(root)) return root.flatMap(child => nodes(child, predicate));
|
||||
if (!root || typeof root !== 'object') return [];
|
||||
return [...(predicate(root) ? [root] : []), ...nodes(root.props?.children, predicate)];
|
||||
}
|
||||
function render(admin = false) {state.cursor = 0; return Images({admin});}
|
||||
beforeEach(() => {state.slots = []; state.cursor = 0; vi.stubGlobal('window', {confirm:vi.fn(() => false)});});
|
||||
beforeEach(() => {state.slots = []; state.cursor = 0; state.emptyImages = false; vi.stubGlobal('window', {confirm:vi.fn(() => false)});});
|
||||
afterEach(() => {vi.restoreAllMocks(); vi.unstubAllGlobals();});
|
||||
it('hides experimental and admission controls from ordinary users', () => {
|
||||
const tree = render();
|
||||
expect(nodes(tree, n => n.type === ReleasePanel)).toHaveLength(0);
|
||||
expect(nodes(render(true), n => n.type === ReleasePanel)).toHaveLength(1);
|
||||
expect(nodes(tree, n => n.type === 'input' && n.props.type === 'checkbox')).toHaveLength(0);
|
||||
expect(nodes(tree, n => n.type === 'button' && n.props.children === '准入管理')).toHaveLength(0);
|
||||
expect(nodes(render(true), n => n.type === 'button' && n.props.children === '准入管理')).toHaveLength(2);
|
||||
});
|
||||
it('canceled build confirmation sends zero requests', () => {
|
||||
const request = vi.spyOn(api, 'startBuild');
|
||||
const check = vi.spyOn(api, 'compatibility');
|
||||
nodes(render(true), n => n.type === 'form')[0].props.onSubmit({preventDefault:vi.fn()});
|
||||
expect(window.confirm).toHaveBeenCalledOnce();
|
||||
expect(request).not.toHaveBeenCalled(); expect(check).not.toHaveBeenCalled();
|
||||
it.each([false, true])('has no legacy build controls (admin=%s)', admin => {
|
||||
const tree = render(admin);
|
||||
expect(nodes(tree, n => n.type === 'form' || n.type === 'input')).toHaveLength(0);
|
||||
expect(JSON.stringify(tree)).not.toMatch(/查询 latest|精确 DSH 版本|提交实验构建/);
|
||||
});
|
||||
it('requires evidence and confirmation; cancel sends no admission mutation', () => {
|
||||
const request = vi.spyOn(api, 'admitImage');
|
||||
@@ -50,17 +50,26 @@ it('requires evidence and confirmation; cancel sends no admission mutation', ()
|
||||
form.props.onSubmit({preventDefault:vi.fn()});
|
||||
expect(window.confirm).toHaveBeenCalledOnce(); expect(request).not.toHaveBeenCalled();
|
||||
});
|
||||
it('unknown latest never reserves an intent, including a second submit', async () => {
|
||||
const request = vi.spyOn(api, 'request').mockResolvedValue({version:'9.9.9', build_allowed:false});
|
||||
vi.spyOn(api, 'compatibility').mockResolvedValue(policy);
|
||||
vi.stubGlobal('FormData', class {get() {return 'name';}});
|
||||
const renderCreate = () => {state.cursor = 0; return CreateInstance({close:vi.fn(), refresh:vi.fn(async () => {})});};
|
||||
let form = renderCreate();
|
||||
it('creates only the selected verified exact image without build or intent requests', async () => {
|
||||
const request = vi.spyOn(api, 'request').mockResolvedValue({id:'created'});
|
||||
vi.stubGlobal('FormData', class {get(key: string) {return key === 'image' ? 'ok' : 'name';}});
|
||||
const close = vi.fn(), refresh = vi.fn(async () => {});
|
||||
const form = CreateInstance({close, refresh});
|
||||
expect(nodes(form, n => n.type === 'select')).toHaveLength(1);
|
||||
expect(nodes(form, n => n.type === 'option' && n.props.value === 'exp')).toHaveLength(0);
|
||||
nodes(form, n => n.type === 'select')[0].props.onChange({target:{value:'latest'}});
|
||||
form = renderCreate(); await form.props.onSubmit({preventDefault:vi.fn(), currentTarget:{}});
|
||||
form = renderCreate(); await form.props.onSubmit({preventDefault:vi.fn(), currentTarget:{}});
|
||||
expect(request.mock.calls).toEqual([['/latest']]);
|
||||
await form.props.onSubmit({preventDefault:vi.fn(), currentTarget:{}});
|
||||
expect(request.mock.calls).toEqual([['/instances', 'POST', {name:'name', image_id:'ok', version:'1.2.3'}]]);
|
||||
expect(refresh).toHaveBeenCalledOnce(); expect(close).toHaveBeenCalledOnce();
|
||||
});
|
||||
it('directs users to an administrator and disables creation when no verified image exists', async () => {
|
||||
state.emptyImages = true;
|
||||
const request = vi.spyOn(api, 'request');
|
||||
vi.stubGlobal('FormData', class {get() {return 'exp';}});
|
||||
const form = CreateInstance({close:vi.fn(), refresh:vi.fn(async () => {})});
|
||||
expect(nodes(form, n => n.type === 'button' && n.props.children === '确认创建')[0].props.disabled).toBe(true);
|
||||
expect(JSON.stringify(form)).toContain('请联系管理员');
|
||||
await form.props.onSubmit({preventDefault:vi.fn(), currentTarget:{}});
|
||||
expect(request).not.toHaveBeenCalled();
|
||||
});
|
||||
it('admission rejects blank/unconfirmed evidence, resets consent on edits, and submits once', async () => {
|
||||
const request = vi.spyOn(api, 'admitImage').mockResolvedValue({});
|
||||
|
||||
+11
-27
@@ -1,6 +1,7 @@
|
||||
import { useState, type FormEvent } from 'react';
|
||||
import { api, normalBuildAllowed, type AdmissionState, type Build } from './api';
|
||||
import { useState } from 'react';
|
||||
import { api, type AdmissionState, type Build } from './api';
|
||||
import { Panel, ResourceState, Empty, Status, date, useAction, useResource } from './ui';
|
||||
import { BuildProvenance, ReleasePanel } from './ReleasePanel';
|
||||
export const activeBuild = (build: Build) => ['queued', 'running', 'validating'].includes(build.status);
|
||||
function BuildLogs({id, close}: {id: string; close: () => void}) {
|
||||
const logs = useResource(() => api.build(id), id, 5000, activeBuild);
|
||||
@@ -9,7 +10,7 @@ function BuildLogs({id, close}: {id: string; close: () => void}) {
|
||||
<button onClick={close}>关闭日志</button>
|
||||
<ResourceState resource={logs}/>{logs.data && <>
|
||||
<p>{logs.data.experimental && <strong className="alert">实验构建 · 不自动准入</strong>}{logs.data.version} · <Status value={logs.data.status}/>
|
||||
</p>{logs.data.error && <p className="alert">{logs.data.error}</p>}<pre className="logs">{logs.data.logs || '任务尚未输出日志'}</pre>
|
||||
</p><BuildProvenance build={logs.data}/>{logs.data.error && <p className="alert">{logs.data.error}</p>}<pre className="logs">{logs.data.logs || '任务尚未输出日志'}</pre>
|
||||
</>}</div>;
|
||||
}
|
||||
function Admission({imageId, refresh}: {imageId:string; refresh:() => Promise<void>}) {
|
||||
@@ -24,7 +25,7 @@ function Admission({imageId, refresh}: {imageId:string; refresh:() => Promise<vo
|
||||
<option value="blocked">禁止使用</option>
|
||||
</select>
|
||||
</label>
|
||||
<p className="callout">构建成功不等于通过验收。请记录启动、登录、终端、文件和配置等实际兼容性验收结果及可追溯依据,不可仅填写“构建成功”。单镜像准入不扩大普通构建版本白名单;未知版本仅可选择已准入的本地镜像 ID 创建或升级。</p>
|
||||
<p className="callout">构建成功不等于通过验收。请记录启动、登录、终端、文件和配置等实际兼容性验收结果及可追溯依据,不可仅填写“构建成功”。创建或升级须选择已准入的本地镜像 ID。</p>
|
||||
<label>验收依据<textarea required aria-invalid={evidenceBytes > 4096} value={evidence} onChange={e => {setEvidence(e.target.value); setConfirmed(false);}}/>
|
||||
</label>
|
||||
<p className={evidenceBytes > 4096 ? 'alert' : 'muted'} role={evidenceBytes > 4096 ? 'alert' : undefined}>验收依据:{evidenceBytes} / 4096 UTF-8 字节(去除首尾空白后);中文通常占 3 字节,表情通常占 4 字节。{evidenceBytes > 4096 && '已超出字节上限,请缩短依据。'}</p>
|
||||
@@ -34,33 +35,16 @@ function Admission({imageId, refresh}: {imageId:string; refresh:() => Promise<vo
|
||||
</form>;
|
||||
}
|
||||
export function Images({admin = false}: {admin?:boolean}) {
|
||||
const [experimental, setExperimental] = useState(false), [admissionId, setAdmissionId] = useState<string>();
|
||||
const [admissionId, setAdmissionId] = useState<string>();
|
||||
const [page, setPage] = useState(0);
|
||||
const compatibility = useResource(api.compatibility);
|
||||
const images = useResource(api.images), builds = useResource(() => api.builds(21, page * 20), String(page), 5000, rows => rows.slice(0, 20).some(activeBuild)); const action = useAction();
|
||||
const [latest, setLatest] = useState<{version:string; checked_at?:string}>(), [version,setVersion] = useState(''), [selected,setSelected] = useState<string>();
|
||||
function build(e: FormEvent) {e.preventDefault(); void action.run(async () => {if (!(admin && experimental) && !normalBuildAllowed(await api.compatibility(), version)) throw new Error('此版本不支持普通构建,请管理员选择实验构建;实验构建不会自动准入。'); await api.startBuild(version, admin && experimental); if (page === 0) await builds.refresh(); else setPage(0);}, '构建任务已提交,请查看任务的实际状态。', (experimental ? '实验构建不会自动准入或用于创建。确认实验构建精确版本 ' : '确认在本机手动构建精确版本 ') + version + '?构建会消耗磁盘、网络与 CPU,不会自动升级实例。');}
|
||||
const [selected, setSelected] = useState<string>();
|
||||
const rows = builds.data?.slice(0, 20);
|
||||
function changePage(next: number) { setSelected(undefined); setPage(next); }
|
||||
return <>{action.feedback}<Panel title="查询与手动构建">
|
||||
<ResourceState resource={compatibility}/>{version && compatibility.data && !normalBuildAllowed(compatibility.data, version) && <p className="alert">此版本不支持普通构建。仅管理员可实验构建,不会自动准入。</p>}<div className="split">
|
||||
<div>
|
||||
<p className="muted">从 npm 查询 dist-tags.latest,仅查询,不自动构建。</p>
|
||||
<button disabled={action.busy} onClick={() => action.run(async () => {const result = await api.request<{version:string; checked_at?:string}>('/latest'); setLatest(result); setVersion(result.version);}, '最新版本查询完成')}>查询 latest</button>{latest && <p>
|
||||
<strong>{latest.version}</strong> <small>查询时间 {date(latest.checked_at)}</small>
|
||||
</p>}</div>
|
||||
<form className="build-form" onSubmit={build}>
|
||||
<label>精确 DSH 版本<input required value={version} onChange={e => setVersion(e.target.value)} placeholder="查询 latest,或填写精确版本" pattern="[0-9]+\.[0-9]+\.[0-9]+.*" />
|
||||
</label>
|
||||
<>{admin && <label>
|
||||
<input type="checkbox" checked={experimental} onChange={e => setExperimental(e.target.checked)}/>管理员实验构建(不自动准入)</label>}{experimental && <p className="alert" role="alert">实验镜像:构建成功后仍不能创建或升级实例,必须另行验收并显式准入。</p>}<button className="primary" disabled={action.busy || !version}>{experimental ? '提交实验构建' : '手动构建'}</button>
|
||||
</>
|
||||
</form>
|
||||
</div>
|
||||
<p className="callout">镜像仅保存在本机。构建成功不等于通过验收或自动准入。仅 available 且允许选择的镜像可创建或升级;existing_only 仅允许存量启停。blocked 禁止新建、升级及新的启动/重启,但不会自动停止已运行实例或断开连接。</p>
|
||||
</Panel>
|
||||
return <>{action.feedback}{admin && <ReleasePanel onBuilt={async () => {if (page === 0) await builds.refresh(); else changePage(0);}}/>}
|
||||
<Panel title="本地镜像" action={<button disabled={images.loading} onClick={images.refresh}>刷新</button>}>
|
||||
<ResourceState resource={images}/>{images.data?.length === 0 && <Empty>本机暂无托管镜像,请先手动构建。</Empty>}{!!images.data?.length && <div className="table-scroll">
|
||||
<p className="callout">镜像仅保存在本机。仅已验证且允许选择的 available 镜像可创建或升级;existing_only 仅允许存量启停。blocked 禁止新建、升级及新的启动/重启,但不会自动停止已运行实例或断开连接。</p>
|
||||
<ResourceState resource={images}/>{images.data?.length === 0 && <Empty>本机暂无托管镜像。请联系管理员构建并验证发布镜像。</Empty>}{!!images.data?.length && <div className="table-scroll">
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
@@ -104,7 +88,7 @@ export function Images({admin = false}: {admin?:boolean}) {
|
||||
</thead>
|
||||
<tbody>{rows!.map(b => <tr key={b.id}>
|
||||
<td className="mono">{b.id}</td>
|
||||
<td>{b.version}{b.experimental && <strong className="alert">实验构建 · 不自动准入</strong>}</td>
|
||||
<td>{b.version}{b.experimental && <strong className="alert">实验构建 · 不自动准入</strong>}<BuildProvenance build={b}/></td>
|
||||
<td>
|
||||
<Status value={b.status}/>
|
||||
</td>
|
||||
|
||||
@@ -0,0 +1,148 @@
|
||||
import { afterEach, beforeEach, expect, it, vi } from 'vitest';
|
||||
const hooks = vi.hoisted(() => ({slots: [] as any[], cursor: 0, effects: [] as (() => void)[]}));
|
||||
vi.mock('react', async original => ({
|
||||
...await original<typeof import('react')>(),
|
||||
useRef(value: unknown) {const i = hooks.cursor++; return hooks.slots[i] ??= {current:value};},
|
||||
useState(value: unknown) {const i = hooks.cursor++; if (!(i in hooks.slots)) hooks.slots[i] = value; return [hooks.slots[i], (v: unknown) => {hooks.slots[i] = v;}];},
|
||||
useCallback(fn: unknown, deps: unknown[]) {
|
||||
const i = hooks.cursor++, previous = hooks.slots[i];
|
||||
if (!previous || deps.some((v, n) => v !== previous.deps[n])) hooks.slots[i] = {deps, fn};
|
||||
return hooks.slots[i].fn;
|
||||
},
|
||||
useEffect(fn: () => (() => void) | undefined, deps: unknown[]) {
|
||||
const i = hooks.cursor++, previous = hooks.slots[i];
|
||||
if (!previous || deps.some((v, n) => v !== previous.deps[n])) hooks.effects.push(() => {previous?.cleanup?.(); hooks.slots[i] = {deps, cleanup:fn()};});
|
||||
},
|
||||
}));
|
||||
import { api, ApiError, createApi, type Release } from './api';
|
||||
import { BuildProvenance, ReleasePanel } from './ReleasePanel';
|
||||
import { ResourceState } from './ui';
|
||||
const release: Release = {release_id:'release-1', version:'1.2.3', commit:'abc123', repository:'owner/repo', branch:'main', recipe_rev:'7', platforms:['linux/amd64'], compatible:true, reason:'supported'};
|
||||
const built = {id:'build-1', version:'1.2.3', status:'queued', created_at:''};
|
||||
const onBuilt = vi.fn(async () => {});
|
||||
function nodes(root: any, predicate: (node: any) => boolean): any[] {
|
||||
if (Array.isArray(root)) return root.flatMap(child => nodes(child, predicate));
|
||||
if (!root || typeof root !== 'object') return [];
|
||||
return [...(predicate(root) ? [root] : []), ...nodes(root.props?.children, predicate)];
|
||||
}
|
||||
function render() {hooks.cursor = 0; const tree = ReleasePanel({onBuilt}); hooks.effects.splice(0).forEach(effect => effect()); return tree;}
|
||||
const button = () => nodes(render(), n => n.type === 'button')[0];
|
||||
const settle = () => vi.advanceTimersByTimeAsync(0);
|
||||
beforeEach(() => {
|
||||
hooks.slots = []; hooks.cursor = 0; hooks.effects = []; onBuilt.mockClear();
|
||||
vi.useFakeTimers(); vi.stubGlobal('window', {setInterval, clearInterval, confirm:vi.fn(() => false)});
|
||||
vi.spyOn(api, 'latestRelease').mockResolvedValue({...release});
|
||||
vi.spyOn(api, 'buildRelease').mockResolvedValue(built);
|
||||
vi.spyOn(api, 'build').mockResolvedValue(built);
|
||||
});
|
||||
afterEach(() => {hooks.slots.forEach(s => s?.cleanup?.()); vi.restoreAllMocks(); vi.useRealTimers(); vi.unstubAllGlobals();});
|
||||
it('checks on mount and every five minutes, supports manual refresh, and never automatically builds', async () => {
|
||||
render(); await settle(); expect(api.latestRelease).toHaveBeenCalledTimes(1);
|
||||
await vi.advanceTimersByTimeAsync(299_999); expect(api.latestRelease).toHaveBeenCalledTimes(1);
|
||||
await vi.advanceTimersByTimeAsync(1); expect(api.latestRelease).toHaveBeenCalledTimes(2);
|
||||
await render().props.action.props.onClick(); expect(api.latestRelease).toHaveBeenCalledTimes(3);
|
||||
expect(api.buildRelease).not.toHaveBeenCalled();
|
||||
hooks.slots.forEach(s => s?.cleanup?.());
|
||||
await vi.advanceTimersByTimeAsync(300_000); expect(api.latestRelease).toHaveBeenCalledTimes(3);
|
||||
});
|
||||
it('cancellation sends no mutation; confirmed duplicate clicks submit the exact displayed commit once', async () => {
|
||||
render(); await settle(); await button().props.onClick();
|
||||
expect(api.buildRelease).not.toHaveBeenCalled();
|
||||
expect(window.confirm).toHaveBeenCalledWith(expect.stringContaining(release.commit));
|
||||
vi.mocked(window.confirm).mockReturnValue(true);
|
||||
const submit = button().props.onClick; await Promise.all([submit(), submit()]);
|
||||
expect(api.buildRelease).toHaveBeenCalledExactlyOnceWith(release.commit);
|
||||
expect(onBuilt).toHaveBeenCalledOnce();
|
||||
expect(button().props.disabled).toBe(true);
|
||||
expect(nodes(render(), n => n.type === BuildProvenance)[0].props.build).toEqual(built);
|
||||
});
|
||||
it.each(['succeeded', 'failed'])('polls the accepted task to %s when latest omits its build', async status => {
|
||||
render(); await settle(); vi.mocked(window.confirm).mockReturnValue(true);
|
||||
await button().props.onClick(); render(); await settle();
|
||||
expect(api.build).toHaveBeenCalledExactlyOnceWith(built.id);
|
||||
expect(button().props.disabled).toBe(true);
|
||||
vi.mocked(api.build).mockResolvedValue({...built, status, error:status === 'failed' ? 'retry failed' : undefined});
|
||||
await vi.advanceTimersByTimeAsync(4999); expect(api.build).toHaveBeenCalledTimes(1);
|
||||
await vi.advanceTimersByTimeAsync(1);
|
||||
expect(nodes(render(), n => n.type === BuildProvenance)[0].props.build.status).toBe(status);
|
||||
expect(button().props.disabled).toBe(false);
|
||||
if (status === 'failed') expect(JSON.stringify(render())).toContain('retry failed');
|
||||
await vi.advanceTimersByTimeAsync(10_000); expect(api.build).toHaveBeenCalledTimes(2);
|
||||
expect(api.latestRelease).toHaveBeenCalledTimes(2);
|
||||
expect(api.buildRelease).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
it('prefers a new retry over an old failed build and ignores earlier same-commit task data', async () => {
|
||||
const old = {...built, id:'old-build', status:'failed', error:'old failure'};
|
||||
vi.mocked(api.latestRelease).mockResolvedValue({...release, build:old});
|
||||
vi.mocked(api.build).mockResolvedValue(old);
|
||||
render(); await settle(); render(); await settle();
|
||||
expect(button().props.disabled).toBe(false);
|
||||
vi.mocked(window.confirm).mockReturnValue(true);
|
||||
await button().props.onClick();
|
||||
vi.mocked(api.build).mockResolvedValue({...built, status:'running'});
|
||||
expect(button().props.disabled).toBe(true);
|
||||
expect(nodes(render(), n => n.type === BuildProvenance)[0].props.build.id).toBe(built.id);
|
||||
await button().props.onClick(); expect(api.buildRelease).toHaveBeenCalledTimes(1);
|
||||
await settle(); expect(api.build).toHaveBeenLastCalledWith(built.id);
|
||||
expect(nodes(render(), n => n.props?.role === 'alert')).toHaveLength(0);
|
||||
vi.mocked(api.latestRelease).mockResolvedValue({...release, build:built});
|
||||
await render().props.action.props.onClick();
|
||||
expect(nodes(render(), n => n.type === BuildProvenance)[0].props.build.status).toBe('running');
|
||||
vi.mocked(api.build).mockResolvedValue({...built, status:'failed', error:'new failure'});
|
||||
await vi.advanceTimersByTimeAsync(5000);
|
||||
expect(button().props.disabled).toBe(false);
|
||||
expect(JSON.stringify(render())).toContain('new failure');
|
||||
vi.mocked(api.buildRelease).mockResolvedValue({...built, id:'build-2'});
|
||||
await button().props.onClick();
|
||||
expect(button().props.disabled).toBe(true);
|
||||
expect(nodes(render(), n => n.type === BuildProvenance)[0].props.build.id).toBe('build-2');
|
||||
});
|
||||
it('blocks incompatible releases and shows the server reason', async () => {
|
||||
vi.mocked(api.latestRelease).mockResolvedValue({...release, compatible:false, reason:'unsupported recipe'});
|
||||
render(); await settle(); expect(button().props.disabled).toBe(true);
|
||||
expect(JSON.stringify(render())).toContain('unsupported recipe');
|
||||
await button().props.onClick(); expect(window.confirm).not.toHaveBeenCalled();
|
||||
expect(api.buildRelease).not.toHaveBeenCalled();
|
||||
});
|
||||
it('blocks stale data after refresh failure and exposes retry feedback', async () => {
|
||||
render(); await settle(); vi.mocked(api.latestRelease).mockRejectedValue(new Error('offline'));
|
||||
await render().props.action.props.onClick();
|
||||
expect(button().props.disabled).toBe(true);
|
||||
expect(nodes(render(), n => n.type === ResourceState)[0].props.resource.error).toBe('offline');
|
||||
});
|
||||
it('409 refreshes the branch without retrying a build, and requires new confirmation', async () => {
|
||||
render(); await settle(); vi.mocked(window.confirm).mockReturnValue(true);
|
||||
vi.mocked(api.buildRelease).mockRejectedValueOnce(new ApiError(409, 'changed'));
|
||||
vi.mocked(api.latestRelease).mockResolvedValue({...release, commit:'def456'});
|
||||
await button().props.onClick();
|
||||
expect(api.buildRelease).toHaveBeenCalledTimes(1);
|
||||
expect(JSON.stringify(render())).toContain('再次明确确认');
|
||||
vi.mocked(window.confirm).mockReturnValue(false); await button().props.onClick();
|
||||
expect(window.confirm).toHaveBeenLastCalledWith(expect.stringContaining('def456'));
|
||||
expect(api.buildRelease).toHaveBeenCalledTimes(1);
|
||||
vi.mocked(window.confirm).mockReturnValue(true); await button().props.onClick();
|
||||
expect(api.buildRelease).toHaveBeenLastCalledWith('def456');
|
||||
});
|
||||
it('shows build errors and optional provenance without affecting legacy builds', async () => {
|
||||
const build = {...built, status:'failed', error:'build failed', release_id:release.release_id, source_commit:release.commit, source_repository:release.repository};
|
||||
vi.mocked(api.latestRelease).mockResolvedValue({...release, build});
|
||||
render(); await settle(); expect(JSON.stringify(render())).toContain('build failed');
|
||||
const provenance = JSON.stringify(BuildProvenance({build}));
|
||||
for (const value of [release.release_id, release.commit, release.repository]) expect(provenance).toContain(value);
|
||||
expect(BuildProvenance({build:built})).toBeNull();
|
||||
});
|
||||
it('surfaces build request failure without refreshing or upgrading', async () => {
|
||||
render(); await settle(); vi.mocked(window.confirm).mockReturnValue(true);
|
||||
vi.mocked(api.buildRelease).mockRejectedValue(new ApiError(500, 'builder unavailable'));
|
||||
await button().props.onClick(); expect(JSON.stringify(render())).toContain('builder unavailable');
|
||||
expect(onBuilt).not.toHaveBeenCalled(); expect(api.latestRelease).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
it('uses the assigned endpoints and sends only the pinned commit', async () => {
|
||||
const fetcher = vi.fn<typeof fetch>().mockResolvedValueOnce(new Response(JSON.stringify(release))).mockResolvedValueOnce(new Response(JSON.stringify(built), {status:202}));
|
||||
const client = createApi(fetcher);
|
||||
expect(await client.latestRelease()).toEqual(release);
|
||||
expect(await client.buildRelease(release.commit)).toEqual(built);
|
||||
expect(fetcher.mock.calls[0][0]).toBe('/api/releases/latest');
|
||||
expect(fetcher.mock.calls[1][0]).toBe('/api/releases/build');
|
||||
expect(fetcher.mock.calls[1][1]).toMatchObject({method:'POST', body:JSON.stringify({commit:release.commit})});
|
||||
});
|
||||
@@ -0,0 +1,61 @@
|
||||
import { useState } from 'react';
|
||||
import { api, ApiError, type Build } from './api';
|
||||
import { Panel, ResourceState, Status, useAction, useResource } from './ui';
|
||||
|
||||
export function BuildProvenance({build}: {build: Build}) {
|
||||
if (!build.release_id && !build.source_commit && !build.source_repository) return null;
|
||||
return <div className="mono wrap">
|
||||
{build.release_id && <p>发布标识:{build.release_id}</p>}
|
||||
{build.source_repository && <p>源码仓库:{build.source_repository}</p>}
|
||||
{build.source_commit && <p>源码提交:{build.source_commit}</p>}
|
||||
</div>;
|
||||
}
|
||||
|
||||
export function ReleasePanel({onBuilt}: {onBuilt: () => Promise<void>}) {
|
||||
const release = useResource(api.latestRelease, '', 300_000);
|
||||
const action = useAction();
|
||||
const [submitted, setSubmitted] = useState<{commit: string; build: Build}>();
|
||||
const current = release.data;
|
||||
const accepted = submitted?.commit === current?.commit ? submitted?.build : undefined;
|
||||
const associated = accepted || current?.build;
|
||||
const task = useResource(() => associated ? api.build(associated.id) : Promise.resolve(undefined), associated?.id, associated ? 5000 : 0,
|
||||
value => !!value && ['queued', 'running', 'validating'].includes(value.status));
|
||||
const build = task.data?.id === associated?.id ? task.data : associated;
|
||||
const active = build && ['queued', 'running', 'validating'].includes(build.status);
|
||||
const disabled = action.busy || release.loading || !!release.error || !current?.compatible || !!active;
|
||||
async function submit() {
|
||||
if (disabled || !current) return;
|
||||
await action.run(async () => {
|
||||
let result: Build;
|
||||
try { result = await api.buildRelease(current.commit); }
|
||||
catch (error) {
|
||||
if (error instanceof ApiError && error.status === 409) {
|
||||
await release.refresh();
|
||||
throw new Error('发布分支已变化,已重新查询。请核对最新提交并再次明确确认构建。');
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
setSubmitted({commit: current.commit, build: result});
|
||||
await Promise.all([release.refresh(), onBuilt()]);
|
||||
}, '源码发布构建任务已提交,不会自动升级实例。',
|
||||
`确认手动构建源码发布 ${current.version}?\n仓库:${current.repository}\n分支:${current.branch}\n提交:${current.commit}\n部署协议:${current.recipe_rev}\n平台:${current.platforms.join(', ')}\n构建会消耗磁盘、网络与 CPU;不会自动升级实例,构建成功不等于通过验收。`);
|
||||
}
|
||||
return <Panel title="源码发布(管理员)" action={<button disabled={release.loading || action.busy} onClick={release.refresh}>刷新源码发布</button>}>
|
||||
<p className="muted">进入页面时查询,此后每 5 分钟检查一次。仅检查,不自动构建或升级。新构建由管理员在此提交,创建或升级须另行选择已验证的镜像。</p>
|
||||
<ResourceState resource={release}/>{associated && <ResourceState resource={task}/>} {action.feedback}
|
||||
{current && <>
|
||||
<h3>{current.version}</h3>
|
||||
<div className="mono wrap">
|
||||
<p>发布标识:{current.release_id}</p>
|
||||
<p>源码仓库:{current.repository}</p>
|
||||
<p>分支:{current.branch}</p>
|
||||
<p>源码提交:{current.commit}</p>
|
||||
</div>
|
||||
<p>部署协议:{current.recipe_rev} · 平台:{current.platforms.join(', ')}</p>
|
||||
<p className={current.compatible ? 'callout' : 'alert'}>兼容性:{current.compatible ? '兼容,可手动构建' : '不兼容,禁止构建'} · {current.reason}</p>
|
||||
{build && <div><p>关联构建:{build.id} · <Status value={build.status}/></p><BuildProvenance build={build}/>{build.error && <p className="alert" role="alert">{build.error}</p>}</div>}
|
||||
<button className="primary" disabled={disabled} onClick={submit}>手动构建此源码发布</button>
|
||||
</>}
|
||||
<p className="callout">构建和启动验证通过后可用于实例;发布者仍需完成上线验收,实例升级须另行确认。</p>
|
||||
</Panel>;
|
||||
}
|
||||
+4
-6
@@ -6,11 +6,9 @@ export interface Session { user: User; csrf_token: string }
|
||||
export interface Instance { id: string; name: string; status: string; image_id: string; version: string; created_at: string }
|
||||
export type AdmissionState = 'recommended' | 'compatible' | 'existing_only' | 'blocked';
|
||||
export interface Image { id: string; image_id?: string; version: string; tag?: string; status?: string; admission?: string; admission_reason?: string; selectable?: boolean; usage_count?: number; recipe_rev?: string; platform?: string; created_at?: string }
|
||||
export interface Build { id: string; version: string; status: string; experimental?: boolean; image_id?: string; logs?: string; error?: string; created_at: string }
|
||||
export interface Build { id: string; version: string; status: string; experimental?: boolean; image_id?: string; logs?: string; error?: string; created_at: string; release_id?: string; source_commit?: string; source_repository?: string }
|
||||
export interface Release { release_id: string; version: string; commit: string; repository: string; branch: string; recipe_rev: string; platforms: string[]; compatible: boolean; reason: string; build?: Build }
|
||||
export const selectableImage = (image: Image) => image.status === 'available' && image.selectable === true;
|
||||
export interface CompatibilityRule { state: string; build_allowed: boolean; reason: string }
|
||||
export interface Compatibility { recipe_rev: string; platform: string; entries: (CompatibilityRule & {version:string; recipe_rev:string; platform:string})[]; unknown: CompatibilityRule; experimental_admin_only: boolean }
|
||||
export const normalBuildAllowed = (policy: Compatibility, version: string) => policy.entries.some(entry => entry.version === version && entry.recipe_rev === policy.recipe_rev && entry.platform === policy.platform && entry.build_allowed === true);
|
||||
export interface Entry { name: string; path?: string; is_dir: boolean; size: number; mod_time: string }
|
||||
export interface Usage { used_bytes: number; quota_bytes: number }
|
||||
export interface Backup { id: string; created_at: string; size: number }
|
||||
@@ -83,8 +81,8 @@ export function createApi(fetcher: typeof fetch = fetch) {
|
||||
async logout() { await request('/logout', 'POST'); csrf = ''; },
|
||||
instances: () => request<Instance[]>('/instances'),
|
||||
images: () => request<Image[]>('/images'),
|
||||
compatibility: () => request<Compatibility>('/compatibility'),
|
||||
startBuild: (version: string, experimental = false) => request<Build>('/builds', 'POST', {version, experimental}),
|
||||
latestRelease: () => request<Release>('/releases/latest'),
|
||||
buildRelease: (commit: string) => request<Build>('/releases/build', 'POST', {commit}),
|
||||
admitImage: (image_id: string, state: AdmissionState, evidence: string) => request('/images/admission', 'POST', {image_id, state, evidence}),
|
||||
builds: (limit = 20, offset = 0) => request<Build[]>(`/builds?limit=${limit}&offset=${offset}`),
|
||||
build: (id: string) => request<Build>('/builds/' + encodeURIComponent(id)),
|
||||
|
||||
@@ -1,30 +1,13 @@
|
||||
import { expect, it, vi } from 'vitest';
|
||||
import { createApi, normalBuildAllowed, selectableImage, type AdmissionState, type Compatibility } from './api';
|
||||
import { createApi, selectableImage, type AdmissionState } from './api';
|
||||
|
||||
it('reads the compatibility contract and accepts only explicit current tuples', async () => {
|
||||
const policy: Compatibility = {recipe_rev:'7', platform:'linux/amd64', entries:[{version:'1.2.3', recipe_rev:'7', platform:'linux/amd64', state:'recommended', build_allowed:true, reason:'tested'}], unknown:{state:'existing_only', build_allowed:false, reason:'unknown'}, experimental_admin_only:true};
|
||||
const fetcher = vi.fn<typeof fetch>().mockResolvedValue(new Response(JSON.stringify(policy)));
|
||||
const result = await createApi(fetcher).compatibility();
|
||||
expect(fetcher.mock.calls[0][0]).toBe('/api/compatibility');
|
||||
expect(result).toEqual(policy);
|
||||
expect(normalBuildAllowed(result, '1.2.3')).toBe(true);
|
||||
expect(normalBuildAllowed(result, '9.9.9')).toBe(false);
|
||||
expect(normalBuildAllowed({...result, recipe_rev:'8'}, '1.2.3')).toBe(false);
|
||||
expect(normalBuildAllowed({...result, platform:'linux/arm64'}, '1.2.3')).toBe(false);
|
||||
expect(normalBuildAllowed({...result, entries:result.entries.map(entry => ({...entry, build_allowed:false}))}, '1.2.3')).toBe(false);
|
||||
});
|
||||
|
||||
it('sends exact experimental build and evidence-bearing admission contracts', async () => {
|
||||
it('sends exact-image evidence-bearing admission contracts', async () => {
|
||||
const fetcher = vi.fn<typeof fetch>().mockImplementation(async () => new Response('{}'));
|
||||
const api = createApi(fetcher);
|
||||
await api.startBuild('1.2.3');
|
||||
await api.startBuild('9.9.9', true);
|
||||
for (const state of ['recommended', 'compatible', 'existing_only', 'blocked'] as AdmissionState[]) {
|
||||
await api.admitImage('sha256:a/b', state, '终端、文件与配置验收记录 #42');
|
||||
}
|
||||
expect(fetcher.mock.calls.map(([url, options]) => [url, options?.method, JSON.parse(options?.body as string)])).toEqual([
|
||||
['/api/builds', 'POST', {version:'1.2.3', experimental:false}],
|
||||
['/api/builds', 'POST', {version:'9.9.9', experimental:true}],
|
||||
...['recommended', 'compatible', 'existing_only', 'blocked'].map(state => ['/api/images/admission', 'POST', {image_id:'sha256:a/b', state, evidence:'终端、文件与配置验收记录 #42'}]),
|
||||
]);
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user