Compare commits

...

75 Commits
v0.1.1 ... main

Author SHA1 Message Date
Peter Steinberger
393f980ad3
test: stabilize directory guard coverage
Some checks failed
ci / Lint workflows (push) Has been cancelled
ci / Node ${{ matrix.node }} check (${{ matrix.os }}) (22, macos-latest) (push) Has been cancelled
ci / Node ${{ matrix.node }} check (${{ matrix.os }}) (22, ubuntu-latest) (push) Has been cancelled
ci / Node ${{ matrix.node }} check (${{ matrix.os }}) (22, windows-latest) (push) Has been cancelled
ci / Node ${{ matrix.node }} check (${{ matrix.os }}) (24, macos-latest) (push) Has been cancelled
ci / Node ${{ matrix.node }} check (${{ matrix.os }}) (24, ubuntu-latest) (push) Has been cancelled
ci / Node ${{ matrix.node }} check (${{ matrix.os }}) (24, windows-latest) (push) Has been cancelled
coverage / Node 22 coverage (push) Has been cancelled
2026-05-08 12:29:15 +01:00
Peter Steinberger
d3f37bc700
chore: release 0.2.1 2026-05-08 08:43:17 +01:00
Peter Steinberger
66a4dfba0e
fix: isolate prepack types and secret symlink reads 2026-05-08 08:41:57 +01:00
Peter Steinberger
638999ca7c
Merge pull request #15 from openclaw/codex/ci-all-oses-node-22-24
ci: test all OSes on Node 22 and 24
2026-05-08 02:50:23 +01:00
Peter Steinberger
b132712635
ci: test all supported OSes on Node 22 and 24 2026-05-08 02:47:56 +01:00
Peter Steinberger
b5ad5e9244
Merge pull request #14 from openclaw/ci/windows-runner
fix: align posix/windows path handling and enable windows ci
2026-05-08 02:46:38 +01:00
Peter Steinberger
edca51be7c
docs(changelog): note windows path handling fix 2026-05-08 02:45:02 +01:00
Peter Steinberger
36dd0888a3
fix(root): align pinned fallback path boundary checks 2026-05-08 02:43:24 +01:00
Sarah Fortune
ecefc5bd53 test: avoid windows 8.3 short-name diff in realpath assertions
On windows fs.realpathSync and fs.realpath (async) can disagree on
8.3 short-name canonicalization. The github actions windows runner
exposes this: fs.realpathSync returns "C:\Users\RUNNER~1\..."
while fs.realpath returns "C:\Users\runneradmin\...". Tests that
compare a sync helper's output against await fs.realpath fail with
the same path printed in two forms.

Compare against fs.realpathSync (imported as realpathSync from
node:fs) on both sides so the test exercises the same canonical
form regardless of which short-name configuration the runner has.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 16:23:32 -07:00
Sarah Fortune
4a1e5d8b72 fix(root): drop spurious ".." prefix rejection on posix write path
The posix write helper rejected any path whose computed
path.relative(root, resolved) string-began with "..", even when the
resolved path was fully inside the root. That matched literal
filenames whose first segment merely starts with the two characters
".." (e.g. "..%2fpwned.txt", "..%00/pwned.txt") and produced an
"outside-workspace" error for paths that do not actually escape
the root. The real boundary is already enforced upstream by
resolvePathInRoot's path.resolve + isPathInside check, so the
extra startsWith("..") guard added no security value while
introducing platform divergence (windows did not have it). Drop the
guard, keep the path.isAbsolute check.

Recategorize the three former SAFE_REJECTED_SUSPICIOUS_WRITE_PAYLOADS
test inputs so the test reflects what each platform actually does:

- "..%2fpwned.txt" and "..%00/pwned.txt" are literal names that
  resolve fully inside root on both platforms; move them to
  LITERAL_SUSPICIOUS_WRITE_PAYLOADS (accepted everywhere).
- "..\pwned.txt" is a real traversal on windows where "\\" is a
  separator, but a literal filename on posix where "\\" is a regular
  name character; move it to POSIX_LITERAL_SUSPICIOUS_WRITE_PAYLOADS
  (accepted on posix, rejected on windows).

The literal-directory check in the same test uses fsp.stat on
windows since safeRoot.list goes through the pinned helper that is
unavailable on win32, matching the pattern already used a few lines
up. Bump the per-test timeout to 15s for slow windows fs under
parallel test load.

Drop a stale explanatory comment in expandHomePrefix.

After: 254 passed, 0 failed, 66 skipped on windows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 16:16:50 -07:00
Sarah Fortune
3be7ba6ee3 ci+test: run check on windows and guard windows-only test behavior
Run the check job on windows-latest in addition to ubuntu so the
windows code paths (no O_NOFOLLOW, node fallbacks for fd-relative
ops, ACL inspection) are exercised on every PR rather than only
documented.

Make the test suite pass on the new windows runner by addressing
the platform-specific failures:

- Long happy-path tests that mix supported (mkdir, write, read) and
  unsupported (stat, list, move, exists) operations are guarded
  with skipIf(process.platform === "win32") since the pinned
  filesystem helper throws "unsupported-platform" on win32 by
  design (src/pinned-python.ts).
- Short focused tests where the unsupported operation is the whole
  point (pinned-python, pinned-write-fallback-coverage,
  write-boundary-bypass symlink-move) split into runIf(non-win32)
  and runIf(win32) tests, with the windows variant asserting
  unsupported-platform.
- The expectFsSafeCode helper accepts unsupported-platform on
  windows; new expectedFsSafeCode helper substitutes for
  per-rejects.toMatchObject sites where the windows code differs
  from posix (e.g. path-alias / not-found returning
  unsupported-platform via the helper layer).
- secure-file-reads test split into a posix happy-path runIf and a
  windows runIf that asserts permission-unverified, since ACL
  inspection has no portable equivalent on windows
  (src/secure-file.ts:177).
- safeFileURLToPath test uses hardcoded platform-specific input/
  output instead of building the URL via pathToFileURL+fileURLToPath
  so the assertion verifies the function directly.
- Fix expandHomePrefix to normalize path separators by splitting via
  path.normalize + path.sep and rejoining via path.join. Apply the
  same segment-based check to resolveHomeRelativePath and
  resolveOsHomeRelativePath. Drop input.trim() — whitespace is a
  valid filename character on both platforms and env-var inputs are
  already trimmed upstream via normalizeOptionalString.
- coverage-more's "normalizes empty temp names" decomposes the
  result with path.dirname/path.basename instead of regex-matching
  a path-separator literal.
- extracted-helpers' path-helpers test builds its root with
  path.resolve so the drive letter is present on windows.
- additional-boundary-bypass guards its "..\evil.txt" sanitizer
  assertion behind a non-win32 check (windows reserves "\" as a
  path separator and cannot have it in a filename).
- coverage-more's sibling temp test guards just the posix file-mode
  assertion (stat.mode & 0o777 === 0o600), which has no analog on
  windows. The syncing behaviour the test actually targets still
  runs on both platforms.
- Raise test/new-primitives.test.ts size budget to 1500 to
  accommodate the secure-file-reads test split.

After: 253 passed, 1 failed, 66 skipped on windows-11-arm64. The
single remaining failure is a separate library-side gap (a
SAFE_REJECTED_SUSPICIOUS_WRITE_PAYLOADS payload resolves on windows
instead of rejecting) and will be tracked in a follow-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 14:59:24 -07:00
Peter Steinberger
c7ccb99d30
chore: release 0.2.0
Some checks failed
ci / Node 22 check (push) Waiting to run
coverage / Node 22 coverage (push) Waiting to run
pages / Deploy docs (push) Has been cancelled
2026-05-07 10:59:47 +01:00
Peter Steinberger
12e617ae50
Merge pull request #12 from openclaw/codex/ensure-absolute-directory
Add safe absolute directory creation helper
2026-05-07 10:56:17 +01:00
Peter Steinberger
a81a2c78e3
docs: note absolute directory helper 2026-05-07 10:54:52 +01:00
Peter Steinberger
a431bfc3b8
fix: harden absolute directory segment validation 2026-05-07 10:52:57 +01:00
Peter Steinberger
fb06663ac6
test: split absolute directory regressions 2026-05-07 10:36:15 +01:00
Peter Steinberger
f9e3d30d2d
refactor: structure absolute directory failures 2026-05-07 10:32:16 +01:00
jesse-merhi
aa02b4fa42
fix: guard absolute directory races 2026-05-07 10:29:16 +01:00
jesse-merhi
509076b3a2
docs: document absolute directory helper 2026-05-07 10:29:15 +01:00
jesse-merhi
e91134e92f
feat: add absolute directory ensure helper 2026-05-07 10:29:15 +01:00
Peter Steinberger
d1c1988174
Merge pull request #9 from sallyom/oc-issue-73655
add non-durable atomic write option
2026-05-07 10:28:25 +01:00
sallyom
e335490a5b
add non-durable atomic write option
Signed-off-by: sallyom <somalley@redhat.com>
2026-05-07 10:26:06 +01:00
Peter Steinberger
7ca0af4bac
Merge pull request #11 from openclaw/fix/manifest-safe-exdev-move
fix: preserve concurrent move fallback writes
2026-05-07 10:19:16 +01:00
Peter Steinberger
71ec9f4c10
fix: detect stale move fallback sources 2026-05-07 10:17:11 +01:00
Peter Steinberger
354ba8e4c9
fix: preserve concurrent move fallback writes 2026-05-07 08:58:42 +01:00
Peter Steinberger
c382eafdb2
fix: fail closed on stale sidecar locks 2026-05-07 08:02:36 +01:00
Peter Steinberger
02897e6879
fix: harden filesystem read and temp paths 2026-05-07 08:02:25 +01:00
Peter Steinberger
ce4137f028
test: harden external output coverage 2026-05-07 04:34:31 +01:00
Peter Steinberger
b57002a6a1 fix: preserve external output path spelling (#7) (thanks @jesse-merhi)
Some checks are pending
ci / Node 22 check (push) Waiting to run
coverage / Node 22 coverage (push) Waiting to run
pages / Deploy docs (push) Waiting to run
2026-05-07 03:05:05 +01:00
jesse-merhi
cfda97c828 test: cover external output traversal rejection 2026-05-07 03:05:05 +01:00
jesse-merhi
f4a7bb1a65 feat: add safe external output writer 2026-05-07 03:05:05 +01:00
Sarah Fortune
3412e03c09
Merge pull request #10 from openclaw/fix/pnpm-workspace-packages-field
fix(workspace): add packages field so pnpm prepare succeeds
2026-05-06 16:37:03 -07:00
Sarah Fortune
2a3db08b8d fix(workspace): add packages field so pnpm prepare succeeds
pnpm-workspace.yaml carried only allowBuilds, with no packages field.
Recent pnpm rejects this with ERR_PNPM_INVALID_WORKSPACE_CONFIGURATION
("packages field missing or empty") during the prepare step it runs
inside any consumer that pulls @openclaw/fs-safe from a github tarball,
breaking installs in downstream repos that pin a commit (e.g. openclaw).

Adding packages: [] keeps the file a valid (empty) workspace root while
preserving the allowBuilds: { esbuild: true } directive. Verified by
extracting the working tree to a clean directory and running
pnpm install -- which now completes without error.
2026-05-06 16:33:34 -07:00
Peter Steinberger
85f5b55050
fix(fs): close fallback mkdir and archive cleanup races 2026-05-07 00:19:59 +01:00
Peter Steinberger
500243f398
Merge pull request #8 from openclaw/codex/centralize-fs-boundary-guards
[codex] centralize filesystem boundary guards
2026-05-06 23:55:07 +01:00
Peter Steinberger
b8f079c999
fix(store): preserve sync read validation failures 2026-05-06 23:53:33 +01:00
Peter Steinberger
261ca3cbc0
fix(fs): preserve prune and trash fallback behavior 2026-05-06 23:05:13 +01:00
Peter Steinberger
feb21f0be6
docs(fs): explain guarded cleanup invariants 2026-05-06 22:35:26 +01:00
Peter Steinberger
d27434b50c
fix(fs): avoid unsafe guarded cleanup paths 2026-05-06 22:32:09 +01:00
Peter Steinberger
70fdf86fde
fix(fs): close guarded fallback handles on post-check failure 2026-05-06 22:22:48 +01:00
Peter Steinberger
5218746972
fix(temp): preserve workspace leaf filename contract 2026-05-06 21:54:15 +01:00
Peter Steinberger
55327c8930
ci: enable clawsweeper dispatch 2026-05-06 21:31:17 +01:00
Peter Steinberger
f305c8be2b
fix(fs): preserve public path modes in guard refactor 2026-05-06 21:31:04 +01:00
Peter Steinberger
c2e5849039
docs(fs): document boundary guardrails 2026-05-06 21:21:42 +01:00
Peter Steinberger
925dbfa29b
test(fs): cover centralized boundary regressions 2026-05-06 21:21:40 +01:00
Peter Steinberger
ed5df29ad2
refactor(fs): centralize boundary guard primitives 2026-05-06 21:21:37 +01:00
Peter Steinberger
039e3aa0c8
docs: document filesystem hardening 2026-05-06 20:57:25 +01:00
Peter Steinberger
948a696af6
test: cover filesystem finding regressions 2026-05-06 20:57:25 +01:00
Peter Steinberger
8d6e8c411a
fix(trash): reject changed trash targets 2026-05-06 20:57:25 +01:00
Peter Steinberger
549a083c56
fix(temp): keep temp helpers in private dirs 2026-05-06 20:57:25 +01:00
Peter Steinberger
9b57c2c564
fix(store): validate durable queue entry ids 2026-05-06 20:57:25 +01:00
Peter Steinberger
41463990b0
fix(json): avoid copy fallback symlink writes 2026-05-06 20:57:25 +01:00
Peter Steinberger
03ffe1b9f8
fix(store): pin sync reads and prune traversal 2026-05-06 20:57:25 +01:00
Peter Steinberger
b293131dd0
fix(root): harden pinned write fallback temps 2026-05-06 20:57:24 +01:00
Peter Steinberger
4658071a89
fix(archive): pin staged merge mutations 2026-05-06 20:57:24 +01:00
Peter Steinberger
ee0eb18a6d
fix(root): guard fallback mutator parents 2026-05-06 20:57:24 +01:00
Peter Steinberger
c8fabd7aee
test: add filesystem race regression hooks 2026-05-06 20:57:24 +01:00
Peter Steinberger
3c508734af
fix: make prepack portable on windows 2026-05-06 07:20:21 +01:00
Peter Steinberger
91f7b74ad6
feat: add root JSON and durable queue helpers 2026-05-06 06:27:23 +01:00
Jesse Merhi
26ffb16001
Merge pull request #6 from openclaw/test/rename-boundary-bypass-tests
test: rename boundary bypass suites
2026-05-06 14:45:23 +10:00
jesse-merhi
9cbee5d1b7 test: rename boundary bypass suites 2026-05-06 14:42:49 +10:00
Peter Steinberger
cbe59d156a
docs: package release docs 2026-05-06 04:54:18 +01:00
Peter Steinberger
60e0390332
chore: release 0.1.2 2026-05-06 04:51:16 +01:00
Peter Steinberger
61fadb4365
test: increase filesystem edge coverage 2026-05-06 04:33:06 +01:00
Peter Steinberger
bac1e5ac39
docs: include new pages in site nav 2026-05-06 04:20:05 +01:00
Peter Steinberger
1b78581bca
docs: add missing subpath pages 2026-05-06 04:19:53 +01:00
Peter Steinberger
56abb17dc9
test: stabilize coverage corpus 2026-05-06 04:19:51 +01:00
Peter Steinberger
0cfde5c5f4
test: increase edge coverage 2026-05-06 04:17:16 +01:00
Peter Steinberger
a6149c2cee
fix: harden file store writes 2026-05-06 04:12:15 +01:00
Peter Steinberger
409874b507
docs: refresh readme presentation 2026-05-06 03:39:50 +01:00
Peter Steinberger
3676664dda
docs: add readme shield 2026-05-06 03:34:45 +01:00
Peter Steinberger
cc0757aa82
refactor: split root internals and security test helpers 2026-05-06 03:29:31 +01:00
Peter Steinberger
f0d5fe8ee6
ci: harden github workflows 2026-05-06 03:16:25 +01:00
Peter Steinberger
057d04aa05
ci: update github actions 2026-05-06 03:13:40 +01:00
Peter Steinberger
90ffaf5168
fix: harden root path validation 2026-05-06 03:04:45 +01:00
103 changed files with 8797 additions and 2511 deletions

10
.github/dependabot.yml vendored Normal file
View File

@ -0,0 +1,10 @@
version: 2
updates:
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
groups:
github-actions:
patterns:
- "*"

View File

@ -16,17 +16,20 @@ jobs:
timeout-minutes: 15
steps:
- name: Check out
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up pnpm
uses: pnpm/action-setup@8912a9102ac27614460f54aedde9e1e7f9aec20d # v6.0.5
with:
version: 10.33.2
run_install: false
- name: Set up Node
uses: actions/setup-node@v4
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: "22"
- name: Enable pnpm
run: |
corepack enable
corepack prepare pnpm@10.33.2 --activate
cache: pnpm
cache-dependency-path: pnpm-lock.yaml
- name: Install dependencies
run: pnpm install --frozen-lockfile
@ -42,7 +45,7 @@ jobs:
pnpm benchmark -- --json dist/benchmarks/results.json --markdown dist/benchmarks/summary.md
- name: Upload benchmark results
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: fs-safe-benchmark-${{ github.run_id }}
path: dist/benchmarks/

View File

@ -11,23 +11,47 @@ permissions:
contents: read
jobs:
check:
name: Node 22 check
lint-workflows:
name: Lint workflows
runs-on: ubuntu-latest
timeout-minutes: 15
timeout-minutes: 5
steps:
- name: Check out
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Lint workflows
uses: rhysd/actionlint@914e7df21a07ef503a81201c76d2b11c789d3fca # v1.7.12
check:
name: Node ${{ matrix.node }} check (${{ matrix.os }})
runs-on: ${{ matrix.os }}
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
node:
- 22
- 24
os:
- ubuntu-latest
- macos-latest
- windows-latest
steps:
- name: Check out
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up pnpm
uses: pnpm/action-setup@8912a9102ac27614460f54aedde9e1e7f9aec20d # v6.0.5
with:
version: 10.33.2
run_install: false
- name: Set up Node
uses: actions/setup-node@v4
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: "22"
- name: Enable pnpm
run: |
corepack enable
corepack prepare pnpm@10.33.2 --activate
node-version: ${{ matrix.node }}
cache: pnpm
cache-dependency-path: pnpm-lock.yaml
- name: Install dependencies
run: pnpm install --frozen-lockfile

View File

@ -0,0 +1,116 @@
name: ClawSweeper Dispatch
on:
issues:
types: [opened, reopened, edited, labeled, unlabeled]
issue_comment:
types: [created, edited]
pull_request_target: # zizmor: ignore[dangerous-triggers] maintainer-owned external dispatch; no checkout or untrusted PR code execution
types: [opened, reopened, synchronize, ready_for_review, edited, labeled, unlabeled]
permissions:
contents: read
concurrency:
group: clawsweeper-dispatch-${{ github.repository }}-${{ github.event.issue.number || github.event.pull_request.number || github.run_id }}
cancel-in-progress: ${{ github.event.action == 'edited' || github.event.action == 'synchronize' || github.event.action == 'ready_for_review' }}
jobs:
dispatch:
runs-on: ubuntu-latest
if: ${{ !(endsWith(github.actor, '[bot]') && (github.event.action == 'labeled' || github.event.action == 'unlabeled')) }}
env:
HAS_CLAWSWEEPER_APP_PRIVATE_KEY: ${{ secrets.CLAWSWEEPER_APP_PRIVATE_KEY != '' }}
CLAWSWEEPER_APP_CLIENT_ID: Iv23liOECG0slfuhz093
SUPERSEDES_IN_PROGRESS: ${{ (github.event.action == 'edited' || github.event.action == 'synchronize' || github.event.action == 'ready_for_review') && 'true' || 'false' }}
steps:
- name: Debounce bursty metadata events
if: ${{ github.event.action == 'labeled' || github.event.action == 'unlabeled' }}
run: sleep 20
- name: Create ClawSweeper dispatch token
id: token
if: ${{ env.HAS_CLAWSWEEPER_APP_PRIVATE_KEY == 'true' }}
uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 # v3.1.1
with:
client-id: ${{ env.CLAWSWEEPER_APP_CLIENT_ID }}
private-key: ${{ secrets.CLAWSWEEPER_APP_PRIVATE_KEY }}
owner: openclaw
repositories: clawsweeper
permission-contents: write
- name: Create target comment token
id: target_token
if: ${{ github.event_name == 'issue_comment' && env.HAS_CLAWSWEEPER_APP_PRIVATE_KEY == 'true' }}
uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 # v3.1.1
with:
client-id: ${{ env.CLAWSWEEPER_APP_CLIENT_ID }}
private-key: ${{ secrets.CLAWSWEEPER_APP_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: ${{ github.event.repository.name }}
permission-issues: write
permission-pull-requests: read
- name: Dispatch exact ClawSweeper review
if: ${{ github.event_name != 'issue_comment' }}
env:
GH_TOKEN: ${{ steps.token.outputs.token }}
TARGET_REPO: ${{ github.repository }}
ITEM_NUMBER: ${{ github.event.issue.number || github.event.pull_request.number }}
ITEM_KIND: ${{ github.event_name == 'pull_request_target' && 'pull_request' || 'issue' }}
SOURCE_EVENT: ${{ github.event_name }}
SOURCE_ACTION: ${{ github.event.action }}
run: |
if [ -z "$GH_TOKEN" ]; then
echo "::notice::Skipping ClawSweeper dispatch because no dispatch credential is configured."
exit 0
fi
payload="$(jq -nc \
--arg target_repo "$TARGET_REPO" \
--argjson item_number "$ITEM_NUMBER" \
--arg item_kind "$ITEM_KIND" \
--arg source_event "$SOURCE_EVENT" \
--arg source_action "$SOURCE_ACTION" \
--argjson supersedes_in_progress "$SUPERSEDES_IN_PROGRESS" \
'{event_type:"clawsweeper_item",client_payload:{target_repo:$target_repo,item_number:$item_number,item_kind:$item_kind,source_event:$source_event,source_action:$source_action,supersedes_in_progress:$supersedes_in_progress}}')"
gh api repos/openclaw/clawsweeper/dispatches \
--method POST \
--input - <<< "$payload"
- name: Acknowledge and dispatch ClawSweeper comment
if: ${{ github.event_name == 'issue_comment' }}
env:
DISPATCH_TOKEN: ${{ steps.token.outputs.token }}
TARGET_TOKEN: ${{ steps.target_token.outputs.token }}
TARGET_REPO: ${{ github.repository }}
ITEM_NUMBER: ${{ github.event.issue.number }}
COMMENT_ID: ${{ github.event.comment.id }}
COMMENT_BODY: ${{ github.event.comment.body }}
SOURCE_ACTION: ${{ github.event.action }}
run: |
if [ -z "$DISPATCH_TOKEN" ]; then
echo "::notice::Skipping ClawSweeper dispatch because no dispatch credential is configured."
exit 0
fi
body_file="$RUNNER_TEMP/clawsweeper-comment-body.txt"
printf '%s\n' "$COMMENT_BODY" > "$body_file"
if ! grep -Eiq '(^|[[:space:]])@clawsweeper\b|(^|[[:space:]])/(clawsweeper|review|re-review|rerun[ -]?review|status|explain|fix|build|implement|create[ -]?pr|fix[ -]?issue|autofix|auto[ -]?fix|automerge|auto[ -]?merge|approve|stop|autoclose)\b' "$body_file"; then
echo "No ClawSweeper command found in comment."
exit 0
fi
if [ -n "$TARGET_TOKEN" ]; then
GH_TOKEN="$TARGET_TOKEN" gh api -X POST \
-H "Accept: application/vnd.github+json" \
"repos/$TARGET_REPO/issues/comments/$COMMENT_ID/reactions" \
-f content="eyes" >/dev/null || true
fi
payload="$(jq -nc \
--arg target_repo "$TARGET_REPO" \
--argjson item_number "$ITEM_NUMBER" \
--argjson comment_id "$COMMENT_ID" \
--arg source_event "issue_comment" \
--arg source_action "$SOURCE_ACTION" \
'{event_type:"clawsweeper_comment",client_payload:{target_repo:$target_repo,item_number:$item_number,comment_id:$comment_id,source_event:$source_event,source_action:$source_action}}')"
GH_TOKEN="$DISPATCH_TOKEN" gh api repos/openclaw/clawsweeper/dispatches \
--method POST \
--input - <<< "$payload"

View File

@ -19,17 +19,20 @@ jobs:
timeout-minutes: 15
steps:
- name: Check out
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up pnpm
uses: pnpm/action-setup@8912a9102ac27614460f54aedde9e1e7f9aec20d # v6.0.5
with:
version: 10.33.2
run_install: false
- name: Set up Node
uses: actions/setup-node@v4
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: "22"
- name: Enable pnpm
run: |
corepack enable
corepack prepare pnpm@10.33.2 --activate
cache: pnpm
cache-dependency-path: pnpm-lock.yaml
- name: Install dependencies
run: pnpm install --frozen-lockfile
@ -64,7 +67,7 @@ jobs:
- name: Upload coverage results
if: always()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: fs-safe-coverage-${{ github.run_id }}
path: coverage/

View File

@ -30,10 +30,10 @@ jobs:
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Check out
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Node
uses: actions/setup-node@v4
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: "22"
@ -41,13 +41,13 @@ jobs:
run: node scripts/build-docs-site.mjs
- name: Configure Pages
uses: actions/configure-pages@v5
uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0
with:
path: dist/docs-site
- name: Deploy
id: deployment
uses: actions/deploy-pages@v4
uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5.0.0

2
.gitignore vendored
View File

@ -2,3 +2,5 @@ dist/
node_modules/
coverage/
*.tsbuildinfo
.deepsec/
.vscode

View File

@ -1,5 +1,73 @@
# Changelog
## Unreleased
## 0.2.1 - 2026-05-08
### Fixes
- Align POSIX and Windows handling for literal `..`-prefixed write targets, preserve whitespace in direct home-relative path inputs, and run the check suite on Windows CI. (#14; thanks @sjf)
- Keep source prepack builds isolated from parent monorepo ambient type packages such as Bun typings. (#13; thanks @Kaspre)
- Let secret-file reads follow symlink paths through the pinned real target unless callers opt into `rejectSymlink: true`.
## 0.2.0 - 2026-05-07
### Features
- Add `writeExternalFileWithinRoot()` for libraries that require an output path while preserving caller-provided destination names. (#7; thanks @jesse-merhi)
- Add root JSON helpers and durable JSON queue helpers for file-backed work queues with pending, delivered, failed, and acknowledgement flows.
- Add `ensureAbsoluteDirectory()` for creating trusted absolute directory paths one segment at a time while rejecting symlink and non-directory components. (#12; thanks @jesse-merhi)
- Add a `durable: false` option to async atomic text and JSON writes so callers can preserve replace semantics while skipping temp-file and parent-directory fsync. (#9; thanks @sallyom)
- Add process-wide sidecar lock defaults while keeping JSON store locking opt-in per resource.
### Security and Correctness
- Harden Root fallback mutators, archive merges, private store reads/writes, durable queue ids, JSON fallback writes, sibling temp writes, temp filename sanitization, and trash moves against symlink-swap and path traversal edge cases.
- Centralize safe path segment validation, directory identity guards, guarded mkdir, and guarded mutation wrappers so filesystem helpers reuse the same race-resistant checks.
- Route archive ZIP staging, temp workspace sync reads, secret-file commits, and atomic move/replace fallbacks through shared pinned-read or guarded-write primitives without applying private-directory modes to public paths.
- Close guarded fallback write handles without following path names if post-write directory verification fails, avoiding descriptor leaks and unsafe cleanup in symlink-swap races.
- Harden temp filename prefixes, local-root reads, private store imports, durable queue reads, and regular-file byte caps against Deepsec-reported path traversal, symlink, and oversized-read races.
- Harden sidecar lock cleanup and stale-lock handling so stale third-party locks fail closed instead of being deleted by path.
### Compatibility
- Make cross-device move fallbacks reject source changes during staged copies and clean up only the source entries copied into the staged destination, preserving concurrent source additions or replacements instead of recursively deleting them.
- Preserve directory modes during cross-device directory moves.
- Preserve empty-directory pruning and broken-symlink trash moves across guarded fallback paths.
- Preserve sync file-store read policy errors for directory and hardlink validation failures.
- Preserve existing temp workspace leaf filename behavior for names such as `.env` and filenames containing spaces.
- Preserve public parent-directory modes when writing JSON, moving files across devices, and extracting archives.
- Make `prepack` portable on Windows and add the missing pnpm workspace `packages` field so package preparation succeeds consistently.
### Tests
- Added regression coverage for the filesystem race and traversal findings fixed in this release.
- Added Deepsec regression coverage for unsafe temp tokens, dangling symlinks, default read caps, private `copyIn()` races, symlinked queue entries, oversized queue entries, and fresh sidecar lock preservation.
- Added regression coverage for external-output traversal rejection, guarded cleanup, sidecar lock stale handling, move fallback cleanup, durable queue validation, sync read policy failures, and absolute-directory validation.
- Added a static filesystem-boundary primitive check that blocks reintroducing known raw copy/read/guard patterns.
### Docs and Tooling
- Added docs for external output writers, durable JSON queue helpers, sidecar lock defaults, boundary guardrails, and absolute-directory creation.
- Enable ClawSweeper dispatch for pull-request review automation.
## 0.1.2 - 2026-05-06
### Fixes
- Reject `fileStore()` and `fileStoreSync()` writes through symlinked parent directories so store commits cannot escape the configured root.
### Tests
- Increased filesystem edge coverage around secure temp fallback handling, sibling-temp cleanup, local-root resolution, file locks, and file identity checks.
- Prevented POSIX test runs from leaving Windows-style secure-temp fallback paths in the repository root.
### Docs
- Added missing docs pages for `@openclaw/fs-safe/config`, `@openclaw/fs-safe/store`, `@openclaw/fs-safe/advanced`, and `@openclaw/fs-safe/test-hooks`.
- Corrected path-helper docs for the synchronous `isPathInsideWithRealpath` and `safeRealpathSync` behavior.
- Included the Markdown docs in the npm package so README links resolve after install.
## 0.1.1 - 2026-05-06
### Fixes

View File

@ -1,4 +1,10 @@
# @openclaw/fs-safe
# 🛡️ @openclaw/fs-safe
[![npm](https://img.shields.io/npm/v/@openclaw/fs-safe.svg?color=10b981&label=npm)](https://www.npmjs.com/package/@openclaw/fs-safe)
[![ci](https://github.com/openclaw/fs-safe/actions/workflows/ci.yml/badge.svg)](https://github.com/openclaw/fs-safe/actions/workflows/ci.yml)
[![node](https://img.shields.io/node/v/@openclaw/fs-safe.svg?color=10b981)](https://nodejs.org)
[![license](https://img.shields.io/npm/l/@openclaw/fs-safe.svg?color=10b981)](LICENSE)
[![docs](https://img.shields.io/badge/docs-fs--safe.io-10b981)](https://fs-safe.io)
Capability-style filesystem roots for Node.js apps that handle untrusted relative paths.
@ -14,6 +20,12 @@ await fs.write("../escape.txt", "x"); // throws FsSafeError("outside-
That's the whole pitch. `root()` is the product; the rest of the package — JSON stores, atomic writes, secret files, archive extraction, temp workspaces — is supporting cast for the same boundary.
Full docs and reference at **[fs-safe.io](https://fs-safe.io)**.
## Contents
[Why this exists](#why-this-exists) · [Not a sandbox](#not-a-sandbox) · [Install](#install) · [Quick start](#quick-start) · [Reading](#reading) · [Subpaths](#subpaths) · [Failure semantics](#failure-semantics-in-the-name) · [Atomic writes](#atomic-writes) · [External outputs](#external-outputs) · [Stores](#stores) · [Secure absolute reads](#secure-absolute-file-reads) · [Walking](#directory-walking) · [Archive extraction](#archive-extraction) · [Path scopes](#advanced-path-scopes) · [Errors](#errors) · [Safety model](#safety-model) · [Limitations](#limitations)
## Why this exists
Most Node code that has to touch caller-controlled paths reaches for:
@ -26,6 +38,13 @@ That validates a *string*. It does not pin the file you opened, defend against a
The same idea has landed in other languages. Go [added `os.Root` and `OpenInRoot`](https://go.dev/blog/osroot); Rust has had [`cap-std`](https://github.com/bytecodealliance/cap-std) for years. Node's `fs` is path-string-oriented and exposes flags like `O_NOFOLLOW` but not an ergonomic "operate inside this root" API. `fs-safe` fills that gap.
| | Root boundary | Atomic writes | Symlink/hardlink defense | TOCTOU resistance | Archive extraction |
|---|---|---|---|---|---|
| `path.resolve().startsWith()` | string check only | | | | |
| [`write-file-atomic`](https://www.npmjs.com/package/write-file-atomic) | | ✓ | | | |
| Go [`os.Root`](https://go.dev/blog/osroot) / Rust [`cap-std`](https://github.com/bytecodealliance/cap-std) | ✓ | platform | ✓ | ✓ | |
| **`@openclaw/fs-safe`** | **✓** | **✓** | **✓** | **✓ (POSIX fd-relative)** | **✓ (ZIP/TAR)** |
## Not a sandbox
This is a **library-level guardrail**, not OS-level isolation. It does not replace containers, seccomp, AppArmor, or filesystem permissions. It is for code that already runs with the privileges of its workspace and wants to stop trivial path tricks from escaping it. If your threat model is a hostile process, you need OS isolation; if your threat model is "an agent, plugin, upload handler, or CLI will eventually be tricked into writing somewhere it shouldn't," `fs-safe` catches that.
@ -153,6 +172,7 @@ that OpenClaw needs to compose higher-level APIs are grouped under
| `@openclaw/fs-safe/config` | process-global Python helper configuration |
| `@openclaw/fs-safe/path` | canonical path checks: `isPathInside`, `safeRealpathSync`, `isNotFoundPathError`, `isSymlinkOpenError` |
| `@openclaw/fs-safe/json` | `tryReadJson`, `readJson`, `readJsonIfExists`, `writeJson`, sync variants |
| `@openclaw/fs-safe/output` | `writeExternalFileWithinRoot` for external libraries that need a temp output path |
| `@openclaw/fs-safe/store` | `fileStore`, `fileStoreSync`, and `jsonStore` |
| `@openclaw/fs-safe/secret` | strict and try-style secret file read/write helpers |
| `@openclaw/fs-safe/atomic` | `replaceFileAtomic`, `replaceFileAtomicSync`, `replaceDirectoryAtomic`, `movePathWithCopyFallback` |
@ -178,9 +198,14 @@ await tryReadJson("./config.json"); // returns null on missing or invalid
await readJson("./manifest.json"); // throws on missing or invalid
```
For one-off structured reads under a trusted root, `readRootJsonObjectSync()`
performs the root-bounded open and JSON object validation in one step. Use
`readRootStructuredFileSync()` when the parser lives outside fs-safe, such as
JSON5-backed plugin manifests.
## Atomic writes
`replaceFileAtomic()` writes a sibling temp file, optionally fsyncs it, and renames it over the destination. Mode preservation, rename retry / copy fallback on `EPERM`, parent-directory fsync, and a `beforeRename` hook for backup or observer flows are all opt-in.
`replaceFileAtomic()` writes a sibling temp file, optionally fsyncs it, and renames it over the destination. Mode preservation, rename retry / copy fallback on `EPERM`, parent-directory fsync, and a `beforeRename` hook for backup or observer flows are all opt-in. `movePathWithCopyFallback()` stages cross-device moves before commit and removes only the copied source entries, so concurrent source additions or replacements are preserved.
```ts
import { replaceFileAtomic } from "@openclaw/fs-safe/atomic";
@ -196,6 +221,32 @@ await replaceFileAtomic({
`replaceFileAtomicSync()` covers the synchronous case with the same options shape. Both accept an injectable `fileSystem` for tests.
## External outputs
Use `writeExternalFileWithinRoot()` when a browser download, renderer, media
tool, or native library needs an absolute path to write to:
```ts
import { writeExternalFileWithinRoot } from "@openclaw/fs-safe/output";
await writeExternalFileWithinRoot({
rootDir: "/safe/workspace/downloads",
path: "reports/today.pdf",
write: async (filePath) => {
await download.saveAs(filePath);
},
});
```
The callback receives a private temp file path, not the final destination. After
the callback returns, fs-safe finalizes the staged file with `Root.copyIn()`,
creating missing parents by default and rejecting traversal, symlink parent
escapes, hardlinked final targets, and size-limit violations.
Use it when the final filename is known before the external writer runs. If the
filename depends on sniffing the produced bytes, write to a private temp
workspace first, then finalize through the normal root APIs after validation.
## Stores
Use `fileStore().json()` for small state files that need explicit fallback
@ -241,6 +292,11 @@ const opened = await media.open("inbound/photo.jpg");
await media.pruneExpired({ ttlMs: 10 * 60 * 1000, recursive: true });
```
The `store` subpath also includes durable JSON queue helpers for the common
"one JSON file per work item" pattern: atomic entry writes, pending-entry loads,
acknowledgement via `.delivered` markers, failed-entry moves, and stale temp
cleanup. Retry, dedupe, and transport semantics stay with the caller.
`tempWorkspace()` exposes `write()`, `writeText()`, `writeJson()`, `copyIn()`, and `read()` for
single-file scratch workflows without hand-rolled path joins, plus a `store: FileStore` view of
the workspace dir for the richer cases (`writeStream`, `readJsonIfExists`, `store.json<T>(rel)`).

110
docs/advanced.md Normal file
View File

@ -0,0 +1,110 @@
---
title: Advanced
description: "Lower-level composition helpers under @openclaw/fs-safe/advanced. Less stable than focused public subpaths."
---
# `@openclaw/fs-safe/advanced`
Composition primitives that OpenClaw uses to build higher-level APIs. They are public — semver applies — but treated as a less stable surface than the focused subpaths (`root`, `json`, `store`, `temp`, `archive`, `errors`). Reach for them only when you are building a primitive of your own and the focused subpaths do not cover it.
```ts
import {
pathScope,
withTimeout,
pathExists,
sanitizeUntrustedFileName,
// …
} from "@openclaw/fs-safe/advanced";
```
## What lives here
The exports group into a handful of themes. Each documented helper has its own page; everything else is reference-only and tracked here.
### Path scopes and root paths
| Export | Page | Notes |
|---|---|---|
| `pathScope`, `PathScope`, `PathScopeOptions`, `PathScopeResolveOptions` | [path-scope.md](path-scope.md) | Absolute-path boundary helper with `Result`-shaped returns. |
| `ensureDirectoryWithinRoot` | | Create a directory while enforcing the root boundary. |
| `resolvePathWithinRoot`, `resolvePathsWithinRoot` | | Resolve one or many relative paths against a trusted root. |
| `resolveExistingPathsWithinRoot`, `resolveStrictExistingPathsWithinRoot` | | Same, but require the targets to exist. |
| `resolveWritablePathWithinRoot` | | Resolve a write target inside a root. |
| `resolveRootPath`, `resolveRootPathSync`, `ResolvedRootPath`, `ROOT_PATH_ALIAS_POLICIES`, `RootPathAliasPolicy` | | Resolve a root directory honoring alias policy. |
| `resolvePathViaExistingAncestorSync` | | Walk to an existing ancestor for paths whose tail does not yet exist. |
### Absolute path helpers
| Export | Page | Notes |
|---|---|---|
| `assertAbsolutePathInput` | | Validate a caller-supplied absolute path string. |
| `ensureAbsoluteDirectory`, `EnsureAbsoluteDirectoryOptions`, `EnsureAbsoluteDirectoryResult` | | Create a trusted absolute directory path one segment at a time, rejecting symlink or non-directory segments. |
| `canonicalPathFromExistingAncestor`, `findExistingAncestor` | | Canonicalize without requiring the leaf to exist. |
| `resolveAbsolutePathForRead`, `resolveAbsolutePathForWrite`, `ResolvedAbsolutePath`, `ResolvedWritableAbsolutePath`, `AbsolutePathSymlinkPolicy` | | Validate an absolute path against a symlink policy before opening. |
`ensureAbsoluteDirectory()` is for paths you already intend to trust as absolute
locations, such as a configured output root. It does not enforce a root boundary;
use `pathScope().ensureDir()` or `ensureDirectoryWithinRoot()` when the caller
supplies a path that must stay under a root.
The helper returns `{ ok: false, code, error }` for path-policy failures such as
relative paths, symlinks, non-directories, or directory swaps during creation.
Operational filesystem failures such as permissions or I/O errors are rethrown.
### Files and identity
| Export | Page | Notes |
|---|---|---|
| `openRootFile`, `openRootFileSync`, `canUseRootFileOpen`, `matchRootFileOpenFailure`, related types | | Low-level no-follow open routed through the root-file path. |
| `appendRegularFile`, `appendRegularFileSync`, `readRegularFile`, `readRegularFileSync`, `statRegularFile`, `statRegularFileSync`, `resolveRegularFileAppendFlags`, `AppendRegularFileOptions`, `RegularFileStatResult` | [regular-file.md](regular-file.md) | Type-checked regular-file I/O. |
| `sameFileIdentity`, `FileIdentityStat` | | Compare two stats for same-inode equality. |
| `pathExists`, `pathExistsSync` | | Boolean existence check that does not throw on `ENOENT`. |
| `assertNoSymlinkParents`, `assertNoSymlinkParentsSync`, `AssertNoSymlinkParentsOptions` | | Reject paths whose ancestor chain contains symlinks. |
| `assertNoHardlinkedFinalPath`, `assertNoPathAliasEscape`, `PATH_ALIAS_POLICIES`, `PathAliasPolicy` | | Hardlink/alias defense building blocks. |
### Local roots and file URLs
| Export | Page | Notes |
|---|---|---|
| `resolveLocalPathFromRootsSync`, `readLocalFileFromRoots`, related options/result types | [local-roots.md](local-roots.md) | Resolve a path against a list of allowed local roots. |
| `assertNoWindowsNetworkPath`, `basenameFromMediaSource`, `hasEncodedFileUrlSeparator`, `isWindowsDriveLetterPath`, `isWindowsNetworkPath`, `safeFileURLToPath`, `trySafeFileURLToPath` | | Defensive helpers around Windows paths and `file://` URLs. |
### Install paths and filenames
| Export | Page | Notes |
|---|---|---|
| `safeDirName`, `safePathSegmentHashed`, `resolveSafeInstallDir`, `assertCanonicalPathWithinBase` | [install-path.md](install-path.md) | Build install-target directories from caller-supplied identifiers. |
| `sanitizeUntrustedFileName` | [filename.md](filename.md) | Coerce an untrusted string into a safe filename. |
| `resolveHomeRelativePath` | | Expand `~`-prefixed paths. |
### Temp targets and sibling-temp writes
| Export | Page | Notes |
|---|---|---|
| `tempFile`, `withTempFile`, `TempFile`, `buildRandomTempFilePath`, `sanitizeTempFileName` | [temp.md](temp.md) | One-file temp primitive; prefer `tempWorkspace` from `@openclaw/fs-safe/temp` for the stable surface. |
| `writeSiblingTempFile`, `writeViaSiblingTempPath`, `WriteSiblingTempFileOptions`, `WriteSiblingTempFileResult` | | Sibling-temp write building block used by `replaceFileAtomic`. |
### Permissions
| Export | Page | Notes |
|---|---|---|
| `formatPosixMode` | [permissions.md](permissions.md) | Format a POSIX mode bitmask. |
| `inspectWindowsAcl`, `summarizeWindowsAcl`, `formatWindowsAclSummary`, `parseIcaclsOutput`, `resolveWindowsUserPrincipal`, `createIcaclsResetCommand`, `formatIcaclsResetCommand`, `IcaclsResetCommandOptions`, `PermissionExec`, `WindowsAclEntry`, `WindowsAclSummary` | [permissions.md](permissions.md) | Windows ACL inspection and remediation. |
### Concurrency, timing, trash
| Export | Page | Notes |
|---|---|---|
| `createAsyncLock` | | In-process async lock (separate from cross-process file locks). |
| `withTimeout` | [timing.md](timing.md) | Wrap a promise with a timeout that raises `FsSafeError("timeout")`. |
| `movePathToTrash`, `MovePathToTrashOptions` | | Best-effort move to the platform trash. |
## Stability
Items in this surface can change shape between minor versions if a higher-level primitive needs them to. Pin to a minor version if you depend on a specific helper, or open an issue at the [GitHub repo](https://github.com/openclaw/fs-safe) and we will discuss promoting it to a focused subpath.
## Related pages
- [Root API](root.md) — built on top of these helpers.
- [Errors](errors.md) — every helper here surfaces failures as `FsSafeError`.
- [Security model](security-model.md) — what the underlying boundary checks promise.

View File

@ -68,7 +68,12 @@ If `beforeRename` throws, the rename is skipped and the temp file is removed —
### `EPERM` and copy fallback
On systems where `rename` fails with `EPERM`/`EEXIST`, pass `copyFallbackOnPermissionError: true` to fall back to copy + unlink. The fallback refuses symlink destinations before copying so it does not write through a replaced destination link.
On systems where `rename` fails with `EPERM`/`EEXIST`, pass
`copyFallbackOnPermissionError: true` to fall back to a non-atomic copy
replacement. The fallback removes the old destination, opens the replacement
with exclusive/no-follow flags where the platform supports them, and refuses
known symlink destinations so it does not write through a replaced destination
link.
### Sync variant
@ -109,21 +114,44 @@ await writeTextAtomic("/srv/workspace/rendered.md", rendered, {
});
```
Options:
```ts
type WriteTextAtomicOptions = {
mode?: number; // file mode (default 0o600)
dirMode?: number; // mode for parent dirs created on demand
trailingNewline?: boolean; // append "\n" if missing
durable?: boolean; // default true; false skips temp/parent fsync
};
```
`durable: false` keeps the sibling-temp replace/rename behavior but skips the
temp-file and parent-directory `fsync` calls. Use it only for reconstructible
metadata where lower latency matters more than crash-durability.
## `movePathWithCopyFallback`
Rename a path. If the rename fails with `EXDEV` (cross-device) or `EPERM`, fall back to copy + remove. Preserves atomicity at the destination by writing the copy through `replaceFileAtomic` (for files) or staged-rename (for directories).
Rename a path. If the rename fails with `EXDEV` (cross-device), fall back to
copying into a staged sibling path, renaming that staged path into place, and
then removing only the source entries that were copied. The fallback avoids
buffering regular files into memory and does not tighten the destination parent
directory mode.
```ts
import { movePathWithCopyFallback } from "@openclaw/fs-safe/atomic";
await movePathWithCopyFallback({
source: "/srv/cache/blob.bin",
destination: "/srv/persistent/blob.bin",
overwrite: true,
from: "/srv/cache/blob.bin",
sourceHardlinks: "reject",
to: "/srv/persistent/blob.bin",
});
```
Use it when source and destination might live on different filesystems (containers, tmpfs, separate volumes).
If another writer changes source entries during the fallback, the staged copy
throws `ESTALE` before commit when possible. If the destination has already
been committed, cleanup still preserves the changed source entries and throws
`ESTALE`.
## Difference from `root()`

94
docs/config.md Normal file
View File

@ -0,0 +1,94 @@
---
title: Config
description: "Process-global defaults for optional fs-safe helpers."
---
# `@openclaw/fs-safe/config`
Process-global configuration knobs for optional fs-safe helpers. The Python helper policy is described in the [Python helper policy](python-helper.md); this page is the API reference.
```ts
import {
configureFsSafePython,
configureFsSafeLocks,
getFsSafePythonConfig,
getFsSafeLockConfig,
type FsSafeLockConfig,
type FsSafePythonConfig,
type FsSafePythonMode,
} from "@openclaw/fs-safe/config";
```
These functions are also re-exported from the main entry point. Prefer the subpath when you only need helper configuration and want the smallest import surface.
## `configureFsSafePython(config)`
```ts
function configureFsSafePython(config: Partial<FsSafePythonConfig>): void;
type FsSafePythonConfig = {
mode: FsSafePythonMode;
pythonPath?: string;
};
type FsSafePythonMode = "auto" | "off" | "require";
```
Set the process-global policy. Calls merge into the existing override config, so passing `{ pythonPath: "/usr/bin/python3" }` keeps any previously set `mode`. Configure once at startup, before the first `root()` call — switching modes mid-process is supported but the helper may already be running.
| Mode | Behavior |
|---|---|
| `auto` | Default. Use the helper when it starts; fall back to Node-only behavior if Python is missing or fails to start. |
| `off` | Never spawn the helper. Read/write/move use Node fallbacks plus pre/post identity checks. |
| `require` | Fail closed if the helper cannot start. Operations that need the helper raise `FsSafeError("helper-unavailable")`. |
## `getFsSafePythonConfig()`
```ts
function getFsSafePythonConfig(): FsSafePythonConfig;
```
Return the effective configuration: programmatic overrides win, then env vars, then the package default (`auto`).
## `configureFsSafeLocks(config)`
```ts
function configureFsSafeLocks(config: Partial<FsSafeLockConfig>): void;
type FsSafeLockConfig = {
staleRecovery: "fail-closed";
staleMs?: number;
timeoutMs?: number;
retry?: FileLockRetryOptions;
};
```
Set process-wide defaults for sidecar lock options. This does **not** turn locking on globally; callers still need to pass `lock: true` or a lock options object for the specific JSON store/resource that needs cross-process coordination.
`staleRecovery` currently supports `"fail-closed"` only. Stale third-party sidecars are not deleted by path because Node cannot atomically bind that deletion to the file that was inspected.
## `getFsSafeLockConfig()`
```ts
function getFsSafeLockConfig(): FsSafeLockConfig;
```
Return the current sidecar lock defaults.
## Environment variables
The same policy can be set without code:
```bash
FS_SAFE_PYTHON_MODE=auto # auto | off | require | true | false | on | off | 1 | 0 | never | required
FS_SAFE_PYTHON=/usr/bin/python3
```
OpenClaw compatibility aliases are accepted: `OPENCLAW_FS_SAFE_PYTHON_MODE`, `OPENCLAW_FS_SAFE_PYTHON`, `OPENCLAW_PINNED_PYTHON`, and `OPENCLAW_PINNED_WRITE_PYTHON`. Programmatic overrides via `configureFsSafePython` always win.
## Related pages
- [Python helper policy](python-helper.md) — when to pick `auto`, `off`, or `require`, and what each mode protects.
- [File lock](sidecar-lock.md) — the per-resource lock API that consumes lock defaults.
- [Root API](root.md) — the API whose POSIX hardening the helper backs.
- [Errors](errors.md) — `helper-unavailable` and `helper-failed`.

View File

@ -49,9 +49,12 @@ await fs.remove("notes/archive/today.txt");
| Surface | Use it for |
|---|---|
| [`root()`](root.md) | One boundary for read/write/move/remove inside a trusted directory. |
| [`@openclaw/fs-safe/config`](config.md) | Process-global Python helper and lock-option defaults. |
| [Python helper policy](python-helper.md) | Choose `auto`, `off`, or `require` for POSIX fd-relative hardening. |
| [`replaceFileAtomic`](atomic.md) | Sibling-temp + rename, fsync hooks, mode preservation, copy fallback. |
| [`writeExternalFileWithinRoot`](output.md) | Stage external-library file output in private temp storage, then finalize under a root. |
| [`writeJson` / `readJson*`](json.md) | JSON state files with strict and lenient read variants. |
| [`@openclaw/fs-safe/store`](store.md) | Overview of `fileStore`, `fileStoreSync`, and `jsonStore`. |
| [`jsonStore`](json-store.md) | Single JSON state file with explicit fallback, atomic writes, and optional locking. |
| [`fileStore`](file-store.md) | Managed multi-file/blob store with modes, stream writes, copy-in, pruning, and private mode. |
| [Private file-store mode](private-file-store.md) | `fileStore({ private: true })` for private JSON/text state at 0600 under 0700 dirs. |
@ -61,9 +64,11 @@ await fs.remove("notes/archive/today.txt");
| [`extractArchive`](archive.md) | ZIP/TAR extraction with size, count, link, and traversal limits. |
| [Secret files](secret-file.md) | Mode-0600 credentials with size and TOCTOU defense. |
| [Permissions](permissions.md) | POSIX mode and Windows ACL inspection/remediation helpers. |
| [`acquireFileLock`](sidecar-lock.md) | Cross-process file lock with retry and stale-lock recovery. |
| [`acquireFileLock`](sidecar-lock.md) | Cross-process file lock with retry and fail-closed stale-lock handling. |
| [`FsSafeError`](errors.md) | Closed code union (with `policy` / `operational` category) you can branch on. |
| [`pathScope()`](path-scope.md) | Lower-level absolute-path boundary helper; lives behind `@openclaw/fs-safe/advanced`. |
| [`@openclaw/fs-safe/advanced`](advanced.md) | Directory of lower-level composition helpers (path scopes, regular-file I/O, install paths, sibling-temp writes, …). |
| [`@openclaw/fs-safe/test-hooks`](test-hooks.md) | Test-only injection hooks for reproducing open/lstat races. |
## Status

View File

@ -53,6 +53,7 @@ type JsonStoreLockOptions = {
staleMs?: number; // default 30_000
timeoutMs?: number; // default 30_000
retry?: FileLockRetryOptions;
staleRecovery?: "fail-closed";
managerKey?: string; // default `fs-safe.json-store:<filePath>`
};
@ -130,6 +131,7 @@ const counter = jsonStore<{ count: number }>({
lock: {
staleMs: 60_000,
timeoutMs: 10_000,
staleRecovery: "fail-closed",
retry: { retries: 30, minTimeout: 100, maxTimeout: 5_000, randomize: true },
},
});
@ -137,6 +139,8 @@ const counter = jsonStore<{ count: number }>({
When `lock` is falsy, `read` / `write` / `update` are unlocked. The `update` shape is still useful — it gives you a single function for the read-modify-write pattern — but it offers no concurrency guarantees if other processes also write to the file.
Process-wide lock defaults from `configureFsSafeLocks()` apply only after locking is explicitly enabled. They do not make JSON stores lock by default.
The default `managerKey` namespaces the in-process `FileLockManager` per absolute file path, so two `jsonStore` calls on the same file share lock state automatically.
## Common patterns

View File

@ -9,6 +9,9 @@ import {
readJsonIfExists,
readJsonSync,
tryReadJsonSync,
readRootJsonSync,
readRootJsonObjectSync,
readRootStructuredFileSync,
writeJson,
writeJsonSync,
JsonFileReadError,
@ -69,6 +72,32 @@ Synchronous strict reader. Throws `JsonFileReadError` on missing or invalid inpu
Synchronous, generic, lenient. Returns `T | null`. Useful in boot paths where you want a typed result without async.
## Root-bounded structured reads
Use the root-bounded readers when you already have a trusted root directory and
a caller-controlled relative path, but you only need one synchronous structured
read instead of a full `root()` handle.
```ts
const result = readRootJsonObjectSync({
rootDir: "/safe/workspace",
relativePath: "plugin/openclaw.plugin.json",
boundaryLabel: "plugin manifest",
});
if (!result.ok) {
// reason is "open", "parse", or "invalid"
throw new Error(result.reason);
}
console.log(result.value);
```
`readRootJsonSync()` parses any JSON value. `readRootJsonObjectSync()` only
accepts objects. `readRootStructuredFileSync()` accepts a custom parser and
validator so callers can layer JSON5, TOML, YAML, or domain-specific validation
without making `fs-safe` depend on those formats.
## Writing
### `writeJson(filePath, value, options?)`
@ -86,9 +115,14 @@ type WriteJsonOptions = {
mode?: number; // file mode (default 0o600)
dirMode?: number; // mode for parent dirs created on demand
trailingNewline?: boolean; // append "\n" if missing (default false)
durable?: boolean; // default true; false skips temp/parent fsync
};
```
`durable: false` preserves atomic temp-file replacement but skips the temp-file
and parent-directory `fsync` calls. Use it only for reconstructible JSON state
where lower latency matters more than crash-durability.
### `writeJsonSync(pathname, data)`
Synchronous variant. Convenience wrapper that uses the sync atomic-write path with sensible defaults.

92
docs/output.md Normal file
View File

@ -0,0 +1,92 @@
# External outputs
`@openclaw/fs-safe/output` covers the case where another library insists on
writing to an absolute path you give it. Browser downloads, renderers, media
tools, and native libraries often have this shape:
```ts
import { writeExternalFileWithinRoot } from "@openclaw/fs-safe/output";
await writeExternalFileWithinRoot({
rootDir: "/srv/workspace/downloads",
path: "reports/today.pdf",
write: async (filePath) => {
await download.saveAs(filePath);
},
});
```
The external writer never receives the final destination path. It receives a
private temp file path instead. After the callback returns, fs-safe copies that
staged file into the requested target through the same root boundary used by
`Root.copyIn()`.
## Signature
```ts
function writeExternalFileWithinRoot<T = void>(
options: ExternalFileWriteOptions<T>,
): Promise<ExternalFileWriteResult<T>>;
type ExternalFileWriteOptions<T = void> = {
rootDir: string;
path: string; // relative or absolute, but must stay under rootDir
write: (filePath: string) => Promise<T>;
maxBytes?: number;
mode?: number;
};
type ExternalFileWriteResult<T = void> = {
path: string; // final absolute path under the canonical root
result: T; // value returned by write()
};
```
The requested `path` must name a file. Missing destination parents are created
by the helper because the operation is "produce this output file under the
root"; callers should choose the filename before calling this API.
Use `maxBytes` when the external producer can create arbitrarily large files.
Use `mode` when the finalized file needs a specific POSIX mode. Both are
enforced during the `Root.copyIn()` finalization step, after the external writer
has produced the staged file and before the final target is committed.
## Why not pass the final path to the library?
If a target parent can be swapped after validation, handing an external library
the final path can make the library write outside the intended root before
fs-safe has a chance to finalize or reject the operation. This helper stages in
a private temp workspace first, then finalizes with `Root.copyIn()`. That keeps
the trust-boundary write inside fs-safe's root-aware copy/atomic-write path.
## Browser download example
```ts
const outputPath = requestedOutputPath || sanitizeBrowserSuggestedName(suggestedFilename);
await writeExternalFileWithinRoot({
rootDir: downloadsRoot,
path: outputPath,
maxBytes: 512 * 1024 * 1024,
write: async (filePath) => {
await download.saveAs(filePath);
},
});
```
The chosen path may be absolute if it is already inside `downloadsRoot`, or
relative to `downloadsRoot`. Traversal, symlink parent escapes, hardlinked final
targets, over-large staged files, and missing temp files surface as
`FsSafeError`s.
This helper is not the right fit when the final filename depends on inspecting
the produced bytes. In that case, write to a private temp workspace, sniff or
validate the file, choose the final name, then copy or write into the root with
the normal root APIs.
## See also
- [Root writes](writing.md) — `write`, `copyIn`, `move`, and `mkdir`.
- [Temp workspaces](temp.md) — private scratch directories for longer workflows.
- [`pathScope()`](path-scope.md) — validation-only helper when you must pass an
absolute path directly to another library.

View File

@ -10,13 +10,14 @@ import {
resolveSafeBaseDir,
safeRealpathSync,
safeStatSync,
assertNoNulPathInput,
isNotFoundPathError,
isSymlinkOpenError,
hasNodeErrorCode,
} from "@openclaw/fs-safe/path";
```
These helpers are also re-exported from the main entry; the `/path` subpath is for callers that want the smallest possible import.
Only `root()`, `FsSafeError`, and the Python helper config live on the main entry. Path helpers are deliberately a subpath import so the main entry stays small.
## Boundary checks
@ -33,15 +34,22 @@ isPathInside("/srv/uploads", "/srv/uploads"); // true (root itsel
The check is platform-aware: on Windows, paths are normalized for case and separator before comparison.
### `isPathInsideWithRealpath(rootDir, target)`
### `isPathInsideWithRealpath(rootDir, target, opts?)`
Async. Same as `isPathInside`, but resolves both inputs through `realpath` first. Use this when you want the canonical answer and either input might be a symlink.
Synchronous. Same as `isPathInside`, but resolves both inputs through `realpath` first. Use this when you want the canonical answer and either input might be a symlink.
```ts
await isPathInsideWithRealpath("/srv/uploads", "/srv/symlink-to-elsewhere"); // false
isPathInsideWithRealpath("/srv/uploads", "/srv/symlink-to-elsewhere"); // false
```
Throws on `realpath` failure for either input. Catch with `isNotFoundPathError(err)` if you want missing inputs to be a "no" rather than an exception.
```ts
type Options = {
requireRealpath?: boolean; // default true
cache?: Map<string, string>;
};
```
Does not throw on missing inputs — `realpath` failures are absorbed by the underlying `safeRealpathSync`. By default (`requireRealpath: true`) the function returns `false` when either input cannot be resolved. Pass `{ requireRealpath: false }` to fall back to the lexical answer from `isPathInside` instead.
### `isWithinDir(rootDir, targetPath)`
@ -59,14 +67,14 @@ const base = resolveSafeBaseDir("/srv/uploads/."); // "/srv/uploads"
### `safeRealpathSync(targetPath, cache?)`
Synchronous `realpath` that returns `null` instead of throwing on missing paths. Pass an optional `Map<string, string>` to cache results across calls within a single operation.
Synchronous `realpath` that returns `null` instead of throwing on any error. Pass an optional `Map<string, string>` to cache results across calls within a single operation.
```ts
const real = safeRealpathSync("/srv/uploads/photo.jpg");
if (real === null) return notFound();
```
Errors other than `ENOENT` propagate normally.
All `realpath` failures collapse to `null` — there is no distinction between `ENOENT`, `EACCES`, and other I/O errors. Use `fs.realpathSync` directly if you need to branch on the error code.
### `safeStatSync(targetPath)`
@ -77,6 +85,10 @@ const stat = safeStatSync("/srv/uploads/photo.jpg");
if (!stat?.isFile()) return notFound();
```
### `assertNoNulPathInput(filePath, message?)`
Throws `FsSafeError` with code `invalid-path` when a path string contains an embedded NUL byte. Use it before calling Node `fs` APIs directly; Node's native error can include raw path text in the message.
## Error inspection
### `isNotFoundPathError(err)`
@ -128,7 +140,7 @@ try {
if (isNotFoundPathError(err)) return reply(404);
throw err;
}
if (!await isPathInsideWithRealpath("/srv/uploads", canonical)) {
if (!isPathInsideWithRealpath("/srv/uploads", canonical)) {
return reply(403);
}
```

View File

@ -27,7 +27,7 @@ type RootDefaults = {
};
```
`root()` resolves the directory through the real filesystem. A symlinked input becomes the canonical path; a non-existent root throws `FsSafeError` with code `not-found`.
`root()` resolves the directory through the real filesystem. A symlinked input becomes the canonical path; a non-existent root throws `FsSafeError` with code `not-found`, and malformed or non-directory roots throw `invalid-path`.
`defaults` apply to every method on the returned handle. Per-call options on individual methods override the defaults for that call only.
@ -127,6 +127,7 @@ Every method throws `FsSafeError` with a `code`. Branch on `err.code`, not messa
| Code | When it fires |
|---|---|
| `invalid-path` | The input path is malformed, including embedded NUL bytes. |
| `outside-workspace` | The input resolves outside the root, or contains a `..` segment that would escape it. |
| `not-found` | The target does not exist (or its parent does not, with `mkdir: false`). |
| `not-file` | A read or copy targeted a non-regular file (directory, FIFO, socket, …). |

View File

@ -1,6 +1,6 @@
# File lock
`acquireFileLock()` and `withFileLock()` provide a cross-process file lock with retry, stale-lock reclaim, and process-exit cleanup. The lock is implemented as a sidecar file (e.g. `state.json``state.json.lock`) — only one acquirer can create the sidecar with `O_CREAT | O_EXCL` at a time.
`acquireFileLock()` and `withFileLock()` provide a cross-process file lock with retry and process-exit cleanup. The lock is implemented as a sidecar file (e.g. `state.json``state.json.lock`) — only one acquirer can create the sidecar with `O_CREAT | O_EXCL` at a time.
```ts
import { acquireFileLock } from "@openclaw/fs-safe/file-lock";
@ -19,9 +19,9 @@ try {
## Why sidecar?
The lock file sits next to the protected resource. If a process crashes mid-lock, the next acquirer notices the held entry, inspects its payload (PID, host, acquired-at timestamp), and decides — via `shouldReclaim` (defaulting to "is the lock older than `staleMs`?") — whether to take it over.
The lock file sits next to the protected resource. If a process crashes mid-lock, the next acquirer notices the held entry, inspects its payload (PID, host, acquired-at timestamp), and decides — via `shouldReclaim` (defaulting to "is the lock older than `staleMs`?") — whether it should keep waiting or fail.
The library installs a `process.on("exit")` handler that releases all currently-held locks synchronously, so well-behaved exits leave no stale sidecars. Crashes still need the reclaim path.
The library installs a `process.on("exit")` handler that releases all currently-held locks synchronously, so well-behaved exits leave no stale sidecars. Crashed holders leave their sidecar behind; remove those through an application-owned recovery path after you have proved the holder cannot still be writing.
## API
@ -48,9 +48,10 @@ function createFileLockManager(key: string): FileLockManager;
type FileLockAcquireOptions<TPayload extends Record<string, unknown>> = {
managerKey?: string; // optional in-process manager namespace
lockPath?: string; // override; defaults to `${targetPath}.lock`
staleMs: number; // how long until a held lock is considered stale
staleMs?: number; // default 30_000
timeoutMs?: number; // overall acquire deadline; default unbounded
retry?: FileLockRetryOptions;
staleRecovery?: "fail-closed"; // default
allowReentrant?: boolean; // if this process already holds it, increment a count instead of failing
payload: () => TPayload | Promise<TPayload>;
shouldReclaim?: (params: {
@ -100,7 +101,7 @@ try {
}
```
If your process dies before `release()` runs and skips the exit handler, the next acquirer reclaims the lock once `staleMs` elapses (or your `shouldReclaim` returns true).
If your process dies before `release()` runs and skips the exit handler, the sidecar remains. Once `staleMs` elapses (or your `shouldReclaim` returns true), acquisition fails closed instead of deleting by path, because Node cannot atomically bind that deletion to the file that was inspected.
## `withFileLock` — common shape made one-liner
@ -139,9 +140,9 @@ await handle.release();
await locks.drain();
```
## Reclaim policy: `shouldReclaim`
## Stale policy: `shouldReclaim`
The default policy reclaims locks whose `acquiredAt` is older than `staleMs`. Pass a custom callback when you want a richer notion of "is the holder still alive":
The default policy treats locks whose `createdAt` is older than `staleMs` as stale. Pass a custom callback when you want a richer notion of "is the holder still alive":
```ts
import { kill } from "node:process";
@ -155,26 +156,26 @@ const handle = await acquireFileLock(targetPath, {
if (!Number.isFinite(pid)) return true;
try {
kill(pid, 0);
return false; // process still alive — don't reclaim
return false; // process still alive — keep waiting
} catch {
return true; // process gone — reclaim
return true; // process gone — fail closed for recovery
}
},
});
```
`heldByThisProcess` is true when this manager already holds the lock (relevant for the reentrant case).
`heldByThisProcess` is true when this manager already holds the lock (relevant for the reentrant case). A `true` result does not delete the sidecar; it lets the acquire loop stop waiting once the retry/timeout policy says to give up.
## What sidecar locks defend against
- **Two processes writing the same file at once.** `acquire` serializes the critical section.
- **A crashed holder leaving a stale lock.** `staleMs` plus optional `shouldReclaim` recovers it.
- **Accidentally deleting a fresh lock during stale recovery.** Stale third-party locks fail closed because safe compare-and-unlink is not available through Node's path APIs.
- **Race between simultaneous acquire attempts.** `O_CREAT | O_EXCL` ensures one wins.
## What they do **not** defend against
- **Misbehaving holders that ignore the lock.** Locks are advisory — only callers that go through `acquire` are bound.
- **Holders that never call `release` and have no liveness check.** Without a real `shouldReclaim`, the lock relies on `staleMs` alone — pick a deadline that is comfortably longer than your real work but short enough to recover from crashes.
- **Automatic stale lock deletion.** If a process crashes, use the payload and your own supervisor/process table to decide when to remove the sidecar.
- **Multi-host coordination over network filesystems.** Behavior depends on the underlying filesystem's `O_EXCL` semantics; treat as best-effort.
## Common patterns

Binary file not shown.

Before

Width:  |  Height:  |  Size: 119 KiB

After

Width:  |  Height:  |  Size: 114 KiB

View File

@ -27,48 +27,48 @@
<rect x="0" y="0" width="1200" height="4" fill="#10b981"/>
<!-- shield mark, large on the left -->
<g transform="translate(150, 205)">
<path d="M110 0 L220 38 L220 134 C220 207 173 260 110 282 C47 260 0 207 0 134 L0 38 Z"
<g transform="translate(120, 222)">
<path d="M93 0 L186 32 L186 113 C186 175 146 220 93 239 C40 220 0 175 0 113 L0 32 Z"
fill="url(#shieldFill)"/>
<path d="M50 134 L97 178 L182 95"
fill="none" stroke="#0a0e16" stroke-width="20"
<path d="M42 113 L82 151 L154 80"
fill="none" stroke="#0a0e16" stroke-width="17"
stroke-linecap="round" stroke-linejoin="round"/>
</g>
<!-- text block -->
<g font-family="Inter, ui-sans-serif, system-ui, -apple-system, Segoe UI, sans-serif" fill="#f1f5f9">
<!-- eyebrow -->
<text x="540" y="245" font-size="22" font-weight="600" letter-spacing="3"
<text x="430" y="252" font-size="20" font-weight="600" letter-spacing="3"
fill="#34d399" opacity="0.95">
NODE.JS · TYPESCRIPT · MIT
</text>
<!-- product name -->
<text x="540" y="335" font-size="100" font-weight="700" letter-spacing="-2">
<text x="430" y="332" font-size="88" font-weight="700" letter-spacing="-2">
fs-safe
</text>
<!-- tagline -->
<text x="540" y="395" font-size="32" font-weight="500" fill="#cbd2dc">
<text x="430" y="386" font-size="28" font-weight="500" fill="#cbd2dc">
Safe filesystem primitives for Node.js
</text>
<!-- description -->
<g font-size="21" fill="#9aa3b2" font-weight="400">
<text x="540" y="450">Race-resistant. Root-bounded.</text>
<text x="540" y="480">Defends symlink swaps, hardlink aliases,</text>
<text x="540" y="510">traversal, and TOCTOU rename races.</text>
<g font-size="19" fill="#9aa3b2" font-weight="400">
<text x="430" y="436">Race-resistant. Root-bounded.</text>
<text x="430" y="463">Defends symlink swaps, hardlink aliases,</text>
<text x="430" y="490">traversal, and TOCTOU rename races.</text>
</g>
<!-- url footer -->
<text x="540" y="575" font-family="JetBrains Mono, SF Mono, ui-monospace, monospace"
font-size="22" font-weight="500" fill="#5eead4">
<text x="430" y="548" font-family="JetBrains Mono, SF Mono, ui-monospace, monospace"
font-size="20" font-weight="500" fill="#5eead4">
fs-safe.io
</text>
</g>
<!-- bottom accent dots -->
<g transform="translate(1040, 575)" opacity="0.85">
<g transform="translate(1050, 548)" opacity="0.85">
<circle cx="0" cy="0" r="6" fill="#10b981"/>
<circle cx="22" cy="0" r="6" fill="#10b981" opacity="0.55"/>
<circle cx="44" cy="0" r="6" fill="#10b981" opacity="0.25"/>

Before

Width:  |  Height:  |  Size: 3.0 KiB

After

Width:  |  Height:  |  Size: 3.0 KiB

77
docs/store.md Normal file
View File

@ -0,0 +1,77 @@
---
title: Store
description: "Overview of @openclaw/fs-safe/store: fileStore, fileStoreSync, and jsonStore."
---
# `@openclaw/fs-safe/store`
The `store` subpath bundles two managed wrappers around the same safe-write primitives `root()` uses:
```ts
import {
ensureJsonDurableQueueDirs,
fileStore,
fileStoreSync,
jsonStore,
loadPendingJsonDurableQueueEntries,
resolveJsonDurableQueueEntryPaths,
writeJsonDurableQueueEntry,
type FileStore,
type FileStoreOptions,
type FileStoreSync,
type JsonStore,
type JsonStoreOptions,
} from "@openclaw/fs-safe/store";
```
| Helper | Use it for |
|---|---|
| [`fileStore()`](file-store.md) | Multi-file directories with safe relative paths, size limits, atomic replacement, stream writes, copy-in, and TTL-based pruning. |
| `fileStoreSync()` | Synchronous variant of `fileStore()` for places that genuinely cannot await. |
| [`jsonStore()`](json-store.md) | A single keyed JSON state file with explicit fallback, atomic writes, and optional sidecar locking around read-modify-write updates. |
| Durable JSON queue helpers | Append/load/ack JSON entry files using atomic writes and delivered markers. |
| [Private file-store mode](private-file-store.md) | `fileStore({ private: true })` for credentials, tokens, and per-agent state at `0600` files under `0700` directories. |
`fileStore().json("rel.json")` and `jsonStore({ filePath })` are intentionally separate primitives. Use `fileStore().json(...)` when JSON state lives alongside other files in the same managed directory; use `jsonStore({ filePath })` when you have a single absolute path and want the keyed JSON shape directly.
## Picking a shape
- **Multi-file directory under one root** — reach for `fileStore()`. It exposes `write`, `writeJson`, `writeText`, `writeStream`, `read*`, `open`, `copyIn`, `remove`, and `pruneExpired` against safe relative paths.
- **One JSON state file** — reach for `jsonStore({ filePath })`. Its `update()` and `updateOr()` methods cover the merge-into-defaults and read-modify-write cases.
- **Credentials or tokens** — pass `private: true` to `fileStore()`. Same store shape; writes route through the secret-file atomic path with `0600`/`0700` permissions.
- **Durable work queues** — use the durable JSON queue helpers when each work item is a standalone JSON file and acknowledgement is represented by moving it through a short-lived `.delivered` marker.
## Durable JSON queues
The durable queue helpers are intentionally low-level. They do not decide retry,
dedupe, or recovery policy; they just provide the filesystem mechanics that
several queue implementations otherwise rewrite by hand.
```ts
await ensureJsonDurableQueueDirs({ queueDir, failedDir });
const paths = resolveJsonDurableQueueEntryPaths(queueDir, id);
await writeJsonDurableQueueEntry({
filePath: paths.jsonPath,
entry,
tempPrefix: "queue",
});
const pending = await loadPendingJsonDurableQueueEntries({ queueDir, tempPrefix: "queue" });
```
`id` must be a single safe path segment: non-empty, not dot-prefixed, and made
from letters, numbers, `_`, `-`, and `.`. Slashes, backslashes, NUL bytes, `.`,
and `..` are rejected.
Use `ackJsonDurableQueueEntry()` after durable processing succeeds and
`moveJsonDurableQueueEntryToFailed()` when the caller wants to quarantine an
entry for inspection.
## Related pages
- [`fileStore`](file-store.md) — full API for the multi-file store.
- [`jsonStore`](json-store.md) — single-file JSON store with locking.
- [Private file-store mode](private-file-store.md) — credential-shaped variant.
- [JSON files](json.md) — lower-level `readJson` / `writeJson` helpers.
- [Atomic writes](atomic.md) — what `fileStore` and `jsonStore` use under the hood.

View File

@ -175,9 +175,15 @@ const result = await writeSiblingTempFile<string>({
`writeSiblingTempFile` chooses a random sibling name in `dir`, calls your `writeTemp()` callback, validates that `resolveFinalPath(result)` is still inside that same directory, and renames the temp file there.
By default it preserves the historical private-helper behavior of chmodding
`dir` to `dirMode` (default `0o700`). Pass `chmodDir: false` when the directory
is a public staging/output path whose existing mode must be preserved.
### `writeViaSiblingTempPath`
A higher-level convenience — write content + rename in one call:
A higher-level convenience for callback-based producers. The callback writes to
a private temp path, then the helper copies the result into `targetPath` through
the root boundary:
```ts
import { writeViaSiblingTempPath } from "@openclaw/fs-safe/advanced";
@ -191,7 +197,9 @@ await writeViaSiblingTempPath({
});
```
If `replaceFileAtomic` does what you need, prefer that — `writeViaSiblingTempPath` is the lower-level building block.
If `replaceFileAtomic` does what you need, prefer that. Use
`writeViaSiblingTempPath` when the producer needs a concrete temp pathname but
the final destination still needs root-boundary checks.
## Secure temp root

74
docs/test-hooks.md Normal file
View File

@ -0,0 +1,74 @@
---
title: Test hooks
description: "Internal-only injection hooks used by the fs-safe test suite. Active only when NODE_ENV=test or VITEST=true."
---
# `@openclaw/fs-safe/test-hooks`
Internal injection points the `fs-safe` test suite uses to deterministically reproduce open/lstat races. They are exposed as a public subpath so downstream test suites can reuse the same harness, but they are **not** part of the supported runtime API.
```ts
import {
getFsSafeTestHooks,
__setFsSafeTestHooksForTest,
type FsSafeTestHooks,
} from "@openclaw/fs-safe/test-hooks";
```
## When the hooks are active
Hooks are only honored when one of the following is true:
- `process.env.NODE_ENV === "test"`
- `process.env.VITEST === "true"`
Calling `__setFsSafeTestHooksForTest(hooks)` outside of those environments throws. `getFsSafeTestHooks()` returns `undefined` when no hooks are registered, regardless of the environment.
## Shape
```ts
type FsSafeTestHooks = {
afterPreOpenLstat?: (filePath: string) => Promise<void> | void;
beforeOpen?: (filePath: string, flags: number) => Promise<void> | void;
afterOpen?: (filePath: string, handle: FileHandle) => Promise<void> | void;
};
```
| Hook | Fires when |
|---|---|
| `afterPreOpenLstat` | A pre-open `lstat` has just resolved. Use this to swap a path between validation and open. |
| `beforeOpen` | The library is about to call `open(path, flags)`. Use this to inject a TOCTOU window. |
| `afterOpen` | An open just succeeded. Use this to mutate state before the post-open identity check runs. |
Each hook may be sync or async; async hooks are awaited.
## Usage
```ts
import { afterEach, beforeEach } from "vitest";
import { __setFsSafeTestHooksForTest } from "@openclaw/fs-safe/test-hooks";
beforeEach(() => {
__setFsSafeTestHooksForTest({
beforeOpen: async (filePath) => {
// swap a victim file with a symlink right before fs-safe opens it
await replaceWithSymlink(filePath);
},
});
});
afterEach(() => {
__setFsSafeTestHooksForTest(undefined);
});
```
Always clear the hooks in `afterEach` so a stuck hook does not leak across tests.
## Stability
The shape can grow new optional fields between minor versions. Treat the surface as test-only and do not rely on it from production code.
## Related pages
- [Testing](testing.md) — broader notes on testing against `fs-safe`.
- [Security model](security-model.md) — the races these hooks help reproduce.

View File

@ -145,6 +145,32 @@ it("writes and reads through the boundary", async () => {
For tests that need a private temp workspace, [`withTempWorkspace`](temp.md) makes the setup-and-teardown story trivial.
## Repo test shards
Run the full local gate before handoff:
```sh
pnpm check
```
Run only the security boundary corpus while iterating on root/path/archive/temp hardening:
```sh
pnpm test:security
```
Run the static primitive guard after changing low-level filesystem helpers:
```sh
pnpm lint:fs-boundary
```
It catches the specific raw fallback patterns that previously led to
check-then-use bugs, such as direct copy-to-destination fallback and sync temp
workspace reads that bypass pinned file descriptors.
`pnpm check` also runs `pnpm lint:file-size`. New source and test files should stay under 500 lines. Existing larger files have explicit budgets in `scripts/check-file-size.mjs`; do not increase those budgets as part of unrelated work.
## See also
- [Security model](security-model.md) — what the boundary is supposed to defend; design tests around the same threats.

View File

@ -1,6 +1,6 @@
{
"name": "@openclaw/fs-safe",
"version": "0.1.1",
"version": "0.2.1",
"description": "Capability-style filesystem roots for Node.js apps that handle untrusted relative paths.",
"license": "MIT",
"repository": {
@ -11,6 +11,7 @@
"dist/**/*.js",
"dist/**/*.d.ts",
"dist/**/*.d.ts.map",
"docs/**/*.md",
"README.md",
"CHANGELOG.md",
"SECURITY.md",
@ -35,6 +36,10 @@
"types": "./dist/path.d.ts",
"default": "./dist/path.js"
},
"./output": {
"types": "./dist/output.d.ts",
"default": "./dist/output.js"
},
"./advanced": {
"types": "./dist/advanced.d.ts",
"default": "./dist/advanced.js"
@ -95,10 +100,13 @@
"scripts": {
"benchmark": "node scripts/benchmark.mjs",
"build": "tsc -p tsconfig.json",
"lint:file-size": "node scripts/check-file-size.mjs",
"lint:fs-boundary": "node scripts/check-fs-boundary-primitives.mjs",
"prepack": "node scripts/prepack-build.mjs",
"test": "vitest run",
"test:coverage": "vitest run --coverage",
"check": "pnpm build && pnpm test",
"test:security": "vitest run test/fs-safe.test.ts test/openclaw-read-bypass-parity.test.ts test/openclaw-write-bypass-parity.test.ts test/additional-bypass-parity.test.ts test/adversarial-boundary-payloads.test.ts",
"check": "pnpm lint:file-size && pnpm lint:fs-boundary && pnpm build && pnpm test",
"docs:site": "node scripts/build-docs-site.mjs"
},
"optionalDependencies": {

View File

@ -1,2 +1,4 @@
packages: []
allowBuilds:
esbuild: true

View File

@ -26,13 +26,13 @@ const productDescription =
const installCmd = "pnpm add @openclaw/fs-safe";
const sections = [
["Start", ["index.md", "install.md", "quickstart.md", "security-model.md", "python-helper.md"]],
["Start", ["index.md", "install.md", "quickstart.md", "security-model.md", "python-helper.md", "config.md"]],
["Root API", ["root.md", "reading.md", "writing.md", "path-scope.md"]],
["Atomic & temp", ["atomic.md", "json.md", "temp.md", "archive.md"]],
["Stores", ["json-store.md", "file-store.md", "private-file-store.md"]],
["Atomic & temp", ["atomic.md", "output.md", "json.md", "temp.md", "archive.md"]],
["Stores", ["store.md", "json-store.md", "file-store.md", "private-file-store.md"]],
["Specialized", ["secret-file.md", "regular-file.md", "sidecar-lock.md", "pinned-open.md", "local-roots.md"]],
["Path & filename", ["path.md", "filename.md", "install-path.md"]],
["Reference", ["errors.md", "types.md", "testing.md", "timing.md", "contributing.md"]],
["Reference", ["errors.md", "types.md", "testing.md", "timing.md", "advanced.md", "test-hooks.md", "contributing.md"]],
];
const buildExcludes = [];

View File

@ -0,0 +1,46 @@
import fs from "node:fs";
import path from "node:path";
const DEFAULT_MAX_LINES = 500;
const LINE_BUDGETS = new Map([
["src/file-store.ts", 580],
["src/permissions.ts", 566],
["src/pinned-python.ts", 655],
["src/root-impl.ts", 1750],
["src/root-path.ts", 862],
["test/api-coverage.test.ts", 983],
["test/new-primitives.test.ts", 1500],
]);
function walk(dir) {
const entries = fs.readdirSync(dir, { withFileTypes: true });
return entries.flatMap((entry) => {
const fullPath = path.join(dir, entry.name);
if (entry.isDirectory()) {
return walk(fullPath);
}
return fullPath.endsWith(".ts") ? [fullPath] : [];
});
}
const rootDir = process.cwd();
const files = [...walk("src"), ...walk("test")].sort();
const failures = [];
for (const file of files) {
const normalized = file.split(path.sep).join("/");
const text = fs.readFileSync(path.join(rootDir, file), "utf8");
const lines = text.length === 0 ? 0 : text.split("\n").length;
const budget = LINE_BUDGETS.get(normalized) ?? DEFAULT_MAX_LINES;
if (lines > budget) {
failures.push(`${normalized}: ${lines} lines > ${budget} budget`);
}
}
if (failures.length > 0) {
console.error("File size budget exceeded:");
for (const failure of failures) {
console.error(`- ${failure}`);
}
process.exit(1);
}

View File

@ -0,0 +1,97 @@
#!/usr/bin/env node
import fs from "node:fs";
import path from "node:path";
const root = process.cwd();
function read(relativePath) {
return fs.readFileSync(path.join(root, relativePath), "utf8");
}
const checks = [
{
file: "src/replace-file.ts",
forbidden: [
{
pattern: /\.copyFile(?:Sync)?\([^,\n]+,\s*(?:params\.)?dest\b/,
message: "replace-file fallback must not copy directly to the destination path",
},
],
},
{
file: "src/move-path.ts",
forbidden: [
{
pattern: /fs\.cp\(\s*options\.from,\s*options\.to\b/,
message: "cross-device move fallback must stage before replacing the destination",
},
{
pattern: /fs\.rm\(\s*options\.from\b/,
message: "move fallback source removal must go through guardedRm",
},
],
},
{
file: "src/private-temp-workspace.ts",
forbidden: [
{
pattern: /readFileSync\(\s*filePath\b/,
message: "tempWorkspaceSync.read must use a pinned root-file fd",
},
],
},
{
file: "src/sibling-temp.ts",
forbidden: [
{
pattern: /type DirectoryGuard\b/,
message: "sibling-temp must use the shared directory guard",
},
],
},
{
file: "src/archive-staging.ts",
forbidden: [
{
pattern: /type DirectoryIdentityGuard\b/,
message: "archive staging must use the shared directory guard",
},
],
},
];
const requiredImports = [
{
file: "src/json-durable-queue.ts",
pattern: /assertSafePathSegment/,
message: "durable queue ids must use the shared safe path segment helper",
},
{
file: "src/temp-target.ts",
pattern: /sanitizeSafePathSegment/,
message: "temp filenames must use the shared safe path segment sanitizer",
},
];
const failures = [];
for (const check of checks) {
const source = read(check.file);
for (const rule of check.forbidden) {
if (rule.pattern.test(source)) {
failures.push(`${check.file}: ${rule.message}`);
}
}
}
for (const check of requiredImports) {
const source = read(check.file);
if (!check.pattern.test(source)) {
failures.push(`${check.file}: ${check.message}`);
}
}
if (failures.length > 0) {
console.error(failures.join("\n"));
process.exit(1);
}

View File

@ -1,14 +1,21 @@
#!/usr/bin/env node
import { existsSync } from "node:fs";
import { spawnSync } from "node:child_process";
import { createRequire } from "node:module";
const tscBin =
process.platform === "win32" ? "node_modules/.bin/tsc.cmd" : "node_modules/.bin/tsc";
const require = createRequire(import.meta.url);
function resolveTypeScriptCompiler() {
try {
return require.resolve("typescript/bin/tsc");
} catch {
return undefined;
}
}
function run(command, args, env = {}) {
const result = spawnSync(command, args, {
stdio: "inherit",
shell: process.platform === "win32",
env: { ...process.env, ...env },
});
if (result.status !== 0) {
@ -16,7 +23,9 @@ function run(command, args, env = {}) {
}
}
if (!existsSync(tscBin)) {
let tscBin = resolveTypeScriptCompiler();
if (!tscBin || !existsSync(tscBin)) {
run(
"pnpm",
[
@ -32,6 +41,10 @@ if (!existsSync(tscBin)) {
PNPM_CONFIG_LOCKFILE_ONLY: "false",
},
);
tscBin = resolveTypeScriptCompiler();
if (!tscBin || !existsSync(tscBin)) {
throw new Error("TypeScript compiler is unavailable after installing dev dependencies");
}
}
run(tscBin, ["-p", "tsconfig.json"]);
run(process.execPath, [tscBin, "-p", "tsconfig.json"]);

View File

@ -1,6 +1,12 @@
import type { Stats } from "node:fs";
import fs from "node:fs/promises";
import path from "node:path";
import { FsSafeError } from "./errors.js";
import {
assertAsyncDirectoryGuard,
type AsyncDirectoryGuard,
createAsyncDirectoryGuard,
} from "./directory-guard.js";
import { FsSafeError, type FsSafeErrorCode } from "./errors.js";
export type AbsolutePathSymlinkPolicy = "reject" | "follow";
@ -14,6 +20,192 @@ export type ResolvedWritableAbsolutePath = ResolvedAbsolutePath & {
parentExists: boolean;
};
export type EnsureAbsoluteDirectoryOptions = {
scopeLabel?: string;
mode?: number;
};
export type EnsureAbsoluteDirectoryResult =
| { ok: true; path: string }
| { ok: false; code: FsSafeErrorCode; error: FsSafeError };
type EnsureAbsoluteDirectoryFailure = Extract<EnsureAbsoluteDirectoryResult, { ok: false }>;
type DirectoryGuardCheckResult = { ok: true } | EnsureAbsoluteDirectoryFailure;
type DirectoryGuardCreateResult =
| { ok: true; guard: AsyncDirectoryGuard }
| EnsureAbsoluteDirectoryFailure;
type DirectoryPrefixResult =
| {
ok: true;
ancestorPath: string;
missingSegments: string[];
}
| EnsureAbsoluteDirectoryFailure;
function ensureDirectoryFailure(
code: FsSafeErrorCode,
message: string,
cause?: unknown,
): EnsureAbsoluteDirectoryFailure {
return {
ok: false,
code,
error: new FsSafeError(code, message, { cause }),
};
}
async function assertGuardResult(
guard: AsyncDirectoryGuard,
scopeLabel: string,
): Promise<DirectoryGuardCheckResult> {
try {
await assertAsyncDirectoryGuard(guard);
return { ok: true };
} catch (err) {
if (err instanceof FsSafeError) {
return await directoryGuardFailure(err, guard.dir, scopeLabel);
}
throw err;
}
}
async function createDirectoryGuardResult(
dir: string,
scopeLabel: string,
): Promise<DirectoryGuardCreateResult> {
try {
return { ok: true, guard: await createAsyncDirectoryGuard(dir) };
} catch (err) {
if (err instanceof FsSafeError) {
return await directoryGuardFailure(err, dir, scopeLabel);
}
throw err;
}
}
function classifyDirectoryLookupError(
err: unknown,
scopeLabel: string,
): EnsureAbsoluteDirectoryFailure | null {
const code = (err as NodeJS.ErrnoException).code;
if (code === "ENOENT") {
return ensureDirectoryFailure(
"not-found",
`directory path must have a real existing ancestor within ${scopeLabel}`,
err,
);
}
if (code === "ENOTDIR") {
return ensureDirectoryFailure(
"not-file",
`path must be a real directory within ${scopeLabel}`,
err,
);
}
return null;
}
function classifyExistingDirectorySegment(
stat: Stats,
scopeLabel: string,
): EnsureAbsoluteDirectoryFailure | null {
if (stat.isSymbolicLink()) {
return ensureDirectoryFailure(
"symlink",
`directory path traverses a symlink within ${scopeLabel}`,
);
}
if (!stat.isDirectory()) {
return ensureDirectoryFailure("not-file", `path must be a real directory within ${scopeLabel}`);
}
return null;
}
async function directoryGuardFailure(
err: FsSafeError,
dir: string,
scopeLabel: string,
): Promise<EnsureAbsoluteDirectoryFailure> {
if (err.code !== "not-file") {
return { ok: false, code: err.code, error: err };
}
try {
const stat = await fs.lstat(dir);
const failure = classifyExistingDirectorySegment(stat, scopeLabel);
if (failure) {
return failure;
}
} catch (lookupErr) {
const failure = classifyDirectoryLookupError(lookupErr, scopeLabel);
if (failure) {
return failure;
}
throw lookupErr;
}
return { ok: false, code: err.code, error: err };
}
async function resolveTrustedDirectoryPrefix(
targetPath: string,
scopeLabel: string,
): Promise<DirectoryPrefixResult> {
const root = path.parse(targetPath).root;
let current = root;
let currentStat: Stats;
try {
currentStat = await fs.lstat(current);
} catch (err) {
const failure = classifyDirectoryLookupError(err, scopeLabel);
if (failure) {
return failure;
}
throw err;
}
const rootFailure = classifyExistingDirectorySegment(currentStat, scopeLabel);
if (rootFailure) {
return rootFailure;
}
// Walk forward with lstat. Looking backward for the "nearest existing
// ancestor" can cross an existing suffix through a symlinked parent before
// this helper gets a chance to reject that parent.
const segments = path.relative(root, targetPath).split(path.sep).filter(Boolean);
for (let index = 0; index < segments.length; index += 1) {
const segment = segments[index];
if (!segment) {
continue;
}
const next = path.join(current, segment);
try {
const nextStat = await fs.lstat(next);
const segmentFailure = classifyExistingDirectorySegment(nextStat, scopeLabel);
if (segmentFailure) {
return segmentFailure;
}
current = next;
currentStat = nextStat;
} catch (err) {
const code = (err as NodeJS.ErrnoException).code;
if (code === "ENOENT") {
return {
ok: true,
ancestorPath: current,
missingSegments: segments.slice(index),
};
}
const failure = classifyDirectoryLookupError(err, scopeLabel);
if (failure) {
return failure;
}
throw err;
}
}
return { ok: true, ancestorPath: current, missingSegments: [] };
}
export function assertAbsolutePathInput(filePath: string): string {
if (!filePath) {
throw new FsSafeError("invalid-path", "path is required");
@ -37,11 +229,17 @@ async function pathExists(filePath: string): Promise<boolean> {
}
export async function findExistingAncestor(filePath: string): Promise<string | null> {
return (await findExistingAncestorWithStat(filePath))?.path ?? null;
}
async function findExistingAncestorWithStat(filePath: string): Promise<{
path: string;
stat: Stats;
} | null> {
let current = path.resolve(filePath);
while (true) {
try {
await fs.lstat(current);
return current;
return { path: current, stat: await fs.lstat(current) };
} catch (err) {
if ((err as NodeJS.ErrnoException).code !== "ENOENT") {
throw err;
@ -55,6 +253,90 @@ export async function findExistingAncestor(filePath: string): Promise<string | n
}
}
export async function ensureAbsoluteDirectory(
dirPath: string,
options: EnsureAbsoluteDirectoryOptions = {},
): Promise<EnsureAbsoluteDirectoryResult> {
const scopeLabel = options.scopeLabel ?? "directory";
let targetPath: string;
try {
targetPath = assertAbsolutePathInput(dirPath);
} catch (err) {
if (err instanceof FsSafeError) {
return { ok: false, code: err.code, error: err };
}
throw err;
}
const prefix = await resolveTrustedDirectoryPrefix(targetPath, scopeLabel);
if (!prefix.ok) {
return prefix;
}
let current = prefix.ancestorPath;
const initialGuard = await createDirectoryGuardResult(prefix.ancestorPath, scopeLabel);
if (!initialGuard.ok) {
return initialGuard;
}
let currentGuard: AsyncDirectoryGuard = initialGuard.guard;
for (const segment of prefix.missingSegments) {
current = path.join(current, segment);
while (true) {
const guardResult = await assertGuardResult(currentGuard, scopeLabel);
if (!guardResult.ok) {
return guardResult;
}
try {
const stat = await fs.lstat(current);
if (stat.isSymbolicLink()) {
return ensureDirectoryFailure(
"symlink",
`directory path traverses a symlink within ${scopeLabel}`,
);
}
if (!stat.isDirectory()) {
return ensureDirectoryFailure(
"not-file",
`path must be a real directory within ${scopeLabel}`,
);
}
break;
} catch (err) {
if ((err as NodeJS.ErrnoException).code !== "ENOENT") {
throw err;
}
const parentStillValid = await assertGuardResult(currentGuard, scopeLabel);
if (!parentStillValid.ok) {
return parentStillValid;
}
try {
await fs.mkdir(current, { mode: options.mode });
} catch (mkdirErr) {
if ((mkdirErr as NodeJS.ErrnoException).code === "EEXIST") {
continue;
}
throw mkdirErr;
}
}
}
const nextGuard = await createDirectoryGuardResult(current, scopeLabel);
if (!nextGuard.ok) {
return nextGuard;
}
const previousGuardStillValid = await assertGuardResult(currentGuard, scopeLabel);
if (!previousGuardStillValid.ok) {
return previousGuardStillValid;
}
currentGuard = nextGuard.guard;
}
const finalGuardResult = await assertGuardResult(currentGuard, scopeLabel);
if (!finalGuardResult.ok) {
return finalGuardResult;
}
return { ok: true, path: targetPath };
}
export async function canonicalPathFromExistingAncestor(filePath: string): Promise<string> {
const ancestor = await findExistingAncestor(filePath);
if (!ancestor) {

View File

@ -5,10 +5,13 @@ export { createAsyncLock } from "./async-lock.js";
export {
assertAbsolutePathInput,
canonicalPathFromExistingAncestor,
ensureAbsoluteDirectory,
findExistingAncestor,
resolveAbsolutePathForRead,
resolveAbsolutePathForWrite,
type AbsolutePathSymlinkPolicy,
type EnsureAbsoluteDirectoryOptions,
type EnsureAbsoluteDirectoryResult,
type ResolvedAbsolutePath,
type ResolvedWritableAbsolutePath,
} from "./absolute-path.js";
@ -34,6 +37,11 @@ export {
trySafeFileURLToPath,
} from "./local-file-access.js";
export { formatPosixMode } from "./mode.js";
export {
configureFsSafeLocks,
getFsSafeLockConfig,
type FsSafeLockConfig,
} from "./lock-config.js";
export {
assertNoHardlinkedFinalPath,
assertNoPathAliasEscape,

View File

@ -1,9 +1,16 @@
import fs from "node:fs/promises";
import fsSync from "node:fs";
import path from "node:path";
import {
assertAsyncDirectoryGuard,
createAsyncDirectoryGuard,
type AsyncDirectoryGuard,
} from "./directory-guard.js";
import { FsSafeError } from "./errors.js";
import { root } from "./root.js";
import { resolveOpenedFileRealPathForHandle, root } from "./root.js";
import { isNotFoundPathError, isPathInside } from "./path.js";
import { resolveSecureTempRoot } from "./secure-temp-dir.js";
import { getFsSafeTestHooks } from "./test-hooks.js";
const ERROR_ARCHIVE_ENTRY_TRAVERSES_SYMLINK = "archive entry traverses symlink in destination";
const ARCHIVE_STAGING_MODE = 0o700;
@ -30,6 +37,31 @@ function symlinkTraversalError(originalPath: string): ArchiveSecurityError {
);
}
async function createDirectoryIdentityGuard(dir: string): Promise<AsyncDirectoryGuard> {
try {
return await createAsyncDirectoryGuard(dir);
} catch (err) {
if (err instanceof FsSafeError && err.code === "not-file") {
throw new ArchiveSecurityError("destination-symlink", "archive destination is a symlink");
}
throw err;
}
}
async function assertDirectoryIdentityGuard(guard: AsyncDirectoryGuard): Promise<void> {
try {
await assertAsyncDirectoryGuard(guard);
} catch (err) {
if (err instanceof FsSafeError) {
throw new ArchiveSecurityError(
"destination-symlink-traversal",
"archive destination changed during extraction",
);
}
throw err;
}
}
export async function prepareArchiveDestinationDir(destDir: string): Promise<string> {
const stat = await fs.lstat(destDir);
if (stat.isSymbolicLink()) {
@ -95,14 +127,20 @@ export async function prepareArchiveOutputPath(params: {
originalPath: string;
isDirectory: boolean;
}): Promise<void> {
const targetRoot = await root(params.destinationRealDir);
const destinationGuard = await createDirectoryIdentityGuard(params.destinationRealDir);
const relPath = params.relPath.split(path.sep).join(path.posix.sep);
await assertNoSymlinkTraversal({
rootDir: params.destinationDir,
relPath: params.relPath,
relPath,
originalPath: params.originalPath,
});
if (params.isDirectory) {
await fs.mkdir(params.outPath, { recursive: true });
await getFsSafeTestHooks()?.beforeArchiveOutputMutation?.("mkdir", params.outPath);
await assertDirectoryIdentityGuard(destinationGuard);
await targetRoot.mkdir(relPath);
await assertDirectoryIdentityGuard(destinationGuard);
await assertResolvedInsideDestination({
destinationRealDir: params.destinationRealDir,
targetPath: params.outPath,
@ -111,15 +149,59 @@ export async function prepareArchiveOutputPath(params: {
return;
}
const parentDir = path.dirname(params.outPath);
await fs.mkdir(parentDir, { recursive: true });
const parentRel = path.posix.dirname(relPath);
if (parentRel !== ".") {
await getFsSafeTestHooks()?.beforeArchiveOutputMutation?.("mkdir", path.dirname(params.outPath));
await assertDirectoryIdentityGuard(destinationGuard);
await targetRoot.mkdir(parentRel);
await assertDirectoryIdentityGuard(destinationGuard);
}
await assertResolvedInsideDestination({
destinationRealDir: params.destinationRealDir,
targetPath: parentDir,
targetPath: path.dirname(params.outPath),
originalPath: params.originalPath,
});
}
async function chmodInsideDestinationBestEffort(params: {
destinationRealDir: string;
destinationPath: string;
mode: number;
originalPath: string;
}): Promise<void> {
await getFsSafeTestHooks()?.beforeArchiveOutputMutation?.("chmod", params.destinationPath);
const destinationGuard = await createDirectoryIdentityGuard(params.destinationRealDir);
await assertDirectoryIdentityGuard(destinationGuard);
const noFollowFlag =
process.platform !== "win32" && "O_NOFOLLOW" in fsSync.constants
? fsSync.constants.O_NOFOLLOW
: 0;
const handle = await fs
.open(params.destinationPath, fsSync.constants.O_RDONLY | noFollowFlag)
.catch(() => null);
if (!handle) {
const stat = await fs.lstat(params.destinationPath).catch(() => null);
if (stat?.isSymbolicLink()) {
throw symlinkTraversalError(params.originalPath);
}
return;
}
try {
const stat = await handle.stat();
if (!stat.isDirectory() && !stat.isFile()) {
return;
}
const realPath = await resolveOpenedFileRealPathForHandle(handle, params.destinationPath);
if (!isPathInside(params.destinationRealDir, realPath)) {
throw symlinkTraversalError(params.originalPath);
}
await handle.chmod(params.mode).catch(() => undefined);
await assertDirectoryIdentityGuard(destinationGuard);
} finally {
await handle.close().catch(() => undefined);
}
}
async function applyStagedEntryMode(params: {
destinationRealDir: string;
relPath: string;
@ -133,7 +215,12 @@ async function applyStagedEntryMode(params: {
originalPath: params.originalPath,
});
if (params.mode !== 0) {
await fs.chmod(destinationPath, params.mode).catch(() => undefined);
await chmodInsideDestinationBestEffort({
destinationRealDir: params.destinationRealDir,
destinationPath,
mode: params.mode,
originalPath: params.originalPath,
});
}
}

View File

@ -1,4 +1,3 @@
import { randomUUID } from "node:crypto";
import { constants as fsConstants } from "node:fs";
import type { FileHandle } from "node:fs/promises";
import fs from "node:fs/promises";
@ -32,7 +31,7 @@ import {
type TarEntryInfo,
} from "./archive-tar.js";
import { loadZipArchiveWithPreflight } from "./archive-zip-preflight.js";
import { isNotFoundPathError } from "./path.js";
import { writeSiblingTempFile } from "./sibling-temp.js";
import { withTimeout } from "./timing.js";
export type ArchiveLogger = {
@ -80,28 +79,6 @@ const OPEN_WRITE_CREATE_FLAGS =
fsConstants.O_EXCL |
(SUPPORTS_NOFOLLOW ? fsConstants.O_NOFOLLOW : 0);
async function cleanupPartialRegularFile(filePath: string): Promise<void> {
let stat: Awaited<ReturnType<typeof fs.lstat>>;
try {
stat = await fs.lstat(filePath);
} catch (err) {
if (isNotFoundPathError(err)) {
return;
}
throw err;
}
if (stat.isFile()) {
await fs.unlink(filePath).catch(() => undefined);
}
}
function buildArchiveAtomicTempPath(targetPath: string): string {
return path.join(
path.dirname(targetPath),
`.${path.basename(targetPath)}.${process.pid}.${randomUUID()}.tmp`,
);
}
type ZipEntry = {
name: string;
dir: boolean;
@ -184,29 +161,34 @@ async function writeZipFileEntry(params: {
const destinationPath = params.outPath;
let tempHandle: FileHandle | null = null;
let tempPath: string | null = null;
let handleClosedByStream = false;
try {
tempPath = buildArchiveAtomicTempPath(destinationPath);
tempHandle = await fs.open(tempPath, OPEN_WRITE_CREATE_FLAGS, 0o666);
const writable = tempHandle.createWriteStream();
writable.once("close", () => {
handleClosedByStream = true;
});
await writeSiblingTempFile({
dir: path.dirname(destinationPath),
tempPrefix: `.${path.basename(destinationPath)}.fs-safe-archive`,
chmodDir: false,
writeTemp: async (tempPath) => {
tempHandle = await fs.open(tempPath, OPEN_WRITE_CREATE_FLAGS, 0o666);
const writable = tempHandle.createWriteStream();
writable.once("close", () => {
handleClosedByStream = true;
});
await pipeline(
readable,
createExtractBudgetTransform({ onChunkBytes: params.budget.addBytes }),
writable,
);
if (!handleClosedByStream) {
await tempHandle.close().catch(() => undefined);
handleClosedByStream = true;
}
tempHandle = null;
await fs.rename(tempPath, destinationPath);
tempPath = null;
await pipeline(
readable,
createExtractBudgetTransform({ onChunkBytes: params.budget.addBytes }),
writable,
);
if (!handleClosedByStream) {
await tempHandle.close().catch(() => undefined);
handleClosedByStream = true;
}
tempHandle = null;
return destinationPath;
},
resolveFinalPath: (filePath) => filePath,
});
// Best-effort permission restore for zip entries created on unix.
if (typeof params.entry.unixPermissions === "number") {
@ -216,15 +198,13 @@ async function writeZipFileEntry(params: {
}
}
} catch (err) {
if (tempPath) {
await fs.rm(tempPath, { force: true }).catch(() => undefined);
} else {
await cleanupPartialRegularFile(destinationPath).catch(() => undefined);
}
// Failures here happen before the temp has been committed. The destination
// parent may already be untrusted, so cleanup must stay limited to temp state.
throw err;
} finally {
if (tempHandle && !handleClosedByStream) {
await tempHandle.close().catch(() => undefined);
const openTempHandle = tempHandle as FileHandle | null;
if (openTempHandle && !handleClosedByStream) {
await openTempHandle.close().catch(() => undefined);
}
}
}

View File

@ -0,0 +1,30 @@
import { Transform } from "node:stream";
import { FsSafeError } from "./errors.js";
export function createMaxBytesTransform(maxBytes: number): Transform {
let bytes = 0;
return new Transform({
transform(chunk, _encoding, callback) {
const buffer = chunk instanceof Buffer ? chunk : Buffer.from(chunk as Uint8Array);
bytes += buffer.byteLength;
if (bytes > maxBytes) {
callback(
new FsSafeError(
"too-large",
`file exceeds limit of ${maxBytes} bytes (got at least ${bytes})`,
),
);
return;
}
callback(null, buffer);
},
});
}
export function createBoundedReadStream(
opened: { handle: { createReadStream(): NodeJS.ReadableStream } },
maxBytes: number | undefined,
): NodeJS.ReadableStream {
const stream = opened.handle.createReadStream();
return maxBytes === undefined ? stream : stream.pipe(createMaxBytesTransform(maxBytes));
}

View File

@ -4,3 +4,8 @@ export {
type FsSafePythonConfig,
type FsSafePythonMode,
} from "./pinned-python-config.js";
export {
configureFsSafeLocks,
getFsSafeLockConfig,
type FsSafeLockConfig,
} from "./lock-config.js";

93
src/directory-guard.ts Normal file
View File

@ -0,0 +1,93 @@
import type { Stats } from "node:fs";
import fsSync from "node:fs";
import fs from "node:fs/promises";
import path from "node:path";
import { FsSafeError } from "./errors.js";
import { sameFileIdentity } from "./file-identity.js";
import { isNotFoundPathError } from "./path.js";
export type AsyncDirectoryGuard = {
dir: string;
realPath: string;
stat: Stats;
};
export type SyncDirectoryGuard = {
dir: string;
realPath: string;
stat: Stats;
};
export async function createAsyncDirectoryGuard(dir: string): Promise<AsyncDirectoryGuard> {
const stat = await fs.lstat(dir);
if (stat.isSymbolicLink() || !stat.isDirectory()) {
throw new FsSafeError("not-file", "directory component must be a directory");
}
return { dir, realPath: await fs.realpath(dir), stat };
}
export async function assertAsyncDirectoryGuard(guard: AsyncDirectoryGuard): Promise<void> {
const stat = await fs.lstat(guard.dir);
if (stat.isSymbolicLink() || !stat.isDirectory()) {
throw new FsSafeError("not-file", "directory component must be a directory");
}
if (!sameFileIdentity(stat, guard.stat) || (await fs.realpath(guard.dir)) !== guard.realPath) {
throw new FsSafeError("path-mismatch", "directory changed during operation");
}
}
export function createSyncDirectoryGuard(dir: string): SyncDirectoryGuard {
const stat = fsSync.lstatSync(dir);
if (stat.isSymbolicLink() || !stat.isDirectory()) {
throw new FsSafeError("not-file", "directory component must be a directory");
}
return { dir, realPath: fsSync.realpathSync(dir), stat };
}
export function assertSyncDirectoryGuard(guard: SyncDirectoryGuard): void {
const stat = fsSync.lstatSync(guard.dir);
if (stat.isSymbolicLink() || !stat.isDirectory()) {
throw new FsSafeError("not-file", "directory component must be a directory");
}
if (!sameFileIdentity(stat, guard.stat) || fsSync.realpathSync(guard.dir) !== guard.realPath) {
throw new FsSafeError("path-mismatch", "directory changed during operation");
}
}
export async function createNearestExistingDirectoryGuard(
rootReal: string,
targetPath: string,
): Promise<AsyncDirectoryGuard> {
let current = path.resolve(targetPath);
const root = path.resolve(rootReal);
while (current !== root) {
try {
return await createAsyncDirectoryGuard(current);
} catch (error) {
if (!isNotFoundPathError(error)) {
throw error;
}
current = path.dirname(current);
}
}
return await createAsyncDirectoryGuard(root);
}
export function createNearestExistingSyncDirectoryGuard(
rootReal: string,
targetPath: string,
): SyncDirectoryGuard {
let current = path.resolve(targetPath);
const root = path.resolve(rootReal);
while (current !== root) {
try {
return createSyncDirectoryGuard(current);
} catch (error) {
if (!isNotFoundPathError(error)) {
throw error;
}
current = path.dirname(current);
}
}
return createSyncDirectoryGuard(root);
}

View File

@ -4,15 +4,19 @@ import {
type SidecarLockHandle,
type SidecarLockHeldEntry,
type SidecarLockRetryOptions,
type SidecarLockStaleRecovery,
} from "./sidecar-lock.js";
import { getFsSafeLockConfig } from "./lock-config.js";
export type FileLockRetryOptions = SidecarLockRetryOptions;
export type FileLockStaleRecovery = SidecarLockStaleRecovery;
export type FileLockAcquireOptions<TPayload extends Record<string, unknown>> = Omit<
SidecarLockAcquireOptions<TPayload>,
"targetPath"
"targetPath" | "staleMs"
> & {
managerKey?: string;
staleMs?: number;
};
export type FileLockHandle = SidecarLockHandle;
@ -37,6 +41,19 @@ function resolveFileLockManagerKey(targetPath: string, managerKey?: string): str
return managerKey ?? `fs-safe.file-lock:${targetPath}`;
}
function withLockDefaults<TPayload extends Record<string, unknown>>(
options: FileLockAcquireOptions<TPayload>,
): Omit<SidecarLockAcquireOptions<TPayload>, "targetPath"> {
const defaults = getFsSafeLockConfig();
return {
...options,
retry: options.retry ?? defaults.retry,
staleMs: options.staleMs ?? defaults.staleMs ?? 30_000,
staleRecovery: options.staleRecovery ?? defaults.staleRecovery,
timeoutMs: options.timeoutMs ?? defaults.timeoutMs,
};
}
export async function acquireFileLock<TPayload extends Record<string, unknown>>(
targetPath: string,
options: FileLockAcquireOptions<TPayload>,
@ -59,11 +76,11 @@ export function createFileLockManager(key: string): FileLockManager {
return {
acquire: async (targetPath, options) => {
const { managerKey: _managerKey, ...acquireOptions } = options;
return await manager.acquire({ ...acquireOptions, targetPath });
return await manager.acquire({ ...withLockDefaults(acquireOptions), targetPath });
},
withLock: async (targetPath, options, fn) => {
const { managerKey: _managerKey, ...acquireOptions } = options;
return await manager.withLock({ ...acquireOptions, targetPath }, fn);
return await manager.withLock({ ...withLockDefaults(acquireOptions), targetPath }, fn);
},
drain: manager.drain,
reset: manager.reset,

201
src/file-store-boundary.ts Normal file
View File

@ -0,0 +1,201 @@
import syncFs from "node:fs";
import fs from "node:fs/promises";
import path from "node:path";
import { Transform, type Readable } from "node:stream";
import { pipeline } from "node:stream/promises";
import {
assertSyncDirectoryGuard as assertDirectoryGuardSync,
createSyncDirectoryGuard,
type SyncDirectoryGuard,
} from "./directory-guard.js";
import { FsSafeError } from "./errors.js";
import { isPathInside } from "./path.js";
import { resolveOpenedFileRealPathForHandle, root, type Root } from "./root.js";
import { resolveSecureTempRoot } from "./secure-temp-dir.js";
export type SyncParentGuard = SyncDirectoryGuard;
function parentRelativePath(relativePath: string): string {
const parent = path.posix.dirname(relativePath);
return parent === "." ? "" : parent;
}
export async function ensureParentInRoot(
scopedRoot: Root,
relativePath: string,
mode: number,
): Promise<void> {
const parent = parentRelativePath(relativePath);
if (!parent) {
return;
}
await scopedRoot.mkdir(parent);
await chmodDirectoryInRootBestEffort(scopedRoot, parent, mode).catch(() => undefined);
}
export async function openWritableStoreRoot(params: {
rootDir: string;
dirMode: number;
maxBytes?: number;
}): Promise<Root> {
await fs.mkdir(params.rootDir, { recursive: true, mode: params.dirMode });
await fs.chmod(params.rootDir, params.dirMode).catch(() => undefined);
return await root(params.rootDir, { hardlinks: "reject", maxBytes: params.maxBytes });
}
async function chmodDirectoryInRootBestEffort(
scopedRoot: Root,
relativePath: string,
mode: number,
): Promise<void> {
const dirPath = await scopedRoot.resolve(relativePath);
const directoryFlag = "O_DIRECTORY" in syncFs.constants ? syncFs.constants.O_DIRECTORY : 0;
const noFollowFlag =
process.platform !== "win32" && "O_NOFOLLOW" in syncFs.constants
? syncFs.constants.O_NOFOLLOW
: 0;
const handle = await fs.open(dirPath, syncFs.constants.O_RDONLY | directoryFlag | noFollowFlag);
try {
const stat = await handle.stat();
if (!stat.isDirectory()) {
return;
}
const realPath = await resolveOpenedFileRealPathForHandle(handle, dirPath);
if (!isPathInside(scopedRoot.rootWithSep, realPath)) {
throw new FsSafeError("outside-workspace", "directory is outside store root");
}
await handle.chmod(mode).catch(() => undefined);
} finally {
await handle.close().catch(() => undefined);
}
}
function createMaxBytesTransform(maxBytes: number | undefined): Transform | undefined {
if (maxBytes === undefined) {
return undefined;
}
let total = 0;
return new Transform({
transform(chunk: Buffer | string, _encoding, callback) {
const buffer = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);
total += buffer.byteLength;
if (total > maxBytes) {
callback(new FsSafeError("too-large", `file exceeds maximum size of ${maxBytes} bytes`));
return;
}
callback(null, buffer);
},
});
}
export async function writeStreamToTempSource(params: {
stream: Readable;
maxBytes?: number;
mode: number;
}): Promise<{ path: string; cleanup: () => Promise<void> }> {
const tempRoot = resolveSecureTempRoot({
fallbackPrefix: "fs-safe-file-store",
unsafeFallbackLabel: "file store temp dir",
warn: () => undefined,
});
const dir = await fs.mkdtemp(path.join(tempRoot, "fs-safe-file-store-"));
const filePath = path.join(dir, "payload");
let handle: Awaited<ReturnType<typeof fs.open>> | null = null;
let handleClosedByStream = false;
try {
handle = await fs.open(filePath, "wx", params.mode);
const writable = handle.createWriteStream();
writable.once("close", () => {
handleClosedByStream = true;
});
const limiter = createMaxBytesTransform(params.maxBytes);
if (limiter) {
await pipeline(params.stream, limiter, writable);
} else {
await pipeline(params.stream, writable);
}
if (!handleClosedByStream) {
await handle.close().catch(() => undefined);
}
await fs.chmod(filePath, params.mode).catch(() => undefined);
return {
path: filePath,
cleanup: async () => {
await fs.rm(dir, { recursive: true, force: true }).catch(() => undefined);
},
};
} catch (err) {
if (handle && !handleClosedByStream) {
await handle.close().catch(() => undefined);
}
await fs.rm(dir, { recursive: true, force: true }).catch(() => undefined);
throw err;
}
}
export function assertSyncDirectoryGuard(guard: SyncParentGuard): void {
try {
assertDirectoryGuardSync(guard);
} catch (error) {
if (error instanceof FsSafeError && error.code === "path-mismatch") {
throw new FsSafeError("path-mismatch", "store directory changed during write", {
cause: error,
});
}
throw error;
}
}
function chmodDirectorySyncBestEffort(dir: string, mode: number): void {
try {
syncFs.chmodSync(dir, mode);
} catch {
// Best-effort on platforms that do not enforce POSIX modes.
}
}
export function ensureParentSync(params: {
rootDir: string;
filePath: string;
mode: number;
}): SyncParentGuard {
const rootDir = path.resolve(params.rootDir);
const dir = path.dirname(path.resolve(params.filePath));
const relative = path.relative(rootDir, dir);
if (relative.startsWith("..") || path.isAbsolute(relative)) {
throw new FsSafeError("outside-workspace", "file path escapes store root");
}
syncFs.mkdirSync(rootDir, { recursive: true, mode: params.mode });
const rootStat = syncFs.lstatSync(rootDir);
if (rootStat.isSymbolicLink() || !rootStat.isDirectory()) {
throw new FsSafeError("not-file", `store root must be a directory: ${rootDir}`);
}
const rootReal = syncFs.realpathSync(rootDir);
chmodDirectorySyncBestEffort(rootDir, params.mode);
let current = rootDir;
for (const segment of path.relative(rootDir, dir).split(path.sep).filter(Boolean)) {
current = path.join(current, segment);
try {
const stat = syncFs.lstatSync(current);
if (stat.isSymbolicLink() || !stat.isDirectory()) {
throw new FsSafeError("not-file", `store directory component must be a directory: ${current}`);
}
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") {
throw error;
}
syncFs.mkdirSync(current, { mode: params.mode });
}
const currentReal = syncFs.realpathSync(current);
if (!isPathInside(rootReal, currentReal)) {
throw new FsSafeError("outside-workspace", "store directory escapes root");
}
chmodDirectorySyncBestEffort(current, params.mode);
}
const guard = createSyncDirectoryGuard(dir);
assertSyncDirectoryGuard(guard);
return guard;
}

106
src/file-store-prune.ts Normal file
View File

@ -0,0 +1,106 @@
import type { Dirent } from "node:fs";
import fs from "node:fs/promises";
import path from "node:path";
import { FsSafeError } from "./errors.js";
import { isPathInside } from "./path.js";
import { root } from "./root.js";
import { getFsSafeTestHooks } from "./test-hooks.js";
export type FileStorePruneOptions = {
ttlMs: number;
recursive?: boolean;
maxDepth?: number;
pruneEmptyDirs?: boolean;
};
export async function pruneExpiredStoreEntries(params: {
rootDir: string;
dirMode: number;
options: FileStorePruneOptions;
}): Promise<void> {
const now = Date.now();
const recursive = params.options.recursive ?? false;
const maxDepth = params.options.maxDepth;
const pruneEmptyDirs =
(recursive || maxDepth !== undefined) && (params.options.pruneEmptyDirs ?? false);
await fs.mkdir(params.rootDir, { recursive: true, mode: params.dirMode });
const rootReal = await fs.realpath(params.rootDir);
const scopedRoot = await root(rootReal);
const rootGuard = {
dir: rootReal,
realPath: rootReal,
stat: await fs.lstat(rootReal),
};
async function assertRootGuard(): Promise<void> {
const stat = await fs.lstat(rootGuard.dir);
if (
stat.isSymbolicLink() ||
!stat.isDirectory() ||
stat.dev !== rootGuard.stat.dev ||
stat.ino !== rootGuard.stat.ino ||
(await fs.realpath(rootGuard.dir)) !== rootGuard.realPath
) {
throw new FsSafeError("path-mismatch", "store root changed during prune");
}
}
async function readStableDirectory(dir: string): Promise<Dirent[] | null> {
const before = await fs.lstat(dir).catch(() => null);
if (!before || before.isSymbolicLink() || !before.isDirectory()) {
return null;
}
const real = await fs.realpath(dir).catch(() => null);
if (!real || !isPathInside(rootReal, real)) {
return null;
}
const entries = await fs.readdir(dir, { withFileTypes: true }).catch(() => null);
if (!entries) {
return null;
}
const after = await fs.lstat(dir).catch(() => null);
if (!after || before.dev !== after.dev || before.ino !== after.ino) {
return null;
}
return entries;
}
async function pruneDir(dir: string, relativeDir: string, depth: number): Promise<boolean> {
const entries = await readStableDirectory(dir);
if (!entries) {
return false;
}
for (const entry of entries) {
const fullPath = path.join(dir, entry.name);
const relativePath = relativeDir ? `${relativeDir}/${entry.name}` : entry.name;
const stat = await fs.lstat(fullPath).catch(() => null);
if (!stat || stat.isSymbolicLink()) {
continue;
}
if (stat.isDirectory()) {
const shouldDescend = maxDepth !== undefined ? depth < maxDepth : recursive;
if (shouldDescend) {
await getFsSafeTestHooks()?.beforeFileStorePruneDescend?.(fullPath);
}
if (shouldDescend && (await pruneDir(fullPath, relativePath, depth + 1))) {
await assertRootGuard();
// Keep empty-dir pruning on the same root-bounded remove path as files;
// the Root fallback handles empty directories without recursive delete.
await scopedRoot.remove(relativePath).catch(() => undefined);
}
continue;
}
if (stat.isFile() && now - stat.mtimeMs > params.options.ttlMs) {
await assertRootGuard();
await scopedRoot.remove(relativePath).catch(() => undefined);
}
}
if (!pruneEmptyDirs) {
return false;
}
const remaining = await readStableDirectory(dir);
return remaining !== null && remaining.length === 0;
}
await pruneDir(rootReal, "", 0);
}

33
src/file-store-source.ts Normal file
View File

@ -0,0 +1,33 @@
import fs from "node:fs/promises";
import { FsSafeError } from "./errors.js";
import { readRegularFile } from "./regular-file.js";
export async function readFileStoreCopySource(params: {
sourcePath: string;
maxBytes?: number;
}): Promise<Buffer> {
const sourceStat = await fs.lstat(params.sourcePath);
if (sourceStat.isSymbolicLink() || !sourceStat.isFile()) {
throw new FsSafeError("not-file", "source path is not a file");
}
if (params.maxBytes !== undefined && sourceStat.size > params.maxBytes) {
throw new FsSafeError("too-large", `file exceeds maximum size of ${params.maxBytes} bytes`);
}
try {
return (await readRegularFile({ filePath: params.sourcePath, maxBytes: params.maxBytes }))
.buffer;
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
if (message.includes("regular file") || message.includes("not a regular file")) {
throw new FsSafeError("not-file", "source path is not a file", {
cause: error instanceof Error ? error : undefined,
});
}
if (params.maxBytes !== undefined && message.includes(`exceeds ${params.maxBytes} bytes`)) {
throw new FsSafeError("too-large", `file exceeds maximum size of ${params.maxBytes} bytes`, {
cause: error instanceof Error ? error : undefined,
});
}
throw error;
}
}

View File

@ -3,13 +3,29 @@ import syncFs from "node:fs";
import fs from "node:fs/promises";
import path from "node:path";
import type { Readable } from "node:stream";
import { pipeline } from "node:stream/promises";
import { createSyncDirectoryGuard } from "./directory-guard.js";
import { FsSafeError } from "./errors.js";
import {
pruneExpiredStoreEntries,
type FileStorePruneOptions,
} from "./file-store-prune.js";
export type { FileStorePruneOptions } from "./file-store-prune.js";
import {
assertSyncDirectoryGuard,
ensureParentInRoot,
ensureParentSync,
openWritableStoreRoot,
type SyncParentGuard,
writeStreamToTempSource,
} from "./file-store-boundary.js";
import { readFileStoreCopySource } from "./file-store-source.js";
import { createJsonStore, type JsonFileStoreOptions, type JsonStore } from "./json-document-store.js";
import { isPathInside, resolveSafeRelativePath } from "./path.js";
import { root, type OpenResult, type ReadResult, type Root, type RootReadOptions } from "./root.js";
import { DEFAULT_ROOT_MAX_BYTES } from "./root-impl.js";
import { matchRootFileOpenFailure, openRootFileSync, type RootFileOpenFailure } from "./root-file.js";
import { writeSecretFileAtomic } from "./secret-file.js";
import { writeSiblingTempFile } from "./sibling-temp.js";
import { getFsSafeTestHooks } from "./test-hooks.js";
export type FileStoreOptions = {
rootDir: string;
@ -28,13 +44,6 @@ export type FileStoreWriteOptions = {
export type FileStoreReadOptions = RootReadOptions & { encoding?: BufferEncoding };
export type FileStorePruneOptions = {
ttlMs: number;
recursive?: boolean;
maxDepth?: number;
pruneEmptyDirs?: boolean;
};
export type FileStore = {
readonly rootDir: string;
path(relativePath: string): string;
@ -114,13 +123,6 @@ function assertStoreFilePath(rootDir: string, filePath: string): void {
throw new FsSafeError("outside-workspace", "file path escapes store root");
}
}
async function ensureParent(filePath: string, mode: number): Promise<void> {
const dir = path.dirname(filePath);
await fs.mkdir(dir, { recursive: true, mode });
await fs.chmod(dir, mode).catch(() => undefined);
}
function assertMaxBytes(size: number, maxBytes?: number): void {
if (maxBytes !== undefined && size > maxBytes) {
throw new FsSafeError("too-large", `file exceeds maximum size of ${maxBytes} bytes`);
@ -128,12 +130,40 @@ function assertMaxBytes(size: number, maxBytes?: number): void {
}
function isNotFound(error: unknown): boolean {
if (!error) {
return false;
}
return error instanceof FsSafeError
? error.code === "not-found"
: (error as NodeJS.ErrnoException).code === "ENOENT" ||
(error as NodeJS.ErrnoException).code === "ENOTDIR";
}
function handleSyncStoreReadOpenFailure(opened: RootFileOpenFailure): null {
return matchRootFileOpenFailure<null>(opened, {
path: (failure) => {
if (isNotFound(failure.error)) {
return null;
}
throw new FsSafeError("path-mismatch", "store target changed during read", {
cause: failure.error instanceof Error ? failure.error : undefined,
});
},
validation: (failure) => {
// Validation failures mean the path existed but violated store policy
// (directory, hardlink, symlink race). Do not report them as missing.
throw new FsSafeError("path-mismatch", "store target failed read validation", {
cause: failure.error instanceof Error ? failure.error : undefined,
});
},
fallback: (failure) => {
throw new FsSafeError("path-mismatch", "store target changed during read", {
cause: failure.error instanceof Error ? failure.error : undefined,
});
},
});
}
async function copyIntoRoot(params: {
rootDir: string;
relativePath: string;
@ -143,26 +173,26 @@ async function copyIntoRoot(params: {
mode?: number;
tempPrefix?: string;
}): Promise<string> {
const destination = resolveStorePath(params.rootDir, params.relativePath);
const relativePath = assertRelativePath(params.relativePath);
const destination = resolveStorePath(params.rootDir, relativePath);
const sourceStat = await fs.lstat(params.sourcePath);
if (sourceStat.isSymbolicLink() || !sourceStat.isFile()) {
throw new FsSafeError("not-file", "source path is not a file");
}
assertMaxBytes(sourceStat.size, params.maxBytes);
await ensureParent(destination, params.dirMode ?? 0o700);
const result = await writeSiblingTempFile({
dir: path.dirname(destination),
dirMode: params.dirMode ?? 0o700,
mode: params.mode ?? 0o600,
tempPrefix: params.tempPrefix ?? `.${path.basename(destination)}`,
writeTemp: async (tempPath) => {
await fs.copyFile(params.sourcePath, tempPath);
},
resolveFinalPath: () => destination,
syncTempFile: true,
syncParentDir: true,
const dirMode = params.dirMode ?? 0o700;
const scopedRoot = await openWritableStoreRoot({
rootDir: params.rootDir,
dirMode,
maxBytes: params.maxBytes,
});
return result.filePath;
await ensureParentInRoot(scopedRoot, relativePath, dirMode);
await scopedRoot.copyIn(relativePath, params.sourcePath, {
maxBytes: params.maxBytes,
mkdir: false,
mode: params.mode ?? 0o600,
});
return destination;
}
export function fileStore(options: FileStoreOptions): FileStore {
@ -181,7 +211,8 @@ export function fileStore(options: FileStoreOptions): FileStore {
data: string | Uint8Array,
writeOptions?: FileStoreWriteOptions,
): Promise<string> {
const destination = resolveStorePath(rootDir, relativePath);
const safeRelativePath = assertRelativePath(relativePath);
const destination = resolveStorePath(rootDir, safeRelativePath);
const content = Buffer.isBuffer(data) ? data : Buffer.from(data);
assertMaxBytes(content.byteLength, writeOptions?.maxBytes ?? maxBytes);
if (privateMode) {
@ -194,20 +225,18 @@ export function fileStore(options: FileStoreOptions): FileStore {
});
return destination;
}
await ensureParent(destination, writeOptions?.dirMode ?? dirMode);
const result = await writeSiblingTempFile({
dir: path.dirname(destination),
dirMode: writeOptions?.dirMode ?? dirMode,
mode: writeOptions?.mode ?? mode,
tempPrefix: writeOptions?.tempPrefix ?? `.${path.basename(destination)}`,
writeTemp: async (tempPath) => {
await fs.writeFile(tempPath, content);
},
resolveFinalPath: () => destination,
syncTempFile: true,
syncParentDir: true,
const writeDirMode = writeOptions?.dirMode ?? dirMode;
const scopedRoot = await openWritableStoreRoot({
rootDir,
dirMode: writeDirMode,
maxBytes: writeOptions?.maxBytes ?? maxBytes,
});
return result.filePath;
await ensureParentInRoot(scopedRoot, safeRelativePath, writeDirMode);
await scopedRoot.write(safeRelativePath, content, {
mkdir: false,
mode: writeOptions?.mode ?? mode,
});
return destination;
}
return {
@ -216,8 +245,9 @@ export function fileStore(options: FileStoreOptions): FileStore {
root: openRoot,
write,
writeStream: async (relativePath, stream, writeOptions) => {
const destination = resolveStorePath(rootDir, relativePath);
const limit = writeOptions?.maxBytes ?? maxBytes;
const safeRelativePath = assertRelativePath(relativePath);
const destination = resolveStorePath(rootDir, safeRelativePath);
const limit = writeOptions?.maxBytes ?? maxBytes ?? (privateMode ? DEFAULT_ROOT_MAX_BYTES : undefined);
if (privateMode) {
const chunks: Buffer[] = [];
let total = 0;
@ -237,45 +267,34 @@ export function fileStore(options: FileStoreOptions): FileStore {
});
return destination;
}
await ensureParent(destination, writeOptions?.dirMode ?? dirMode);
let total = 0;
const result = await writeSiblingTempFile({
dir: path.dirname(destination),
dirMode: writeOptions?.dirMode ?? dirMode,
const staged = await writeStreamToTempSource({
stream,
maxBytes: limit,
mode: writeOptions?.mode ?? mode,
tempPrefix: writeOptions?.tempPrefix ?? `.${path.basename(destination)}`,
writeTemp: async (tempPath) => {
const writable = await fs.open(tempPath, "w", writeOptions?.mode ?? mode);
try {
const out = writable.createWriteStream();
stream.on("data", (chunk: Buffer | string) => {
total += Buffer.byteLength(chunk);
if (limit !== undefined && total > limit) {
stream.destroy(
new FsSafeError("too-large", `file exceeds maximum size of ${limit} bytes`),
);
}
});
await pipeline(stream, out);
} finally {
await writable.close().catch(() => undefined);
}
},
resolveFinalPath: () => destination,
syncTempFile: true,
syncParentDir: true,
});
return result.filePath;
try {
await copyIntoRoot({
rootDir,
relativePath: safeRelativePath,
sourcePath: staged.path,
maxBytes: limit,
mode: writeOptions?.mode ?? mode,
tempPrefix: writeOptions?.tempPrefix,
dirMode: writeOptions?.dirMode ?? dirMode,
});
} finally {
await staged.cleanup();
}
return destination;
},
copyIn: async (relativePath, sourcePath, writeOptions) =>
privateMode
? await (async () => {
const sourceStat = await fs.lstat(sourcePath);
if (sourceStat.isSymbolicLink() || !sourceStat.isFile()) {
throw new FsSafeError("not-file", "source path is not a file");
}
assertMaxBytes(sourceStat.size, writeOptions?.maxBytes ?? maxBytes);
return await write(relativePath, await fs.readFile(sourcePath), writeOptions);
const buffer = await readFileStoreCopySource({
sourcePath,
maxBytes: writeOptions?.maxBytes ?? maxBytes ?? DEFAULT_ROOT_MAX_BYTES,
});
return await write(relativePath, buffer, writeOptions);
})()
: await copyIntoRoot({
rootDir,
@ -369,53 +388,12 @@ export function fileStore(options: FileStoreOptions): FileStore {
);
},
pruneExpired: async (pruneOptions) => {
const now = Date.now();
const recursive = pruneOptions.recursive ?? false;
const maxDepth = pruneOptions.maxDepth;
const pruneEmptyDirs =
(recursive || maxDepth !== undefined) && (pruneOptions.pruneEmptyDirs ?? false);
async function pruneDir(dir: string, depth: number): Promise<boolean> {
const entries = await fs.readdir(dir, { withFileTypes: true }).catch(() => []);
for (const entry of entries) {
const fullPath = path.join(dir, entry.name);
const stat = await fs.lstat(fullPath).catch(() => null);
if (!stat || stat.isSymbolicLink()) {
continue;
}
if (stat.isDirectory()) {
const shouldDescend = maxDepth !== undefined ? depth < maxDepth : recursive;
if (shouldDescend && (await pruneDir(fullPath, depth + 1))) {
await fs.rmdir(fullPath).catch(() => undefined);
}
continue;
}
if (stat.isFile() && now - stat.mtimeMs > pruneOptions.ttlMs) {
await fs.rm(fullPath, { force: true }).catch(() => undefined);
}
}
if (!pruneEmptyDirs) {
return false;
}
const remaining = await fs.readdir(dir).catch(() => null);
return remaining !== null && remaining.length === 0;
}
await fs.mkdir(rootDir, { recursive: true, mode: dirMode });
await pruneDir(rootDir, 0);
await pruneExpiredStoreEntries({ rootDir, dirMode, options: pruneOptions });
},
};
}
function ensureParentSync(filePath: string, mode: number): void {
const dir = path.dirname(filePath);
syncFs.mkdirSync(dir, { recursive: true, mode });
try {
syncFs.chmodSync(dir, mode);
} catch {
// Best-effort on platforms that do not enforce POSIX modes.
}
}
function ensurePrivateDirectorySync(rootDir: string, targetDir: string, mode: number): void {
function ensurePrivateDirectorySync(rootDir: string, targetDir: string, mode: number): SyncParentGuard {
const root = path.resolve(rootDir);
const target = path.resolve(targetDir);
assertStoreFilePath(root, target);
@ -457,6 +435,9 @@ function ensurePrivateDirectorySync(rootDir: string, targetDir: string, mode: nu
// Best-effort on platforms that do not enforce POSIX modes.
}
}
const guard = createSyncDirectoryGuard(target);
assertSyncDirectoryGuard(guard);
return guard;
}
function writeFileSyncAtomic(params: {
@ -469,8 +450,9 @@ function writeFileSyncAtomic(params: {
}): string {
const filePath = path.resolve(params.filePath);
assertStoreFilePath(params.rootDir, filePath);
let parentGuard: SyncParentGuard | undefined;
if (params.privateMode) {
ensurePrivateDirectorySync(params.rootDir, path.dirname(filePath), params.dirMode);
parentGuard = ensurePrivateDirectorySync(params.rootDir, path.dirname(filePath), params.dirMode);
try {
const stat = syncFs.lstatSync(filePath);
if (stat.isSymbolicLink() || !stat.isFile()) {
@ -482,11 +464,22 @@ function writeFileSyncAtomic(params: {
}
}
} else {
ensureParentSync(filePath, params.dirMode);
parentGuard = ensureParentSync({
rootDir: params.rootDir,
filePath,
mode: params.dirMode,
});
}
const tempPath = path.join(path.dirname(filePath), `.fs-safe-${process.pid}-${randomUUID()}.tmp`);
const tempPath = path.join(
parentGuard?.dir ?? path.dirname(filePath),
`.fs-safe-${process.pid}-${randomUUID()}.tmp`,
);
let tempExists = false;
try {
getFsSafeTestHooks()?.beforeFileStoreSyncPrivateWrite?.(filePath);
if (parentGuard) {
assertSyncDirectoryGuard(parentGuard);
}
syncFs.writeFileSync(tempPath, params.content, { flag: "wx", mode: params.mode });
tempExists = true;
try {
@ -494,8 +487,14 @@ function writeFileSyncAtomic(params: {
} catch {
// Best-effort on platforms that do not enforce POSIX modes.
}
if (parentGuard) {
assertSyncDirectoryGuard(parentGuard);
}
syncFs.renameSync(tempPath, filePath);
tempExists = false;
if (parentGuard) {
assertSyncDirectoryGuard(parentGuard);
}
try {
syncFs.chmodSync(filePath, params.mode);
} catch {
@ -543,21 +542,22 @@ export function fileStoreSync(options: FileStoreOptions): FileStoreSync {
path: (relativePath) => resolveStorePath(rootDir, relativePath),
readTextIfExists: (relativePath, readOptions) => {
const targetPath = resolveStorePath(rootDir, relativePath);
const opened = openRootFileSync({
absolutePath: targetPath,
rootPath: rootDir,
boundaryLabel: "store root",
rejectHardlinks: privateMode,
});
if (!opened.ok) {
return handleSyncStoreReadOpenFailure(opened);
}
try {
const stat = syncFs.lstatSync(targetPath);
if (stat.isSymbolicLink() || !stat.isFile()) {
throw new FsSafeError("not-file", "store target is not a file");
}
assertMaxBytes(stat.size, readOptions?.maxBytes ?? maxBytes);
if (privateMode && stat.nlink > 1) {
throw new FsSafeError("hardlink", "private store target must not be hardlinked");
}
return syncFs.readFileSync(targetPath, "utf8");
} catch (error) {
if (isNotFound(error)) {
return null;
}
throw error;
assertMaxBytes(opened.stat.size, readOptions?.maxBytes ?? maxBytes);
const raw = syncFs.readFileSync(opened.fd, "utf8");
assertMaxBytes(Buffer.byteLength(raw, "utf8"), readOptions?.maxBytes ?? maxBytes);
return raw;
} finally {
syncFs.closeSync(opened.fd);
}
},
readJsonIfExists: <T = unknown>(relativePath: string, readOptions?: { maxBytes?: number }) => {

51
src/guarded-mkdir.ts Normal file
View File

@ -0,0 +1,51 @@
import fs from "node:fs/promises";
import path from "node:path";
import { assertAsyncDirectoryGuard, createAsyncDirectoryGuard } from "./directory-guard.js";
import { FsSafeError } from "./errors.js";
function isSameOrChildPath(candidate: string, parent: string): boolean {
return candidate === parent || candidate.startsWith(`${parent}${path.sep}`);
}
function isPathEscape(relativePath: string): boolean {
return relativePath === ".." || relativePath.startsWith(`..${path.sep}`) || path.isAbsolute(relativePath);
}
export async function mkdirPathComponentsWithGuards(params: {
rootReal: string;
targetPath: string;
beforeComponent?: (componentPath: string) => Promise<void> | void;
}): Promise<void> {
const root = path.resolve(params.rootReal);
const target = path.resolve(params.targetPath);
const relative = path.relative(root, target);
if (isPathEscape(relative)) {
throw new FsSafeError("outside-workspace", "directory is outside workspace root");
}
let current = root;
for (const part of relative.split(path.sep).filter(Boolean)) {
const next = path.join(current, part);
const parentGuard = await createAsyncDirectoryGuard(current);
await params.beforeComponent?.(next);
await assertAsyncDirectoryGuard(parentGuard);
try {
await fs.mkdir(next);
} catch (error) {
if (!error || typeof error !== "object" || !("code" in error) || error.code !== "EEXIST") {
throw error;
}
}
const stat = await fs.lstat(next);
if (stat.isSymbolicLink() || !stat.isDirectory()) {
throw new FsSafeError("not-file", "directory component must be a directory");
}
// Node's recursive mkdir follows symlinks in missing components. Build one
// segment at a time and realpath-check each segment before descending.
if (!isSameOrChildPath(path.resolve(await fs.realpath(next)), root)) {
throw new FsSafeError("outside-workspace", "directory escaped workspace root");
}
await createAsyncDirectoryGuard(next);
await assertAsyncDirectoryGuard(parentGuard);
current = next;
}
}

137
src/guarded-mutation.ts Normal file
View File

@ -0,0 +1,137 @@
import fsSync from "node:fs";
import fs from "node:fs/promises";
import path from "node:path";
import {
assertAsyncDirectoryGuard,
assertSyncDirectoryGuard,
createAsyncDirectoryGuard,
createNearestExistingDirectoryGuard,
createNearestExistingSyncDirectoryGuard,
createSyncDirectoryGuard,
type AsyncDirectoryGuard,
type SyncDirectoryGuard,
} from "./directory-guard.js";
export async function withAsyncDirectoryGuards<T>(
guards: readonly AsyncDirectoryGuard[],
mutate: () => Promise<T>,
options: {
verifyAfter?: boolean;
onPostGuardFailure?: (result: T, error: unknown) => Promise<void> | void;
} = {},
): Promise<T> {
for (const guard of guards) {
await assertAsyncDirectoryGuard(guard);
}
const result = await mutate();
if (options.verifyAfter !== false) {
try {
for (const guard of guards) {
await assertAsyncDirectoryGuard(guard);
}
} catch (error) {
if (options.onPostGuardFailure) {
try {
// The mutation may have returned an owned resource before the post-guard
// check detected a swapped directory. Give callers one chance to close
// handles without letting cleanup hide the boundary failure.
await options.onPostGuardFailure(result, error);
} catch {
// Preserve the boundary failure. Cleanup is best-effort.
}
}
throw error;
}
}
return result;
}
export function withSyncDirectoryGuards<T>(
guards: readonly SyncDirectoryGuard[],
mutate: () => T,
options: { verifyAfter?: boolean } = {},
): T {
for (const guard of guards) {
assertSyncDirectoryGuard(guard);
}
const result = mutate();
if (options.verifyAfter !== false) {
for (const guard of guards) {
assertSyncDirectoryGuard(guard);
}
}
return result;
}
export async function guardedRename(params: {
from: string;
to: string;
targetRoot?: string;
verifyAfter?: boolean;
}): Promise<void> {
const sourceGuard = await createAsyncDirectoryGuard(path.dirname(params.from));
const targetGuard = params.targetRoot
? await createNearestExistingDirectoryGuard(params.targetRoot, path.dirname(params.to))
: await createAsyncDirectoryGuard(path.dirname(params.to));
await withAsyncDirectoryGuards(
[sourceGuard, targetGuard],
async () => {
await fs.rename(params.from, params.to);
},
{ verifyAfter: params.verifyAfter },
);
}
export function guardedRenameSync(params: {
from: string;
to: string;
targetRoot?: string;
verifyAfter?: boolean;
}): void {
const sourceGuard = createSyncDirectoryGuard(path.dirname(params.from));
const targetGuard = params.targetRoot
? createNearestExistingSyncDirectoryGuard(params.targetRoot, path.dirname(params.to))
: createSyncDirectoryGuard(path.dirname(params.to));
withSyncDirectoryGuards(
[sourceGuard, targetGuard],
() => fsSync.renameSync(params.from, params.to),
{ verifyAfter: params.verifyAfter },
);
}
export async function guardedRm(params: {
target: string;
recursive?: boolean;
force?: boolean;
verifyAfter?: boolean;
}): Promise<void> {
const guard = await createAsyncDirectoryGuard(path.dirname(params.target));
await withAsyncDirectoryGuards(
[guard],
async () => {
await fs.rm(params.target, {
...(params.recursive !== undefined ? { recursive: params.recursive } : {}),
...(params.force !== undefined ? { force: params.force } : {}),
});
},
{ verifyAfter: params.verifyAfter },
);
}
export function guardedRmSync(params: {
target: string;
recursive?: boolean;
force?: boolean;
verifyAfter?: boolean;
}): void {
const guard = createSyncDirectoryGuard(path.dirname(params.target));
withSyncDirectoryGuards(
[guard],
() =>
fsSync.rmSync(params.target, {
...(params.recursive !== undefined ? { recursive: params.recursive } : {}),
...(params.force !== undefined ? { force: params.force } : {}),
}),
{ verifyAfter: params.verifyAfter },
);
}

View File

@ -31,18 +31,21 @@ export function resolveOsHomeDir(
function resolveRawHomeDir(env: NodeJS.ProcessEnv, homedir: () => string): string | undefined {
const explicitHome = normalize(env.OPENCLAW_HOME);
if (explicitHome) {
if (explicitHome === "~" || explicitHome.startsWith("~/") || explicitHome.startsWith("~\\")) {
const fallbackHome = resolveRawOsHomeDir(env, homedir);
if (fallbackHome) {
return explicitHome.replace(/^~(?=$|[\\/])/, fallbackHome);
}
return undefined;
}
if (!explicitHome) {
return resolveRawOsHomeDir(env, homedir);
}
const segments = path.normalize(explicitHome).split(path.sep);
if (segments[0] !== "~") {
return explicitHome;
}
return resolveRawOsHomeDir(env, homedir);
// OPENCLAW_HOME starts with "~"; expand against the os home dir. Fall
// back to undefined when there is no os home to expand against rather
// than returning a raw "~"-prefixed path the caller cannot use.
const fallbackHome = resolveRawOsHomeDir(env, homedir);
if (!fallbackHome) {
return undefined;
}
return expandHomePrefix(explicitHome, { home: fallbackHome });
}
function resolveRawOsHomeDir(env: NodeJS.ProcessEnv, homedir: () => string): string | undefined {
@ -87,7 +90,8 @@ export function expandHomePrefix(
homedir?: () => string;
},
): string {
if (!input.startsWith("~")) {
const segments = path.normalize(input).split(path.sep);
if (segments[0] !== "~") {
return input;
}
const home =
@ -96,7 +100,7 @@ export function expandHomePrefix(
if (!home) {
return input;
}
return input.replace(/^~(?=$|[\\/])/, home);
return path.join(home, ...segments.slice(1));
}
export function resolveHomeRelativePath(
@ -106,19 +110,19 @@ export function resolveHomeRelativePath(
homedir?: () => string;
},
): string {
const trimmed = input.trim();
if (!trimmed) {
return trimmed;
if (!input) {
return input;
}
if (trimmed.startsWith("~")) {
const expanded = expandHomePrefix(trimmed, {
home: resolveRequiredHomeDir(opts?.env ?? process.env, opts?.homedir ?? os.homedir),
env: opts?.env,
homedir: opts?.homedir,
});
return path.resolve(expanded);
const segments = path.normalize(input).split(path.sep)
if (segments[0] !== "~") {
return path.resolve(input);
}
return path.resolve(trimmed);
const expanded = expandHomePrefix(input, {
home: resolveRequiredHomeDir(opts?.env ?? process.env, opts?.homedir ?? os.homedir),
env: opts?.env,
homedir: opts?.homedir,
});
return path.resolve(expanded);
}
export function resolveUserPath(
@ -145,17 +149,17 @@ export function resolveOsHomeRelativePath(
homedir?: () => string;
},
): string {
const trimmed = input.trim();
if (!trimmed) {
return trimmed;
if (!input) {
return input;
}
if (trimmed.startsWith("~")) {
const expanded = expandHomePrefix(trimmed, {
home: resolveRequiredOsHomeDir(opts?.env ?? process.env, opts?.homedir ?? os.homedir),
env: opts?.env,
homedir: opts?.homedir,
});
return path.resolve(expanded);
const segments = path.normalize(input).split(path.sep);
if (segments[0] !== "~") {
return path.resolve(input);
}
return path.resolve(trimmed);
const expanded = expandHomePrefix(input, {
home: resolveRequiredOsHomeDir(opts?.env ?? process.env, opts?.homedir ?? os.homedir),
env: opts?.env,
homedir: opts?.homedir,
});
return path.resolve(expanded);
}

View File

@ -32,3 +32,13 @@ export {
type FsSafePythonConfig,
type FsSafePythonMode,
} from "./pinned-python-config.js";
export {
writeExternalFileWithinRoot,
type ExternalFileWriteOptions,
type ExternalFileWriteResult,
} from "./output.js";
export {
configureFsSafeLocks,
getFsSafeLockConfig,
type FsSafeLockConfig,
} from "./lock-config.js";

View File

@ -1,10 +1,13 @@
import type { FileLockRetryOptions } from "./file-lock.js";
import { getFsSafeLockConfig } from "./lock-config.js";
import { createSidecarLockManager } from "./sidecar-lock.js";
import type { SidecarLockStaleRecovery } from "./sidecar-lock.js";
export type JsonStoreLockOptions = {
staleMs?: number;
timeoutMs?: number;
retry?: FileLockRetryOptions;
staleRecovery?: SidecarLockStaleRecovery;
managerKey?: string;
};
@ -45,11 +48,13 @@ function resolveLockOptions(
return null;
}
const lockOptions = options.lock === true ? {} : options.lock;
const defaults = getFsSafeLockConfig();
return {
managerKey: lockOptions.managerKey ?? `fs-safe.json-store:${filePath}`,
retry: lockOptions.retry ?? {},
staleMs: lockOptions.staleMs ?? 30_000,
timeoutMs: lockOptions.timeoutMs ?? 30_000,
retry: lockOptions.retry ?? defaults.retry ?? {},
staleMs: lockOptions.staleMs ?? defaults.staleMs ?? 30_000,
staleRecovery: lockOptions.staleRecovery ?? defaults.staleRecovery,
timeoutMs: lockOptions.timeoutMs ?? defaults.timeoutMs ?? 30_000,
};
}
@ -84,6 +89,7 @@ export function createJsonStore<T>(
staleMs: lockOptions.staleMs,
timeoutMs: lockOptions.timeoutMs,
retry: lockOptions.retry,
staleRecovery: lockOptions.staleRecovery,
allowReentrant: true,
payload: () => ({ pid: process.pid, createdAt: new Date().toISOString() }),
},

270
src/json-durable-queue.ts Normal file
View File

@ -0,0 +1,270 @@
import fs from "node:fs";
import path from "node:path";
import { sameFileIdentity } from "./file-identity.js";
import { replaceFileAtomic } from "./replace-file.js";
import { assertSafePathSegment } from "./safe-path-segment.js";
export type JsonDurableQueueEntryPaths = {
jsonPath: string;
deliveredPath: string;
};
export type JsonDurableQueueReadResult<T> = {
entry: T;
migrated?: boolean;
};
export type JsonDurableQueueLoadOptions<T> = {
queueDir: string;
tempPrefix: string;
read?: (entry: T, filePath: string) => Promise<JsonDurableQueueReadResult<T>>;
cleanupTmpMaxAgeMs?: number;
maxBytes?: number;
};
export const DEFAULT_JSON_DURABLE_QUEUE_ENTRY_MAX_BYTES = 16 * 1024 * 1024;
function getErrnoCode(error: unknown): string | null {
return error && typeof error === "object" && "code" in error
? String((error as { code?: unknown }).code)
: null;
}
function assertSafeQueueEntryId(id: string): void {
assertSafePathSegment(id, { label: "queue entry id" });
}
export async function unlinkBestEffort(filePath: string): Promise<void> {
await fs.promises.unlink(filePath).catch(() => undefined);
}
export async function jsonDurableQueueEntryExists(filePath: string): Promise<boolean> {
try {
const stat = await fs.promises.lstat(filePath);
return stat.isFile();
} catch (error) {
if (getErrnoCode(error) === "ENOENT") {
return false;
}
throw error;
}
}
async function unlinkStaleTmpBestEffort(
filePath: string,
now: number,
maxAgeMs: number,
): Promise<void> {
try {
const stat = await fs.promises.stat(filePath);
if (stat.isFile() && now - stat.mtimeMs >= maxAgeMs) {
await unlinkBestEffort(filePath);
}
} catch (error) {
if (getErrnoCode(error) !== "ENOENT") {
throw error;
}
}
}
export function resolveJsonDurableQueueEntryPaths(
queueDir: string,
id: string,
): JsonDurableQueueEntryPaths {
assertSafeQueueEntryId(id);
return {
jsonPath: path.join(queueDir, `${id}.json`),
deliveredPath: path.join(queueDir, `${id}.delivered`),
};
}
export async function ensureJsonDurableQueueDirs(params: {
queueDir: string;
failedDir: string;
}): Promise<void> {
await fs.promises.mkdir(params.queueDir, { recursive: true, mode: 0o700 });
await fs.promises.mkdir(params.failedDir, { recursive: true, mode: 0o700 });
}
export async function writeJsonDurableQueueEntry(params: {
filePath: string;
entry: unknown;
tempPrefix: string;
}): Promise<void> {
await replaceFileAtomic({
filePath: params.filePath,
content: JSON.stringify(params.entry, null, 2),
mode: 0o600,
tempPrefix: params.tempPrefix,
});
}
async function readBoundedUtf8File(params: {
filePath: string;
maxBytes: number;
}): Promise<string> {
const initialStat = await fs.promises.lstat(params.filePath);
if (initialStat.isSymbolicLink() || !initialStat.isFile()) {
throw new Error("queue entry is not a regular file");
}
if (initialStat.size > params.maxBytes) {
throw new Error(`queue entry exceeds ${params.maxBytes} bytes`);
}
const noFollow =
typeof fs.constants.O_NOFOLLOW === "number" && process.platform !== "win32"
? fs.constants.O_NOFOLLOW
: 0;
const handle = await fs.promises.open(params.filePath, fs.constants.O_RDONLY | noFollow);
try {
const openedStat = await handle.stat();
const pathStat = await fs.promises.lstat(params.filePath);
if (
!openedStat.isFile() ||
pathStat.isSymbolicLink() ||
!pathStat.isFile() ||
!sameFileIdentity(initialStat, openedStat) ||
!sameFileIdentity(pathStat, openedStat)
) {
throw new Error("queue entry changed during read");
}
const chunks: Buffer[] = [];
const scratch = Buffer.allocUnsafe(Math.min(64 * 1024, params.maxBytes + 1));
let total = 0;
while (true) {
const { bytesRead } = await handle.read(scratch, 0, scratch.length, null);
if (bytesRead === 0) {
return Buffer.concat(chunks, total).toString("utf8");
}
total += bytesRead;
if (total > params.maxBytes) {
throw new Error(`queue entry exceeds ${params.maxBytes} bytes`);
}
chunks.push(Buffer.from(scratch.subarray(0, bytesRead)));
}
} finally {
await handle.close();
}
}
export async function readJsonDurableQueueEntry<T>(
filePath: string,
options: { maxBytes?: number } = {},
): Promise<T> {
return JSON.parse(
await readBoundedUtf8File({
filePath,
maxBytes: options.maxBytes ?? DEFAULT_JSON_DURABLE_QUEUE_ENTRY_MAX_BYTES,
}),
) as T;
}
export async function ackJsonDurableQueueEntry(paths: JsonDurableQueueEntryPaths): Promise<void> {
try {
await fs.promises.rename(paths.jsonPath, paths.deliveredPath);
} catch (error) {
if (getErrnoCode(error) === "ENOENT") {
await unlinkBestEffort(paths.deliveredPath);
return;
}
throw error;
}
await unlinkBestEffort(paths.deliveredPath);
}
export async function loadJsonDurableQueueEntry<T>(params: {
paths: JsonDurableQueueEntryPaths;
tempPrefix: string;
read?: (entry: T, filePath: string) => Promise<JsonDurableQueueReadResult<T>>;
maxBytes?: number;
}): Promise<T | null> {
try {
const stat = await fs.promises.lstat(params.paths.jsonPath);
if (!stat.isFile()) {
return null;
}
const raw = await readJsonDurableQueueEntry<T>(params.paths.jsonPath, {
maxBytes: params.maxBytes,
});
const result = params.read ? await params.read(raw, params.paths.jsonPath) : { entry: raw };
if (result.migrated) {
await writeJsonDurableQueueEntry({
filePath: params.paths.jsonPath,
entry: result.entry,
tempPrefix: params.tempPrefix,
});
}
return result.entry;
} catch (error) {
if (getErrnoCode(error) === "ENOENT") {
return null;
}
throw error;
}
}
export async function loadPendingJsonDurableQueueEntries<T>(
options: JsonDurableQueueLoadOptions<T>,
): Promise<T[]> {
let files: string[];
try {
files = await fs.promises.readdir(options.queueDir);
} catch (error) {
if (getErrnoCode(error) === "ENOENT") {
return [];
}
throw error;
}
const now = Date.now();
for (const file of files) {
if (file.endsWith(".delivered")) {
await unlinkBestEffort(path.join(options.queueDir, file));
} else if (options.cleanupTmpMaxAgeMs !== undefined && file.endsWith(".tmp")) {
await unlinkStaleTmpBestEffort(
path.join(options.queueDir, file),
now,
options.cleanupTmpMaxAgeMs,
);
}
}
const entries: T[] = [];
for (const file of files) {
if (!file.endsWith(".json")) {
continue;
}
const filePath = path.join(options.queueDir, file);
try {
const stat = await fs.promises.lstat(filePath);
if (!stat.isFile()) {
continue;
}
const raw = await readJsonDurableQueueEntry<T>(filePath, { maxBytes: options.maxBytes });
const result = options.read ? await options.read(raw, filePath) : { entry: raw };
if (result.migrated) {
await writeJsonDurableQueueEntry({
filePath,
entry: result.entry,
tempPrefix: options.tempPrefix,
});
}
entries.push(result.entry);
} catch {
continue;
}
}
return entries;
}
export async function moveJsonDurableQueueEntryToFailed(params: {
queueDir: string;
failedDir: string;
id: string;
}): Promise<void> {
assertSafeQueueEntryId(params.id);
await fs.promises.mkdir(params.failedDir, { recursive: true, mode: 0o700 });
await fs.promises.rename(
path.join(params.queueDir, `${params.id}.json`),
path.join(params.failedDir, `${params.id}.json`),
);
}

View File

@ -2,7 +2,8 @@ import { randomUUID } from "node:crypto";
import fsSync from "node:fs";
import path from "node:path";
import { readRegularFile, readRegularFileSync } from "./regular-file.js";
import { writeTextAtomic } from "./text-atomic.js";
import { openRootFileSync, type RootFileOpenFailure } from "./root-file.js";
import { writeTextAtomic, type WriteTextAtomicOptions } from "./text-atomic.js";
const JSON_FILE_MODE = 0o600;
const JSON_DIR_MODE = 0o700;
@ -74,8 +75,8 @@ function renameJsonFileWithFallback(tmpPath: string, pathname: string) {
fsSync.renameSync(tmpPath, pathname);
return;
}
fsSync.copyFileSync(tmpPath, pathname);
fsSync.rmSync(tmpPath, { force: true });
fsSync.rmSync(pathname, { force: true });
fsSync.renameSync(tmpPath, pathname);
return;
}
throw error;
@ -134,6 +135,106 @@ export class JsonFileReadError extends Error {
}
}
export type RootStructuredFileReadResult<T> =
| { ok: true; value: T; stat: fsSync.Stats; path: string; rootRealPath: string }
| { ok: false; reason: "open"; failure: RootFileOpenFailure }
| { ok: false; reason: "invalid" | "parse"; error: string };
export type ReadRootStructuredFileSyncOptions<T> = {
rootDir: string;
rootRealPath?: string;
relativePath: string;
boundaryLabel: string;
rejectHardlinks?: boolean;
maxBytes?: number;
parse: (raw: string) => unknown;
validate?: (value: unknown) => value is T;
invalidMessage?: string | ((relativePath: string) => string);
};
export type ReadRootJsonSyncOptions = Omit<
ReadRootStructuredFileSyncOptions<unknown>,
"parse" | "validate" | "invalidMessage"
>;
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
function resolveInvalidMessage(
invalidMessage: ReadRootStructuredFileSyncOptions<unknown>["invalidMessage"],
relativePath: string,
): string {
if (typeof invalidMessage === "function") {
return invalidMessage(relativePath);
}
return invalidMessage ?? `${relativePath} has an unexpected shape`;
}
export function readRootStructuredFileSync<T>(
options: ReadRootStructuredFileSyncOptions<T>,
): RootStructuredFileReadResult<T> {
const absolutePath = path.resolve(options.rootDir, options.relativePath);
const opened = openRootFileSync({
absolutePath,
rootPath: options.rootDir,
...(options.rootRealPath !== undefined ? { rootRealPath: options.rootRealPath } : {}),
boundaryLabel: options.boundaryLabel,
rejectHardlinks: options.rejectHardlinks,
maxBytes: options.maxBytes,
allowedType: "file",
});
if (!opened.ok) {
return { ok: false, reason: "open", failure: opened };
}
try {
const parsed = options.parse(fsSync.readFileSync(opened.fd, "utf8"));
if (options.validate && !options.validate(parsed)) {
return {
ok: false,
reason: "invalid",
error: resolveInvalidMessage(options.invalidMessage, options.relativePath),
};
}
return {
ok: true,
value: parsed as T,
stat: opened.stat,
path: opened.path,
rootRealPath: opened.rootRealPath,
};
} catch (error) {
return {
ok: false,
reason: "parse",
error: `failed to parse ${options.relativePath}: ${String(error)}`,
};
} finally {
fsSync.closeSync(opened.fd);
}
}
export function readRootJsonSync<T = unknown>(
options: ReadRootJsonSyncOptions,
): RootStructuredFileReadResult<T> {
return readRootStructuredFileSync<T>({
...options,
parse: (raw) => JSON.parse(raw),
});
}
export function readRootJsonObjectSync(
options: ReadRootJsonSyncOptions,
): RootStructuredFileReadResult<Record<string, unknown>> {
return readRootStructuredFileSync<Record<string, unknown>>({
...options,
parse: (raw) => JSON.parse(raw),
validate: isRecord,
invalidMessage: (relativePath) => `${relativePath} must contain a JSON object`,
});
}
export async function tryReadJson<T>(filePath: string): Promise<T | null> {
try {
const raw = (await readRegularFile({ filePath })).buffer.toString("utf8");
@ -188,15 +289,21 @@ export function readJsonSync<T = unknown>(filePath: string): T {
}
}
export type WriteJsonOptions = Pick<
WriteTextAtomicOptions,
"dirMode" | "durable" | "mode" | "trailingNewline"
>;
export async function writeJson(
filePath: string,
value: unknown,
options?: { mode?: number; trailingNewline?: boolean; dirMode?: number },
options?: WriteJsonOptions,
) {
const text = JSON.stringify(value, null, 2);
await writeTextAtomic(filePath, text, {
mode: options?.mode,
dirMode: options?.dirMode,
trailingNewline: options?.trailingNewline,
durable: options?.durable,
});
}

View File

@ -84,8 +84,10 @@ function resolveRootRealSync(rootDir: string): string | null {
function resolveCandidateCanonicalSync(
filePath: string,
): { exists: true; canonicalPath: string; isFile: boolean } | { exists: false; canonicalPath: string } {
let sawExistingLeaf = false;
try {
const stat = fsSync.lstatSync(filePath);
sawExistingLeaf = true;
return {
exists: true,
canonicalPath: fsSync.realpathSync(filePath),
@ -96,6 +98,11 @@ function resolveCandidateCanonicalSync(
throw err;
}
}
if (sawExistingLeaf) {
// lstat succeeded but realpath failed: this is an existing dangling
// symlink, not a missing path callers may safely create through.
throw new FsSafeError("symlink", "local roots candidate is a dangling symlink");
}
let cursor = filePath;
const missingSegments: string[] = [];
@ -115,6 +122,8 @@ function resolveCandidateCanonicalSync(
};
} catch (err) {
if ((err as NodeJS.ErrnoException).code !== "ENOENT") {
// Existing ancestors that cannot be canonicalized are symlink/error
// terrain; do not reconstruct a trusted missing path through them.
throw err;
}
}
@ -175,12 +184,17 @@ export async function readLocalFileFromRoots(
}
try {
const result = await scopedRoot.read(relativePath, {
const readOptions: Parameters<typeof scopedRoot.read>[1] = {
hardlinks: options.hardlinks,
maxBytes: options.maxBytes,
nonBlockingRead: options.nonBlockingRead,
symlinks: options.symlinks,
});
};
// Leave maxBytes absent when the caller omits it so Root's own default
// cap remains in force instead of being overwritten by undefined.
if (options.maxBytes !== undefined) {
readOptions.maxBytes = options.maxBytes;
}
const result = await scopedRoot.read(relativePath, readOptions);
return { ...result, root: scopedRoot.rootReal };
} catch {
continue;

25
src/lock-config.ts Normal file
View File

@ -0,0 +1,25 @@
import type { SidecarLockRetryOptions, SidecarLockStaleRecovery } from "./sidecar-lock.js";
export type FsSafeLockConfig = {
staleRecovery: SidecarLockStaleRecovery;
staleMs?: number;
timeoutMs?: number;
retry?: SidecarLockRetryOptions;
};
const DEFAULT_LOCK_CONFIG: FsSafeLockConfig = {
staleRecovery: "fail-closed",
};
let lockConfig: FsSafeLockConfig = { ...DEFAULT_LOCK_CONFIG };
export function configureFsSafeLocks(config: Partial<FsSafeLockConfig>): void {
// Process defaults only fill lock options after a caller explicitly enables
// locking for a resource; this must never turn sidecar locks on globally.
lockConfig = { ...lockConfig, ...config };
}
export function getFsSafeLockConfig(): FsSafeLockConfig {
return { ...lockConfig, retry: lockConfig.retry ? { ...lockConfig.retry } : undefined };
}

View File

@ -1,25 +1,282 @@
import { randomUUID } from "node:crypto";
import { constants as fsConstants } from "node:fs";
import type { FileHandle } from "node:fs/promises";
import fs from "node:fs/promises";
import path from "node:path";
import { guardedRename } from "./guarded-mutation.js";
export type MovePathWithCopyFallbackOptions = {
from: string;
sourceHardlinks?: "allow" | "reject";
to: string;
};
type EntryIdentity = {
ctimeMs: number;
dev: number;
ino: number;
mode: number;
mtimeMs: number;
size: number;
};
type CopiedEntryManifest =
| (EntryIdentity & {
children: Array<{ name: string; manifest: CopiedEntryManifest }>;
kind: "directory";
})
| (EntryIdentity & { kind: "leaf" });
type CleanupCopiedEntryResult = "removed" | "stale";
function entryIdentity(stat: {
ctimeMs: number;
dev: number;
ino: number;
mode: number;
mtimeMs: number;
size: number;
}): EntryIdentity {
return {
ctimeMs: stat.ctimeMs,
dev: stat.dev,
ino: stat.ino,
mode: stat.mode,
mtimeMs: stat.mtimeMs,
size: stat.size,
};
}
function sameIdentity(a: EntryIdentity, b: EntryIdentity): boolean {
return (
a.dev === b.dev &&
a.ino === b.ino &&
a.mode === b.mode &&
a.size === b.size &&
a.mtimeMs === b.mtimeMs &&
a.ctimeMs === b.ctimeMs
);
}
function sameDirectoryNode(a: EntryIdentity, b: EntryIdentity): boolean {
return a.dev === b.dev && a.ino === b.ino;
}
function modeBits(mode: number): number {
return mode & 0o777;
}
function sourceChangedError(sourcePath: string): Error {
return Object.assign(new Error(`Source changed during move fallback: ${sourcePath}`), {
code: "ESTALE",
});
}
async function assertSourceStillMatches(
sourcePath: string,
identity: EntryIdentity,
): Promise<void> {
if (!sameIdentity(identity, entryIdentity(await fs.lstat(sourcePath)))) {
throw sourceChangedError(sourcePath);
}
}
function regularReadFlags(): number {
return (
fsConstants.O_RDONLY |
(typeof fsConstants.O_NOFOLLOW === "number" && process.platform !== "win32"
? fsConstants.O_NOFOLLOW
: 0)
);
}
async function writeAll(handle: FileHandle, buffer: Buffer, bytesRead: number): Promise<void> {
let offset = 0;
while (offset < bytesRead) {
const { bytesWritten } = await handle.write(buffer, offset, bytesRead - offset);
offset += bytesWritten;
}
}
async function copyRegularFilePinned(params: {
from: string;
identity: EntryIdentity;
mode: number;
to: string;
}): Promise<void> {
let destinationCreated = false;
let sourceHandle: FileHandle;
try {
sourceHandle = await fs.open(params.from, regularReadFlags());
} catch (error) {
const code = (error as NodeJS.ErrnoException | null)?.code;
if (code === "ELOOP" || code === "ENOENT" || code === "ENOTDIR") {
throw sourceChangedError(params.from);
}
throw error;
}
try {
const openedStat = await sourceHandle.stat();
if (!openedStat.isFile() || !sameIdentity(params.identity, entryIdentity(openedStat))) {
throw sourceChangedError(params.from);
}
const destinationHandle = await fs.open(
params.to,
fsConstants.O_WRONLY | fsConstants.O_CREAT | fsConstants.O_EXCL,
modeBits(params.mode) || 0o666,
);
destinationCreated = true;
try {
const scratch = Buffer.allocUnsafe(64 * 1024);
while (true) {
const { bytesRead } = await sourceHandle.read(scratch, 0, scratch.length, null);
if (bytesRead === 0) {
break;
}
await writeAll(destinationHandle, scratch, bytesRead);
}
} finally {
await destinationHandle.close();
}
// Re-check the opened handle before the staged tree can be committed. If
// the source changed while we copied, the caller should retry the move.
if (!sameIdentity(params.identity, entryIdentity(await sourceHandle.stat()))) {
throw sourceChangedError(params.from);
}
await fs.chmod(params.to, modeBits(params.mode)).catch(() => undefined);
} catch (error) {
if (destinationCreated) {
await fs.rm(params.to, { force: true }).catch(() => undefined);
}
throw error;
} finally {
await sourceHandle.close();
}
}
async function copyEntryWithManifest(
from: string,
to: string,
options: { sourceHardlinks: "allow" | "reject" },
): Promise<CopiedEntryManifest> {
const sourceStat = await fs.lstat(from);
const identity = entryIdentity(sourceStat);
if (sourceStat.isSymbolicLink()) {
await fs.symlink(await fs.readlink(from), to);
// readlink() is path-based; verify the symlink we copied is still the one
// we inspected before letting the staged destination become visible.
await assertSourceStillMatches(from, identity);
return { ...identity, kind: "leaf" };
}
if (sourceStat.isDirectory()) {
await fs.mkdir(to, { mode: modeBits(sourceStat.mode) || 0o755 });
const children: Array<{ name: string; manifest: CopiedEntryManifest }> = [];
for (const child of await fs.readdir(from)) {
children.push({
name: child,
manifest: await copyEntryWithManifest(path.join(from, child), path.join(to, child), options),
});
}
// Directory traversal is path-based in Node. Treat a changed parent as a
// stale move before committing so swapped-in outside trees are not imported.
await assertSourceStillMatches(from, identity);
// mkdir() honors process umask. Restore the source mode before commit so
// EXDEV fallback preserves directory permissions like fs.cp did.
await fs.chmod(to, modeBits(sourceStat.mode));
return { ...identity, children, kind: "directory" };
}
if (!sourceStat.isFile()) {
throw new Error(`Refusing to move non-file path with copy fallback: ${from}`);
}
if (options.sourceHardlinks === "reject" && sourceStat.nlink > 1) {
throw new Error(`Refusing to move hardlinked file with copy fallback: ${from}`);
}
await copyRegularFilePinned({ from, identity, mode: sourceStat.mode, to });
return { ...identity, kind: "leaf" };
}
function mergeCleanupResults(
a: CleanupCopiedEntryResult,
b: CleanupCopiedEntryResult,
): CleanupCopiedEntryResult {
return a === "stale" || b === "stale" ? "stale" : "removed";
}
async function cleanupCopiedEntry(
sourcePath: string,
manifest: CopiedEntryManifest,
): Promise<CleanupCopiedEntryResult> {
let currentStat: Awaited<ReturnType<typeof fs.lstat>>;
try {
currentStat = await fs.lstat(sourcePath);
} catch (error) {
if ((error as NodeJS.ErrnoException | null)?.code === "ENOENT") {
return "removed";
}
throw error;
}
if (manifest.kind === "directory") {
if (!currentStat.isDirectory() || !sameDirectoryNode(manifest, entryIdentity(currentStat))) {
return "stale";
}
// A same-inode directory can gain unrelated children after commit. Still
// clean manifest children so the fallback does not duplicate copied files.
let result: CleanupCopiedEntryResult = "removed";
for (const child of manifest.children) {
result = mergeCleanupResults(
result,
await cleanupCopiedEntry(path.join(sourcePath, child.name), child.manifest),
);
}
try {
await fs.rmdir(sourcePath);
} catch (error) {
const code = (error as NodeJS.ErrnoException | null)?.code;
if (code === "ENOTEMPTY" || code === "EEXIST") {
return "stale";
}
throw error;
}
return result;
}
if (!sameIdentity(manifest, entryIdentity(currentStat))) {
return "stale";
}
await fs.unlink(sourcePath);
return "removed";
}
export async function movePathWithCopyFallback(
options: MovePathWithCopyFallbackOptions,
): Promise<void> {
try {
await fs.rename(options.from, options.to);
await guardedRename({ from: options.from, to: options.to });
return;
} catch (error) {
if ((error as NodeJS.ErrnoException | null)?.code !== "EXDEV") {
throw error;
}
}
await fs.cp(options.from, options.to, {
recursive: true,
force: true,
dereference: false,
});
await fs.rm(options.from, { recursive: true, force: true });
const targetDir = path.dirname(path.resolve(options.to));
const staged = path.join(targetDir, `.fs-safe-move-${process.pid}-${randomUUID()}.tmp`);
try {
const manifest = await copyEntryWithManifest(options.from, staged, {
sourceHardlinks: options.sourceHardlinks ?? "allow",
});
await guardedRename({ from: staged, to: options.to });
const cleanupResult = await cleanupCopiedEntry(options.from, manifest);
if (cleanupResult === "stale") {
throw sourceChangedError(options.from);
}
} finally {
await fs.rm(staged, { recursive: true, force: true }).catch(() => undefined);
}
}

96
src/output.ts Normal file
View File

@ -0,0 +1,96 @@
import path from "node:path";
import { FsSafeError } from "./errors.js";
import { sanitizeUntrustedFileName } from "./filename.js";
import { isPathInside } from "./path.js";
import { root } from "./root.js";
import { tempFile } from "./temp-target.js";
export type ExternalFileWriteOptions<T = void> = {
rootDir: string;
path: string;
write: (filePath: string) => Promise<T>;
maxBytes?: number;
mode?: number;
};
export type ExternalFileWriteResult<T = void> = {
path: string;
result: T;
};
function tempFileNameForTarget(targetPath: string): string {
return sanitizeUntrustedFileName(path.basename(targetPath), "output.bin");
}
function ensureTrailingSep(value: string): string {
return value.endsWith(path.sep) ? value : `${value}${path.sep}`;
}
function toRootPathInput(params: {
rootDir: string;
rootReal: string;
targetPath: string;
}): string {
if (!path.isAbsolute(params.targetPath)) {
return params.targetPath;
}
const absoluteTarget = path.resolve(params.targetPath);
const rootDir = path.resolve(params.rootDir);
if (isPathInside(ensureTrailingSep(rootDir), absoluteTarget)) {
return path.relative(rootDir, absoluteTarget);
}
if (isPathInside(ensureTrailingSep(params.rootReal), absoluteTarget)) {
return path.relative(params.rootReal, absoluteTarget);
}
return params.targetPath;
}
function assertFileTargetPath(targetPath: string): void {
const basename = path.basename(targetPath);
if (
!targetPath ||
targetPath === "." ||
targetPath.endsWith("/") ||
targetPath.endsWith("\\") ||
!basename ||
basename === "." ||
basename === ".."
) {
throw new FsSafeError("invalid-path", "target path must name a file");
}
}
export async function writeExternalFileWithinRoot<T = void>(
options: ExternalFileWriteOptions<T>,
): Promise<ExternalFileWriteResult<T>> {
const targetRoot = await root(options.rootDir);
const requestedTargetPath = options.path;
if (requestedTargetPath.length === 0) {
throw new FsSafeError("invalid-path", "target path is required");
}
const targetPath = toRootPathInput({
rootDir: targetRoot.rootDir,
rootReal: targetRoot.rootReal,
targetPath: requestedTargetPath,
});
assertFileTargetPath(targetPath);
const finalPath = await targetRoot.resolve(targetPath);
const staged = await tempFile({
prefix: "fs-safe-output",
fileName: tempFileNameForTarget(targetPath),
});
try {
const result = await options.write(staged.path);
await targetRoot.copyIn(targetPath, staged.path, {
maxBytes: options.maxBytes,
mode: options.mode,
mkdir: true,
sourceHardlinks: "reject",
});
return { path: finalPath, result };
} finally {
await staged.cleanup();
}
}

18
src/path-stat.ts Normal file
View File

@ -0,0 +1,18 @@
import type { Stats } from "node:fs";
import type { PathStat } from "./types.js";
export function pathStatFromStats(stat: Stats): PathStat {
return {
dev: Number(stat.dev),
gid: Number(stat.gid),
ino: Number(stat.ino),
isDirectory: stat.isDirectory(),
isFile: stat.isFile(),
isSymbolicLink: stat.isSymbolicLink(),
mode: stat.mode,
mtimeMs: stat.mtimeMs,
nlink: stat.nlink,
size: stat.size,
uid: stat.uid,
};
}

View File

@ -6,7 +6,6 @@ import { normalizeLowercaseStringOrEmpty } from "./string-coerce.js";
const NOT_FOUND_CODES = new Set(["ENOENT", "ENOTDIR"]);
const SYMLINK_OPEN_CODES = new Set(["ELOOP", "EINVAL", "ENOTSUP"]);
const PARENT_SEGMENT_PREFIX = /^\.\.(?:[\\/]|$)/u;
const POSIX_SEPARATOR_CHAR_CODE = 0x2f;
export function normalizeWindowsPathForComparison(input: string): string {
@ -30,6 +29,12 @@ export function hasNodeErrorCode(value: unknown, code: string): boolean {
return isNodeError(value) && value.code === code;
}
export function assertNoNulPathInput(filePath: string, message = "path contains a NUL byte"): void {
if (filePath.includes("\0")) {
throw new FsSafeError("invalid-path", message);
}
}
export function isNotFoundPathError(value: unknown): boolean {
return isNodeError(value) && typeof value.code === "string" && NOT_FOUND_CODES.has(value.code);
}
@ -43,8 +48,9 @@ export function isPathInside(root: string, target: string): boolean {
const rootForCompare = normalizeWindowsPathForComparison(path.win32.resolve(root));
const targetForCompare = normalizeWindowsPathForComparison(path.win32.resolve(target));
const relative = path.win32.relative(rootForCompare, targetForCompare);
const firstSegment = relative.split(path.win32.sep)[0];
return (
relative === "" || (!PARENT_SEGMENT_PREFIX.test(relative) && !path.win32.isAbsolute(relative))
relative === "" || (firstSegment !== ".." && !path.win32.isAbsolute(relative))
);
}
@ -63,7 +69,8 @@ export function isPathInside(root: string, target: string): boolean {
const resolvedRoot = path.resolve(root);
const resolvedTarget = path.resolve(target);
const relative = path.relative(resolvedRoot, resolvedTarget);
return relative === "" || (!PARENT_SEGMENT_PREFIX.test(relative) && !path.isAbsolute(relative));
const firstSegment = relative.split(path.posix.sep)[0];
return relative === "" || (firstSegment !== ".." && !path.isAbsolute(relative));
}
export function resolveSafeBaseDir(rootDir: string): string {
@ -118,9 +125,7 @@ export function splitSafeRelativePath(relativePath: string): string[] {
if (relativePath.length === 0 || relativePath === ".") {
return [];
}
if (relativePath.includes("\0")) {
throw new FsSafeError("invalid-path", "relative path contains a NUL byte");
}
assertNoNulPathInput(relativePath, "relative path contains a NUL byte");
if (relativePath.includes("\\")) {
throw new FsSafeError("invalid-path", "relative path must use forward slashes");
}

View File

@ -1,10 +1,13 @@
import { randomUUID } from "node:crypto";
import fsSync from "node:fs";
import fs from "node:fs/promises";
import path from "node:path";
import { Transform, type Readable } from "node:stream";
import { pipeline } from "node:stream/promises";
import { createNearestExistingDirectoryGuard } from "./directory-guard.js";
import { FsSafeError } from "./errors.js";
import type { FileIdentityStat } from "./file-identity.js";
import { withAsyncDirectoryGuards } from "./guarded-mutation.js";
import { canFallbackFromPythonError, getFsSafePythonConfig } from "./pinned-python-config.js";
import {
assertPinnedPythonOperationAvailable,
@ -111,13 +114,13 @@ export async function runPinnedWriteHelper(params: {
maxBytes?: number;
input: PinnedWriteInput;
}): Promise<FileIdentityStat> {
if (getFsSafePythonConfig().mode === "off") {
return await runPinnedWriteFallback(params);
}
assertSafeBasename(params.basename);
validatePinnedOperationPayload({
relativeParentPath: params.relativeParentPath,
});
if (getFsSafePythonConfig().mode === "off") {
return await runPinnedWriteFallback(params);
}
if (params.input.kind === "stream") {
try {
assertPinnedPythonOperationAvailable();
@ -196,15 +199,29 @@ async function runPinnedWriteFallback(params: {
const parentPath = params.relativeParentPath
? path.join(params.rootPath, ...params.relativeParentPath.split("/"))
: params.rootPath;
const parentGuard = await createNearestExistingDirectoryGuard(params.rootPath, parentPath);
if (params.mkdir) {
await fs.mkdir(parentPath, { recursive: true });
await withAsyncDirectoryGuards([parentGuard], async () => {
await fs.mkdir(parentPath, { recursive: true });
});
}
const targetPath = path.join(parentPath, params.basename);
if (params.overwrite === false) {
const handle = await fs.open(
targetPath,
fsSync.constants.O_WRONLY | fsSync.constants.O_CREAT | fsSync.constants.O_EXCL,
params.mode,
let handle = await withAsyncDirectoryGuards(
[parentGuard],
async () =>
await fs.open(
targetPath,
fsSync.constants.O_WRONLY | fsSync.constants.O_CREAT | fsSync.constants.O_EXCL,
params.mode,
),
{
onPostGuardFailure: async (openedHandle) => {
// The parent failed verification, so targetPath may now resolve
// somewhere else. Close the fd, but do not clean up by path.
await openedHandle.close().catch(() => undefined);
},
},
);
let created = true;
try {
@ -236,35 +253,46 @@ async function runPinnedWriteFallback(params: {
}
}
const tempPath = path.join(parentPath, `.${params.basename}.fallback.tmp`);
const tempPath = path.join(parentPath, `.${params.basename}.${randomUUID()}.fallback.tmp`);
const tempFlags =
fsSync.constants.O_WRONLY |
fsSync.constants.O_CREAT |
fsSync.constants.O_EXCL |
(process.platform !== "win32" && "O_NOFOLLOW" in fsSync.constants
? fsSync.constants.O_NOFOLLOW
: 0);
let handle: Awaited<ReturnType<typeof fs.open>> | undefined;
let handleClosedByStream = false;
try {
handle = await fs.open(tempPath, tempFlags, params.mode);
if (params.input.kind === "buffer") {
assertWithinMaxBytes(
byteLength(params.input.data, params.input.encoding),
params.maxBytes,
);
if (typeof params.input.data === "string") {
await fs.writeFile(tempPath, params.input.data, {
encoding: params.input.encoding ?? "utf8",
mode: params.mode,
});
await handle.writeFile(params.input.data, params.input.encoding ?? "utf8");
} else {
await fs.writeFile(tempPath, params.input.data, { mode: params.mode });
await handle.writeFile(params.input.data);
}
} else {
const handle = await fs.open(tempPath, "w", params.mode);
try {
await pipelineWithMaxBytes(
params.input.stream,
handle.createWriteStream(),
params.maxBytes,
);
} finally {
await handle.close().catch(() => {});
}
const writable = handle.createWriteStream();
writable.once("close", () => {
handleClosedByStream = true;
});
await pipelineWithMaxBytes(params.input.stream, writable, params.maxBytes);
}
await fs.rename(tempPath, targetPath);
if (!handleClosedByStream) {
await handle.close().catch(() => undefined);
handle = undefined;
}
await withAsyncDirectoryGuards([parentGuard], async () => {
await fs.rename(tempPath, targetPath);
});
} catch (error) {
if (handle && !handleClosedByStream) {
await handle.close().catch(() => undefined);
}
await fs.rm(tempPath, { force: true }).catch(() => undefined);
throw error;
}

View File

@ -8,6 +8,7 @@ import {
type FileStore,
type FileStoreSync,
} from "./file-store.js";
import { openRootFileSync } from "./root-file.js";
import { registerTempPathForExit } from "./temp-cleanup.js";
export type TempWorkspaceOptions = {
@ -55,24 +56,23 @@ function sanitizeTempPrefix(prefix: string): string {
}
function resolveWorkspaceLeaf(dir: string, fileName: string): string {
const raw = fileName.trim();
if (
!raw ||
raw === "." ||
raw === ".." ||
raw.includes("\0") ||
raw.includes("/") ||
raw.includes("\\") ||
path.basename(raw) !== raw
) {
throw new Error(`Invalid temp workspace file name: ${JSON.stringify(fileName)}`);
}
return path.join(dir, raw);
return path.join(dir, assertWorkspaceFileName(fileName));
}
function assertWorkspaceFileName(fileName: string): string {
resolveWorkspaceLeaf(".", fileName);
return fileName.trim();
const value = fileName.trim();
if (
!value ||
value === "." ||
value === ".." ||
value.includes("\0") ||
value.includes("/") ||
value.includes("\\") ||
path.basename(value) !== value
) {
throw new Error(`Invalid temp workspace file name: ${JSON.stringify(fileName)}`);
}
return value;
}
async function ensurePrivateDirectory(dir: string, mode: number): Promise<void> {
@ -208,8 +208,20 @@ export function tempWorkspaceSync(
trailingNewline: writeOptions?.trailingNewline,
}),
read: (fileName) => {
const filePath = store.path(assertWorkspaceFileName(fileName));
return fsSync.readFileSync(filePath);
const opened = openRootFileSync({
absolutePath: store.path(assertWorkspaceFileName(fileName)),
rootPath: dir,
boundaryLabel: "temp workspace",
rejectHardlinks: true,
});
if (!opened.ok) {
throw Object.assign(new Error(`File not found: ${fileName}`), { code: "ENOENT" });
}
try {
return fsSync.readFileSync(opened.fd);
} finally {
fsSync.closeSync(opened.fd);
}
},
cleanup: () => {
try {

View File

@ -1,5 +1,6 @@
import type { Stats } from "node:fs";
import fsSync from "node:fs";
import type { FileHandle } from "node:fs/promises";
import fs from "node:fs/promises";
import path from "node:path";
import { sameFileIdentity } from "./file-identity.js";
@ -44,6 +45,54 @@ function resolveRegularFileReadFlags(): number {
);
}
async function readFileHandleBounded(params: {
handle: FileHandle;
filePath: string;
maxBytes?: number;
}): Promise<Buffer> {
if (params.maxBytes === undefined) {
return await params.handle.readFile();
}
const chunks: Buffer[] = [];
const scratch = Buffer.allocUnsafe(Math.min(64 * 1024, Math.max(1, params.maxBytes + 1)));
let total = 0;
while (true) {
const { bytesRead } = await params.handle.read(scratch, 0, scratch.length, null);
if (bytesRead === 0) {
return Buffer.concat(chunks, total);
}
total += bytesRead;
if (total > params.maxBytes) {
throw new Error(`File exceeds ${params.maxBytes} bytes: ${params.filePath}`);
}
chunks.push(Buffer.from(scratch.subarray(0, bytesRead)));
}
}
function readFileDescriptorBounded(params: {
fd: number;
filePath: string;
maxBytes?: number;
}): Buffer {
if (params.maxBytes === undefined) {
return fsSync.readFileSync(params.fd);
}
const chunks: Buffer[] = [];
const scratch = Buffer.allocUnsafe(Math.min(64 * 1024, Math.max(1, params.maxBytes + 1)));
let total = 0;
while (true) {
const bytesRead = fsSync.readSync(params.fd, scratch, 0, scratch.length, null);
if (bytesRead === 0) {
return Buffer.concat(chunks, total);
}
total += bytesRead;
if (total > params.maxBytes) {
throw new Error(`File exceeds ${params.maxBytes} bytes: ${params.filePath}`);
}
chunks.push(Buffer.from(scratch.subarray(0, bytesRead)));
}
}
export async function statRegularFile(filePath: string): Promise<RegularFileStatResult> {
let stat: Stats;
try {
@ -100,10 +149,13 @@ export async function readRegularFile(params: {
if (params.maxBytes !== undefined && stat.size > params.maxBytes) {
throw new Error(`File exceeds ${params.maxBytes} bytes: ${params.filePath}`);
}
const buffer = await handle.readFile();
if (params.maxBytes !== undefined && buffer.byteLength > params.maxBytes) {
throw new Error(`File exceeds ${params.maxBytes} bytes: ${params.filePath}`);
}
// With a byte cap, avoid readFile(): a raced file growth would allocate
// the oversized content before the post-read check could reject it.
const buffer = await readFileHandleBounded({
handle,
filePath: params.filePath,
maxBytes: params.maxBytes,
});
return { buffer, stat };
} finally {
await handle.close();
@ -143,10 +195,13 @@ function readOpenedRegularFileSync(params: {
if (params.maxBytes !== undefined && stat.size > params.maxBytes) {
throw new Error(`File exceeds ${params.maxBytes} bytes: ${params.filePath}`);
}
const buffer = fsSync.readFileSync(params.fd);
if (params.maxBytes !== undefined && buffer.byteLength > params.maxBytes) {
throw new Error(`File exceeds ${params.maxBytes} bytes: ${params.filePath}`);
}
// Keep capped sync reads incremental for the same reason as async reads:
// readFileSync(fd) would buffer a raced oversized file before throwing.
const buffer = readFileDescriptorBounded({
fd: params.fd,
filePath: params.filePath,
maxBytes: params.maxBytes,
});
return { buffer, stat };
}

View File

@ -1,5 +1,6 @@
import fs from "node:fs/promises";
import path from "node:path";
import { guardedRename, guardedRm } from "./guarded-mutation.js";
export type ReplaceDirectoryAtomicOptions = {
stagedDir: string;
@ -21,7 +22,7 @@ export async function replaceDirectoryAtomic(
await fs.mkdir(parentDir, { recursive: true });
try {
await fs.rename(targetDir, backupDir);
await guardedRename({ from: targetDir, to: backupDir });
backupCreated = true;
} catch (err) {
if ((err as NodeJS.ErrnoException).code !== "ENOENT") {
@ -30,16 +31,16 @@ export async function replaceDirectoryAtomic(
}
try {
await fs.rename(stagedDir, targetDir);
await guardedRename({ from: stagedDir, to: targetDir });
} catch (err) {
if (backupCreated) {
await fs.rename(backupDir, targetDir).catch(() => undefined);
await guardedRename({ from: backupDir, to: targetDir }).catch(() => undefined);
backupCreated = false;
}
throw err;
}
if (backupCreated) {
await fs.rm(backupDir, { recursive: true, force: true });
await guardedRm({ target: backupDir, recursive: true, force: true, verifyAfter: false });
}
}

View File

@ -3,6 +3,7 @@ import syncFs from "node:fs";
import type { Stats } from "node:fs";
import fs from "node:fs/promises";
import path from "node:path";
import { assertSafePathPrefix } from "./safe-path-segment.js";
import { registerTempPathForExit } from "./temp-cleanup.js";
import { serializePathWrite } from "./write-queue.js";
@ -26,6 +27,7 @@ export type ReplaceFileAtomicSyncFileSystem = Pick<
typeof syncFs,
| "mkdirSync"
| "chmodSync"
| "readFileSync"
| "writeFileSync"
| "renameSync"
| "copyFileSync"
@ -76,6 +78,15 @@ function isPermissionRenameError(error: unknown): boolean {
return code === "EPERM" || code === "EEXIST";
}
const SUPPORTS_NOFOLLOW = process.platform !== "win32" && "O_NOFOLLOW" in syncFs.constants;
const OPEN_READ_FLAGS =
syncFs.constants.O_RDONLY | (SUPPORTS_NOFOLLOW ? syncFs.constants.O_NOFOLLOW : 0);
const OPEN_WRITE_EXCLUSIVE_FLAGS =
syncFs.constants.O_WRONLY |
syncFs.constants.O_CREAT |
syncFs.constants.O_EXCL |
(SUPPORTS_NOFOLLOW ? syncFs.constants.O_NOFOLLOW : 0);
async function sleep(ms: number): Promise<void> {
await new Promise((resolve) => setTimeout(resolve, ms));
}
@ -98,17 +109,7 @@ async function renameWithRetry(params: {
continue;
}
if (params.copyFallbackOnPermissionError && isPermissionRenameError(error)) {
const stat = await params.fsModule.lstat(params.dest).catch((lstatError) => {
if ((lstatError as NodeJS.ErrnoException).code === "ENOENT") {
return null;
}
throw lstatError;
});
if (stat?.isSymbolicLink()) {
throw new Error(`Refusing copy fallback through symlink destination: ${params.dest}`);
}
await params.fsModule.copyFile(params.src, params.dest);
await params.fsModule.unlink(params.src).catch(() => undefined);
await copyFallbackReplace(params.fsModule, params.src, params.dest);
return { method: "copy-fallback" };
}
throw error;
@ -142,23 +143,7 @@ function renameWithRetrySync(params: {
continue;
}
if (params.copyFallbackOnPermissionError && isPermissionRenameError(error)) {
let stat: Stats | null = null;
try {
stat = params.fsModule.lstatSync(params.dest);
} catch (lstatError) {
if ((lstatError as NodeJS.ErrnoException).code !== "ENOENT") {
throw lstatError;
}
}
if (stat?.isSymbolicLink()) {
throw new Error(`Refusing copy fallback through symlink destination: ${params.dest}`);
}
params.fsModule.copyFileSync(params.src, params.dest);
try {
params.fsModule.unlinkSync(params.src);
} catch {
// Best-effort cleanup after fallback replacement.
}
copyFallbackReplaceSync(params.fsModule, params.src, params.dest);
return { method: "copy-fallback" };
}
throw error;
@ -167,6 +152,96 @@ function renameWithRetrySync(params: {
throw new Error("Atomic rename retry loop exhausted.");
}
async function copyFallbackReplace(
fsModule: ReplaceFileAtomicFileSystem["promises"],
src: string,
dest: string,
): Promise<void> {
const sourceStat = await fsModule.lstat(src);
if (sourceStat.isSymbolicLink() || !sourceStat.isFile()) {
throw new Error(`Refusing copy fallback from non-file source: ${src}`);
}
const destStat = await fsModule.lstat(dest).catch((lstatError) => {
if ((lstatError as NodeJS.ErrnoException).code === "ENOENT") {
return null;
}
throw lstatError;
});
if (destStat?.isSymbolicLink()) {
throw new Error(`Refusing copy fallback through symlink destination: ${dest}`);
}
if (destStat) {
await fsModule.rm(dest, { force: true });
}
let sourceHandle: Awaited<ReturnType<ReplaceFileAtomicFileSystem["promises"]["open"]>> | null =
null;
let destHandle: Awaited<ReturnType<ReplaceFileAtomicFileSystem["promises"]["open"]>> | null =
null;
try {
sourceHandle = await fsModule.open(src, OPEN_READ_FLAGS);
destHandle = await fsModule.open(dest, OPEN_WRITE_EXCLUSIVE_FLAGS, sourceStat.mode & 0o777);
await destHandle.writeFile(await sourceHandle.readFile());
} finally {
await destHandle?.close().catch(() => undefined);
await sourceHandle?.close().catch(() => undefined);
}
await fsModule.unlink(src).catch(() => undefined);
}
function copyFallbackReplaceSync(
fsModule: ReplaceFileAtomicSyncFileSystem,
src: string,
dest: string,
): void {
const sourceStat = fsModule.lstatSync(src);
if (sourceStat.isSymbolicLink() || !sourceStat.isFile()) {
throw new Error(`Refusing copy fallback from non-file source: ${src}`);
}
let destStat: Stats | null = null;
try {
destStat = fsModule.lstatSync(dest);
} catch (lstatError) {
if ((lstatError as NodeJS.ErrnoException).code !== "ENOENT") {
throw lstatError;
}
}
if (destStat?.isSymbolicLink()) {
throw new Error(`Refusing copy fallback through symlink destination: ${dest}`);
}
if (destStat) {
fsModule.rmSync(dest, { force: true });
}
let sourceFd: number | undefined;
let destFd: number | undefined;
try {
sourceFd = fsModule.openSync(src, OPEN_READ_FLAGS);
destFd = fsModule.openSync(dest, OPEN_WRITE_EXCLUSIVE_FLAGS, sourceStat.mode & 0o777);
fsModule.writeFileSync(destFd, fsModule.readFileSync(sourceFd));
} finally {
if (destFd !== undefined) {
try {
fsModule.closeSync(destFd);
} catch {
// Best-effort close after fallback replacement.
}
}
if (sourceFd !== undefined) {
try {
fsModule.closeSync(sourceFd);
} catch {
// Best-effort close after fallback replacement.
}
}
}
try {
fsModule.unlinkSync(src);
} catch {
// Best-effort cleanup after fallback replacement.
}
}
function validateReplaceFilePath(filePath: string): void {
if (!filePath || filePath.includes("\0")) {
throw new Error("Atomic replace file path must be non-empty.");
@ -175,7 +250,8 @@ function validateReplaceFilePath(filePath: string): void {
function buildReplaceTempPath(filePath: string, tempPrefix?: string): string {
const dir = path.dirname(filePath);
return path.join(dir, `${tempPrefix ?? ".fs-safe-replace"}.${process.pid}.${randomUUID()}.tmp`);
const safePrefix = assertSafePathPrefix(tempPrefix ?? ".fs-safe-replace", { label: "atomic replace temp prefix" });
return path.join(dir, `${safePrefix}.${process.pid}.${randomUUID()}.tmp`);
}
async function resolveMode(options: ReplaceFileAtomicOptions): Promise<number> {

80
src/root-context.ts Normal file
View File

@ -0,0 +1,80 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { FsSafeError } from "./errors.js";
import { expandHomePrefix } from "./home-dir.js";
import { assertNoNulPathInput, isNotFoundPathError, isPathInside } from "./path.js";
export type RootContext = {
rootDir: string;
rootReal: string;
rootWithSep: string;
};
export const ensureTrailingSep = (value: string) =>
value.endsWith(path.sep) ? value : value + path.sep;
export function assertValidRootRelativePath(relativePath: string): void {
assertNoNulPathInput(relativePath, "relative path contains a NUL byte");
}
let cachedHomePath: { raw: string; real: string } | undefined;
export async function expandRelativePathWithHome(relativePath: string): Promise<string> {
const rawHome = process.env.HOME || process.env.USERPROFILE || os.homedir();
if (cachedHomePath?.raw !== rawHome) {
let realHome = rawHome;
try {
realHome = await fs.realpath(rawHome);
} catch {
// If the home dir cannot be canonicalized, keep lexical expansion behavior.
}
cachedHomePath = { raw: rawHome, real: realHome };
}
return expandHomePrefix(relativePath, { home: cachedHomePath.real });
}
export async function resolveRootContext(rootDir: string): Promise<RootContext> {
assertNoNulPathInput(rootDir, "root dir contains a NUL byte");
let rootReal: string;
try {
rootReal = await fs.realpath(rootDir);
const rootStat = await fs.stat(rootReal);
if (!rootStat.isDirectory()) {
throw new FsSafeError("invalid-path", "root dir is not a directory");
}
} catch (err) {
if (err instanceof FsSafeError) {
throw err;
}
if (isNotFoundPathError(err)) {
throw new FsSafeError("not-found", "root dir not found");
}
throw err;
}
return {
rootDir: path.resolve(rootDir),
rootReal,
rootWithSep: ensureTrailingSep(rootReal),
};
}
export async function resolvePathInRoot(
root: RootContext,
relativePath: string,
): Promise<{ rootReal: string; rootWithSep: string; resolved: string }> {
assertValidRootRelativePath(relativePath);
const expanded = await expandRelativePathWithHome(relativePath);
const resolved = path.resolve(root.rootWithSep, expanded);
if (!isPathInside(root.rootWithSep, resolved)) {
throw new FsSafeError("outside-workspace", "file is outside workspace root");
}
return { rootReal: root.rootReal, rootWithSep: root.rootWithSep, resolved };
}
export async function resolvePathWithinRoot(params: {
rootDir: string;
relativePath: string;
}): Promise<{ rootReal: string; rootWithSep: string; resolved: string }> {
return await resolvePathInRoot(await resolveRootContext(params.rootDir), params.relativePath);
}

24
src/root-errors.ts Normal file
View File

@ -0,0 +1,24 @@
import { FsSafeError } from "./errors.js";
import { hasNodeErrorCode } from "./path.js";
export function isAlreadyExistsError(error: unknown): boolean {
return hasNodeErrorCode(error, "EEXIST") || /File exists|EEXIST/i.test(String(error));
}
export function normalizePinnedWriteError(error: unknown): Error {
if (error instanceof FsSafeError) {
return error;
}
return new FsSafeError("invalid-path", "path is not a regular file under root", {
cause: error instanceof Error ? error : undefined,
});
}
export function normalizePinnedPathError(error: unknown): Error {
if (error instanceof FsSafeError) {
return error;
}
return new FsSafeError("path-alias", "path is not under root", {
cause: error instanceof Error ? error : undefined,
});
}

1746
src/root-impl.ts Normal file

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

79
src/safe-path-segment.ts Normal file
View File

@ -0,0 +1,79 @@
import { FsSafeError } from "./errors.js";
const SAFE_PATH_SEGMENT_PATTERN = /^[A-Za-z0-9_-][A-Za-z0-9._-]*$/;
const SAFE_DOT_PREFIX_PATH_SEGMENT_PATTERN = /^[A-Za-z0-9._-]+$/;
export type SafePathSegmentOptions = {
allowDotPrefix?: boolean;
label?: string;
};
export function isSafePathSegment(
segment: string,
options: SafePathSegmentOptions = {},
): boolean {
return (
segment !== "" &&
segment !== "." &&
segment !== ".." &&
!segment.includes("/") &&
!segment.includes("\\") &&
!segment.includes("\0") &&
(options.allowDotPrefix === true || !segment.startsWith(".")) &&
(options.allowDotPrefix === true
? SAFE_DOT_PREFIX_PATH_SEGMENT_PATTERN.test(segment)
: SAFE_PATH_SEGMENT_PATTERN.test(segment))
);
}
export function assertSafePathSegment(
segment: string,
options: SafePathSegmentOptions = {},
): string {
// Validate the exact value callers will later join into paths; trimming here
// would let whitespace-padded ids pass and then be used verbatim.
if (!isSafePathSegment(segment, options)) {
throw new FsSafeError(
"invalid-path",
`${options.label ?? "path segment"} must be a safe path segment`,
);
}
return segment;
}
export function sanitizeSafePathSegment(
value: string,
fallback: string,
options: SafePathSegmentOptions = {},
): string {
const sanitized = value
.trim()
.replace(/[\\/]+/g, "-")
.replace(/\0/g, "")
.replace(/[^A-Za-z0-9._-]+/g, "-")
.replace(/^-+|-+$/g, "");
if (isSafePathSegment(sanitized, options)) {
return sanitized;
}
return assertSafePathSegment(fallback, { ...options, label: "fallback path segment" });
}
export function assertSafePathPrefix(
prefix: string,
options: SafePathSegmentOptions = {},
): string {
// Prefixes are often derived from safe filenames. Normalize harmless
// filename characters first, but still reject real path-control bytes.
if (prefix.includes("/") || prefix.includes("\\") || prefix.includes("\0")) {
return assertSafePathSegment(prefix, {
allowDotPrefix: true,
...options,
label: options.label ?? "path prefix",
});
}
return assertSafePathSegment(prefix.replace(/[^A-Za-z0-9._-]+/g, "-"), {
allowDotPrefix: true,
...options,
label: options.label ?? "path prefix",
});
}

View File

@ -2,7 +2,9 @@ import { randomBytes } from "node:crypto";
import fs from "node:fs";
import fsp from "node:fs/promises";
import path from "node:path";
import { createAsyncDirectoryGuard } from "./directory-guard.js";
import { FsSafeError, type FsSafeErrorCode } from "./errors.js";
import { withAsyncDirectoryGuards } from "./guarded-mutation.js";
import { resolveHomeRelativePath } from "./home-dir.js";
import { openPinnedFileSync } from "./pinned-open.js";
@ -53,12 +55,26 @@ function readSecretFileOutcomeSync(
};
}
if (options.rejectSymlink && previewStat.isSymbolicLink()) {
return {
ok: false,
code: "symlink",
message: `${label} file at ${resolvedPath} must not be a symlink.`,
};
if (previewStat.isSymbolicLink()) {
if (!options.rejectSymlink) {
try {
previewStat = fs.statSync(resolvedPath);
} catch (error) {
const normalized = normalizeSecretReadError(error);
return {
ok: false,
code: (error as NodeJS.ErrnoException).code === "ENOENT" ? "not-found" : "invalid-path",
error: normalized,
message: `Failed to inspect ${label} file at ${resolvedPath}: ${String(normalized)}`,
};
}
} else {
return {
ok: false,
code: "symlink",
message: `${label} file at ${resolvedPath} must not be a symlink.`,
};
}
}
if (!previewStat.isFile()) {
return {
@ -241,6 +257,7 @@ export async function writeSecretFileAtomic(params: {
const resolvedRootReal = await fsp.realpath(resolvedRoot);
const parentDir = await fsp.realpath(intendedParentDir);
assertRealPathWithinRoot(resolvedRootReal, parentDir);
const parentGuard = await createAsyncDirectoryGuard(parentDir);
const fileName = path.basename(resolvedFile);
const finalFilePath = path.join(parentDir, fileName);
@ -276,7 +293,9 @@ export async function writeSecretFileAtomic(params: {
if (refreshedParentReal !== parentDir) {
throw new Error(`Private secret parent directory changed during write for ${finalFilePath}.`);
}
await fsp.rename(tempPath, finalFilePath);
await withAsyncDirectoryGuards([parentGuard], async () => {
await fsp.rename(tempPath, finalFilePath);
});
createdTemp = false;
await enforcePrivatePathMode(finalFilePath, mode, "file");
} finally {

View File

@ -1,9 +1,14 @@
import crypto, { randomUUID } from "node:crypto";
import fs from "node:fs/promises";
import path from "node:path";
import { assertAsyncDirectoryGuard, createAsyncDirectoryGuard } from "./directory-guard.js";
import { withAsyncDirectoryGuards } from "./guarded-mutation.js";
import { sanitizeUntrustedFileName } from "./filename.js";
import { root } from "./root.js";
import { assertSafePathPrefix } from "./safe-path-segment.js";
import { resolveSecureTempRoot } from "./secure-temp-dir.js";
import { registerTempPathForExit } from "./temp-cleanup.js";
import { getFsSafeTestHooks } from "./test-hooks.js";
import { serializePathWrite } from "./write-queue.js";
export type WriteSiblingTempFileOptions<T> = {
@ -12,6 +17,7 @@ export type WriteSiblingTempFileOptions<T> = {
resolveFinalPath: (result: T) => string;
tempPrefix?: string;
dirMode?: number;
chmodDir?: boolean;
mode?: number;
syncTempFile?: boolean;
syncParentDir?: boolean;
@ -23,7 +29,10 @@ export type WriteSiblingTempFileResult<T> = {
};
function buildTempPath(dir: string, tempPrefix?: string): string {
return path.join(dir, `${tempPrefix ?? ".fs-safe-stream"}.${process.pid}.${randomUUID()}.tmp`);
const safePrefix = assertSafePathPrefix(tempPrefix ?? ".fs-safe-stream", {
label: "sibling temp prefix",
});
return path.join(dir, `${safePrefix}.${process.pid}.${randomUUID()}.tmp`);
}
async function syncFileBestEffort(filePath: string): Promise<void> {
@ -64,7 +73,10 @@ export async function writeSiblingTempFile<T>(
): Promise<WriteSiblingTempFileResult<T>> {
const dir = path.resolve(options.dir);
await fs.mkdir(dir, { recursive: true, mode: options.dirMode ?? 0o700 });
await fs.chmod(dir, options.dirMode ?? 0o700).catch(() => undefined);
if (options.chmodDir !== false) {
await fs.chmod(dir, options.dirMode ?? 0o700).catch(() => undefined);
}
const dirGuard = await createAsyncDirectoryGuard(dir);
const tempPath = buildTempPath(dir, options.tempPrefix);
const unregisterTempPath = registerTempPathForExit(tempPath);
let tempExists = false;
@ -80,7 +92,9 @@ export async function writeSiblingTempFile<T>(
const filePath = path.resolve(options.resolveFinalPath(result));
assertFinalPathIsSibling(dir, filePath);
await serializePathWrite(filePath, async () => {
await fs.rename(tempPath, filePath);
await withAsyncDirectoryGuards([dirGuard], async () => {
await fs.rename(tempPath, filePath);
});
tempExists = false;
unregisterTempPath();
if (options.mode !== undefined) {
@ -105,11 +119,14 @@ function buildSiblingTempPath(params: {
tempPrefix: string;
}): string {
const id = crypto.randomUUID();
const safePrefix = assertSafePathPrefix(params.tempPrefix, {
label: "sibling temp prefix",
});
const safeTail = sanitizeUntrustedFileName(
path.basename(params.targetPath),
params.fallbackFileName,
);
return path.join(path.dirname(params.targetPath), `${params.tempPrefix}${id}-${safeTail}.part`);
return path.join(path.dirname(params.targetPath), `${safePrefix}${id}-${safeTail}.part`);
}
export async function writeViaSiblingTempPath(params: {
@ -136,18 +153,32 @@ export async function writeViaSiblingTempPath(params: {
) {
throw new Error("Target path is outside the allowed root");
}
const rootGuard = await createAsyncDirectoryGuard(rootDir);
const tempDir = await fs.mkdtemp(
path.join(
resolveSecureTempRoot({
fallbackPrefix: "fs-safe-output",
unsafeFallbackLabel: "sibling temp output dir",
warn: () => undefined,
}),
"fs-safe-output-",
),
);
const tempPath = buildSiblingTempPath({
targetPath,
targetPath: path.join(tempDir, path.basename(targetPath)),
fallbackFileName: params.fallbackFileName ?? "output.bin",
tempPrefix: params.tempPrefix ?? ".fs-safe-output-",
});
const unregisterTempPath = registerTempPathForExit(tempPath);
const unregisterTempPath = registerTempPathForExit(tempDir, { recursive: true });
try {
await getFsSafeTestHooks()?.beforeSiblingTempWrite?.(tempPath);
await params.writeTemp(tempPath);
await assertAsyncDirectoryGuard(rootGuard);
const targetRoot = await root(rootDir);
await targetRoot.copyIn(relativeTargetPath, tempPath, { mkdir: false });
await assertAsyncDirectoryGuard(rootGuard);
} finally {
await fs.rm(tempPath, { force: true }).catch(() => {});
await fs.rm(tempDir, { recursive: true, force: true }).catch(() => {});
unregisterTempPath();
}
}

View File

@ -1,7 +1,9 @@
import fsSync from "node:fs";
import type { Stats } from "node:fs";
import type { FileHandle } from "node:fs/promises";
import fs from "node:fs/promises";
import path from "node:path";
import { sameFileIdentity } from "./file-identity.js";
export type SidecarLockRetryOptions = {
retries?: number;
@ -11,12 +13,15 @@ export type SidecarLockRetryOptions = {
randomize?: boolean;
};
export type SidecarLockStaleRecovery = "fail-closed";
export type SidecarLockAcquireOptions<TPayload extends Record<string, unknown>> = {
targetPath: string;
lockPath?: string;
staleMs: number;
timeoutMs?: number;
retry?: SidecarLockRetryOptions;
staleRecovery?: SidecarLockStaleRecovery;
allowReentrant?: boolean;
payload: () => TPayload | Promise<TPayload>;
shouldReclaim?: (params: {
@ -56,6 +61,7 @@ type HeldLock = {
count: number;
handle: FileHandle;
lockPath: string;
snapshot: LockSnapshot;
acquiredAt: number;
metadata: Record<string, unknown>;
releasePromise?: Promise<void>;
@ -88,17 +94,78 @@ function resolveManagerState(key: string): SidecarLockManagerState {
return state;
}
async function readJsonPayload(lockPath: string): Promise<Record<string, unknown> | null> {
type LockSnapshot = {
raw?: string;
payload: Record<string, unknown> | null;
stat?: Stats;
};
async function readLockSnapshot(lockPath: string): Promise<LockSnapshot | null> {
try {
const parsed = JSON.parse(await fs.readFile(lockPath, "utf8")) as unknown;
return parsed && typeof parsed === "object" && !Array.isArray(parsed)
? (parsed as Record<string, unknown>)
: null;
const stat = await fs.lstat(lockPath);
const raw = await fs.readFile(lockPath, "utf8");
try {
const parsed = JSON.parse(raw) as unknown;
const payload =
parsed && typeof parsed === "object" && !Array.isArray(parsed)
? (parsed as Record<string, unknown>)
: null;
return { raw, payload, stat };
} catch {
return { raw, payload: null, stat };
}
} catch {
return null;
}
}
function snapshotMatches(current: LockSnapshot, observed: LockSnapshot): boolean {
if (observed.stat && current.stat && !sameFileIdentity(observed.stat, current.stat)) {
return false;
}
if (observed.raw !== undefined) {
return current.raw === observed.raw;
}
return observed.stat !== undefined && current.stat !== undefined;
}
async function removeLockIfUnchanged(
lockPath: string,
observed: LockSnapshot | null,
): Promise<boolean> {
const current = await readLockSnapshot(lockPath);
if (!current || !observed) {
return false;
}
if (!snapshotMatches(current, observed)) {
// The lock changed after we decided it was stale. Leave the fresh holder's
// file alone; deleting by path here would break mutual exclusion.
return false;
}
await fs.rm(lockPath, { force: true }).catch(() => undefined);
return true;
}
async function lockSnapshotStillPresent(
lockPath: string,
observed: LockSnapshot | null,
): Promise<boolean> {
const current = await readLockSnapshot(lockPath);
return !!current && !!observed && snapshotMatches(current, observed);
}
function snapshotMatchesSync(lockPath: string, observed: LockSnapshot): boolean {
try {
const stat = fsSync.lstatSync(lockPath);
if (observed.stat && !sameFileIdentity(observed.stat, stat)) {
return false;
}
return observed.raw === undefined || fsSync.readFileSync(lockPath, "utf8") === observed.raw;
} catch {
return false;
}
}
async function resolveNormalizedTargetPath(targetPath: string): Promise<string> {
const resolved = path.resolve(targetPath);
const dir = path.dirname(resolved);
@ -142,7 +209,9 @@ function releaseAllLocksSync(state: SidecarLockManagerState): void {
for (const [normalizedTargetPath, held] of state.held) {
void held.handle.close().catch(() => undefined);
try {
fsSync.rmSync(held.lockPath, { force: true });
if (snapshotMatchesSync(held.lockPath, held.snapshot)) {
fsSync.rmSync(held.lockPath, { force: true });
}
} catch {
// Best-effort process-exit cleanup.
}
@ -175,7 +244,7 @@ async function releaseHeldLock(
state.held.delete(normalizedTargetPath);
held.releasePromise = (async () => {
await held.handle.close().catch(() => undefined);
await fs.rm(held.lockPath, { force: true }).catch(() => undefined);
await removeLockIfUnchanged(held.lockPath, held.snapshot);
})();
try {
await held.releasePromise;
@ -222,15 +291,19 @@ export function createSidecarLockManager(key: string) {
let handle: FileHandle | null = null;
try {
handle = await fs.open(lockPath, "wx");
const payload = await options.payload();
const raw = `${JSON.stringify(payload, null, 2)}\n`;
await handle.writeFile(raw, "utf8");
const snapshot = { raw, payload, stat: await handle.stat() };
const createdHeld: HeldLock = {
count: 1,
handle,
lockPath,
snapshot,
acquiredAt: Date.now(),
metadata: options.metadata ?? {},
};
state.held.set(normalizedTargetPath, createdHeld);
await handle.writeFile(`${JSON.stringify(await options.payload(), null, 2)}\n`, "utf8");
const release = () =>
releaseHeldLock(state, normalizedTargetPath, createdHeld).then(() => undefined);
return {
@ -241,31 +314,57 @@ export function createSidecarLockManager(key: string) {
};
} catch (err) {
if (handle) {
const failedSnapshot: LockSnapshot = { payload: null };
try {
failedSnapshot.stat = await handle.stat();
} catch {
// Best-effort cleanup of a failed exclusive create.
}
const current = state.held.get(normalizedTargetPath);
if (current?.handle === handle) {
state.held.delete(normalizedTargetPath);
}
await handle.close().catch(() => undefined);
// If payload serialization/write fails, the file may be empty or
// partial JSON, so remove while our exclusive handle is still open.
await fs.rm(lockPath, { force: true }).catch(() => undefined);
await handle.close().catch(() => undefined);
// Windows can refuse removing an open file; retry after close but
// only if the path still points at the file identity we created.
await removeLockIfUnchanged(lockPath, failedSnapshot);
}
if ((err as { code?: unknown }).code !== "EEXIST") {
throw err;
}
const nowMs = Date.now();
const payload = await readJsonPayload(lockPath);
const snapshot = await readLockSnapshot(lockPath);
if (!snapshot) {
continue;
}
const shouldReclaim = options.shouldReclaim ?? defaultShouldReclaim;
if (
await shouldReclaim({
lockPath,
normalizedTargetPath,
payload,
payload: snapshot?.payload ?? null,
staleMs: options.staleMs,
nowMs,
heldByThisProcess: state.held.has(normalizedTargetPath),
})
) {
await fs.rm(lockPath, { force: true }).catch(() => undefined);
continue;
if (!(await lockSnapshotStillPresent(lockPath, snapshot))) {
continue;
}
// Node exposes only path-based unlink/rename here. A stale-lock
// reclaimer cannot bind the delete to the file it inspected, so a
// concurrent release+fresh-acquire could otherwise lose its lock.
// Fail closed and let callers choose a higher-level recovery path.
if ((options.staleRecovery ?? "fail-closed") === "fail-closed") {
throw Object.assign(new Error(`file lock stale for ${normalizedTargetPath}`), {
code: "file_lock_stale",
lockPath,
normalizedTargetPath,
});
}
}
const elapsed = Date.now() - startedAt;
if (

View File

@ -15,3 +15,18 @@ export {
type JsonStoreLockOptions,
type JsonStoreOptions,
} from "./json-store.js";
export {
ackJsonDurableQueueEntry,
ensureJsonDurableQueueDirs,
jsonDurableQueueEntryExists,
loadJsonDurableQueueEntry,
loadPendingJsonDurableQueueEntries,
moveJsonDurableQueueEntryToFailed,
readJsonDurableQueueEntry,
resolveJsonDurableQueueEntryPaths,
unlinkBestEffort,
writeJsonDurableQueueEntry,
type JsonDurableQueueEntryPaths,
type JsonDurableQueueLoadOptions,
type JsonDurableQueueReadResult,
} from "./json-durable-queue.js";

View File

@ -1,6 +1,7 @@
import crypto from "node:crypto";
import { mkdtemp, rm } from "node:fs/promises";
import path from "node:path";
import { assertSafePathSegment, sanitizeSafePathSegment } from "./safe-path-segment.js";
import { resolveSecureTempRoot } from "./secure-temp-dir.js";
import { registerTempPathForExit } from "./temp-cleanup.js";
@ -28,9 +29,9 @@ function sanitizeExtension(extension?: string): string {
}
export function sanitizeTempFileName(fileName: string): string {
const base = path.basename(fileName).replace(/[^a-zA-Z0-9._-]+/g, "-");
const normalized = base.replace(/^-+|-+$/g, "");
return normalized || "download.bin";
return sanitizeSafePathSegment(path.basename(fileName), "download.bin", {
allowDotPrefix: true,
});
}
export function buildRandomTempFilePath(params: {
@ -48,7 +49,9 @@ export function buildRandomTempFilePath(params: {
typeof nowCandidate === "number" && Number.isFinite(nowCandidate)
? Math.trunc(nowCandidate)
: Date.now();
const uuid = params.uuid?.trim() || crypto.randomUUID();
const uuid = params.uuid
? assertSafePathSegment(params.uuid.trim(), { label: "temp uuid" })
: crypto.randomUUID();
return path.join(rootDir, `${prefix}-${now}-${uuid}${extension}`);
}

View File

@ -4,6 +4,18 @@ export type FsSafeTestHooks = {
afterPreOpenLstat?: (filePath: string) => Promise<void> | void;
beforeOpen?: (filePath: string, flags: number) => Promise<void> | void;
afterOpen?: (filePath: string, handle: FileHandle) => Promise<void> | void;
beforeArchiveOutputMutation?: (
operation: "mkdir" | "chmod",
targetPath: string,
) => Promise<void> | void;
beforeFileStorePruneDescend?: (dirPath: string) => Promise<void> | void;
beforeFileStoreSyncPrivateWrite?: (filePath: string) => void;
beforeRootFallbackMutation?: (
operation: "mkdir" | "move" | "remove",
targetPath: string,
) => Promise<void> | void;
beforeSiblingTempWrite?: (tempPath: string) => Promise<void> | void;
beforeTrashMove?: (targetPath: string, destPath: string) => void;
};
let fsSafeTestHooks: FsSafeTestHooks | undefined;

View File

@ -4,6 +4,13 @@ export type WriteTextAtomicOptions = {
mode?: number;
dirMode?: number;
trailingNewline?: boolean;
/**
* When false, skip the temp-file and parent-directory fsync calls while
* preserving the temp-file replace/rename behavior.
*
* Defaults to true.
*/
durable?: boolean;
};
export async function writeTextAtomic(
@ -12,13 +19,14 @@ export async function writeTextAtomic(
options?: WriteTextAtomicOptions,
): Promise<void> {
const payload = options?.trailingNewline && !content.endsWith("\n") ? `${content}\n` : content;
const durable = options?.durable ?? true;
await replaceFileAtomic({
filePath,
content: payload,
mode: options?.mode ?? 0o600,
dirMode: options?.dirMode ?? (0o777 & ~process.umask()),
copyFallbackOnPermissionError: true,
syncTempFile: true,
syncParentDir: true,
syncTempFile: durable,
syncParentDir: durable,
});
}

View File

@ -1,6 +1,9 @@
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { sameFileIdentity } from "./file-identity.js";
import { guardedRenameSync, guardedRmSync } from "./guarded-mutation.js";
import { getFsSafeTestHooks } from "./test-hooks.js";
export type MovePathToTrashOptions = {
allowedRoots?: Iterable<string>;
@ -27,29 +30,69 @@ function isSameOrChildPath(candidate: string, parent: string): boolean {
}
function resolveAllowedTrashRoots(allowedRoots?: Iterable<string>): string[] {
const roots = [...(allowedRoots ?? [os.homedir(), os.tmpdir()])].map((root) => {
const roots = [...(allowedRoots ?? [os.homedir(), os.tmpdir()])].flatMap((root) => {
const lexicalRoot = path.resolve(root);
try {
return path.resolve(fs.realpathSync.native(root));
// Keep both spellings: broken symlink targets cannot be realpathed and
// may only compare equal to the caller's lexical allowed root.
return [path.resolve(fs.realpathSync.native(root)), lexicalRoot];
} catch {
return path.resolve(root);
return [lexicalRoot];
}
});
return [...new Set(roots)];
}
function assertAllowedTrashTarget(targetPath: string, allowedRoots?: Iterable<string>): void {
let resolvedTargetPath = path.resolve(targetPath);
type TrashTargetGuard = {
path: string;
realPath: string;
realPathResolved: boolean;
stat: fs.Stats;
};
function resolveTrashTargetPath(targetPath: string): { path: string; resolved: boolean } {
try {
resolvedTargetPath = path.resolve(fs.realpathSync.native(targetPath));
return { path: path.resolve(fs.realpathSync.native(targetPath)), resolved: true };
} catch {
// The subsequent move will surface missing or inaccessible targets.
// Broken symlinks are valid trash targets. Fall back to the lexical path,
// then rely on lstat identity so the move renames the symlink itself.
return { path: path.resolve(targetPath), resolved: false };
}
}
function assertAllowedTrashTarget(
targetPath: string,
allowedRoots?: Iterable<string>,
): TrashTargetGuard {
const stat = fs.lstatSync(path.resolve(targetPath));
const resolvedTarget = resolveTrashTargetPath(targetPath);
const resolvedTargetPath = resolvedTarget.path;
const isAllowed = resolveAllowedTrashRoots(allowedRoots).some(
(root) => resolvedTargetPath !== root && isSameOrChildPath(resolvedTargetPath, root),
);
if (!isAllowed) {
throw new Error(`Refusing to trash path outside allowed roots: ${targetPath}`);
}
return {
path: path.resolve(targetPath),
realPath: resolvedTargetPath,
realPathResolved: resolvedTarget.resolved,
stat,
};
}
function assertTrashTargetGuard(guard: TrashTargetGuard): void {
const stat = fs.lstatSync(guard.path);
if (!sameFileIdentity(stat, guard.stat)) {
throw new Error(`Refusing to trash path after it changed: ${guard.path}`);
}
const current = resolveTrashTargetPath(guard.path);
if (guard.realPathResolved && (!current.resolved || current.path !== guard.realPath)) {
throw new Error(`Refusing to trash path after it changed: ${guard.path}`);
}
if (!guard.realPathResolved && current.resolved) {
throw new Error(`Refusing to trash path after it changed: ${guard.path}`);
}
}
function resolveTrashDir(): string {
@ -103,9 +146,11 @@ function reserveTrashDestination(trashDir: string, base: string, timestamp: numb
return resolveContainedPath(container, base);
}
function movePathToDestination(targetPath: string, dest: string): boolean {
function movePathToDestination(target: TrashTargetGuard, dest: string): boolean {
getFsSafeTestHooks()?.beforeTrashMove?.(target.path, dest);
assertTrashTargetGuard(target);
try {
fs.renameSync(targetPath, dest);
guardedRenameSync({ from: target.path, to: dest });
return true;
} catch (error) {
if (getFsErrorCode(error) !== "EXDEV") {
@ -117,8 +162,10 @@ function movePathToDestination(targetPath: string, dest: string): boolean {
}
try {
fs.cpSync(targetPath, dest, { recursive: true, force: false, errorOnExist: true });
fs.rmSync(targetPath, { recursive: true, force: false });
assertTrashTargetGuard(target);
fs.cpSync(target.path, dest, { recursive: true, force: false, errorOnExist: true });
assertTrashTargetGuard(target);
guardedRmSync({ target: target.path, recursive: true, force: false, verifyAfter: false });
return true;
} catch (error) {
if (isTrashDestinationCollision(error)) {
@ -134,12 +181,12 @@ export async function movePathToTrash(
): Promise<string> {
// Avoid resolving external trash helpers through the service PATH during cleanup.
const base = trashBaseName(targetPath);
assertAllowedTrashTarget(targetPath, options.allowedRoots);
const target = assertAllowedTrashTarget(targetPath, options.allowedRoots);
const trashDir = resolveTrashDir();
const timestamp = Date.now();
for (let attempt = 0; attempt < TRASH_DESTINATION_RETRY_LIMIT; attempt += 1) {
const dest = reserveTrashDestination(trashDir, base, timestamp);
if (movePathToDestination(targetPath, dest)) {
if (movePathToDestination(target, dest)) {
return dest;
}
}

View File

@ -0,0 +1,185 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import { ensureAbsoluteDirectory } from "../src/absolute-path.js";
const tempDirs: string[] = [];
async function tempRoot(prefix: string): Promise<string> {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), prefix));
tempDirs.push(dir);
return dir;
}
afterEach(async () => {
vi.restoreAllMocks();
await Promise.all(tempDirs.splice(0).map((dir) => fs.rm(dir, { force: true, recursive: true })));
});
describe("ensureAbsoluteDirectory", () => {
it("safely creates missing absolute directory parents from a real ancestor", async () => {
const root = await fs.realpath(await tempRoot("fs-safe-absolute-dir-"));
const targetDir = path.join(root, "nested", "deeper");
await expect(
ensureAbsoluteDirectory(targetDir, { scopeLabel: "output directory", mode: 0o700 }),
).resolves.toEqual({ ok: true, path: targetDir });
expect((await fs.stat(targetDir)).isDirectory()).toBe(true);
});
it("rejects relative absolute-directory inputs", async () => {
await expect(
ensureAbsoluteDirectory(path.join("..", "..", "..", "escape"), {
scopeLabel: "output directory",
}),
).resolves.toMatchObject({ ok: false, code: "invalid-path" });
});
it("rejects absolute directory creation when the existing target is not a directory", async () => {
const root = await fs.realpath(await tempRoot("fs-safe-absolute-dir-file-"));
const targetPath = path.join(root, "file.txt");
await fs.writeFile(targetPath, "file", "utf8");
await expect(
ensureAbsoluteDirectory(targetPath, { scopeLabel: "output directory" }),
).resolves.toMatchObject({ ok: false, code: "not-file" });
});
it.runIf(process.platform !== "win32")(
"rejects absolute directory creation through symlinked existing segments",
async () => {
const root = await fs.realpath(await tempRoot("fs-safe-absolute-dir-link-"));
const outside = await fs.realpath(await tempRoot("fs-safe-absolute-dir-outside-"));
const linkDir = path.join(root, "link");
await fs.symlink(outside, linkDir);
await expect(
ensureAbsoluteDirectory(path.join(linkDir, "nested"), {
scopeLabel: "output directory",
}),
).resolves.toMatchObject({ ok: false, code: "symlink" });
await expect(fs.readdir(outside)).resolves.toEqual([]);
},
);
it.runIf(process.platform !== "win32")(
"rejects symlinked parents even when the requested suffix already exists",
async () => {
const root = await fs.realpath(await tempRoot("fs-safe-absolute-dir-link-existing-"));
const outside = await fs.realpath(
await tempRoot("fs-safe-absolute-dir-link-existing-outside-"),
);
const existing = path.join(outside, "existing");
const linkDir = path.join(root, "link");
await fs.mkdir(existing);
await fs.symlink(outside, linkDir);
await expect(
ensureAbsoluteDirectory(path.join(linkDir, "existing", "new"), {
scopeLabel: "output directory",
}),
).resolves.toMatchObject({ ok: false, code: "symlink" });
await expect(fs.stat(path.join(existing, "new"))).rejects.toMatchObject({ code: "ENOENT" });
},
);
it("returns a policy failure when an intermediate component is a file", async () => {
const root = await fs.realpath(await tempRoot("fs-safe-absolute-dir-file-component-"));
const filePath = path.join(root, "file");
await fs.writeFile(filePath, "file", "utf8");
await expect(
ensureAbsoluteDirectory(path.join(filePath, "child"), {
scopeLabel: "output directory",
}),
).resolves.toMatchObject({ ok: false, code: "not-file" });
});
it.runIf(process.platform !== "win32")(
"rejects absolute directory creation when an existing parent is swapped before mkdir",
async () => {
const root = await fs.realpath(await tempRoot("fs-safe-absolute-dir-race-"));
const outside = await fs.realpath(await tempRoot("fs-safe-absolute-dir-race-outside-"));
const parentDir = path.join(root, "parent");
const targetDir = path.join(parentDir, "child");
await fs.mkdir(parentDir);
const realLstat = fs.lstat.bind(fs);
let swapped = false;
const lstatSpy = vi.spyOn(fs, "lstat").mockImplementation(async (...args) => {
const candidate = String(args[0]);
if (!swapped && candidate === targetDir) {
swapped = true;
await fs.rename(parentDir, `${parentDir}-real`);
await fs.symlink(outside, parentDir, "dir");
}
return await realLstat(...args);
});
try {
await expect(
ensureAbsoluteDirectory(targetDir, { scopeLabel: "output directory" }),
).resolves.toMatchObject({ ok: false, code: "symlink" });
} finally {
lstatSpy.mockRestore();
}
await expect(fs.stat(path.join(outside, "child"))).rejects.toMatchObject({ code: "ENOENT" });
},
);
it.runIf(process.platform !== "win32")(
"rejects absolute directory creation when the existing target changes before return",
async () => {
const root = await fs.realpath(await tempRoot("fs-safe-absolute-dir-target-race-"));
const outside = await fs.realpath(
await tempRoot("fs-safe-absolute-dir-target-race-outside-"),
);
const targetDir = path.join(root, "target");
await fs.mkdir(targetDir);
const realRealpath = fs.realpath.bind(fs);
let swapped = false;
const realpathSpy = vi.spyOn(fs, "realpath").mockImplementation(async (...args) => {
const candidate = String(args[0]);
if (!swapped && candidate === targetDir) {
swapped = true;
const resolved = await realRealpath(...args);
await fs.rename(targetDir, `${targetDir}-real`);
await fs.symlink(outside, targetDir, "dir");
return resolved;
}
return await realRealpath(...args);
});
try {
await expect(
ensureAbsoluteDirectory(targetDir, { scopeLabel: "output directory" }),
).resolves.toMatchObject({ ok: false, code: "symlink" });
} finally {
realpathSpy.mockRestore();
}
},
);
it("rethrows operational absolute directory creation failures", async () => {
const root = await fs.realpath(await tempRoot("fs-safe-absolute-dir-io-"));
const targetDir = path.join(root, "nested");
const realMkdir = fs.mkdir.bind(fs);
const mkdirSpy = vi.spyOn(fs, "mkdir").mockImplementation(async (...args) => {
if (String(args[0]) === targetDir) {
throw Object.assign(new Error("permission denied"), { code: "EACCES" });
}
return await realMkdir(...args);
});
try {
await expect(
ensureAbsoluteDirectory(targetDir, { scopeLabel: "output directory" }),
).rejects.toMatchObject({ code: "EACCES" });
} finally {
mkdirSpy.mockRestore();
}
});
});

View File

@ -47,7 +47,7 @@ afterEach(async () => {
await Promise.all(tempDirs.splice(0).map((dir) => fsp.rm(dir, { force: true, recursive: true })));
});
describe("additional bypass parity", () => {
describe("additional helper boundary bypass attempts", () => {
it("rejects archive traversal payloads before resolving output paths", async () => {
const layout = await makeTempLayout("fs-safe-archive-payloads");
@ -73,7 +73,12 @@ describe("additional bypass parity", () => {
it("sanitizes temp file names and keeps temp file helpers inside their created directory", async () => {
const layout = await makeTempLayout("fs-safe-temp");
expect(sanitizeTempFileName("../../evil.txt")).toBe("evil.txt");
expect(sanitizeTempFileName("..\\evil.txt")).toBe("..-evil.txt");
if (process.platform !== "win32") {
// On windows "\" is a reserved path separator and cannot appear in a
// filename, so this case only exercises the posix sanitizer where "\"
// is a literal name character that needs neutralizing.
expect(sanitizeTempFileName("..\\evil.txt")).toBe("..-evil.txt");
}
expect(sanitizeTempFileName("\u0000../evil.txt")).toBe("evil.txt");
const target = await tempFile({ rootDir: layout.base, prefix: "../../prefix", fileName: "../../evil.txt" });

View File

@ -0,0 +1,144 @@
import fsp from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { root as openRoot } from "../src/index.js";
type TempLayout = {
outside: string;
outsideFile: string;
root: string;
};
const tempDirs: string[] = [];
const DOT_SEGMENTS = ["..", "...", ". .", ".. ", " ..", "%2e%2e", "%252e%252e"] as const;
const SEPARATORS = ["/", "//", "\\", "\\\\", "%2f", "%5c"] as const;
const TARGETS = ["secret.txt", "etc/passwd", "Windows/win.ini", "CON", "NUL", "aux.txt"] as const;
const CONTROL_PAYLOADS = [
"..\u0000/secret.txt",
"..\u0001/secret.txt",
"..\u001f/secret.txt",
"safe\u0000name.txt",
"safe\u202ename.txt",
"safe\ufeffname.txt",
] as const;
async function makeTempLayout(prefix: string): Promise<TempLayout> {
const root = await fsp.mkdtemp(path.join(os.tmpdir(), `${prefix}-root-`));
const outside = await fsp.mkdtemp(path.join(os.tmpdir(), `${prefix}-outside-`));
tempDirs.push(root, outside);
const outsideFile = path.join(outside, "secret.txt");
await fsp.writeFile(outsideFile, "outside secret");
return { outside, outsideFile, root };
}
function buildPayloadCorpus(): string[] {
const payloads = new Set<string>();
for (const dot of DOT_SEGMENTS) {
for (const sep of SEPARATORS) {
for (const target of TARGETS) {
payloads.add(`${dot}${sep}${target}`);
payloads.add(`nested${sep}${dot}${sep}${dot}${sep}${target}`);
}
}
}
payloads.add("/etc/passwd");
payloads.add("//server/share/secret.txt");
payloads.add("C:/Windows/win.ini");
payloads.add("C:\\Windows\\win.ini");
for (const payload of CONTROL_PAYLOADS) {
payloads.add(payload);
}
return [...payloads];
}
async function expectOutsideUntouched(layout: TempLayout): Promise<void> {
await expect(fsp.readFile(layout.outsideFile, "utf8")).resolves.toBe("outside secret");
}
async function closeIfOpened(value: unknown): Promise<void> {
if (typeof value !== "object" || value === null) {
return;
}
if (Symbol.asyncDispose in value) {
await (value as { [Symbol.asyncDispose](): Promise<void> })[Symbol.asyncDispose]();
return;
}
if ("handle" in value) {
await (value as { handle: { close(): Promise<void> } }).handle.close();
return;
}
if ("close" in value) {
await (value as { close(): Promise<void> }).close();
}
}
async function attemptAll(rootDir: Awaited<ReturnType<typeof openRoot>>, payload: string): Promise<void> {
const opened = await rootDir.open(payload).catch((error: unknown) => error);
await closeIfOpened(opened);
const writable = await rootDir.openWritable(payload).catch((error: unknown) => error);
await closeIfOpened(writable);
await Promise.allSettled([
rootDir.read(payload),
rootDir.stat(payload),
rootDir.write(payload, "payload"),
rootDir.create(payload, "payload"),
rootDir.append(payload, "payload"),
]);
}
afterEach(async () => {
await Promise.all(tempDirs.splice(0).map((dir) => fsp.rm(dir, { force: true, recursive: true })));
});
describe("adversarial boundary payloads", () => {
it("never reads, writes, or deletes outside the root for a generated traversal corpus", async () => {
const layout = await makeTempLayout("fs-safe-adversarial-corpus");
await fsp.mkdir(path.join(layout.root, "nested"), { recursive: true });
await fsp.mkdir(path.join(layout.root, "safe"), { recursive: true });
const safeRoot = await openRoot(layout.root);
const payloads = buildPayloadCorpus().slice(0, 96);
for (const payload of payloads) {
await attemptAll(safeRoot, payload);
await expectOutsideUntouched(layout);
}
}, 15_000);
it("rejects chained symlink parent escapes across read and write surfaces", async () => {
const layout = await makeTempLayout("fs-safe-symlink-chain");
await fsp.mkdir(path.join(layout.root, "a"), { recursive: true });
await fsp.symlink(path.join(layout.root, "a"), path.join(layout.root, "link-a"), "dir");
await fsp.symlink(layout.outside, path.join(layout.root, "a", "link-out"), "dir");
const safeRoot = await openRoot(layout.root);
for (const payload of ["link-a/link-out/secret.txt", "a/link-out/secret.txt"]) {
await expect(safeRoot.read(payload), `read ${payload}`).rejects.toBeTruthy();
await expect(safeRoot.write(payload, "pwned"), `write ${payload}`).rejects.toBeTruthy();
await expect(safeRoot.remove(payload), `remove ${payload}`).rejects.toBeTruthy();
}
await expectOutsideUntouched(layout);
});
it("does not clobber outside files when copy and move payloads mix source and destination attacks", async () => {
const layout = await makeTempLayout("fs-safe-copy-move-corpus");
const source = path.join(layout.root, "source.txt");
await fsp.writeFile(source, "source");
await fsp.symlink(layout.outsideFile, path.join(layout.root, "outside-link.txt"), "file");
const safeRoot = await openRoot(layout.root);
const payloads = buildPayloadCorpus().slice(0, 48);
for (const payload of payloads) {
await Promise.allSettled([
safeRoot.copyIn(payload, source),
safeRoot.copyIn("copied.txt", layout.outsideFile),
safeRoot.move("source.txt", payload, { overwrite: true }),
safeRoot.move(payload, "moved.txt", { overwrite: true }),
safeRoot.move("outside-link.txt", "moved-link.txt", { overwrite: true }),
]);
await expectOutsideUntouched(layout);
await fsp.writeFile(source, "source");
}
}, 15_000);
});

View File

@ -1,4 +1,5 @@
import fs from "node:fs/promises";
import { realpathSync } from "node:fs";
import os from "node:os";
import path from "node:path";
import { Readable } from "node:stream";
@ -37,6 +38,7 @@ import {
import { resolveLocalPathFromRootsSync } from "../src/local-roots.js";
import { movePathWithCopyFallback } from "../src/move-path.js";
import {
assertNoNulPathInput,
hasNodeErrorCode,
isNotFoundPathError,
isPathInside,
@ -207,7 +209,7 @@ describe("root handle coverage", () => {
await opened[Symbol.asyncDispose]();
await fs.mkdir(path.join(rootDir, "dir"));
await expect(scoped.openWritable("dir")).rejects.toMatchObject({ code: "EISDIR" });
await expect(scoped.openWritable("dir")).rejects.toMatchObject({ code: "not-file" });
await expect(scoped.openWritable("inside-hardlink.txt")).rejects.toMatchObject({
code: "path-alias",
});
@ -241,8 +243,11 @@ describe("path helpers", () => {
expect(isSymlinkOpenError(Object.assign(new Error("x"), { code: "ELOOP" }))).toBe(true);
expect(isPathInside(root, file)).toBe(true);
expect(resolveSafeBaseDir(root)).toBe(`${path.resolve(root)}${path.sep}`);
expect(safeRealpathSync(file, cache)).toBe(await fs.realpath(file));
expect(safeRealpathSync(file, cache)).toBe(await fs.realpath(file));
// Use the sync realpath to compare against safeRealpathSync. On windows
// fs.realpathSync and fs.realpath (async) sometimes disagree on 8.3
// short-name canonicalization (e.g. "RUNNER~1" vs "runneradmin").
expect(safeRealpathSync(file, cache)).toBe(realpathSync(file));
expect(safeRealpathSync(file, cache)).toBe(realpathSync(file));
expect(safeRealpathSync(path.join(root, "missing"), cache)).toBeNull();
expect(isPathInsideWithRealpath(root, file, { cache })).toBe(true);
expect(isPathInsideWithRealpath(root, path.join(root, "missing"), { requireRealpath: false }))
@ -250,6 +255,7 @@ describe("path helpers", () => {
expect(isPathInsideWithRealpath(root, path.join(root, "missing"))).toBe(false);
expect(safeStatSync(file)?.isFile()).toBe(true);
expect(safeStatSync(path.join(root, "missing"))).toBeNull();
expect(() => assertNoNulPathInput("a\0b")).toThrow("NUL");
expect(splitSafeRelativePath("./a//b")).toEqual(["a", "b"]);
for (const bad of ["../x", "/x", "C:\\x", "a\\b", "a\0b"]) {
expect(() => splitSafeRelativePath(bad)).toThrow();
@ -455,7 +461,7 @@ describe("URL, install, and local-root helpers", () => {
label: "media roots",
requireFile: true,
}),
).toMatchObject({ path: await fs.realpath(file) });
).toMatchObject({ path: realpathSync(file) });
expect(() =>
resolveLocalPathFromRootsSync({
filePath: "bad\0path",
@ -786,7 +792,7 @@ describe("temporary workspace and symlink parent helpers", () => {
});
describe("file stores and private stores", () => {
it("writes, streams, copies, reads, removes, and prunes file-store entries", async () => {
it.skipIf(process.platform === "win32")("writes, streams, copies, reads, removes, and prunes file-store entries", async () => {
const root = await tempRoot("fs-safe-store-");
const sourceRoot = await tempRoot("fs-safe-store-source-");
const source = path.join(sourceRoot, "source.txt");
@ -826,7 +832,7 @@ describe("file stores and private stores", () => {
await expect(fs.stat(old)).rejects.toMatchObject({ code: "ENOENT" });
});
it("covers private file store mode", async () => {
it.skipIf(process.platform === "win32")("covers private file store mode", async () => {
const root = await tempRoot("fs-safe-private-store-");
const store = fileStore({ rootDir: root, private: true });
@ -889,16 +895,13 @@ describe("secret files and temp roots", () => {
expect(resolveSecureTempRoot({ fallbackPrefix: "fallback", preferredDir: secure })).toBe(
path.resolve(secure),
);
expect(
resolveSecureTempRoot({
fallbackPrefix: "fallback",
preferredDir: secure,
skipPreferredOnWindows: true,
platform: "win32",
tmpdir: () => root,
getuid: () => undefined,
}),
).toBe(path.win32.join(root, "fallback"));
const winFallback = path.win32.join(root, "fallback");
const winFallbackStat = { isDirectory: () => true, isSymbolicLink: () => false };
expect(resolveSecureTempRoot({
accessSync: vi.fn(), chmodSync: vi.fn(), fallbackPrefix: "fallback",
getuid: () => undefined, lstatSync: vi.fn(() => winFallbackStat), mkdirSync: vi.fn(),
platform: "win32", preferredDir: secure, skipPreferredOnWindows: true, tmpdir: () => root,
})).toBe(winFallback);
await expect(withTimeout(Promise.resolve("ok"), 10, { message: "slow" })).resolves.toBe("ok");
await expect(withTimeout(new Promise(() => undefined), 1, { message: "slow" }))
.rejects

View File

@ -79,11 +79,13 @@ describe("archive extraction", () => {
const zip = new JSZip();
zip.file("package/hello.txt", "hi");
zip.file("package/my file.txt", "space");
await fs.writeFile(archivePath, await zip.generateAsync({ type: "nodebuffer" }));
await extractArchive({ archivePath, destDir, timeoutMs: 15_000 });
const packageDir = await resolvePackedRootDir(destDir);
await expect(fs.readFile(path.join(packageDir, "hello.txt"), "utf8")).resolves.toBe("hi");
await expect(fs.readFile(path.join(packageDir, "my file.txt"), "utf8")).resolves.toBe("space");
});
it("does not truncate existing destination files when zip extraction fails", async () => {
@ -169,6 +171,38 @@ describe("archive extraction", () => {
},
);
it.runIf(process.platform !== "win32")(
"does not cleanup through a swapped zip entry parent before commit",
async () => {
const root = await tempRoot("fs-safe-archive-cleanup-race-");
const archivePath = path.join(root, "pkg.zip");
const destDir = path.join(root, "dest");
const outsideDir = path.join(root, "outside");
const outsideFile = path.join(outsideDir, "payload.txt");
await fs.mkdir(destDir);
await fs.mkdir(outsideDir);
await fs.writeFile(outsideFile, "outside");
const zip = new JSZip();
zip.file("nested/payload.txt", "inside");
await fs.writeFile(archivePath, await zip.generateAsync({ type: "nodebuffer" }));
const realMkdir = fs.mkdir.bind(fs);
let swapped = false;
vi.spyOn(fs, "mkdir").mockImplementation(async (...args: Parameters<typeof fs.mkdir>) => {
const candidate = String(args[0]);
if (!swapped && path.basename(candidate) === "nested" && await fs.lstat(candidate).then(() => true, () => false)) {
swapped = true;
await fs.rename(candidate, path.join(path.dirname(candidate), "nested-real"));
await fs.symlink(outsideDir, candidate, "dir");
}
return await realMkdir(...args);
});
await expect(extractArchive({ archivePath, destDir, kind: "zip", timeoutMs: 15_000 }))
.rejects.toBeTruthy();
await expect(fs.readFile(outsideFile, "utf8")).resolves.toBe("outside");
},
);
it.runIf(process.platform !== "win32")(
"rejects zip extraction when a hardlink appears after write",
async () => {

View File

@ -121,9 +121,6 @@ describe("home directory helpers", () => {
expect(resolveHomeRelativePath("~/state", { env })).toBe(path.resolve("/configured/state"));
expect(resolveOsHomeRelativePath("~/state", { env })).toBe(path.resolve("/home/tester/state"));
expect(resolveUserPath("~/state", env)).toBe(path.resolve("/configured/state"));
expect(resolveUserPath(" ./relative ", { env })).toBe(path.resolve("./relative"));
expect(resolveHomeRelativePath(" ", { env })).toBe("");
expect(resolveOsHomeRelativePath(" ", { env })).toBe("");
});
it("ignores unusable home values", () => {
@ -242,6 +239,7 @@ describe("absolute path helpers", () => {
code: "symlink",
});
});
});
describe("filesystem utility helpers", () => {
@ -329,20 +327,24 @@ describe("sidecar lock manager", () => {
manager.reset();
});
it("times out and reclaims stale locks", async () => {
it("times out on stale locks without deleting them by path", async () => {
const root = await tempRoot("fs-safe-sidecar-timeout-");
const targetPath = path.join(root, "state.json");
const lockPath = `${targetPath}.lock`;
const manager = createSidecarLockManager(`coverage-timeout-${Date.now()}-${Math.random()}`);
await fs.writeFile(lockPath, "{\"createdAt\":\"2000-01-01T00:00:00.000Z\"}\n", "utf8");
const reclaimed = await manager.acquire({
targetPath,
lockPath,
staleMs: 1,
payload: () => ({ owner: "coverage" }),
});
await reclaimed.release();
await expect(
manager.acquire({
targetPath,
lockPath,
staleMs: 1,
timeoutMs: 1,
retry: { retries: 0, minTimeout: 1, maxTimeout: 1 },
payload: () => ({ owner: "coverage" }),
}),
).rejects.toMatchObject({ code: "file_lock_stale" });
await expect(fs.readFile(lockPath, "utf8")).resolves.toContain("2000");
await fs.writeFile(lockPath, "{\"createdAt\":\"2999-01-01T00:00:00.000Z\"}\n", "utf8");
await expect(

329
test/coverage-more.test.ts Normal file
View File

@ -0,0 +1,329 @@
import fsSync from "node:fs";
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { Readable } from "node:stream";
import { afterEach, describe, expect, it, vi } from "vitest";
import { createBoundedReadStream, createMaxBytesTransform } from "../src/bounded-read-stream.js";
import {
assertAsyncDirectoryGuard,
assertSyncDirectoryGuard,
createAsyncDirectoryGuard,
createNearestExistingDirectoryGuard,
createNearestExistingSyncDirectoryGuard,
createSyncDirectoryGuard,
} from "../src/directory-guard.js";
import { drainFileLockManagerForTest, resetFileLockManagerForTest } from "../src/file-lock.js";
import { sameFileIdentity } from "../src/file-identity.js";
import { readLocalFileFromRoots, resolveLocalPathFromRootsSync } from "../src/local-roots.js";
import { resolveSecureTempRoot, type ResolveSecureTempRootOptions } from "../src/secure-temp-dir.js";
import { writeSiblingTempFile, writeViaSiblingTempPath } from "../src/sibling-temp.js";
import {
buildRandomTempFilePath,
sanitizeTempFileName,
tempFile,
withTempFile,
} from "../src/temp-target.js";
type SecureDirStat = NonNullable<ResolveSecureTempRootOptions["lstatSync"]> extends (
path: string,
) => infer Result
? Result
: never;
const tempDirs = new Set<string>();
async function tempRoot(prefix: string): Promise<string> {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), prefix));
tempDirs.add(dir);
return dir;
}
function nodeError(code: string): Error & { code: string } {
return Object.assign(new Error(code), { code });
}
function dirStat(params?: {
isDirectory?: boolean;
isSymbolicLink?: boolean;
mode?: number;
uid?: number;
}): SecureDirStat {
return {
isDirectory: () => params?.isDirectory ?? true,
isSymbolicLink: () => params?.isSymbolicLink ?? false,
mode: params?.mode ?? 0o40700,
uid: params?.uid,
};
}
afterEach(async () => {
vi.restoreAllMocks();
for (const dir of tempDirs) {
await fs.rm(dir, { recursive: true, force: true });
}
tempDirs.clear();
});
describe("secure temp root fallback coverage", () => {
it("creates the uid-less fallback when no preferred directory is configured", () => {
const fallbackPath = path.join("/tmp", "fs-safe-test");
let created = false;
const resolved = resolveSecureTempRoot({
fallbackPrefix: "fs-safe-test",
getuid: () => undefined,
lstatSync: vi.fn((candidate: string) => {
if (candidate === fallbackPath && !created) {
throw nodeError("ENOENT");
}
return dirStat();
}),
mkdirSync: vi.fn((candidate: string) => {
if (candidate === fallbackPath) {
created = true;
}
}),
chmodSync: vi.fn(),
accessSync: vi.fn(),
tmpdir: () => "/tmp",
warn: vi.fn(),
});
expect(resolved).toBe(fallbackPath);
});
it("rejects an unsafe fallback directory that cannot be repaired", () => {
expect(() =>
resolveSecureTempRoot({
fallbackPrefix: "fs-safe-test",
getuid: () => 501,
lstatSync: vi.fn(() => dirStat({ isSymbolicLink: true, uid: 501 })),
mkdirSync: vi.fn(),
chmodSync: vi.fn(),
accessSync: vi.fn(),
tmpdir: () => "/tmp",
unsafeFallbackLabel: "test temp",
warn: vi.fn(),
}),
).toThrow("Unsafe fallback test temp");
});
it("accepts a fallback directory after a chmod-denied recheck proves it safe", () => {
const fallbackPath = path.join("/tmp", "fs-safe-test-501");
let calls = 0;
const resolved = resolveSecureTempRoot({
fallbackPrefix: "fs-safe-test",
getuid: () => 501,
lstatSync: vi.fn((candidate: string) => {
expect(candidate).toBe(fallbackPath);
calls += 1;
return dirStat({ mode: calls < 3 ? 0o40777 : 0o40700, uid: 501 });
}),
mkdirSync: vi.fn(),
chmodSync: vi.fn(() => {
throw nodeError("EPERM");
}),
accessSync: vi.fn(),
tmpdir: () => "/tmp",
warn: vi.fn(),
});
expect(resolved).toBe(fallbackPath);
});
});
describe("small identity and lock wrappers", () => {
it("compares file identities across POSIX and Windows zero-device stats", async () => {
expect(sameFileIdentity({ dev: 1, ino: 2 }, { dev: 1, ino: 2 }, "linux")).toBe(true);
expect(sameFileIdentity({ dev: 1, ino: 2 }, { dev: 1, ino: 3 }, "linux")).toBe(false);
expect(sameFileIdentity({ dev: 0, ino: 2 }, { dev: 99, ino: 2 }, "win32")).toBe(true);
expect(sameFileIdentity({ dev: 0n, ino: 2n }, { dev: 99n, ino: 2n }, "linux")).toBe(false);
const root = await tempRoot("fs-safe-file-lock-wrapper-");
const targetPath = path.join(root, "state.json");
await drainFileLockManagerForTest(targetPath, "coverage-lock-wrapper");
resetFileLockManagerForTest(targetPath, "coverage-lock-wrapper");
});
});
describe("bounded streams and directory guard coverage", () => {
it("returns raw streams without limits and rejects oversized limited streams", async () => {
const raw = Readable.from(["ok"]);
const returned = createBoundedReadStream({ handle: { createReadStream: () => raw } }, undefined);
expect(returned).toBe(raw);
await expect(async () => {
for await (const _chunk of Readable.from(["ab", "cd"]).pipe(createMaxBytesTransform(3))) {
// Drain the stream so transform errors surface.
}
}).rejects.toMatchObject({ code: "too-large" });
});
it("detects changed or invalid directory guards", async () => {
const root = await tempRoot("fs-safe-dir-guard-more-");
const nested = path.join(root, "nested");
const filePath = path.join(root, "file.txt");
await fs.mkdir(nested);
await fs.writeFile(filePath, "not a dir", "utf8");
await expect(createAsyncDirectoryGuard(filePath)).rejects.toMatchObject({ code: "not-file" });
expect(() => createSyncDirectoryGuard(filePath)).toThrow("directory component");
const asyncGuard = await createAsyncDirectoryGuard(nested);
const syncGuard = createSyncDirectoryGuard(nested);
await expect(assertAsyncDirectoryGuard({ ...asyncGuard, realPath: root })).rejects.toMatchObject({
code: "path-mismatch",
});
expect(() => assertSyncDirectoryGuard({ ...syncGuard, realPath: root })).toThrow(
"directory changed",
);
await fs.rm(nested, { recursive: true });
await fs.writeFile(nested, "not a dir", "utf8");
await expect(assertAsyncDirectoryGuard(asyncGuard)).rejects.toMatchObject({
code: "not-file",
});
expect(() => assertSyncDirectoryGuard(syncGuard)).toThrow("directory component");
const nearest = await createNearestExistingDirectoryGuard(root, path.join(root, "missing", "x"));
expect(nearest.dir).toBe(root);
expect(createNearestExistingSyncDirectoryGuard(root, path.join(root, "missing", "x")).dir)
.toBe(root);
});
});
describe("sibling temp coverage", () => {
it("syncs temp files and parent dirs when requested", async () => {
const root = await tempRoot("fs-safe-sibling-more-");
const result = await writeSiblingTempFile({
dir: root,
mode: 0o600,
syncParentDir: true,
syncTempFile: true,
tempPrefix: ".coverage",
writeTemp: async (tempPath) => {
await fs.writeFile(tempPath, "synced", "utf8");
return { name: "final.txt" };
},
resolveFinalPath: ({ name }) => path.join(root, name),
});
expect(result.filePath).toBe(path.join(root, "final.txt"));
await expect(fs.readFile(result.filePath, "utf8")).resolves.toBe("synced");
if (process.platform !== "win32") {
// POSIX file modes don't fully apply on Windows.
expect((await fs.stat(result.filePath)).mode & 0o777).toBe(0o600);
}
});
it("removes sibling temp files when copy-in rejects the staged source", async () => {
const root = await tempRoot("fs-safe-sibling-copyin-");
const outside = await tempRoot("fs-safe-sibling-copyin-outside-");
const targetPath = path.join(root, "nested", "out.txt");
let stagedPath = "";
await fs.mkdir(path.dirname(targetPath), { recursive: true });
await expect(
writeViaSiblingTempPath({
rootDir: root,
targetPath,
writeTemp: async (candidate) => {
stagedPath = candidate;
await fs.writeFile(candidate, "bad", "utf8");
await fs.symlink(outside, path.dirname(targetPath), "dir");
},
}),
).rejects.toBeTruthy();
await expect(fs.stat(stagedPath)).rejects.toMatchObject({ code: "ENOENT" });
await expect(fs.readdir(outside)).resolves.toEqual([]);
});
});
describe("temp target edge coverage", () => {
it("normalizes empty temp names, extensions, timestamps, and cleanup on thrown callbacks", async () => {
const root = await tempRoot("fs-safe-temp-more-");
expect(sanitizeTempFileName("???")).toBe("download.bin");
const built = buildRandomTempFilePath({
rootDir: root,
prefix: "!!!",
extension: "._-",
now: Number.NaN,
uuid: "id",
});
expect(path.dirname(built)).toBe(root);
expect(path.basename(built)).toMatch(/^tmp-\d+-id$/);
const tmp = await tempFile({ rootDir: root, prefix: "???", fileName: "???" });
expect(path.basename(tmp.dir)).toMatch(/^tmp-/);
expect(path.basename(tmp.path)).toBe("download.bin");
await tmp.cleanup();
let scopedPath = "";
await expect(
withTempFile({ rootDir: root, prefix: "throwing", fileName: "x.txt" }, async (tmpPath) => {
scopedPath = tmpPath;
await fs.writeFile(tmpPath, "cleanup", "utf8");
throw new Error("boom");
}),
).rejects.toThrow("boom");
await expect(fs.stat(path.dirname(scopedPath))).rejects.toMatchObject({ code: "ENOENT" });
});
});
describe("local roots edge coverage", () => {
it("rejects invalid root entries and file URL inputs", async () => {
const root = await tempRoot("fs-safe-local-roots-more-");
await expect(async () =>
resolveLocalPathFromRootsSync({ filePath: root, roots: [" "], label: "media roots" }),
).rejects.toThrow("media roots entry is required");
expect(() =>
resolveLocalPathFromRootsSync({ filePath: root, roots: ["relative"], label: "media roots" }),
).toThrow("absolute paths");
expect(() =>
resolveLocalPathFromRootsSync({
filePath: root,
roots: [`${root}\0bad`],
label: "media roots",
}),
).toThrow("NUL");
expect(() =>
resolveLocalPathFromRootsSync({
filePath: "file://remote.example/path.txt",
roots: [root],
}),
).toThrow("Invalid file:// URL");
});
it("skips unusable roots and non-file candidates", async () => {
const root = await tempRoot("fs-safe-local-roots-valid-");
const missingRoot = path.join(root, "missing-root");
const realRoot = path.join(root, "real-root");
const childDir = path.join(realRoot, "child");
const filePath = path.join(realRoot, "ok.txt");
await fs.mkdir(childDir, { recursive: true });
await fs.writeFile(filePath, "ok", "utf8");
expect(
resolveLocalPathFromRootsSync({
filePath,
roots: [missingRoot, realRoot],
requireFile: true,
}),
).toMatchObject({ path: fsSync.realpathSync(filePath) });
expect(
resolveLocalPathFromRootsSync({
filePath: childDir,
roots: [realRoot],
requireFile: true,
}),
).toBeNull();
await expect(
readLocalFileFromRoots({ filePath: realRoot, roots: [missingRoot, realRoot] }),
).resolves.toBeNull();
});
});

View File

@ -0,0 +1,158 @@
import fsp from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import { fileStore } from "../src/file-store.js";
import { loadPendingJsonDurableQueueEntries } from "../src/json-durable-queue.js";
import { readLocalFileFromRoots, resolveLocalPathFromRootsSync } from "../src/local-roots.js";
import { replaceFileAtomic } from "../src/replace-file.js";
import { writeViaSiblingTempPath } from "../src/sibling-temp.js";
import { buildRandomTempFilePath } from "../src/temp-target.js";
const tempDirs: string[] = [];
async function tempRoot(prefix: string): Promise<string> {
const dir = await fsp.mkdtemp(path.join(os.tmpdir(), prefix));
tempDirs.push(dir);
return dir;
}
afterEach(async () => {
vi.restoreAllMocks();
await Promise.all(tempDirs.splice(0).map((dir) => fsp.rm(dir, { recursive: true, force: true })));
});
describe("deepsec regressions", () => {
it("keeps caller-provided temp tokens as single path segments", async () => {
const base = await tempRoot("fs-safe-temp-token-");
const target = path.join(base, "out.txt");
expect(() =>
buildRandomTempFilePath({ rootDir: base, prefix: "tmp", uuid: "../escape" }),
).toThrow();
await expect(
replaceFileAtomic({ filePath: target, content: "x", tempPrefix: "../escape" }),
).rejects.toThrow();
await expect(
writeViaSiblingTempPath({
rootDir: base,
targetPath: target,
tempPrefix: "../escape",
writeTemp: async (tempPath) => {
await fsp.writeFile(tempPath, "x");
},
}),
).rejects.toThrow();
await expect(fsp.stat(path.join(path.dirname(base), "escape"))).rejects.toMatchObject({
code: "ENOENT",
});
await expect(
writeViaSiblingTempPath({
rootDir: base,
targetPath: target,
tempPrefix: ".derived file prefix",
writeTemp: async (tempPath) => {
await fsp.writeFile(tempPath, "ok");
},
}),
).resolves.toBeUndefined();
await expect(fsp.readFile(target, "utf8")).resolves.toBe("ok");
});
it.runIf(process.platform !== "win32")(
"does not treat dangling symlinks as safe missing local-root paths",
async () => {
const base = await tempRoot("fs-safe-local-roots-");
const outside = await tempRoot("fs-safe-local-roots-outside-");
const linkPath = path.join(base, "dangling");
await fsp.symlink(path.join(outside, "missing.txt"), linkPath, "file");
expect(
resolveLocalPathFromRootsSync({
filePath: linkPath,
roots: [base],
allowMissing: true,
}),
).toBeNull();
},
);
it("preserves Root's default read cap for local-root reads", async () => {
const base = await tempRoot("fs-safe-local-root-cap-");
const filePath = path.join(base, "large.bin");
await fsp.writeFile(filePath, Buffer.alloc(16 * 1024 * 1024 + 1));
await expect(readLocalFileFromRoots({ filePath, roots: [base] })).resolves.toBeNull();
const uncapped = await readLocalFileFromRoots({
filePath,
roots: [base],
maxBytes: 16 * 1024 * 1024 + 1,
});
expect(uncapped?.buffer.byteLength).toBe(16 * 1024 * 1024 + 1);
});
it.runIf(process.platform !== "win32")("pins private copyIn sources after validation", async () => {
const base = await tempRoot("fs-safe-private-copyin-");
const sourceDir = await tempRoot("fs-safe-private-copyin-source-");
const outside = await tempRoot("fs-safe-private-copyin-outside-");
const sourcePath = path.join(sourceDir, "upload.txt");
const outsideFile = path.join(outside, "secret.txt");
await fsp.writeFile(sourcePath, "upload");
await fsp.writeFile(outsideFile, "secret");
const originalLstat = fsp.lstat;
let swapped = false;
vi.spyOn(fsp, "lstat").mockImplementation(async (candidate, options) => {
const stat = await originalLstat(candidate, options as never);
if (!swapped && candidate === sourcePath) {
swapped = true;
await fsp.rm(sourcePath);
await fsp.symlink(outsideFile, sourcePath, "file");
}
return stat;
});
const store = fileStore({ rootDir: base, private: true });
await expect(store.copyIn("copied.txt", sourcePath)).rejects.toBeTruthy();
await expect(fsp.stat(path.join(base, "copied.txt"))).rejects.toMatchObject({ code: "ENOENT" });
});
it.runIf(process.platform !== "win32")("preserves private copyIn source error codes", async () => {
const base = await tempRoot("fs-safe-private-copyin-codes-");
const sourceDir = await tempRoot("fs-safe-private-copyin-codes-source-");
const source = path.join(sourceDir, "source.txt");
const link = path.join(sourceDir, "source-link.txt");
await fsp.writeFile(source, "1234567890");
await fsp.symlink(source, link, "file");
const store = fileStore({ rootDir: base, private: true });
await expect(store.copyIn("dir.txt", sourceDir)).rejects.toMatchObject({ code: "not-file" });
await expect(store.copyIn("link.txt", link)).rejects.toMatchObject({ code: "not-file" });
await expect(store.copyIn("large.txt", source, { maxBytes: 4 })).rejects.toMatchObject({
code: "too-large",
});
});
it.runIf(process.platform !== "win32")("skips symlinked durable queue entries", async () => {
const base = await tempRoot("fs-safe-queue-symlink-");
const queueDir = path.join(base, "queue");
const outside = await tempRoot("fs-safe-queue-outside-");
await fsp.mkdir(queueDir);
await fsp.writeFile(path.join(outside, "outside.json"), JSON.stringify({ leaked: true }));
await fsp.symlink(path.join(outside, "outside.json"), path.join(queueDir, "leak.json"), "file");
await expect(
loadPendingJsonDurableQueueEntries<{ leaked: boolean }>({ queueDir, tempPrefix: "queue" }),
).resolves.toEqual([]);
});
it("rejects oversized durable queue entries before parsing", async () => {
const base = await tempRoot("fs-safe-queue-size-");
const queueDir = path.join(base, "queue");
await fsp.mkdir(queueDir);
await fsp.writeFile(path.join(queueDir, "large.json"), JSON.stringify({ data: "0123456789" }));
await expect(
loadPendingJsonDurableQueueEntries({ queueDir, tempPrefix: "queue", maxBytes: 4 }),
).resolves.toEqual([]);
});
});

249
test/edge-coverage.test.ts Normal file
View File

@ -0,0 +1,249 @@
import fsSync from "node:fs";
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { Readable } from "node:stream";
import { afterEach, describe, expect, it, vi } from "vitest";
import { FsSafeError } from "../src/errors.js";
import {
assertSyncDirectoryGuard,
ensureParentSync,
writeStreamToTempSource,
} from "../src/file-store-boundary.js";
import {
assertCanonicalPathWithinBase,
resolveSafeInstallDir,
safePathSegmentHashed,
} from "../src/install-path.js";
import { replaceDirectoryAtomic } from "../src/replace-directory.js";
import {
isAlreadyExistsError,
normalizePinnedPathError,
normalizePinnedWriteError,
} from "../src/root-errors.js";
import { movePathToTrash } from "../src/trash.js";
const tempDirs = new Set<string>();
async function tempRoot(prefix: string): Promise<string> {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), prefix));
tempDirs.add(dir);
return dir;
}
afterEach(async () => {
vi.restoreAllMocks();
for (const dir of tempDirs) {
await fs.rm(dir, { recursive: true, force: true });
}
tempDirs.clear();
});
describe("root error helpers", () => {
it("normalizes existing and unknown low-level errors", () => {
const existsError = Object.assign(new Error("File exists"), { code: "EEXIST" });
expect(isAlreadyExistsError(existsError)).toBe(true);
expect(isAlreadyExistsError("EEXIST: File exists")).toBe(true);
expect(isAlreadyExistsError(new Error("different"))).toBe(false);
const fsSafe = new FsSafeError("not-file", "already normalized");
expect(normalizePinnedWriteError(fsSafe)).toBe(fsSafe);
expect(normalizePinnedPathError(fsSafe)).toBe(fsSafe);
expect(normalizePinnedWriteError(new Error("raw"))).toMatchObject({
code: "invalid-path",
message: "path is not a regular file under root",
});
expect(normalizePinnedWriteError("raw string")).toMatchObject({ code: "invalid-path" });
expect(normalizePinnedPathError(new Error("raw"))).toMatchObject({
code: "path-alias",
message: "path is not under root",
});
expect(normalizePinnedPathError("raw string")).toMatchObject({ code: "path-alias" });
});
});
describe("directory replacement and file store boundary helpers", () => {
it("rolls back directory replacement when the staged rename fails", async () => {
const root = await tempRoot("fs-safe-replace-dir-");
const target = path.join(root, "target");
const staged = path.join(root, "staged");
await fs.mkdir(target);
await fs.writeFile(path.join(target, "old.txt"), "old", "utf8");
await fs.mkdir(staged);
await fs.writeFile(path.join(staged, "new.txt"), "new", "utf8");
const realRename = fs.rename.bind(fs);
vi.spyOn(fs, "rename").mockImplementation(async (from, to) => {
if (from === staged && to === target) {
throw Object.assign(new Error("boom"), { code: "EACCES" });
}
return await realRename(from, to);
});
await expect(replaceDirectoryAtomic({ stagedDir: staged, targetDir: target })).rejects
.toMatchObject({ code: "EACCES" });
await expect(fs.readFile(path.join(target, "old.txt"), "utf8")).resolves.toBe("old");
await expect(fs.readFile(path.join(staged, "new.txt"), "utf8")).resolves.toBe("new");
});
it("guards sync parents and rejects escapes or swapped directories", async () => {
const root = await tempRoot("fs-safe-store-boundary-");
const guard = ensureParentSync({
rootDir: root,
filePath: path.join(root, "nested", "file.txt"),
mode: 0o700,
});
expect(path.basename(guard.dir)).toBe("nested");
expect(() => assertSyncDirectoryGuard(guard)).not.toThrow();
expect(() => assertSyncDirectoryGuard({ ...guard, realPath: path.join(root, "other") }))
.toThrow("changed during write");
expect(() =>
ensureParentSync({
rootDir: root,
filePath: path.join(path.dirname(root), "outside.txt"),
mode: 0o700,
}),
).toThrow("escapes store root");
const badRoot = await tempRoot("fs-safe-store-boundary-bad-");
await fs.writeFile(path.join(badRoot, "file-parent"), "not a dir", "utf8");
expect(() =>
ensureParentSync({
rootDir: badRoot,
filePath: path.join(badRoot, "file-parent", "child.txt"),
mode: 0o700,
}),
).toThrow("must be a directory");
});
it("stages streams and cleans failed temp sources", async () => {
const staged = await writeStreamToTempSource({
stream: Readable.from(["hello"]),
mode: 0o600,
});
try {
await expect(fs.readFile(staged.path, "utf8")).resolves.toBe("hello");
} finally {
await staged.cleanup();
}
await expect(fs.stat(path.dirname(staged.path))).rejects.toMatchObject({ code: "ENOENT" });
await expect(
writeStreamToTempSource({
stream: Readable.from(["123", "456"]),
maxBytes: 4,
mode: 0o600,
}),
).rejects.toMatchObject({ code: "too-large" });
});
});
describe("install path edge paths", () => {
it("covers hashed segment fallbacks and canonical base failures", async () => {
expect(safePathSegmentHashed(".")).toMatch(/^skill-[a-f0-9]{10}$/);
expect(safePathSegmentHashed("!!!")).toMatch(/^skill-[a-f0-9]{10}$/);
expect(safePathSegmentHashed("ok-name")).toBe("ok-name");
expect(
resolveSafeInstallDir({
baseDir: "/tmp/plugins",
id: "same",
invalidNameMessage: "bad",
nameEncoder: () => "",
}),
).toEqual({ ok: false, error: "bad" });
const root = await tempRoot("fs-safe-install-edge-");
const realBase = path.join(root, "real-base");
const linkBase = path.join(root, "link-base");
await fs.mkdir(realBase);
await fs.symlink(realBase, linkBase, "dir");
await expect(
assertCanonicalPathWithinBase({
baseDir: linkBase,
candidatePath: path.join(linkBase, "future.txt"),
boundaryLabel: "install root",
}),
).resolves.toBeUndefined();
const baseFile = path.join(root, "base-file");
await fs.writeFile(baseFile, "not a directory", "utf8");
await expect(
assertCanonicalPathWithinBase({
baseDir: baseFile,
candidatePath: path.join(baseFile, "future.txt"),
boundaryLabel: "install root",
}),
).rejects.toThrow("base directory");
const outside = await tempRoot("fs-safe-install-edge-outside-");
await fs.symlink(outside, path.join(realBase, "outside-link"), "dir");
await expect(
assertCanonicalPathWithinBase({
baseDir: realBase,
candidatePath: path.join(realBase, "outside-link", "future.txt"),
boundaryLabel: "install root",
}),
).rejects.toThrow("within");
});
});
describe("trash edge paths", () => {
it("refuses root paths, retries name collisions, and falls back across devices", async () => {
const root = await tempRoot("fs-safe-trash-extra-");
const filePath = path.join(root, "retry.txt");
await fs.writeFile(filePath, "trash", "utf8");
await expect(movePathToTrash(path.parse(root).root, { allowedRoots: [root] }))
.rejects
.toThrow("Refusing to trash root path");
const realRename = fsSync.renameSync.bind(fsSync);
let collision = true;
vi.spyOn(fsSync, "renameSync").mockImplementation((from, to) => {
if (from === filePath && collision) {
collision = false;
throw Object.assign(new Error("exists"), { code: "EEXIST" });
}
return realRename(from, to);
});
const retriedDest = await movePathToTrash(filePath, { allowedRoots: [root] });
try {
expect(path.basename(retriedDest)).toBe("retry.txt");
} finally {
await fs.rm(path.dirname(retriedDest), { recursive: true, force: true });
}
vi.restoreAllMocks();
const crossDevice = path.join(root, "cross-device.txt");
await fs.writeFile(crossDevice, "copy fallback", "utf8");
vi.spyOn(fsSync, "renameSync").mockImplementation((from, to) => {
if (from === crossDevice) {
throw Object.assign(new Error("cross-device"), { code: "EXDEV" });
}
return realRename(from, to);
});
const copiedDest = await movePathToTrash(crossDevice, { allowedRoots: [root] });
try {
expect(fsSync.readFileSync(copiedDest, "utf8")).toBe("copy fallback");
expect(fsSync.existsSync(crossDevice)).toBe(false);
} finally {
await fs.rm(path.dirname(copiedDest), { recursive: true, force: true });
}
});
it.runIf(process.platform !== "win32")("moves broken symlinks to trash", async () => {
const root = await tempRoot("fs-safe-trash-broken-link-");
const linkPath = path.join(root, "broken-link");
const missingTarget = path.join(root, "missing-target");
await fs.symlink(missingTarget, linkPath);
const dest = await movePathToTrash(linkPath, { allowedRoots: [root] });
try {
// Broken links cannot be realpathed; the guard keeps lstat identity and
// renames the link itself instead of requiring the target to exist.
await expect(fs.readlink(dest)).resolves.toBe(missingTarget);
await expect(fs.lstat(linkPath)).rejects.toMatchObject({ code: "ENOENT" });
} finally {
await fs.rm(path.dirname(dest), { recursive: true, force: true });
}
});
});

View File

@ -1,5 +1,4 @@
import fs from "node:fs/promises";
import { fileURLToPath } from "node:url";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
@ -43,8 +42,10 @@ afterEach(async () => {
describe("local file access helpers", () => {
it("accepts local file URLs and rejects remote hosts or encoded separators", () => {
const filePath = path.join(path.sep, "tmp", "demo.txt");
expect(safeFileURLToPath(new URL(`file://${filePath}`).href)).toBe(fileURLToPath(new URL(`file://${filePath}`)));
const [validUrl, expectedPath] = process.platform === "win32"
? ["file:///C:/tmp/demo.txt", "C:\\tmp\\demo.txt"]
: ["file:///tmp/demo.txt", "/tmp/demo.txt"];
expect(safeFileURLToPath(validUrl)).toBe(expectedPath);
expect(() => safeFileURLToPath("file://example.com/tmp/demo.txt")).toThrow(/remote hosts/);
expect(() => safeFileURLToPath("file:///tmp/a%2Fb.txt")).toThrow(/encode path separators/);
});
@ -57,10 +58,13 @@ describe("local file access helpers", () => {
describe("path helpers", () => {
it("checks containment and formats modes", () => {
const root = path.join(path.sep, "tmp", "root");
// Use path.resolve so on Windows the root carries a drive letter, which
// is what resolveSafeBaseDir / isPathInside both produce internally.
const root = path.resolve(path.sep, "tmp", "root");
const otherRoot = path.resolve(path.sep, "tmp", "root-other");
expect(resolveSafeBaseDir(root)).toBe(`${root}${path.sep}`);
expect(isWithinDir(root, path.join(root, "file.txt"))).toBe(true);
expect(isPathInside(root, path.join(path.sep, "tmp", "root-other"))).toBe(false);
expect(isPathInside(root, otherRoot)).toBe(false);
expect(formatPosixMode(0o100755)).toBe("755");
});
});

View File

@ -0,0 +1,42 @@
import fsSync from "node:fs";
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { fileStoreSync } from "../src/file-store.js";
const tempDirs: string[] = [];
async function tempRoot(prefix: string): Promise<string> {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), prefix));
tempDirs.push(dir);
return dir;
}
afterEach(async () => {
await Promise.all(tempDirs.splice(0).map((dir) => fs.rm(dir, { recursive: true, force: true })));
});
describe("sync file-store read validation failures", () => {
it("surfaces directories as filesystem safety errors", async () => {
const root = await tempRoot("fs-safe-sync-store-validation-");
await fs.mkdir(path.join(root, "not-a-file"));
const store = fileStoreSync({ rootDir: root, private: true });
expect(() => store.readTextIfExists("not-a-file")).toThrow(
expect.objectContaining({ code: "path-mismatch" }),
);
});
it.runIf(process.platform !== "win32")("surfaces hardlinks as filesystem safety errors", async () => {
const root = await tempRoot("fs-safe-sync-store-hardlink-");
const filePath = path.join(root, "value.txt");
await fs.writeFile(filePath, "secret");
fsSync.linkSync(filePath, path.join(root, "alias.txt"));
const store = fileStoreSync({ rootDir: root, private: true });
expect(() => store.readTextIfExists("value.txt")).toThrow(
expect.objectContaining({ code: "path-mismatch" }),
);
});
});

View File

@ -0,0 +1,478 @@
import fsSync from "node:fs";
import fsp from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import JSZip from "jszip";
import { afterEach, describe, expect, it, vi } from "vitest";
import { extractArchive } from "../src/archive.js";
import { configureFsSafePython, root as openRoot } from "../src/index.js";
import { prepareArchiveDestinationDir, prepareArchiveOutputPath, mergeExtractedTreeIntoDestination } from "../src/archive-staging.js";
import { fileStore, fileStoreSync } from "../src/file-store.js";
import { writeJsonSync } from "../src/json.js";
import {
moveJsonDurableQueueEntryToFailed,
resolveJsonDurableQueueEntryPaths,
} from "../src/json-durable-queue.js";
import { movePathWithCopyFallback } from "../src/move-path.js";
import { runPinnedWriteHelper } from "../src/pinned-write.js";
import { replaceFileAtomic } from "../src/replace-file.js";
import { writeViaSiblingTempPath } from "../src/sibling-temp.js";
import { sanitizeTempFileName, tempFile } from "../src/temp-target.js";
import { tempWorkspace, tempWorkspaceSync } from "../src/private-temp-workspace.js";
import { __setFsSafeTestHooksForTest } from "../src/test-hooks.js";
import { movePathToTrash } from "../src/trash.js";
const tempDirs: string[] = [];
async function tempRoot(prefix: string): Promise<string> {
const dir = await fsp.mkdtemp(path.join(os.tmpdir(), prefix));
tempDirs.push(dir);
return dir;
}
async function writeOldFile(filePath: string, content = "old"): Promise<void> {
await fsp.writeFile(filePath, content);
const old = new Date(Date.now() - 60_000);
await fsp.utimes(filePath, old, old);
}
afterEach(async () => {
vi.restoreAllMocks();
configureFsSafePython({ mode: "auto", pythonPath: undefined });
__setFsSafeTestHooksForTest(undefined);
await Promise.all(tempDirs.splice(0).map((dir) => fsp.rm(dir, { recursive: true, force: true })));
});
describe("security finding regressions", () => {
it.runIf(process.platform !== "win32")("guards Root fallback mutators against parent swaps", async () => {
configureFsSafePython({ mode: "off" });
const base = await tempRoot("fs-safe-root-fallback-race-");
const outside = await tempRoot("fs-safe-root-fallback-outside-");
await fsp.mkdir(path.join(base, "nested"));
await fsp.writeFile(path.join(base, "nested", "delete.txt"), "inside");
await fsp.writeFile(path.join(base, "from.txt"), "move");
await fsp.writeFile(path.join(outside, "delete.txt"), "outside");
const scoped = await openRoot(base);
__setFsSafeTestHooksForTest({
async beforeRootFallbackMutation(operation) {
if (operation !== "remove") return;
await fsp.rename(path.join(base, "nested"), path.join(base, "nested-real"));
await fsp.symlink(outside, path.join(base, "nested"), "dir");
},
});
await expect(scoped.remove("nested/delete.txt")).rejects.toBeTruthy();
await expect(fsp.readFile(path.join(outside, "delete.txt"), "utf8")).resolves.toBe("outside");
await fsp.rm(path.join(base, "nested"));
await fsp.rename(path.join(base, "nested-real"), path.join(base, "nested"));
__setFsSafeTestHooksForTest({
async beforeRootFallbackMutation(operation) {
if (operation !== "mkdir") return;
await fsp.rename(path.join(base, "nested"), path.join(base, "nested-real"));
await fsp.symlink(outside, path.join(base, "nested"), "dir");
},
});
await expect(scoped.mkdir("nested/created")).rejects.toBeTruthy();
await expect(fsp.stat(path.join(outside, "created"))).rejects.toMatchObject({ code: "ENOENT" });
await fsp.rm(path.join(base, "nested"));
await fsp.rename(path.join(base, "nested-real"), path.join(base, "nested"));
__setFsSafeTestHooksForTest({
async beforeRootFallbackMutation(operation) {
if (operation !== "move") return;
await fsp.rename(path.join(base, "nested"), path.join(base, "nested-real"));
await fsp.symlink(outside, path.join(base, "nested"), "dir");
},
});
await expect(scoped.move("from.txt", "nested/moved.txt")).rejects.toBeTruthy();
await expect(fsp.stat(path.join(outside, "moved.txt"))).rejects.toMatchObject({ code: "ENOENT" });
});
it.runIf(process.platform !== "win32")("does not create archive directories through a swapped destination", async () => {
const base = await tempRoot("fs-safe-archive-dir-race-");
const dest = path.join(base, "dest");
const outside = await tempRoot("fs-safe-archive-dir-outside-");
await fsp.mkdir(dest);
const destinationRealDir = await prepareArchiveDestinationDir(dest);
__setFsSafeTestHooksForTest({
async beforeArchiveOutputMutation(operation) {
if (operation !== "mkdir") return;
await fsp.rename(dest, path.join(base, "dest-real"));
await fsp.symlink(outside, dest, "dir");
},
});
await expect(
prepareArchiveOutputPath({
destinationDir: dest,
destinationRealDir,
relPath: "nested/payload.txt",
outPath: path.join(dest, "nested", "payload.txt"),
originalPath: "nested/payload.txt",
isDirectory: false,
}),
).rejects.toBeTruthy();
await expect(fsp.stat(path.join(outside, "nested"))).rejects.toMatchObject({ code: "ENOENT" });
});
it.runIf(process.platform !== "win32")("does not chmod through an archive entry symlink swap", async () => {
const base = await tempRoot("fs-safe-archive-chmod-race-");
const source = path.join(base, "source");
const dest = path.join(base, "dest");
const outside = await tempRoot("fs-safe-archive-chmod-outside-");
const outsideFile = path.join(outside, "outside.txt");
await fsp.mkdir(source);
await fsp.mkdir(dest);
await fsp.writeFile(path.join(source, "payload.txt"), "payload");
await fsp.writeFile(outsideFile, "outside");
await fsp.chmod(outsideFile, 0o600);
const destinationRealDir = await prepareArchiveDestinationDir(dest);
__setFsSafeTestHooksForTest({
async beforeArchiveOutputMutation(operation, targetPath) {
if (operation !== "chmod" || !targetPath.endsWith("payload.txt")) return;
await fsp.rm(targetPath, { force: true });
await fsp.symlink(outsideFile, targetPath, "file");
},
});
await expect(
mergeExtractedTreeIntoDestination({ sourceDir: source, destinationDir: dest, destinationRealDir }),
).rejects.toBeTruthy();
expect((await fsp.stat(outsideFile)).mode & 0o777).toBe(0o600);
});
it.runIf(process.platform !== "win32")("uses unguessable no-follow temp files in pinned write fallback", async () => {
configureFsSafePython({ mode: "off" });
const base = await tempRoot("fs-safe-pinned-write-fallback-");
const outside = await tempRoot("fs-safe-pinned-write-outside-");
const outsideFile = path.join(outside, "outside.txt");
await fsp.writeFile(outsideFile, "outside");
await fsp.symlink(outsideFile, path.join(base, ".victim.txt.fallback.tmp"), "file");
await runPinnedWriteHelper({
rootPath: base,
relativeParentPath: "",
basename: "victim.txt",
mkdir: true,
mode: 0o600,
overwrite: true,
input: { kind: "buffer", data: "safe" },
});
await expect(fsp.readFile(path.join(base, "victim.txt"), "utf8")).resolves.toBe("safe");
await expect(fsp.readFile(outsideFile, "utf8")).resolves.toBe("outside");
});
it("validates pinned write fallback payloads even when Python mode is off", async () => {
configureFsSafePython({ mode: "off" });
const base = await tempRoot("fs-safe-pinned-write-validation-");
await expect(
runPinnedWriteHelper({
rootPath: base,
relativeParentPath: "../escape",
basename: "victim.txt",
mkdir: true,
mode: 0o600,
overwrite: true,
input: { kind: "buffer", data: "bad" },
}),
).rejects.toBeTruthy();
});
it.runIf(process.platform !== "win32")("guards private sync store writes against parent swaps", async () => {
const base = await tempRoot("fs-safe-sync-private-write-");
const outside = await tempRoot("fs-safe-sync-private-outside-");
await fsp.mkdir(path.join(base, "nested"));
const store = fileStoreSync({ rootDir: base, private: true });
__setFsSafeTestHooksForTest({
beforeFileStoreSyncPrivateWrite() {
fsSync.renameSync(path.join(base, "nested"), path.join(base, "nested-real"));
fsSync.symlinkSync(outside, path.join(base, "nested"), "dir");
},
});
expect(() => store.writeText("nested/value.txt", "secret")).toThrow();
await expect(fsp.stat(path.join(outside, "value.txt"))).rejects.toMatchObject({ code: "ENOENT" });
});
it.runIf(process.platform !== "win32")("pins sync store reads against final symlink swaps", async () => {
const base = await tempRoot("fs-safe-sync-read-race-");
const outside = await tempRoot("fs-safe-sync-read-outside-");
const filePath = path.join(base, "value.txt");
const outsideFile = path.join(outside, "outside.txt");
await fsp.writeFile(filePath, "inside");
await fsp.writeFile(outsideFile, "outside");
const originalReadFileSync = fsSync.readFileSync;
let swapped = false;
vi.spyOn(fsSync, "readFileSync").mockImplementation((target, options) => {
if (!swapped && typeof target === "number") {
swapped = true;
fsSync.rmSync(filePath);
fsSync.symlinkSync(outsideFile, filePath, "file");
}
return originalReadFileSync.call(fsSync, target, options as never);
});
const store = fileStoreSync({ rootDir: base, private: true });
expect(store.readTextIfExists("value.txt")).toBe("inside");
expect(fsSync.readFileSync(outsideFile, "utf8")).toBe("outside");
});
it.runIf(process.platform !== "win32")("does not recurse prune through a swapped directory symlink", async () => {
const base = await tempRoot("fs-safe-prune-race-");
const outside = await tempRoot("fs-safe-prune-outside-");
await fsp.mkdir(path.join(base, "cache"));
await writeOldFile(path.join(base, "cache", "old.txt"));
await writeOldFile(path.join(outside, "old.txt"), "outside");
const store = fileStore({ rootDir: base });
__setFsSafeTestHooksForTest({
async beforeFileStorePruneDescend(dirPath) {
if (!dirPath.endsWith("cache")) return;
await fsp.rename(path.join(base, "cache"), path.join(base, "cache-real"));
await fsp.symlink(outside, path.join(base, "cache"), "dir");
},
});
await store.pruneExpired({ ttlMs: 1, recursive: true });
await expect(fsp.readFile(path.join(outside, "old.txt"), "utf8")).resolves.toBe("outside");
});
it.runIf(process.platform !== "win32")("does not copy JSON through a raced fallback symlink", async () => {
const base = await tempRoot("fs-safe-json-fallback-race-");
const outside = await tempRoot("fs-safe-json-fallback-outside-");
const target = path.join(base, "state.json");
const outsideFile = path.join(outside, "outside.json");
await fsp.writeFile(target, "{}");
await fsp.writeFile(outsideFile, "outside");
const originalRenameSync = fsSync.renameSync;
const originalLstatSync = fsSync.lstatSync;
let forced = false;
let swapped = false;
vi.spyOn(fsSync, "renameSync").mockImplementation((from, to) => {
if (!forced && to === target) {
forced = true;
throw Object.assign(new Error("forced EPERM"), { code: "EPERM" });
}
return originalRenameSync.call(fsSync, from, to);
});
vi.spyOn(fsSync, "lstatSync").mockImplementation((candidate, options) => {
const stat = originalLstatSync.call(fsSync, candidate, options as never);
if (!swapped && candidate === target && forced) {
swapped = true;
fsSync.rmSync(target);
fsSync.symlinkSync(outsideFile, target, "file");
}
return stat;
});
writeJsonSync(target, { ok: true });
await expect(fsp.readFile(outsideFile, "utf8")).resolves.toBe("outside");
await expect(fsp.readFile(target, "utf8")).resolves.toContain('"ok": true');
});
it.runIf(process.platform !== "win32")("does not chmod existing parents during sync JSON writes", async () => {
const base = await tempRoot("fs-safe-json-parent-mode-");
const parent = path.join(base, "shared");
await fsp.mkdir(parent, { mode: 0o755 });
await fsp.chmod(parent, 0o755);
writeJsonSync(path.join(parent, "state.json"), { ok: true });
expect((await fsp.stat(parent)).mode & 0o777).toBe(0o755);
});
it.runIf(process.platform !== "win32")("does not copy atomic fallback through a raced destination symlink", async () => {
const base = await tempRoot("fs-safe-atomic-fallback-race-");
const outside = await tempRoot("fs-safe-atomic-fallback-outside-");
const target = path.join(base, "state.txt");
const outsideFile = path.join(outside, "outside.txt");
await fsp.writeFile(target, "old");
await fsp.writeFile(outsideFile, "outside");
const originalLstat = fsp.lstat;
let swapped = false;
await replaceFileAtomic({
filePath: target,
content: "new",
copyFallbackOnPermissionError: true,
fileSystem: {
promises: {
...fsp,
rename: async () => {
throw Object.assign(new Error("forced EPERM"), { code: "EPERM" });
},
lstat: async (candidate) => {
const stat = await originalLstat(candidate);
if (!swapped && candidate === target) {
swapped = true;
await fsp.rm(target);
await fsp.symlink(outsideFile, target, "file");
}
return stat;
},
},
},
});
await expect(fsp.readFile(outsideFile, "utf8")).resolves.toBe("outside");
await expect(fsp.readFile(target, "utf8")).resolves.toBe("new");
});
it.runIf(process.platform !== "win32")("stages EXDEV file moves without buffering or chmodding parents", async () => {
const base = await tempRoot("fs-safe-move-exdev-mode-");
const source = path.join(base, "source.bin");
const destDir = path.join(base, "public");
const dest = path.join(destDir, "dest.bin");
await fsp.mkdir(destDir, { mode: 0o755 });
await fsp.chmod(destDir, 0o755);
await fsp.writeFile(source, Buffer.alloc(1024 * 1024, 7));
const realRename = fsp.rename;
const realReadFile = fsp.readFile;
vi.spyOn(fsp, "rename").mockImplementation(async (from, to) => {
if (from === source && to === dest) {
throw Object.assign(new Error("cross-device"), { code: "EXDEV" });
}
return await realRename(from, to);
});
vi.spyOn(fsp, "readFile").mockImplementation(async (target, options) => {
if (target === source) {
throw new Error("move fallback must not buffer source files");
}
return await realReadFile(target, options as never);
});
await movePathWithCopyFallback({ from: source, to: dest });
expect((await fsp.stat(destDir)).mode & 0o777).toBe(0o755);
await expect(fsp.stat(source)).rejects.toMatchObject({ code: "ENOENT" });
expect((await fsp.readFile(dest)).byteLength).toBe(1024 * 1024);
});
it.runIf(process.platform !== "win32")("preserves public directory modes for zip staging parents", async () => {
const oldUmask = process.umask(0o022);
try {
const base = await tempRoot("fs-safe-zip-dir-mode-");
const archivePath = path.join(base, "pkg.zip");
const destDir = path.join(base, "dest");
await fsp.mkdir(destDir);
const zip = new JSZip();
zip.file("assets/app.js", "console.log('ok');");
await fsp.writeFile(archivePath, await zip.generateAsync({ type: "nodebuffer" }));
await extractArchive({ archivePath, destDir, kind: "zip", timeoutMs: 15_000 });
expect((await fsp.stat(path.join(destDir, "assets"))).mode & 0o777).toBe(0o755);
} finally {
process.umask(oldUmask);
}
});
it("rejects durable queue ids that are not safe path segments", async () => {
const base = await tempRoot("fs-safe-queue-id-");
expect(() => resolveJsonDurableQueueEntryPaths(base, "../escape")).toThrow();
await expect(
moveJsonDurableQueueEntryToFailed({ queueDir: base, failedDir: path.join(base, "failed"), id: "nested/escape" }),
).rejects.toBeTruthy();
});
it("keeps dot-only temp filenames inside the private temp directory", async () => {
const base = await tempRoot("fs-safe-temp-dot-");
expect(sanitizeTempFileName("..")).toBe("download.bin");
expect(sanitizeTempFileName("./..")).toBe("download.bin");
const target = await tempFile({ rootDir: base, prefix: "download", fileName: ".." });
try {
expect(target.path).toBe(path.join(target.dir, "download.bin"));
expect(target.file("..")).toBe(path.join(target.dir, "download.bin"));
} finally {
await target.cleanup();
}
});
it("accepts safe temp workspace leaf names with spaces and dot prefixes", async () => {
await using workspace = await tempWorkspace({
rootDir: await tempRoot("fs-safe-workspace-leaf-"),
prefix: "work-",
});
await workspace.writeText("report 2026.txt", "ok");
await workspace.writeText(".env", "TOKEN=ok");
await expect(workspace.read("report 2026.txt")).resolves.toEqual(Buffer.from("ok"));
await expect(workspace.read(".env")).resolves.toEqual(Buffer.from("TOKEN=ok"));
});
it.runIf(process.platform !== "win32")("pins sync temp workspace reads against final symlink swaps", async () => {
const base = await tempRoot("fs-safe-temp-workspace-sync-read-");
const outside = await tempRoot("fs-safe-temp-workspace-sync-outside-");
const workspace = tempWorkspaceSync({ rootDir: base, prefix: "ws-" });
try {
workspace.write("value.bin", Buffer.from([0, 1, 2, 3]));
const outsideFile = path.join(outside, "outside.bin");
fsSync.writeFileSync(outsideFile, "outside");
const targetPath = workspace.path("value.bin");
const originalReadFileSync = fsSync.readFileSync;
let swapped = false;
vi.spyOn(fsSync, "readFileSync").mockImplementation((target, options) => {
if (!swapped && typeof target === "number") {
swapped = true;
fsSync.rmSync(targetPath);
fsSync.symlinkSync(outsideFile, targetPath, "file");
}
return originalReadFileSync.call(fsSync, target, options as never);
});
expect([...workspace.read("value.bin")]).toEqual([0, 1, 2, 3]);
expect(fsSync.readFileSync(outsideFile, "utf8")).toBe("outside");
} finally {
workspace.cleanup();
}
});
it.runIf(process.platform !== "win32")("writes sibling-temp content from a private temp path, not a swapped target parent", async () => {
const base = await tempRoot("fs-safe-sibling-temp-race-");
const outside = await tempRoot("fs-safe-sibling-temp-outside-");
await fsp.mkdir(path.join(base, "nested"));
__setFsSafeTestHooksForTest({
async beforeSiblingTempWrite() {
await fsp.rename(path.join(base, "nested"), path.join(base, "nested-real"));
await fsp.symlink(outside, path.join(base, "nested"), "dir");
},
});
await expect(
writeViaSiblingTempPath({
rootDir: base,
targetPath: path.join(base, "nested", "out.txt"),
writeTemp: async (tempPath) => {
await fsp.writeFile(tempPath, "secret");
},
}),
).rejects.toBeTruthy();
await expect(fsp.stat(path.join(outside, "out.txt"))).rejects.toMatchObject({ code: "ENOENT" });
});
it.runIf(process.platform !== "win32")("does not trash a different path after an allowed parent swap", async () => {
const base = await tempRoot("fs-safe-trash-race-");
const outside = await tempRoot("fs-safe-trash-outside-");
await fsp.mkdir(path.join(base, "dir"));
await fsp.writeFile(path.join(base, "dir", "victim.txt"), "inside");
await fsp.writeFile(path.join(outside, "victim.txt"), "outside");
__setFsSafeTestHooksForTest({
beforeTrashMove() {
fsSync.renameSync(path.join(base, "dir"), path.join(base, "dir-real"));
fsSync.symlinkSync(outside, path.join(base, "dir"), "dir");
},
});
await expect(movePathToTrash(path.join(base, "dir", "victim.txt"), { allowedRoots: [base] }))
.rejects.toBeTruthy();
await expect(fsp.readFile(path.join(outside, "victim.txt"), "utf8")).resolves.toBe("outside");
});
});

View File

@ -4,7 +4,11 @@ import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { configureFsSafePython, FsSafeError, root as openRoot } from "../src/index.js";
import { openLocalFileSafely, readLocalFileSafely } from "../src/root.js";
import { __setFsSafeTestHooksForTest } from "../src/test-hooks.js";
import { expectedFsSafeCode } from "./helpers/security.js";
const skipOnWindows = process.platform === "win32";
const tempDirs: string[] = [];
@ -22,7 +26,7 @@ afterEach(async () => {
});
describe("@openclaw/fs-safe", () => {
it("reuses a root capability across filesystem operations", async () => {
it.skipIf(skipOnWindows)("reuses a root capability across filesystem operations", async () => {
const rootPath = await tempRoot("fs-root-object-");
const root = await openRoot(rootPath);
@ -50,7 +54,18 @@ describe("@openclaw/fs-safe", () => {
});
});
it("can disable the Python helper and keep root operations available", async () => {
it("rejects non-directory roots before creating a capability", async () => {
const rootPath = await tempRoot("fs-safe-root-file-");
const filePath = path.join(rootPath, "file.txt");
await writeFile(filePath, "not a directory");
await expect(openRoot(filePath)).rejects.toMatchObject({
code: "invalid-path",
message: "root dir is not a directory",
});
});
it.skipIf(skipOnWindows)("can disable the Python helper and keep root operations available", async () => {
configureFsSafePython({ mode: "off" });
const rootPath = await tempRoot("fs-safe-python-off-");
const sourceRoot = await tempRoot("fs-safe-python-off-source-");
@ -113,7 +128,7 @@ describe("@openclaw/fs-safe", () => {
);
});
it("writes, reads, stats, and lists files within a root", async () => {
it.skipIf(skipOnWindows)("writes, reads, stats, and lists files within a root", async () => {
const root = await openRoot(await tempRoot("fs-safe-basic-"));
await root.mkdir("nested");
@ -146,6 +161,61 @@ describe("@openclaw/fs-safe", () => {
});
});
it("rejects NUL bytes with FsSafeError before reaching Node fs", async () => {
const root = await openRoot(await tempRoot("fs-safe-nul-"));
for (const operation of [
() => root.resolve("x\0y"),
() => root.open("x\0y"),
() => root.openWritable("x\0y"),
() => root.read("x\0y"),
() => root.readBytes("x\0y"),
() => root.readText("x\0y"),
() => root.readJson("x\0y"),
() => root.write("x\0y", "data"),
() => root.append("x\0y", "data"),
() => root.copyIn("x\0y", path.join(root.rootDir, "source.txt")),
() => root.exists("x\0y"),
() => root.stat("x\0y"),
() => root.list("x\0y"),
() => root.move("x\0y", "dest.txt"),
() => root.move("source.txt", "x\0y"),
() => root.remove("x\0y"),
() => root.mkdir("x\0y"),
]) {
await expect(operation()).rejects.toMatchObject({
code: "invalid-path",
message: "relative path contains a NUL byte",
});
}
await expect(root.copyIn("dest.txt", `${root.rootDir}/source\0.txt`)).rejects.toMatchObject({
code: "invalid-path",
message: "source path contains a NUL byte",
});
});
it("rejects NUL bytes on public root and local-file entry points", async () => {
const rootPath = await tempRoot("fs-safe-public-nul-");
const filePath = path.join(rootPath, "file.txt");
await writeFile(filePath, "ok");
for (const operation of [
() => openRoot(`${rootPath}\0bad`),
() => openLocalFileSafely({ filePath: `${filePath}\0bad` }),
() => readLocalFileSafely({ filePath: `${filePath}\0bad` }),
]) {
let thrown: unknown;
try {
await operation();
} catch (error) {
thrown = error;
}
expect(thrown).toMatchObject({ code: "invalid-path" });
expect(String(thrown)).not.toContain(rootPath);
}
});
it("rejects reader callbacks for absolute paths outside the root", async () => {
const root = await openRoot(await tempRoot("fs-safe-reader-root-"));
const outside = await tempRoot("fs-safe-reader-outside-");
@ -167,7 +237,7 @@ describe("@openclaw/fs-safe", () => {
await expect(root.read("link/secret.txt")).rejects.toMatchObject({
code: "outside-workspace",
});
await expect(root.list("link")).rejects.toMatchObject({ code: "path-alias" });
await expect(root.list("link")).rejects.toMatchObject({ code: expectedFsSafeCode("path-alias") });
});
it("rejects symlink leaves for stat and read", async () => {
@ -177,11 +247,11 @@ describe("@openclaw/fs-safe", () => {
await writeFile(path.join(outside, "secret.txt"), "secret");
await symlink(path.join(outside, "secret.txt"), path.join(rootPath, "secret-link"), "file");
await expect(root.stat("secret-link")).rejects.toMatchObject({ code: "path-alias" });
await expect(root.stat("secret-link")).rejects.toMatchObject({ code: expectedFsSafeCode("path-alias") });
await expect(root.read("secret-link")).rejects.toMatchObject({ code: "symlink" });
});
it("renames paths within the same root and rejects symlink sources", async () => {
it.skipIf(skipOnWindows)("renames paths within the same root and rejects symlink sources", async () => {
const rootPath = await tempRoot("fs-safe-rename-");
const root = await openRoot(rootPath);
const outside = await tempRoot("fs-safe-outside-");
@ -197,7 +267,7 @@ describe("@openclaw/fs-safe", () => {
});
});
it("requires explicit overwrite for moves that replace a target", async () => {
it.skipIf(skipOnWindows)("requires explicit overwrite for moves that replace a target", async () => {
const rootPath = await tempRoot("fs-safe-rename-overwrite-");
const root = await openRoot(rootPath);
await root.write("from.txt", "source");
@ -212,7 +282,7 @@ describe("@openclaw/fs-safe", () => {
await expect(readFile(path.join(rootPath, "to.txt"), "utf8")).resolves.toBe("source");
});
it("enforces copyIn maxBytes while streaming", async () => {
it.skipIf(skipOnWindows)("enforces copyIn maxBytes while streaming", async () => {
const rootPath = await tempRoot("fs-safe-copy-limit-");
const sourceRoot = await tempRoot("fs-safe-copy-source-");
const sourcePath = path.join(sourceRoot, "source.txt");
@ -287,7 +357,7 @@ describe("@openclaw/fs-safe", () => {
await expect(readFile(outsideFile, "utf8")).resolves.toBe("kept");
await expect(root.stat("link")).rejects.toMatchObject({
code: "not-found",
code: expectedFsSafeCode("not-found"),
});
});

View File

@ -0,0 +1,106 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import { configureFsSafePython } from "../src/pinned-python-config.js";
import { runPinnedWriteHelper } from "../src/pinned-write.js";
const tempDirs: string[] = [];
async function tempRoot(prefix: string): Promise<string> {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), prefix));
tempDirs.push(dir);
return dir;
}
async function replaceParentAfterOpen(params: {
targetPath: string;
parentPath: string;
movedParentPath: string;
symlinkTargetPath: string;
}): Promise<() => void> {
const originalOpen = fs.open;
let closeSpy: ReturnType<typeof vi.spyOn> | undefined;
const openSpy = vi.spyOn(fs, "open").mockImplementation(async (...args) => {
const handle = await originalOpen(...args);
if (String(args[0]) === params.targetPath) {
closeSpy = vi.spyOn(handle, "close");
await fs.rename(params.parentPath, params.movedParentPath);
await fs.symlink(params.symlinkTargetPath, params.parentPath, "dir");
}
return handle;
});
return () => {
openSpy.mockRestore();
expect(closeSpy).toHaveBeenCalledTimes(1);
};
}
afterEach(async () => {
vi.restoreAllMocks();
configureFsSafePython({ mode: "auto", pythonPath: undefined });
Object.defineProperty(process, "platform", originalPlatformDescriptor);
await Promise.all(tempDirs.splice(0).map((dir) => fs.rm(dir, { recursive: true, force: true })));
});
const originalPlatformDescriptor = Object.getOwnPropertyDescriptor(process, "platform")!;
describe("guarded fallback write cleanup", () => {
it.runIf(process.platform !== "win32")("closes pinned no-overwrite handles when post guards fail", async () => {
configureFsSafePython({ mode: "off" });
const base = await tempRoot("fs-safe-pinned-post-guard-");
const parentPath = path.join(base, "nested");
const movedParentPath = path.join(base, "nested-real");
const targetPath = path.join(parentPath, "created.txt");
const outside = await tempRoot("fs-safe-pinned-post-guard-outside-");
const outsideFile = path.join(outside, "created.txt");
await fs.mkdir(parentPath);
await fs.writeFile(outsideFile, "outside");
const assertClosed = await replaceParentAfterOpen({
targetPath,
parentPath,
movedParentPath,
symlinkTargetPath: outside,
});
await expect(
runPinnedWriteHelper({
rootPath: base,
relativeParentPath: "nested",
basename: "created.txt",
mkdir: false,
mode: 0o600,
overwrite: false,
input: { kind: "buffer", data: "payload" },
}),
).rejects.toBeTruthy();
assertClosed();
await expect(fs.readFile(outsideFile, "utf8")).resolves.toBe("outside");
});
it.runIf(process.platform !== "win32")("closes root no-overwrite handles when post guards fail", async () => {
Object.defineProperty(process, "platform", { configurable: true, value: "win32" });
const { root: openRoot } = await import("../src/index.js");
const base = await tempRoot("fs-safe-root-post-guard-");
const parentPath = path.join(base, "nested");
const movedParentPath = path.join(base, "nested-real");
const outside = await tempRoot("fs-safe-root-post-guard-outside-");
const outsideFile = path.join(outside, "created.txt");
await fs.mkdir(parentPath);
await fs.writeFile(outsideFile, "outside");
const targetPath = path.join(await fs.realpath(parentPath), "created.txt");
const assertClosed = await replaceParentAfterOpen({
targetPath,
parentPath,
movedParentPath,
symlinkTargetPath: outside,
});
const scoped = await openRoot(base);
await expect(scoped.create("nested/created.txt", "payload")).rejects.toBeTruthy();
assertClosed();
await expect(fs.readFile(outsideFile, "utf8")).resolves.toBe("outside");
});
});

124
test/helpers/security.ts Normal file
View File

@ -0,0 +1,124 @@
import fsp from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { expect } from "vitest";
import { FsSafeError } from "../../src/index.js";
export type TempLayout = {
outside: string;
outsideFile: string;
root: string;
};
export const TRAVERSAL_PAYLOADS = [
"../secret.txt",
"../../secret.txt",
"nested/../../secret.txt",
"nested/../../../secret.txt",
"./../secret.txt",
"nested/..//../secret.txt",
"nested/%2e%2e/secret.txt",
"%2e%2e/secret.txt",
"%2e%2e%2fsecret.txt",
"..%2fsecret.txt",
"%252e%252e%252fsecret.txt",
"..%00/secret.txt",
"..\\secret.txt",
"nested\\..\\..\\secret.txt",
"C:\\Windows\\win.ini",
"\\\\server\\share\\secret.txt",
] as const;
export const LIST_TRAVERSAL_PAYLOADS = [
"..",
"../",
"../../",
"nested/../..",
"nested/../../outside",
"%2e%2e",
"%2e%2e%2f",
"..\\",
"C:\\Windows",
"\\\\server\\share",
] as const;
export const ESCAPING_WRITE_PAYLOADS = [
"../pwned.txt",
"../../pwned.txt",
"nested/../../pwned.txt",
"nested/../../../pwned.txt",
"./../pwned.txt",
"nested/..//../pwned.txt",
] as const;
export const LITERAL_SUSPICIOUS_WRITE_PAYLOADS = [
"nested/%2e%2e/pwned.txt",
"%2e%2e/pwned.txt",
"%2e%2e%2fpwned.txt",
"%252e%252e%252fpwned.txt",
// ".." prefix without an actual separator: a single literal filename
// ("..%2fpwned.txt") or two literal segments ("..%00", "pwned.txt") that
// resolve fully inside root. Accepted on both platforms.
"..%2fpwned.txt",
"..%00/pwned.txt",
] as const;
export const POSIX_LITERAL_SUSPICIOUS_WRITE_PAYLOADS = [
"nested\\..\\..\\pwned.txt",
"C:\\Windows\\win.ini",
"\\\\server\\share\\pwned.txt",
// "..\\" is a real traversal on Windows (separator) but a literal filename
// on POSIX (where "\\" is a regular name character).
"..\\pwned.txt",
] as const;
export const ESCAPING_DIRECTORY_PAYLOADS = [
"..",
"../",
"../../",
"nested/../..",
"nested/../../outside",
] as const;
export const LITERAL_SUSPICIOUS_DIRECTORY_PAYLOADS = ["%2e%2e", "%2e%2e%2f"] as const;
export const SAFE_REJECTED_SUSPICIOUS_DIRECTORY_PAYLOADS = ["..\\"] as const;
export const WINDOWS_REJECTED_SUSPICIOUS_DIRECTORY_PAYLOADS = [
"C:\\Windows",
"\\\\server\\share",
] as const;
export async function makeTempLayout(
prefix: string,
tempDirs: string[],
): Promise<TempLayout> {
const root = await fsp.mkdtemp(path.join(os.tmpdir(), `${prefix}-root-`));
const outside = await fsp.mkdtemp(path.join(os.tmpdir(), `${prefix}-outside-`));
tempDirs.push(root, outside);
const outsideFile = path.join(outside, "secret.txt");
await fsp.writeFile(outsideFile, "outside secret");
return { outside, outsideFile, root };
}
export function expectFsSafeCode(
error: unknown,
codes: readonly string[],
opts: { allowUnsupportedPlatformOnWindows?: boolean } = {},
): void {
expect(error).toBeInstanceOf(FsSafeError);
const accepted = process.platform === "win32" && opts.allowUnsupportedPlatformOnWindows
? [...codes, "unsupported-platform"]
: codes;
expect(accepted).toContain((error as FsSafeError).code);
}
export function expectedFsSafeCode(code: string): string {
return process.platform === "win32" ? "unsupported-platform" : code;
}
export async function expectNoOutsideWrite(
layout: TempLayout,
expected = "outside secret",
): Promise<void> {
await expect(fsp.readFile(layout.outsideFile, "utf8")).resolves.toBe(expected);
}

View File

@ -0,0 +1,97 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import {
ackJsonDurableQueueEntry,
ensureJsonDurableQueueDirs,
jsonDurableQueueEntryExists,
loadJsonDurableQueueEntry,
loadPendingJsonDurableQueueEntries,
moveJsonDurableQueueEntryToFailed,
resolveJsonDurableQueueEntryPaths,
writeJsonDurableQueueEntry,
} from "../src/json-durable-queue.js";
let root: string;
beforeEach(async () => {
root = await fs.mkdtemp(path.join(os.tmpdir(), "fs-safe-json-queue-"));
});
afterEach(async () => {
await fs.rm(root, { recursive: true, force: true });
});
describe("durable JSON queues", () => {
it("writes, migrates, loads, acks, and moves entries", async () => {
const queueDir = path.join(root, "queue");
const failedDir = path.join(root, "failed");
await ensureJsonDurableQueueDirs({ queueDir, failedDir });
const paths = resolveJsonDurableQueueEntryPaths(queueDir, "entry-1");
await writeJsonDurableQueueEntry({
filePath: paths.jsonPath,
entry: { id: "entry-1", version: 1 },
tempPrefix: "queue",
});
await expect(jsonDurableQueueEntryExists(paths.jsonPath)).resolves.toBe(true);
await expect(
loadJsonDurableQueueEntry<{ id: string; version: number }>({
paths,
tempPrefix: "queue",
read: async (entry) => ({ entry: { ...entry, version: 2 }, migrated: true }),
}),
).resolves.toEqual({ id: "entry-1", version: 2 });
await expect(fs.readFile(paths.jsonPath, "utf8")).resolves.toContain("\"version\": 2");
await expect(
loadPendingJsonDurableQueueEntries<{ id: string; version: number }>({
queueDir,
tempPrefix: "queue",
}),
).resolves.toEqual([{ id: "entry-1", version: 2 }]);
await ackJsonDurableQueueEntry(paths);
await expect(jsonDurableQueueEntryExists(paths.jsonPath)).resolves.toBe(false);
await writeJsonDurableQueueEntry({
filePath: paths.jsonPath,
entry: { id: "entry-1", version: 3 },
tempPrefix: "queue",
});
await moveJsonDurableQueueEntryToFailed({ queueDir, failedDir, id: "entry-1" });
await expect(fs.readFile(path.join(failedDir, "entry-1.json"), "utf8")).resolves.toContain(
"\"version\": 3",
);
});
it("skips bad pending entries and cleans stale temp files", async () => {
const queueDir = path.join(root, "queue");
const failedDir = path.join(root, "failed");
await ensureJsonDurableQueueDirs({ queueDir, failedDir });
await fs.writeFile(path.join(queueDir, "good.json"), JSON.stringify({ ok: true }), "utf8");
await fs.writeFile(path.join(queueDir, "bad.json"), "{", "utf8");
const tempPath = path.join(queueDir, "orphan.tmp");
await fs.writeFile(tempPath, "tmp", "utf8");
const old = new Date(Date.now() - 10_000);
await fs.utimes(tempPath, old, old);
await expect(
loadPendingJsonDurableQueueEntries<{ ok: boolean }>({
queueDir,
tempPrefix: "queue",
cleanupTmpMaxAgeMs: 1,
}),
).resolves.toEqual([{ ok: true }]);
await expect(fs.access(tempPath)).rejects.toMatchObject({ code: "ENOENT" });
});
it("rejects whitespace-padded queue ids", async () => {
const queueDir = path.join(root, "queue");
expect(() => resolveJsonDurableQueueEntryPaths(queueDir, "job ")).toThrow();
expect(() => resolveJsonDurableQueueEntryPaths(queueDir, " job")).toThrow();
});
});

View File

@ -6,6 +6,8 @@ import { createAsyncLock } from "../src/async-lock.js";
import { writeTextAtomic } from "../src/atomic.js";
import {
JsonFileReadError,
readRootJsonObjectSync,
readRootStructuredFileSync,
readJson,
readJsonIfExists,
readJsonSync,
@ -26,6 +28,24 @@ afterEach(async () => {
await Promise.all(tempDirs.splice(0).map((dir) => fs.rm(dir, { force: true, recursive: true })));
});
function mockOpenForSyncCounting(): { readonly syncCalls: number; restore: () => void } {
let syncCalls = 0;
const openSpy = vi.spyOn(fs, "open").mockImplementation(async () => {
return {
sync: async () => {
syncCalls += 1;
},
close: async () => undefined,
} as Awaited<ReturnType<typeof fs.open>>;
});
return {
get syncCalls() {
return syncCalls;
},
restore: () => openSpy.mockRestore(),
};
}
describe("json file helpers", () => {
it("writes formatted JSON atomically with an optional trailing newline", async () => {
const root = await tempRoot("fs-safe-json-");
@ -57,6 +77,54 @@ describe("json file helpers", () => {
}
});
it("syncs temp file and parent directory by default for text writes", async () => {
const root = await tempRoot("fs-safe-json-");
const filePath = path.join(root, "default-durable.txt");
const syncCounter = mockOpenForSyncCounting();
try {
await writeTextAtomic(filePath, "data");
} finally {
syncCounter.restore();
}
expect(syncCounter.syncCalls).toBe(2);
await expect(fs.readFile(filePath, "utf8")).resolves.toBe("data");
});
it("skips fsync when text writes opt out of durability", async () => {
const root = await tempRoot("fs-safe-json-");
const filePath = path.join(root, "store.json");
await fs.writeFile(filePath, "old", "utf8");
const syncCounter = mockOpenForSyncCounting();
try {
await writeTextAtomic(filePath, "new", { durable: false });
} finally {
syncCounter.restore();
}
expect(syncCounter.syncCalls).toBe(0);
await expect(fs.readFile(filePath, "utf8")).resolves.toBe("new");
const dirEntries = await fs.readdir(root);
expect(dirEntries.some((entry) => entry.endsWith(".tmp"))).toBe(false);
});
it("threads durable option through JSON writes", async () => {
const root = await tempRoot("fs-safe-json-");
const filePath = path.join(root, "state.json");
const syncCounter = mockOpenForSyncCounting();
try {
await writeJson(filePath, { ok: true }, { durable: false });
} finally {
syncCounter.restore();
}
expect(syncCounter.syncCalls).toBe(0);
await expect(fs.readFile(filePath, "utf8")).resolves.toBe("{\n \"ok\": true\n}");
});
it("separates nullable and durable read failure semantics", async () => {
const root = await tempRoot("fs-safe-json-");
const missing = path.join(root, "missing.json");
@ -141,4 +209,62 @@ describe("json file helpers", () => {
await expect(Promise.all([first, second])).resolves.toEqual([1, 2]);
expect(events).toEqual(["first:start", "first:end", "second"]);
});
it("reads JSON objects through a root-bounded open", async () => {
const root = await tempRoot("fs-safe-root-json-");
await fs.writeFile(path.join(root, "config.json"), JSON.stringify({ name: "demo" }), "utf8");
const result = readRootJsonObjectSync({
rootDir: root,
relativePath: "config.json",
boundaryLabel: "test root",
rejectHardlinks: true,
});
expect(result).toMatchObject({ ok: true, value: { name: "demo" } });
});
it("rejects invalid root-bounded JSON shapes and escapes", async () => {
const root = await tempRoot("fs-safe-root-json-");
const outside = path.join(path.dirname(root), `${path.basename(root)}.json`);
await fs.writeFile(path.join(root, "array.json"), "[]", "utf8");
await fs.writeFile(outside, JSON.stringify({ name: "outside" }), "utf8");
try {
expect(
readRootJsonObjectSync({
rootDir: root,
relativePath: "array.json",
boundaryLabel: "test root",
}),
).toMatchObject({ ok: false, reason: "invalid" });
expect(
readRootJsonObjectSync({
rootDir: root,
relativePath: "../outside-root-json-test.json",
boundaryLabel: "test root",
}),
).toMatchObject({ ok: false, reason: "open" });
} finally {
await fs.rm(outside, { force: true });
}
});
it("lets callers provide parser and validation for root-bounded structured files", async () => {
const root = await tempRoot("fs-safe-root-structured-");
await fs.writeFile(path.join(root, "config.txt"), "name=demo", "utf8");
const result = readRootStructuredFileSync<{ name: string }>({
rootDir: root,
relativePath: "config.txt",
boundaryLabel: "test root",
parse: (raw) => ({ name: raw.split("=")[1]?.trim() }),
validate: (value): value is { name: string } =>
typeof value === "object" &&
value !== null &&
"name" in value &&
typeof value.name === "string",
});
expect(result).toMatchObject({ ok: true, value: { name: "demo" } });
});
});

View File

@ -0,0 +1,179 @@
import fsp from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import { movePathWithCopyFallback } from "../src/move-path.js";
const tempDirs: string[] = [];
async function tempRoot(prefix: string): Promise<string> {
const dir = await fsp.mkdtemp(path.join(os.tmpdir(), prefix));
tempDirs.push(dir);
return dir;
}
afterEach(async () => {
vi.restoreAllMocks();
await Promise.all(tempDirs.splice(0).map((dir) => fsp.rm(dir, { recursive: true, force: true })));
});
describe("movePathWithCopyFallback regressions", () => {
it.runIf(process.platform !== "win32")(
"does not delete source entries replaced after an EXDEV copy",
async () => {
const base = await tempRoot("fs-safe-move-exdev-replaced-source-");
const source = path.join(base, "source-dir");
const dest = path.join(base, "dest-dir");
await fsp.mkdir(source);
await fsp.writeFile(path.join(source, "copied.txt"), "copied");
const realRename = fsp.rename;
vi.spyOn(fsp, "rename").mockImplementation(async (from, to) => {
if (from === source && to === dest) {
throw Object.assign(new Error("cross-device"), { code: "EXDEV" });
}
await realRename(from, to);
if (to === dest && String(from).includes(".fs-safe-move-")) {
await fsp.rm(path.join(source, "copied.txt"));
await fsp.writeFile(path.join(source, "copied.txt"), "replacement");
await fsp.writeFile(path.join(source, "late.txt"), "late");
}
});
await expect(movePathWithCopyFallback({ from: source, to: dest })).rejects.toMatchObject({
code: "ESTALE",
});
await expect(fsp.readFile(path.join(dest, "copied.txt"), "utf8")).resolves.toBe("copied");
await expect(fsp.readFile(path.join(source, "copied.txt"), "utf8")).resolves.toBe(
"replacement",
);
await expect(fsp.readFile(path.join(source, "late.txt"), "utf8")).resolves.toBe("late");
},
);
it.runIf(process.platform !== "win32")(
"can reject hardlinked files during EXDEV move fallback",
async () => {
const base = await tempRoot("fs-safe-move-exdev-hardlink-");
const source = path.join(base, "source.txt");
const hardlink = path.join(base, "hardlink.txt");
const dest = path.join(base, "dest.txt");
await fsp.writeFile(source, "source");
await fsp.link(source, hardlink);
const realRename = fsp.rename;
vi.spyOn(fsp, "rename").mockImplementation(async (from, to) => {
if (from === source && to === dest) {
throw Object.assign(new Error("cross-device"), { code: "EXDEV" });
}
return await realRename(from, to);
});
await expect(
movePathWithCopyFallback({ from: source, sourceHardlinks: "reject", to: dest }),
).rejects.toThrow("Refusing to move hardlinked file");
await expect(fsp.readFile(source, "utf8")).resolves.toBe("source");
await expect(fsp.stat(dest)).rejects.toMatchObject({ code: "ENOENT" });
},
);
it.runIf(process.platform !== "win32")(
"preserves directory modes during EXDEV move fallback",
async () => {
const base = await tempRoot("fs-safe-move-exdev-dir-mode-");
const source = path.join(base, "source-dir");
const dest = path.join(base, "dest-dir");
await fsp.mkdir(source);
await fsp.writeFile(path.join(source, "copied.txt"), "copied");
await fsp.chmod(source, 0o777);
const realRename = fsp.rename;
vi.spyOn(fsp, "rename").mockImplementation(async (from, to) => {
if (from === source && to === dest) {
throw Object.assign(new Error("cross-device"), { code: "EXDEV" });
}
return await realRename(from, to);
});
const realMkdir = fsp.mkdir;
vi.spyOn(fsp, "mkdir").mockImplementation(async (target, options) => {
const result = await realMkdir(target, options as never);
if (String(target).includes(".fs-safe-move-")) {
await fsp.chmod(target, 0o700);
}
return result;
});
await movePathWithCopyFallback({ from: source, to: dest });
expect((await fsp.stat(dest)).mode & 0o777).toBe(0o777);
await expect(fsp.readFile(path.join(dest, "copied.txt"), "utf8")).resolves.toBe("copied");
},
);
it.runIf(process.platform !== "win32")(
"removes unchanged copied children when source directory gains a late child",
async () => {
const base = await tempRoot("fs-safe-move-exdev-added-source-");
const source = path.join(base, "source-dir");
const dest = path.join(base, "dest-dir");
await fsp.mkdir(source);
await fsp.writeFile(path.join(source, "copied.txt"), "copied");
const realRename = fsp.rename;
vi.spyOn(fsp, "rename").mockImplementation(async (from, to) => {
if (from === source && to === dest) {
throw Object.assign(new Error("cross-device"), { code: "EXDEV" });
}
await realRename(from, to);
if (to === dest && String(from).includes(".fs-safe-move-")) {
await fsp.writeFile(path.join(source, "late.txt"), "late");
}
});
await expect(movePathWithCopyFallback({ from: source, to: dest })).rejects.toMatchObject({
code: "ESTALE",
});
await expect(fsp.readFile(path.join(dest, "copied.txt"), "utf8")).resolves.toBe("copied");
await expect(fsp.stat(path.join(source, "copied.txt"))).rejects.toMatchObject({
code: "ENOENT",
});
await expect(fsp.readFile(path.join(source, "late.txt"), "utf8")).resolves.toBe("late");
},
);
it.runIf(process.platform !== "win32")(
"does not commit bytes from a source swapped after validation",
async () => {
const base = await tempRoot("fs-safe-move-exdev-source-swap-");
const outside = await tempRoot("fs-safe-move-exdev-source-swap-outside-");
const source = path.join(base, "source.txt");
const dest = path.join(base, "dest.txt");
const outsideFile = path.join(outside, "secret.txt");
await fsp.writeFile(source, "inside");
await fsp.writeFile(outsideFile, "secret");
const realRename = fsp.rename;
vi.spyOn(fsp, "rename").mockImplementation(async (from, to) => {
if (from === source && to === dest) {
throw Object.assign(new Error("cross-device"), { code: "EXDEV" });
}
return await realRename(from, to);
});
const realLstat = fsp.lstat;
let swapped = false;
vi.spyOn(fsp, "lstat").mockImplementation(async (candidate, options) => {
const stat = await realLstat(candidate, options as never);
if (!swapped && candidate === source) {
swapped = true;
await fsp.rm(source);
await fsp.symlink(outsideFile, source, "file");
}
return stat;
});
await expect(movePathWithCopyFallback({ from: source, to: dest })).rejects.toMatchObject({
code: "ESTALE",
});
await expect(fsp.stat(dest)).rejects.toMatchObject({ code: "ENOENT" });
},
);
});

View File

@ -125,7 +125,7 @@ describe("private temp workspaces", () => {
});
describe("file store", () => {
it("writes, reads, copies, and prunes files under the store root", async () => {
it.skipIf(process.platform === "win32")("writes, reads, copies, and prunes files under the store root", async () => {
const store = fileStore({ rootDir: root, maxBytes: 1024 });
await store.write("media/a.txt", "hello");
await expect(store.readBytes("media/a.txt")).resolves.toEqual(Buffer.from("hello"));
@ -175,7 +175,7 @@ describe("json store", () => {
});
describe("secure file reads", () => {
it("reads from a validated file handle", async () => {
it.runIf(process.platform !== "win32")("reads from a validated file handle", async () => {
const filePath = path.join(root, "secret.json");
await fs.writeFile(filePath, '{"token":"ok"}', { mode: 0o600 });
await fs.chmod(filePath, 0o600).catch(() => undefined);
@ -190,6 +190,24 @@ describe("secure file reads", () => {
expect(result.realPath).toBe(await fs.realpath(filePath));
});
it.runIf(process.platform === "win32")(
"fails closed on windows when ACL inspection is unavailable",
async () => {
// See src/secure-file.ts:177 — readSecureFile throws permission-unverified
// on Windows because ACL inspection has no portable equivalent.
const filePath = path.join(root, "secret.json");
await fs.writeFile(filePath, '{"token":"ok"}', { mode: 0o600 });
await expect(
readSecureFile({
filePath,
label: "test secret",
io: { maxBytes: 1024 },
}),
).rejects.toMatchObject({ code: "permission-unverified" });
},
);
it("rejects symlinks and files outside trusted dirs", async () => {
const trusted = path.join(root, "trusted");
const outside = path.join(root, "outside");

278
test/output.test.ts Normal file
View File

@ -0,0 +1,278 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { writeExternalFileWithinRoot } from "../src/output.js";
const tempDirs = new Set<string>();
async function tempRoot(prefix: string): Promise<string> {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), prefix));
tempDirs.add(dir);
return dir;
}
afterEach(async () => {
for (const dir of tempDirs) {
await fs.rm(dir, { force: true, recursive: true });
}
tempDirs.clear();
});
describe("writeExternalFileWithinRoot", () => {
it("stages an external writer in private temp storage and finalizes under the root", async () => {
const rootDir = await tempRoot("fs-safe-output-root-");
const targetPath = path.join(rootDir, "downloads", "report.txt");
let tempPath = "";
const result = await writeExternalFileWithinRoot({
rootDir,
path: targetPath,
write: async (candidate) => {
tempPath = candidate;
await fs.writeFile(candidate, "downloaded", "utf8");
return { bytes: 10 };
},
});
expect(result.path).toBe(path.join(await fs.realpath(rootDir), "downloads", "report.txt"));
expect(result.result).toEqual({ bytes: 10 });
expect(path.dirname(tempPath)).not.toBe(path.dirname(targetPath));
await expect(fs.readFile(targetPath, "utf8")).resolves.toBe("downloaded");
await expect(fs.stat(tempPath)).rejects.toMatchObject({ code: "ENOENT" });
});
it("preserves caller-provided destination filename spacing", async () => {
const rootDir = await tempRoot("fs-safe-output-spaces-");
const fileName = " report .txt ";
const result = await writeExternalFileWithinRoot({
rootDir,
path: fileName,
write: async (candidate) => {
await fs.writeFile(candidate, "spaced", "utf8");
},
});
const finalPath = path.join(rootDir, fileName);
expect(result.path).toBe(path.join(await fs.realpath(rootDir), fileName));
await expect(fs.readFile(finalPath, "utf8")).resolves.toBe("spaced");
await expect(fs.stat(path.join(rootDir, fileName.trim()))).rejects.toMatchObject({
code: "ENOENT",
});
});
it("accepts absolute target paths that resolve inside the root", async () => {
const rootDir = await tempRoot("fs-safe-output-absolute-");
const targetPath = path.join(rootDir, "nested", "report.txt");
const result = await writeExternalFileWithinRoot({
rootDir,
path: targetPath,
write: async (candidate) => {
await fs.writeFile(candidate, "absolute", "utf8");
},
});
expect(result.path).toBe(path.join(await fs.realpath(rootDir), "nested", "report.txt"));
await expect(fs.readFile(targetPath, "utf8")).resolves.toBe("absolute");
});
it("enforces byte limits while leaving the final target absent", async () => {
const rootDir = await tempRoot("fs-safe-output-max-bytes-");
const targetPath = path.join(rootDir, "too-large.bin");
await expect(
writeExternalFileWithinRoot({
rootDir,
path: "too-large.bin",
maxBytes: 3,
write: async (candidate) => {
await fs.writeFile(candidate, "larger", "utf8");
},
}),
).rejects.toMatchObject({ code: "too-large" });
await expect(fs.stat(targetPath)).rejects.toMatchObject({ code: "ENOENT" });
});
it.runIf(process.platform !== "win32")("applies the requested final file mode", async () => {
const rootDir = await tempRoot("fs-safe-output-mode-");
const targetPath = path.join(rootDir, "private.txt");
await writeExternalFileWithinRoot({
rootDir,
path: "private.txt",
mode: 0o600,
write: async (candidate) => {
await fs.writeFile(candidate, "private", { encoding: "utf8", mode: 0o644 });
},
});
const stat = await fs.stat(targetPath);
expect(stat.mode & 0o777).toBe(0o600);
});
it("rejects empty target paths before invoking the external writer", async () => {
const rootDir = await tempRoot("fs-safe-output-default-");
let called = false;
await expect(
writeExternalFileWithinRoot({
rootDir,
path: "",
write: async (candidate) => {
called = true;
await fs.writeFile(candidate, "named", "utf8");
},
}),
).rejects.toMatchObject({ code: "invalid-path" });
expect(called).toBe(false);
});
it("rejects targets outside the root before invoking the external writer", async () => {
const rootDir = await tempRoot("fs-safe-output-reject-root-");
const outsideDir = await tempRoot("fs-safe-output-reject-outside-");
const outsidePath = path.join(outsideDir, "pwned.txt");
let called = false;
await expect(
writeExternalFileWithinRoot({
rootDir,
path: outsidePath,
write: async (candidate) => {
called = true;
await fs.writeFile(candidate, "pwned", "utf8");
},
}),
).rejects.toMatchObject({ code: "outside-workspace" });
expect(called).toBe(false);
await expect(fs.stat(outsidePath)).rejects.toMatchObject({ code: "ENOENT" });
});
it("rejects traversal targets before invoking the external writer", async () => {
const rootDir = await tempRoot("fs-safe-output-traversal-root-");
let called = false;
await expect(
writeExternalFileWithinRoot({
rootDir,
path: "../../../pwned.txt",
write: async (candidate) => {
called = true;
await fs.writeFile(candidate, "pwned", "utf8");
},
}),
).rejects.toMatchObject({ code: "outside-workspace" });
expect(called).toBe(false);
});
it("rejects root directory targets before invoking the external writer", async () => {
const rootDir = await tempRoot("fs-safe-output-root-target-");
let called = false;
await expect(
writeExternalFileWithinRoot({
rootDir,
path: rootDir,
write: async (candidate) => {
called = true;
await fs.writeFile(candidate, "not a file target", "utf8");
},
}),
).rejects.toMatchObject({ code: "invalid-path" });
expect(called).toBe(false);
});
it("rejects trailing-separator targets before invoking the external writer", async () => {
const rootDir = await tempRoot("fs-safe-output-dir-target-");
let called = false;
await expect(
writeExternalFileWithinRoot({
rootDir,
path: "nested/",
write: async (candidate) => {
called = true;
await fs.writeFile(candidate, "not a file target", "utf8");
},
}),
).rejects.toMatchObject({ code: "invalid-path" });
expect(called).toBe(false);
});
it.runIf(process.platform !== "win32")(
"does not let symlinked target parents redirect the external temp write",
async () => {
const rootDir = await tempRoot("fs-safe-output-link-root-");
const outsideDir = await tempRoot("fs-safe-output-link-outside-");
await fs.symlink(outsideDir, path.join(rootDir, "link"), "dir");
let tempPath = "";
await expect(
writeExternalFileWithinRoot({
rootDir,
path: "link/out.txt",
write: async (candidate) => {
tempPath = candidate;
await fs.writeFile(candidate, "pwned", "utf8");
},
}),
).rejects.toBeTruthy();
await expect(fs.stat(tempPath)).rejects.toMatchObject({ code: "ENOENT" });
await expect(fs.readdir(outsideDir)).resolves.toEqual([]);
},
);
it.runIf(process.platform !== "win32")(
"rejects hardlinked final targets and preserves the existing file",
async () => {
const rootDir = await tempRoot("fs-safe-output-hardlink-");
const sourcePath = path.join(rootDir, "source.txt");
const hardlinkPath = path.join(rootDir, "hardlink.txt");
await fs.writeFile(sourcePath, "original", "utf8");
await fs.link(sourcePath, hardlinkPath);
await expect(
writeExternalFileWithinRoot({
rootDir,
path: "hardlink.txt",
write: async (candidate) => {
await fs.writeFile(candidate, "replacement", "utf8");
},
}),
).rejects.toBeTruthy();
await expect(fs.readFile(sourcePath, "utf8")).resolves.toBe("original");
await expect(fs.readFile(hardlinkPath, "utf8")).resolves.toBe("original");
},
);
it("cleans private temp files when the external writer fails", async () => {
const rootDir = await tempRoot("fs-safe-output-fail-root-");
let tempPath = "";
await expect(
writeExternalFileWithinRoot({
rootDir,
path: "out.txt",
write: async (candidate) => {
tempPath = candidate;
await fs.writeFile(candidate, "partial", "utf8");
throw new Error("download failed");
},
}),
).rejects.toThrow("download failed");
await expect(fs.stat(tempPath)).rejects.toMatchObject({ code: "ENOENT" });
await expect(fs.stat(path.join(rootDir, "out.txt"))).rejects.toMatchObject({
code: "ENOENT",
});
});
});

View File

@ -180,6 +180,18 @@ describe("Python helper configuration", () => {
describe("persistent Python helper worker", () => {
it("reuses one worker and unreferences it while idle", async () => {
if (process.platform === "win32") {
configureFsSafePython({ mode: "auto", pythonPath: "/tmp/fake-python" });
await expect(
runPinnedPythonOperation<{ ok: boolean }>({
operation: "stat",
rootPath: "/tmp/root",
payload: { relativePath: "a.txt" },
}),
).rejects.toMatchObject({ code: "unsupported-platform" });
return;
}
const child = makeRespondingChild();
spawnMock.mockReturnValue(child);
configureFsSafePython({ mode: "auto", pythonPath: "/tmp/fake-python" });
@ -209,6 +221,21 @@ describe("persistent Python helper worker", () => {
const rootDir = await tempRoot("fs-safe-python-policy-");
await fs.writeFile(path.join(rootDir, "file.txt"), "ok");
if (process.platform === "win32") {
spawnMock.mockImplementation(makeFailingChild);
configureFsSafePython({ mode: "auto", pythonPath: "/tmp/missing-python" });
const autoRoot = await root(rootDir);
await expect(autoRoot.stat("file.txt")).rejects.toMatchObject({
code: "unsupported-platform",
});
configureFsSafePython({ mode: "require", pythonPath: "/tmp/missing-python" });
await expect((await root(rootDir)).stat("file.txt")).rejects.toMatchObject({
code: "unsupported-platform",
});
return;
}
spawnMock.mockImplementation(makeFailingChild);
configureFsSafePython({ mode: "auto", pythonPath: "/tmp/missing-python" });
const autoRoot = await root(rootDir);

View File

@ -38,64 +38,89 @@ afterEach(async () => {
});
describe("pinned write fallback coverage", () => {
it("writes buffers, creates only when missing, streams, and enforces limits", async () => {
const { runPinnedWriteHelper } = await import("../src/pinned-write.js");
const root = await tempRoot("fs-safe-pinned-write-fallback-");
it.runIf(process.platform !== "win32")(
"writes buffers, creates only when missing, streams, and enforces limits",
async () => {
const { runPinnedWriteHelper } = await import("../src/pinned-write.js");
const root = await tempRoot("fs-safe-pinned-write-fallback-");
const created = await runPinnedWriteHelper({
rootPath: root,
relativeParentPath: "nested",
basename: "created.txt",
mkdir: true,
mode: 0o600,
overwrite: false,
input: { kind: "buffer", data: "created", encoding: "utf8" },
});
expect(created.ino).toBeGreaterThan(0);
await expect(fs.readFile(path.join(root, "nested", "created.txt"), "utf8")).resolves.toBe(
"created",
);
await expect(
runPinnedWriteHelper({
const created = await runPinnedWriteHelper({
rootPath: root,
relativeParentPath: "nested",
basename: "created.txt",
mkdir: true,
mode: 0o600,
overwrite: false,
input: { kind: "buffer", data: "again" },
}),
).rejects.toMatchObject({ code: "EEXIST" });
input: { kind: "buffer", data: "created", encoding: "utf8" },
});
expect(created.ino).toBeGreaterThan(0);
await expect(fs.readFile(path.join(root, "nested", "created.txt"), "utf8")).resolves.toBe(
"created",
);
await expect(
runPinnedWriteHelper({
rootPath: root,
relativeParentPath: "nested",
basename: "created.txt",
mkdir: true,
mode: 0o600,
overwrite: false,
input: { kind: "buffer", data: "again" },
}),
).rejects.toMatchObject({ code: "EEXIST" });
const streamed = await runPinnedWriteHelper({
rootPath: root,
relativeParentPath: "nested",
basename: "streamed.txt",
mkdir: true,
mode: 0o600,
overwrite: true,
maxBytes: 16,
input: { kind: "stream", stream: Readable.from(["stream", "ed"]) },
});
expect(streamed.dev).toBeGreaterThan(0);
await expect(fs.readFile(path.join(root, "nested", "streamed.txt"), "utf8")).resolves.toBe(
"streamed",
);
await expect(
runPinnedWriteHelper({
const streamed = await runPinnedWriteHelper({
rootPath: root,
relativeParentPath: "nested",
basename: "too-large.txt",
basename: "streamed.txt",
mkdir: true,
mode: 0o600,
overwrite: true,
maxBytes: 2,
input: { kind: "buffer", data: Buffer.from("large") },
}),
).rejects.toMatchObject({ code: "too-large" });
await expect(fs.stat(path.join(root, "nested", "too-large.txt"))).rejects.toMatchObject({
code: "ENOENT",
});
});
maxBytes: 16,
input: { kind: "stream", stream: Readable.from(["stream", "ed"]) },
});
expect(streamed.dev).toBeGreaterThan(0);
await expect(fs.readFile(path.join(root, "nested", "streamed.txt"), "utf8")).resolves.toBe(
"streamed",
);
await expect(
runPinnedWriteHelper({
rootPath: root,
relativeParentPath: "nested",
basename: "too-large.txt",
mkdir: true,
mode: 0o600,
overwrite: true,
maxBytes: 2,
input: { kind: "buffer", data: Buffer.from("large") },
}),
).rejects.toMatchObject({ code: "too-large" });
await expect(fs.stat(path.join(root, "nested", "too-large.txt"))).rejects.toMatchObject({
code: "ENOENT",
});
},
);
it.runIf(process.platform === "win32")(
"rejects with unsupported-platform on windows",
async () => {
// fd-relative pinned filesystem operations are unavailable on windows
// (see src/pinned-python.ts), so the helper fails closed before any
// posix-only logic runs.
const { runPinnedWriteHelper } = await import("../src/pinned-write.js");
const root = await tempRoot("fs-safe-pinned-write-fallback-");
await expect(
runPinnedWriteHelper({
rootPath: root,
relativeParentPath: "nested",
basename: "created.txt",
mkdir: true,
mode: 0o600,
overwrite: false,
input: { kind: "buffer", data: "created", encoding: "utf8" },
}),
).rejects.toMatchObject({ code: "unsupported-platform" });
},
);
});

View File

@ -72,4 +72,39 @@ describe("platform fallback coverage", () => {
code: "ENOENT",
});
});
it("prunes empty directories through the Windows remove fallback", async () => {
await importRootForPlatform("win32");
const { fileStore } = await import("../src/file-store.js");
const rootDir = await tempRoot("fs-safe-win-prune-");
const store = fileStore({ rootDir });
const stalePath = path.join(rootDir, "old", "stale.txt");
await fs.mkdir(path.dirname(stalePath), { recursive: true });
await fs.writeFile(stalePath, "stale", "utf8");
await fs.utimes(stalePath, new Date(0), new Date(0));
await store.pruneExpired({ ttlMs: 1, recursive: true, pruneEmptyDirs: true });
// Root.remove's Node fallback must use rmdir for empty directories; fs.rm
// without recursive rejects dirs and would silently leave pruneEmptyDirs work.
await expect(fs.stat(path.join(rootDir, "old"))).rejects.toMatchObject({ code: "ENOENT" });
});
it.runIf(process.platform !== "win32")("rejects symlinked missing mkdir components in fallback", async () => {
const { root: openRoot } = await importRootForPlatform("win32");
const { __setFsSafeTestHooksForTest } = await import("../src/test-hooks.js");
const rootDir = await tempRoot("fs-safe-win-mkdir-race-");
const outsideDir = await tempRoot("fs-safe-win-mkdir-outside-");
const scoped = await openRoot(rootDir);
const racedComponent = path.join(rootDir, "link");
__setFsSafeTestHooksForTest({
async beforeRootFallbackMutation(operation, targetPath) {
if (operation !== "mkdir" || path.basename(targetPath) !== "link") return;
await fs.symlink(outsideDir, targetPath, "dir");
},
});
await expect(scoped.mkdir("link/created")).rejects.toBeTruthy();
await expect(fs.stat(path.join(outsideDir, "created"))).rejects.toMatchObject({ code: "ENOENT" });
});
});

View File

@ -1,61 +1,23 @@
import fs from "node:fs";
import fsp from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { resolveAbsolutePathForRead } from "../src/absolute-path.js";
import { FsSafeError, root as openRoot } from "../src/index.js";
import { root as openRoot } from "../src/index.js";
import { openPinnedFileSync } from "../src/pinned-open.js";
import { pathScope } from "../src/root-paths.js";
import { openRootFile, openRootFileSync } from "../src/root-file.js";
type TempLayout = {
outside: string;
outsideFile: string;
root: string;
};
import {
expectFsSafeCode,
LIST_TRAVERSAL_PAYLOADS,
makeTempLayout as makeSecurityTempLayout,
TRAVERSAL_PAYLOADS,
} from "./helpers/security.js";
const tempDirs: string[] = [];
const TRAVERSAL_PAYLOADS = [
"../secret.txt",
"../../secret.txt",
"nested/../../secret.txt",
"nested/../../../secret.txt",
"./../secret.txt",
"nested/..//../secret.txt",
"nested/%2e%2e/secret.txt",
"%2e%2e/secret.txt",
"%2e%2e%2fsecret.txt",
"..%2fsecret.txt",
"%252e%252e%252fsecret.txt",
"..%00/secret.txt",
"..\\secret.txt",
"nested\\..\\..\\secret.txt",
"C:\\Windows\\win.ini",
"\\\\server\\share\\secret.txt",
] as const;
const LIST_TRAVERSAL_PAYLOADS = [
"..",
"../",
"../../",
"nested/../..",
"nested/../../outside",
"%2e%2e",
"%2e%2e%2f",
"..\\",
"C:\\Windows",
"\\\\server\\share",
] as const;
async function makeTempLayout(prefix: string): Promise<TempLayout> {
const root = await fsp.mkdtemp(path.join(os.tmpdir(), `${prefix}-root-`));
const outside = await fsp.mkdtemp(path.join(os.tmpdir(), `${prefix}-outside-`));
tempDirs.push(root, outside);
const outsideFile = path.join(outside, "secret.txt");
await fsp.writeFile(outsideFile, "outside secret");
return { outside, outsideFile, root };
async function makeTempLayout(prefix: string) {
return await makeSecurityTempLayout(prefix, tempDirs);
}
async function closeIfOpen(value: unknown): Promise<void> {
@ -67,16 +29,11 @@ async function closeIfOpen(value: unknown): Promise<void> {
}
}
function expectFsSafeCode(error: unknown, codes: readonly string[]): void {
expect(error).toBeInstanceOf(FsSafeError);
expect(codes).toContain((error as FsSafeError).code);
}
afterEach(async () => {
await Promise.all(tempDirs.splice(0).map((dir) => fsp.rm(dir, { force: true, recursive: true })));
});
describe("OpenClaw read bypass parity", () => {
describe("read boundary bypass attempts", () => {
it("rejects a payload corpus of traversal, encoded, NUL, Windows, and UNC read attempts", async () => {
const layout = await makeTempLayout("fs-safe-read-payloads");
await fsp.mkdir(path.join(layout.root, "nested"), { recursive: true });
@ -115,11 +72,15 @@ describe("OpenClaw read bypass parity", () => {
return true;
});
await expect(safeRoot.stat("../secret.txt")).rejects.toSatisfy((error: unknown) => {
expectFsSafeCode(error, ["outside-workspace", "invalid-path", "path-alias"]);
expectFsSafeCode(error, ["outside-workspace", "invalid-path", "path-alias"], {
allowUnsupportedPlatformOnWindows: true,
});
return true;
});
await expect(safeRoot.list(".." as string)).rejects.toSatisfy((error: unknown) => {
expectFsSafeCode(error, ["outside-workspace", "invalid-path", "path-alias"]);
expectFsSafeCode(error, ["outside-workspace", "invalid-path", "path-alias"], {
allowUnsupportedPlatformOnWindows: true,
});
return true;
});
await expect(scope.files(["../secret.txt"])).resolves.toMatchObject({ ok: false });
@ -139,11 +100,15 @@ describe("OpenClaw read bypass parity", () => {
return true;
});
await expect(safeRoot.stat("link/secret.txt")).rejects.toSatisfy((error: unknown) => {
expectFsSafeCode(error, ["outside-workspace", "path-alias", "symlink"]);
expectFsSafeCode(error, ["outside-workspace", "path-alias", "symlink"], {
allowUnsupportedPlatformOnWindows: true,
});
return true;
});
await expect(safeRoot.list("link")).rejects.toSatisfy((error: unknown) => {
expectFsSafeCode(error, ["outside-workspace", "path-alias", "symlink"]);
expectFsSafeCode(error, ["outside-workspace", "path-alias", "symlink"], {
allowUnsupportedPlatformOnWindows: true,
});
return true;
});
});
@ -163,7 +128,9 @@ describe("OpenClaw read bypass parity", () => {
return true;
});
await expect(safeRoot.stat("secret-link.txt")).rejects.toSatisfy((error: unknown) => {
expectFsSafeCode(error, ["outside-workspace", "path-alias", "symlink"]);
expectFsSafeCode(error, ["outside-workspace", "path-alias", "symlink"], {
allowUnsupportedPlatformOnWindows: true,
});
return true;
});
@ -211,7 +178,9 @@ describe("OpenClaw read bypass parity", () => {
return true;
});
await expect(safeRoot.stat(layout.outsideFile)).rejects.toSatisfy((error: unknown) => {
expectFsSafeCode(error, ["outside-workspace", "path-alias", "invalid-path"]);
expectFsSafeCode(error, ["outside-workspace", "path-alias", "invalid-path"], {
allowUnsupportedPlatformOnWindows: true,
});
return true;
});

View File

@ -1,4 +1,5 @@
import fs from "node:fs/promises";
import { realpathSync } from "node:fs";
import os from "node:os";
import path from "node:path";
import { describe, expect, it } from "vitest";
@ -173,7 +174,10 @@ describe("local roots helpers", () => {
const missingPath = path.join(uploadsDir, "new", "later.txt");
const outsidePath = path.join(baseDir, "outside.txt");
await fs.writeFile(filePath, "ok", "utf8");
const uploadsReal = await fs.realpath(uploadsDir);
// Use the sync realpath to compare against resolveLocalPathFromRootsSync.
// On windows fs.realpathSync and fs.realpath (async) sometimes disagree
// on 8.3 short-name canonicalization (e.g. "RUNNER~1" vs "runneradmin").
const uploadsReal = realpathSync(uploadsDir);
expect(
resolveLocalPathFromRootsSync({
@ -182,7 +186,7 @@ describe("local roots helpers", () => {
label: "media roots",
requireFile: true,
}),
).toEqual({ path: await fs.realpath(filePath), root: uploadsReal });
).toEqual({ path: realpathSync(filePath), root: uploadsReal });
expect(
resolveLocalPathFromRootsSync({
filePath: missingPath,

View File

@ -68,9 +68,15 @@ describe("secret file helpers", () => {
const root = await tempRoot("fs-safe-secret-");
const target = path.join(root, "target.txt");
const link = path.join(root, "link.txt");
const broken = path.join(root, "broken.txt");
await fs.writeFile(target, "secret", "utf8");
await fs.symlink(target, link);
await fs.symlink(path.join(root, "missing.txt"), broken);
expect(readSecretFileSync(link, "API token")).toBe("secret");
expect(tryReadSecretFileSync(link, "API token")).toBe("secret");
expectSecretReadCode(() => readSecretFileSync(broken, "API token"), "not-found");
expect(tryReadSecretFileSync(broken, "API token")).toBeUndefined();
expect(() => readSecretFileSync(link, "API token", { rejectSymlink: true })).toThrow(
`API token file at ${link} must not be a symlink.`,
);

Some files were not shown because too many files have changed in this diff Show More