Compare commits

...

3 Commits

Author SHA1 Message Date
Peter Steinberger
d144817e7c
fix: preserve external output path spelling (#7) (thanks @jesse-merhi) 2026-05-07 03:03:33 +01:00
jesse-merhi
a6c7e86dc2
test: cover external output traversal rejection 2026-05-07 03:02:57 +01:00
jesse-merhi
11d28f9130
feat: add safe external output writer 2026-05-07 03:02:57 +01:00
9 changed files with 441 additions and 2 deletions

View File

@ -4,6 +4,7 @@
### Fixes
- Add `writeExternalFileWithinRoot()` for libraries that require an output path while preserving caller-provided destination names. (#7; thanks @jesse-merhi)
- Reject `fileStore()` and `fileStoreSync()` writes through symlinked parent directories so store commits cannot escape the configured root.
- Harden Root fallback mutators, archive merges, private store reads/writes, durable queue ids, JSON fallback writes, sibling temp writes, temp filename sanitization, and trash moves against symlink-swap and path traversal edge cases.
- Centralize safe path segment validation, directory identity guards, and guarded mutation wrappers so future filesystem helpers reuse the same race-resistant checks.

View File

@ -24,7 +24,7 @@ Full docs and reference at **[fs-safe.io](https://fs-safe.io)**.
## Contents
[Why this exists](#why-this-exists) · [Not a sandbox](#not-a-sandbox) · [Install](#install) · [Quick start](#quick-start) · [Reading](#reading) · [Subpaths](#subpaths) · [Failure semantics](#failure-semantics-in-the-name) · [Atomic writes](#atomic-writes) · [Stores](#stores) · [Secure absolute reads](#secure-absolute-file-reads) · [Walking](#directory-walking) · [Archive extraction](#archive-extraction) · [Path scopes](#advanced-path-scopes) · [Errors](#errors) · [Safety model](#safety-model) · [Limitations](#limitations)
[Why this exists](#why-this-exists) · [Not a sandbox](#not-a-sandbox) · [Install](#install) · [Quick start](#quick-start) · [Reading](#reading) · [Subpaths](#subpaths) · [Failure semantics](#failure-semantics-in-the-name) · [Atomic writes](#atomic-writes) · [External outputs](#external-outputs) · [Stores](#stores) · [Secure absolute reads](#secure-absolute-file-reads) · [Walking](#directory-walking) · [Archive extraction](#archive-extraction) · [Path scopes](#advanced-path-scopes) · [Errors](#errors) · [Safety model](#safety-model) · [Limitations](#limitations)
## Why this exists
@ -172,6 +172,7 @@ that OpenClaw needs to compose higher-level APIs are grouped under
| `@openclaw/fs-safe/config` | process-global Python helper configuration |
| `@openclaw/fs-safe/path` | canonical path checks: `isPathInside`, `safeRealpathSync`, `isNotFoundPathError`, `isSymlinkOpenError` |
| `@openclaw/fs-safe/json` | `tryReadJson`, `readJson`, `readJsonIfExists`, `writeJson`, sync variants |
| `@openclaw/fs-safe/output` | `writeExternalFileWithinRoot` for external libraries that need a temp output path |
| `@openclaw/fs-safe/store` | `fileStore`, `fileStoreSync`, and `jsonStore` |
| `@openclaw/fs-safe/secret` | strict and try-style secret file read/write helpers |
| `@openclaw/fs-safe/atomic` | `replaceFileAtomic`, `replaceFileAtomicSync`, `replaceDirectoryAtomic`, `movePathWithCopyFallback` |
@ -220,6 +221,28 @@ await replaceFileAtomic({
`replaceFileAtomicSync()` covers the synchronous case with the same options shape. Both accept an injectable `fileSystem` for tests.
## External outputs
Use `writeExternalFileWithinRoot()` when a browser download, renderer, media
tool, or native library needs an absolute path to write to:
```ts
import { writeExternalFileWithinRoot } from "@openclaw/fs-safe/output";
await writeExternalFileWithinRoot({
rootDir: "/safe/workspace/downloads",
path: "reports/today.pdf",
write: async (filePath) => {
await download.saveAs(filePath);
},
});
```
The callback receives a private temp file path, not the final destination. After
the callback returns, fs-safe finalizes the staged file with `Root.copyIn()`,
creating missing parents by default and rejecting traversal, symlink parent
escapes, hardlinked final targets, and size-limit violations.
## Stores
Use `fileStore().json()` for small state files that need explicit fallback

View File

@ -52,6 +52,7 @@ await fs.remove("notes/archive/today.txt");
| [`@openclaw/fs-safe/config`](config.md) | Process-global Python helper configuration (`configureFsSafePython`, `getFsSafePythonConfig`). |
| [Python helper policy](python-helper.md) | Choose `auto`, `off`, or `require` for POSIX fd-relative hardening. |
| [`replaceFileAtomic`](atomic.md) | Sibling-temp + rename, fsync hooks, mode preservation, copy fallback. |
| [`writeExternalFileWithinRoot`](output.md) | Stage external-library file output in private temp storage, then finalize under a root. |
| [`writeJson` / `readJson*`](json.md) | JSON state files with strict and lenient read variants. |
| [`@openclaw/fs-safe/store`](store.md) | Overview of `fileStore`, `fileStoreSync`, and `jsonStore`. |
| [`jsonStore`](json-store.md) | Single JSON state file with explicit fallback, atomic writes, and optional locking. |

82
docs/output.md Normal file
View File

@ -0,0 +1,82 @@
# External outputs
`@openclaw/fs-safe/output` covers the case where another library insists on
writing to an absolute path you give it. Browser downloads, renderers, media
tools, and native libraries often have this shape:
```ts
import { writeExternalFileWithinRoot } from "@openclaw/fs-safe/output";
await writeExternalFileWithinRoot({
rootDir: "/srv/workspace/downloads",
path: "reports/today.pdf",
write: async (filePath) => {
await download.saveAs(filePath);
},
});
```
The external writer never receives the final destination path. It receives a
private temp file path instead. After the callback returns, fs-safe copies that
staged file into the requested target through the same root boundary used by
`Root.copyIn()`.
## Signature
```ts
function writeExternalFileWithinRoot<T = void>(
options: ExternalFileWriteOptions<T>,
): Promise<ExternalFileWriteResult<T>>;
type ExternalFileWriteOptions<T = void> = {
rootDir: string;
path: string; // relative or absolute, but must stay under rootDir
write: (filePath: string) => Promise<T>;
maxBytes?: number;
mode?: number;
};
type ExternalFileWriteResult<T = void> = {
path: string; // final absolute path under the canonical root
result: T; // value returned by write()
};
```
The requested `path` must name a file. Missing destination parents are created
by the helper because the operation is "produce this output file under the
root"; callers should choose the filename before calling this API.
## Why not pass the final path to the library?
If a target parent can be swapped after validation, handing an external library
the final path can make the library write outside the intended root before
fs-safe has a chance to finalize or reject the operation. This helper stages in
a private temp workspace first, then finalizes with `Root.copyIn()`. That keeps
the trust-boundary write inside fs-safe's root-aware copy/atomic-write path.
## Browser download example
```ts
const outputPath = requestedOutputPath || sanitizeBrowserSuggestedName(suggestedFilename);
await writeExternalFileWithinRoot({
rootDir: downloadsRoot,
path: outputPath,
maxBytes: 512 * 1024 * 1024,
write: async (filePath) => {
await download.saveAs(filePath);
},
});
```
The chosen path may be absolute if it is already inside `downloadsRoot`, or
relative to `downloadsRoot`. Traversal, symlink parent escapes, hardlinked final
targets, over-large staged files, and missing temp files surface as
`FsSafeError`s.
## See also
- [Root writes](writing.md) — `write`, `copyIn`, `move`, and `mkdir`.
- [Temp workspaces](temp.md) — private scratch directories for longer workflows.
- [`pathScope()`](path-scope.md) — validation-only helper when you must pass an
absolute path directly to another library.

View File

@ -36,6 +36,10 @@
"types": "./dist/path.d.ts",
"default": "./dist/path.js"
},
"./output": {
"types": "./dist/output.d.ts",
"default": "./dist/output.js"
},
"./advanced": {
"types": "./dist/advanced.d.ts",
"default": "./dist/advanced.js"

View File

@ -28,7 +28,7 @@ const installCmd = "pnpm add @openclaw/fs-safe";
const sections = [
["Start", ["index.md", "install.md", "quickstart.md", "security-model.md", "python-helper.md", "config.md"]],
["Root API", ["root.md", "reading.md", "writing.md", "path-scope.md"]],
["Atomic & temp", ["atomic.md", "json.md", "temp.md", "archive.md"]],
["Atomic & temp", ["atomic.md", "output.md", "json.md", "temp.md", "archive.md"]],
["Stores", ["store.md", "json-store.md", "file-store.md", "private-file-store.md"]],
["Specialized", ["secret-file.md", "regular-file.md", "sidecar-lock.md", "pinned-open.md", "local-roots.md"]],
["Path & filename", ["path.md", "filename.md", "install-path.md"]],

View File

@ -32,3 +32,8 @@ export {
type FsSafePythonConfig,
type FsSafePythonMode,
} from "./pinned-python-config.js";
export {
writeExternalFileWithinRoot,
type ExternalFileWriteOptions,
type ExternalFileWriteResult,
} from "./output.js";

96
src/output.ts Normal file
View File

@ -0,0 +1,96 @@
import path from "node:path";
import { FsSafeError } from "./errors.js";
import { sanitizeUntrustedFileName } from "./filename.js";
import { isPathInside } from "./path.js";
import { root } from "./root.js";
import { tempFile } from "./temp-target.js";
export type ExternalFileWriteOptions<T = void> = {
rootDir: string;
path: string;
write: (filePath: string) => Promise<T>;
maxBytes?: number;
mode?: number;
};
export type ExternalFileWriteResult<T = void> = {
path: string;
result: T;
};
function tempFileNameForTarget(targetPath: string): string {
return sanitizeUntrustedFileName(path.basename(targetPath), "output.bin");
}
function ensureTrailingSep(value: string): string {
return value.endsWith(path.sep) ? value : `${value}${path.sep}`;
}
function toRootPathInput(params: {
rootDir: string;
rootReal: string;
targetPath: string;
}): string {
if (!path.isAbsolute(params.targetPath)) {
return params.targetPath;
}
const absoluteTarget = path.resolve(params.targetPath);
const rootDir = path.resolve(params.rootDir);
if (isPathInside(ensureTrailingSep(rootDir), absoluteTarget)) {
return path.relative(rootDir, absoluteTarget);
}
if (isPathInside(ensureTrailingSep(params.rootReal), absoluteTarget)) {
return path.relative(params.rootReal, absoluteTarget);
}
return params.targetPath;
}
function assertFileTargetPath(targetPath: string): void {
const basename = path.basename(targetPath);
if (
!targetPath ||
targetPath === "." ||
targetPath.endsWith("/") ||
targetPath.endsWith("\\") ||
!basename ||
basename === "." ||
basename === ".."
) {
throw new FsSafeError("invalid-path", "target path must name a file");
}
}
export async function writeExternalFileWithinRoot<T = void>(
options: ExternalFileWriteOptions<T>,
): Promise<ExternalFileWriteResult<T>> {
const targetRoot = await root(options.rootDir);
const requestedTargetPath = options.path;
if (requestedTargetPath.length === 0) {
throw new FsSafeError("invalid-path", "target path is required");
}
const targetPath = toRootPathInput({
rootDir: targetRoot.rootDir,
rootReal: targetRoot.rootReal,
targetPath: requestedTargetPath,
});
assertFileTargetPath(targetPath);
const finalPath = await targetRoot.resolve(targetPath);
const staged = await tempFile({
prefix: "fs-safe-output",
fileName: tempFileNameForTarget(targetPath),
});
try {
const result = await options.write(staged.path);
await targetRoot.copyIn(targetPath, staged.path, {
maxBytes: options.maxBytes,
mode: options.mode,
mkdir: true,
sourceHardlinks: "reject",
});
return { path: finalPath, result };
} finally {
await staged.cleanup();
}
}

227
test/output.test.ts Normal file
View File

@ -0,0 +1,227 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { writeExternalFileWithinRoot } from "../src/output.js";
const tempDirs = new Set<string>();
async function tempRoot(prefix: string): Promise<string> {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), prefix));
tempDirs.add(dir);
return dir;
}
afterEach(async () => {
for (const dir of tempDirs) {
await fs.rm(dir, { force: true, recursive: true });
}
tempDirs.clear();
});
describe("writeExternalFileWithinRoot", () => {
it("stages an external writer in private temp storage and finalizes under the root", async () => {
const rootDir = await tempRoot("fs-safe-output-root-");
const targetPath = path.join(rootDir, "downloads", "report.txt");
let tempPath = "";
const result = await writeExternalFileWithinRoot({
rootDir,
path: targetPath,
write: async (candidate) => {
tempPath = candidate;
await fs.writeFile(candidate, "downloaded", "utf8");
return { bytes: 10 };
},
});
expect(result.path).toBe(path.join(await fs.realpath(rootDir), "downloads", "report.txt"));
expect(result.result).toEqual({ bytes: 10 });
expect(path.dirname(tempPath)).not.toBe(path.dirname(targetPath));
await expect(fs.readFile(targetPath, "utf8")).resolves.toBe("downloaded");
await expect(fs.stat(tempPath)).rejects.toMatchObject({ code: "ENOENT" });
});
it("preserves caller-provided destination filename spacing", async () => {
const rootDir = await tempRoot("fs-safe-output-spaces-");
const fileName = " report .txt ";
const result = await writeExternalFileWithinRoot({
rootDir,
path: fileName,
write: async (candidate) => {
await fs.writeFile(candidate, "spaced", "utf8");
},
});
const finalPath = path.join(rootDir, fileName);
expect(result.path).toBe(path.join(await fs.realpath(rootDir), fileName));
await expect(fs.readFile(finalPath, "utf8")).resolves.toBe("spaced");
await expect(fs.stat(path.join(rootDir, fileName.trim()))).rejects.toMatchObject({
code: "ENOENT",
});
});
it("rejects empty target paths before invoking the external writer", async () => {
const rootDir = await tempRoot("fs-safe-output-default-");
let called = false;
await expect(
writeExternalFileWithinRoot({
rootDir,
path: "",
write: async (candidate) => {
called = true;
await fs.writeFile(candidate, "named", "utf8");
},
}),
).rejects.toMatchObject({ code: "invalid-path" });
expect(called).toBe(false);
});
it("rejects targets outside the root before invoking the external writer", async () => {
const rootDir = await tempRoot("fs-safe-output-reject-root-");
const outsideDir = await tempRoot("fs-safe-output-reject-outside-");
const outsidePath = path.join(outsideDir, "pwned.txt");
let called = false;
await expect(
writeExternalFileWithinRoot({
rootDir,
path: outsidePath,
write: async (candidate) => {
called = true;
await fs.writeFile(candidate, "pwned", "utf8");
},
}),
).rejects.toMatchObject({ code: "outside-workspace" });
expect(called).toBe(false);
await expect(fs.stat(outsidePath)).rejects.toMatchObject({ code: "ENOENT" });
});
it("rejects traversal targets before invoking the external writer", async () => {
const rootDir = await tempRoot("fs-safe-output-traversal-root-");
let called = false;
await expect(
writeExternalFileWithinRoot({
rootDir,
path: "../../../pwned.txt",
write: async (candidate) => {
called = true;
await fs.writeFile(candidate, "pwned", "utf8");
},
}),
).rejects.toMatchObject({ code: "outside-workspace" });
expect(called).toBe(false);
});
it("rejects root directory targets before invoking the external writer", async () => {
const rootDir = await tempRoot("fs-safe-output-root-target-");
let called = false;
await expect(
writeExternalFileWithinRoot({
rootDir,
path: rootDir,
write: async (candidate) => {
called = true;
await fs.writeFile(candidate, "not a file target", "utf8");
},
}),
).rejects.toMatchObject({ code: "invalid-path" });
expect(called).toBe(false);
});
it("rejects trailing-separator targets before invoking the external writer", async () => {
const rootDir = await tempRoot("fs-safe-output-dir-target-");
let called = false;
await expect(
writeExternalFileWithinRoot({
rootDir,
path: "nested/",
write: async (candidate) => {
called = true;
await fs.writeFile(candidate, "not a file target", "utf8");
},
}),
).rejects.toMatchObject({ code: "invalid-path" });
expect(called).toBe(false);
});
it.runIf(process.platform !== "win32")(
"does not let symlinked target parents redirect the external temp write",
async () => {
const rootDir = await tempRoot("fs-safe-output-link-root-");
const outsideDir = await tempRoot("fs-safe-output-link-outside-");
await fs.symlink(outsideDir, path.join(rootDir, "link"), "dir");
let tempPath = "";
await expect(
writeExternalFileWithinRoot({
rootDir,
path: "link/out.txt",
write: async (candidate) => {
tempPath = candidate;
await fs.writeFile(candidate, "pwned", "utf8");
},
}),
).rejects.toBeTruthy();
await expect(fs.stat(tempPath)).rejects.toMatchObject({ code: "ENOENT" });
await expect(fs.readdir(outsideDir)).resolves.toEqual([]);
},
);
it.runIf(process.platform !== "win32")(
"rejects hardlinked final targets and preserves the existing file",
async () => {
const rootDir = await tempRoot("fs-safe-output-hardlink-");
const sourcePath = path.join(rootDir, "source.txt");
const hardlinkPath = path.join(rootDir, "hardlink.txt");
await fs.writeFile(sourcePath, "original", "utf8");
await fs.link(sourcePath, hardlinkPath);
await expect(
writeExternalFileWithinRoot({
rootDir,
path: "hardlink.txt",
write: async (candidate) => {
await fs.writeFile(candidate, "replacement", "utf8");
},
}),
).rejects.toBeTruthy();
await expect(fs.readFile(sourcePath, "utf8")).resolves.toBe("original");
await expect(fs.readFile(hardlinkPath, "utf8")).resolves.toBe("original");
},
);
it("cleans private temp files when the external writer fails", async () => {
const rootDir = await tempRoot("fs-safe-output-fail-root-");
let tempPath = "";
await expect(
writeExternalFileWithinRoot({
rootDir,
path: "out.txt",
write: async (candidate) => {
tempPath = candidate;
await fs.writeFile(candidate, "partial", "utf8");
throw new Error("download failed");
},
}),
).rejects.toThrow("download failed");
await expect(fs.stat(tempPath)).rejects.toMatchObject({ code: "ENOENT" });
await expect(fs.stat(path.join(rootDir, "out.txt"))).rejects.toMatchObject({
code: "ENOENT",
});
});
});