Compare commits
3 Commits
main
...
codex/exte
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d144817e7c | ||
|
|
a6c7e86dc2 | ||
|
|
11d28f9130 |
@ -4,6 +4,7 @@
|
||||
|
||||
### Fixes
|
||||
|
||||
- Add `writeExternalFileWithinRoot()` for libraries that require an output path while preserving caller-provided destination names. (#7; thanks @jesse-merhi)
|
||||
- Reject `fileStore()` and `fileStoreSync()` writes through symlinked parent directories so store commits cannot escape the configured root.
|
||||
- Harden Root fallback mutators, archive merges, private store reads/writes, durable queue ids, JSON fallback writes, sibling temp writes, temp filename sanitization, and trash moves against symlink-swap and path traversal edge cases.
|
||||
- Centralize safe path segment validation, directory identity guards, and guarded mutation wrappers so future filesystem helpers reuse the same race-resistant checks.
|
||||
|
||||
25
README.md
25
README.md
@ -24,7 +24,7 @@ Full docs and reference at **[fs-safe.io](https://fs-safe.io)**.
|
||||
|
||||
## Contents
|
||||
|
||||
[Why this exists](#why-this-exists) · [Not a sandbox](#not-a-sandbox) · [Install](#install) · [Quick start](#quick-start) · [Reading](#reading) · [Subpaths](#subpaths) · [Failure semantics](#failure-semantics-in-the-name) · [Atomic writes](#atomic-writes) · [Stores](#stores) · [Secure absolute reads](#secure-absolute-file-reads) · [Walking](#directory-walking) · [Archive extraction](#archive-extraction) · [Path scopes](#advanced-path-scopes) · [Errors](#errors) · [Safety model](#safety-model) · [Limitations](#limitations)
|
||||
[Why this exists](#why-this-exists) · [Not a sandbox](#not-a-sandbox) · [Install](#install) · [Quick start](#quick-start) · [Reading](#reading) · [Subpaths](#subpaths) · [Failure semantics](#failure-semantics-in-the-name) · [Atomic writes](#atomic-writes) · [External outputs](#external-outputs) · [Stores](#stores) · [Secure absolute reads](#secure-absolute-file-reads) · [Walking](#directory-walking) · [Archive extraction](#archive-extraction) · [Path scopes](#advanced-path-scopes) · [Errors](#errors) · [Safety model](#safety-model) · [Limitations](#limitations)
|
||||
|
||||
## Why this exists
|
||||
|
||||
@ -172,6 +172,7 @@ that OpenClaw needs to compose higher-level APIs are grouped under
|
||||
| `@openclaw/fs-safe/config` | process-global Python helper configuration |
|
||||
| `@openclaw/fs-safe/path` | canonical path checks: `isPathInside`, `safeRealpathSync`, `isNotFoundPathError`, `isSymlinkOpenError` |
|
||||
| `@openclaw/fs-safe/json` | `tryReadJson`, `readJson`, `readJsonIfExists`, `writeJson`, sync variants |
|
||||
| `@openclaw/fs-safe/output` | `writeExternalFileWithinRoot` for external libraries that need a temp output path |
|
||||
| `@openclaw/fs-safe/store` | `fileStore`, `fileStoreSync`, and `jsonStore` |
|
||||
| `@openclaw/fs-safe/secret` | strict and try-style secret file read/write helpers |
|
||||
| `@openclaw/fs-safe/atomic` | `replaceFileAtomic`, `replaceFileAtomicSync`, `replaceDirectoryAtomic`, `movePathWithCopyFallback` |
|
||||
@ -220,6 +221,28 @@ await replaceFileAtomic({
|
||||
|
||||
`replaceFileAtomicSync()` covers the synchronous case with the same options shape. Both accept an injectable `fileSystem` for tests.
|
||||
|
||||
## External outputs
|
||||
|
||||
Use `writeExternalFileWithinRoot()` when a browser download, renderer, media
|
||||
tool, or native library needs an absolute path to write to:
|
||||
|
||||
```ts
|
||||
import { writeExternalFileWithinRoot } from "@openclaw/fs-safe/output";
|
||||
|
||||
await writeExternalFileWithinRoot({
|
||||
rootDir: "/safe/workspace/downloads",
|
||||
path: "reports/today.pdf",
|
||||
write: async (filePath) => {
|
||||
await download.saveAs(filePath);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
The callback receives a private temp file path, not the final destination. After
|
||||
the callback returns, fs-safe finalizes the staged file with `Root.copyIn()`,
|
||||
creating missing parents by default and rejecting traversal, symlink parent
|
||||
escapes, hardlinked final targets, and size-limit violations.
|
||||
|
||||
## Stores
|
||||
|
||||
Use `fileStore().json()` for small state files that need explicit fallback
|
||||
|
||||
@ -52,6 +52,7 @@ await fs.remove("notes/archive/today.txt");
|
||||
| [`@openclaw/fs-safe/config`](config.md) | Process-global Python helper configuration (`configureFsSafePython`, `getFsSafePythonConfig`). |
|
||||
| [Python helper policy](python-helper.md) | Choose `auto`, `off`, or `require` for POSIX fd-relative hardening. |
|
||||
| [`replaceFileAtomic`](atomic.md) | Sibling-temp + rename, fsync hooks, mode preservation, copy fallback. |
|
||||
| [`writeExternalFileWithinRoot`](output.md) | Stage external-library file output in private temp storage, then finalize under a root. |
|
||||
| [`writeJson` / `readJson*`](json.md) | JSON state files with strict and lenient read variants. |
|
||||
| [`@openclaw/fs-safe/store`](store.md) | Overview of `fileStore`, `fileStoreSync`, and `jsonStore`. |
|
||||
| [`jsonStore`](json-store.md) | Single JSON state file with explicit fallback, atomic writes, and optional locking. |
|
||||
|
||||
82
docs/output.md
Normal file
82
docs/output.md
Normal file
@ -0,0 +1,82 @@
|
||||
# External outputs
|
||||
|
||||
`@openclaw/fs-safe/output` covers the case where another library insists on
|
||||
writing to an absolute path you give it. Browser downloads, renderers, media
|
||||
tools, and native libraries often have this shape:
|
||||
|
||||
```ts
|
||||
import { writeExternalFileWithinRoot } from "@openclaw/fs-safe/output";
|
||||
|
||||
await writeExternalFileWithinRoot({
|
||||
rootDir: "/srv/workspace/downloads",
|
||||
path: "reports/today.pdf",
|
||||
write: async (filePath) => {
|
||||
await download.saveAs(filePath);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
The external writer never receives the final destination path. It receives a
|
||||
private temp file path instead. After the callback returns, fs-safe copies that
|
||||
staged file into the requested target through the same root boundary used by
|
||||
`Root.copyIn()`.
|
||||
|
||||
## Signature
|
||||
|
||||
```ts
|
||||
function writeExternalFileWithinRoot<T = void>(
|
||||
options: ExternalFileWriteOptions<T>,
|
||||
): Promise<ExternalFileWriteResult<T>>;
|
||||
|
||||
type ExternalFileWriteOptions<T = void> = {
|
||||
rootDir: string;
|
||||
path: string; // relative or absolute, but must stay under rootDir
|
||||
write: (filePath: string) => Promise<T>;
|
||||
maxBytes?: number;
|
||||
mode?: number;
|
||||
};
|
||||
|
||||
type ExternalFileWriteResult<T = void> = {
|
||||
path: string; // final absolute path under the canonical root
|
||||
result: T; // value returned by write()
|
||||
};
|
||||
```
|
||||
|
||||
The requested `path` must name a file. Missing destination parents are created
|
||||
by the helper because the operation is "produce this output file under the
|
||||
root"; callers should choose the filename before calling this API.
|
||||
|
||||
## Why not pass the final path to the library?
|
||||
|
||||
If a target parent can be swapped after validation, handing an external library
|
||||
the final path can make the library write outside the intended root before
|
||||
fs-safe has a chance to finalize or reject the operation. This helper stages in
|
||||
a private temp workspace first, then finalizes with `Root.copyIn()`. That keeps
|
||||
the trust-boundary write inside fs-safe's root-aware copy/atomic-write path.
|
||||
|
||||
## Browser download example
|
||||
|
||||
```ts
|
||||
const outputPath = requestedOutputPath || sanitizeBrowserSuggestedName(suggestedFilename);
|
||||
|
||||
await writeExternalFileWithinRoot({
|
||||
rootDir: downloadsRoot,
|
||||
path: outputPath,
|
||||
maxBytes: 512 * 1024 * 1024,
|
||||
write: async (filePath) => {
|
||||
await download.saveAs(filePath);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
The chosen path may be absolute if it is already inside `downloadsRoot`, or
|
||||
relative to `downloadsRoot`. Traversal, symlink parent escapes, hardlinked final
|
||||
targets, over-large staged files, and missing temp files surface as
|
||||
`FsSafeError`s.
|
||||
|
||||
## See also
|
||||
|
||||
- [Root writes](writing.md) — `write`, `copyIn`, `move`, and `mkdir`.
|
||||
- [Temp workspaces](temp.md) — private scratch directories for longer workflows.
|
||||
- [`pathScope()`](path-scope.md) — validation-only helper when you must pass an
|
||||
absolute path directly to another library.
|
||||
@ -36,6 +36,10 @@
|
||||
"types": "./dist/path.d.ts",
|
||||
"default": "./dist/path.js"
|
||||
},
|
||||
"./output": {
|
||||
"types": "./dist/output.d.ts",
|
||||
"default": "./dist/output.js"
|
||||
},
|
||||
"./advanced": {
|
||||
"types": "./dist/advanced.d.ts",
|
||||
"default": "./dist/advanced.js"
|
||||
|
||||
@ -28,7 +28,7 @@ const installCmd = "pnpm add @openclaw/fs-safe";
|
||||
const sections = [
|
||||
["Start", ["index.md", "install.md", "quickstart.md", "security-model.md", "python-helper.md", "config.md"]],
|
||||
["Root API", ["root.md", "reading.md", "writing.md", "path-scope.md"]],
|
||||
["Atomic & temp", ["atomic.md", "json.md", "temp.md", "archive.md"]],
|
||||
["Atomic & temp", ["atomic.md", "output.md", "json.md", "temp.md", "archive.md"]],
|
||||
["Stores", ["store.md", "json-store.md", "file-store.md", "private-file-store.md"]],
|
||||
["Specialized", ["secret-file.md", "regular-file.md", "sidecar-lock.md", "pinned-open.md", "local-roots.md"]],
|
||||
["Path & filename", ["path.md", "filename.md", "install-path.md"]],
|
||||
|
||||
@ -32,3 +32,8 @@ export {
|
||||
type FsSafePythonConfig,
|
||||
type FsSafePythonMode,
|
||||
} from "./pinned-python-config.js";
|
||||
export {
|
||||
writeExternalFileWithinRoot,
|
||||
type ExternalFileWriteOptions,
|
||||
type ExternalFileWriteResult,
|
||||
} from "./output.js";
|
||||
|
||||
96
src/output.ts
Normal file
96
src/output.ts
Normal file
@ -0,0 +1,96 @@
|
||||
import path from "node:path";
|
||||
import { FsSafeError } from "./errors.js";
|
||||
import { sanitizeUntrustedFileName } from "./filename.js";
|
||||
import { isPathInside } from "./path.js";
|
||||
import { root } from "./root.js";
|
||||
import { tempFile } from "./temp-target.js";
|
||||
|
||||
export type ExternalFileWriteOptions<T = void> = {
|
||||
rootDir: string;
|
||||
path: string;
|
||||
write: (filePath: string) => Promise<T>;
|
||||
maxBytes?: number;
|
||||
mode?: number;
|
||||
};
|
||||
|
||||
export type ExternalFileWriteResult<T = void> = {
|
||||
path: string;
|
||||
result: T;
|
||||
};
|
||||
|
||||
function tempFileNameForTarget(targetPath: string): string {
|
||||
return sanitizeUntrustedFileName(path.basename(targetPath), "output.bin");
|
||||
}
|
||||
|
||||
function ensureTrailingSep(value: string): string {
|
||||
return value.endsWith(path.sep) ? value : `${value}${path.sep}`;
|
||||
}
|
||||
|
||||
function toRootPathInput(params: {
|
||||
rootDir: string;
|
||||
rootReal: string;
|
||||
targetPath: string;
|
||||
}): string {
|
||||
if (!path.isAbsolute(params.targetPath)) {
|
||||
return params.targetPath;
|
||||
}
|
||||
|
||||
const absoluteTarget = path.resolve(params.targetPath);
|
||||
const rootDir = path.resolve(params.rootDir);
|
||||
if (isPathInside(ensureTrailingSep(rootDir), absoluteTarget)) {
|
||||
return path.relative(rootDir, absoluteTarget);
|
||||
}
|
||||
if (isPathInside(ensureTrailingSep(params.rootReal), absoluteTarget)) {
|
||||
return path.relative(params.rootReal, absoluteTarget);
|
||||
}
|
||||
return params.targetPath;
|
||||
}
|
||||
|
||||
function assertFileTargetPath(targetPath: string): void {
|
||||
const basename = path.basename(targetPath);
|
||||
if (
|
||||
!targetPath ||
|
||||
targetPath === "." ||
|
||||
targetPath.endsWith("/") ||
|
||||
targetPath.endsWith("\\") ||
|
||||
!basename ||
|
||||
basename === "." ||
|
||||
basename === ".."
|
||||
) {
|
||||
throw new FsSafeError("invalid-path", "target path must name a file");
|
||||
}
|
||||
}
|
||||
|
||||
export async function writeExternalFileWithinRoot<T = void>(
|
||||
options: ExternalFileWriteOptions<T>,
|
||||
): Promise<ExternalFileWriteResult<T>> {
|
||||
const targetRoot = await root(options.rootDir);
|
||||
const requestedTargetPath = options.path;
|
||||
if (requestedTargetPath.length === 0) {
|
||||
throw new FsSafeError("invalid-path", "target path is required");
|
||||
}
|
||||
const targetPath = toRootPathInput({
|
||||
rootDir: targetRoot.rootDir,
|
||||
rootReal: targetRoot.rootReal,
|
||||
targetPath: requestedTargetPath,
|
||||
});
|
||||
assertFileTargetPath(targetPath);
|
||||
const finalPath = await targetRoot.resolve(targetPath);
|
||||
const staged = await tempFile({
|
||||
prefix: "fs-safe-output",
|
||||
fileName: tempFileNameForTarget(targetPath),
|
||||
});
|
||||
|
||||
try {
|
||||
const result = await options.write(staged.path);
|
||||
await targetRoot.copyIn(targetPath, staged.path, {
|
||||
maxBytes: options.maxBytes,
|
||||
mode: options.mode,
|
||||
mkdir: true,
|
||||
sourceHardlinks: "reject",
|
||||
});
|
||||
return { path: finalPath, result };
|
||||
} finally {
|
||||
await staged.cleanup();
|
||||
}
|
||||
}
|
||||
227
test/output.test.ts
Normal file
227
test/output.test.ts
Normal file
@ -0,0 +1,227 @@
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import { writeExternalFileWithinRoot } from "../src/output.js";
|
||||
|
||||
const tempDirs = new Set<string>();
|
||||
|
||||
async function tempRoot(prefix: string): Promise<string> {
|
||||
const dir = await fs.mkdtemp(path.join(os.tmpdir(), prefix));
|
||||
tempDirs.add(dir);
|
||||
return dir;
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
for (const dir of tempDirs) {
|
||||
await fs.rm(dir, { force: true, recursive: true });
|
||||
}
|
||||
tempDirs.clear();
|
||||
});
|
||||
|
||||
describe("writeExternalFileWithinRoot", () => {
|
||||
it("stages an external writer in private temp storage and finalizes under the root", async () => {
|
||||
const rootDir = await tempRoot("fs-safe-output-root-");
|
||||
const targetPath = path.join(rootDir, "downloads", "report.txt");
|
||||
let tempPath = "";
|
||||
|
||||
const result = await writeExternalFileWithinRoot({
|
||||
rootDir,
|
||||
path: targetPath,
|
||||
write: async (candidate) => {
|
||||
tempPath = candidate;
|
||||
await fs.writeFile(candidate, "downloaded", "utf8");
|
||||
return { bytes: 10 };
|
||||
},
|
||||
});
|
||||
|
||||
expect(result.path).toBe(path.join(await fs.realpath(rootDir), "downloads", "report.txt"));
|
||||
expect(result.result).toEqual({ bytes: 10 });
|
||||
expect(path.dirname(tempPath)).not.toBe(path.dirname(targetPath));
|
||||
await expect(fs.readFile(targetPath, "utf8")).resolves.toBe("downloaded");
|
||||
await expect(fs.stat(tempPath)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
});
|
||||
|
||||
it("preserves caller-provided destination filename spacing", async () => {
|
||||
const rootDir = await tempRoot("fs-safe-output-spaces-");
|
||||
const fileName = " report .txt ";
|
||||
|
||||
const result = await writeExternalFileWithinRoot({
|
||||
rootDir,
|
||||
path: fileName,
|
||||
write: async (candidate) => {
|
||||
await fs.writeFile(candidate, "spaced", "utf8");
|
||||
},
|
||||
});
|
||||
|
||||
const finalPath = path.join(rootDir, fileName);
|
||||
expect(result.path).toBe(path.join(await fs.realpath(rootDir), fileName));
|
||||
await expect(fs.readFile(finalPath, "utf8")).resolves.toBe("spaced");
|
||||
await expect(fs.stat(path.join(rootDir, fileName.trim()))).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects empty target paths before invoking the external writer", async () => {
|
||||
const rootDir = await tempRoot("fs-safe-output-default-");
|
||||
let called = false;
|
||||
|
||||
await expect(
|
||||
writeExternalFileWithinRoot({
|
||||
rootDir,
|
||||
path: "",
|
||||
write: async (candidate) => {
|
||||
called = true;
|
||||
await fs.writeFile(candidate, "named", "utf8");
|
||||
},
|
||||
}),
|
||||
).rejects.toMatchObject({ code: "invalid-path" });
|
||||
|
||||
expect(called).toBe(false);
|
||||
});
|
||||
|
||||
it("rejects targets outside the root before invoking the external writer", async () => {
|
||||
const rootDir = await tempRoot("fs-safe-output-reject-root-");
|
||||
const outsideDir = await tempRoot("fs-safe-output-reject-outside-");
|
||||
const outsidePath = path.join(outsideDir, "pwned.txt");
|
||||
let called = false;
|
||||
|
||||
await expect(
|
||||
writeExternalFileWithinRoot({
|
||||
rootDir,
|
||||
path: outsidePath,
|
||||
write: async (candidate) => {
|
||||
called = true;
|
||||
await fs.writeFile(candidate, "pwned", "utf8");
|
||||
},
|
||||
}),
|
||||
).rejects.toMatchObject({ code: "outside-workspace" });
|
||||
|
||||
expect(called).toBe(false);
|
||||
await expect(fs.stat(outsidePath)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
});
|
||||
|
||||
it("rejects traversal targets before invoking the external writer", async () => {
|
||||
const rootDir = await tempRoot("fs-safe-output-traversal-root-");
|
||||
let called = false;
|
||||
|
||||
await expect(
|
||||
writeExternalFileWithinRoot({
|
||||
rootDir,
|
||||
path: "../../../pwned.txt",
|
||||
write: async (candidate) => {
|
||||
called = true;
|
||||
await fs.writeFile(candidate, "pwned", "utf8");
|
||||
},
|
||||
}),
|
||||
).rejects.toMatchObject({ code: "outside-workspace" });
|
||||
|
||||
expect(called).toBe(false);
|
||||
});
|
||||
|
||||
it("rejects root directory targets before invoking the external writer", async () => {
|
||||
const rootDir = await tempRoot("fs-safe-output-root-target-");
|
||||
let called = false;
|
||||
|
||||
await expect(
|
||||
writeExternalFileWithinRoot({
|
||||
rootDir,
|
||||
path: rootDir,
|
||||
write: async (candidate) => {
|
||||
called = true;
|
||||
await fs.writeFile(candidate, "not a file target", "utf8");
|
||||
},
|
||||
}),
|
||||
).rejects.toMatchObject({ code: "invalid-path" });
|
||||
|
||||
expect(called).toBe(false);
|
||||
});
|
||||
|
||||
it("rejects trailing-separator targets before invoking the external writer", async () => {
|
||||
const rootDir = await tempRoot("fs-safe-output-dir-target-");
|
||||
let called = false;
|
||||
|
||||
await expect(
|
||||
writeExternalFileWithinRoot({
|
||||
rootDir,
|
||||
path: "nested/",
|
||||
write: async (candidate) => {
|
||||
called = true;
|
||||
await fs.writeFile(candidate, "not a file target", "utf8");
|
||||
},
|
||||
}),
|
||||
).rejects.toMatchObject({ code: "invalid-path" });
|
||||
|
||||
expect(called).toBe(false);
|
||||
});
|
||||
|
||||
it.runIf(process.platform !== "win32")(
|
||||
"does not let symlinked target parents redirect the external temp write",
|
||||
async () => {
|
||||
const rootDir = await tempRoot("fs-safe-output-link-root-");
|
||||
const outsideDir = await tempRoot("fs-safe-output-link-outside-");
|
||||
await fs.symlink(outsideDir, path.join(rootDir, "link"), "dir");
|
||||
let tempPath = "";
|
||||
|
||||
await expect(
|
||||
writeExternalFileWithinRoot({
|
||||
rootDir,
|
||||
path: "link/out.txt",
|
||||
write: async (candidate) => {
|
||||
tempPath = candidate;
|
||||
await fs.writeFile(candidate, "pwned", "utf8");
|
||||
},
|
||||
}),
|
||||
).rejects.toBeTruthy();
|
||||
|
||||
await expect(fs.stat(tempPath)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
await expect(fs.readdir(outsideDir)).resolves.toEqual([]);
|
||||
},
|
||||
);
|
||||
|
||||
it.runIf(process.platform !== "win32")(
|
||||
"rejects hardlinked final targets and preserves the existing file",
|
||||
async () => {
|
||||
const rootDir = await tempRoot("fs-safe-output-hardlink-");
|
||||
const sourcePath = path.join(rootDir, "source.txt");
|
||||
const hardlinkPath = path.join(rootDir, "hardlink.txt");
|
||||
await fs.writeFile(sourcePath, "original", "utf8");
|
||||
await fs.link(sourcePath, hardlinkPath);
|
||||
|
||||
await expect(
|
||||
writeExternalFileWithinRoot({
|
||||
rootDir,
|
||||
path: "hardlink.txt",
|
||||
write: async (candidate) => {
|
||||
await fs.writeFile(candidate, "replacement", "utf8");
|
||||
},
|
||||
}),
|
||||
).rejects.toBeTruthy();
|
||||
|
||||
await expect(fs.readFile(sourcePath, "utf8")).resolves.toBe("original");
|
||||
await expect(fs.readFile(hardlinkPath, "utf8")).resolves.toBe("original");
|
||||
},
|
||||
);
|
||||
|
||||
it("cleans private temp files when the external writer fails", async () => {
|
||||
const rootDir = await tempRoot("fs-safe-output-fail-root-");
|
||||
let tempPath = "";
|
||||
|
||||
await expect(
|
||||
writeExternalFileWithinRoot({
|
||||
rootDir,
|
||||
path: "out.txt",
|
||||
write: async (candidate) => {
|
||||
tempPath = candidate;
|
||||
await fs.writeFile(candidate, "partial", "utf8");
|
||||
throw new Error("download failed");
|
||||
},
|
||||
}),
|
||||
).rejects.toThrow("download failed");
|
||||
|
||||
await expect(fs.stat(tempPath)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
await expect(fs.stat(path.join(rootDir, "out.txt"))).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
});
|
||||
});
|
||||
Loading…
Reference in New Issue
Block a user