Compare commits

..
Author SHA1 Message Date
Vito Sansevero 4afb042b07 fix(tests): pin glibc's mmap threshold in the browser test server (#516)
Each tests/browser/server.py grew from about 13 MB to 0.6-1 GB RSS over
a page-workflow shard while its Python heap stayed near 12 MB. glibc
raises its mmap threshold (up to 32 MiB) after a large free, so the
seeder's image-sized buffers came from per-thread malloc arenas, which
fragmented and kept their pages. The server now fixes the threshold at
glibc's 128 KiB start value (mallopt M_MMAP_THRESHOLD) before it starts,
so large blocks go back to the OS on free. No memory limit is set; off
glibc it does nothing.

Closes #516
2026-10-29 17:43:11 -07:00
Vito 62815c9e3e fix(tests): close the twin database before the bulk-delete temp dir goes (#510) (#511)
tearDown removed the temp folder before the test's addCleanup closed
twin.db, so Windows refused the delete (WinError 32). Register the folder
and main database cleanups in setUp so they run after every cleanup a
test adds.
2026-10-29 22:56:38 +00:00
Vito a1ec65442d fix(images): say plainly why Windows offers no Recycle Bin (#500) (#508)
Maintainer ruling (a): Windows never uses the Recycle Bin, because it deletes
files larger than the bin's capacity permanently and IFileOperation's
PreDeleteItem guard cannot detect it (native probe runs 40951037575,
40953347904). Each file in the Delete review now reads "Moving files to the
Recycle Bin isn't available on Windows: files larger than the bin's capacity
would be deleted permanently." (code windows-recycle) instead of "This drive has
no trash.", and the summary line drops its "yet": "Moving to the trash is not
available for these files; each one lists why."
2026-10-29 22:39:53 +00:00
15 changed files with 188 additions and 284 deletions
+6 -10
View File
@@ -53,14 +53,6 @@ stdin open and the server (`--exit-on-stdin-eof`) shuts down cleanly, removing i
temp root, when that pipe closes, so a worker killed without teardown leaves no
server behind. A test that fails gets the stderr its server wrote during the test
attached to the report as `server-stderr` (an aiohttp traceback behind a 500).
Each worker also records its browser's own log (#513, `tools/vendor/browser-logs.mjs`):
Playwright's `pw:browser` channel, kept as a 300-line tail without printing it
unless `DEBUG` asks for it. A browser that exits abnormally writes one
`pm-browser-exit| browser pid=<n> exited: exitCode=<c>, signal=<s>` line to the job
log, and a test that fails on a closed or crashed target ("Target page, context or
browser has been closed") gets the tail attached as `browser-stderr.txt`, which the
uploaded test-results carry. `tests/browser/browser-logs.spec.mjs` kills a browser of
its own on purpose, so each browser job logs that line once for it.
A `/__test__/.../reset` still running after 2 s writes a report to the job log,
passing test or not: every thread's stack taken while it waits, the default
executor's threads and queue, the open connections and WAL size, then the total
@@ -2496,8 +2488,12 @@ files to the system trash. **No code path unlinks or overwrites a media file.**
`.Trash/$uid` or `.Trash-$uid`), reserve the name with `O_EXCL` and move with
`renameat2(RENAME_NOREPLACE)`; without it, across devices, on a read-only mount or with
a trash inside an output root the move is refused (`no-trash`, `trash-in-root`). Windows
and macOS report `no-trash` until their own backends land; library-only removal works
everywhere. Tests use `tests/fake_trash.py`; the browser test server installs it, so no
never uses the Recycle Bin (#500, maintainer ruling (a)): files larger than the bin's
capacity are deleted permanently and IFileOperation's `PreDeleteItem` guard cannot detect
it (native probe runs 40951037575, 40953347904), so every file answers `windows-recycle`
("Moving files to the Recycle Bin isn't available on Windows: files larger than the bin's
capacity would be deleted permanently."). macOS reports `no-trash` until #499 lands;
library-only removal works everywhere. Tests use `tests/fake_trash.py`; the browser test server installs it, so no
test touches a real trash.
### Prompt library cards and recorded deviations (#249 PR E)
+4
View File
@@ -83,6 +83,10 @@ REASONS = {
trash.TRASH_IN_ROOT: "This drive's trash is inside an output folder.",
trash.LOCKED: "The file is open in another program. Close it and try again.",
trash.FAILED: "The file could not be moved to the trash.",
trash.WINDOWS_RECYCLE: (
"Moving files to the Recycle Bin isn't available on Windows: files larger "
"than the bin's capacity would be deleted permanently."
),
"not-attempted": "Not attempted: the job stopped after an error.",
}
_PROBLEM_CODES = {
-26
View File
@@ -1,26 +0,0 @@
// #513: a worker records its browser's own log, so a browser that dies between two tests leaves its exit and stderr
// behind instead of only "Target page, context or browser has been closed" on the next test.
import { test, expect } from './fixtures.mjs';
import { isTargetGone } from '../../tools/vendor/browser-logs.mjs';
test('the worker records the launch of the browser its tests use', async ({ browser, browserLogs }) => {
expect(browser.isConnected()).toBe(true);
expect(browserLogs.launches().length).toBeGreaterThan(0);
});
test('a killed browser leaves its signal and stderr in the worker log', async ({ playwright, browserName, browserLogs }) => {
const before = browserLogs.launches();
const victim = await playwright[browserName].launch();
const pid = browserLogs.launches().find(item => !before.includes(item));
expect(pid, 'the new browser\'s launch was recorded').toBeGreaterThan(0);
const disconnected = new Promise(resolve => victim.once('disconnected', resolve));
// Playwright starts a browser in a process group of its own; WebKit's launched pid is a wrapper script.
process.kill(-pid, 'SIGKILL');
await disconnected;
const error = await victim.newContext().then(() => null, item => item);
expect(error, 'a dead browser refuses a new context').not.toBeNull();
expect(isTargetGone(error.message)).toBe(true);
// The pipe can close before the launcher has logged the exit.
await expect.poll(() => browserLogs.exits()).toContainEqual({ pid, exitCode: null, signal: 'SIGKILL' });
expect(browserLogs.tail()).toContain(`[pid=${pid}] <process did exit: exitCode=null, signal=SIGKILL>`);
});
+1 -17
View File
@@ -4,29 +4,13 @@
// points at that worker's server. A server that dies fails the running test
// with its stderr tail, and the next test in the worker gets a fresh server.
// A test that fails gets the stderr its server wrote during it attached as
// `server-stderr`. The worker also records its browser's own log (#513): a
// browser that exits abnormally prints one `pm-browser-exit|` line, and a test
// that fails on a closed or crashed target gets that log attached as
// `browser-stderr`.
// `server-stderr`.
import { test as base, expect } from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import { attachServerStderr, createServerSupervisor, startBrowserServer } from '../../tools/vendor/start-browser-server.mjs';
import { attachBrowserLogs, createBrowserLogRecorder } from '../../tools/vendor/browser-logs.mjs';
const SERVER_FIXTURE_TIMEOUT_MS = 90_000;
export const test = base.extend({
browserLogs: [async ({}, use) => {
const recorder = createBrowserLogRecorder();
try {
await use(recorder);
} finally {
recorder.uninstall();
}
}, { scope: 'worker', auto: true }],
browserLogsOnFailure: [async ({ browserLogs }, use, testInfo) => {
await use();
await attachBrowserLogs(testInfo, browserLogs);
}, { auto: true }],
testServer: [async ({}, use) => {
const supervisor = createServerSupervisor({ start: () => startBrowserServer() });
try {
+31
View File
@@ -0,0 +1,31 @@
"""Keep the browser test server's large frees going back to the OS (#516).
glibc serves an allocation of at least the mmap threshold (128 KiB at start) with its
own mmap and unmaps it on free. Freeing such a block also raises the threshold, up to
32 MiB, so the server's later image-sized buffers (PIL pixel data, PNG encoding,
whole-file reads in the executor threads) come from per-thread malloc arenas instead.
Those arenas fragment and keep their pages: a server grew from 13 MB to 0.6-1 GB RSS
over one page-workflow shard while its Python heap stayed near 12 MB. Setting
M_MMAP_THRESHOLD fixes the threshold at its start value and turns the raising off.
"""
import ctypes
import sys
M_MMAP_THRESHOLD = -3
MMAP_THRESHOLD_BYTES = 128 * 1024
def pin_mmap_threshold(libc=None) -> bool:
"""Fix glibc's mmap threshold; False where there is no glibc ``mallopt``."""
if not sys.platform.startswith("linux"):
return False
if libc is None:
try:
libc = ctypes.CDLL(None)
except OSError:
return False
mallopt = getattr(libc, "mallopt", None)
if mallopt is None:
return False
return bool(mallopt(M_MMAP_THRESHOLD, MMAP_THRESHOLD_BYTES))
+3
View File
@@ -20,6 +20,7 @@ from PIL import Image
from PIL.PngImagePlugin import PngInfo
from tests.browser.folder_trash import close_trash, replace_folder
from tests.browser.malloc_tuning import pin_mmap_threshold
from tests.browser.request_hold import register_request_holds
from tests.browser import slow_reset_watch
from tests.browser.fast_reset import SEED_ON_LIVE_ENV, ScratchSeeder
@@ -755,6 +756,8 @@ def main(argv: list[str] | None = None) -> None:
help="shut down when stdin closes (the launcher's parent-death guard)",
)
args = parser.parse_args(argv)
# Before any thread allocates (#516): large frees then return to the OS.
pin_mmap_threshold()
listener = bind_listener(args.port)
# One machine-readable line tells the launcher which port this process holds.
print(f"{PORT_ANNOUNCEMENT}{listener.getsockname()[1]}", flush=True)
+2
View File
@@ -65,6 +65,8 @@ test('sentences follow the mode and the backup policy', () => {
assert.deepEqual(deleteSentences(plan(), 'trash'), [SENTENCES.trash, USE_LIBRARY_ONLY]);
const nowhere = plan({ trash: { count: 0 }, trash_refused: { count: 3, rows: [] } });
assert.deepEqual(deleteSentences(nowhere, 'trash'), [SENTENCES.trash, NO_TRASH_HERE, USE_LIBRARY_ONLY]);
// #500: no "yet" (Windows is unavailable by design); each file's reason says why.
assert.equal(NO_TRASH_HERE, 'Moving to the trash is not available for these files; each one lists why.');
assert.deepEqual(deleteSentences(plan({ trash_refused: { count: 0 } }), 'trash'), [SENTENCES.trash]);
});
+92
View File
@@ -0,0 +1,92 @@
"""#516: the browser test server pins glibc's mmap threshold.
Large frees then go back to the OS instead of fragmenting per-thread arenas.
"""
import ctypes
import sys
import unittest
from unittest import mock
from tests.browser import malloc_tuning
class FakeLibc:
def __init__(self, result=1):
self.calls = []
self.result = result
def mallopt(self, option, value):
self.calls.append((option, value))
return self.result
class PinMmapThresholdTests(unittest.TestCase):
def test_sets_a_fixed_threshold_through_mallopt(self):
libc = FakeLibc()
with mock.patch.object(sys, "platform", "linux"):
self.assertTrue(malloc_tuning.pin_mmap_threshold(libc))
# M_MMAP_THRESHOLD is -3; setting it also turns off glibc's dynamic
# raising of the threshold.
self.assertEqual(libc.calls, [(-3, 128 * 1024)])
def test_reports_a_refused_setting(self):
with mock.patch.object(sys, "platform", "linux"):
self.assertFalse(malloc_tuning.pin_mmap_threshold(FakeLibc(result=0)))
def test_skips_a_libc_without_mallopt(self):
with mock.patch.object(sys, "platform", "linux"):
self.assertFalse(malloc_tuning.pin_mmap_threshold(object()))
def test_skips_other_platforms_without_loading_a_libc(self):
for name in ("win32", "darwin"):
with (
self.subTest(name),
mock.patch.object(sys, "platform", name),
mock.patch.object(
ctypes, "CDLL", side_effect=AssertionError("no libc load")
),
):
self.assertFalse(malloc_tuning.pin_mmap_threshold())
def test_skips_when_the_process_libc_cannot_be_loaded(self):
with (
mock.patch.object(sys, "platform", "linux"),
mock.patch.object(ctypes, "CDLL", side_effect=OSError("no libc")),
):
self.assertFalse(malloc_tuning.pin_mmap_threshold())
@unittest.skipUnless(
sys.platform.startswith("linux")
and hasattr(ctypes.CDLL(None), "gnu_get_libc_version"),
"glibc only",
)
def test_glibc_accepts_the_setting(self):
# This pins the threshold for the rest of this test process too; the
# test server does the same at start, so nothing here depends on it.
self.assertTrue(malloc_tuning.pin_mmap_threshold())
class ServerMainTests(unittest.TestCase):
def test_main_pins_the_threshold_before_it_starts_serving(self):
from tests.browser import server
order = []
def bind(port):
order.append("bind")
raise RuntimeError("stop before serving")
with (
mock.patch.object(
server, "pin_mmap_threshold", side_effect=lambda: order.append("pin")
),
mock.patch.object(server, "bind_listener", side_effect=bind),
):
with self.assertRaises(RuntimeError):
server.main(["--port", "0"])
self.assertEqual(order, ["pin", "bind"])
if __name__ == "__main__":
unittest.main()
+5 -4
View File
@@ -64,14 +64,15 @@ def table_dump(db):
class LibraryDeleteCase(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
# Cleanups run last-in first-out and after every cleanup a test adds,
# so a database a test opens (and closes via addCleanup) is closed
# before the folder goes; Windows cannot delete an open file (#510).
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
self.db = PromptDatabase(str(self.root / "delete.db"))
self.addCleanup(self.db.close_all)
self.epoch = self.db.get_library_options()["library_epoch"]
def tearDown(self):
self.db.close_all()
self.temp.cleanup()
def paired(self, positive, negative="blur"):
pair = self.db.save_prompt_pair(positive, negative)
found = self.db.get_prompt_pair(pair)
+9 -1
View File
@@ -306,6 +306,13 @@ class TestFileCheck(ImageDeleteTestCase):
file_check(str(self.output_dir / "ok.png"), root, "ok.png")[0]
)
def test_windows_reason_says_why_plainly(self):
self.assertEqual(
image_delete.REASONS["windows-recycle"],
"Moving files to the Recycle Bin isn't available on Windows: files "
"larger than the bin's capacity would be deleted permanently.",
)
def test_reasons_cover_every_code(self):
from utils import trash
@@ -324,7 +331,8 @@ class TestNativeBackend(ImageDeleteTestCase):
native_trash = sys.platform not in ("win32", "darwin")
if not native_trash:
self.assertEqual(plan["trash"], {"count": 0})
self.assertEqual(plan["trash_refused"]["counts"], {"no-trash": 1})
code = "windows-recycle" if sys.platform == "win32" else "no-trash"
self.assertEqual(plan["trash_refused"]["counts"], {code: 1})
body = await self.post(APPLY, {"token": plan["token"], "mode": "library"}, 202)
job = self.api._jobs.get(body["job"]["id"])
await asyncio.wait_for(asyncio.shield(job.task), DISK_BOUND_SECONDS)
+14
View File
@@ -285,6 +285,20 @@ class UnavailableBackendTests(unittest.TestCase):
)
self.assertIsInstance(trash.default_backend("linux"), trash.FreedesktopBackend)
def test_windows_says_why_the_recycle_bin_is_not_used(self):
# #500 ruling (a): the Recycle Bin can delete permanently, so it is never used.
backend = trash.default_backend("win32")
self.assertEqual(
backend.available("C:/x/a.png", ["C:/x"]), trash.WINDOWS_RECYCLE
)
with self.assertRaises(trash.TrashRefused) as caught:
backend.move("C:/x/a.png", ["C:/x"])
self.assertEqual(caught.exception.code, trash.WINDOWS_RECYCLE)
self.assertIn(trash.WINDOWS_RECYCLE, trash.CODES)
self.assertEqual(
trash.default_backend("darwin").available("/x", ["/"]), trash.NO_TRASH
)
def test_module_functions_use_the_replaceable_backend(self):
fake = mock.Mock()
fake.available.return_value = "network"
-127
View File
@@ -1,127 +0,0 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { mkdtempSync, readFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { attachBrowserLogs, BROWSER_EXIT_PREFIX, createBrowserLogRecorder, isAbnormalExit, isTargetGone, parseExitLine,
playwrightDebug } from '../../tools/vendor/browser-logs.mjs';
const debug = playwrightDebug();
/** Runs `body` with DEBUG set to `namespaces` and restores the caller's setting afterwards. */
function withDebug(namespaces, body) {
const saved = debug.disable();
const savedLog = debug.log;
if (namespaces) debug.enable(namespaces);
try {
return body();
} finally {
debug.log = savedLog;
debug.disable();
if (saved) debug.enable(saved);
}
}
const outputDir = mkdtempSync(join(tmpdir(), 'pm-browser-logs-'));
test.after(() => rmSync(outputDir, { recursive: true, force: true }));
function testInfo({ status = 'failed', errors = [] } = {}) {
const attachments = [];
const dir = mkdtempSync(join(outputDir, 'test-'));
return { status, expectedStatus: 'passed', errors, attachments, outputPath: name => join(dir, name),
attach: async (name, options) => { attachments.push({ name, ...options }); } };
}
test('parses the exit line Playwright logs for a browser process', () => {
assert.deepEqual(parseExitLine('[pid=7124] <process did exit: exitCode=null, signal=SIGKILL> +21ms'),
{ pid: 7124, exitCode: null, signal: 'SIGKILL' });
assert.deepEqual(parseExitLine('[pid=9] <process did exit: exitCode=0, signal=null>'), { pid: 9, exitCode: 0, signal: null });
assert.deepEqual(parseExitLine('[pid=9] <process did exit: exitCode=11, signal=null>'), { pid: 9, exitCode: 11, signal: null });
assert.equal(parseExitLine('[pid=9][err] JavaScript warning'), null);
assert.equal(isAbnormalExit({ pid: 9, exitCode: 0, signal: null }), false);
assert.equal(isAbnormalExit({ pid: 9, exitCode: null, signal: 'SIGKILL' }), true);
assert.equal(isAbnormalExit({ pid: 9, exitCode: 139, signal: null }), true);
});
test('recognises the errors of a page, context or browser that went away', () => {
for (const message of ['Error: browser.newContext: Target page, context or browser has been closed',
'Protocol error (Page.navigate): Target closed', 'page.goto: Page crashed', 'Target crashed']) {
assert.equal(isTargetGone(message), true, message);
}
for (const message of ['expect(locator).toBeVisible() failed', undefined, 'Test timeout of 30000ms exceeded.']) {
assert.equal(isTargetGone(message), false, String(message));
}
});
test('records the browser channel into a bounded tail without printing it', () => withDebug('', () => {
const printed = [];
debug.log = (...args) => printed.push(args.join(' '));
const written = [];
const recorder = createBrowserLogRecorder({ debug, limit: 3, write: text => written.push(text) });
const browser = debug('pw:browser');
browser('<launched> pid=5');
for (const n of [1, 2, 3, 4]) browser(`[pid=5][err] line ${n}`);
assert.deepEqual(printed, [], 'the browser channel stays quiet when DEBUG did not ask for it');
assert.match(recorder.tail(), /line 2[\s\S]*line 3[\s\S]*line 4/);
assert.doesNotMatch(recorder.tail(), /line 1/);
assert.deepEqual(written, []);
assert.deepEqual(recorder.launches(), [5], 'launches outlive the bounded tail');
browser('[pid=5] <process did exit: exitCode=null, signal=SIGKILL>');
assert.deepEqual(recorder.exits(), [{ pid: 5, exitCode: null, signal: 'SIGKILL' }]);
assert.deepEqual(written, [`${BROWSER_EXIT_PREFIX}browser pid=5 exited: exitCode=null, signal=SIGKILL\n`]);
browser('[pid=6] <process did exit: exitCode=0, signal=null>');
assert.equal(written.length, 1, 'a clean exit prints nothing');
recorder.uninstall();
assert.equal(debug.enabled('pw:browser'), false, 'uninstall restores the caller\'s DEBUG');
}));
test('keeps printing the channels the user enabled, the browser one included', () => withDebug('pw:api,pw:browser', () => {
const printed = [];
debug.log = function (...args) { printed.push(`${this.namespace} ${args.join(' ')}`); };
const recorder = createBrowserLogRecorder({ debug, write: () => {} });
debug('pw:api')('call');
debug('pw:browser')('[pid=1][out] hello');
debug('pw:protocol')('not enabled');
assert.equal(printed.length, 2);
assert.match(printed[0], /^pw:api /);
assert.match(printed[1], /^pw:browser /);
assert.doesNotMatch(recorder.tail(), /call/);
recorder.uninstall();
assert.equal(debug.enabled('pw:api'), true);
assert.equal(debug.enabled('pw:browser'), true);
}));
test('attaches the tail and the exit only to a test whose target went away', async () => withDebug('', async () => {
debug.log = () => {};
const recorder = createBrowserLogRecorder({ debug, write: () => {} });
const browser = debug('pw:browser');
browser('\u001b[30m[pid=8][err] Crash Annotation GraphicsCriticalError\u001b[0m');
browser('[pid=8] <process did exit: exitCode=null, signal=SIGKILL>');
const passed = testInfo({ status: 'passed' });
assert.equal(await attachBrowserLogs(passed, recorder), false);
const assertion = testInfo({ errors: [{ message: 'expect(locator).toBeVisible() failed' }] });
assert.equal(await attachBrowserLogs(assertion, recorder), false);
assert.deepEqual(assertion.attachments, []);
const closed = testInfo({ errors: [{ message: 'Error: browser.newContext: Target page, context or browser has been closed' }] });
assert.equal(await attachBrowserLogs(closed, recorder), true);
assert.equal(closed.attachments.length, 1);
const [{ name, path, contentType }] = closed.attachments;
assert.equal(name, 'browser-stderr');
assert.equal(contentType, 'text/plain');
const body = readFileSync(path, 'utf8');
assert.match(body, /^browser pid=8 exited: exitCode=null, signal=SIGKILL\n/);
assert.match(body, /\[pid=8\]\[err\] Crash Annotation GraphicsCriticalError/);
assert.doesNotMatch(body, /\u001b/, 'the attached log has no terminal colours');
recorder.uninstall();
}));
test('says so when the worker logged no abnormal exit', async () => withDebug('', async () => {
debug.log = () => {};
const recorder = createBrowserLogRecorder({ debug, write: () => {} });
const closed = testInfo({ errors: [{ message: 'x', stack: 'Error: Target closed\n at y' }] });
assert.equal(await attachBrowserLogs(closed, recorder), true);
assert.match(readFileSync(closed.attachments[0].path, 'utf8'), /^no abnormal browser exit was logged in this worker\n/);
recorder.uninstall();
}));
-91
View File
@@ -1,91 +0,0 @@
// What a browser printed before it died (#513). Playwright writes a browser's stderr, its launch and its exit
// (`<process did exit: exitCode=…, signal=…>`) only to its `pw:browser` debug channel, which is off unless DEBUG is
// set, so a browser that dies between two tests leaves the next test only "Target page, context or browser has been
// closed". A worker records that channel into a bounded tail, prints one line when a browser exits abnormally and
// attaches the tail to a test that fails on a closed or crashed target. Nothing is retried or hidden.
import { createRequire } from 'node:module';
import { writeFile } from 'node:fs/promises';
import { format, stripVTControlCharacters } from 'node:util';
const BROWSER_NAMESPACE = 'pw:browser';
export const BROWSER_LOG_LIMIT = 300;
export const BROWSER_EXIT_PREFIX = 'pm-browser-exit| ';
const LAUNCH_LINE = /<launched> pid=(\d+)/;
const EXIT_LINE = /\[pid=(\d+)\] <process did exit: exitCode=([^,]+), signal=([^>]+)>/;
// The client's TargetClosedError, a protocol error of a closed or crashed target, and a crashed page.
const TARGET_GONE = /Target page, context or browser has been closed|Target (?:closed|crashed)|Browser closed|browser has disconnected|Page crashed/i;
/** The `debug` module Playwright logs through (the instance this worker's tests launch browsers with). */
export function playwrightDebug() {
return createRequire(import.meta.url)('playwright-core/lib/utilsBundle').debug;
}
/** The pid, exit code and signal of a `process did exit` line, or null. */
export function parseExitLine(line) {
const match = EXIT_LINE.exec(line);
if (!match) return null;
const value = text => (text === 'null' ? null : text);
const code = value(match[2]);
return { pid: Number(match[1]), exitCode: code === null ? null : Number(code), signal: value(match[3]) };
}
export const isAbnormalExit = exit => exit !== null && (exit.exitCode !== 0 || exit.signal !== null);
/** True for an error message that says the page, context or browser under the test went away. */
export const isTargetGone = message => TARGET_GONE.test(String(message ?? ''));
/**
* Records `pw:browser` into a tail of at most `limit` lines. Other debug channels keep printing as before; the
* browser channel prints only when the user's own DEBUG asked for it. `write` gets one line per abnormal exit.
*/
export function createBrowserLogRecorder({ debug = playwrightDebug(), limit = BROWSER_LOG_LIMIT,
write = text => process.stderr.write(text) } = {}) {
const lines = [];
const launches = [];
const exits = [];
const userWantsBrowser = debug.enabled(BROWSER_NAMESPACE);
const previous = debug.disable();
const original = debug.log;
debug.log = function record(...args) {
if (this?.namespace !== BROWSER_NAMESPACE || userWantsBrowser) {
(original ?? (() => {})).apply(this, args);
}
if (this?.namespace !== BROWSER_NAMESPACE) return;
const line = stripVTControlCharacters(format(...args));
lines.push(line);
if (lines.length > limit) lines.splice(0, lines.length - limit);
const launched = LAUNCH_LINE.exec(line);
if (launched) launches.push(Number(launched[1]));
const exit = parseExitLine(line);
if (!exit) return;
exits.push(exit);
if (isAbnormalExit(exit)) {
write(`${BROWSER_EXIT_PREFIX}browser pid=${exit.pid} exited: exitCode=${exit.exitCode}, signal=${exit.signal}\n`);
}
};
debug.enable([previous, BROWSER_NAMESPACE].filter(Boolean).join(','));
return {
tail: () => lines.join('\n'),
launches: () => [...launches],
exits: () => [...exits],
uninstall() {
debug.log = original;
debug.disable();
if (previous) debug.enable(previous);
},
};
}
/** Attaches the browser log tail to a test that failed because its page, context or browser went away. */
export async function attachBrowserLogs(testInfo, recorder) {
if (testInfo.status === testInfo.expectedStatus) return false;
if (!testInfo.errors.some(error => isTargetGone(error.message) || isTargetGone(error.stack))) return false;
const exits = recorder.exits().filter(isAbnormalExit)
.map(exit => `browser pid=${exit.pid} exited: exitCode=${exit.exitCode}, signal=${exit.signal}`);
const summary = exits.length ? exits.join('\n') : 'no abnormal browser exit was logged in this worker';
// A file, so the log travels with the uploaded test-results (the list reporter shortens a long body).
const path = testInfo.outputPath('browser-stderr.txt');
await writeFile(path, `${summary}\n--- last ${BROWSER_LOG_LIMIT} pw:browser lines ---\n${recorder.tail()}\n`);
await testInfo.attach('browser-stderr', { path, contentType: 'text/plain' });
return true;
}
+20 -7
View File
@@ -14,8 +14,9 @@ same mount, else the file's mount's ``.Trash/$uid`` (only under a real, sticky
``.Trash``) or ``.Trash-$uid``. Names are reserved by creating the
``.trashinfo`` with ``O_EXCL`` and the file is moved with
``renameat2(RENAME_NOREPLACE)``; without that call the move is refused.
Windows and macOS have no backend yet (:class:`UnavailableBackend`): their
own PRs add one with a native proof.
Windows never uses the Recycle Bin (#500: files over its capacity are deleted
permanently, undetectably), so it answers ``windows-recycle``; macOS has no
backend yet (#499). Both use :class:`UnavailableBackend`.
``BACKEND`` is the one replaceable seam; tests install a fake.
"""
@@ -36,7 +37,11 @@ TRASH_IN_ROOT = "trash-in-root"
PATH_TOO_LONG = "path-too-long"
LOCKED = "locked"
FAILED = "failed"
CODES = frozenset({NO_TRASH, NETWORK, TRASH_IN_ROOT, PATH_TOO_LONG, LOCKED, FAILED})
# Windows never uses the Recycle Bin (#500): it can delete permanently.
WINDOWS_RECYCLE = "windows-recycle"
CODES = frozenset(
{NO_TRASH, NETWORK, TRASH_IN_ROOT, PATH_TOO_LONG, LOCKED, FAILED, WINDOWS_RECYCLE}
)
HOME_TRASH_LABEL = "home trash"
RENAME_NOREPLACE = 1
@@ -177,13 +182,16 @@ def _owned_folder(path: str, uid: int, *, create: bool) -> bool:
class UnavailableBackend:
"""No trash on this system: every move is refused, nothing is touched."""
"""No trash on this system: every move is refused with *code*, untouched."""
def __init__(self, code: str = NO_TRASH):
self.code = code
def available(self, path: str, roots) -> str | None:
return NO_TRASH
return self.code
def move(self, path: str, roots) -> str:
raise TrashRefused(NO_TRASH)
raise TrashRefused(self.code)
class FreedesktopBackend:
@@ -346,7 +354,12 @@ class FreedesktopBackend:
def default_backend(platform: str = sys.platform):
"""The backend for *platform*; Windows and macOS have none yet (#480)."""
if platform in ("win32", "darwin") or platform.startswith("cygwin"):
if platform == "win32" or platform.startswith("cygwin"):
# #500, ruling (a): Windows deletes files larger than the Recycle Bin's
# capacity permanently, and IFileOperation's PreDeleteItem guard cannot
# tell (native probe runs 40951037575, 40953347904).
return UnavailableBackend(WINDOWS_RECYCLE)
if platform == "darwin":
return UnavailableBackend()
return FreedesktopBackend()
+1 -1
View File
@@ -32,7 +32,7 @@ export const REVIEW_FAILED = 'The review failed. Nothing was changed.';
export const SOURCE_CHANGED = 'The sources changed. Refresh the images and try again.';
export const STORAGE_CHANGED = 'Prompt storage was restored. Reopen Delete.';
export const USE_LIBRARY_ONLY = 'To remove these files from the library anyway, choose “Remove from the library only”.';
export const NO_TRASH_HERE = 'Moving to the trash is not available on this system yet.';
export const NO_TRASH_HERE = 'Moving to the trash is not available for these files; each one lists why.';
const SENTENCES = Object.freeze({
libraryBackup: 'Removed images stay hidden unless the file is modified or replaced. Their tags, category, collections '
+ 'and favorite can only be recovered by restoring the safety backup, which also undoes every later change.',