Vito
d0339128e4
* ci: pin black and flake8, let the bandit hook fail, cache npm in the JS job (#352) The lint job installs black==26.10.1 and flake8==7.4.1 from requirements-dev.txt (the versions CI resolved on main today), so a new release cannot turn CI red with no code change. The local bandit hook ran 'bash -c ... "$@"; true': it could never fail, and bash bound the first staged file to $0 so it was never scanned. It now runs 'bandit -lll -q' directly, failing on high severity exactly like code-quality.yml. The frontend modules job's setup-node caches npm keyed on tools/vendor/package-lock.json. * test(history): wait on the writer's state, not short wall-clock pauses (#352) Four 0.08 s pauses before asserting that the worker had not probed yet now wait until the idle worker has run a whole loop pass at the current manual clock value (three clock reads), so a slow runner cannot end the pause before the worker decides. A mutation that probes 20 ms early now fails two tests deterministically. The real-clock bounded close no longer asserts close(0.02) returned in under 0.2 s wall time; it asserts the in-flight write is still blocked and the worker alive, like the output writer test. The manual-clock bound stays exact. The last 0.1 s non-blocking submit check uses the 1 s NON_BLOCKING_CEILING from #351. * test(api): give each API fixture database its own temp folder (#346) Every PromptManagerAPI sweeps its database's folder at startup for the stages, uploads and safety-backup copies an interrupted restore left. The API test fixtures created their database directly in the system temp folder, so any other test process starting an API there (another worktree's suite, a pre-push hook, an agent's targeted run) deleted this process's in-flight restore files. The restore, epoch and retirement tests then failed with 'staged restore changed', 'unable to open database file' or 503 instead of 409. Busy machines fail because they are running other suites, not because they are slow: the same CPU load with a private TMPDIR passes. private_temp_db() puts the fixture database in a new folder removed at cleanup (with any safety backups a restore left beside it). A regression test sweeps a stand-in shared folder in the middle of a multipart restore, as another suite's startup would. * test(history): count worker parks, hold saturation blocks until released (#352) wait_for_worker_pass now counts the idle worker's parks on its wake event (two parks from now = one whole pass begun after the call) instead of assuming a fixed number of clock reads per pass, which a future clock read would have made vacuous. The real-clock bounded close keeps a loose upper bound (0.02 s + NON_BLOCKING_CEILING). The two saturation tests (history and output writer) blocked the worker with release.wait(3) while they queued 1023 more items. On a loaded runner the fill took longer than 3 s, the worker unblocked on its own and drained part of the queue (pending 868..1017 != 1024), and cleanup then reported a leaked worker. The block now holds until the test releases it; 60 s is only a hang guard, and cleanup always sets the release. Under one core with 8 busy loops: 8/8 runs failed before, 6/6 pass after. * test(maintenance): wait on duplicate jobs with a disk-bound hang guard (#409) On Windows py3.13 CI (PR #405) a one-file duplicate cleanup missed its 10 s WAIT. The test already waits on the job's own task; the 10 s bound acted as a deadline on disk-bound work. In the same 50 s window the following tests' setups took up to 9 s against 0.1-0.6 s around it while the test bodies stayed fast, which is consistent with a process-wide disk I/O stall on the runner rather than a lease, handle, executor or job stall (the log has no thread dump to show more). tests/disk_waits.py holds the one disk-bound hang guard (90 s per wait, under the 120 s per-test watchdog), shared with RESTORE_SECONDS (#382). Waits on find and cleanup job tasks in the duplicate-files tests use it; gates and server-state polls keep WAIT. The keeper-validation test turns the safety backup off, since it does not test backups.
419 lines
15 KiB
Python
419 lines
15 KiB
Python
"""
|
|
API endpoint integration tests for PromptManager.
|
|
|
|
Uses aiohttp test client to test request/response contracts
|
|
without requiring a full ComfyUI server.
|
|
"""
|
|
|
|
import json
|
|
import os
|
|
import sys
|
|
import unittest
|
|
|
|
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
|
|
|
from aiohttp import web
|
|
from aiohttp.test_utils import AioHTTPTestCase
|
|
|
|
from database.operations import PromptDatabase
|
|
from py.api import PromptManagerAPI
|
|
from tests.private_temp import private_temp_db
|
|
from utils.hashing import generate_prompt_hash
|
|
|
|
|
|
class APITestCase(AioHTTPTestCase):
|
|
"""Base class that stands up a test aiohttp app with PromptManager routes."""
|
|
|
|
async def get_application(self):
|
|
self._temp_db = private_temp_db(self)
|
|
|
|
app = web.Application()
|
|
routes = web.RouteTableDef()
|
|
|
|
self.api = PromptManagerAPI(db=PromptDatabase(self._temp_db.name))
|
|
self.api.add_routes(routes)
|
|
app.router.add_routes(routes)
|
|
return app
|
|
|
|
async def tearDownAsync(self):
|
|
await super().tearDownAsync() # closes the aiohttp test client
|
|
# Close every thread's connection first: Windows refuses to delete
|
|
# an open database file, and executor threads hold connections too.
|
|
self.api.db.close_all()
|
|
for path in (
|
|
self._temp_db.name,
|
|
self._temp_db.name + "-wal",
|
|
self._temp_db.name + "-shm",
|
|
):
|
|
if os.path.exists(path):
|
|
os.unlink(path)
|
|
|
|
# ── helpers ────────────────────────────────────────────────────────
|
|
|
|
def _save_prompt(self, text="Test prompt", category=None, tags=None, rating=None):
|
|
return self.api.db.save_prompt(
|
|
text=text,
|
|
category=category,
|
|
tags=tags or [],
|
|
rating=rating,
|
|
prompt_hash=generate_prompt_hash(text),
|
|
)
|
|
|
|
|
|
class TestHealthEndpoint(APITestCase):
|
|
|
|
async def test_test_route(self):
|
|
resp = await self.client.request("GET", "/prompt_manager/test")
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
self.assertIn("timestamp", data)
|
|
|
|
|
|
class TestRecentPrompts(APITestCase):
|
|
|
|
async def test_recent_empty(self):
|
|
resp = await self.client.request("GET", "/prompt_manager/recent")
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
self.assertEqual(data["pagination"]["total"], 0)
|
|
self.assertEqual(len(data["results"]), 0)
|
|
|
|
async def test_recent_with_data(self):
|
|
self._save_prompt("First prompt")
|
|
self._save_prompt("Second prompt")
|
|
resp = await self.client.request("GET", "/prompt_manager/recent")
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertEqual(data["pagination"]["total"], 2)
|
|
self.assertEqual(len(data["results"]), 2)
|
|
|
|
async def test_recent_pagination(self):
|
|
for i in range(15):
|
|
self._save_prompt(f"Prompt {i:02d}")
|
|
resp = await self.client.request(
|
|
"GET", "/prompt_manager/recent?limit=5&offset=0"
|
|
)
|
|
data = await resp.json()
|
|
self.assertEqual(len(data["results"]), 5)
|
|
self.assertEqual(data["pagination"]["total"], 15)
|
|
self.assertTrue(data["pagination"]["has_more"])
|
|
|
|
async def test_recent_last_page(self):
|
|
for i in range(7):
|
|
self._save_prompt(f"Prompt {i}")
|
|
resp = await self.client.request(
|
|
"GET", "/prompt_manager/recent?limit=5&offset=5"
|
|
)
|
|
data = await resp.json()
|
|
self.assertEqual(len(data["results"]), 2)
|
|
self.assertFalse(data["pagination"]["has_more"])
|
|
|
|
|
|
class TestSearch(APITestCase):
|
|
|
|
async def test_search_by_text(self):
|
|
self._save_prompt("Beautiful mountain landscape")
|
|
self._save_prompt("City skyline at night")
|
|
resp = await self.client.request("GET", "/prompt_manager/search?text=mountain")
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
self.assertEqual(len(data["results"]), 1)
|
|
self.assertIn("mountain", data["results"][0]["text"])
|
|
|
|
async def test_search_by_category(self):
|
|
self._save_prompt("Nature scene", category="nature")
|
|
self._save_prompt("Urban view", category="urban")
|
|
resp = await self.client.request(
|
|
"GET", "/prompt_manager/search?category=nature"
|
|
)
|
|
data = await resp.json()
|
|
self.assertEqual(len(data["results"]), 1)
|
|
|
|
async def test_search_by_tag(self):
|
|
self._save_prompt("Tagged prompt", tags=["landscape", "sunset"])
|
|
self._save_prompt("Other prompt", tags=["portrait"])
|
|
resp = await self.client.request("GET", "/prompt_manager/search?tags=landscape")
|
|
data = await resp.json()
|
|
self.assertEqual(len(data["results"]), 1)
|
|
|
|
async def test_search_empty_result(self):
|
|
resp = await self.client.request(
|
|
"GET", "/prompt_manager/search?text=nonexistent_xyz"
|
|
)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
self.assertEqual(len(data["results"]), 0)
|
|
|
|
|
|
class TestSaveAndDelete(APITestCase):
|
|
|
|
async def test_save_prompt(self):
|
|
resp = await self.client.request(
|
|
"POST",
|
|
"/prompt_manager/save",
|
|
json={
|
|
"text": "New prompt via API",
|
|
"category": "test",
|
|
"tags": ["api", "test"],
|
|
},
|
|
)
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
self.assertIn("prompt_id", data)
|
|
|
|
async def test_save_empty_text(self):
|
|
resp = await self.client.request(
|
|
"POST", "/prompt_manager/save", json={"text": ""}
|
|
)
|
|
self.assertEqual(resp.status, 400)
|
|
data = await resp.json()
|
|
self.assertFalse(data["success"])
|
|
|
|
async def test_save_with_null_optional_fields(self):
|
|
"""Regression for #143: UI sends null for empty optional fields."""
|
|
resp = await self.client.request(
|
|
"POST",
|
|
"/prompt_manager/save",
|
|
json={"text": "Null notes prompt", "category": None, "notes": None},
|
|
)
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
|
|
async def test_save_with_null_text(self):
|
|
resp = await self.client.request(
|
|
"POST", "/prompt_manager/save", json={"text": None}
|
|
)
|
|
self.assertEqual(resp.status, 400)
|
|
|
|
async def test_delete_prompt(self):
|
|
pid = self._save_prompt("To delete")
|
|
resp = await self.client.request("DELETE", f"/prompt_manager/delete/{pid}")
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
|
|
async def test_delete_nonexistent(self):
|
|
resp = await self.client.request("DELETE", "/prompt_manager/delete/99999")
|
|
self.assertEqual(resp.status, 404)
|
|
data = await resp.json()
|
|
self.assertFalse(data["success"])
|
|
|
|
|
|
class TestUpdatePrompt(APITestCase):
|
|
|
|
async def test_update_text(self):
|
|
pid = self._save_prompt("Original text")
|
|
resp = await self.client.request(
|
|
"PUT",
|
|
f"/prompt_manager/prompts/{pid}",
|
|
json={"text": "Updated text"},
|
|
)
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
|
|
async def test_update_rating(self):
|
|
pid = self._save_prompt("Ratable")
|
|
resp = await self.client.request(
|
|
"PUT",
|
|
f"/prompt_manager/prompts/{pid}/rating",
|
|
json={"rating": 4},
|
|
)
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
|
|
async def test_add_tag(self):
|
|
pid = self._save_prompt("Taggable", tags=["old"])
|
|
resp = await self.client.request(
|
|
"POST",
|
|
f"/prompt_manager/prompts/{pid}/tags",
|
|
json={"tag": "new_tag"},
|
|
)
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
|
|
|
|
class TestUsageSorting(APITestCase):
|
|
"""Server-side sort for /recent and /search (3.2.4, #90.1)."""
|
|
|
|
def _use(self, prompt_id, times=1):
|
|
for _ in range(times):
|
|
self.api.db.record_prompt_use(prompt_id)
|
|
|
|
async def _ids(self, path):
|
|
resp = await self.client.request("GET", path)
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
return [p["id"] for p in data["results"]]
|
|
|
|
async def test_recent_sorted_by_last_used_bubbles_rerun_prompt(self):
|
|
old = self._save_prompt("Old prompt")
|
|
self._save_prompt("New prompt")
|
|
self._use(old)
|
|
ids = await self._ids("/prompt_manager/recent?limit=1&sort=last_used_desc")
|
|
self.assertEqual(ids, [old])
|
|
|
|
async def test_recent_sorted_by_run_count(self):
|
|
# Most-run prompt is the oldest, so newest-first would give the opposite order
|
|
most = self._save_prompt("Run thrice")
|
|
least = self._save_prompt("Run once")
|
|
self._use(most, times=3)
|
|
self._use(least)
|
|
ids = await self._ids("/prompt_manager/recent?sort=run_count_desc")
|
|
self.assertEqual(ids, [most, least])
|
|
|
|
async def test_recent_default_is_newest_first(self):
|
|
first = self._save_prompt("First")
|
|
second = self._save_prompt("Second")
|
|
self._use(first)
|
|
self.assertEqual(await self._ids("/prompt_manager/recent"), [second, first])
|
|
|
|
async def test_unknown_sort_falls_back_instead_of_erroring(self):
|
|
self._save_prompt("Only")
|
|
ids = await self._ids(
|
|
"/prompt_manager/recent?sort=created_at;DROP%20TABLE%20prompts"
|
|
)
|
|
self.assertEqual(len(ids), 1)
|
|
|
|
async def test_search_respects_sort(self):
|
|
a = self._save_prompt("castle at dawn")
|
|
self._save_prompt("castle at dusk")
|
|
self._use(a)
|
|
ids = await self._ids("/prompt_manager/search?text=castle&sort=last_used_desc")
|
|
self.assertEqual(ids[0], a)
|
|
|
|
async def test_results_include_usage_fields(self):
|
|
pid = self._save_prompt("Tracked")
|
|
self._use(pid, times=2)
|
|
resp = await self.client.request("GET", "/prompt_manager/recent")
|
|
prompt = (await resp.json())["results"][0]
|
|
self.assertEqual(prompt["run_count"], 2)
|
|
self.assertTrue(prompt["last_used_at"])
|
|
|
|
|
|
class TestTags(APITestCase):
|
|
|
|
async def test_get_all_tags(self):
|
|
self._save_prompt("P1", tags=["alpha", "beta"])
|
|
self._save_prompt("P2", tags=["beta", "gamma"])
|
|
resp = await self.client.request("GET", "/prompt_manager/tags")
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
self.assertIn("tags", data)
|
|
|
|
async def test_get_tag_stats(self):
|
|
self._save_prompt("P1", tags=["common", "rare"])
|
|
self._save_prompt("P2", tags=["common"])
|
|
resp = await self.client.request("GET", "/prompt_manager/tags/stats")
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
|
|
async def test_rename_tag(self):
|
|
self._save_prompt("P1", tags=["old_name"])
|
|
resp = await self.client.request(
|
|
"PUT",
|
|
"/prompt_manager/tags/old_name",
|
|
json={"new_name": "new_name"},
|
|
)
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
|
|
async def test_delete_tag(self):
|
|
self._save_prompt("P1", tags=["removable"])
|
|
resp = await self.client.request("DELETE", "/prompt_manager/tags/removable")
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
|
|
async def test_merge_tags(self):
|
|
self._save_prompt("P1", tags=["target"])
|
|
self._save_prompt("P2", tags=["source"])
|
|
resp = await self.client.request(
|
|
"POST",
|
|
"/prompt_manager/tags/merge",
|
|
json={"source_tags": ["source"], "target_tag": "target"},
|
|
)
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
|
|
|
|
class TestCategories(APITestCase):
|
|
|
|
async def test_get_categories(self):
|
|
self._save_prompt("P1", category="nature")
|
|
self._save_prompt("P2", category="urban")
|
|
resp = await self.client.request("GET", "/prompt_manager/categories")
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
self.assertIn("categories", data)
|
|
|
|
|
|
class TestStats(APITestCase):
|
|
|
|
async def test_get_stats(self):
|
|
self._save_prompt("P1", category="test", tags=["t1"], rating=4)
|
|
resp = await self.client.request("GET", "/prompt_manager/stats")
|
|
self.assertEqual(resp.status, 200)
|
|
data = await resp.json()
|
|
self.assertTrue(data["success"])
|
|
self.assertIn("stats", data)
|
|
|
|
|
|
class TestExport(APITestCase):
|
|
|
|
async def test_export_json(self):
|
|
self._save_prompt("Exportable", category="test", tags=["export"])
|
|
resp = await self.client.request("GET", "/prompt_manager/export?format=json")
|
|
self.assertEqual(resp.status, 200)
|
|
# Export returns raw JSON file download, not a success envelope
|
|
self.assertIn("Content-Disposition", resp.headers)
|
|
text = await resp.text()
|
|
data = json.loads(text)
|
|
self.assertIn("prompts", data)
|
|
self.assertEqual(len(data["prompts"]), 1)
|
|
|
|
|
|
class TestResponseEnvelope(APITestCase):
|
|
"""Verify all responses follow the {success: bool, ...} envelope."""
|
|
|
|
async def test_success_responses_have_success_true(self):
|
|
self._save_prompt("Envelope test")
|
|
endpoints = [
|
|
("GET", "/prompt_manager/test"),
|
|
("GET", "/prompt_manager/recent"),
|
|
("GET", "/prompt_manager/search"),
|
|
("GET", "/prompt_manager/categories"),
|
|
("GET", "/prompt_manager/tags"),
|
|
("GET", "/prompt_manager/stats"),
|
|
]
|
|
for method, path in endpoints:
|
|
resp = await self.client.request(method, path)
|
|
data = await resp.json()
|
|
self.assertIn("success", data, f"Missing 'success' key in {method} {path}")
|
|
self.assertTrue(
|
|
data["success"], f"Expected success=True for {method} {path}"
|
|
)
|
|
|
|
async def test_error_responses_have_success_false(self):
|
|
resp = await self.client.request("DELETE", "/prompt_manager/delete/99999")
|
|
data = await resp.json()
|
|
self.assertIn("success", data)
|
|
self.assertFalse(data["success"])
|
|
|
|
|
|
if __name__ == "__main__":
|
|
unittest.main()
|