Local Deep Research
LearningCircuit/local-deep-research
overview.
Local Deep Research runs multi-step web and academic research locally over per-user SQLCipher databases, 34 search engines, and isolated document RAG collections. I have 35 pull requests approved by the maintainer (all PRs) spanning log-redaction security, database and engine invariants, and deterministic citation scanning.
Work across the 35 maintainer-approved PRs covers closing Unicode format-character (U+00AD, U+2065), surrogateescape, and URL-credential bypasses in redact_for_log and auth/parser error paths (PR #8824, PR #8848, PR #8823); preserving _temp_attributes plain-dict invariants across 34 search engines, immutable DownloadFailure tokens, competing RAGDocumentStatus deletes, and SQLCipher rekey diagnostics (PR #8853, PR #8811, PR #8812, PR #8865); and enforcing deterministic _MeasuredStr / _TrackedStr linear-scan bounds in CitationFormatter (PR #8804, PR #8810, PR #8826).
key features.
- PR #8824, PR #8848 & PR #8823: closed Unicode format-character (U+00AD, U+2065), surrogateescape, and URL-credential bypasses in redact_for_log and auth/parser error paths
- PR #8853, PR #8811, PR #8812 & PR #8865: preserved _temp_attributes plain-dict invariants across 34 search engines, immutable DownloadFailure tokens, competing RAGDocumentStatus deletes, and SQLCipher rekey diagnostics
- PR #8804, PR #8810 & PR #8826: enforced deterministic _MeasuredStr and _TrackedStr linear-scan bounds in CitationFormatter
challenges & solutions.
Challenge: Soft hyphens (U+00AD), invisible format characters (U+2065), and surrogateescape bytes split bearer tokens across regex boundaries in redact_for_log.
Solution: Hardened redact_for_log against Unicode and surrogateescape bypasses, scrubbed URL credentials, and switched auth/parser logs to log_exception_type and value-free line/column errors.
Challenge: Wall-clock timing assertions for CitationFormatter flaked on shared CI runners while quadratic scan regressions needed strict enforcement.
Solution: Used instrumented _MeasuredStr and _TrackedStr string subclasses to pin exact deterministic linear character-scan bounds independent of runner CPU speed.