Local Deep Research logo

Local Deep Research

LearningCircuit/local-deep-research

Source Code35 approved9.6k
PythonSQLCipherSQLAlchemyRAGSecurityFlask

overview.

Local Deep Research runs multi-step web and academic research locally over per-user SQLCipher databases, 34 search engines, and isolated document RAG collections. I have 35 pull requests approved by the maintainer (all PRs) spanning log-redaction security, database and engine invariants, and deterministic citation scanning.

Work across the 35 maintainer-approved PRs covers closing Unicode format-character (U+00AD, U+2065), surrogateescape, and URL-credential bypasses in redact_for_log and auth/parser error paths (PR #8824, PR #8848, PR #8823); preserving _temp_attributes plain-dict invariants across 34 search engines, immutable DownloadFailure tokens, competing RAGDocumentStatus deletes, and SQLCipher rekey diagnostics (PR #8853, PR #8811, PR #8812, PR #8865); and enforcing deterministic _MeasuredStr / _TrackedStr linear-scan bounds in CitationFormatter (PR #8804, PR #8810, PR #8826).

key features.

  • PR #8824, PR #8848 & PR #8823: closed Unicode format-character (U+00AD, U+2065), surrogateescape, and URL-credential bypasses in redact_for_log and auth/parser error paths
  • PR #8853, PR #8811, PR #8812 & PR #8865: preserved _temp_attributes plain-dict invariants across 34 search engines, immutable DownloadFailure tokens, competing RAGDocumentStatus deletes, and SQLCipher rekey diagnostics
  • PR #8804, PR #8810 & PR #8826: enforced deterministic _MeasuredStr and _TrackedStr linear-scan bounds in CitationFormatter

challenges & solutions.

Challenge: Soft hyphens (U+00AD), invisible format characters (U+2065), and surrogateescape bytes split bearer tokens across regex boundaries in redact_for_log.

Solution: Hardened redact_for_log against Unicode and surrogateescape bypasses, scrubbed URL credentials, and switched auth/parser logs to log_exception_type and value-free line/column errors.

Challenge: Wall-clock timing assertions for CitationFormatter flaked on shared CI runners while quadratic scan regressions needed strict enforcement.

Solution: Used instrumented _MeasuredStr and _TrackedStr string subclasses to pin exact deterministic linear character-scan bounds independent of runner CPU speed.

project impact.

35 Approved
Approved Pulls
34 Engines
Search Engines
Log Redaction
Security
9.6k+ Stars
Repo Stars

Engineering Deep Dive