Late November Open Source: 10 Merges in TokenEconomizer and 35 Approved PRs in Local Deep Research

· 3 min readoriginal postPost views

In the second half of November, five more pull requests merged upstream: four in microsoft/TokenEconomizer (7.7k stars, bringing my merged total there to ten) and one in heymrun/heym (1.8k stars). Alongside those merges, I went deep into LearningCircuit/local-deep-research (9.6k stars), where 35 of my pull requests have been approved by the maintainer and are queued for merge.

Three of the TokenEconomizer merges closed JSONL and schema edge cases. A line of valid JSON is not guaranteed to be a JSON object: null, 42, "str", and [...] all pass json.loads and then crash .get() callers with AttributeError. PR #199 hardened microskills call loading against non-dict JSON lines, PR #198 guarded parse_payload when a proposer returns wrongly-typed fields, and PR #200 fixed a subtler bug in Store.read_lines and accounting.summarize: filtering non-object JSONL records before slicing limit, so trailing non-dict lines in a ledger cannot silently evict real records from a bounded tail read.

The fourth TokenEconomizer merge, PR #259, tackled multi-channel sessions in Hermes Agent. Hermes runs background gateway sessions across Discord, Telegram, an API server, and scheduled cron jobs, so relying on a process-wide TERMINAL_CWD cross-contaminates repository state when two chats point at different repos. PR #259 resolves each session's repository working directory independently through SQLite sessions.cwd, gateway_routing terminal_cwd, and scoped_session_cwd(), and registers the /te slash command. Separately, heym PR #1030 fixed run_tests.sh so a parallel test worker that crashes before writing its status file fails closed instead of letting the script exit 0.

The rest of late November went into LearningCircuit/local-deep-research. It is a rewarding codebase to work on: encrypted SQLCipher per-user databases, 34 pluggable academic and web search engines, isolated document loaders, RAG indexing, and strict structural and redaction test contracts. To keep status clear, these 35 pull requests are maintainer-approved and awaiting merge rather than merged yet, and they group into three engineering areas.

First, log, error, and parser secret redaction (10 approved PRs). PR #8824 closed redact_for_log bypasses where soft hyphens (U+00AD), invisible format characters (U+2065), or surrogateescape undecodable bytes split bearer tokens and API keys across regex boundaries. PR #8848 and PR #8816 scrubbed URL credentials in categorization errors and routed OpenAlex error bodies through patcher-safe redaction. PR #8823, PR #8863, and PR #8864 replaced raw exception and path interpolation with log_exception_type, value-free YAML and JSON line-and-column parse errors, and short_id logging, while PR #8802 raised the soupsieve>=4.0 floor for CVE-2026-126190.

Second, SQLCipher, RAG, and 34-engine search invariants (13 approved PRs). PR #8853 kept _temp_attributes plain-dict invariants intact across all 34 search engines. PR #8811 replaced live exception retention in the download tracker with an immutable DownloadFailure token so tracebacks and locals do not stay pinned in memory. PR #8812 handled competing RAGDocumentStatus row deletes and SQL NULL counter drift, PR #8868 returned _AUTO_PERSIST_PENDING admission permits cleanly, PR #8832 protected Markdown code spans and URLs during Quarto citation rewriting, and PR #8865 and PR #8869 surfaced SQLCipher describe_rekey_fault diagnostics and distinct post-rekey backup skip reasons.

Third, deterministic linear-scan bounds and frontend state (12 approved PRs). Wall-clock timing assertions flake on slow CI runners, so PR #8804, PR #8810, and PR #8826 used instrumented str subclasses (_MeasuredStr, _TrackedStr) to pin exact linear character-scan bounds in CitationFormatter and _line_lead. Rounding out this set, PR #8827 verified socket-free over-budget HTTPS proxy re-entry, PR #8845 pinned wheel RECORD inventory on refreshed_contents, and PR #8856 kept superseded settings save state consistent across React and DOM re-renders.

Across 17 merged pull requests in November (12 in the first half, 5 in the second) and 35 maintainer-approved ones in local-deep-research, the sharpest tests were structural rather than mock-heavy. Counting exact character comparisons beats time.perf_counter() on shared CI runners, and checking SQL or JSONL boundaries before limit slices catches data loss that clean fixtures hide. Treating every log call in an auth or parser path as an untrusted sink pays off immediately once you see how many redaction bypasses hide inside invisible Unicode characters and raw exception strings.

Related projects