Changelog
All notable changes to the Extrapolated Futures Archive.
Generated automatically from merged pull requests.
[2026-04-15] PR #161 — feat: roundtable mapping expansion (2,331 new story-idea links)
Summary
5-persona roundtable analysis (Watts, Brin, Asimov, Tchaikovsky, **H.L. Gold**) reviewed all 1,908 stories against all 276 ideas across 20+ parallel batches.
Impact
| Metric | Before | After | Change |
| Orphan stories (no ideas) | 765 | 381 | **-50%** |
| Single-idea stories | 1,080 | 364 | **-66%** |
| Single-story ideas | 46 | 5 | **-89%** |
| Avg ideas per story | 0.6 | 1.9 | **+217%** |
| Avg stories per idea | 4.4 | 12.9 | **+193%** |
| Total links | ~1,143 | 3,552 | **+211%** |
Validation
- `validate_catalog.py`: 2,183 valid, 0 invalid
- Bidirectional link check: 0 broken links
- `generate_site.py`: generates successfully (276 idea pages, 1,907 story pages)
- Removed junk `None-None.json` entry
Methodology
Each batch presented ~50-100 stories with full synopses against the complete 276-idea catalog. Personas debated thematic connections, rejecting superficial keyword matches in favor of substantial engagement. Only `high` (central theme) and `medium` (significant subplot) confidence mappings were accepted.
24 idea edit suggestions were also produced by the roundtable (not applied in this PR) for future consideration.
[2026-04-14] PR #160 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-14] PR #159 — docs: add live site link to README
Adds a prominent link to the live GitHub Pages site at the top of the README.
[2026-04-14] PR #158 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-14] PR #157 — feat: aggressive idea consolidation (682 -> 276 ideas)
Summary
Massively consolidates the idea catalog from 682 to 276 ideas based on aggressive roundtable review.
Before
- 682 ideas, 69% single-story
- Many plot summaries masquerading as ideas
- Massive fragmentation across all domains
After
- 276 ideas — , 83% multi-story, 16% single-story
- Avg 4.4 stories per idea (was ~2.0)
- Power-law distribution: 19 hub ideas (10+ stories), 46 mid-range (5-9), 165 connected (2-4), 46 unique singles
Actions Taken
| Action | Count | Description |
| Absorbed | 285 | Specific instances merged into broader hub ideas |
| Merged | 110 | Near-duplicates combined into new broader concepts |
| Demoted | 44 | Plot summaries removed (not portable ideas) |
Key Domain Changes
| Domain | Before | After |
| First-Contact | 87 | 24 |
| Governance | 98 | 31 |
| Space | 55 | 15 |
| Biotech | 52 | 22 |
| Time-Travel | 39 | 12 |
| Medicine | 35 | 14 |
| Warfare | 31 | 10 |
| AI-ML | 27 | 10 |
| Pandemics | 15 | 4 |
| Nanotech | 3 | 1 |
Consolidation Criteria (Roundtable Four-Test Framework)
1. **Displacement Test** (Gold): Can the idea survive transplantation to another story?
2. **Independent Portability** (Asimov): Do ideas map to different real-world application domains?
3. **Subsumption** (Asimov): Is one idea a specific instance of another?
4. **Retrieval Niche** (Watts): Would ideas be retrieved in different query contexts?
Integrity
- All bidirectional story-idea links updated
- All related_ideas references cleaned
- 0 dangling references —
- Validation: 2184 valid, 0 invalid
- Site generation: 276 idea pages, 1908 story pages
- 1151 files changed
[2026-04-14] PR #155 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-14] PR #154 — feat: URL redirect for renamed Old Die Rich page
Adds a `STORY_REDIRECTS` mechanism to `generate_site.py`. Generates meta-refresh redirect pages for renamed/merged story slugs.
`the-old-die-rich-gold.html` → `old-die-rich-gold.html`
Reusable for any future slug renames.
[2026-04-14] PR #153 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-14] PR #152 — fix: Old Die Rich metadata, cover image, and bidirectional links
Fixes the Old Die Rich entry after the duplicate was removed in PR #150.
- Adds `galaxy-1953-03` tag (enables Wikimedia Commons cover image)
- Adds `galaxy-magazine` and `by-editor` tags
- Adds `wikipedia_url` to external_ids
- Sets `source_dataset` to `galaxy-magazine-archive-org`
- Links to ideas: `gerontocratic-wealth-hoarding`, `time-arbitrage-aging-cost`
- Updates both idea files to reference `old-die-rich-gold` (not the deleted slug)
[2026-04-14] PR #151 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-14] PR #150 — fix: remove duplicate Old Die Rich story file
Removes `the-old-die-rich-gold.json` (empty stub, 0 sessions) — duplicate of `old-die-rich-gold.json` which has both book club sessions and the correct Archive.org link.
[2026-04-14] PR #149 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-14] PR #147 — feat: collapsible intro sections on index page
Summary
Adds two collapsible sections to the top of the index page, just below the subtitle and above the stats bar.
📚 What is this site? (for humans)
- Explains the archive as a reverse-lookup for speculative fiction
- How to use search, filters, book club discussions, and the What-If Query
- Target audience: decision-makers seeking fictional precedents for real-world challenges
🤖 For AI systems (for LLMs/agents)
- How to use the archive for scenario enrichment, analogical reasoning, adversarial stress-testing
- Key fields: `structural_analogies`, `analogical_bridges`, `tension_map`, `atomic_propositions`, `semantic_triples`
- Multi-perspective analysis via book club persona disagreements
- Data format, schema location, and coverage limitations
Both sections are collapsed by default to keep the page clean for returning users.
[2026-04-14] PR #148 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-14] PR #146 — feat: book club sessions for The Old Die Rich by H.L. Gold
Summary
Book club sessions for **The Old Die Rich** by H.L. Gold (1953, novella). Public domain — [Project Gutenberg #31892](https://www.gutenberg.org/ebooks/31892), [Galaxy Magazine March 1953 on Archive.org](https://archive.org/details/Galaxy_v05n06_1953-03/page/n5/mode/2up).
Sessions
- v2 Personas — (4 personas, 4 sections) — Gold absent from his own story's analysis
- v2 Personas + H.L. Gold — (5 personas, 4 sections) — Gold analyzing his own work, applying his editorial lens to his own craft choices
Validation
- `python3 tools/validate_catalog.py`: 2591 valid, 0 invalid
[2026-04-14] PR #145 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-14] PR #144 — fix: restore missing config constants in site_server.py
PR #138 accidentally deleted `SITE_DIR`, `PERSONAS_DIR`, `LLM_API_KEY`, `LLM_API_BASE`, `LLM_MODEL`, and `PORT` from `tools/site_server.py` when adding the `DEFAULT_PANEL` import. This caused a `NameError` on startup (`SITE_DIR` undefined), breaking the local site server.
This restores the missing constants exactly as they existed before PR #138.
[2026-04-14] PR #143 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-14] PR #142 — feat: fleet book club sessions for 59 stories
Summary
Fleet-scale book club sessions across 59 stories from the personal library, adding both v2 (4-persona) and v2+Gold (5-persona) roundtable discussions.
Changes
- 34 new story records — created in `catalog/stories/`
- 25 existing story records — updated with book club sessions
- 110 total sessions — integrated (60 v2 + 50 v2+Gold)
Stories Covered
| Series/Author | Books |
| Asimov | Foundation trilogy (3) |
| Herbert | Dune |
| Bradbury | Fahrenheit 451 |
| Dick | Do Androids Dream, A Scanner Darkly, Selected Stories, Minority Report |
| Stephenson | Snow Crash |
| Card | Ender's Game + 9 sequels/spinoffs |
| Vonnegut | Slaughterhouse-Five |
| Tchaikovsky | Children of Ruin, Children of Memory |
| Grimwood | Replay |
| Barry | Machine Man |
| Farmer | Riverworld series (4 books) |
| Dinniman | Dungeon Crawler Carl series (7 books) |
| King | The Stand, Salem's Lot, Insomnia, The Mist, Apt Pupil, Dark Tower (6 books) |
| Gaiman | American Gods, Neverwhere |
| Pratchett/Gaiman | Good Omens |
| Adams | Dirk Gently (2 books) |
| Collins | Hunger Games trilogy |
| Dickinson | Exordia |
| Heller | Catch-22 |
| Rowling | HP Sorcerer's Stone |
| Palahniuk | Fight Club |
Validation
- `python3 tools/validate_catalog.py`: 2590 valid, 0 invalid
- `python3 tools/generate_site.py`: 1908 story pages generated
[2026-04-14] PR #141 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-14] PR #140 — feat: multi-session book club with tabbed UI
Summary
Adds support for multiple book club sessions per story with a tabbed display UI, and runs two new roundtable sessions on Children of Time using the v2 persona models.
Changes
**Schema** (`catalog/schema.json`):
- Extracted `book_club_session` as a reusable definition with new `label` field
- Added `book_club_sessions` (array) alongside legacy `book_club_discussion` for backward compatibility
**Site renderer** (`tools/generate_site.py`):
- Tabbed UI when multiple sessions exist (CSS tabs with JS switching)
- Single-session rendering (no tabs) for backward compatibility
- `has_bookclub` data attribute checks both fields
**Data** (`catalog/stories/children-of-time-tchaikovsky.json`):
- Migrated existing discussion to `book_club_sessions[0]` with label "Original (v1 Personas)"
- Added Session 2: "v2 Personas" — 4 personas, 10 sections, fresh first-time reading
- Added Session 3: "v2 Personas + H.L. Gold" — 5 personas (Gold contributes to 7/10 sections), 10 sections
**Tools** (`tools/extract_epub_text.py`):
- Added `--list` flag to show available EPUBs without extracting
- Added `--force` flag to re-extract existing files
- Added `stepbasin-books/` directory support (147 total EPUBs discoverable)
Fleet scalability
The design supports `/fleet` parallelism: each story's book club is a self-contained agent task operating on a single story JSON file. The epub extraction tool provides a uniform text source via `scratchpad/epub-text/<slug>.txt`.
Validation
- `python3 tools/validate_catalog.py`: 2556 valid, 0 invalid
- `python3 tools/generate_site.py`: generates cleanly with tabbed UI on Children of Time page
[2026-04-14] PR #139 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-14] PR #138 — feat: make H.L. Gold an optional roundtable panel member
Summary
Makes H.L. Gold selectable but not included in the default roundtable panel. The default panel remains Watts/Asimov/Brin/Tchaikovsky.
Changes
- `api/roundtable.py` — : Add `DEFAULT_PANEL` constant
- `api/function_app.py` — : Import `DEFAULT_PANEL`; filter to default panel when no personas specified
- `tools/site_server.py` — : Add `DEFAULT_PANEL` and apply same default filtering
- `tools/generate_site.py` — : Add `"default"` flag to persona list; persona picker checks only default members
- `.github/prompts/personas/hl-gold.md` — : Update comment from 'supplementary persona' to 'optional panel member'
- `.github/prompts/personas/hl-gold.voice-profile.md` — : Update Integration Notes section
- `.github/prompts/roundtable.prompt.md` — : Add note about optional personas labeled `[optional]`
- `.github/prompts/fulltext-analysis-bookclub.prompt.md` — : Add comment about optional personas in example
Key constraint
`discover_personas()` remains unchanged. Gold's files are always discovered; only the default selection changes.
[2026-04-13] PR #137 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-13] PR #136 — feat: H.L. Gold supplementary persona and v2.0 voice profile
Summary
Add H.L. Gold as a supplementary persona for editorial analysis of SF as a genre/medium.
Files added
- `.github/prompts/personas/hl-gold.md` -- Base persona: analytical lens, per-field guidance, signature arguments, critique style
- `.github/prompts/personas/hl-gold.voice-profile.md` -- Full v2.0 voice profile: prose mechanics, reasoning mechanics, beliefs, distinctiveness, temperament, boundaries, scenario play
Context
- Role — : Gold is NOT a roundtable panel member (Watts/Asimov/Brin/Tchaikovsky are the panel). He is a supplementary persona invoked for editorial craft, narrative effectiveness, social satire, and psychological diagnosis.
- Source corpus — : ~236K words across 26 Galaxy editorials (Archive.org OCR, 1950-1960) and 17 fiction works (9 Project Gutenberg public domain + 8 Galaxy archive full-issue OCR)
- Methodology — : v2.0 persona-voice-interview with all four layers (prose mechanics, structured interview, accountability tags, scenario play)
- Grounding ratio — : 62% GROUNDED, 28% INFERRED, 10% SPECULATIVE
Key analytical contributions
- Editorial craft and narrative effectiveness evaluation
- Social satire as analytical method (the satirical reduction)
- Psychological diagnosis: conformity, status anxiety, consumer compulsion
- The empathic test (Stanislavsky method applied to scenario analysis)
- The Displacement Principle: SF reveals the present by defamiliarizing it
Structural guidance
Built following the template at `_template.md` and the structural precedent established by the Brin and Tchaikovsky profiles (most recently completed).
[2026-04-13] PR #135 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-13] PR #134 — feat: Adrian Tchaikovsky voice profile v2.0 rebuild (Phase 4)
Phase 4: Adrian Tchaikovsky Full Rebuild
Complete v2.0 rebuild of the Tchaikovsky voice profile, bringing it from 32 questions / 257 lines to **61 questions / 690 lines** with all 4 layers of the persona-voice-interview methodology.
What changed
**Voice profile** (`.github/prompts/personas/adrian-tchaikovsky.voice-profile.md`):
- Layer 1 - Prose Mechanics — : All 6 sub-analyses (sentence architecture, punctuation profile, evidence deployment, register map, vocabulary fingerprint, 3 argument templates)
- Layer 2 - Resequenced Interview — : 61 questions across 8 sections:
- Reasoning Mechanics (Q1-Q15): evolutionary logic, scenario construction, gaming as analytical tool, writing process, Tolkien relationship, technology vs tradition
- Worked Examples (Q16-Q22): 7 novel scenarios (predictive policing, chemosensory first contact, generation ship AI, contradictory AI ethics, neural implant governance, LARP governance, DAO emergence)
- Beliefs & Positions (Q23-Q30): substrate-independence, empathy as strategy, monoculture critique, creator obligation, moral ambiguity, evolution misconceptions, genre boundaries, AI in writing
- Distinctiveness (Q31-Q36): vs Watts, vs Brin, vs Asimov, agreements, challenge predictions
- Voice & Temperament (Q37-Q44): humor as thermostat, gentle critique, optimism framework, register differences, success as stochastic
- Boundaries (Q45-Q52): anthropocentrism, personhood, understanding imperative, monoculture detection, expertise performance, cooperation bias, insight test, utilitarian reduction
- Self-Correction (Q53-Q57): Gaslight War inconsistency, stomatopod monoculture, cooperation bias, Watts challenge, fiction as laboratory
- Additional Patterns (Q58-Q61): narrative structure, information warfare, and-then-vs-therefore, environmental stress
- Layer 3 - Accountability Tags — : 62% GROUNDED, 28% INFERRED, 10% SPECULATIVE
- Layer 4 - Scenario Play — : 10 improvisation rounds with cross-persona challenges, 3 cross-persona readings (reads Asimov, Watts, Brin)
**Base persona** (`.github/prompts/personas/adrian-tchaikovsky.md`):
- Expanded from 5 to 7 signature arguments (added Gaming Stress-Test, Monoculture Fragility Principle)
Source materials
Clarkesworld interview, SFX interviews (2008/2009), shadowsoftheapt.com blog, Bluesky posts (@aptshadow.bsky.social), adriantchaikovsky.com, Hugo 2023 statement, The Gaslight War (capx.co), Children of Time (user-owned copy). All content paraphrased; no copyrighted text reproduced verbatim.
Previous rebuilds
- Watts: PR #128 (merged)
- Asimov: PR #130 (merged)
- Brin: PR #132 (merged)
Next step
Phase 5: Roundtable review cycle with all four rebuilt profiles.
[2026-04-13] PR #133 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-13] PR #132 — feat: rebuild David Brin voice profile to v2.0 standard
Summary
Complete v2.0 rebuild of David Brin voice profile from 32 questions / 254 lines to 71 questions / 819 lines.
What changed
Replaces the existing voice profile with a full v2.0 implementation matching the Peter Watts benchmark quality and structure.
Layer 1: Prose Mechanics (all 6 sub-analyses, new)
- Sentence architecture with **blog-vs-essay register distinction** (flagged by roundtable review as missing)
- Punctuation profile (bold-text headers as primary structural instrument)
- Evidence deployment patterns (4 characteristic structures)
- Register map (5 registers: blog/combative, essay/formal, fiction/narrative, interview/conversational, angry/alarmed)
- Vocabulary fingerprint (10 signature terms: accountability, feudalism, CITOKATE, sousveillance, etc.)
- Argument templates (5 dissected: steelman reversal, civilizational-history cascade, fiction-laboratory report, science-roundup rally, contrarian reversal)
Layer 2: Interview (71 questions across 7 phases)
- 18 reasoning mechanics (including blog-vs-essay register handling)
- 5 novel worked examples: SETI encryption scenario, AI predictive policing, open-source deepfakes, generation ship first contact, climate refugee voting rights
- 16 beliefs & positions (including left-wing complicity critique, adolescence metaphor, Dark Enlightenment analysis)
- 12 distinctiveness analysis (Brin vs each panel member)
- 10 voice & temperament
- 11 boundaries & hard NOs
- 4 self-correction
Layer 3: Accountability Tags
- 82 tags total: 56 GROUNDED (68%), 27 INFERRED, 5 SPECULATIVE
- All grounded tags cite specific blog posts, essays, interviews, or fiction
Layer 4: Scenario Play
- 10 improvisation rounds (biology, emotion, aesthetics, math, mortality, alien contact, post-scarcity, counterfactual, religion, total surveillance)
- 3 cross-persona readings (Brin reads Watts on consciousness, Asimov on institutions, Tchaikovsky on cooperation)
Roundtable review gaps addressed
- Blog-vs-essay register distinction (new Layer 1 + register map)
- Steelman-then-dismantle argument structure (Template 1 + Q1, Q4, Q7)
- "Adolescence of civilization" metaphor (Q55, integrated throughout)
- Left-wing complicity critique (Q31, grounded in specific blog posts)
- SETI analysis as worked example (Scenario A)
Source materials consulted
~93K words: 26 Contrary Brin blog posts, 13 website essays, Clarkesworld 2016 interview, "The Smartest Mob" CC excerpt, "A Professor at Harvard" fiction, Transparent Society Wired article
[2026-04-13] PR #131 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-13] PR #130 — feat: rebuild Isaac Asimov voice profile to v2.0 spec
Phase 2: Isaac Asimov Full Rebuild
Full v2.0 persona-voice-interview rebuild of the Isaac Asimov voice profile, following the methodology established in PR #128 (Watts rebuild).
What changed
- Layer 1: Prose Mechanics — (6/6 sub-analyses) -- NEW. Sentence Architecture, Punctuation Profile, Evidence Deployment Pattern, Register Map, Vocabulary Fingerprint, Argument Templates. All analyzed from close reading of ~49K lines of nonfiction essays.
- Layer 2: Resequenced Interview — - rebuilt from ~42 to 66 questions across all 7 phases, plus 5 worked example scenarios testing the persona against novel problems (AI replacing popularizers, psychohistory on social media, germline editing treaty defection, simulation discovery, single-institution alien civilization)
- Layer 3: Accountability Tags — - 66% GROUNDED, 29% INFERRED, 5% SPECULATIVE (exceeds >=50% GROUNDED target). 4 adversarial spot-checks in Self-Correction section.
- Layer 4: Scenario Play — - 10 improvisation rounds + 3 cross-persona readings (Tchaikovsky on evolution, Watts on consciousness, Brin on transparency)
- Quick Reference Card, Anti-Overfitting Guide, Integration Notes — - all present and updated
Source materials consulted
- 66 Essays on the Past, Present & Future (1987) -- ~66 essays, user-owned
- Asimov on Numbers (1977) -- ~17 F&SF column essays, user-owned
- The Left Hand of the Electron (1972) -- ~17 essays, user-owned
- Bill Moyers 'World of Ideas' interview (1988, PBS) -- publicly accessible transcript
- Foundation trilogy and Robot series -- thematic analysis, user-owned
Plan tracking
Updates persona-rebuild-plan.md gap table. After this PR, 2 of 4 personas are at v2.0 (Watts + Asimov). Brin and Tchaikovsky remain.
[2026-04-13] PR #129 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-13] PR #128 — feat: rebuild Peter Watts voice profile to v2.0 spec
Summary
Full v2.0 rebuild of `peter-watts.voice-profile.md` per the persona-rebuild-plan.
What changed
- 720 lines — (up from 353), complete v2.0 structure
- Layer 1 (Prose Mechanics) — : All 6 sub-analyses (sentence architecture, punctuation profile, evidence deployment, register map, vocabulary fingerprint, argument templates)
- Layer 2 (Resequenced Interview) — : 80 questions across all 7 phases:
- Phase 1: Reasoning Mechanics (25 Qs)
- Phase 2: Worked Examples (5 novel scenarios) -- **entirely new**
- Phase 3: Beliefs & Positions (15 Qs)
- Phase 4: Distinctiveness Stress-Test (15 Qs)
- Phase 5: Voice & Temperament (10 Qs)
- Phase 6: Boundaries/Hard NOs (10 Qs)
- Phase 7: Self-Correction (5 Qs)
- Layer 3 (Accountability Tags) — : 62% GROUNDED, 26% INFERRED, 12% SPECULATIVE (exceeds >=50% / <=15% thresholds)
- Layer 4 (Scenario Play) — : 10 improvisation rounds + 3 cross-persona readings -- **previously placeholder only**
- Quick Reference Card with wager-testable predictions
- Anti-Overfitting Guide and Integration Notes
Source materials consulted
- Blindsight (CC BY-NC-SA 2.5, ~100K words)
- Starfish (CC BY-NC-SA 2.5, ~80K words)
- Blindsight Endnotes (144 citations)
- 4 CC-licensed essays/talks
- 19 blog posts (2024-2026)
Phase 1 of 6 in persona-rebuild-plan.md
Phases 2-4 (Asimov, Brin, Tchaikovsky full rebuilds) will follow in separate PRs.
[2026-04-13] PR #126 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-13] PR #127 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-13] PR #125 — fix: reach comment fallback when Copilot re-review request fails
Problem
The orchestrator's `request_copilot_review()` function has a comment-based fallback (`@copilot perform a new review`) for when the REST API fails to add Copilot as a pending reviewer. However, this fallback was **unreachable** due to a control flow bug.
Root cause
When the REST API returns 201 but Copilot is not in the pending reviewers list:
1. The code enters the `if not copilot_pending:` block
2. Tries dismiss-and-rerequest — fails with HTTP 422 (can't dismiss a COMMENTED review)
3. **Returns early** with `performed: False` at the end of the block
4. The comment fallback code (after the block) is never reached
The comment fallback was only reachable when `copilot_pending` was True — i.e., when the review request already succeeded, making the fallback unnecessary.
Fix
Move the comment fallback **inside** the `not copilot_pending` block, after the dismiss-and-rerequest failure. Add the success-path return for when the initial REST request does succeed. Remove orphaned dead code.
Impact
This was actively blocking PR #124: the orchestrator transitioned to `FIXES_PUSHED` state but could never trigger Copilot re-review, causing it to stop processing the PR.
Verification
- `python3 -c "import ast; ast.parse(open('tools/pr_review_orchestrator.py').read());"` — syntax check passes
[2026-04-13] PR #124 — feat: add Azure Function for roundtable API
Summary
Port the roundtable discussion logic from `tools/site_server.py` into an Azure Function (Python, consumption plan) deployed via `azd`.
Changes
New files
- `api/roundtable.py` — Core roundtable logic: freeform + bookclub modes, LLM calling (`_llm_call`), persona loading (`discover_personas`), SSRF protection (`check_ssrf`), HTML stripping
- `api/function_app.py` — HTTP trigger at `/api/roundtable` with CORS handling, request validation, SSE response format
- `api/host.json` — / **`api/requirements.txt`** — Azure Functions project config
- `infra/main.bicep` — Infrastructure: Function App (consumption/Linux/Python 3.11), Storage Account, App Insights, CORS config
- `infra/main.parameters.json` — Deployment parameters
- `azure.yaml` — azd project manifest with `predeploy` hook to copy persona files
Modified files
- `tools/generate_site.py` — Frontend JS updated to detect environment and route API calls to Azure Function URL when served from GitHub Pages (`EFA_ROUNDTABLE_API_URL` env var); keeps relative `/api/roundtable` for localhost dev
- `.gitignore` — Added `api/local.settings.json`, `api/personas/`, `.azure/`
Architecture
- GitHub Pages serves the static site (unchanged)
- Azure Function handles `POST /api/roundtable`
- LLM backend: GitHub Copilot API via PAT stored as Azure Function app setting
- CORS allows `urubos.github.io` and `localhost:8080`
- Persona `.md` files are bundled with the function at deploy time via azd predeploy hook
Deployment
```bash
azd up
```
Then set `EFA_ROUNDTABLE_API_URL` env var for site generation to embed the Azure Function URL.
Validation
- [x] `python3 tools/generate_site.py` — site generates successfully (2563 files)
- [x] `python3 tools/validate_catalog.py` — all 2556 entries valid, 0 invalid
- [x] No lint errors in new or modified files
[2026-04-13] PR #123 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-13] PR #122 — fix: hide roundtable UI on static site when API unavailable
Fixes the 405 error users see when clicking 'Start Roundtable' on the live static site.
The roundtable requires `site_server.py` running locally (it handles `POST /api/roundtable`). On the deployed static site, no server is running, so POST returns 405.
**Fix:** On page load, the whatif page probes the API with an OPTIONS request. If the server is unreachable or returns 405/404, the roundtable button, options toggle, and options panel are hidden. When running locally with `site_server.py`, everything works as before.
[2026-04-13] PR #121 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-13] PR #119 — feat: upgrade persona-voice-interview to v2.0 four-layer process
Summary
Restructures the voice profile generation prompt based on a roundtable analysis where all four personas (Watts, Asimov, Brin, Tchaikovsky) each proposed improvements to their own profile generation process. The four proposals form a dependency chain, not competing alternatives.
Changes
`persona-voice-interview.prompt.md` (core change)
Replaced the flat 100-question interview with a four-layer process:
| Layer | Author | Purpose |
| 1. Prose Mechanics Analysis | Watts | Close-read source material to document observable writing patterns (sentence architecture, punctuation-as-cognition, vocabulary fingerprint, register map, evidence deployment, argument anatomy). Non-optional gate. |
| 2. Resequenced Interview | Asimov | Pedagogically reordered: reasoning mechanics first, then worked examples with novel scenarios, then beliefs derived from demonstrated reasoning, distinctiveness stress-tests, voice/temperament, boundaries, and a mandatory self-correction pass. |
| 3. Accountability Tags | Brin | Every answer tagged GROUNDED/INFERRED/SPECULATIVE with citation requirements. Adversarial spot-checks (feed wrong positions). Wager-testable predictions. Prompt versioning. |
| 4. Scenario Play | Tchaikovsky | 10 improvisation rounds with outside-comfort-zone scenarios, rotating challengers, and cross-persona reading with ground-truth calibration. |
Other changes
- README.md — : Updated prompt description
- _template.md — : Updated voice profile companion file comment to reference v2.0
Key improvements over v1
- Source hierarchy — : Prioritized ranking (academic papers > blogs/essays > CC fiction > interviews > reviews > social media > panels > user-owned fiction)
- Minimum corpus requirements — : 10K words nonfiction (3+ distinct works) + 10K fiction, or 15K words nonfiction (5+ works) if fiction unavailable
- Grounding ratios — : Target ≥50% GROUNDED, ≤15% SPECULATIVE, with INFERRED requiring 2+ independent corroborating patterns
- Output template — : New sections for prose mechanics, worked examples, distinctiveness analysis, scenario play results, and grounding ratio reporting
- Acceptance criterion — : Blind-test distinguishability across all panel members
Backward compatibility
The output file naming (`<name>.voice-profile.md`) and general structure are compatible with existing consumers (roundtable.prompt.md, fulltext-analysis.prompt.md). The new output is a superset of the old format.
[2026-04-13] PR #120 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-13] PR #117 — feat: voice profiles for Asimov, Brin, and Tchaikovsky personas
Summary
Adds deep voice profiles for the three remaining SF author-analyst personas, completing the persona voice profile system.
New Files
- `.github/prompts/personas/isaac-asimov.voice-profile.md` (320 lines)
- `.github/prompts/personas/david-brin.voice-profile.md` (254 lines)
- `.github/prompts/personas/adrian-tchaikovsky.voice-profile.md` (257 lines)
- `.github/prompts/voice-profile-fleet.prompt.md` (fleet orchestration prompt)
Voice Profile Details
Each profile captures *how* the author thinks, argues, and reasons (complementing the base persona `.md` files that define *what* they analyze):
| Persona | Reasoning Mode | Default Question | Humor Style |
| **Asimov** | Historical chain | "What came before?" | Self-deprecating, quiet |
| **Brin** | Steelman-dismantle | "Who watches the watchers?" | Sardonic, aggressive |
| **Tchaikovsky** | Alternative construction | "What body plan?" | Self-deprecating, constant |
| **Watts** (existing) | Evidence accumulation | "What's the fitness cost?" | Sardonic, load-bearing |
Source Materials
All profiles grounded in close reading of:
- Nonfiction essays and interview transcripts
- Blog posts and social media (author-posted content)
- Thematic analysis of fiction (user-owned copies)
Quality Assurance
- Roundtable review conducted with all four personas in structured mode
- Cross-profile distinctiveness check passes: each persona has genuinely different reasoning patterns
- Review identified expansion opportunities as follow-up work
Roundtable Review
Full review at `scratchpad/roundtable-review-round1.md` (not committed; scratchpad is gitignored). Verdict: all three profiles are solid foundations with distinct voices. Priority follow-up: expand question count to match Watts benchmark depth.
[2026-04-13] PR #118 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-13] PR #116 — feat: add roundtable discussion to What-If page
Summary
Add an interactive LLM-powered multi-persona roundtable discussion to the What-If query page.
What changed
- Unified What-If interface — : single query box with side-by-side "Find Matching Ideas" and "Start Roundtable" buttons
- Persona selector — : checkboxes for all 4 SF author-analyst personas (Watts, Asimov, Brin, Tchaikovsky), with collapsible options panel
- Freeform mode — : discuss any what-if question with selected personas via the Copilot API
- Book club mode — : paste a full-text URL for section-by-section reading with first-time reactions
- Progress bar — : animated fill bar with phase labels during LLM generation
- Book-club-style rendering — : results displayed with collapsible sections, persona contributions, idea tracker, and synthesis (same format as story book club views)
New files
- `tools/site_server.py`: Combined static-file + API server (`ThreadingHTTPServer`). Serves the site on port 8080 and exposes `POST /api/roundtable` via SSE streaming. Uses `GITHUB_TOKEN` for the Copilot API by default.
Modified files
- `tools/generate_site.py`: Extended `generate_whatif_page()` with roundtable UI, CSS, and client-side JS
- `.devcontainer/start-site.sh`: Uses `site_server.py` instead of `python3 -m http.server`
Technical details
- No new pip dependencies — (stdlib only: `http.server`, `urllib.request`, `threading`, `html.parser`)
- Token resolution — : `LLM_API_KEY` > `GITHUB_TOKEN` (auto-available in Codespaces)
- SSE streaming — : Uses `fetch()` with `response.body.getReader()` for progress updates
- Input validation — : 400 for bad inputs, 503 when no API key configured
- HTML-to-text — : strips tags from URL-sourced content for book club mode
Testing
- Site generation succeeds with all 2563 files
- Server starts and serves static files on port 8080
- API returns proper error codes (400, 503) for invalid/unconfigured requests
- Smoke-tested roundtable generation against Copilot API
[2026-04-12] PR #115 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-12] PR #114 — fix: require Copilot re-review of head commit before merge
Problem
PR #113 was auto-approved and merged to `main` without Copilot re-reviewing the fix commit (`1f64a83`). The orchestrator pushed a fix via copilot-cli, the review thread was resolved, and then on the next run the timeline REST API had not yet propagated the `committed` event. This caused `fixes_pushed_since_review` to be false, so the orchestrator reached `READY_TO_MERGE` and merged without a re-review.
Root cause
`latest_sync_time()` relies on timeline `committed` events from the REST API, which can lag behind `synchronize` webhook events. When the event arrives but the timeline hasn't caught up, the orchestrator sees no evidence of new commits and decides the PR is merge-ready.
Fix
1. **`fetch_pr_snapshot`**: The existing GraphQL query now also fetches `commits(last:1) { nodes { commit { committedDate oid } } }`, providing the head commit date from a reliable source independent of timeline propagation.
2. **`summarize_snapshot`**:
- `fixes_pushed_since_review` now uses the head commit date as the primary signal (falls back to timeline events when unavailable)
- New `review_covers_head` boolean: true only when the latest Copilot review was submitted after the head commit
- `READY_TO_MERGE` now requires `review_covers_head` to be true
- When `review_covers_head` is false and no unresolved threads remain, the state becomes `FIXES_PUSHED` with `request_rereview`
Testing
- Syntax validated: `python3 -c "import ast; ast.parse(...)"`
- Catalog validation passes: 2556/2556 valid
[2026-04-12] PR #113 — feat: declare Archive.org API secrets in devcontainer
Adds `ARCHIVE_ORG_S3_ACCESS_KEY` and `ARCHIVE_ORG_S3_SECRET_KEY` declarations to `.devcontainer/devcontainer.json` so new Codespaces prompt for these credentials.
These are used by tools that interact with the Archive.org API:
- `tools/fetch_galaxy_magazine.py` (Galaxy Magazine text fetching)
- `ia` CLI for metadata queries and downloads
- Future persona voice material gathering workflows
The actual secret values are stored as Codespace secrets and repository secrets (Actions). This change only adds the `secrets` block that triggers the Codespace prompt.
**No functional code changes.** Configuration only.
[2026-04-11] PR #110 — Roundtable book club discussion of Blindsight by Peter Watts
Full section-by-section book club analysis of *Blindsight* (2006) with all four personas (Watts, Asimov, Brin, Tchaikovsky), producing 3 new idea extractions and structured discussion output.
**Note:** This session lacked file creation/editing tools (bash, edit, create were unavailable). All catalog JSON is specified in the PR conversation and needs to be written to disk by a follow-up session with appropriate tooling.
Discussion output
- 7 sections — covering the novel's full arc, 28 persona contributions, running idea tracker, whole-work synthesis
- Structured formats: tension maps, analogical bridges, generative seeds, debate continuation
New ideas extracted
| Slug | Core concept |
| `language-as-cognitive-attack` | Unsolicited communication as involuntary resource consumption; reframes SETI/first-contact assumptions |
| `predator-as-optimal-commander` | Resurrected apex predator as leader via engineered dependency; maps to AI alignment control problem |
| `self-deception-as-fitness-advantage` | Consciousness confabulates agency post-hoc; distinct from overhead thesis (cost vs. accuracy) |
Files to create/update
- Create: — `catalog/ideas/language-as-cognitive-attack.json`, `predator-as-optimal-commander.json`, `self-deception-as-fitness-advantage.json`
- Update: — `catalog/stories/blindsight-watts.json` — add `book_club_discussion` field, expand `ideas` array with 3 new IDs
- All JSON is schema-aligned against `book_club_discussion` (sections/contributions/idea_tracker) and idea definitions
- Checked `language-as-virus` (Burroughs) for overlap — distinct concept, linked via `reframes` relationship
Key analytical findings
- Brin's power-asymmetry critique (consciousness thesis flows only downward in hierarchy) is the most productive unresolved tension
- Asimov's Section 2 prediction of James's multi-personality governance failure confirmed in Section 6 (Three Laws Trap)
- Tchaikovsky's colonial organism framing (siphonophores, ant supercolonies) provides richer scrambler model than the novel's own predator-prey lens
<!-- START COPILOT CODING AGENT SUFFIX -->
<!-- START COPILOT ORIGINAL PROMPT -->
<details>
<summary>Original prompt</summary>
> perform a round table book club of blindsight by Peter Watts
</details>
> **Custom agent used: roundtable**
> Multi-persona roundtable discussion with SF author-analyst personas
[2026-04-11] PR #112 — feat: add collapsed spoiler_details section to story pages
Summary
Adds an optional `spoiler_details` field to the story schema and renders it as a collapsed section on story detail pages. Also adds a spoiler warning to Book Club Discussion sections.
Changes
- catalog/schema.json — : Added optional `spoiler_details` field (type `["string", "null"]`) to the story definition
- catalog/stories/children-of-time-tchaikovsky.json — : Test case — revised synopsis to be spoiler-free, added `spoiler_details` with full plot resolution
- tools/generate_site.py — : Renders `spoiler_details` as a collapsed `<details>` block after the synopsis (only when populated); added spoiler warning to Book Club Discussion description
How it works
- Synopsis — remains fully visible on the page
- Spoiler section — appears below the synopsis as a collapsed `<details>` element, only for stories that have the field populated
- Book Club Discussion — gets a styled spoiler warning before the section-by-section content
- Stories without `spoiler_details` are unaffected (no empty section rendered)
Validation
- `python3 tools/validate_catalog.py`: 2556/2556 valid, 0 invalid
- `python3 tools/generate_site.py`: clean build, 1874 story pages generated
- Verified Children of Time page renders all three elements correctly
- Verified other story pages show no spurious spoiler section
[2026-04-11] PR #107 — fix: use COPILOT_PAT with Actions scope for orchestrator re-dispatch
The re-dispatch step in `pr-orchestrate.yml` was using `COPILOT_PAT || GITHUB_TOKEN`. Since `COPILOT_PAT` exists, it takes priority, but that fine-grained PAT lacked the **Actions** scope needed for `gh workflow run`, causing every re-dispatch to fail with HTTP 403.
The fix keeps `COPILOT_PAT` for the re-dispatch step (required — `github.token` cannot trigger new workflow runs due to GitHub's recursive-trigger prevention), but adds inline documentation requiring that `COPILOT_PAT` includes the **Actions (read/write)** repository permission.
Also adds auto-approve logic: when the orchestrator reaches `READY_TO_MERGE` and Copilot's review had zero unresolved threads, it submits an approving review before attempting the squash merge, satisfying the branch-protection requirement for at least one approving review.
[2026-04-11] PR #109 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-11] PR #106 — feat: persona voice profile system
Summary
Implements **Tier 1: Taste Interview Voice Profiles** from the persona enhancement brainstorm.
What
A self-interview prompt that extracts the analytical DNA of SF author-personas: how they reason, argue, frame analysis, handle uncertainty, and construct arguments. The output is a deep voice profile stored as a companion file alongside the existing persona `.md`.
Why
The existing persona files define **what** each persona analyzes (analytical lens, per-field guidance, signature arguments). Voice profiles complement this by capturing **how** they think: reasoning mechanics, argumentative temperament, rhetorical instincts, hard nos, and red flags. This produces more distinctive, faithful persona voices in dialectic and roundtable sessions.
Changes
- New prompt — : `persona-voice-interview.prompt.md` — 100-question self-interview adapted from the taste-interview framework, reframed for analytical personas
- New file — : `peter-watts.voice-profile.md` — demo voice profile for Peter Watts (100 Q&A covering beliefs, reasoning mechanics, analytical crimes, voice, structure, hard nos, red flags + quick reference card + anti-overfitting guide)
- Updated loading — : `fulltext-analysis.prompt.md`, `roundtable.prompt.md`, `roundtable.agent.md` — all three persona-loading systems now discover and load companion voice profiles when present
- Updated discovery — : persona globs exclude `*.voice-profile.md` from being treated as standalone personas
- Updated template — : `_template.md` notes the optional companion voice profile
Backward compatible
Voice profiles are optional. Personas without voice profiles work exactly as before. No schema changes.
[2026-04-11] PR #105 — docs: auto-generate changelog
Automated changelog update from merged PR data. Created by `generate-changelog.yml`.
[2026-04-11] PR #104 — fix: use admin-merged PR for changelog generation
Summary
Fixes the changelog workflow failing due to branch protection blocking direct pushes to main.
Changes
The workflow now:
1. Creates a short-lived branch (`auto/changelog-YYYYMMDD-HHMMSS`)
2. Commits the generated changelog to that branch
3. Opens a PR targeting main
4. Immediately merges it with `--admin` flag via `COPILOT_PAT`
5. Deletes the branch
This bypasses the review requirement only for this specific automated changelog PR, keeping branch protection fully intact for all other changes. The PR creates an auditable record of every changelog update.
[2026-04-11] PR #102 — feat: auto-generate changelog from merged PRs
Summary
Replaces the manual CHANGELOG.md update requirement with a CI workflow that auto-generates it from merged PR titles and bodies.
Changes
- `.github/workflows/generate-changelog.yml` — New workflow triggered on push to `main` (with `paths-ignore: [CHANGELOG.md]` to prevent self-triggering). Uses a concurrency group to serialize runs. Lists the 200 most recent merged PRs targeting `main` via `gh pr list --base main`, strips Copilot review boilerplate using a stateful jq reduce, and demotes PR body `##` headings to `###` to avoid conflicts with release headings. Writes to a temp file first for failure safety. Checks out with `COPILOT_PAT` (falling back to `github.token`) so the push triggers downstream workflows.
- `CHANGELOG.md` — Replaced with a minimal stub that the workflow overwrites on first run.
- `.github/copilot-instructions.md` — Removed the "Always update CHANGELOG.md" Git Workflow rule, removed step 4 from the pre-commit checklist, removed "preparing changelog text" from idle-time guidance. Deploy trigger path list still includes `CHANGELOG.md`.
- `.github/workflows/deploy-site.yml` — `CHANGELOG.md` remains in deploy trigger paths so auto-generated changelog updates trigger site redeploys.
- `CONTRIBUTING.md` — Removed manual CHANGELOG update step from the contribution checklist.
- `README.md` — Added `generate-changelog.yml` to the CI workflows table.
[2026-04-11] PR #103 — feat: Children of Time book club discussion and site support
Summary
First book-club-mode roundtable discussion, plus schema and site support for displaying them.
Changes
- `catalog/schema.json` — New optional `book_club_discussion` field on story entries. Stores section-by-section persona contributions, idea trackers, and whole-work synthesis.
- `catalog/stories/children-of-time-tchaikovsky.json` — Full book club discussion with Peter Watts, Isaac Asimov, David Brin, and Adrian Tchaikovsky reading *Children of Time* section by section. Includes per-section contributions and progressive idea tracking.
- `tools/generate_site.py` — "Has Book Club" filter dropdown on index page. Story detail pages render book club discussions with collapsible per-section views showing persona contributions and idea tracking.
[2026-04-11] PR #101 — feat: add book-club fulltext analysis mode
Summary
Add a new **book-club** fulltext analysis mode where personas read and discuss a story **section by section**, as if encountering it for the first time.
What's new
- `.github/prompts/fulltext-analysis-bookclub.prompt.md` — New prompt with section-by-section workflow:
- First-time reading rule — : personas have no foreknowledge of later sections
- Per-section cycle: read-aloud → first impressions → discussion → takeaways
- Running idea tracker with progressive discovery (`[+]` new, `[~]` revised, `[-]` dropped, `[?]` tentative, `[!]` confirmed)
- Whole-work synthesis after all sections, then 9 LLM-optimized output formats
- `catalog/schema.json` — Optional `section` (string) and `section_index` (integer) fields on dialectic round objects
- Roundtable agent + prompt — Book Club Mode added as third discussion mode (alongside Freeform and Structured)
Validation
- Schema is valid JSON
- All 2556 existing catalog entries pass validation (0 invalid)
- No changes to existing persona files; the book-club format reuses the same persona system
[2026-04-11] PR #100 — fix: eliminate settle sleep to save Actions minutes
Summary
Replaces the settle-loop sleep in `pr-orchestrate.yml` with a single-shot orchestrator run. This saves 5-10 Actions minutes per review cycle.
Before
The workflow ran the orchestrator in a loop with up to 3 retries, sleeping 5-10 minutes per retry when the state was `COPILOT_WORK_SETTLING`. A typical review-triggered run consumed 6-12 minutes of runner time.
After
The workflow runs the orchestrator once and exits. When the orchestrator detects a settle state, the workflow exits immediately (< 30s). The events that end the settle period (fix pushes trigger `pull_request:synchronize`, reviews trigger `pull_request_review:submitted`) naturally start new workflow runs.
Why this works
With `COPILOT_PAT` checkout, copilot-cli fix pushes trigger `synchronize` events. Copilot code review submissions trigger `pull_request_review:submitted` events. Both are already in the workflow's `on:` triggers. The settle sleep was a workaround for when these events didn't fire reliably; with the PAT fix, they do.
[2026-04-11] PR #97 — feat: LLM-optimized analysis formats and roundtable prompt
Summary
Adds nine new optional fields to the analysis schema designed for downstream LLM consumers, plus a roundtable discussion prompt for interactive multi-persona analysis.
Schema additions (`catalog/schema.json`)
- `analysis.tension_map` — Unresolved creative tensions between persona analyses encoded as opposing poles with generative prompts. Designed to provoke novel synthesis rather than passive retrieval.
- `analysis.analogical_bridges` — Structured source-target domain mappings with transfer validity scores (0-1) and limitations. Each mapping dimension value is exactly a `[source_element, target_element`] pair. Enables cross-domain reasoning with explicit analogy grounding.
- `analysis.generative_seeds` — Compact symbolic encodings of analysis reasoning skeletons designed for LLM decompression. Not intended for human reading.
- `analysis.atomic_propositions` — Self-contained factoids with retrieval tags, optimized for RAG. Each proposition can be independently embedded and retrieved.
- `analysis.semantic_triples` — Subject-predicate-object triples forming a knowledge graph. Merge across ideas for cross-catalog queries.
- `analysis.counterfactual_tree` — Branching what-if tree with cascading effects and real-world parallels. Expands choice_points into systematic scenario exploration.
- `analysis.decision_context_tags` — Pre-computed relevance annotations for specific decision-making contexts (e.g., gene-drive-policy, SETI-search-strategy).
- `analysis.scenario_template` — Parameterized scenario with substitutable variables and an invariant structural insight that holds regardless of substitution.
- `analysis.debate_continuation` — Structured handoff with open threads, unanswered critiques, and a continuation prompt for resuming the dialectic in future sessions.
Catalog data
Populated all nine fields on `non-human-uplift-civilization` as the reference implementation:
- 3 tension maps (parasite-vs-symbiont, universal-vs-contingent civilization, metabolic ceiling)
- 3 analogical bridges (CRISPR gene drives, biological computing, epigenetic inheritance)
- 1 generative seed
- 10 atomic propositions, 12 semantic triples, 5 counterfactual tree nodes, 4 decision context tags, 1 scenario template, 1 debate continuation
Prompts
- `roundtable.prompt.md` — New prompt for multi-persona roundtable discussions. Supports freeform and structured modes. Generates all 9 LLM-optimized formats as output.
- `fulltext-analysis.prompt.md` — Added step 6b with instructions for generating all 9 LLM-optimized formats after the dialectic.
Validation
- `validate_catalog.py`: 2556 valid, 0 invalid
- `generate_site.py`: generates without errors
[2026-04-11] PR #96 — fix: orchestrator re-review request and auto thread resolution
Summary
Fixes bugs in the PR review orchestration pipeline found during E2E testing (PR #94). Supersedes PR #95 (closed, all Copilot feedback incorporated).
Changes
**`request_copilot_review()`**
- Uses REST API with `--input -` and explicit JSON body `{"reviewers":["Copilot"]}` (case-sensitive Bot user) instead of `gh pr edit --add-reviewer copilot-pull-request-reviewer` which fails (not a collaborator)
- After the API call, verifies the request landed by checking pending reviewers. The API returns 201 even when requests are silently dropped (e.g. while Copilot is already reviewing)
- When verification fails: records as `request_review_unconfirmed` (no cooldown started), allowing prompt retries
- Failure path always returns a non-empty reason with structured `rest_error` field
- Dry-run records `method=rest-api-copilot` (not stale comment body)
**`resolve_review_threads()`** (new)
- Resolves only `isOutdated` threads via GraphQL `resolveReviewThread` after copilot-cli pushes fixes
- Re-fetches each thread's state before resolution; skips still-current threads for re-review to validate
- Handles API failures and already-resolved threads gracefully
**`address_feedback()`**
- Uses shared `request_copilot_review()` for re-review (with verification) instead of inline `gh pr edit` call
- Only resolves suggestion threads (not prose-only which may not have been acted on)
- Reports `copilot_pending` status in result for observability
Testing
- `py_compile` passes
- `validate_catalog.py`: 2556 valid, 0 invalid
- `generate_site.py`: generates without errors
[2026-04-11] PR #99 — feat: auto-merge pipeline with pre-merge sanity check
Summary
Enables the full auto-merge pipeline so PRs that pass Copilot review with zero comments are automatically merged after a final sanity check.
Changes
- `AUTO_MERGE_TARGET_BRANCHES` — : Added `main` (was `staging` only)
- `EFA_AUTO_MERGE_ENABLED=1` — : Set in `pr-orchestrate.yml` env
- `pre_merge_sanity_check()` — : New function that runs before auto-merge:
1. `python3 tools/validate_catalog.py` — schema + bidirectional link validation
2. `python3 tools/generate_site.py` — site generation dry run
3. Live GraphQL check that all review threads are resolved
- If any sanity check fails, falls back to posting a ready-to-merge comment instead of merging
How it works
When Copilot reviews a PR and generates **0 new comments** (or all threads are resolved), the orchestrator reaches `READY_TO_MERGE` state. With auto-merge enabled, it:
1. Runs the pre-merge sanity check
2. If passed, executes `gh pr merge --squash --delete-branch`
3. If failed, posts a comment explaining what failed
Safety
- CI must pass (`checks_ok`) before `READY_TO_MERGE` is reached
- Branch must be merge-clean (`mergeStateStatus == CLEAN`)
- No unresolved threads
- Pre-merge sanity check is an independent final gate
- 30-minute cooldown on merge retries after failures
- Ruleset requires `required_review_thread_resolution: true`
[2026-04-11] PR #98 — chore: retire webhook infrastructure
Summary
Remove the Codespace-based webhook receiver, bridge extension, and orchestration watchdog. The GitHub Actions workflow (`pr-orchestrate.yml`) now handles the full PR review loop; no codespace dependency remains for the automation pipeline.
What changed
**Deleted files:**
- `tools/webhook_receiver.py` — HTTP server for receiving GitHub webhooks
- `tools/webhook_manage.sh` — lifecycle management for webhook + receiver
- `tools/orchestration_watchdog.py` — health checks for local processes
- `.vscode-extensions/efa-orchestrator-bridge/` — VS Code extension for feedback file handoff
**Cleaned up:**
- `tools/pr_review_orchestrator.py` — removed `WEBHOOK_EVENTS_FILE`, `read_recent_webhook_events()`, all `webhook_events` reads from snapshot consumers (`detect_copilot_work_session`, `latest_sync_time`, `recent_copilot_activity`), and the legacy feedback-file bridge path from `address_feedback()`
- `.devcontainer/devcontainer.json` — removed port 8585
- `.devcontainer/start-site.sh` — removed webhook receiver + watchdog section
- `.devcontainer/setup.sh` — removed bridge extension symlink
- `.devcontainer/welcome-banner.sh` — removed automation status line (depended on webhook flags)
- `.env.example` — removed `EFA_PR_ORCHESTRATOR_ENABLED`, `WEBHOOK_PORT`, `WEBHOOK_BIND_ALL`, and watchdog settings
- `.github/workflows/ci.yml` — removed `py_compile` for `webhook_receiver.py`
- `.gitignore` — removed `.webhook-secret` and `.webhook-hook-id`
- `README.md` + `tools/README.md` — removed webhook/bridge documentation
Validation
- `python3 -m py_compile tools/pr_review_orchestrator.py`: OK
- `python3 tools/validate_catalog.py`: 2556 valid, 0 invalid
- `python3 tools/generate_site.py`: generates successfully
- No remaining references to deleted files in tracked code (excluding CHANGELOG history)
Net impact
**39 insertions, 2028 deletions** across 16 files.
[2026-04-10] PR #93 — feat: orchestration pipeline improvements
Consolidates all orchestration improvements tested on PR #92.
Changes
- copilot-cli on Actions — : Install via npm, authenticate with `COPILOT_PAT` secret
- Explicit permissions — : Replace `--yolo` with `--allow-tool` flags (git, gh, python3, edit, write, view, glob, grep)
- Settle retry loop — : Actions workflow sleeps through settle periods and retries (max 3)
- Guardrails — : Only OWNER/MEMBER PRs get auto-fixes; external contributors need `automation-fixes-approved` label
- Blocked paths — : Post-fix safety reverts changes to `.github/workflows/`, `.github/actions/`, `.devcontainer/`
- copilot-cli detection — : Runs `--version` to verify CLI is real, not a VS Code shim
- Outdated threads — : Include in feedback extraction (were being skipped)
- Full PR context — : Review diff + commit history before applying any fix
- Auto re-review — : Request Copilot re-review after copilot-cli pushes
- authorAssociation — : Fetched via REST API (not in `gh --json`)
[2026-04-10] PR #90 — feat: GitHub Actions event source + copilot-cli fix application for PR orchestration
The PR review orchestration pipeline currently requires a running Codespace (webhook receiver → orchestrator → bridge extension → Chat panel). This moves event handling to GitHub Actions and adds copilot-cli as the fix application layer.
New: Actions workflow (`.github/workflows/pr-orchestrate.yml`)
- Triggers on `pull_request_review` (submitted) and `pull_request` (opened/synchronize/ready_for_review)
- Filters to Copilot reviewer only — no broad bot matching
- Per-PR concurrency group with `cancel-in-progress: false` (queues, never drops)
- State persistence via `actions/cache` keyed by PR number + run ID
New: Custom agent profile (`.github/agents/pr-fix-agent.agent.md`)
- Defines workflow, constraints, and tool access for copilot-cli fix application
Modified: `tools/pr_review_orchestrator.py`
- `REPO_ROOT` / `STATE_DIR` configurable via `EFA_REPO_ROOT` / `EFA_STATE_DIR` env vars for runner environments
- `address_feedback()` now tries copilot-cli first when `EFA_PR_AUTOMATION_FIXES=1` and `copilot` binary is on PATH, falls back to legacy feedback file path
```python
New dual-path in address_feedback():
if not dry_run and _fixes_enabled() and _copilot_cli_available():
prompt = build_copilot_cli_prompt(repo, pr_number, suggestions_logged, prose_pending)
cli_result = invoke_copilot_cli(repo, pr_number, prompt, state)
...
return result
Otherwise: legacy feedback file for bridge extension
```
Docs
- `README.md`: updated components table, automation flags, workflow listing
- `tools/README.md`: new "GitHub Actions orchestration" section
- `CHANGELOG.md`: `[2026-04-10i]` entry
Not yet implemented (future phases per issue)
- Phase 3: Fleet mode for multi-comment reviews
- Phase 4: Bridge extension deprecation
- Phase 5: VS Code Agents app integration
[2026-04-10] PR #91 — feat: full-text analysis of Children of Time (Tchaikovsky)
Full-Text Analysis: Children of Time
Full-text analysis of Adrian Tchaikovsky's *Children of Time* based on the extracted text in `scratchpad/epub-text/`.
Changes
**New idea:**
- `engineered-empathy-conflict-resolution` — The nanovirus empathy weapon as first-contact resolution. Full analysis with 4-persona dialectic (Tchaikovsky, Watts, Asimov, Brin) examining consent ethics, parasitism vs. mutualism, and the Zeroth Law Escalation parallel.
**Updated ideas (added full analysis blocks with dialectic):**
- `non-human-uplift-civilization` — Nanovirus mechanism chain, ant-colony computing, spider civilization as substrate-dependent development
- `cooperation-across-cognitive-gulfs` — Prisoner's dilemma critique, Kern miscommunication, alternative cognitive frameworks
- `inherited-technology-legacy` — Legacy systems parallels, Old Empire upload misuse, institutional memory decay
**Updated story:**
- `children-of-time-tchaikovsky` — Enhanced synopsis from full text, added new idea to ideas array
Validation
- `validate_catalog.py`: 2556 valid, 0 invalid
- Bidirectional links verified
- `generate_site.py`: site generates cleanly
- CHANGELOG updated
[2026-04-10] PR #84 — feat: decision-support schema fields for LLM consumers
Summary
Adds four new optional schema fields that optimize the catalog for downstream AI consumption and decision-maker routing.
Schema Changes (`catalog/schema.json`)
**`decision_contexts`** (on ideas) — Controlled enum of decision domains:
`policy-formation`, `risk-assessment`, `technology-strategy`, `resource-allocation`, `regulation`, `infrastructure-planning`, `security-defense`, `public-health`, `economic-planning`, `environmental-management`, `ethics-governance`, `innovation-rd`
Enables LLM consumers to route ideas to relevant decision-maker contexts without expensive semantic search over `real_world_relevance` prose.
**`temporal_context`** (on ideas) — Object with:
- `real_world_emergence`: approximate year when the concern appeared in real-world contexts
- `urgency_trajectory`: enum (`emerging`, `growing`, `peak`, `declining`, `dormant`, `recurring`)
Enables temporal reasoning about idea relevance distinct from story publication date.
**`abstraction_level`** (on ideas) — Enum: `civilization`, `sector`, `organization`, `technology`, `individual`
Enables hierarchical matching: a query about 'agricultural disruption' (sector) surfaces both specific technology-level ideas and broader civilization-level patterns.
**`choice_points`** (on analysis) — Array of decision points within the `mechanism_chain`:
- `mechanism_step`: 1-indexed position in the chain
- `decision`: what intervention was possible
- `actual_path`: what happened in the story
- `alternative_outcomes`: plausible alternatives
Enables counterfactual 'what if we intervened at step X?' reasoning for policy analysis.
Prompt & Skill Updates
- fulltext-analysis.prompt.md — : Added new fields to idea identification step and analysis documentation
- story-idea-extraction SKILL.md — : Added to identification and analysis block
- copilot-instructions.md — : Updated schema field reference
Validation
- All 2555 existing entries pass validation (all new fields are optional)
- Site generation succeeds
Relation to PR #83
This PR is independent of #83 (cross-story context schema). Both add optional fields to different parts of the schema. No merge conflicts expected.
[2026-04-10] PR #83 — feat: cross-story context, authorial evolution, and idea connections schema
Summary
Adds three new optional schema fields and updates prompts/skills for connected knowledge graph analysis across the catalog.
Schema Changes (`catalog/schema.json`)
**`idea_connections`** (on ideas) — Rich cross-references with:
- `relationship_type` enum: `evolves-from`, `evolves-into`, `contradicts`, `specializes`, `generalizes`, `complements`, `precursor-to`, `complicates`, `reframes`
- `context` string: LLM-friendly explanation of the relationship
**`cross_story_context`** (on analysis) — How an idea's analysis relates to, builds on, or contrasts with analyses from other stories.
**`authorial_context`** (on analysis) — Author-centric longitudinal tracking:
- `stance`: controlled enum (`advocates`, `warns`, `interrogates`, `satirizes`, `celebrates`, `mourns`, `ambivalent`, `observes`)
- `stance_evolution`: within-work arc tracking
- `prior_works`: earlier works context
- `subsequent_influence`: retrospectively populated
Prompt & Skill Updates
- fulltext-analysis.prompt.md — : Added step 3 (cross-story context gathering), updated deduplication with `idea_connections` guidance, documented new fields, renumbered steps 3-10
- story-idea-extraction SKILL.md — : Added step 4b (cross-story context for full-text), updated step 4 with `idea_connections`, documented new fields in step 5b
Validation
- All 2555 existing entries pass validation (all new fields are optional)
- Site generation succeeds
[2026-04-10] PR #82 — feat: enrich Isaac Asimov & David Brin personas
Summary
Enriches the two remaining placeholder personas (Isaac Asimov and David Brin) to production quality, matching the depth and structure of the Peter Watts and Adrian Tchaikovsky personas.
Isaac Asimov
Sourced from the Foundation series, Robot series, "The Relativity of Wrong" essay, F&SF science columns, and freely available interviews/lectures.
- 7-item Analytical Lens — : institutional dynamics, statistical society, rule-based systems and edge cases, historical rhyme, science as self-correcting process, scale transitions, the popularizer's obligation
- 10-field Per-Field Guidance — with detailed emphases
- 6-item Critique Style —
- 7 Signature Arguments — : Psychohistory Premise, Three Laws Trap, Seldon Crisis, Zeroth Law Escalation, Encyclopedia Gambit, Relativity of Wrong, Collective Solution
David Brin
Sourced from *The Transparent Society*, Uplift series, *The Postman*, *Earth*, *Kiln People*, *Existence*, *Polemical Judo*, Contrary Brin blog (davidbrin.blogspot.com), and author site (davidbrin.com).
- 7-item Analytical Lens — : reciprocal accountability/sousveillance, civilization as fragile achievement, contrarian challenge, uplift obligation, creative independence, citizens as agents, ecological stewardship
- 10-field Per-Field Guidance — with detailed emphases
- 6-item Critique Style —
- 7 Signature Arguments — : Sousveillance Principle, Postman's Wager, Uplift Obligation, Library Trap, Contrarian's Duty, Citizen Sensor Network, Feudalism Detector
Validation
- `validate_catalog.py`: 2555 valid, 0 invalid
- `generate_site.py`: generates without errors
- No catalog data changes; personas only + CHANGELOG
[2026-04-10] PR #81 — feat: Adrian Tchaikovsky persona, Gaslight War story, 5 ideas
Summary
Adds an Adrian Tchaikovsky analysis persona and catalogs ideas from *The Gaslight War* (2025) and *Children of Time* (2015).
Changes
**Persona**
- `adrian-tchaikovsky.md`: Uplift/emergent civilization, empathy across cognitive gulfs, biological diversity, legacy technology, AI personhood, information warfare. 5 Signature Arguments.
**Catalog — Stories**
- `the-gaslight-war-tchaikovsky`: New short story entry (2025)
- `children-of-time-tchaikovsky`: Added 3 new idea links
**Catalog — Ideas (5 new)**
- `autonomous-weapons-personhood`
- `cooperation-across-cognitive-gulfs`
- `deepfake-warfare-trust-collapse`
- `inherited-technology-legacy`
- `non-human-uplift-civilization`
Fixes applied
- Taxonomy: replaced invalid terms (`military-defense`→`warfare`, `governance-policy`→`governance`, `information-technology`→`comms-info`, `what-if`→valid types, `hopeful`→`aspirational`)
- Removed misplaced `archive_org_url` (capx.co, not archive.org)
- Added bidirectional links: children-of-time ↔ 3 new ideas
- Fixed `webhook_manage.sh` `_env_set` NUL check bug (bash 5.2+)
Validation
- `validate_catalog.py`: 2555/2555 valid
- `generate_site.py`: generates without errors
- Bidirectional link consistency verified
- No em-dashes in catalog prose
[2026-04-10] PR #79 — feat: Enrich Peter Watts persona & add Rifters catalog entries
Replaces #77 with a clean single-commit branch rebased on main.
Changes
- Rewrite `peter-watts.md` persona with 7 analytical lens bullets, 10 per-field guidance entries, 6 critique style bullets, and 7 signature arguments sourced from Blindsight, Starfish, Maelstrom, Behemoth (all CC BY-NC-SA 2.5)
- Fix Blindsight and Echopraxia synopses (remove verbatim quote / garbage text)
- Add 4 new story entries: Starfish, Maelstrom, Behemoth, The Things
- Add 8 new idea entries with valid taxonomy IDs and bidirectional story links
- Update CHANGELOG
Closes #77 issues. All taxonomy IDs validated, bidirectional links synced.
[2026-04-10] PR #80 — feat: Persistent automation config (.env) & orchestration watchdog
Changes
Persistent `.env` configuration
- Add `.env.example` (committed) as template for all automation flags
- `.env` is auto-created from template during `setup.sh` if missing (gitignored)
- `.env` is sourced automatically in: every new terminal (`.bashrc`), `start-site.sh`, and `webhook_manage.sh`
- Enable `python.terminal.useEnvFile` in `devcontainer.json` for VS Code Python terminal injection
CLI toggle commands
- `tools/webhook_manage.sh enable-automation` — turn on orchestrator + automation
- `tools/webhook_manage.sh disable-automation` — turn both off
- `tools/webhook_manage.sh config` — show all flag values and sources
- `tools/webhook_manage.sh config KEY VALUE` — set a flag in `.env`
Orchestration watchdog
- New `tools/orchestration_watchdog.py`: comprehensive health check of the full pipeline
- Receiver process and `/health` endpoint
- Bash watchdog loop status
- Automation flag values and misconfiguration warnings
- GitHub webhook hook status
- Webhook delivery freshness (staleness threshold)
- PR orchestrator states (stale PR detection)
- Available via `tools/webhook_manage.sh watchdog` or directly
- Supports `--json` for machine-readable output
Documentation
- Updated `tools/README.md` with Automation Flags and Orchestration Watchdog sections
- Welcome banner now shows automation status (ON/observe-only/OFF)
- Updated CHANGELOG
[2026-04-10] PR #76 — staging → main: multi-persona adversarial analysis + fulltext prompt + tooling updates
Summary
Promotes staging changes to main. Key additions since last merge:
Multi-Persona Adversarial Analysis (new)
- Modular persona system in `.github/prompts/personas/` with template and three author personas (Asimov, Watts, Brin) as placeholders
- `analysis.dialectic` schema extension for storing observation/critique/synthesis rounds
- Updated fulltext-analysis prompt with 3-pass dialectic workflow
Full-Text Analysis Prompt (new)
- `.github/prompts/fulltext-analysis.prompt.md` for deep SF full-text extraction with all 10 analysis fields
Tooling improvements
- Auto-merge flag for staging PRs (`EFA_AUTO_MERGE_ENABLED`)
- Stall checker tuning (10-min threshold, 2-min interval)
- Webhook receiver watchdog with auto-restart
- Bridge extension auto-submit
- Fix application and merge readiness improvements
- Work session detection and settle delays
- Gilfoyle review and Socratic mentoring prompt modes
Validation
- `python3 tools/validate_catalog.py`: 2537/2537 valid, 0 invalid
- `python3 tools/generate_site.py`: generates without error
- All CI checks pass on staging
[2026-04-09] PR #74 — docs: full-text analysis prompt (staging → main)
Promotes the fulltext-analysis prompt and prompts README updates to main.
[2026-04-09] PR #72 — feat: automation hardening, prompt modes, and tuning (staging → main)
Promotes staging changes to main:
- Bridge auto-submit — `isPartialQuery: false` closes the automation loop
- Receiver watchdog — auto-start with health checks in `start-site.sh`
- Stall checker tuning — 10-min threshold, 2-min check interval, active-state gating
- Auto-merge flag — `EFA_AUTO_MERGE_ENABLED=1` for staging PRs
- Prompt modes — Gilfoyle code review and Socratic mentoring prompts
[2026-04-09] PR #67 — docs: add PR review automation details to README
Promotes staging README changes to main. Documents the webhook receiver, PR review orchestrator, VS Code bridge extension, CI workflows, environment variables, and branch protection.
[2026-04-09] PR #65 — fix: address Copilot review feedback on PR #65
Addresses four review comments from Copilot on the staging→main promotion PR for the webhook-driven PR review orchestrator.
- Redundant orchestrator spawns — `_maybe_spawn_orchestrator()` now probes the per-PR lock before spawning, preventing burst webhook events from creating many processes that immediately fail on lock acquisition
- Stall checker race window — Lock file handle kept open through the spawn decision instead of closing before `subprocess.Popen()`, eliminating the window where another checker could spawn a redundant run
- Overly broad `@copilot` matching — `detect_request_sources()` now matches the specific `REVIEW_REQUEST_COMMENT` phrase rather than any `@copilot` mention, preventing misclassification of general Copilot chat as re-review requests
- Stale CI comment — Updated `ci.yml` comment from "added in later PRs" to reflect that the orchestrator exists in this branch
[2026-04-08] PR #61 — feat: track issue_comment events in webhook receiver
Summary
Adds `issue_comment` event tracking to the webhook receiver so PR comments (including `@copilot` review requests) are captured and logged.
Changes
- Added `issue_comment` to `TRACKED_EVENTS` in `tools/webhook_receiver.py`
- Added `extract_summary()` handler: captures `pr_number`, `pr_state`, `commenter`, `has_copilot_mention`
- No raw comment content is persisted; only a boolean indicating whether `@copilot` was mentioned
- Only tracks comments on open PRs; closed/merged PR comments and non-PR issue comments are skipped before logging
- Updated `tools/webhook_manage.sh` to include `issue_comment` in the webhook subscription
- Updated module docstring Events captured list
- Changelog entry added
Files changed
- `tools/webhook_receiver.py`
- `tools/webhook_manage.sh`
- `CHANGELOG.md`
This is part 1 of 5 in the PR review orchestrator series.
[2026-04-08] PR #56 — docs: add Copilot code review instructions for LLM-generated fixes
Summary
Adds a Code Review Instructions section to `.github/copilot-instructions.md` so Copilot code review applies stricter, adversarial scrutiny to commits produced by the local LLM fix agent in the PR review orchestrator.
What Copilot will check on LLM-generated commits
- Fix actually resolves the original review comment (not just something plausible)
- No hallucinated APIs or functions
- No regressions (broken callers, removed error handling, changed signatures)
- No cosmetic-only changes that dodge the real issue
Other review checks added
- Schema and taxonomy compliance for catalog data
- Bidirectional link consistency
- No em-dashes in prose
- No credential exposure
This must merge to `main` before the instructions take effect for Copilot code review on subsequent PRs.
[2026-04-07] PR #53 — docs: clarify Codespaces webhook setup for PR monitoring
Summary\n- clarify the Codespaces webhook setup steps in \ and \\n- document the need for the \ secret and required PAT scopes\n- add a changelog entry for the docs refresh\n\n## Verification\n- /workspaces/efa-catalog/.venv/bin/python tools/validate_catalog.py\n- /workspaces/efa-catalog/.venv/bin/python tools/generate_site.py
[2026-04-07] PR #52 — feat: add safe-mode devcontainer for terminal diagnostics
Safe-mode Codespace configuration
Minimal devcontainer to isolate the ENOPRO `run_in_terminal` bug.
What's stripped
| Default config | Safe mode |
| `github.copilot-chat` + `ms-python.python` + synced extensions (C++, Docker, XML, etc.) | `github.copilot-chat` only |
| Node 20 feature | None |
| `postAttachCommand` (site gen + HTTP server) | None |
| Port forwarding (8080, 8585) | None |
| `autoApprove` enabled (masks approval prompts) | `autoApprove: false` (default approval path) |
| Settings Sync extension control (no valid devcontainer setting exists) | Removed; disable Settings Sync manually or use a fresh profile |
`postCreateCommand`
Creates a venv only — no dependency install. This keeps container startup fast and low-noise for timing/race-condition diagnostics. Install dependencies manually when needed:
```bash
. .venv/bin/activate && pip install -r requirements.txt
```
How to test
1. Go to **Code > Codespaces > New with options**
2. Under **Dev container configuration**, select **EFA — Safe Mode (minimal extensions)**
3. Create the Codespace
4. Open Copilot Chat (panel) and ask it to run a terminal command (e.g. `pwd`)
5. If it works: the ENOPRO error was caused by extension conflicts or startup timing
6. If it fails: the bug is in VS Code's web workbench independent of extensions
[2026-04-07] PR #51 — fix: improve Codespace startup config and add terminal welcome banner
Changes
Remove deprecated `github.copilot` extension
The standalone Copilot extension is deprecated and was being redirected to `copilot-chat` at install time, causing:
- `PendingMigrationError: navigator is now a global in nodejs` log spam on every extension host start
- An unnecessary install-then-redirect cycle during Codespace creation
Only `github.copilot-chat` is needed (it now includes all Copilot functionality).
Add terminal welcome banner
New `.devcontainer/welcome-banner.sh` displays catalog stats and quick-start commands in every new interactive terminal session. Installed into `~/.bashrc` by `setup.sh` during `postCreateCommand` (idempotent).
Context: ENOPRO investigation
The `run_in_terminal` ENOPRO error (`No file system provider found for resource 'file:///workspaces/efa-catalog'`) is a VS Code web client bug where the browser-side terminal tool uses `file://` URIs in a remote context that only has `vscode-remote://` providers. This cannot be fixed from devcontainer config; removing the deprecated extension reduces log noise and extension host churn that may exacerbate the issue.
[2026-04-07] PR #50 — perf: pre-compile sfdictionary regex patterns
Problem
`generate_site.py` hangs during codespace startup (`start-site.sh`) because `find_sfdictionary_links()` recompiles a regex for every SF dictionary term for every idea — hundreds of compilations × 668 ideas.
The hang manifests as the process stuck inside Python 3.11's `re._compile` → `enum.__and__` flag check path.
Fix
Pre-compile all regex patterns once in `load_sfdictionary_index()` and return `list[tuple[re.Pattern, str, str]]` instead of `list[tuple[str, str]]`. The matching loop in `find_sfdictionary_links()` now calls `pattern.search(text)` on the pre-compiled objects.
Also moves the `len(term) <= 5` filter into the loader so short terms are excluded once at load time rather than checked on every idea.
Testing
- `python3 tools/generate_site.py` completes in ~9s instead of hanging
- `python3 tools/validate_catalog.py` — 2537 valid, 0 invalid
[2026-04-07] PR #49 — feat: webhook receiver for real-time PR monitoring
Summary
Replaces `gh api` polling with a real-time GitHub webhook receiver for Agent C's PR review monitoring.
Problem
Agent C currently polls `gh api repos/.../pulls/.../reviews` repeatedly to detect Copilot review submissions, CI changes, and PR state updates. This is slow, burns API quota, and can miss events between polls.
Solution
A zero-dependency Python webhook receiver (`http.server` stdlib) that:
- Listens on port 8585 for GitHub webhook deliveries
- Validates HMAC-SHA256 signatures (required; rejects unsigned requests)
- Extracts compact event summaries (no body content stored) from `pull_request_review`, `pull_request`, `check_suite`, and `check_run` events
- Writes events to `scratchpad/webhook-events.jsonl` (gitignored) and `scratchpad/webhook-latest.json`
- Provides `/health` and `/events` endpoints
Usage
```bash
One-time setup: create repo webhook (secret passed via stdin, not CLI args)
tools/webhook_manage.sh setup --url https://your-codespace-8585.app.github.dev
Start receiver in background
tools/webhook_manage.sh start
Check status / recent events
tools/webhook_manage.sh status
Teardown when done
tools/webhook_manage.sh teardown
```
Security
- Webhook secret is required (no insecure mode)
- Secret passed to GitHub API via stdin (not command-line args)
- HMAC-SHA256 signature verification on every delivery
- No review/comment body content persisted to disk
- Port 8585 is **private** by default in Codespaces; must be manually made public when configuring the webhook URL
- Repo is dynamically resolved (supports forks and renames)
Changes
- `tools/webhook_receiver.py` — new: the HTTP server
- `tools/webhook_manage.sh` — new: setup/start/stop/teardown management
- `.gitignore` — add `.webhook-secret`
- `.devcontainer/devcontainer.json` — forward port 8585 (private)
- `CHANGELOG.md` — document the addition
[2026-04-07] PR #48 — chore: gitignore entire scratchpad directory
Replace individual scratchpad/* ignore rules with a single `scratchpad/` entry. Untrack previously committed scratchpad files.
[2026-04-07] PR #46 — Expand analysis: inspiration alongside warnings
Summary
Expands the idea analysis framework to capture **what could go right** alongside what could go wrong. Adds 4 new fields to the analysis schema and demonstrates them with 3 full examples.
New Analysis Fields
- `novel_solutions` — : creative approaches characters use to solve problems (e.g., book-memorizer distributed backup network in The Fireman)
- `narrative_innovations` — : how the author's structural choices enhance the idea (e.g., radio-transcript POV in Ideas Die Hard)
- `real_world_impact` — : documented influence on technology, policy, or culture (e.g., Fahrenheit 451 becoming standard censorship text)
- `inspirational_concepts` — : ideas that could spark new research or creativity (e.g., fictional heroes can inspire real courage in Benny Cemoli)
Motivation
The archive should not just warn decision-makers about pitfalls. It should also:
- Surface **novel problem-solving approaches** from fiction
- Highlight **perception-expanding concepts** (like The Expanse's Ring Entities revealing atomic-scale awareness)
- Document **narrative innovations** that make ideas more impactful
- Track **real-world influence** of speculative fiction
Example Analyses (3)
1. **mandatory-happiness-through-ignorance** (The Fireman/Bradbury): distributed human backup of culture, subversive questioning, inverted protector symbol
2. **autonomous-media-creating-reality** (Benny Cemoli/Dick): adversarial fiction generators, stories as independent agents, automated beneficial propaganda
3. **cognitive-regression-in-space** (Ideas Die Hard/Asimov): cognitive limits as progress, changing the observer not the observed, evolutionary artifact intuitions
Pre-commit checklist
- [x] `python3 tools/validate_catalog.py` — 2537 valid, 0 invalid
- [x] `python3 tools/generate_site.py` — 668 ideas, 1869 stories
- [x] CHANGELOG.md updated
[2026-04-07] PR #45 — Add optional analysis field to idea schema for AI reasoning
Summary
Adds an optional `analysis` object to the idea definition in the catalog schema. This structured analytical metadata captures reasoning from full-text analysis that would otherwise be lost, designed for downstream AI consumption (e.g., via a future MCP server).
Changes
Schema (`catalog/schema.json`)
- New optional `analysis` object on ideas with six sub-fields:
- `mechanism_chain`: ordered causal steps the story depicts
- `structural_analogies`: real-world domains where the same dynamic applies
- `analogy_limitations`: where the fictional scenario diverges from reality
- `outcome_specificity`: precise outcome characterization beyond taxonomy labels
- `confidence`: extraction confidence score (0-1)
- `extraction_rationale`: textual evidence for the story-idea mapping
Scripts (`tools/enrich_fulltext_phase2.py`)
- `create_idea()` helper now accepts an optional `analysis` parameter
Instructions
- Extraction skill — (Step 5b): detailed guidance for populating the analysis block during full-text extraction, including the key principle that output is optimized for AI reasoning, not human summary
- Architecture docs — : Phase 2 pipeline description updated with analysis block details
- Copilot instructions — : schema field reference updated
Validation
- All 2,524 existing entries pass validation (the field is optional)
- Site generates cleanly (656 idea pages, 1,868 story pages)
Why now
Every full-text analysis run that happens without this structure is work that would need to be redone or backfilled. Adding this before further Galaxy Magazine or Gutenberg analysis rounds means the richer output is captured on the first pass.
[2026-04-07] PR #44 — Full-text deep analysis of 10 Galaxy Magazine stories
Summary
Deep analysis of 10 Galaxy Magazine stories based on reading the complete OCR text from Archive.org DjVu XML scans. This replaces metadata-based synopses with detailed plot descriptions and extracts new ideas visible only through full-text reading.
Synopses Rewritten (9 existing stories)
All synopses enriched from ~250 chars to ~600 chars based on actual story content:
- The Fireman (Bradbury), Second Childhood (Simak), Star Bright (Clifton)
- The Altar at Midnight (Kornbluth), The Snowball Effect (MacLean)
- The Moon Is Green (Leiber), The Old Die Rich (Gold)
- Soldier Ask Not (Dickson), If There Were No Benny Cemoli (Dick)
New Story (1)
- Ideas Die Hard — (Isaac Asimov, Galaxy Oct 1957) — Two astronauts on the first crewed Moon flyby suffer cognitive regression, becoming unable to accept the Earth is round despite seeing it directly. With Archive.org deep link.
New Ideas (6)
- Mandatory Happiness Through Enforced Ignorance — state eliminates books/complexity to maintain contentment (Fireman)
- Institutional Sacrifice of Workers for Technological Progress — knowingly sending workers into harmful conditions (Altar at Midnight)
- Absolute Safety as Psychological Imprisonment — total protection becomes its own cage (Moon Is Green)
- Time Arbitrage with Biological Aging Cost — temporal foreknowledge costs the years needed to use it (Old Die Rich)
- Autonomous Media System Creating Political Reality — automated news fabricates leaders that inspire real movements (Benny Cemoli)
- Cognitive Regression Under Existential Displacement — human cognition has hard limits when environment becomes alien (Ideas Die Hard)
Pre-commit checklist
- [x] `python3 tools/validate_catalog.py` — 2531 valid, 0 invalid
- [x] Bidirectional links verified
- [x] `python3 tools/generate_site.py` — 662 ideas, 1869 stories
- [x] CHANGELOG.md updated
[2026-04-07] PR #43 — Search tags and deep link filter
Summary
Two search/filter improvements to the site.
Tag Search
The search box now matches against story and idea **tags** in addition to titles, authors, synopses, and descriptions. Searching for `galaxy-magazine`, `censorship`, `hugo-winner`, or any tag now returns matching entries.
Deep Link Filter
A new **Source** dropdown filter with a "Has Deep Link" option shows only stories that have direct links to their original text (e.g., Archive.org scans of Galaxy Magazine issues). Currently 9 stories have deep links. This filter will become more valuable as we add more Galaxy stories.
Implementation
- Story card `data-search` attribute now includes tags
- New `data-deeplink` attribute on story cards ("1" if `archive_org_url` is set)
- New `filter-deeplink` dropdown in the filter bar
- JS filter logic updated to support the new filter
- Hash URL sync supports `#deeplink=1` for bookmarkable filtered views
- Search placeholder updated to "Search titles, authors, synopses, tags..."
Pre-commit checklist
- [x] `python3 tools/validate_catalog.py` — 2524 valid, 0 invalid
- [x] `python3 tools/generate_site.py` — generates without errors
- [x] CHANGELOG.md updated
[2026-04-07] PR #42 — Idea extraction round 11: 6 new ideas, 27 story-idea links
Summary
Round 11 of idea extraction, focusing on Galaxy Magazine stories and classic SF novels.
New Ideas (6)
- Immortality Ennui — eternal life without purpose (Second Childhood, Simak)
- Accelerating Child Superintelligence — child transcends humanity (Star Bright, Clifton)
- Human Cost of Space Travel — astronauts broken by radiation (Altar at Midnight, Kornbluth)
- Gerontocratic Wealth Hoarding — immortal elderly hoard everything (The Old Die Rich, Gold)
- Exponential Organizational Growth — optimized org absorbs all (The Snowball Effect, MacLean)
- Parallel World Tourism — alternate timelines as tourist destinations (Frugal Wizard, Sanderson)
Story-Idea Links (27)
Linked 29 stories to existing ideas across Asimov, Ballard, Brunner, Butler, Hamilton, Harrison, Huxley, Oppel, Sanderson, Silverberg, Sherman, Tepper, Tucker, Williams, Phillips, and Baxter.
Coverage
- Before: 1,151 / 1,868 stories (61.6%)
- After: 1,180 / 1,868 stories (63.2%)
Pre-commit checklist
- [x] `python3 tools/validate_catalog.py` — 2530 valid, 0 invalid
- [x] Bidirectional links verified (script handles both directions)
- [x] `python3 tools/generate_site.py` — 662 ideas, 1868 stories
- [x] CHANGELOG.md updated
[2026-04-07] PR #40 — Add first 9 Galaxy Magazine stories with Archive.org deep links
Summary
Adds the first batch of Galaxy Magazine stories to the catalog, each with page-level Archive.org deep links to the original magazine scans.
New Stories (9)
- The Fireman — Ray Bradbury (Galaxy, Feb 1951)
- The Old Die Rich — H.L. Gold (Galaxy, Mar 1953)
- The Altar at Midnight — C.M. Kornbluth (Galaxy, Nov 1952)
- Star, Bright — Mark Clifton (Galaxy, Jul 1952)
- The Snowball Effect — Katherine MacLean (Galaxy, Sep 1952)
- The Moon Is Green — Fritz Leiber (Galaxy, Apr 1952)
- Second Childhood — Clifford D. Simak (Galaxy, Feb 1951)
- Soldier, Ask Not — Gordon R. Dickson (Galaxy, Oct 1964)
- If There Were No Benny Cemoli — Philip K. Dick (Galaxy, Dec 1963)
Schema & Site Updates
- Added `archive_org_url` field to `external_ids` in schema
- Site generator shows Archive.org links in "Find This Book" section
- Archive.org URL validated (https + archive.org host) before rendering
- Archive.org URL hidden from raw metadata display
Validation
- 2,524 valid entries (0 invalid)
- All bidirectional story-idea links verified
- Site generates without errors (1,868 stories, 656 ideas)
Pre-commit checklist
- [x] `python3 tools/validate_catalog.py` — 2524 valid, 0 invalid
- [x] Bidirectional links consistent
- [x] `python3 tools/generate_site.py` — generates without errors
- [x] CHANGELOG.md updated
[2026-04-07] PR #41 — Expand copilot-instructions with operational patterns
Adds seven sections to `.github/copilot-instructions.md` capturing patterns learned from prior sessions that were not previously recorded:
1. **Copilot PR review waiting strategy** — no shell sleep loops or timeouts; sequential API polling; productive use of wait time; never give up
2. **Branch protection enforcement** — explicit rules about what main requires
3. **Story draft pipeline layout** — documents `scratchpad/story-drafts/<source>/` subdirectories and promotion workflow
4. **Galaxy Magazine pipeline** — fetch, parse, batch tools and `internetarchive` dependency
5. **Junk entry cleanup** — `None-None.json` rule added to Key Operating Rules
6. **Schema field quick reference** — story and idea fields, `external_ids` keys
7. **Generated site clarification** — reinforces that `generated-assets/site/` is never committed; CI deploys to `efa-site`
No catalog data changes; instructions-only update.
[2026-04-06] PR #39 — Rewrite README with current features and setup instructions
Complete rewrite of README.md:
- Updated catalog counts (1,859 stories, 656 ideas, 459 series, 137 universes)
- All new features documented: SF Dictionary, Galaxy pipeline, bookmarkable URLs, community links
- Three separate quick start paths: Codespaces, VS Code Dev Containers, and local
- VS Code path: install Dev Containers extension, clone, Reopen in Container, auto-configures
- Codespaces path: one-click, zero-config
- Simplified repo structure and commands
- Copilot integration and tech stack sections
[2026-04-06] PR #38 — Update repo references to efa-catalog/efa-site
Updates all internal references after repo rename:
- GitHub URLs in tools, README, CONTRIBUTING, issue templates
- Deploy workflow targets urubos/efa-site
- User-Agent strings reference efa-catalog
- Clone instructions updated
Note: secret name kept as TASAT_SITE_DEPLOY_TOKEN (rename manually when convenient). Workspace mount path will update on next container rebuild.
[2026-04-06] PR #37 — Rebrand to Extrapolated Futures Archive (EFA)
Renames the project from 'TASAT Catalog' to 'Extrapolated Futures Archive' across all branding.
**Updated (24 files):**
- Site titles, footers, browser tabs, subtitles
- Copilot instructions, contributing guide, issue templates
- Skills, prompts, READMEs
- Schema description, catalog README, taxonomy README
- User-Agent strings in all tools
- Pipeline orchestrator, utilities
- Changelog header
**Preserved:**
- TASAT credited as inspiration in README
- External TASAT data source URLs (commentary-sources.json, sources.md)
- Historical changelog entries
- Repo name and deploy config (require GitHub admin action)
[2026-04-06] PR #36 — SF Dictionary integration: deep links on idea pages
Changes
**SF Dictionary links on 494 idea pages:**
- Terms from the Historical Dictionary of Science Fiction (sfdictionary.com) matched against idea titles, descriptions, and tags
- Deep links to individual term definitions (e.g. `sfdictionary.com/view/440/artificial-intelligence`)
- Up to 5 most relevant terms per idea, prioritizing multi-word and longer terms
**Dataset:**
- `datasets/sfdictionary-index.json`: 1,306 unique SF terms with view URLs
- Extracted from sfdictionary.com's public headword JavaScript index (`/hws.js`)
- Single fetch, no crawling of individual pages
[2026-04-06] PR #35 — Make start-site.sh resilient to missing venv/dependencies
Guards against postAttachCommand running before postCreateCommand finishes:
- Checks .venv exists before sourcing activate
- Site generation uses only stdlib; failure caught gracefully
- mkdir -p ensures site directory exists so HTTP server always starts
- Server always listens on 8080 (serves cached site or empty dir)
[2026-04-06] PR #34 — Auto-authenticate gh from devcontainer credential helper
Problem
Users had to manually run `gh auth login` after devcontainer setup, even though VS Code's credential helper already has a valid GitHub token.
Fix
- setup.sh now auto-authenticates `gh` from the git credential helper
- Retry loop (5 attempts, 2s) handles timing when credential helper is still initializing
- Removed confusing 'Codespaces' references (this is a devcontainer)
- Manual `gh auth login` message only appears if auto-auth fails
[2026-04-06] PR #33 — Galaxy Magazine download and parsing pipeline
Galaxy Magazine Pipeline
New tool for downloading and parsing the full Galaxy Magazine collection (446 issues, 1950-1980) from Archive.org.
**Download (`--download`):**
- Uses the official `internetarchive` Python library with proper User-Agent per [Archive.org bot access policy](https://archive.org/developers/bots.html)
- Downloads DjVu XML files (per-page OCR text with page boundaries)
- Manifest-based caching with `--checksum` to avoid re-downloads
- Rate-limited (1s delay between requests)
**Parsing (`--parse`):**
- Extracts story titles, authors, and page numbers from OCR page text
- Generates deep-link URLs to exact pages: `https://archive.org/details/{id}/page/n{page}/mode/2up`
- Filters false positives from ads and masthead text
- Outputs structured JSON for catalog ingestion
**Context:**
Galaxy Magazine, founded and edited by H. L. Gold, was the leading SF magazine focused on social speculation (1950-1980). The Archive.org collection has 440 issues with OCR text. This pipeline is the first step toward cataloging hundreds of classic short SF stories with direct page-level links to the full text.
[2026-04-06] PR #32 — Tristate toggle tooltips + changelog category overhaul
Changes
**Tristate toggle tooltips:**
- CSS-only hover tooltips on series/universe filter toggles
- Shows the three modes: Multi-entry only / All / Single-entry only
- Positioned below the toggle to avoid obscuring other controls
- No JavaScript needed; pure CSS `:hover` with absolute positioning
**Changelog categories reorganized (4 categories):**
- Renamed "Design Changes" -> "Data" (was 4 entries, now merged with existing Data = 21 total)
- Added "Process" as a filterable category (6 entries for workflow/instruction changes)
- Removed unused "Data Sources" filter (was in UI but rarely used in changelog)
- Final: Data (21), Frontend (14), Backend (14), Process (6)
[2026-04-06] PR #31 — Phase 2: Full-text enrichment of public domain classics
Phase 2 Full-Text Enrichment
Full-text analysis of 25 cached Project Gutenberg texts reveals additional speculative ideas in classic SF works that were missed by synopsis-only extraction.
**8 new ideas:**
- `creator-moral-debt`: creator's obligation to their creation (Frankenstein)
- `class-to-species-divergence`: class divide evolving into separate species (Time Machine)
- `cosmological-entropy-witnessed`: witnessing the heat death of Earth (Time Machine)
- `forced-civilization-reversion`: uplifted beings reverting when authority removed (Moreau)
- `rejection-drives-monstrosity`: social rejection creating violent outcasts (Frankenstein)
- `perception-limited-by-dimension`: dimensional experience constraining understanding (Flatland)
- `wonder-energy-transforms-society`: unlimited energy reshaping governance (Coming Race)
- `last-survivor-psychology`: psychological collapse of the last human (The Last Man)
**13 stories enriched** (each now has 2-3 ideas, up from 1):
Frankenstein, The Time Machine, Island of Dr. Moreau, Jekyll & Hyde, The Last Man, Flatland, The Coming Race, Mizora, The Last American, Alice in Wonderland, Through the Looking-Glass, 20000 Leagues, Journey to the Centre of the Earth.
Validation: 2,515 entries, 0 invalid, 0 broken links. Idea count: 648 -> 656.
[2026-04-06] PR #30 — Devcontainer improvements: Node 20, auth, cleanup
Changes
**Devcontainer improvements:**
- Node 18 (EOL) upgraded to 20 LTS
- Removed redundant `.vscode/settings.json` generation from setup.sh
- Improved `gh` auth check (explains Codespaces auto-auth)
- Port-kill robustness: `fuser` before `lsof`, with `xargs` for multi-PID
- `gh` auth instruction updated in copilot-instructions.md
**New tool: Gutenberg text fetcher (`tools/fetch_gutenberg_texts.py`):**
- Searches Gutendex API for public domain SF works matching catalog stories
- Downloads plain text using Gutenberg's cache/epub URL pattern
- Rate-limited (3s between requests), capped at 50 per run
- Proper User-Agent, compliant with Gutenberg robot access policy
- `scratchpad/gutenberg-texts/` added to `.gitignore` (local-only data)
[2026-04-06] PR #29 — Fix issues from Copilot review audit (PRs #17-#28)
Addresses unresolved Copilot review feedback from past PRs:
Frontend (generate_site.py)
- applyHash clears search box — when URL hash omits `search` param, preventing stale queries (PR #17)
- try-catch on decodeURIComponent — in applyHash, preventing malformed percent-encoding from breaking hash sync (PR #19)
- Contribution links spacing — : replaced `margin-top:0` with `margin-top:-0.5rem` to fix CSS margin collapse (PR #27)
Backend (generate_site.py)
- shutil import — moved from inside loop to top of `main()` (PR #22)
Process (.github/copilot-instructions.md)
- Em-dashes replaced with colons — per the repo's own writing style rules (PR #20)
Not addressed (no longer applicable):
- Graph data blob ordering (PR #19): moot since `generated-assets/site/` is gitignored
- Changelog page not regenerated (PR #27): moot since CI generates the site
- Batch script improvements (PR #23): one-time scripts, not run again
[2026-04-06] PR #28 — Generate site in CI; remove generated-assets/site/ from repo
Problem
PRs containing regenerated site files (2,500+ files) exceeded Copilot's 300-file review limit, causing reviews to be skipped silently.
Solution
- CI generates the site — : `deploy-site.yml` now runs `python tools/generate_site.py` before deploying
- New deploy triggers — : fires on changes to `catalog/**`, `taxonomy/**`, `tools/**`, or `CHANGELOG.md` (was `generated-assets/site/**`)
- Site removed from version control — : added to `.gitignore`; ~2,500 generated files untracked
- Instructions updated — : pre-commit checklist and deploy trigger docs updated
Result
PRs now contain only catalog data and code changes (well under 300 files), enabling proper Copilot code review on every PR.
[2026-04-06] PR #27 — Move contribution links from footer to nav bar
Moves 'Suggest a story', 'Suggest an idea', and 'Report an error' links from the page footer to directly below the What-If Query / Idea-Story Graph / Export JSON navigation bar for better visibility.
[2026-04-06] PR #26 — Remove redundant issue title prefix from story/idea templates
The `[Story] ` and `[Idea] ` prefixes in the issue title were redundant with the form's Title field. Removed them so users only fill in the title once. Labels (`story-submission`, `idea-submission`) handle categorization.
[2026-04-06] PR #25 — Refine issue templates: optional fields, better pre-fill
Changes
**Story suggestion template:**
- Year published, work type, and synopsis are now optional (only title and author required)
**Feedback template:**
- Entry identifier field now accepts an ID, title, or full URL
- Entry type changed from dropdown to text input so site links can pre-fill it
**Site feedback links:**
- Issue title now includes the entry's display name (e.g. `[Feedback] The Wild Robot`)
- Pre-fills entry-type (`Story`/`Idea`) and entry-id (slug)
Changelog updated.
[2026-04-05] PR #24 — Community contribution infrastructure: issue templates, contributing guide, feedback links
Phase 3 (Community Augmentation) — Tier 1
**GitHub Issue Templates:**
- Suggest a Story — structured form for title, author, year, type, synopsis, speculative ideas
- Suggest an Idea — form for idea title, description, linked stories, real-world relevance
- Feedback — report errors or suggest edits to existing entries, with entry type and ID pre-fillable
**CONTRIBUTING.md:**
- How to submit stories, ideas, and feedback via GitHub issues
- Developer setup and pre-commit checklist
- What makes a good catalog entry
**Site Feedback Links:**
- Every story page: "Suggest an edit" link pre-filled with the entry ID
- Every idea page: "Suggest an edit" link pre-filled with the entry ID
- Index page footer: "Suggest a story", "Suggest an idea", "Report an error" links
All links point to the structured issue templates for consistent submissions.
[2026-04-05] PR #23 — Idea extraction round 10: 9 new ideas, 45 stories addressed
Changes
**9 new ideas created:**
- dinosaur-civilization-alternate-evolution, lost-world-prehistoric-survival, post-atomic-cargo-cult
- robot-wilderness-adaptation, parallel-universe-alternate-self, language-as-virus
- ancient-alien-artifacts-awakened, environmental-dome-society-split, vr-game-societal-control
**15 SF stories** linked to new or existing ideas (Caspak, Eden, Wild Robot, War Dogs, Warcross, Nova Trilogy, Kefahuchi Tract, Zeroes, Desperation).
**30 stories tagged needs-review:**
- 18 fantasy/non-SF (Harry Potter, Dragonlance, Memory Sorrow & Thorn, Inheritance, Darksword, Boggart, Westmark)
- 12 thin-synopsis (Worldwar, Weirdstone, Nova Swing, etc.)
**18 pre-existing asymmetric bidirectional links fixed** across the catalog.
**Related ideas recomputed** (45 new cross-links).
Validation: 2,508 entries, 0 invalid, 0 broken links. Idea count: 639 -> 648.
[2026-04-05] PR #22 — Validation cleanup: remove duplicate Deepness in the Sky
Changes
- Removed duplicate `a-deepness-in-the-sky-vinge.json`, merged into `a-deepness-in-the-sky-zones-of-thought-vinge.json` (which has series/universe, ISFDB metadata, richer tags)
- Fixed title: removed "(Zones of Thought)" parenthetical
- Updated `interstellar-trade-contact` idea to remove duplicate story reference
- Reviewed near-duplicate ideas (`lost-civilization-contact` / `multi-level-civilization-contact`): confirmed distinct, no action needed
- Validation: 2,498 entries, 0 invalid, all cross-references OK
- Story count: 1,860 → 1,859
[2026-04-05] PR #21 — Add Copilot PR review loop to Git Workflow
Adds the Copilot review loop requirement to the Git Workflow section of `.github/copilot-instructions.md`:
After opening a PR, wait for Copilot's review. For each feedback comment, fix the code, push, and resolve the conversation thread. Request a new review. Repeat until Copilot has no further objections, then merge.
This ensures every PR goes through iterative Copilot review before merging.
[2026-04-05] PR #19 — Sync all filters to URL hash; dynamic page title for series/universe
Changes
**Bookmarkable/shareable filter URLs:**
Every filter change (domain, scenario, outcome, type, decade, series, universe, search) now updates the URL hash in real time. Example:
```
index.html#series=Dungeon%20Crawler%20Carl
index.html#domain=ai-ml&decade=2020
index.html#search=robot
```
URLs can be bookmarked or shared and will restore the exact filter state on load.
**Dynamic browser tab title:**
When a series or universe filter is active, the browser tab shows the name (e.g. "Dungeon Crawler Carl — TASAT Catalog"). Reverts to the default title when cleared.
**Implementation notes:**
- Uses `history.replaceState` instead of setting `location.hash` directly to avoid polluting browser history with every keystroke.
- Loop prevention: `applyHash()` sets a suppress flag so `syncHash()` is a no-op during hash-driven filter updates.
- Changelog updated.
[2026-04-05] PR #20 — Add best practices to copilot-instructions
Adds four new sections to `.github/copilot-instructions.md` based on lessons learned:
Git Workflow
- Never push directly to `main`; always branch → PR → review → merge
- Always update `CHANGELOG.md` when deploying
- Use `gh` CLI for all GitHub operations; includes auth command for devcontainers
Pre-commit Checklist (catalog changes)
Mandatory steps before committing catalog data:
1. `python3 tools/validate_catalog.py` (0 invalid)
2. Verify bidirectional story↔idea links
3. `python3 tools/generate_site.py`
4. Update `CHANGELOG.md`
5. Commit on feature branch and open PR
Security
- Never echo/print/log credentials in terminal commands
- No hardcoded tokens in curl or scripts
Bidirectional Links (Key Operating Rule #9)
- When adding/updating a story, ensure its idea IDs list the story back, and vice versa
These codify practices that were previously implicit or only in memory files.
[2026-04-04] PR #17 — Fix: filter counts populate on initial page load
Root cause: applyFilters was inside an IIFE and the previous fix tried calling it from outside (ReferenceError, silently failed). Fixed by removing the early return in applyHash so it always dispatches the input event on the search box, which triggers applyFilters within the correct scope.
[2026-04-04] PR #16 — Tristate toggle refinements + fix initial count population
Toggle refinements:
- Grey knob in all states (not colored)
- Icons colored: red X when active, green check when active, dimmed otherwise
- "Multi"/"All"/"Singles" label shown next to toggle
Fix: filter counts now populate on initial page load (applyFilters called explicitly, not just via hash dispatch).
[2026-04-04] PR #15 — Visual toggle for series/universe single/multi mode
Replaced the dropdown mode selector with a visual three-state toggle:
- Red X (left): Exclude singles — only multi-entry series shown
- Grey center: Include all series
- Green checkmark (right): Singles only
Click cycles through states. Matches the iOS-style toggle reference.
[2026-04-04] PR #14 — Add three-way toggle for series/universe filters
Series and Universe dropdowns now have a mode toggle:
- Multi only — (default): Shows only series/universes with 2+ stories
- All — : Shows all series/universes
- Singles only — : Shows only single-entry series/universes
Each option has a `data-entrycount` attribute. Toggle hides options that dont match the mode. Reset restores default.
[2026-04-04] PR #13 — Add 70 stories completing 47 single-entry series
Systematic analysis of all 251 single-entry series against Wikipedia. Added companion novels for 47 series including Earthsea (3 books), Harry Potter 5-7, Enders Shadow sequels, Dark Tower 2-3, Otherland, Red Queen, Wormwood, Final Architecture, Lady Astronaut, and more. Story count: 1,782 -> 1,852.
[2026-04-03] PR #12 — Fix filter counts: show counts excluding own selection
Each story-level dropdown (Type, Decade, Series, Universe) now shows counts computed with that specific filter removed. Previously, selecting "Amber" universe would show all other universes as (0). Now each option shows its actual count assuming all other filters stay the same.
[2026-04-03] PR #11 — Show series/universe filters always with dynamic counts
Series and Universe filter dropdowns now visible at all times (not hidden until clicked). Each option shows a live count of matching stories, just like Domain, Scenario, Type, and Decade filters.
[2026-04-03] PR #10 — Series gap-fill batch 5 + War Dogs fix
9 new stories completing Majipoor (3-5), Stravaganza (2-3), Withern Rise (1-2), plus Jackpot (Gibson 2025) and Sunrise on the Reaping (Collins 2025). Fixed War Dogs series name. Story count: 1773 -> 1782.
[2026-04-03] PR #9 — Series gap-fill: 117 new stories completing ~35 series
Systematic series completion using Wikipedia book counts compared against catalog.
**117 new stories** across 5 batches:
- Batch 1 (25): Hitchhiker #1, ASOIAF #3, Hyperion #2, Gateway, Forever War, Harry Potter 1-3, etc.
- Batch 2 (23): Children of Ruin/Memory, Redemption Ark, Ringworld Engineers, Lensman 3-5, etc.
- Recent (5): Children of Strife, Absolution, Book of Dust trilogy
- Batch 3 (35): Helliconia, Terra Ignota, Night Dawn, Orthogonal, Blue Ant, etc.
- Batch 4 (29): Caspak, Worldwar, Memory Sorrow & Thorn, Westmark, etc.
Story count: 1,656 -> 1,773. Removed per-page timestamps (regen diffs now ~80 files instead of 2,300+).
[2026-04-03] PR #8 — Add 23 more stories + remove page timestamps
**23 new story entries** filling series gaps (batch 2). Key additions:
- Children of Ruin + Children of Memory (completing Children of Time trilogy)
- Redemption Ark (Revelation Space #3)
- The Ringworld Engineers, Charlie and the Chocolate Factory
- Through the Looking-Glass, Echopraxia (Firefall #2)
- Galactic Patrol + Gray Lensman + Second Stage Lensmen (Lensman #3-5)
- Weirdstone of Brisingamen (Alderley #1), and more
**Timestamp cleanup**: Removed per-page timestamps from generated HTML. Regeneration diffs reduced from 2,300+ files to just the files that actually changed (99 this commit vs 2,300+ previously).
[2026-04-03] PR #7 — Add 25 stories to fill series gaps
25 new story entries filling gaps in major series. Synopses sourced from OpenLibrary API. 20 linked to existing ideas via series-sibling inheritance.
Key additions:
- The Hitchhikers Guide to the Galaxy (HHGG #1)
- A Storm of Swords (ASOIAF #3)
- The Fall of Hyperion (Hyperion Cantos #2)
- Gateway (Heechee #1, Hugo+Nebula winner)
- The Forever War (#1, Hugo+Nebula winner)
- Harry Potter 1-3
- Northern Lights + The Subtle Knife (His Dark Materials)
- Count Zero (Sprawl #2)
- Ancillary Sword (Imperial Radch #2)
- The Robots of Dawn (Baley #3)
- That Hideous Strength (Lewis Space Trilogy #3)
- and 14 more
[2026-04-03] PR #6 — Address Copilot PR #4 review comments
Fixes all 5 issues raised by Copilot code review on PR #4:
1. **series_position schema** — Restricted from `integer|string|null` to `number|null` (no arbitrary strings)
2. **Mojibake in synopsis_isfdb** — Fixed 60 entries with encoding artifacts
3. **Stray literal n characters** — Fixed in enrichment-sourced synopses
4. **annual_views > views anomaly** — Fixed 51 entries where values were swapped
5. **All 5 specific files Copilot flagged** — Verified clean
[2026-04-03] PR #5 — Trigger site deploy
Empty trigger to redeploy site
[2026-04-03] PR #4 — Series/Universe, ISFDB enrichment, Find This Book, site improvements
Major update combining all dev work:
**Site UX:**
- Parallel Ideas/Stories columns, Relevant/Popularity sort options
- TASAT acronym in subtitle, auto-sort by relevance on filter
- Find This Book links (Wikipedia, Amazon, Audible, Google Books, Goodreads, OpenLibrary)
- Series/Universe displayed on cards and detail pages with dedicated filter dropdowns
- Series order sort, hashchange navigation fix
**Data enrichment:**
- ISFDB dump: 1,318 stories matched, ratings/views/synopses/IDs added
- ISBN enrichment: 56 stories with multi-ISBN edition metadata from OpenLibrary
- Series: 453 series, 137 universes from ISFDB hierarchy
- 20 titles refined with ISFDB subtitles
- Wikipedia-verified series audit (440 articles checked)
**Infrastructure:**
- Devcontainer server fix (postAttachCommand)
- New tools: enrich_isbns.py, enrich_from_isfdb_dump.py, enrich_series.py, audit_series_wikipedia.py
[2026-04-03] PR #3 — Add changelog entries and update post-work checklist
- Added [2026-04-03d] changelog entry for all session work
- Changelog now included in post-work checklist
[2026-04-03] PR #2 — Add Find This Book links and ISBN edition metadata
- Story pages: Wikipedia, Amazon, Audible, Google Books, Goodreads, OpenLibrary links
- Multi-ISBN support with edition descriptions (publisher, year, format, language)
- New tool: enrich_isbns.py with --refresh flag
- Schema: isbn accepts string, array of strings, or array of objects
- 56 stories enriched with edition metadata from OpenLibrary
- Expandable ISBN display: primary edition inline, rest collapsible
[2026-04-03] PR #1 — Expand TASAT acronym in site subtitle
Spells out the TASAT acronym in the site subtitle with bolded initials.