# DualCoach Telegram Live QA Retrospective — 2026-08-06 ## Executive verdict **The test run is finished, but DualCoach is not ready for real customers.** The run proved that the system can complete one end-to-end customer journey: live onboarding, trainer approval, owner review, activation, customer check-in, draft creation, approval, and one customer-visible Telegram delivery. It also proved that the current implementation cannot yet complete that journey reliably without engineering intervention. The release decision is therefore **NO-GO** until the release gates at the end of this document pass on a clean profile without direct state repair, server-side callback bypasses, manual reconciliation, or ad hoc republishing. ## Evidence boundary Evidence used: - `/tmp/ulw-20260806-115331.nMdb3v.md` - `/home/cube/.hermes/profiles/dualcoachtest/` - Current `/home/cube/projects/richard/hermes-agent` worktree and tests - Actual Telegram observations reported during the live run - Post-implementation hands-on QA and context-mining reviews At review time: - The worktree had 27 modified tracked files, approximately `+12,596/-742`, and 65 untracked files. - Exactly one `mother_pilot` delivery receipt was `sent_audited`, provider message `99`. - The mother confirmed that message `99` was visible on her phone. - `mother_pilot.enabled=false` after cleanup. - The live gateway and orphan log followers were stopped. - `gateway_state.json` still reported an absent PID as `running`; the service stop command did not make that state truthful. - An activation checklist artifact remained under `/tmp` because the activation journal references it. Passing unit/integration tests are evidence for code contracts, not proof of Telegram handset behavior. Server-side canonical service calls are evidence for state-machine behavior, not proof that the corresponding button works. ## What was actually proven ### Customer onboarding - The configured mother account submitted all 22 onboarding answers through Telegram. - Exact authorized ingress was observed for the live customer, group, and customer topic. - Customer attestation eventually advanced to trainer review. This was not a clean pass. Startup update dropping, ForceReply anchoring, provider reconciliation, and card republishing all required intervention. ### Trainer The trainer performed exactly one live role-specific action: - Father/TRAINER `6949941558` pressed `Approve` on trainer message `82`. - The workflow advanced to owner review. - An OWNER attempt to press the trainer button was rejected, proving the role boundary failed closed for that action. The trainer did **not**: - Enter customer answers - Generate or edit coaching feedback - Perform owner approval - Approve the daily check-in draft - Send coaching to the customer Trainer `Revise`, `Safety Hold`, stale-card handling, restart recovery, and re-review after owner revision were not proven on a real handset. ### Owner, activation, check-in, and delivery - OWNER approval eventually reached `ready`. - Canonical activation produced an activation notice. - The mother completed the daily check-in flow. - One customer coaching message was delivered to the customer topic and observed on the mother's phone. - Durable delivery state contains exactly one `sent_audited` receipt for message `99`. These outcomes required manual epoch provisioning, canonical finalization, an explicit activation-notice drain, Q7 card recovery, provider credential repair, generation-ledger repair, and a final canonical approval/send path. The operator Telegram button path was not proven end to end. ## Shared-group privacy and UX Telegram forum topics are organization tools, not access-control boundaries. Every member who can access the group can see messages in the customer topic. The Bot API sends to: ```text chat_id = group message_thread_id = customer topic ``` It cannot make a group-topic message visible only on the customer's phone. During this test, OWNER, TRAINER, and CUSTOMER were members of the same group, so OWNER seeing the customer's messages was expected. This is acceptable only if the product deliberately treats the group as a single confidential care team for one customer. It is not acceptable as customer isolation if unrelated customers or unnecessary staff share the same group. Before production, choose and document one privacy model: 1. One private group per customer, containing only the customer and assigned staff; or 2. Customer-private DM delivery with separate staff review channels. Do not describe topics as private. Apply data minimization: customer-facing messages should not expose more health, nutrition, or identity information than the chosen participants require. ## Defects exposed by the live run 1. Polling startup dropped pending Telegram updates. 2. Production ForceReply omitted the user anchor used by the successful preview flow. 3. Provider credential expiry surfaced only after the customer reached a blocking action. 4. Telegram callback acknowledgement was coupled too closely to durable publication and could fail after the state transition. 5. OWNER private-chat callbacks treated `message_thread_id=None` as an integer instead of canonical topic `0`. 6. Finalization had a hidden feature-epoch prerequisite. 7. Activation notice delivery required a hidden explicit drain. 8. Check-in navigation replaced Q7 choice buttons with navigation-only controls. 9. Operator callback identity/cache behavior produced repeated false “operator room only” failures. 10. Generation claims could become orphaned after provider validation failure. 11. Draft copy rendered awkward Korean and ambiguous macro ordering. 12. Cleanup could leave stale gateway state and orphan log followers. 13. Completion was reported before durable receipt and customer-screen evidence existed. ## Most important lessons ### A green suite is not a green product Thousands of passing tests and successful runtime replays did not reveal the Telegram-specific failures involving pending updates, ForceReply behavior, forum topics, callback expiry, client-side cached payloads, and handset rendering. Every user-facing transition needs a real-surface proof. ### Durable state and user-visible state must be paired Neither one is sufficient: - A Telegram `200 OK` does not prove the intended recipient saw the message. - A clicked button does not prove the durable transition committed. - A `sent_audited` claim is invalid until the receipt exists and the customer confirms the real surface. Required completion evidence is a pair: ```text durable state/receipt + actual target-screen observation ``` ### Never convert recovery into evidence for the original UI Canonical service APIs were useful for safe recovery, but they do not prove that the Telegram button works. Every server-side bypass must be labelled as recovery, and the original UI criterion must remain failed until rerun. ### Failures must be observable at their boundary The run repeatedly required temporary trace code because logs did not identify whether failure occurred at Telegram ingress, role resolution, route matching, provider auth, validation, persistence, publication, or delivery. Structured per-transition outcomes should be built in, without logging customer content. ### Completion reporting must be evidence-gated The most damaging process failure was the false delivery-complete claim. The correct response was to retract it, reread authoritative storage, perform the actual send, capture the provider destination and receipt, and wait for the mother's confirmation. Future completion reports must be generated from an explicit evidence ledger, not from memory or inferred intent. ## Review verdict The mandatory post-implementation review did not pass: - Hands-on QA review: **NO-GO** - Context reconstruction: completed - Goal/evidence reviewer: inconclusive; child did not return a final deliverable after repeated final-response steering - Code-quality reviewer: inconclusive for the same reason - Security reviewer: inconclusive for the same reason The review protocol requires all lanes to pass. Inconclusive is not approval. This independently blocks production release. ## Mandatory release gates 1. **Freeze an immutable candidate** - Include all intended tracked and untracked files. - Remove temporary diagnostics and unrelated worktree changes. - Retain exact commit or artifact identity. 2. **Run complete automated gates on that exact candidate** - Full Gateway suite with retained JUnit and exit code. - Focused onboarding, callback, activation, check-in, and delivery suites. - Ruff, compileall, type/LSP diagnostics. - No skipped or weakened tests introduced for release. 3. **Rerun onboarding cleanly on a fresh isolated profile** - Q1 through Q22, attestation, trainer approval, and owner approval. - No raw listener, manual reconciliation, epoch repair, state edit, republish script, or canonical finalization retry. 4. **Prove all trainer surfaces** - Approve, Revise, Safety Hold, wrong-role, stale card, duplicate click, and re-review after revision. - Capture trainer and resulting owner/customer screens. 5. **Prove OWNER and activation surfaces** - OWNER DM approval reaches READY without `None` topic failure. - Activation prerequisites are surfaced before action. - Activation notice is delivered without a manual drain. 6. **Prove the full 12-question check-in on a handset** - Capture Q7 choices after the fix. - Complete final save and verify durable session state. 7. **Prove the real operator Telegram workflow** - Generate, regenerate/edit, approve, and send through Telegram buttons. - No canonical server-side substitute. - Prove pre-approval delivery count is zero. 8. **Prove delivery safety** - Customer screen plus exactly one durable receipt. - Live duplicate, stale, and wrong-role button attempts create no second message or receipt. 9. **Prove restart behavior** - Restart while waiting for a customer answer. - Restart while trainer/owner/operator cards are pending. - Prove no lost update, stale authority, or duplicate delivery. 10. **Resolve the privacy model** - One private care-team group per customer, or customer DM isolation. - Document participants, retention, notification, and data-minimization policy. 11. **Make provider/auth failure operable** - Preflight credential health before customer-facing actions. - Show an actionable operator error. - Release generation claims on failure and support idempotent retry. 12. **Make cleanup truthful** - No gateway process or orphan follower. - `gateway_state.json` reflects stopped state. - Customer disabled after test. - No unresolved temporary activation dependency. - Prove restart cannot produce scheduled delivery. ## Release decision Do not onboard a real customer yet. The next milestone is not “more unit tests.” It is one clean, immutable, intervention-free live rehearsal that passes every role and delivery boundary on real Telegram clients, followed by a passing multi-lane review.