Zendesk digest: link unflagged tickets, list new non-English ones; sogs-ban: show posted names

deploy/env/zendesk.env.example

@@ -19,8 +19,5 @@ ZENDESK_WEBHOOK_SECRET=
 # text, so it is not in the repo; copy it by hand:
 #     install -o zendesk -g zendesk -m 640 house_answers.json /var/lib/session-ops/zendesk-digest/
 #ZENDESK_HOUSE_ANSWERS=/var/lib/session-ops/zendesk-digest/house_answers.json
-# Optional. A non-customer-visible multi-line text ticket field (its id is in its URL)
-# that gets an English rendering of each non-English ticket the digest posts.
-#ZENDESK_ENGLISH_FIELD_ID=
 # The whole `claude:` note path, writing nothing to Zendesk. Set it for the first deploy.
 #RELAY_DRY_RUN=1

docs/jobs/zendesk-digest.md

@@ -49,7 +49,7 @@ The star-floor machinery (`partition_reviews`, `--review-star-floor`, `--include
 
 Twitter DM tickets arrive with `description` identical to `subject` — both just `"Conversation with <handle>"` — which is 15% of non-review tickets and unclassifiable as fetched. For those only, `hydrate_descriptions` fetches a page of up to 10 comments and joins every body that differs from the subject into the description; later replies often carry the actual detail. Hydration is an enrichment, so an HTTP error or an unreachable endpoint leaves the ticket as-is rather than failing the run (`--no-hydrate` to skip it entirely).
 
-The script (`zendesk-triage`, [triage.py](../../src/session_ops/zendesk/triage.py)) fetches the tickets in a rolling time window, classifies the whole batch in one schema-enforced request through the Claude Code CLI, and posts a Discord digest: a short header, then one line per ticket worth looking into.
+The script (`zendesk-triage`, [triage.py](../../src/session_ops/zendesk/triage.py)) fetches the tickets in a rolling time window, classifies the whole batch in one schema-enforced request through the Claude Code CLI, and posts a Discord digest: a short header, the new tickets not in English, one line per ticket worth looking into, then the abuse reports and the unflagged tickets.
 
 Each line leads with a severity marker, a category emoji and a platform icon, links the ticket id, and carries the model's one-line summary plus its root-cause guess:
 
@@ -59,11 +59,14 @@ Backlog: **428** awaiting a reply, excluding app-store reviews (**5,252** more a
 **9** worth looking into.
 ⭐ **6** · 🐛 **3** · ❓ **2** · 🔇 **2** · 🔑 **1** · ⚖️ **1** · 🔒 **1**
 Likely duplicates: **push-notifications-not-delivered** ×5 (#27637, #27610, #27606, #27605)
+🌐 **2** new, not in English — #27610 German, #27648 Russian
 🚨 | ⚖️ | ❔ | #27632 · Police summons demanding user details for a Session ID
 🚨 | 🔒 | 🤖 | #27603 · Exported component lets another app obtain internal SharedPreferences | Likely cause: Improperly exported provider allowing external apps to trigger file sharing
 🟠 | ⭐ | 🍎 | #27610 · Messages not delivered for days; nothing shows even after opening | Likely cause: Push notification delivery / message retrieval failure
 🟠 | 🐛 | 🤖 | 🔄 #27605 · Message and call notifications only appear when the app is opened | Likely cause: Push notification service failure on Android
 🔇 **2** abuse reports — reported Session IDs, nothing actionable (#27640, #27641)
+💤 🔑 **2** not flagged — #27650, #27652
+💤 ❓ **1** not flagged — #27648
 ```
 
 | Column | Values |
@@ -72,7 +75,9 @@ Likely duplicates: **push-notifications-not-delivered** ×5 (#27637, #27610, #27
 | Category | The emoji from `CATEGORY_SPECS`, so it matches the tally line |
 | Platform | 🤖 Android · 🍎 iOS · 🖥️ desktop (all three) · 🌐 multiple · ❔ unknown |
 
-The header accounts for the batch in full, so nothing is dropped silently. The backlog line is scoped exactly like the analysis — `status<pending`, **excluding app-store reviews** — so the number and the tickets under it mean the same thing. That matters because 92% of unsolved tickets are AppFollow reviews, so the unqualified number reads as roughly 13× the queue that actually needs a human (5,680 against 428). Both counts come from Zendesk's count-only search endpoint, one request each and both best-effort — if the review-excluded count fails, the line falls back to the plain total rather than disappearing. The category tally counts abuse reports like anything else, so the numbers still sum to what was analyzed; the collapsed line at the bottom is where they are listed, and its links stop at a character budget (the remainder counted as `+N more`) so a heavy day cannot push a message past 2,000.
+The header accounts for the batch in full, so nothing is dropped silently. The backlog line is scoped exactly like the analysis — `status<pending`, **excluding app-store reviews** — so the number and the tickets under it mean the same thing. That matters because 92% of unsolved tickets are AppFollow reviews, so the unqualified number reads as roughly 13× the queue that actually needs a human (5,680 against 428). Both counts come from Zendesk's count-only search endpoint, one request each and both best-effort — if the review-excluded count fails, the line falls back to the plain total rather than disappearing. The category tally counts abuse reports like anything else, so the numbers still sum to what was analyzed; the collapsed line at the bottom is where they are listed, and its links stop at a character budget (the remainder counted as `+N more`) so a heavy day cannot push a message past 2,000. Every other ticket the model did not flag gets a 💤 line per category, most common first, each with its own budget and linking its oldest tickets first; a ticket past it is not recorded as reported, so it stays eligible for the next run.
+
+The 🌐 line lists every ticket reported for the first time whose classified `language` is not English (a blank or `unknown` one counts as English), whatever line it also sits on: it is the to-do list for answering `claude: english` on each. A 🔄 ticket is left off it. It records nothing as reported, so a ticket past the highlight cap is listed again next run.
 
 **One block per ticket, and the digest is read-only.** Each line is its own Components V2 Text Display inside one Container, so the digest is skimmed rather than read as a wall. Nothing in it is interactive: replies are written on the ticket itself (see [Zendesk Reply from a Private Note](zendesk-relay.md)), and a button here would open a compose flow that no longer exists.
 

docs/jobs/zendesk-relay.md

@@ -40,9 +40,9 @@ there are several.
 
 | Command | What it does | Touches the customer |
 | --- | --- | --- |
-| `claude: draft - <brief>` | Compose the reply from the brief, in the requester's language. A second `draft` amends the one already there rather than starting over | no |
-| `claude: reply [n]` | Publish the chosen option verbatim, status -> `pending` | **yes** |
-| `claude: english` | Post the conversation, both sides, in English. Says so and writes nothing when the ticket is already English | no |
+| `claude: draft - <brief>` | Compose the reply from the brief, in the requester's language. A second `draft` amends the one already there rather than starting over, unless it was sent or the customer wrote after it. Once support has replied, the draft answers only the customer's newest message | no |
+| `claude: reply [n] [anyway]` | Publish the chosen option verbatim, status -> `pending`. If the customer wrote after the draft, or it was already sent, it sends nothing until asked again with `anyway` | **yes** |
+| `claude: english` | Post the conversation, both sides, in English. Says so and writes nothing when the ticket is already English. Too long to send whole, it drops the oldest turns and says how many | no |
 | `claude: explain` | Post what support usually replied to this kind of ticket, what was actually done about it, and the caveats | no |
 | `claude: solve [reason]` | Solve without writing to the customer, for tickets that need no reply. The note records who decided and why | no comment, but **solving fires the CSAT automation** |
 

docs/tools/sogs-ban.md

@@ -30,6 +30,20 @@ the deletion has already walked.
 There is no flag to keep the messages or to narrow the scope: the tool exists for abuse
 reports, where both are always wanted.
 
+## Who is about to be banned
+
+Before the prompt, and in a dry run, each id is listed with the display name of its newest
+post in our rooms' last 256 messages:
+
+```
+  05abc...def  posted as 'Mallory'
+  05123...456  no recent posts
+```
+
+SOGS stores no profiles, so a name exists only in a message the account posted: an
+account that has not posted recently has none to show. A room that cannot be read is
+named under the list, since its posters would otherwise read as "no recent posts".
+
 ## Confirming the ban
 
 Each step's HTTP status is the confirmation, and the script prints one line per step:

src/session_ops/sogs/ban.py

@@ -112,6 +112,9 @@
 import requests
 from nacl.signing import SigningKey
 
+from session_ops.sogs import names as sogs_names
+from session_ops.sogs.errors import SogsError
+
 try:
     from session_util import blinding, xed25519
 except ImportError as e:  # pragma: no cover - environment, not logic
@@ -133,10 +136,6 @@
 
 
 
-class SogsError(RuntimeError):
-    pass
-
-
 class PartialBan(SogsError):
     """A run that stopped part-way: a /sequence that aborted, or a deletion that never
     reached the server. Carries the steps that did run.
@@ -392,6 +391,12 @@ def read_ids(args):
     return out
 
 
+def describe_poster(sid, names):
+    if sid not in names:
+        return "no recent posts"
+    return f"posted as {names[sid]!r}" if names[sid] else "posted, with no display name"
+
+
 def confirm(assume_yes):
     """Asks before acting. A closed stdin is a refusal, not a traceback: these run from
     cron and CI, where the prompt would otherwise fail with an unhandled EOFError."""
@@ -478,10 +483,16 @@ def main():
     for step in steps:
         print(f"  {step}")
 
+    print()
+    owners = {form: sid for sid in ids for form in (sid, *blinded_ids(sid, server_pubkey))}
+    names, unread = sogs_names.recent_names(client, rooms, owners)
+    for sid in ids:
+        print(f"  {sid}  {describe_poster(sid, names)}")
+    for token, code in unread:
+        print(f"  (could not read {token}: {code}; its posters are missing above)")
+
     if args.dry_run:
         print("\nDry run: no ban or deletion was sent.")
-        for sid in ids:
-            print(f"  {sid}")
         return 0
 
     if not confirm(args.yes):

src/session_ops/sogs/errors.py

@@ -0,0 +1,2 @@
+class SogsError(RuntimeError):
+    pass

src/session_ops/sogs/names.py

@@ -0,0 +1,110 @@
+"""
+The display names accounts last posted under, read from our rooms' recent messages.
+
+SOGS stores no profiles: a name exists on the server only inside the messages an account
+has posted, as the profile its client attaches to each one. An account that has not
+posted recently has no name to find.
+"""
+
+from base64 import b64decode
+
+from session_ops.sogs.errors import SogsError
+
+# The most GET /room/<token>/messages/recent returns.
+RECENT_LIMIT = 256
+
+CONTENT_DATA_MESSAGE = 1
+DATA_MESSAGE_PROFILE = 101
+PROFILE_DISPLAY_NAME = 1
+
+
+def _varint(buf, i):
+    out = shift = 0
+    while True:
+        b = buf[i]
+        i += 1
+        out |= (b & 0x7F) << shift
+        shift += 7
+        if not b & 0x80:
+            return out, i
+
+
+def _fields(buf):
+    """(field number, value) for each field of a protobuf message; bytes for
+    length-delimited fields, which is all a name lookup descends into."""
+    i = 0
+    while i < len(buf):
+        key, i = _varint(buf, i)
+        number, wire_type = key >> 3, key & 7
+        if wire_type == 0:
+            value, i = _varint(buf, i)
+        elif wire_type == 1:
+            value, i = buf[i:i + 8], i + 8
+        elif wire_type == 2:
+            size, i = _varint(buf, i)
+            value, i = buf[i:i + size], i + size
+        elif wire_type == 5:
+            value, i = buf[i:i + 4], i + 4
+        else:
+            raise ValueError(f"unsupported wire type {wire_type}")
+        yield number, value
+
+
+def _field(buf, number):
+    return next((v for n, v in _fields(buf) if n == number and isinstance(v, bytes)), None)
+
+
+def display_name(data_b64):
+    """The display name in a SOGS message's `data`, or None if it carries none or does
+    not parse."""
+    try:
+        raw = b64decode(data_b64)
+    except (TypeError, ValueError):
+        return None
+    # Clients pad the Content with 0x80 and then zeros.
+    raw = raw.rstrip(b'\x00')
+    if raw.endswith(b'\x80'):
+        raw = raw[:-1]
+    try:
+        name = raw
+        for number in (CONTENT_DATA_MESSAGE, DATA_MESSAGE_PROFILE, PROFILE_DISPLAY_NAME):
+            name = _field(name, number)
+            if name is None:
+                return None
+    except (IndexError, ValueError):
+        return None
+    return name.decode('utf-8', 'replace') or None
+
+
+def recent_names(client, rooms, owners):
+    """Looks accounts up in every room's recent messages.
+
+    `owners` maps each id form an account can post under (05 and both blinded 15
+    forms) to the 05 id to report it as. Returns ({id: name}, [(room token, why)]).
+    An id that posted is in the dict, with the name from its newest post (None if
+    that post carried none); an id that did not is absent. The list names the rooms
+    that could not be read, so a missing id is not mistaken for one that never
+    posted. Never raises: the lookup is informational and must not stop a ban.
+    """
+    newest, unread = {}, []
+    for room in rooms:
+        token = room['token']
+        try:
+            code, messages = client.request('GET', f'/room/{token}/messages/recent?limit={RECENT_LIMIT}')
+        except SogsError as e:
+            unread.append((token, str(e)))
+            continue
+        if code != 200 or not isinstance(messages, list):
+            unread.append((token, code))
+            continue
+        try:
+            for m in messages:
+                sid = owners.get(m.get('session_id'))
+                # A deleted message stays in the list with its data nulled.
+                if not sid or not m.get('data'):
+                    continue
+                if sid not in newest or m['posted'] > newest[sid][0]:
+                    newest[sid] = (m['posted'], display_name(m['data']))
+        except (AttributeError, KeyError, TypeError) as e:
+            unread.append((token, f"unexpected message shape ({type(e).__name__})"))
+    return {sid: name for sid, (_, name) in newest.items()}, unread

src/session_ops/zendesk/api.py

@@ -281,7 +281,7 @@ def customer_authors(session, subdomain, ticket, comments):
     return customers or {requester}
 
 
-def customer_text(session, subdomain, ticket, comments, limit):
+def customer_text(session, subdomain, ticket, comments, limit, authors=None):
     """What the customer wrote, as the signal for which language to reply in.
 
     Their words only. An agent's earlier English reply is still text on the ticket,
@@ -291,7 +291,8 @@ def customer_text(session, subdomain, ticket, comments, limit):
     # Public only. A private note is internal annotation — including the `claude:`
     # commands and the drafts this tool writes — and never the customer speaking.
     comments = [c for c in comments if c.get("public")]
-    authors = customer_authors(session, subdomain, ticket, comments)
+    if authors is None:
+        authors = customer_authors(session, subdomain, ticket, comments)
     subject = squash(ticket.get("subject"))
     parts = []
     description = (ticket.get("description") or "").strip()