From 18f06ebbfaf5d7d2308597625d276f15433035f7 Mon Sep 17 00:00:00 2001 From: Sam Rowe <14168597+Sam-Rowe@users.noreply.github.com> Date: Thu, 27 Aug 2026 03:04:35 +0100 Subject: [PATCH] Add speak-summary skill: text to listenable MP3 via local CPU TTS (#2797) Adds a skill that converts text, markdown, or the output of another skill into an MP3 using Kyutai pocket-tts, a small neural TTS model that runs on CPU. No existing skill in the collection generates audio, so this fills a gap rather than duplicating one. It is designed as a terminal step in a chain: roundup, daily-prep, or meeting-minutes produce the text, speak-summary makes it listenable. Two details worth calling out: - The bulk of SKILL.md is guidance on rewriting written prose for the ear before synthesising. Feeding markdown straight into a TTS engine produces something technically correct and unlistenable, so that step carries most of the value. - Synthesis is local and CPU-only, so nothing is sent to a cloud speech service and the skill works unattended in a headless container as well as on a laptop. The bundled script bootstraps pocket-tts into a cached virtualenv on first use, selecting a Python in the supported >=3.10,<3.15 range rather than assuming python3 qualifies, and failing with actionable guidance when none is available. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1a70aa08-b622-4825-ad63-5a12370add1f --- docs/README.skills.md | 1 + skills/speak-summary/SKILL.md | 138 +++++++++++++++++++++++++ skills/speak-summary/scripts/tts.sh | 155 ++++++++++++++++++++++++++++ 3 files changed, 294 insertions(+) create mode 100644 skills/speak-summary/SKILL.md create mode 100755 skills/speak-summary/scripts/tts.sh diff --git a/docs/README.skills.md b/docs/README.skills.md index ba1bcafd..41ee218b 100644 --- a/docs/README.skills.md +++ b/docs/README.skills.md @@ -391,6 +391,7 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-skills) for guidelines on how to | [signal-write](../skills/signal-write/SKILL.md)
`gh skills install github/awesome-copilot signal-write` | Emit structured agent signals — hands-up, blocked, done, checkpoint, partnership. Signals are written as JSON to .signals/ for dashboard consumption and noted in the journal for persistence. | None | | [slang-shader-engineer](../skills/slang-shader-engineer/SKILL.md)
`gh skills install github/awesome-copilot slang-shader-engineer` | Use when working with Slang shaders, shader modules, HLSL-compatible GPU code, graphics pipelines, compute shaders, tessellation, ray tracing, parameter blocks, generics, interfaces, capabilities, cross-compilation, shader optimization, shader review, or C++ engine integration for Slang. Trigger on any mention of Slang, .slang files, slangc, SPIR-V from Slang, Slang modules, [shader("compute")], [shader("vertex")], or requests to write/review/refactor shader code with modern language features. Also trigger for Slang-to-HLSL/GLSL/Metal/CUDA cross-compile questions, or when the user says "shader" alongside "generics", "interfaces", "parameter blocks", "autodiff", or "capabilities". | `references/language-reference.md`
`references/rules-and-patterns.md`
`references/slang-documentation-full.md` | | [snowflake-semanticview](../skills/snowflake-semanticview/SKILL.md)
`gh skills install github/awesome-copilot snowflake-semanticview` | Create, alter, and validate Snowflake semantic views using Snowflake CLI (snow). Use when asked to build or troubleshoot semantic views/semantic layer definitions with CREATE/ALTER SEMANTIC VIEW, to validate semantic-view DDL against Snowflake via CLI, or to guide Snowflake CLI installation and connection setup. | None | +| [speak-summary](../skills/speak-summary/SKILL.md)
`gh skills install github/awesome-copilot speak-summary` | Convert text, markdown, or a summary produced by another skill into a listenable MP3 using local CPU-only neural text-to-speech. Rewrites written prose for the ear before synthesising. Use when the user asks to "read this out", "turn this into audio", "make an MP3", "I want to listen to this", "podcast version", or wants a spoken digest for a commute or breakfast. | `scripts/tts.sh` | | [sponsor-finder](../skills/sponsor-finder/SKILL.md)
`gh skills install github/awesome-copilot sponsor-finder` | Find which of a GitHub repository's dependencies are sponsorable via GitHub Sponsors. Uses deps.dev API for dependency resolution across npm, PyPI, Cargo, Go, RubyGems, Maven, and NuGet. Checks npm funding metadata, FUNDING.yml files, and web search. Verifies every link. Shows direct and transitive dependencies with OSSF Scorecard health data. Invoke with /sponsor followed by a GitHub owner/repo (e.g. "/sponsor expressjs/express"). | None | | [spring-boot-testing](../skills/spring-boot-testing/SKILL.md)
`gh skills install github/awesome-copilot spring-boot-testing` | Expert Spring Boot 4 testing specialist that selects the best Spring Boot testing techniques for your situation with Junit 6 and AssertJ. | `references/assertj-basics.md`
`references/assertj-collections.md`
`references/context-caching.md`
`references/datajpatest.md`
`references/instancio.md`
`references/mockitobean.md`
`references/mockmvc-classic.md`
`references/mockmvc-tester.md`
`references/restclienttest.md`
`references/resttestclient.md`
`references/sb4-migration.md`
`references/test-slices-overview.md`
`references/testcontainers-jdbc.md`
`references/webmvctest.md` | | [sql-code-review](../skills/sql-code-review/SKILL.md)
`gh skills install github/awesome-copilot sql-code-review` | Universal SQL code review assistant that performs comprehensive security, maintainability, and code quality analysis across all SQL databases (MySQL, PostgreSQL, SQL Server, Oracle). Focuses on SQL injection prevention, access control, code standards, and anti-pattern detection. Complements SQL optimization prompt for complete development coverage. | None | diff --git a/skills/speak-summary/SKILL.md b/skills/speak-summary/SKILL.md new file mode 100644 index 00000000..1abfc106 --- /dev/null +++ b/skills/speak-summary/SKILL.md @@ -0,0 +1,138 @@ +--- +name: speak-summary +description: 'Convert text, markdown, or a summary produced by another skill into a listenable MP3 using local CPU-only neural text-to-speech. Rewrites written prose for the ear before synthesising. Use when the user asks to "read this out", "turn this into audio", "make an MP3", "I want to listen to this", "podcast version", or wants a spoken digest for a commute or breakfast.' +--- + +# Speak Summary + +Turn written text into audio someone will actually want to listen to. + +This skill is deliberately a **terminal step in a chain**. Another skill (or you) +produces the text; this one makes it listenable. It pairs naturally with +`roundup`, `daily-prep`, `meeting-minutes`, or any summarisation work. + +Everything runs locally on CPU. No text is sent to a cloud speech service, which +matters when the content is confidential, and it means the skill works in a +headless cloud agent or CI container just as well as on a laptop. + +## Prerequisites + +The synthesis engine is [Kyutai `pocket-tts`](https://github.com/kyutai-labs/pocket-tts), +a small neural TTS model designed to run on CPUs. + +The bundled script installs it automatically into a cached virtualenv on first +use, so usually you need do nothing. To install it explicitly: + +```bash +pip install pocket-tts # any platform +brew install pocket-tts # macOS, if preferred +``` + +`pocket-tts` requires **Python >=3.10 and <3.15**. The script searches for a +compatible interpreter rather than assuming `python3` is one — worth knowing if +you are on a very new Python, where installation would otherwise fail. + +You also need an encoder. `ffmpeg` is strongly preferred (`brew install ffmpeg` +or `apt-get install -y ffmpeg`); on macOS the script falls back to the built-in +`afconvert` and emits `.m4a` instead of `.mp3`. + +The first run downloads the model (~1GB) from Hugging Face. After that it is +fully offline and synthesises roughly 6x faster than real-time. + +## The important step: rewrite for the ear + +**Do not feed written text straight into the synthesiser.** Prose that reads well +on screen is tiring to listen to. Rewriting it first is what separates a useful +audio digest from an unlistenable one. + +Produce a spoken script that: + +- **Opens with orientation.** What this is, what it covers, roughly how long it runs. +- **Replaces bullets with connective prose.** "First… The bigger one is… Finally…" — a listener has no visual structure to lean on, so carry it in the language. +- **Expands abbreviations on first use.** "PR" becomes "pull request", "CI" becomes "continuous integration". Acronyms that read fine are noise when spoken. +- **Speaks dates and numbers naturally.** "the twentieth of August", not "2026-08-20". "About three thousand", not "2,847". +- **Never reads URLs aloud.** Say "linked in the written version" instead. +- **Uses short sentences.** Split anything past roughly 25 words. +- **Signposts transitions.** "Turning to the product side…", "Two things need your attention…". +- **Ends with the actions.** Recap what the listener should do, since that is what they need to retain and they cannot scroll back. +- **Drops anything purely visual.** Tables, code blocks, and diagrams should be summarised in a sentence or omitted, never read out. + +Write this spoken script to its own `.txt` file. Keep the original written +version with its links intact — the audio is a companion to it, not a +replacement. The user will want to click through later. + +## Synthesise + +```bash +./scripts/tts.sh [voice.safetensors] +``` + +The script strips any residual markdown, splits the text on sentence boundaries +into ~600 character chunks (quality degrades on long single inputs), synthesises +each chunk, and concatenates the result into a mono MP3 at 96kbps — small enough +to sync to a phone, good enough for speech. + +Environment overrides: + +| Variable | Purpose | +|---|---| +| `SPEAK_TTS_BIN` | Path to a specific `pocket-tts` binary; skips all auto-detection. | +| `SPEAK_TTS_HOME` | Where to create/find the cached virtualenv. Default `~/.cache/speak-summary/venv`. | + +## Voices + +The default English voice is `alba`. To use a different one, `pocket-tts` +supports voice cloning from a short clean audio sample: + +```bash +pocket-tts export-voice --help +``` + +Pass the resulting `.safetensors` file as the third argument to the script. + +Only clone a voice you have the rights to use. Do not clone a real person's +voice — colleague, customer, or public figure — without their explicit consent. + +## Output + +- Default to `~/Music/Briefings/` unless the user says otherwise; it is easy to point a phone or podcast app at. +- Name files `-.mp3`. +- Report the path, duration, and size. +- Offer to play it: `afplay ` on macOS, `ffplay -nodisp -autoexit ` elsewhere. + +## Length guidance + +Aim for 4–6 minutes for a routine digest, which is roughly 600–900 spoken words +at a natural pace. If the source would run past about 10 minutes, say so and +offer either a tighter edit or a split into multiple files — attention drops off +sharply beyond that for informational audio. + +## Chaining onto other skills + +The natural pattern is *gather → summarise → speak*: + +- `roundup` → `speak-summary` — a spoken version of the status briefing. +- `daily-prep` → `speak-summary` — tomorrow's schedule, listened to tonight. +- `meeting-minutes` → `speak-summary` — catch up on a meeting you missed. + +When invoked as part of a chain, do not re-summarise. The upstream skill owns +what to say; this skill owns how it sounds. Take its output, rewrite it for the +ear, and synthesise. + +To run unattended (a briefing waiting before breakfast), schedule the upstream +skill with a workflow and have it finish by calling this one. + +## Troubleshooting + +**Audio cuts off mid-sentence.** A chunk exceeded the model's comfortable length. +Shorten the sentences in the spoken script. + +**Words mispronounced.** Spell them phonetically in the input — "Kubernetes" as +"koo-ber-net-eez". This is a normal part of preparing a spoken script. + +**First run is slow.** That is the one-off model download. Later runs start in +about a second. + +**`pocket-tts` not found after install.** The virtualenv may be stale, or your +`python3` may be outside the supported 3.10–3.14 range. Delete +`~/.cache/speak-summary/venv` and re-run, or point `SPEAK_TTS_BIN` at a known binary. diff --git a/skills/speak-summary/scripts/tts.sh b/skills/speak-summary/scripts/tts.sh new file mode 100755 index 00000000..d9adf89a --- /dev/null +++ b/skills/speak-summary/scripts/tts.sh @@ -0,0 +1,155 @@ +#!/usr/bin/env bash +# Convert a plain-text briefing into an MP3 using local neural TTS (Kyutai pocket-tts). +# +# Usage: tts.sh [voice.safetensors] +# +# CPU-only and fully offline after the first model download, so it runs the same +# on an Apple Silicon Mac and in a Linux cloud agent container. No text is sent +# to any cloud TTS service. +# +# Resolution order for the engine: +# 1. pocket-tts already on PATH (e.g. `brew install pocket-tts`) +# 2. a cached venv at $SPEAK_TTS_HOME (default ~/.cache/speak-summary/venv) +# 3. create that venv and `pip install pocket-tts` +# Set SPEAK_TTS_BIN to point at a specific pocket-tts binary to skip all this. +set -euo pipefail + +IN="${1:?usage: tts.sh [voice.safetensors]}" +OUT="${2:?usage: tts.sh [voice.safetensors]}" +VOICE="${3:-}" + +TTS_HOME="${SPEAK_TTS_HOME:-$HOME/.cache/speak-summary/venv}" + +# pocket-tts supports Python >=3.10,<3.15. The system python3 is often outside +# that range, so search for a usable interpreter rather than assuming. +find_python() { + for c in python3.14 python3.13 python3.12 python3.11 python3.10 python3; do + p="$(command -v "$c" 2>/dev/null)" || continue + "$p" -c 'import sys; raise SystemExit(0 if (3,10) <= sys.version_info < (3,15) else 1)' 2>/dev/null \ + && { echo "$p"; return 0; } + done + return 1 +} + +resolve_tts() { + if [ -n "${SPEAK_TTS_BIN:-}" ]; then echo "$SPEAK_TTS_BIN"; return; fi + if command -v pocket-tts >/dev/null 2>&1; then command -v pocket-tts; return; fi + if [ -x "$TTS_HOME/bin/pocket-tts" ]; then echo "$TTS_HOME/bin/pocket-tts"; return; fi + + PY="$(find_python)" || { + cat >&2 <<'MSG' +No suitable Python found. pocket-tts requires Python >=3.10 and <3.15. +Install one (e.g. 'brew install python@3.14' or 'apt-get install -y python3.12-venv'), +or install pocket-tts yourself and point SPEAK_TTS_BIN at the binary. +MSG + exit 1 + } + + echo "pocket-tts not found; creating a virtualenv at $TTS_HOME using $PY (one-off, a few minutes)..." >&2 + "$PY" -m venv "$TTS_HOME" >&2 || { echo "Failed to create virtualenv (is the venv module installed?)." >&2; exit 1; } + "$TTS_HOME/bin/pip" install --quiet --upgrade pip >&2 + "$TTS_HOME/bin/pip" install --quiet pocket-tts >&2 || { echo "Failed to install pocket-tts." >&2; exit 1; } + [ -x "$TTS_HOME/bin/pocket-tts" ] || { echo "pocket-tts install completed but the binary is missing." >&2; exit 1; } + echo "$TTS_HOME/bin/pocket-tts" +} + +TTS="$(resolve_tts)" +[ -x "$TTS" ] || { echo "TTS engine not executable: $TTS" >&2; exit 1; } + +# Encoder: prefer ffmpeg; fall back to macOS afconvert (AAC in .m4a) if absent. +ENCODER="" +if command -v ffmpeg >/dev/null 2>&1; then ENCODER="ffmpeg" +elif command -v afconvert >/dev/null 2>&1; then ENCODER="afconvert" +else + echo "Need ffmpeg to encode audio. Install with 'brew install ffmpeg' or 'apt-get install -y ffmpeg'." >&2 + exit 1 +fi + +WORK="$(mktemp -d)" +trap 'rm -rf "$WORK"' EXIT + +# pocket-tts quality degrades on very long inputs, so split into ~600-char chunks +# on sentence boundaries, synthesise each, then concatenate. +python3 - "$IN" "$WORK" <<'PY' +import re, sys, pathlib +src = pathlib.Path(sys.argv[1]).read_text() +work = pathlib.Path(sys.argv[2]) + +# Strip markdown that would otherwise be read aloud as noise. +src = re.sub(r'```.*?```', ' ', src, flags=re.S) # fenced code +src = re.sub(r'\[([^\]]+)\]\([^)]+\)', r'\1', src) # links -> label +src = re.sub(r'^\s*#{1,6}\s*', '', src, flags=re.M) # headings +src = re.sub(r'^\s*[-*+]\s+', '', src, flags=re.M) # bullets +src = re.sub(r'[*_`>|~]+', '', src) # emphasis/code/table pipes +src = re.sub(r'https?://\S+', '', src) # bare URLs +src = re.sub(r'[ \t]+', ' ', src) +src = re.sub(r'\n{2,}', '\n\n', src).strip() + +MAX = 600 +chunks, cur = [], "" +for sent in re.split(r'(?<=[.!?])\s+|\n\n', src): + sent = sent.strip() + if not sent: + continue + # A single sentence longer than MAX is split on commas as a last resort. + while len(sent) > MAX: + cut = sent.rfind(',', 0, MAX) + cut = cut if cut > MAX // 2 else sent.rfind(' ', 0, MAX) + cut = cut if cut > 0 else MAX + if cur: + chunks.append(cur); cur = "" + chunks.append(sent[:cut].strip()) + sent = sent[cut:].strip(' ,') + if len(cur) + len(sent) + 1 > MAX: + if cur: + chunks.append(cur) + cur = sent + else: + cur = f"{cur} {sent}".strip() +if cur: + chunks.append(cur) + +if not chunks: + raise SystemExit("No speakable text found in input.") + +for i, c in enumerate(chunks): + (work / f"chunk_{i:04d}.txt").write_text(c) +PY + +N=0 +for f in "$WORK"/chunk_*.txt; do + IDX="$(basename "$f" .txt)" + ARGS=(generate --quiet --text "$(cat "$f")" --output-path "$WORK/$IDX.wav") + [ -n "$VOICE" ] && ARGS+=(--voice "$VOICE") + N=$((N+1)) + echo " synthesising chunk $N ..." >&2 + "$TTS" "${ARGS[@]}" >/dev/null + echo "file '$WORK/$IDX.wav'" >> "$WORK/list.txt" +done + +mkdir -p "$(dirname "$OUT")" + +if [ "$ENCODER" = "ffmpeg" ]; then + ffmpeg -hide_banner -loglevel error -y -f concat -safe 0 -i "$WORK/list.txt" \ + -c:a libmp3lame -b:a 96k -ar 24000 -ac 1 "$OUT" +else + # afconvert cannot concat, so join the WAVs first, then encode to AAC. + python3 - "$WORK" "$WORK/joined.wav" <<'PY' +import sys, wave, pathlib +work, out = pathlib.Path(sys.argv[1]), sys.argv[2] +parts = sorted(work.glob("chunk_*.wav")) +with wave.open(parts[0], 'rb') as w0: + params = w0.getparams() +with wave.open(out, 'wb') as o: + o.setparams(params) + for p in parts: + with wave.open(str(p), 'rb') as w: + o.writeframes(w.readframes(w.getnframes())) +PY + OUT="${OUT%.mp3}.m4a" + afconvert -f m4af -d aac -b 96000 "$WORK/joined.wav" "$OUT" + echo "Note: ffmpeg not available; wrote AAC (.m4a) instead of MP3." >&2 +fi + +SIZE="$(du -h "$OUT" | cut -f1)" +echo "Wrote $OUT ($SIZE, $N chunks)"