* Add poka-yoke skill: make misuse unrepresentable rather than warned against
Mistake-proofing, applied to code. The skill pushes toward devices that make
a wrong action impossible or self-announcing, rather than instructions asking
someone to avoid it, and refuses to accept a comment or a convention as a fix.
The gap it addresses. Given a design, models readily list what to fix and
rarely state what the fix makes impossible. Measured across 591 blind-graded
runs and six model families, responses named the foreclosed set 42% of the
time unprompted and 81% with this skill applied. Assertions were written
before the runs and graded blind to configuration.
Stated with its limits, because they matter: every run was the first turn of
a fresh session, so this measures the ceiling rather than what survives a
long working session; the comparison was against no methodology rather than a
different one, so it does not establish that this particular method caused
the gain; and the skill costs something measurable, making responses somewhat
worse at spotting the specific defect already on the page while better at
changing the shape that allowed it. All of that is in the skill body rather
than omitted.
Bundled, all self-contained, no network access and no dependencies:
scripts/detect_hazards.py standard-library scanner, 42 pattern rules
across 20 hazard shapes, five languages
references/hazard-catalog.md the taxonomy with device per shape
references/lang-*.md Python, TypeScript, Go and Rust patterns
Verified with npm run skill:validate and npm run build. Raw benchmark runs,
the harness and the assertion checklists are public at
https://github.com/rainmanjam/poka-yoke
* Rename HasTable to HasFrom in the TypeScript typestate example
codespell failed the PR: it reads HasTable as a misspelling of hashtable.
The identifier is a legitimate generic parameter on a typestate builder,
QueryBuilder<HasTable, HasWhere>, so this is a false positive, but it is our
file breaking their gate and the fix costs nothing.
HasFrom is also the better name: the flag tracks whether .from() has been
called, not whether a table exists.
Renamed in the upstream repository too, so the two copies do not diverge on
day one. Verified the whole bundle is codespell-clean locally.
* Correct the effect figures in the poka-yoke skill
The submitted numbers (42% -> 81%) could not be reproduced from the upstream
gradings. Recomputed over the six models the sentence describes: 45% -> 80%
across 132 verdicts. Adds the per-scenario breakdown, which is the more useful
claim: the gains are in tasks where nobody asked for a design review.
6.7 KiB
Python Devices
Python's type hints are optional and unenforced at runtime, which splits every device into two questions: what the checker catches, and what actually holds when the code runs.
Prerequisite: mypy --strict (or pyright in strict mode) as a required CI check.
Without it, annotations are documentation, rung zero. Pair it with ruff at error level.
Contact, NewType for cheap distinctness
from typing import NewType
UserId = NewType("UserId", str)
OrderId = NewType("OrderId", str)
def transfer(src: UserId, dst: UserId) -> None: ...
transfer(order_id, user_id) # mypy: error: zero runtime cost
NewType is free at runtime and stops the mix-up at check time. It does not validate, use it
when the concepts differ but the shape doesn't need checking.
Contact, parse at the boundary with Pydantic
When the value needs checking, parse into a model and let the type carry the proof:
from pydantic import BaseModel, EmailStr, Field, ConfigDict
class CreateUser(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
email: EmailStr
age: int = Field(ge=0, le=150)
Two settings do most of the work. extra="forbid" turns a typo'd field into an error instead
of a silently ignored key: the difference between a 400 and a user whose preference never
saved. frozen=True blocks reassignment of the model's fields, so nothing downstream can
quietly replace what you verified. It is shallow, though: a list or dict field is still
mutable in place, so reach for tuple, frozenset, or a nested frozen model where that
matters.
Apply at every edge: request bodies, queue messages, third-party responses, file loads.
Contact, keyword-only arguments
Python's answer to swapped parameters, and it costs one character:
def transfer(*, source: AccountId, dest: AccountId, amount: Money) -> None: ...
transfer(source=a, dest=b, amount=m) # the only legal form
transfer(a, b, m) # TypeError
Force keyword-only for anything with more than two parameters, and always when two share a type. This is Warning-rung. It makes the mistake visible rather than impossible, but it is the highest-value one-character change in the language.
Fixed-value, exhaustiveness
from typing import assert_never, Literal
Status = Literal["pending", "active", "closed"]
def label(s: Status) -> str:
match s:
case "pending": return "Pending"
case "active": return "Active"
case "closed": return "Closed"
case _: assert_never(s) # mypy errors here if a variant is unhandled
assert_never turns "someone added a status" into a build failure at every site that must
change. Works with Literal, Enum, and tagged dataclass unions.
Fixed-value, config validated at startup
from pydantic_settings import BaseSettings
class Settings(BaseSettings):
database_url: str
stripe_key: str
region: str # no default: an unset value should stop the deploy
settings = Settings() # raises at import, before the service reports healthy
Import this once at startup and pass the object down. Every os.getenv buried in a handler is
a 3am page waiting for the one request that reaches it.
Contact, immutable value objects
from dataclasses import dataclass
@dataclass(frozen=True, slots=True, kw_only=True)
class Money:
cents: int
currency: str
def __add__(self, other: "Money") -> "Money":
if self.currency != other.currency:
raise ValueError(f"cannot add {self.currency} to {other.currency}")
return Money(cents=self.cents + other.cents, currency=self.currency)
frozen=True prevents mutation after validation, slots=True makes a typo'd attribute
assignment an AttributeError rather than a silently-created new attribute, and kw_only=True
kills positional swaps. Three flags, three hazard classes closed.
Motion-step, context managers
Any acquire/release pair belongs in a context manager. Never expose open()/close() as
separate public methods: the error path will leak, and only under load.
from contextlib import contextmanager
@contextmanager
def transaction(conn):
tx = conn.begin()
try:
yield tx
tx.commit()
except Exception:
tx.rollback()
raise # re-raise: swallowing here would be X1
Python-specific traps worth checking every time
- Mutable default arguments:
def f(items=[])shares one list across every call. UseNoneand construct inside. Caught by ruffB006. - Bare
except:catchesKeyboardInterruptandSystemExittoo. Caught byE722. assertfor validation is stripped underpython -O. Never use it for anything security- or correctness-critical; raise instead.- Naive
datetime.now(): usedatetime.now(timezone.utc), and inject a clock so time is testable. Caught by ruffDTZ. - Float money: use
intcents ordecimal.Decimal, neverfloat. ==vsison strings and ints works by accident via interning and breaks in production on longer values. Caught byF632.asyncio.create_taskwithout keeping a reference: the task can be garbage collected mid-flight, so the work silently doesn't happen. Caught by ruffRUF006.
Ruff rule sets that are poka-yoke
Style rules aren't mistake-proofing; these are, which is why E appears only as its
bug-shaped subsets and not whole. Select at error level:
[tool.ruff.lint]
select = [
"F", # pyflakes: undefined names, unused imports
"E4", "E7", "E9", # pycodestyle's bug-shaped rules: bare except, `== None`, syntax errors
"B", # bugbear: mutable defaults, loop variable capture, assert-on-tuple
"S", # bandit: hardcoded secrets, unsafe subprocess, weak crypto
"DTZ", # naive datetimes
"ASYNC", # blocking calls inside async functions
"RUF006", # dangling asyncio tasks
"PLE", # pylint errors: genuine bugs only
"T20", # stray print/pprint
]
Known limits
- Annotations are not enforced at runtime. Anything crossing a boundary, or reachable from unchecked code, needs a real runtime parse. Pydantic is how you get Control; mypy alone gives you Control only over code mypy actually checks.
Anyis contagious and an untyped dependency reintroduces it silently. Setdisallow_any_unimportedandwarn_return_any; audit# type: ignorecomments and require a reason on each.- No affine types, so use-after-close isn't preventable; context managers are the answer.
- Monkey-patching means no encapsulation is absolute. Push invariants that truly must hold into the database rather than into a class.