Information Density and Semantic Determinacy - Formal vs Natural Language

Narrative

Independent writers engaging this topic mostly work downstream of the academic literature rather than generating new measurements, but several add genuine analytical value by testing claims against practice. Simon Willison's 2025 posts on vibe coding and vibe engineering are the clearest case: he distinguishes prompts accepted on trust from prompts whose output is tested and reviewed, which amounts to an informal statement of the brief's central question, how much formal residue has to sit alongside a natural-language prompt before behaviour is pinned down. Hillel Wayne's blog, written from inside the formal-methods community, makes the same point from the other direction: his 2019 piece on why people do not use formal methods argues the barrier is pedagogical rather than technical, and his TLA+ business-case post catalogues where formal specification has and has not paid off industrially.

On the ambiguity-as-optimum thread, Sean Trott's research-review blog and languagehat.com both engage directly with Coupé et al.'s 39 bits per second finding, explaining the syllable-rate methodology rather than just repeating the headline number, while Guy Wilson's Abstract Nonsense post raises the question of what 'efficiency' means once speech rate and information density are multiplied together. None of these sources overturns the finding, but they show independent writers doing the work of checking method rather than passing along the popular summary uncritically. LessWrong contributes similarly close readings: one 2015 post tests the ambiguity-is-optimal claim against constructed languages like Lojban, and a 2022 post revisits Shannon's original human-prediction entropy estimates for English, asking what exactly the 0.6 to 1.3 bits-per-character range is holding constant.

On the anti-natural-language-programming tradition, the clearest engagement is with Naur rather than Dijkstra: three independent blogs (codeutopia.net, cekrem.github.io, adropincalm.com) restate and extend Programming as Theory Building, with the most recent, from 2026, explicitly applying Naur's artefact-versus-theory distinction to code substantially written by language models. Dijkstra's EWD667 is itself treated as a primary text rather than commentary, and hxa.name's 2010 rebuttal is one of the few sustained independent counter-arguments found. On notation, Hillel Wayne's post on the J language and Ruben Verg's blog both push back on Iverson's 1979 claim that terse notation is a cognitive tool, arguing that context-dependence and terseness cut against comprehension rather than for it.

A minority of blogs repeat the folk claim that natural language carries less information per unit than code or formal notation, framing this as low density rather than low determinacy. Where sources in this lane make that move (for example some of the prompt-as-code Substack writing), it inverts what Hindle et al. actually measured, since their n-gram entropy estimates put source code at roughly 3 to 4 bits, lower than typical estimates for English. The more careful writers, Trott, Wayne, and the LessWrong entropy posts, keep the two properties, predictability and determinacy, analytically separate, which is the distinction the brief asks this sweep to preserve.