Review: Ash strengths and weaknesses

Review: ash-critique (notes/research/05-ash-strengths-weaknesses.md)

Review of: Ash strengths and weaknesses.

VERDICT: ACCEPT

(Round 1 verdict was REJECT. See ## Round 2 re-verification at the end.)

Reviewer: independent fact-check, 2026-10-01. I fetched every source below myself: HN through the Algolia items API, ElixirForum through the Discourse JSON at forum.elixirforum.com/t/<id>.json (the elixirforum.com host returns 406, the forum. host works) and r.jina.ai, GitHub through gh api, and blogs through r.jina.ai. I checked local docs against scratch/ash-src/ash v3.33.11.

Why REJECT rather than ACCEPT-WITH-FIXES. Most quotes are worded correctly. But the document’s load-bearing finding (criterion 7) misreads its sources, and it misses direct evidence against its own conclusion: a 2026 benchmark of declarative frameworks, and a 2026 report that Opus 4.6 and Codex 5.3 struggle with Ash. Several other problems add to that:

• it uses a source that does not exist (a “résumé” at HN 41649763);
• it gives a commenter an identity he does not have (skybrian as “Brian Harry, Microsoft”);
• it puts another person’s words in the maintainer’s mouth (spiderice’s “alternative to Ecto”);
• it states a 3.0 migration fact backwards (public?), and one of its mesh design lessons depends on that;
• it gets the bus-factor statistic wrong (96% vs the real 79%);
• it never mentions the strongest “do not use Ash” voice (egeersoz), who appears in the very thread it says it could not finish.

Sections §6, §8 and §4 and the Summary need rework, not touch-ups.

---

1. Error table

---

2. Quote sample statistics

I checked 85 quotes: 72 individual quotes plus 13 “stated reason” rows of the migration table. The sample covers every section the brief named. For the verdict see the per-item notes in §1.

On top of these, about 25 quotes from “My thoughts on Ash” carry a systematically wrong date (2025-03-22 instead of 2025-03-07/08), and most of their post-number URLs are off by one.

---

3. Two-independent-sources check

---

4. Cause classification check

• W1 [DO]+[IR]: reasonable, but delete the invented “caslu’s fixes were configuration changes”. egeersoz (below) blames concept overload (calculations vs aggregates vs preparations vs policies), which supports a larger [IR] share.
• W2 [IR]+[EM]: labelled “no inference needed”, but it is inference, and arguably [IR] (see §1).
• W3 [IR]+[EC]: fine. The “ecosystem is no longer one person” mitigation should use the corrected 79% figure.
• W4 [EM]: “by definition an artefact of… compile-time Elixir macros rather than data parsed at runtime” is inference. Ash’s docs say the resource is data read by code (design-principles.md:19). Label it as inference.
• W6 [EM]+[IR]: the [EM] part has no source (see §1).
• W8 [EM]+[EC]: the evidence is real and better than the document shows (see check 9 below). sezaru’s numbers point squarely at code-interface macro expansion, which supports [EM]. Zach: “the code interface logic is not really the greatest macro code anyone ever wrote”.

---

5. Criteria table

---

6. Check 5: Ash and LLM coding agents

What survives from the researcher’s finding:

• The working-with-llms.md warning is real and quoted verbatim. Its meaning is “don’t bring unexplained LLM code to support”, not “the DSL attracts hallucinations”. The same doc says “It is also quite debatable whether it is a good idea to use them at all” (:9).
• The maintainer does attribute the big improvement to usage rules. The researcher missed the strongest source for this: Zach, LLMs & Elixir: Windfall or Deathblow?, https://www.zachdaniel.dev/p/llms-and-elixir-windfall-or-deathblow (2025-06-01): “We’ve done this [usage-rules.md] for the main Ash packages, and the transformation is remarkable. We went from LLM agents being practically useless for Ash development to being able to generate idiomatic, production-ready code.” The same post says: “a huge increase in fully off the wall and weird questions driven by LLM hallucinations”.
• There is no Ash-specific benchmark. Zach says so himself in the same post: “I see plenty of python stuff there, but not no Elixir”. In 70980 he describes his evidence as “My personal and subjective experimation shows a night and day difference”.

What does not survive. The “rather than declarativeness” framing. The maintainer side also claims the declarative, compile-checked structure helps agents:

• zachdaniel, “I concur”, replying to ghannam80’s “its declarative model makes it perfect to generate high quality code using LLM” (https://elixirforum.com/t/ash-framework-official-llm-development-tooling-and-guidance/70980, 2025-05).
• Alembic, 2025-05-15 (URL above): “it still provides a ton of guardrails for agents… So much of Ash is verified at compile time, and provides explanatory errors”.

Evidence the researcher missed: against.

1. A benchmark of declarative frameworks. Ahmed & Deshpande, Declarative by Design, Assistable Only by Convention: Benchmarking Multi-Agent Frameworks for AI-Assistability, arXiv 2602.11198 (submitted 2026-02-03, revised 2026-07-13), https://arxiv.org/abs/2602.11198. Abstract, verbatim: “Our results challenge the intuition that declarative framework design guarantees AI-assistability: Agno, with a single canonical pattern and convention-aligned API, achieves the highest AI score (0.55), while DSPy – the most declarative framework by design – scores lowest (0.07), as its novel abstractions are insufficiently represented in AI training data. We find that convention alignment, not declarative design alone, is the primary driver of AI-assistability”. It covers Python agent frameworks, not Ash, but it is the only controlled evidence on the mesh premise I found, and the report’s “no benchmark exists either way” is false at the general level.
2. egeersoz, 2026-03-15, https://elixirforum.com/t/ash-with-ai-split-thread/74717 post 2: “I’ve found that even Opus 4.6 and Codex 5.3 struggle with Ash. For some reason they can’t figure out the ‘Ash way’ of doing standard operations and always overcomplicate things. Stuff like ‘pull the entire dataset from db and sort in memory’ are anti patterns they constantly fall for with Ash. With Ecto they don’t. Maybe because Ecto is so similar to SQL.” Post 7 (2026-03-17): “over the past year of working with AI, it has never ever gotten moderately complex expressions right”.
3. llamex, https://github.com/dmitriid/llamex (created 2026-08-11), is “Credo Plugin that detects issues that LLM-assisted Elixir refactors commonly introduce”. Its checks include NoAdHocAshQueries, NoDBWorkInMemory and NoAuthorizeBypass. dmitriid is the HN user troupo (profile links dmitriid.com), the same person the document cites twice as LLM “support”. His later tooling targets Ash-specific LLM failure modes, and it matches egeersoz’s in-memory anti-pattern.
4. A Llama 3.2 hallucination about Ash, with Zach’s “Wildly incorrect”, at HN 41649763 (2024-09-25). This is the item the document misdescribes as a résumé.
5. olivermt, 69829 post 28 (2025-03-16): “An LLM by its very nature has unstable output… Ash main feature is to simply remove a whole class of code you don’t have ownership of any more. An LLM will bring tons more of code under your ownership”.
6. garrison, 70980 (2025-05): the CaMeL prompt-injection paper “elected to create a DSL with Python syntax… just to improve the results”. This bears directly on MX’s novel syntax.
7. ash_ai/README.md:72 (v1.1.1): “We are still experimenting to see what tools (if any) are useful while developing with agents.”

Evidence the researcher missed: for (all anecdotal).

• AndyL (74717 post 3, 2026-03-15): “I’ve seen good results with Claude, Ash and usage_rules.”
• egze (post 11, 2026-03-21): “My experience with Ash and AI (Claude Code) has been great. usage_rules is nice… paired together with Tidewave makes it even better.”
• mau013 (post 10, 2026-03-21): “my experience with Claude has been rather good.”
• redrapids (69829 post 24, 2025-03-14): “coding agents are going to need better structure.”
• A language-level claim that Elixir tops Tencent’s AutoCodeBench at 97.5% comes only from a secondary blog (https://codemyspec.com/blog/why-elixir-is-the-best-language-for-llms). [unverified], since I did not reach the primary paper. Even if true, it concerns the language, not Ash.

Correct summary for the document. No Ash benchmark exists. The one controlled study of declarative frameworks (arXiv 2602.11198) finds declarativeness alone does not help, and that convention alignment with training data does. Practitioner reports split: positive reports almost always credit usage_rules and Tidewave, and the strongest negative report (2026, frontier models) describes Ash-specific anti-patterns. The maintainers claim both levers (fresh context, and compile-time guardrails) but show no measurements. Mesh adds no training data, so it starts further off-distribution than Ash. The researcher could note in the analysis section that generated TypeScript in a common style is the “convention-aligned” surface.

---

7. Check 6: the 23 unread replies in “My thoughts on Ash”

Source: https://forum.elixirforum.com/t/69829.json, 34 posts, 2025-03-07 to 2026-08-20. Posts 12-34 add the following:

• egeersoz, post 18 (2025-03-08). This is the strongest negative case anywhere in the record, and the document has none of it. He has 8 years of Elixir and 4 months of daily Ash. Quotes:
  • “Often, it feels like a straitjacket: you don’t fully realize how it limits you until you attempt something ordinary and run into cryptic errors. It’s also unclear when and how you’re supposed to deviate from ‘the Ash way,’ which isn’t well documented.”
  • “once I built a new feature with it, I realized it drastically reduced both my productivity and my enjoyment of programming. I’m not the only team member who feels this way.”
  • Macro count: Phoenix has “only 19 defmacro statements, whereas Ash—around the same size—has 62”.
  • Docs “often feels auto-generated”; support is split across Slack and Discord.
  • His follow-up a year later (74717 posts 2 and 7, 2026-03) reports that the team ripped Ash out after 2 years. It is a long, specific list: errors with missing stack traces; concept overload across calculations, aggregates, preparations and policies; expressions that “look like Elixir but don’t behave like Elixir” (SQL NULL semantics); AshJsonApi gaps; large migration snapshot JSON files bloating PRs; no DB trigger support; “every layer of the framework feels somewhere between 50% and 80% finished, docs are maybe 30% there”. His bottom line: “it should not be the default choice for any team”.
  • Zach’s reply (post 19): “would hardly call docs an ‘afterthought’ but there is room for significant improvement 100%”. sevenseacat (post 20): “It’s a work in progress.”
• zachdaniel, post 21: the new igniter installers “generate the actions directly into your app which makes it much easier to understand what Ash is doing, and also to see what kinds of things you might want to test”. This is the maintainer’s answer to the testing complaint, and it supports mesh’s “emit visible code” idea.
• caslu, posts 13 and 16: partly walks back his complaint (“perhaps an overstatement on my part”; the book “will make myself regret my decision to give up Ash”).
• Stefano1990, posts 15 and 17: “sounds like a skill issue”. Depending on DRF carries the same maintainer risk.
• mudspot, post 22: early adopter. “Our productivity were further increased by another factor of 2 with the adoption of Ash”; “Ash’s implementation of Policies is gold!”
• dewetblomerus, post 23: a learning-curve blocker, but complex logic went into “a normal Elixir module with good old tests”. This is an example of an escape hatch.
• redrapids (24), ghannam80 (27) and olivermt (28): the LLM exchange quoted in check 5.
• sodapopcan, post 29, and binarypaladin, post 32 (2025-11-05): the “bespoke in-house framework” argument, which is the most common pro argument and is missing from the document. binarypaladin, a year in production: “Yeah, maybe there is a risk of @zachdaniel vanishes one day, but if you’re running a bespoke framework, you’re already at risk for that with fewer safety nets.” He updates weekly and adopted Ash incrementally. This is a direct counter to W3.
• FedericoAlcantara, post 34 (2026-08-20): early modules “were refactored several times because I made a mixture of custom functions and Ash having validations in both places”.

---

8. Check 8: important omissions

1. The egeersoz posts (above). The fairness section is not fair without them.
2. “Reasons not to use Ash (split thread)”, https://elixirforum.com/t/reasons-not-to-use-ash-split-thread/69802 (2025-03-06): acrolink: “I prefer to avoid adding another layers of tools over existing working ones”. Zach’s reply argues for the integrated stack.
3. The forms pain point in the migration blog the document already cites: “This was the only real headache I had in the whole process… I had a hell of a time getting this working… over the course of a few days and 20+ messages… a working (but hacky) solution”. Its conclusion is also left out: “For an app this simple, Ash might not provide any game-changing benefits”.
4. Silent per-record work in module calculations: egeersoz (74717 post 7) and dimitarvp (post 9): “our DB queries seemed to have been pulling too much. Implicit behavior is met with ‘nope’ from me.”
5. #1792’s actual content is a data leak: pubsub notifications carry “the full actor resource” to any subscriber. The table reduces it to “a Proposals(4.0) discipline”.
6. Spark’s ElixirSense plugin, which is the real W10 evidence (§3).
7. The Brett Kolodny (MEW) testimonial at what-is-ash.md:123-125 is omitted from §7’s list of docs testimonials (minor).

---

9. Check 9: performance numbers (“unquantified” is false)

Compile time

• sezaru, “Using code_interface makes resource slow to compile”, https://elixirforum.com/t/using-code-interface-makes-resource-slow-to-compile/73196 (2025-11-05):
  • one resource: 11260ms with code_interface, 2270ms without;
  • its domain: 6397ms with, 285ms without;
  • public repro (github.com/sezaru/test_slow_code_interface): 4148 vs 533 ms (resource), 3719 vs 368 ms (domain).
  • zachdaniel: “Wow, that is pretty damn hefty!… the code interface logic is not really the greatest macro code anyone ever wrote”.
• sezaru, “Strategies to make resources compile faster”, https://elixirforum.com/t/strategies-to-make-resources-compile-faster/72265 (2025-08-28): “some of them take more than 30 seconds”. Zach’s advice: “remove as many anonymous functions from your resources as possible”. nocsi published ash_profiler for this.
• ash#2267, “Initiative: Reduce compilation time” (chazwatkins, 2025-08-09, open), https://github.com/ash-project/ash/issues/2267: “Ash currently has 187 compilation cycles, which causes slower compilation time and most of the project to recompile for even small code changes”. Down to 185 after PR #2266.
• ash#2798 (2026-07-23): “several minutes for just 1 file”, closed as fixed in Elixir main, so it was an Elixir compiler bug.
• Compile deadlocks continue in 2026: ash#2670 “Compilation deadlock in policy checks due to new init/1 callback in 3.23.1”, and ash#2685.
• Still not found: a whole-project comparison of Ash against plain Phoenix + Ecto compile time.

Run-time overhead

• ash#2921, “Allow compiling out Ash’s telemetry instrumentation” (lardcanoe, 2026-09-09), https://github.com/ash-project/ash/issues/2921:
  • “Our suite makes 3,853,289 Ash telemetry spans per full run. Stripping this out shaved off ~14s per run, making it about 3% faster.”
  • On a bare ETS resource, removing it “saves 15–23% of Ash.Changeset.for_create/4 and 10–21% of Ash.create!/1”.
  • Ash is the only library in their dependency tree calling :telemetry.list_handlers on every span.
  • zachdaniel: “Open to it 👍”.
  • The numbers were produced with Claude’s help, as the issue says. Treat them as one team’s measurement.
• Still not found: an Ash vs raw Ecto query benchmark. Zach’s gist “Comparing usage of Ash & Ecto” (https://gist.github.com/zachdaniel/79042f1cc546535e495d7e599ca9f21b, 2024-04-28) compares code, not speed.
• A search-engine snippet attributes “there is additional overhead… just optimizations waiting to be made” to https://elixirforum.com/t/ash-framework-a-declarative-resource-oriented-application-development-framework-for-elixir/51119 (2022). I could not find it on page 1. Unconfirmed; do not cite it.

---

10. Fix list for the researcher (in priority order)

1. Rewrite §6, Summary 10, and implication 4 as in check 5. Add arXiv 2602.11198, egeersoz 2026, llamex, Zach’s 2025-06-01 post, “I concur”, and the Alembic guardrails claim. Re-label troupo 2025 and delete “hallucination magnet”.
2. §8 and §7: add egeersoz (2025 and 2026), binarypaladin, mudspot, and caslu’s hedges and walk-back. Replace the fake résumé item.
3. Fix the attributions: spiderice (S8, §8), DylanSp (W5), Woodall (S2), skybrian (delete the identity), and the thread and date for sodapopcan and caslu. Re-date every 69829 quote and fix the post-number URLs.
4. §4: fix the public? row and lesson 1, replace the invented reasons with the guide’s text, delete lesson 2, and add Ash.Flow.
5. W3 and implication 6: 79%, not 96%.
6. W6: replace the blog cite. W2: label the inference. W1: delete “caslu’s fixes…”. W8 and Open questions 2-3: add the numbers from check 9.
7. Bring S2, S4, S5, S7, S8, S9 and S10 up to two independent sources, or merge or drop them.
8. Move mesh analysis out of W4 and W10. State the usage_rules and ash_ai versions. Trim to about 6,000 words.

---

Round 2 re-verification

Document checked: revised 05-ash-strengths-weaknesses.md, 897 lines and 10,572 words, which ends with ## Revision log (round 2).

Method. I extracted every double-quoted string in the document (172) with a script. Each one was matched, after normalising quote marks and whitespace, against the full text of the cited sources:

• Discourse JSON of forum threads 69829 (34 posts), 70080, 69802 and 74717;
• r.jina.ai page captures of 70980 (pages 1 and 2), 73196 and 72265;
• HN Algolia item trees;
• GitHub issue bodies and comments for #2267, #2397, #2670, #2798, #2921 and #1792;
• the two Zach Daniel blog posts, the Alembic post and Matt’s blog;
• the arXiv HTML of 2602.11198;
• the local docs.

I then compared, by hand, each match’s post number and author with the attribution the document gives. One limit: the forum JSON endpoint was rate-limited during this pass, so the 70980 posts’ exact dates (all given as 2025-05-22) and 73196 post 6’s date (2025-11-15) were not confirmed post by post. Both are consistent with the page context: the “Claude 4 just came out” post matches Claude 4’s release day.

Verdict: ACCEPT-WITH-FIXES. The round-1 problems that were load-bearing are fixed against the source:

• the public? inversion;
• the invented migration reasons;
• 79% rather than 96%;
• the misattributions to spiderice, DylanSp and Woodall;
• the fabricated résumé and the fabricated skybrian identity;
• the misreading of the docs’ hallucination warning;
• the egeersoz omission;
• the re-dating of the 69829 thread;
• the performance numbers.

What remains is a set of specific attribution and URL errors, one misclassified LLM item, an overstated Summary bullet about the arXiv study, a false claim in the revision log about W9, and some verified content that was lost in the rewrite.

R2-A. Round-1 findings: confirmed fixed against the source

R2-B. Still wrong: fix each as stated

R2-C. Verified content lost in the rewrite (not mentioned in the revision log)

The rewrite silently dropped material that round 1 had right and I had confirmed. Restore at least the LLM items, since criterion 7 is supposed to list all evidence:

• Criterion 7, “for”:
  • dartos, HN 41896590, 2024-10-20: “I recently picked up the Ash framework for elixir and it does all that too, but in a declarative, precise language which codegens the implementation in a deterministic way.”
  • stray, HN 45431182, 2025-09-30: “Elixir with Ash Framework, backed by PostgreSQL. Claude Code did just fine – but this was back in June before Claude got nerfed.”
• Criterion 7, “against”:
  • troupo, HN 37380622, 2023-09-04: “Currently working on an Elixir project with Ash Framework, and I wouldn’t trust a single Chat GPT output for either of them.”
• Criterion 6: the slogan the brief names explicitly, “Model your domain, derive the rest”, is no longer in §5. It appears at what-is-ash.md:112 and as the ash-hq.org headline. caslu also uses it in 69829 post 1. Restore it in §5.
• S3 / W4:
  • troupo, HN 37632243, 2023-09-24: “yes, there are escape hatches everywhere, and a lot of magic is quite thin… I just drop directly into Elixir and/or Ecto”. This is a supporter’s voice that was removed.
  • The same post’s report of Zach’s ElixirConf DX goal.
• W5: the 2023 HN reports that readers could not tell Ash was for Elixir (avarun 37637169, kkarpkkarp 37630792, bradrn 37630837). W5 now rests on egeersoz plus a single 2023 HN pair.
• W2: ash#791 (2023-12-04, “cryptic error messages (and compile deadlocks if using extensions)”) was dropped. W2 now has no GitHub evidence. ash#2274 (“Using exists() in policy with syntax error produces very obtuse stack trace”, 2025-08-15) and #2392 (“cryptic compile error”, 2025-10-16) are better replacements.

R2-D. Quote statistics (round 2)

R2-E. Requested checks

• Criterion 7 structure. 6a-6d lay out what each kind of source says (docs, maintainer blog, maintainer guardrail claim, study), followed by two lists: 6e “for” and 6f “against”. No conclusion is drawn inside §6, except one mild synthesis sentence in 6c (“So the maintainer side argues both levers”), which is accurate. Each item states what the source literally says, apart from B6 and B7. Everything I found in round 1 is present. What is missing is the three dropped anecdotes (R2-C) and the arXiv details (B7).
• Two-independent-sources recount.
  • Meet the rule: S1, S2, S3, S4, S6, S7, S8; W1, W2, W3, W4, W5, W6, W7, W8, and W10 (hiring half).
  • Do not meet it: S5 (B10); W9 (B8, unlabelled); the W10 editor half (labelled).
  • Labelled single source: S9 (but see B9) and S10.
• Fairness. The critics are now at full strength: egeersoz in his own words across three posts, caslu with his hedges, acrolink, napsterbr, Dmk and Matt’s forms headache. The supporters’ case is intact for adoption (binarypaladin, sodapopcan, mudspot, ken-kost, Stefano1990), but it is weaker on LLMs and escape hatches because dartos, stray and troupo 2023 were dropped (R2-C). B4 hands a supporter’s line to the wrong person.
• Completeness after the truncation. All 9 criteria sections are present (§1-§8 plus the LLM section), along with Implications, Open questions, Sources and the Revision log. None is visibly cut off mid-sentence. The losses listed in R2-C read as content that did not survive the rewrite.
• Criteria status. 1, 2, 3, 5, 7, 8 and 9: complete once B1-B17 are applied. 4: complete once B12 and B13 are fixed. 6: partial until the “Model your domain, derive the rest” slogan is restored.
• Length. 10,572 words against a 6,000 ceiling. I accept the researcher’s flag; the lead decides.

---

Round 3 final check

Document checked: round 3 of 05-ash-strengths-weaknesses.md, 1,022 lines and 11,991 words, with truthful Round 2 and Round 3 revision logs.

Method. Same as round 2:

• The script extracted all 186 double-quoted strings and matched each against the full text of the cited sources: forum JSON or page captures, HN Algolia item trees, GitHub issues and comments, the blogs, the local docs, and the arXiv HTML for 2602.11198.
• A second script compared each match’s author handle and post number with the attribution in the three lines before it. Every mismatch it flagged was a false positive on manual inspection, meaning the attribution sat further up the paragraph.
• The 16 strings the script could not match were checked by hand: the GitHub titles, the arXiv quotes, the llamex repository description and the HN story title.

Final verdict: ACCEPT.

• All 18 round-2 findings are fixed correctly against the source.
• All restored content checks out at its source.
• Criterion 7 is now accurate and keeps evidence and conclusions apart.
• No correct round-2 content was lost.

The residual errors below are minor. Most sit in the labelled analysis section, and none changes a finding. A reader can use the document as is, with the errata below.

R3-A. Round-2 items B1-B18

R3-B. Restored content checked at source

All confirmed:

• dartos, HN 41896590, 2024-10-20: verbatim.
• stray, HN 45431182, 2025-09-30: verbatim.
• troupo, HN 37380622, 2023-09-04: verbatim.
• troupo, HN 37632243, 2023-09-24: escape-hatch text verbatim. The DX sentence is verbatim and correctly labelled as troupo’s summary of the talk.
• The slogan. what-is-ash.md:112 reads “#### Model your domain, derive the rest”. The ash-hq.org page text contains “Model your domain, derive the rest.” caslu in 69829 post 1 wrote: “there’s no better slogan than ‘model your domain, derive the rest’ for describing what Ash is”.
• The three HN readers: kkarpkkarp 37630792, bradrn 37630837 and avarun 37637169, all 2023-09-24, verbatim.
• ash#2274: olivermt, 2025-08-15, open; the title is exact.
• ash#2392: jimsynz, 2025-10-16, closed; the title is exact.

R3-C. Criterion 7

• Lists. Every 6e item is positive or a pro-hypothesis (ken-kost, egze, mau013, AndyL, dimitarvp, mindok, anibal, redrapids, ghannam80, dartos, stray). Every 6f item is negative (egeersoz ×2, troupo 2023, olivermt, garrison, llamex, troupo’s Llama post). dimitarvp sits on the line: positive overall, but the episode itself was “unpleasant”. It is acceptable as placed because the text states both.
• arXiv. The description is exact and not overstated, in 6d and in Summary 8. The document says outright that the study “is not evidence against declarativeness as such” and “says nothing about Ash”.
• No conclusion inside §6. One mild sentence of synthesis remains (6c, “So the maintainer side argues both levers”). It is accurate. Conclusions are confined to analysis point 4.
• Nothing missing. Every LLM item I found in rounds 1 and 2 is present.

R3-D. Two-source labels and cause tags

• S5, S9, S10, W9 and the editor half of W10 are labelled single source, consistently in the headings, the bodies and Open question 5.
• Meeting the rule: S1-S4, S6-S8; W1-W8, and the hiring half of W10.
• Cause tags. W9 is [EM] (inference), defensible. W6 is “[EM]/implementation (inference)”, which is an off-legend category (residual 2).

R3-E. Fresh sample of 30 quotes not checked in rounds 1-2

These are the round-3 additions plus strings not sampled before:

• troupo 37632243 ×2; kkarpkkarp; bradrn; avarun; dartos; stray; troupo 37380622; troupo 41655676 ×2;
• arXiv: “configured with Claude models…”, the Agno sentence, the scope sentence, the title, “functional correctness (pass@1)”;
• Stefano1990 post 17 in full; lardcanoe’s comment;
• the Summary-1 chain; the W4 heading;
• the slogan ×3 (doc, site, caslu);
• the #2274 and #2392 titles; the 74717 thread title;
• “Over the last few months I’ve observed:”;
• Brett Kolodny what-is-ash.md:123-125;
• what-is-ash.md:90; what-is-ash.md:104.

Whole-document scan. 186 quoted strings; 3 are the researcher’s own phrases or revision-log wording and were excluded. That leaves 183 source quotes, all found at the cited source with the cited author. One carries a silent typo correction (residual 8).

R3-F. Did round 3 lose or break correct content?

No. Every round-2 item I had confirmed is still present and unchanged, apart from the deliberate removal of S9’s unquoted Matt line (B9) and the binarypaladin row in §3 (B13). No new attribution errors were introduced. All 9 criteria sections are complete, and none is cut off.

Residual errors (for readers)

Every statement in the document that is still wrong or overstated, with the correct fact:

1. l.797 (analysis 6), “Worse than round 1’s 96%”. Backwards: 79% is a lower concentration than 96%, so the corrected figure is better than round 1 claimed, not worse. Correct fact: zachdaniel has 5,315 of 6,719 human contributions, 79.1% (GitHub contributors API, paginated, 2026-10-01).
2. l.312 (W6 cause), “[EM]/implementation”. “implementation” is not one of the document’s four cause categories (l.191-192), and point-release regressions (#2397 UUID decode, #2670 policy-check compile deadlock) are not shown to be macro-caused. Read it as “implementation regressions; cause category not established”, as an inference.
3. l.411-413 (§3), “Pattern (fact): the asks cluster on … safety defaults”. No ask in the table concerns safety defaults. The asks are: internals docs (caslu post 9, swrenn post 14), error messages, DB triggers, expression semantics, and docs that motivate features (egeersoz, 74717 post 7). Drop “safety defaults” from the pattern.
4. l.766-767 (analysis 1), “Two maintainers and three users independently name the same mechanism — five query-shaping concepts, four authorisation paths”. No maintainer names this mechanism in the cited sources. It comes from egeersoz (74717 post 7: “policies, filter checks, preparations, and expression calculations… four different abstraction paths”), echoed by dimitarvp (74717 post 9, on module calculations). Treat it as one user’s detailed claim, echoed by one other.
5. l.214 (W1 classification), “five distinct concepts all reduce queries”. egeersoz names four query-shaping paths (policies, filter checks, preparations, expression calculations). His list of concepts to keep apart is calculations, aggregations, preparations, policies “and other Ash-specific concepts”. The quotation marks around “which page do I read” (l.215) wrap the researcher’s own phrase, not a source quote.
6. l.801 (analysis 6), “because it is the one thing that would have changed caslu’s decision”. caslu gave three reasons for leaving: testing, learning curve, and lock-in/bus factor (69829 post 1). No source says any single change would have reversed his decision. This is speculation.
7. l.50 (Summary 9), “frontier models actively fight Ash”. egeersoz’s words are “struggle with Ash… always overcomplicate things” (74717 post 2, 2026-03-15). “Actively fight” is the researcher’s gloss.
8. l.517-518 (6b), “the more effective these tools are”. The source reads “thse” (a typo in Zach’s post, https://www.zachdaniel.dev/p/usage-rules-leveling-the-playing). The meaning is unchanged; mark it [sic] if quoting exactly.
9. l.572-574 (6e, ken-kost), “so the ‘this’ that Ash amplifies is that productivity claim”. This is the researcher’s inference from the split-thread title, which appears only inside egeersoz’s cross-quote. The parent post was not retrieved. Treat it as inference.
10. l.780-792 (analysis 4), “the evidence supports the former, not the latter”. This overstates. The evidence is one controlled study on ten Python agent frameworks with a single task and Claude-backed assistants, plus anecdotes on both sides. It suggests that convention alignment matters more than declarativeness. It does not establish this for Ash or for TypeScript.
11. Length. 11,991 words against a 6,000-word target. Not a factual error; the lead decides.

Not errors, but known limits a reader should keep in mind:

• The 70980 per-post dates (all 2025-05-22) and 73196 post 6’s date (2025-11-15) come from page context, not from per-post JSON.
• llamex = troupo is supported (his HN profile links dmitriid.com), not proven.
• ash#2921’s run-time numbers come from one team and were produced with AI assistance, as the document says.
• There is still no Ash-specific LLM benchmark and no Ash-vs-Ecto run-time benchmark.