In my last article, (Building a Fiction Editing Pipeline with Regex and Python \u2014 Part 1 by Eddie Tonkoi<\/a>), I introduced my pipeline, including the rationale behind my Narrative Analysis toolkit, ETNA. This time, I\u2019ll be looking at the more subjective part, starting with grammar.<\/p>\n Unlike syntax and spelling, grammar is a fairly subjective ruleset, especially in a fiction novel. There are plenty of rules that just look plain wrong if they are broken, but there are also others that are more like guidelines, some that are dependent on region, and others that are broken willy-nilly in dialogue.<\/p>\n Last time, I said that one of my requirements was,<\/p>\n This is one of the reasons why I didn’t want to go with Grammarly or other solutions that offered some control, but not enough. Or at least, not on the free tier. Not that I blame them. LanguageTool<\/a> started life back in 2003 as a university project by Daniel Naber and has since grown into a widely used, open-source grammar and style checker. At its heart it\u2019s a rule-based, language-independent engine designed to support many languages, especially those that don\u2019t have expensive commercial tools, and it\u2019s now maintained by a community of volunteer contributors. No Thai yet, by the looks of it, though. In recent years it\u2019s added optional cloud-based, AI-powered checks, but the core engine remains free, offline-capable, and fully customisable. That combination\u2014transparent rules, strong grammar coverage, and the ability to run my own local server and tweak what it flags\u2014makes it a perfect fit for building a bespoke fiction-editing pipeline.<\/p>\n LT starts by segmenting the text into sentences and tokens, which means that it breaks the text following a stringent set of rules into words and phrases. It is then able to tag these into Parts-of-Speech (POS), for example, identifying verbs, nouns, adjectives, etc. It’s also able to group things by knowing that children<\/em> is the plural form of child<\/em>, for example.<\/p>\n Now, with sentences abstracted away into Parts-of-Speech rather than worrying about what the words are, it can run its extensive suite of rules on the sentence or phrase to see if it breaks any rules.<\/p>\n If something is found to break a rule, LT returns the suspect along with an explanation of the rule and a suggestion for what it should be. Depending on the mistake, the wording used could be forceful (Sentences must\u2026), or gentle (You might have meant\u2026).<\/p>\n LT can be run with a CLI, a command-line interface, and that lets me set it up to use with my Python scripts.<\/p>\n At a high level, the process is:<\/p>\n That’s LT, and it runs beautifully by itself, but I’ve added some extra suppressions and house rules<\/p>\n Once LanguageTool is running, I add two layers on top to make it behave like a fiction editor rather than a schoolteacher.<\/p>\n First, I load a small suppression list for this book. It contains:<\/p>\n That way, the report stays readable, and I\u2019m not re-litigating the same deliberate choices chapter after chapter, time after time.<\/p>\n The importance of this is due to the fact that I want to clear every single issue. Let’s say there are 15 instances of LT warning me about the use of “he shoulda known better”. I checked each one, and they are all in dialogue, all idiomatically correct. By adding the rule that checks for “shoulda” to my ignore list, these all stop showing, which reduces noise and lets me see clearly. I think this method greatly reduces the chance of overlooking an error.<\/p>\n Then come my actual House Rules: custom LanguageTool XML rules that encode the way Jern writes in this series\u2014preferred forms, banned forms, and recurring patterns I want caught consistently. These aren\u2019t \u201cignore it when it happens\u201d; they\u2019re \u201cflag it every time, because it matters\u201d.<\/p>\n Put together, suppressions stop the tool wasting my time, and House Rules turn it into a repeatable, book-specific style checker.<\/p>\n One of my simplest and most important House Rules is a British style preference: use \u201ctowards\u201d rather than \u201ctoward\u201d, and more generally prefer \u201c-wards\u201d adverbs like backwards, downwards, inwards.<\/p>\n LanguageTool will flag the shorter forms and suggest the \u201c-s\u201d version\u2014but the rule is smart enough not to fire when the word is acting like an adjective before a noun. So it will catch \u201cHe stumbled backward into the chair\u201d \u2192 backwards, but it won\u2019t nag about \u201cbackward compatibility\u201d or \u201can upward trend\u201d.<\/p>\n The importance of this is that “backwards” and “backward are both technically correct, so LT wouldn’t normally flag things, but I’ve decided that I want to be consistent with all of our books, so I wanted a clear rule. LT allowed me to create my own, clear tool.<\/p>\n Now, if you’ll forgive me, I’ll spend a couple of minutes descibing that rule a bit more, then get back to the overview.<\/p>\n In LanguageTool, a \u201cHouse Rule\u201d is basically a little XML pattern-matcher. Each rule has a unique id (so I can turn it on\/off later), a human-readable name, and then a <pattern><\/strong> made of one or more <token><\/strong> elements.<\/p>\n A token is what it sounds like: one word (or sometimes punctuation), but crucially it can be case-insensitive and regex-based. So my \u201ctoward \u2192 towards\u201d rule is just a single-token pattern that matches toward regardless of case, then a <message><\/strong> explaining why it\u2019s flagged and a <suggestion><\/strong> offering the replacement.<\/p>\n The slightly clever bit is where it becomes POS-aware. In the -ward \u2192 -wards rule, the token matches a whole list of words using regexp=”yes” (so one rule covers backward, upward, inward\u2026). Then I add an <exception scope=”next” postag=”NN.*” postag_regexp=”yes”\/><\/strong>, which means \u201cdon\u2019t fire if the next token is a noun\u201d\u2014because backward compatibility is adjectival, not an adverb.<\/p>\n For the suggestion, I\u2019m not hard-coding every replacement; I use <match \u2026 regexp_replace=\u2026><\/strong> to rewrite the matched word, so (.*)ward becomes $1wards. That\u2019s the general pattern: match tokens, optionally exempt based on neighbouring POS tags, then generate a suggestion, with a couple of <example><\/strong> lines so I can sanity-check it later.<\/p>\n The output in the report, designed and generated by my Python code, has been formatted to provide that information back to me in a meaningful way. Here is an example:<\/p>\n Notice how the report groups everything by rule ID<\/strong> first\u2014that\u2019s deliberate. It means I can make one decision and apply it consistently: fix all instances, suppress a noisy phrase, or disable the rule entirely. Under that, it tells me the human explanation<\/strong> LanguageTool provides, the category<\/strong> I filed it under (in this case my ETNA rules), and the issue type<\/strong>, which in theory helps me triage what\u2019s \u201creal grammar\u201d versus \u201cstyle preference\u201d, but in practice I’ve ignored so far. Then for each hit I get the suggestion<\/strong> and a short context snippet<\/strong> showing exactly where it fired, plus which chapter chunk it came from\u2014enough to judge whether it\u2019s narration, dialogue, or a deliberate voice choice before I accept the change.<\/p>\n If necessary, I can quickly copy some text from the Context<\/em> and use Find<\/strong> in Scrivener to see it in full context and decide whether to make changes.<\/p>\n As I have said before, I will make all changes myself manually, and I never let the tool change the text.<\/p>\n LanguageTool is an amazing piece of software that does superb grammar analysis, handling higher-level issues such as:<\/p>\n This has overlap with previous tools, but also does a whole lot more. LanguageTool is the closest thing to a human editor\u2014but it’s not perfect, and it can take quite some time to go through all of its suggestions, choosing which to take on board. That’s why I implemented some of the previous checks, so that by the time I get to this script with LanguageTool, a lot of the low-hanging fruit has already been dealt with in a calm, systematic manner.<\/p>\n I will run this script countless times over the process, picking things off bit by bit, starting with the certain mistakes and leaving any style choices until the book has cleared Jern’s final edit. This script’s report always has the most entries and is always the last to be cleared, but it really does help me polish the novel.<\/p>\n I have a few more small scripts still, with specific tasks.<\/p>\n This Python script uses fuzzy matching to detect lines that should match exactly but don\u2019t. In Murder in Treggan Bay<\/strong>, there’s a shop sign that recurs in the book. This script was able to tell me that the phrase written on it was incorrect in one instance:<\/p>\n Again, it is using RapidFuzz<\/strong>, but with quite tight settings, including a fuzzy score \u2265 95, so that we don’t get swamped by false positives.<\/p>\n I was so impressed when this actually worked!<\/p>\n This is on my todo list to find out what else RapidFuzz can do for me.<\/p>\n This script gives me a 40,000 ft view of stylistic patterns, to see if anything stands out. It does a few things that I thought might be useful, but I have yet to see if it will bring up anything of substance:<\/p>\n Once again, the goal here is to offer insight into voice and habits without prescribing solutions. This and the next script were added as final additions when I asked ChatGPT if there was anything I could, or rather should, add to my pipeline without moving to AI.<\/p>\n This final script tracks habitual gestures & intensifiers:<\/p>\n I’ve only just added this one and not really tried it yet. The idea is to see if these gestures, or adverbs occur frequently and to flag them.<\/p>\n To provide consistent, customisable spelling and grammar checking for Jern’s novels, I created a pipeline based around Python scripts that uses regex, Hunspell, and LanguageTool to generate reports for different potential issues.<\/p>\n This gives me confidence that I can work through a myriad of potential issues without missing any, and it allows me to customise on a per-book basis.<\/p>\n With these reports, I am able to choose what is an error and what was deliberate or just noise, and then manually make changes to either my custom rules or the manuscript. If you’d like to know more about how this editing pipeline works, or you have other solutions for these problems, please come over to podfeet.com\/slack<\/a>, where you can find me and all the other lovely NosillaCastaways enjoying a chat. I’ve also started trying out Mastodon, where I’m @EddieTonkoi<\/a>.<\/p>\n To find out more about Jern’s writing you can head over to jerntonkoi.com<\/a>, which has detailed information about all her books, or follow her on Instagram where she is @tonkoibooks<\/a>. You can also find out how I created that website on previous segments for the NosillaCast<\/a>.<\/p>\n Happy editing, and happy reading.<\/p>\n","protected":false},"excerpt":{"rendered":" Intro In my last article, (Building a Fiction Editing Pipeline with Regex and Python \u2014 Part 1 by Eddie Tonkoi), I introduced my pipeline, including the rationale behind my Narrative Analysis toolkit, ETNA. This time, I\u2019ll be looking at the more subjective part, starting with grammar. The problem with grammar Unlike syntax and spelling, grammar […]<\/p>\n","protected":false},"author":34,"featured_media":35149,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[147],"tags":[1105,1107,1934,7739,1899],"class_list":["post-35063","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog-posts","tag-book","tag-editing","tag-grammar","tag-language-tool","tag-open-source"],"jetpack_featured_media_url":"https:\/\/www.podfeet.com\/blog\/wp-content\/uploads\/2025\/12\/Eddie-very-silly-certificate-of-success-listing-all-of-the-elements-he-has-described-1040x520-1.jpg","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.podfeet.com\/blog\/wp-json\/wp\/v2\/posts\/35063","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.podfeet.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.podfeet.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.podfeet.com\/blog\/wp-json\/wp\/v2\/users\/34"}],"replies":[{"embeddable":true,"href":"https:\/\/www.podfeet.com\/blog\/wp-json\/wp\/v2\/comments?post=35063"}],"version-history":[{"count":16,"href":"https:\/\/www.podfeet.com\/blog\/wp-json\/wp\/v2\/posts\/35063\/revisions"}],"predecessor-version":[{"id":35143,"href":"https:\/\/www.podfeet.com\/blog\/wp-json\/wp\/v2\/posts\/35063\/revisions\/35143"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.podfeet.com\/blog\/wp-json\/wp\/v2\/media\/35149"}],"wp:attachment":[{"href":"https:\/\/www.podfeet.com\/blog\/wp-json\/wp\/v2\/media?parent=35063"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.podfeet.com\/blog\/wp-json\/wp\/v2\/categories?post=35063"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.podfeet.com\/blog\/wp-json\/wp\/v2\/tags?post=35063"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}The problem with grammar<\/h2>\n
\n
\nInstead, I came across LanguageTool via OpenOffice and a conversation with ChatGPT.<\/p>\nLanguageTool (LT)<\/h2>\n
LT Pipeline<\/h3>\n
Running a server<\/h3>\n
\n
Suppressions and House Rules<\/h3>\n
Suppressions (keep the signal, drop the noise)<\/h4>\n
\n
House Rules (teach LT my style)<\/h4>\n
Technical detail<\/h4>\n
\ud83e\udde9 Rule: 'ETNA_WARD_ADD_S'\nMessage: Use the '-wards' form for consistency.\nCategory: ETNA_WARD_RULES\nIssue type: uncategorized\n\nOccurrences: 12\n\n\ud83d\udccd Chunk: 'chapter_004.txt'\nSuggestion:\n\u200b onwards\n\nContext:\n...rry on, unaffected. The golden one zips onward, bouncing off invisible edges, zigzaggi...\n<\/code><\/pre>\nChecking the grammar<\/h3>\n
\n
Duplicate Phrase Drift<\/h2>\n
\n
\nvs<\/li>\nSimile & Crutch Word Analysis<\/h2>\n
\n
Gesture & Intensifier Repetition<\/h2>\n
\n
Recap<\/h1>\n
\n
\nAs I do this, the reports get shorter until they are totally clear, at which point I have now arranged for it to provide an official Certificate of Correctness, awarded to the novel when the full pipeline runs without any flagged errors.<\/p>\n