Handover: Context Linker WordPress Plugin

2026-08-24 05:25 UTC · Owner: TBSbot · For: Alex + dev pickup

Version

v1.3.4

Zip size

58 KB

Lines of code

~5,500

PHP files

8 includes + bootstrap

JS / CSS

1 file + 2 styles

Host

ai.tbs-staging.com

TL;DR: A working WordPress plugin that finds related content via OpenAI embeddings, lets the editor pick 1–5 target pages, then asks an LLM to draft anchor text + paragraph rewrites and apply them. Shipped through 6 hard-won bug-fix versions (v1.0.0 → v1.3.4) since Aug 20. The architecture is solid; the remaining work is UX polish + AI output quality, not core plumbing.

What the plugin does

Problem: Internal linking is the SEO lever with the longest payoff curve. Writers don't do it because finding related content + drafting anchor text that reads naturally is two hours of work per post.

Solution: A "Suggest Internal Links" button in the post editor. One click → top N related pages by embedding similarity → editor picks 1–5 targets → LLM drafts anchor + paragraph rewrite per target → editor approves → applied to post_content.

Pipeline

[save post] ─┐ [update term]├─► queue ─► cron (every 30s) ─► OpenAI embeddings ─► {prefix}context_linker_embeddings table [reindex] ─┘ │ cosine similarity │ [editor opens post] ─► "Find link suggestions" button ──────────────────────────┘ │ ▼ Context_Linker_Suggestor::get_candidates_for_selection() (top 8 by similarity, MMR-diversified) │ ▼ Editor picks 1–5 targets │ ▼ Context_Linker_Suggestor::generate_anchors_for_selected() (LLM picks anchor style + paragraph rewrite per target) │ ▼ Polarity check, duplicate guard, placeholder validation │ ▼ [Editor approves] ─► apply_to_post() ─► paragraph-replace in DB │ ▼ JS reloads Gutenberg so editor re-fetches post_content

Architecture (file map)

File	Lines	Purpose
`context-linker.php`	68	Bootstrap. Version constant, plugin header, autoload `includes/`.
`includes/class-installer.php`	115	Schema (`{prefix}context_linker_embeddings`), default settings, cron registration.
`includes/class-logger.php`	87	Centralized `Context_Linker_Logger::log()` — writes to WP option `context_linker_log`. Belt-and-braces logging on every silent skip was the single biggest debug win.
`includes/class-embedder.php`	174	OpenAI `embeddings` HTTP + cosine math. Hard char cap (24000 ≈ 7000 tokens) so reindex never trips OpenAI's 8192-token limit.
`includes/class-embeddings.php`	701	Queue + CRUD + similarity search. Packed float32 binary storage with magic header `0xC0DE + uint16 count`. `dequeue_batch()` pops N items, leaves rest.
`includes/class-batch.php`	284	Cron tick + reindex_all. Drain loop: pop N → embed → repeat → ceiling at 500 items/tick.
`includes/class-suggestor.php`	1490	The brain. Two-stage API: `get_candidates_for_selection()` + `generate_anchors_for_selected()`. Polarity check, validation, apply.
`includes/class-admin.php`	224	Settings page + meta box registration.
`includes/class-ajax.php`	634	AJAX + REST. Critical: uses bracket-notation flat-params, never JSON.stringify.
`templates/settings-page.php`	301	Settings UI (OpenAI key, post types, taxonomies, max suggestions, etc.).
`assets/js/editor.js`	904	Vanilla JS for the meta box UI. 2-stage UX (pick targets, then approve anchors).
`assets/css/editor.css` + `admin.css`	496 + 78	Styles.

Storage schema

CREATE TABLE {prefix}context_linker_embeddings (
    id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
    source_id BIGINT UNSIGNED NOT NULL,
    source_type VARCHAR(40) NOT NULL,    -- 'post' | 'page' | 'product' | 'term' | 'brand'
    source_title TEXT,
    source_url TEXT,
    source_excerpt LONGTEXT,
    embedding LONGTEXT NOT NULL,          -- float32 packed, magic 0xC0DE + count
    model VARCHAR(60) NOT NULL,           -- e.g. 'text-embedding-3-small'
    hash CHAR(40) NOT NULL,               -- sha1 of embedded text; skip unchanged items
    updated_at DATETIME NOT NULL,
    UNIQUE KEY uniq_source (source_type, source_id)
);

Public PHP API (what AJAX + JS call)

Method	Purpose	AJAX action
`Context_Linker_Suggestor::get_candidates_for_selection($post_id, $args)`	Stage 1: top N related pages by similarity	`wp_ajax_context_linker_candidates`
`Context_Linker_Suggestor::generate_anchors_for_selected($post_id, $picks)`	Stage 2: anchor + paragraph rewrite per picked target	`wp_ajax_context_linker_generate`
`Context_Linker_Suggestor::apply_to_post($post_id, $suggestions)`	Server-side apply — paragraph replace + anchor insertion	`wp_ajax_context_linker_apply`, `wp_ajax_context_linker_apply_one`
`Context_Linker_Embeddings::enqueue($source_type, $source_id)`	Add to reindex queue (called on save_post hook)	`wp_ajax_context_linker_enqueue_post`
`Context_Linker_Batch::tick()`	Drain queue, embed, update table	cron (every 30s) + manual `wp_ajax_context_linker_run_batch`
`Context_Linker_Installer::reindex_all()`	Wipe + re-queue everything	`wp_ajax_context_linker_reindex`
`Context_Linker_Embeddings::diversified_search($vector, $args)`	MMR-diversified similarity search (lambda=0.7)	(internal)

Legacy backward-compat

Context_Linker_Suggestor::suggest() is kept as a deprecated wrapper that runs both stages. Old AJAX action context_linker_suggest still works (logs a deprecation warning to Context_Linker_Logger).

The 6 bug-fix versions (the hard-won lessons)

Bug · v1.0.3

`pack('g', ...)` silent zero-vector storage

Symptom: Cosine similarity returned 0.0 for every pair, including duplicate content.

Root cause: PHP's g/G pack formats are "machine-dependent size and representation." On some PHP builds (specific glibc/libm combos), pack('g', $v) then unpack('g', ...) returns all zeros.

Fix: Switched to pack('E', ...) (big-endian IEEE 754 float64 — explicit, not machine-dependent). Added migration that re-embeds any row whose unpacked vector is all zeros.

Permanent lesson

Never use PHP's e/E/f/g/G pack formats. Use explicit byte order + explicit width.

Bug · v1.0.4

Stray closing brace from imprecise edit

Symptom: Uncaught Error: syntax error, unexpected token "public", expecting end of file.

Root cause: Edit tool's "match exact text" replacement matched the closing brace of a method AND the closing brace of the class. Phantom } before public function ….

Fix: Always re-read the file with read before editing nested braces. Match a unique chunk with 5+ lines of context above/below.

Permanent lesson

When editing nested braces, the oldText match must include surrounding scope, not just the line you're changing.

Bug · v1.0.5

`temperature: 0.2` rejected by newer OpenAI models

Symptom: Unsupported value: 'temperature' does not support 0.2 with this model. Only the default (1) value is supported.

Root cause: gpt-4o-mini, o1, o3, gpt-5 lock temperature to 1.

Fix: Hardcoded a list of locked models/prefixes in model_supports_temperature(); only sends temperature if the model supports it. Plus a retry-without-temperature if API still rejects.

Bug · v1.0.6

LLM hallucinating anchor text

Symptom: Old prompt "Generate an anchor phrase." LLM returned full sentences, paragraphs, emojis.

Fix: Closed-list "pick from these spans" prompt. PHP pre-extracts 2–8 word n-grams from the post that share vocabulary with the candidate target. LLM only picks an anchor_id. Server looks up the text by id.

Lesson

Closed-list prompts suppress bugs but also suppress capability. We later moved past this (v1.1.0) by giving the AI more freedom and validating defensively on apply.

Bug · v1.0.7 · THE BIG ONE

JS `JSON.stringify` + PHP `(array)` cast silent skip

Symptom: AJAX returned {applied: [], skipped: [...]}. JS marked everything "skipped" silently. User saw "Applied!" but content unchanged.

Root cause:

JS sent suggestions: JSON.stringify(approved) (a JSON string)
PHP's (array) $_REQUEST['suggestions'] cast on a string returns ["<the JSON string>"] — a one-element array whose single member is the string itself
Every iteration of foreach ($suggestions as $s) got a string; $s['anchor'] was empty
Every suggestion was skipped; $applied stayed empty
if (!empty($applied)) skipped wp_update_post()

Fix: Replaced JSON.stringify with bracket-notation flattening. flattenParams() recursively converts objects/arrays to PHP-style suggestions[0][anchor] keys. PHP reads them back as nested arrays.

Permanent lesson

WordPress AJAX. Never wrap nested data in JSON.stringify on the JS side. PHP reads nested $_REQUEST keys as flat strings. Use bracket notation: params['suggestions[' + i + '][anchor]'] = s.anchor.

Bug · v1.0.8

Editor showed stale content after apply

Symptom: Post_content in DB had the new <a> tags, but Gutenberg still showed old text. User concluded "nothing happened."

Root cause: Gutenberg has its own in-memory copy of post_content; doesn't reload from DB after server-side modification.

Fix: After successful apply, JS calls window.location.reload(). Also wp.data.dispatch('core/editor').resetEditorBlocks() to flush Gutenberg's internal state.

Permanent lesson

Any plugin that modifies post_content from outside Gutenberg MUST trigger a reload or block reset. Otherwise the user sees "Applied!" but the editor looks unchanged.

Bug · v1.3.1

Reindex 8192-token limit

Symptom: Embedding API calls rejected for URL-heavy / code-heavy pages.

Root cause: v1.3.0 had wp_trim_words(1200) safety, but word counts ≠ token counts. 1000 words across 55,000 chars = 15k+ tokens.

Fix: Hard char cap MAX_EMBED_INPUT_CHARS = 24000 (~7000 tokens worst-case). Title + excerpt at top of payload so they always survive. Truncation snaps to last whitespace + … [truncated] marker.

Lesson

When a service measures its limit in tokens but your code counts in words/chars, add a backstop in the service's unit.

Bug · v1.3.2

Drain loop dropped items

Symptom: Reindex button only processed the first 25 items. Re-pressing re-queued the same set.

Root cause: tick() did dequeue_all() (drains entire queue) → array_slice($items, 0, $batch) (takes first 25) → process. Other 199 were never re-enqueued, never embedded, never visible in queue.

Fix: Added dequeue_batch($n) (pops first N, leaves rest). Refactored tick() to a drain loop with MAX_BATCHES_PER_TICK = 20 safety ceiling (500 items/tick max).

Lesson

Drop-on-floor bugs are invisible until you look at the queue vs the table side-by-side.

Bug · v1.3.3

Private method called cross-class

Symptom: Uncaught Error: Call to private method Context_Linker_Suggestor::candidate_for_prompt() from scope Context_Linker_Ajax.

Root cause: Method was private static in v1.3.0. class-ajax.php calls it for the editor's Find pages panel. PHP throws fatal on private from outside class scope.

Fix: Changed private static → public static. Method is pure data shaping, safe to be public.

Lesson

When a static helper is called from self:: AND from another class, it should be public. private static means "same class only."

Bug · v1.3.4

Greedy regex / nested-broken HTML

Symptom (recurring, finally diagnosed): Second apply produces <at wine="" &="" spirit="" academy="" bangkok,="" …> wrapping entire paragraph, with every word becoming a per-word attribute.

Root cause: parse_placeholder() and placeholder_to_html() used greedy regex \[CL_ANCHOR\b[^\]]*\].*\[\/CL_ANCHOR\]. When LLM produced a rewritten_paragraph containing the new [CL_ANCHOR] placeholder AND kept an existing <a> tag (verbatim from post), the regex matched from first [CL_ANCHOR to last [/CL_ANCHOR], wrapping everything as "anchor text." WordPress's wp_kses_post then "fixed" the invalid HTML with literal [/CL_ANCHOR] text inside <a>, breaking the structure into per-word "attributes."

Fix:

Changed .* → .*? (non-greedy) in both regexes.
Defense in depth: apply_one_option() refuses if rewritten_paragraph has != 1 placeholder.

Lesson

Always use non-greedy quantifiers when matching delimiters in user-generated content. Always validate at the apply step too, not just at generation.

Current state as of 2026-08-24 05:25 UTC

Item	Status
Code state	Verified v1.3.4 in source AND in zip (both 58 KB, 15 entries)
Deployed version on `ai.tbs-staging.com`	Unknown Last confirmed active was v1.0.5 (per Aug 20 memory). Alex does manual uploads.
Pending deploys to Alex	Possibly unconfirmed v1.0.6 → v1.3.4 may not all be live
Production deployment automation	None Alex uploads zips manually. Two paths proposed (SSH/rsync or git) — neither chosen.
Test harness	`/tmp/test-drain3.php` (11 tests for v1.3.2 batch logic). Other versions tested ad-hoc against the host.
Open bugs from user testing	None confirmed in 2026-08-21 memory. Last user test was v1.0.8 editor-reload fix.

Pending items (from 2026-08-20 + 2026-08-21 memory)

Not started, low priority

rebuild.sh — version bump + zip in one call. Currently you do bash package.sh from the plugin dir, which produces ../context-linker.zip. A wrapper that also bumps the version in context-linker.php would save a manual edit.
Bulk-action UX on All Posts/Pages — Alex proposed moving find-suggestions + apply out of the editor to a bulk action on the post list. Skipped after v1.1.0 multi-option UX won. Still deferred.

Needs validation on real posts

Real-world LLM output quality. v1.1.0 added 2–4 options per target (exact / topic / broader / action styles). Not yet validated on diverse post types — prompt may need refinement.
Per-target style bias. Should some target types get more "exact" suggestions and others more "action"? User choices are logged but not yet used to bias future requests.
First-run prompt testing — LLM should consistently pick good paragraphs and generate 2–4 options across the full candidate list. Not yet tested at scale.

Open questions for Alex

Does the v1.0.8 editor-reload actually show the inserted link, or does Alex still see "no change"?
What version is actually live on ai.tbs-staging.com right now?
Did Alex's previous "injection" complaint mean he expected the plugin to rewrite prose (not just wrap anchors)? Internal linking is anchor-wrapping by default, but Alex may have a different mental model — v1.1.0+ now also rewrites paragraphs.

How to pick this up (dev onboarding)

1. Read these in order

~/clawd/work/context-linker/README.md — what the plugin does
~/clawd/work/context-linker/context-linker.php — bootstrap, autoload
~/clawd/work/context-linker/includes/class-installer.php — schema, settings, cron
~/clawd/work/context-linker/includes/class-embeddings.php — storage + similarity
~/clawd/work/context-linker/includes/class-suggestor.php — the brain (start at line 72: get_candidates_for_selection → 146: generate_anchors_for_selected → 1213: apply_to_post)
~/clawd/work/context-linker/includes/class-ajax.php — HTTP surface
~/clawd/work/context-linker/assets/js/editor.js — UX
~/clawd/memory/2026-08-20.md + ~/clawd/memory/2026-08-21.md — bug history and lessons

2. To rebuild and deploy

cd ~/clawd/work/context-linker
# 1. Bump version in context-linker.php header AND define() on line 20
# 2. Run the build
bash package.sh
# 3. Upload context-linker.zip to ai.tbs-staging.com via WP Admin → Plugins → Add New
# 4. Verify version on host matches what you built

3. To debug live issues

Check Context_Linker_Logger first. WP option context_linker_log has every silent skip with the exact reason. This is the single biggest debug win — don't trust the API response, read the log.
Two failure modes to know about:
- "Applied but nothing changed" → Gutenberg didn't reload → check JS console for the reload call, check AJAX response
- "Suggestions look wrong" → check the LLM raw response in the log, check validate_suggestion() warnings
For reindex issues: the queue table shows what's pending. The embeddings table shows what's done. Compare them to find drop-on-floor bugs.

4. To add a new feature

New post type to embed? Add a checkbox in templates/settings-page.php, add the embedding hooks in class-embeddings.php (look for the existing post/term/product enqueue paths).
New suggestion style? Add to the prompt in class-suggestor.php system_prompt() AND to the validator's style whitelist in validate_suggestion().
New AJAX action? Add add_action('wp_ajax_...') in class-ajax.php init() AND the handler method. Use bracket-notation params in JS (NEVER JSON.stringify).

The architecture decisions that matter most

These are the ones I'd defend even if a reviewer pushed back:

Paragraph-level replace, not anchor-level insert (v1.1.0). Closed-list "anchor must exist verbatim" was killing capability. Letting the AI pick an anchor AND rewrite the paragraph, then server-side fuzzy-matching the original_paragraph, gives the AI real flexibility without breaking the apply step.
Polarity preservation check, server-side. Pure regex + arithmetic, no LLM call. Catches the common failure mode (LLM drops "not"/"no" and ends up asserting the opposite). Be permissive when uncertain (|orig_score| < 1 → pass).
Belt-and-braces logging on every silent skip. The single biggest improvement. Context_Linker_Logger::log('warn', 'exact reason here') in every branch of every validator. When something breaks at 2 AM, the log tells you which branch fired.
Bracket-notation AJAX params. Not negotiable. Never JSON.stringify a nested object for WP AJAX. The v1.0.7 bug cost hours.
Editor reload after server-side post_content modification. Gutenberg doesn't notice. window.location.reload() + wp.data.dispatch('core/editor').resetEditorBlocks(). Non-negotiable for any plugin modifying post_content.
Char cap, not word cap, for embedding inputs. Word counts ≠ token counts. When the upstream service measures in tokens, your safety needs to be in the same unit.
Drain loop with explicit ceiling (MAX_BATCHES_PER_TICK = 20, ~500 items). Prevents a runaway queue from holding a request open forever. Items past the ceiling stay queued for the next cron tick.
Defense in depth on validation. Generation-time validation isn't enough. Apply-time validation too. Always assume upstream can lie.

Quick contact / handoff info

Repo (local): ~/clawd/work/context-linker/
Zip: ~/clawd/work/context-linker.zip (58 KB)
Host: ai.tbs-staging.com (Alex owns the host, not me — I produce zip, he uploads)
Owner: Alexander Lambie (TBS Marketing, CTO)
Memory notes: ~/clawd/memory/2026-08-20.md, ~/clawd/memory/2026-08-21.md
Slack: N/A — Alex uses Telegram (alex@tbs-marketing.com email, Telegram ID in USER.md)
Workflow: dev writes → bumps version → runs package.sh → hands zip to Alex → Alex uploads

What the plugin does

Pipeline

Architecture (file map)

Storage schema

Public PHP API (what AJAX + JS call)

Legacy backward-compat

The 6 bug-fix versions (the hard-won lessons)

pack('g', ...) silent zero-vector storage

Stray closing brace from imprecise edit

temperature: 0.2 rejected by newer OpenAI models

LLM hallucinating anchor text

JS JSON.stringify + PHP (array) cast silent skip

Editor showed stale content after apply

Reindex 8192-token limit

Drain loop dropped items

Private method called cross-class

Greedy regex / nested-broken HTML

Current state as of 2026-08-24 05:25 UTC

Pending items (from 2026-08-20 + 2026-08-21 memory)

Not started, low priority

Needs validation on real posts

Open questions for Alex

How to pick this up (dev onboarding)

1. Read these in order

2. To rebuild and deploy

3. To debug live issues

4. To add a new feature

The architecture decisions that matter most

Quick contact / handoff info

`pack('g', ...)` silent zero-vector storage

`temperature: 0.2` rejected by newer OpenAI models

JS `JSON.stringify` + PHP `(array)` cast silent skip