The system at a glance: four artifacts, one pipeline
The upload is a working set, not four separate ideas. The hybrid workflow is the full pipeline. The code module is its brain, lifted out as a standalone file. The monolith is a shortcut that trades visibility for brevity. The bridge is the front door for approvals that arrive from outside n8n.
SAL Content Pipeline Hybrid — 13 nodes
The full workflow. Every stage is an explicit, inspectable step: two triggers, a file-type filter, ingest, intent, the parallel-agents code node, then two independent branches (auto-publish check, YouTube check). You can watch a run move and stop it at any seam.
Hybrid Parallel Agents — the code module
The exact source of the hybrid workflow's code node, as its own file. Loads three prompt templates from the gatekeeper, substitutes the intent JSON and the Substack CTA, fires the three Ollama generations in parallel, validates, selectively regenerates failed branches (max 3 cycles), and stages the bundle.
SAL Pipeline Monolith (shortcut) — 7 nodes
The same triggers and filter, then one call: POST the path to /sal/pipeline and let the gatekeeper run ingest, intent, agents, and validation internally. Fewer moving parts in n8n; less visibility into which stage failed.
Approval distribution bridge — 2 nodes
A webhook (POST approve-publish) that forwards approvals to the gatekeeper's /approve endpoint. This is how a human (or another system) approves a staged bundle without opening n8n: send a run_id, get a publish.
Status note: these are workflow definitions and module source as uploaded. The gatekeeper service and the n8n instance are not part of the upload, so whether this pipeline is currently deployed and running is unknown from these files alone. The runner in Section 05 simulates the documented flow in your browser; it does not call a live gatekeeper or model.
The hybrid workflow: 13 nodes, two triggers, two exit branches
| Node | Type | What it does, per the export |
|---|---|---|
| Manual Test | manualTrigger | Starts a run by hand. |
| Fixture Path | set v3.4 | Sets path = /content/raw/sample-article.md so a manual test has something to process. |
| Watch Raw Folder | localFileTrigger | Watches /content/raw for add events. The production entry point. |
| Filter MD or TXT | if v2.2 (OR) | Passes only paths ending .md or .txt. The false output is empty: anything else stops here, silently. |
| Ingest Path | httpRequest | POST { path } to gatekeeper:7441/ingest-path. |
| Sal Intent | httpRequest | POST { path } to /sal/intent. Returns the intent object and metadata the agents consume (publish_mode, youtube_url, substack_cta_url). |
| Hybrid Parallel Agents | code v2 | The module of Section 04. Node note in the export: “Parallel Ollama via Promise.all; /validate retries up to 3 cycles; selective regenerate using retry_hints.” |
| Auto publish? | if v2.2 (AND) | True only when publish_mode == "auto_after_validation". |
| Approve and Publish | httpRequest | POST /approve with { run_id, sources: [typefully, buffer, repurpose], move_to_published: true } and the X-Approval-Secret header (value from the APPROVAL_WEBHOOK_SECRET environment variable; the secret itself is not in the file). |
| Has YouTube URL? | if v2.2 (AND) | True when youtube_url is non-empty. |
| Opus Submit | httpRequest | POST { run_id, youtube_url } to /video/opus-submit for clip generation. |
The gatekeeper: every call the four artifacts make
All intelligence and all authority sit behind one internal service, gatekeeper:7441. The workflows are thin: they move files and flags. The gatekeeper holds the templates, the validator, the staging area, and the publish action. Its surface, assembled from the four files:
| Endpoint | Called by | Contract |
|---|---|---|
| POST /ingest-path | Hybrid | Body { path }. Returns the ingest record (filename, meta) the later stages read. |
| POST /sal/intent | Hybrid | Body { path }. Returns { intent, meta }: the structured intent the prompt templates consume, plus publish metadata. |
| GET /prompt-template/<name> | Agents module | Returns raw template text for x-thread.txt, linkedin-carousel.txt, video-hooks.txt. Templates live server-side, not in the workflow. |
| POST /validate | Agents module | Body: the full pack { x_thread, linkedin, video_hooks }. Returns { valid, outputs, errors, retry_hints }; retry_hints names the branches to regenerate. |
| POST /stage-bundle | Agents module | Body: intent, the validated outputs, meta, youtube_url, source_file. Returns the staged run (its run_id drives every later call). |
| POST /sal/pipeline | Monolith | Body { path }. The whole hybrid sequence (ingest, intent, agents, validate, stage) behind one endpoint. |
| POST /approve | Hybrid, Monolith, Bridge | Body { run_id, sources, move_to_published }, header X-Approval-Secret. The single publish action every path converges on. |
| POST /video/opus-submit | Hybrid | Body { run_id, youtube_url }. Hands a staged run's video to Opus for clipping. |
Plus the model itself: Ollama at OLLAMA_HOST (default http://host.docker.internal:11434), model from OLLAMA_MODEL (default llama3), called at /api/generate with stream: false; the X-thread call additionally requests JSON format. Local model, local templates, local staging: the pipeline runs without a cloud LLM in the loop.
The Hybrid Parallel Agents: how the code node thinks
One code node does the creative work, and its discipline is the interesting part: generate in parallel, validate as a whole, regenerate only the branches the validator names.
Prompt assembly
On every cycle the module fetches the three templates from the gatekeeper and substitutes two tokens: __INTENT_JSON__ (the intent object, stringified) into all three, and __SUBSTACK_CTA__ (from meta, defaulting to the placeholder https://your.substack.com) into the X and LinkedIn templates. Intent falls back cleanly: sal.meta if non-empty, else ingest.meta. The workflows carry no prompt text at all; prompts are server-side assets the gatekeeper owns.
The three agents and their parsers
| Agent | Output shape | Parsing rule, from the code |
|---|---|---|
| X thread | JSON array of strings | Response is fence-stripped (``` wrappers removed), JSON-parsed, and must be an array or the run throws x_thread_not_array. Every element is coerced to string. |
| Plain text | Trimmed string, taken as the carousel/post body. | |
| Video hooks | Up to 3 lines | Split by line, leading numbering (1. / 2) / 3]) stripped, empties dropped, hard-capped at 3. |
All three fire together via Promise.all: wall-clock cost is the slowest agent, not the sum. Each writes into a shared pack object, which is what makes partial regeneration possible.
The validation loop
The failure mode is loud by design: a pack that cannot validate in three cycles kills the run with the validator's errors and hints in the message, rather than staging something half-right. And because regeneration merges into the existing pack, a branch that already passed is never re-rolled: its approved text survives untouched while its siblings retry.
Configuration surface
| Setting | Default in code | Notes |
|---|---|---|
| OLLAMA_HOST | http://host.docker.internal:11434 | Env override; trailing slash stripped. |
| OLLAMA_MODEL | llama3 | Env override. One model serves all three agents. |
| APPROVAL_WEBHOOK_SECRET | (env only) | Sent as the X-Approval-Secret header on /approve. Never present in any of the four files. |
| maxCycles | 3 | Hard-coded in the module. |
| publish_mode | stage_only | From article meta; only the exact value auto_after_validation unlocks auto-publish. |
The pipeline runner: the documented flow, simulated
Run the hybrid pipeline end to end in this page. The simulation follows the artifacts' logic: the file filter, ingest and intent, three parallel generations, the validator with selective regeneration (max 3 cycles), staging, then the publish and Opus branches. The validator here is a stand-in applying the parsers' own constraints (X thread must be an array of posts each within the 280-character platform limit; LinkedIn non-empty; exactly the hooks the parser keeps, capped at 3). Generation is simulated locally; no gatekeeper or model is called.
The monolith shortcut: same gates, one call
The monolith keeps the hybrid's triggers, filter, and publish gate, and replaces the four middle steps with a single endpoint. What changes is where the seams are, and therefore what you can see when something breaks.
| Dimension | Hybrid (13 nodes) | Monolith (7 nodes) |
|---|---|---|
| Pipeline body | Ingest, intent, agents, validate, stage: separate nodes and calls, visible per run. | One POST to /sal/pipeline; the gatekeeper runs the sequence internally. |
| Failure location | The failing node is the failing stage; retry from that seam. | The call fails as a unit; diagnosis depends on the gatekeeper's error response and logs. |
| YouTube / Opus branch | Present: Has YouTube URL? → Opus Submit. | Absent. The monolith has no Opus branch; video handling stays a hybrid-only path in these exports. |
| Publish gate | Identical: publish_mode == auto_after_validation → /approve with the same body and secret header. | |
| Manual path filter | Identical in both: the fixture path bypasses the filter; only the folder-watcher path is filtered. | |
Read: the monolith is not a lesser copy; it is the same contract with the orchestration moved server-side. The hybrid is the debuggable form, the monolith the operable form, and the gatekeeper's /sal/pipeline endpoint is what makes them interchangeable.
The approval bridge: publishing without opening n8n
Two nodes. A webhook receives an approval from anywhere (a button, a chat command, another system) and relays it to the gatekeeper with the caller's choices resolved against sensible defaults.
| Field | Default when omitted | Effect |
|---|---|---|
| sources | ['typefully', 'buffer'] | Note the difference from the workflows' own approve calls, which send ['typefully', 'buffer', 'repurpose']. The bridge's default pair is narrower; a caller wanting repurpose must name it. |
| move_to_published | true | Only an explicit false prevents the move. Approval through the bridge means publish, unless the caller says otherwise in so many words. |
| run_id | (required) | The staged run being approved. No default exists; without it there is nothing to approve. |
Publish modes and the safety shape
stage_only
If the article's meta carries no publish_mode, the module defaults to stage_only: the bundle is staged and the run ends. A human approves later, through the bridge or the workflow. Silence means staged, never published.
auto_after_validation
Only the exact string in an article's own metadata unlocks the auto-approve call, and even then it fires only after the validator passed. The content opts itself in; the workflow does not decide for it.
One endpoint, one secret
Every publish in all four artifacts is the same call: POST /approve with the X-Approval-Secret header drawn from the environment. There is no second, quieter way to publish in this system.