Should I rebuild my AI Employee Factory on an open-source agent framework? I ran the evaluation the way I’d want any architecture decision run: adversarially, with evidence.
Someone pointed me at Mastra — the open-source TypeScript agent framework from the Gatsby founders. Agents, workflows, durable execution, memory, MCP, evals, observability, a local Studio. It’s a serious project: 1.0 shipped in January 2026, a $22M Series A, real production users. The obvious question: should any of it replace parts of my own AI Employee Factory — the governed execution engine, the autonomy loop, the labor routing, the evidence layer I’ve been building all year?
The temptation with a question like this is to answer it from enthusiasm (new framework, professional team, shiny docs) or from defensiveness (I built mine, therefore mine is right). Both are bias. So I ran it as a formal buy/build/steal evaluation instead: six parallel research tracks — Mastra’s actual current architecture, Mastra’s production track record, an archaeology of my own estate (what’s actually live versus aspirational), my assurance invariants, my cognitive-efficiency and labor-routing model, and my Folder Chief product. Then the synthesized recommendation went to an independent adversarial reviewer — a different vendor’s model, with no access to how the packet was produced — whose explicit job was to find framework enthusiasm, not-invented-here bias, and claims unsupported by repository evidence.
The reviewer’s verdict on the packet: sound, with findings. It confirmed the direction, caught two of my evidence numbers being stale, and made me engage the strongest pro-Mastra argument head-on instead of around. That process — produce, then have a different model try to tear it down, then reconcile — is the same discipline my platform enforces on its own AI workers, and it earned its keep again here.
The verdict
No, not as a runtime for the AI Employee Factory. Yes, as a pattern library to steal from. Plausibly yes as a delivery stack for client work — a use, not a migration.
Three findings drive it.
1. Mastra is weakest exactly where my system is strongest
The subsystem Mastra would replace — workflow orchestration with durable state — is Mastra’s best-documented failure surface. Its suspend/resume layer is snapshot-based, not replay-based, and the GitHub issue history tells the story: resume restarting from the first step, nested workflow suspend/resume broken, runtime context lost on resume, human-in-the-loop deployments producing 45–91MB snapshot rows with resume latencies up to 99 seconds, and storage-adapter errors silently swallowed while user messages went unpersisted. Mastra’s own answer for hard durability is telling: it ships Temporal and Inngest integrations, and serious adopters run Mastra on top of those rather than trusting its persistence.
Meanwhile the layer of my own system this would replace is its most battle-hardened code: sealed playbook snapshots so a run executes the contract that was reviewed rather than whatever the file says now; hash-chained evidence manifests verified before anything renders; resume paths that refuse divergent evidence; a validation-only resume that structurally cannot convert a failed independent evaluation into a passed one. Nearly every one of those rules exists because something specific went wrong once, and the incident is written down next to the rule.
You don’t trade a hardened subsystem for a framework’s weakest subsystem because the framework’s docs are nicer.
2. The labor model doesn’t fit the abstraction
My platform’s unit of labor is not “a model.” It’s a worker: a binding of model @ harness @ funding channel. The same model reached through a different CLI or paid for through a different subscription is a different worker, with different validation status, different capabilities, and different quota consequences. Workers are dispatched as CLI subprocesses — Codex, Claude Code, Grok, Cursor, OpenCode, and friends — under subscription economics where per-token API spending is prohibited by default and exceptions are capped in dollars. Review independence is enforced at execution time: a reviewer whose route matches the producer’s route is refused, not warned.
Mastra’s model abstraction is Vercel AI SDK provider strings — elegant for API-key economics, and genuinely nice for per-step model swapping. But it has no vocabulary for harness identity, funding channels, quota ledgers, or route-identity independence. Every load-bearing part of my labor model would have to be built around the framework. At that point Mastra is reduced to a step-runner — and I already have a step-runner with governance the framework can’t express.
3. The differentiating IP is the invariants, and invariants don’t port
The estate archaeology catalogued more than sixty operated invariants — not aspirations, but rules with enforcement code and a named incident behind them. A sample of the ones that would be most at risk in any framework rebuild:
- Routing mandates are machine-compared, not persuaded. The prose version of “the reviewer must be independent” was once violated by eleven of thirteen generated playbooks while technically satisfying the letter of the check. Now the mandate is enforced by comparison in code. Frameworks express routing as configuration the agent is asked to honor — which is exactly the failure mode.
- Stale runs are reconciled by audited operator facts, never cleaned up automatically. A framework’s instinct is auto-cleanup and auto-resume. Here, that instinct is three distinct audit failures with names.
- A judge can fail a step but never pass one. LLM-as-judge is advisory-blocking, schema-validated, route-separated, and never the only gate. Frameworks model judge-passes-therefore-step-passes as a feature.
- Declaration is not execution. A tool call recorded is not proof the command ran; claimed commands get re-executed or reconciled against trusted records. Most harnesses record “tool called” and treat it as truth.
- Approvals are hash-bound. An approval binds the exact content approved; editing the request voids the approval. A generic HITL “approve” button carries none of that.
None of these exist in Mastra, and none of them would survive a rewrite by default. They’d have to be re-earned — and most of them were originally earned the expensive way.
The steelman, engaged honestly
The strongest pro-Mastra case isn’t “replace the governance.” It’s the hybrid: keep the invariants as a governance layer and let Mastra be the commodity substrate underneath — because my estate’s genuine weaknesses (single-host coupling, a very large CLI-output parser, a bespoke Python stack that’s hard for anyone else to own) are exactly what a mainstream TypeScript framework fixes.
The answer is a boundary question: where does governance end and substrate begin? Walking down the stack from the invariants, the sealed snapshots, evidence chains, route conformance, fail-closed resume, and OS sandboxing are the execution layer — they live in the same code paths that run steps. The commodity remainder underneath (spawn a subprocess, await it, persist a file) is a few hundred lines I already own and rarely touch. The hybrid doesn’t delete the hard 90 percent; it re-hosts the easy 10 percent and then re-implements the hard 90 percent against a foreign engine whose durability layer is currently its top bug cluster. The genuine weaknesses have cheaper direct fixes that don’t route through a framework.
And because architecture by aesthetics is exactly what I’m trying to avoid, the plan still includes a bounded proof: a two-day vertical-slice spike that rebuilds one real workflow on self-hosted Mastra and A/B-measures it against the current path — correctness, model calls, tokens, kill-and-resume fidelity, evidence quality, failure legibility. If the measurements contradict the reasoning, the verdict changes. That’s what measurements are for.
What I’m stealing
Honesty cuts both ways, and here’s the uncomfortable part: Mastra’s observability is better than mine. Every agent step, tool call, and model call produces a span with automatic token counts and cost estimates, exportable to any OpenTelemetry endpoint. My own platform’s documentation admits the mirror image: no trusted per-invocation usage artifact exists, and token budgets were deliberately rejected for exactly that reason. A meaningful fraction of the cost questions I designed my measurement store to answer are still unanswerable.
So that’s a steal, not a buy: instrument the CLI-adapter seam, emit span-shaped usage records with provenance labels (measured versus estimated versus absent — and absent stays absent; no synthesized numbers), keep the TypeScript runtime out of it. Second steal: Mastra’s live-sampled scorer pattern — rubric-based evaluation sampled against real production runs rather than only episodic benchmark suites — folded into my existing eval layer under its existing rules: advisory, route-separated, never a gate.
Where Mastra genuinely fits
There is one place the hybrid argument holds: client delivery. Consulting clients increasingly want agents on a mainstream stack their own engineers can hire for and own after I leave. A bespoke single-tenant Python platform is the wrong deliverable for that job; Apache-2.0 Mastra on their own infrastructure is a defensible one.
So the plausible product is a lightweight-governance template on Mastra: route separation and approval gates and per-run evidence directories expressed in Mastra primitives, pinned dependencies with a delayed-update policy (Mastra had a real supply-chain incident in June 2026 — 144 packages briefly republished with malware via an ex-contributor’s unrevoked npm rights), and an honest assurance disclosure. Honest means naming the tier: this is lightweight governance, deliberately below what my own platform enforces, because the OS-level sandboxing, sealed evidence chains, and hash-bound approvals don’t port. Selling “governed” without the enforcement would be exactly the prose-masquerading-as-interlock failure my own doctrine exists to prevent.
And Folder Chief? Absolutely not
My other product is a chief of staff that is literally a folder — no daemon, no server, no database, Markdown memory the user owns, inert unless invoked. Its security posture is the absence of an attack surface: a folder has no port. Adding a TypeScript runtime with a Node server and a database isn’t a trade-off for that product; it’s a category change that deletes the product. Frameworks remain useful to it only as a pattern library to mine on paper — suspended-task conventions, eval scorecards, memory-compaction rituals — through its existing adopt-or-reject discipline.
The principle
The governing rule for the whole exercise, and the one I’d offer anyone facing a build-versus-framework decision with an existing system:
Own what differentiates you. Use what others have engineered better. Steal every good idea. Delete everything you no longer need.
Research decides — and an adversary should review the research. In this case the research said: the framework is real, funded, and genuinely good at things I’m bad at. And the right amount of it to put inside my platform is none.