Home › Tools › AI ProdOps overview
Overview · From AI assistance to agentic operations

Where this prototype sits between AI assistance and agentic operations

AI can help draft a document without changing how an organization works. An agentic system goes further: it retrieves context, does multi-step work, uses tools, pauses for review, records decisions, and responds to new information. This site has a working prototype of the middle ground and a proposed architecture for the rest.

Published June 2026 Updated October 2026
AI-assisted workflow Human review Target architecture
In this piece

Getting oriented

The problem is continuity

Product teams already produce plenty of documents: research summaries, initiative proposals, business cases, prioritization records, roadmaps, requirements, stories, and delivery plans. The harder part is keeping them connected.

Evidence is summarized into an initiative. The initiative becomes a business case. The business case becomes a score. The score influences a roadmap. The roadmap becomes requirements. Requirements become a backlog. The backlog becomes a delivery plan. At every translation, context can be lost, assumptions can become facts, and a decision made for one reason can later appear to have been made for another.

What the prototype tests: whether a structured AI-assisted workflow can reduce first-draft effort and lost context across product documents, as long as evidence, assumptions, generated content, code checks, and human decisions stay distinguishable. That still needs outside evaluation.

Where the line falls

Product Management vs. Product Operations

Product Operations builds and runs the systems Product Managers work in: data tools, feedback pipelines, templates, and the planning cadence. Product Managers use what those systems produce to decide what to build and explain why. Who owns which piece varies by organization.

The tools here support Product Management work: describing customers, framing initiatives, writing business cases, prioritizing, specifying, and planning delivery. Building them was Product Operations work: designing the templates, the handoffs between documents, the review points, and the checks that run in code.

Scope. Using a six-area view of Product Operations to scope the project, the toolchain covers three: data collection, feedback management, and roadmap alignment. Release operations, enablement and training, and performance analytics are not built.

The progression

Three operating modes

The prototype sits in the middle one.

1. Manual workflow
A person carries the context
  • A practitioner gathers information, writes each document, and carries the relevant context into the next one.
  • Full human control, with a lot of repeated drafting and reformatting. Continuity depends on individual discipline.
2. AI-assisted workflow
A person still runs every step
  • A practitioner supplies context, gets a draft, reviews it, and decides whether it moves forward.
  • Drafting gets faster, and so can weak reasoning: if generated claims aren't marked apart from supplied evidence, polished output can pass for validated output. This prototype adds structured handoffs, fixed multi-step AI chains, and code checks.
3. Agentic system
Proposed, not built
  • Would retrieve live context, choose and use approved tools, work toward a defined objective, check parts of its own output, stop at enforceable approval points, record its decisions, recover from failures, and respond to scheduled or event-driven triggers.
  • It would also need identity, access control, observability, cost limits, data governance, schema versioning, and clear limits on what agents may change. Most of that is not built.

Several tools run fixed chains of two to five AI calls, but a person starts each chain and reviews what it produces. That keeps them in the middle mode: the workflow doesn't choose its own objectives, pick among tools, pull live enterprise data, write into other systems, or watch outcomes after release.

Inventory

What's built and what isn't

Built and testable on this site

A person starts and reviews every stage.

Proposed, not built

The sample evidence shows how evidence could be shaped and cited. It doesn't show that a live connector would work or that real evidence would hold up.

Human oversight

Why human review is still required

A reviewer has to be able to tell what came from evidence, what was inferred, what was generated, and what is still open. Five questions for each stage:

The more consequential the decision, the more the review has to rest on evidence and on authority the reviewer actually holds.

In a January 2026 Anthropic study, 52 mostly junior engineers learned a new Python library, some with AI help and some without. The AI group scored 50% on a quiz taken right afterward, against 67% for those who coded by hand, and finished only about two minutes faster, a difference that was not statistically significant. How people used the AI mattered: those who asked it questions to understand the code kept more of what they learned. For review, that matters: if the AI did the work, the reviewer may not know enough to catch its mistakes.

David Marquet, a former U.S. Navy submarine commander, argues that no one should approve a decision they don't understand. My article Is it Safe to Use AI? applies that idea to AI-assisted work.

Why code checks matter

Language models are useful for synthesis and drafting. They are not reliable for arithmetic, schema rules, dependency order, or whether every requirement made it downstream. So the prototype recalculates a few structural conditions in code:

These checks are narrow. They don't tell you:

Correct arithmetic can also make weak inputs look authoritative. In the Epic Prioritizer, changing one weight can move an architecture epic to #1, which is why someone has to own the weights.

“Computed” should never be treated as “true.” The Four Guardrails covers this idea more generally.

What comes next

Connecting to live data

A production version would need live evidence. Model Context Protocol (MCP), an open standard for connecting AI applications to approved tools and data, is one way to provide it. Direct APIs and ordinary code are others.

A connection alone wouldn't make the workflow agentic, accurate, or safe. Quality would still depend on which sources are authorized, what an agent may read or change, how evidence provenance is kept, how stale or conflicting data is handled, where approval is enforceable, what gets logged, and how failures are caught.

The network diagram maps the proposed architecture, and its legend says which parts are built. The document flow shows what each built tool takes in and produces.

Go deeper

Explore the evidence

Sources

Anthropic, "How AI assistance impacts the formation of coding skills," January 29, 2026. Randomized trial with 52 engineers learning a new Python library: 50% vs. 67% on a quiz taken right after the task. anthropic.com/research

Dull, R., "Is it Safe to Use AI? A Submarine Commander vs. YOLO Mode and Loss of Skill," robdull.com/perspectives, May 2026. David Marquet's Leader-Leader model applied to AI-assisted work.