Building useful AI tools without turning everything into a chatbot

The chatbot is usually not the system

A lot of AI products start with the question, “How do we add chat to this?” In many real workflows, the better question is, “Where does ambiguity, language, judgment, or synthesis actually enter the process?”

Chat can be useful, especially when the user needs flexibility. But many workflows are not primarily conversations. They are structured processes involving documents, fields, rules, decisions, evidence, and review steps.

The problem is not that chatbots are bad. The problem is treating chat as the architecture instead of the surface layer.

Chatbot-first vs. workflow-first design

The difference is mostly about where responsibility lives.

Chatbot-first

• The user enters a broad request.
• The model decides what matters.
• The model retrieves, reasons, summarizes, drafts, or recommends.
• The user has to judge whether the answer is complete, accurate, and useful.

Workflow-first

• The system defines the object being worked on.
• Inputs are structured.
• Rules and filters narrow the space.
• Retrieval brings relevant context into the model.
• The model performs a narrow task.
• The output is checked, reviewed, or linked back to source data.

In a chatbot-first design, the model carries too much responsibility. In a workflow-first design, the model is one component in a system that already understands the object, stage, evidence, and desired output.

Where models actually help

Extraction from messy inputs

Language models are good at turning messy documents, webpages, notes, PDFs, emails, or long-form text into structured fields. The key is not asking the model to “understand everything,” but asking it to extract specific pieces of information into a known schema.

Instead of asking, “Is this opportunity relevant?”, ask the model to extract deadline, eligibility, award size, topic area, required applicants, cost share, and submission requirements.

Normalization

Different sources describe similar things in inconsistent language. Models can help map that messy language into common categories. One document might say “synthetic biology,” another says “bio-manufacturing,” another says “engineered microbial production.” A workflow may need to normalize these into comparable categories before ranking or matching.

Scoped summarization

Models are good at summarizing when the scope is narrow.

• Bad prompt: “Summarize this document.”
• Better prompt: “Summarize only the eligibility criteria and explain what evidence supports the summary.”

Drafting against a known template

Models help when the template, audience, and purpose are already defined. A model may help draft a proposal section, a technical summary, an executive overview, or a diligence memo, but the structure should come from the workflow, not from the model improvising the entire format.

Ranking after deterministic filtering

Models are often more useful after rules have already reduced the search space. Do not ask a model to search every possible funding opportunity from scratch. First filter by date, eligibility, topic, award size, and applicant type. Then use the model to help compare borderline matches.

Where models should be given a smaller job

Model limitations do not mean “do not use AI.” They mean the model should be given a narrower role.

Open-ended planning across long horizons

Models can produce plausible plans, but long-horizon execution requires state, accountability, changing constraints, and feedback loops that the model does not own.

Precise arithmetic or financial logic

Models can explain assumptions, summarize scenarios, or draft narratives, but actual calculations should be deterministic.

Tasks that need guarantees

If the system needs compliance, eligibility certainty, exact calculations, or legal/regulatory precision, the model should not be the sole source of truth.

Evidence and inference blending

Models can mix what the source says with what seems reasonable. Workflows need to separate extracted evidence from interpretation.

Overconfident language

The model may sound confident even when the source data is incomplete, ambiguous, or stale.

A pattern that has worked

The pieces below are not a framework. They are a rough order of operations that has held up across several projects.

• Define the object. What is the workflow operating on: a company, a grant opportunity, a technology, a patent, a molecule, a proposal, a financial model, a regulatory document, a study?
• Define the workflow state. Imported, normalized, filtered, reviewed, drafted, flagged, approved, rejected, needs follow-up.
• Use deterministic filters first. Dates, eligibility, applicant type, budget range, technology area, required documentation, TRL range, excluded categories.
• Retrieve relevant context. Before generation, retrieve source excerpts, prior notes, templates, or structured fields.
• Call the model for a narrow transformation. Extract, classify, summarize, compare, draft, rewrite, explain, rank.
• Check the output. Schema validation, citations, source links, deterministic rules, human review, comparison to known examples, calculation checks where relevant.
• Add chat last, if useful. Chat can be helpful as a control surface or explanation layer, but it should not be the foundation of the system.

Four examples

1. Grant or opportunity matching

Bad version: “Ask the chatbot to find grants.”

Better version: Ingest opportunities, normalize eligibility fields, filter by deadline and applicant type, retrieve relevant excerpts, then use the model to explain fit and identify missing information.

The model should not be responsible for the entire search space. It should help after the system has organized the search space.

2. Technical diligence

Bad version: “Ask the model if this startup or technology is strong.”

Better version: Separate technical maturity, IP position, market need, customer evidence, regulatory issues, and adoption risk into structured fields. Then use the model to summarize evidence and identify open questions.

Diligence is not one answer. It is a structured judgment across multiple dimensions.

3. Financial modeling

Bad version: “Ask AI to build a financial model.”

Better version: Define assumptions, map revenue and cost drivers, calculate outputs deterministically, and use the model to explain scenario differences or identify assumptions that need support.

The spreadsheet or calculation engine should own the math. The model can help with interpretation, documentation, and scenario explanation.

4. Scientific or regulatory review

Bad version: “Ask AI what this study means.”

Better version: Extract endpoints, population, mechanism, dose, uncertainty, evidence quality, and source references. Then use the model to produce a scoped summary or comparison.

The model is more useful when the workflow distinguishes source evidence from interpretation.

The hard part is evaluation

The honest open question across all of this is evaluation. How do you know if a workflow improved, when there is no single correct answer? How do you know if the model improved the process or only made it feel faster? How do you measure errors introduced by summarization? How do you tell whether a recommendation is based on evidence or inference?

A partial list of things that have helped:

• Schema validation on extracted fields.
• Source traceability for every generated claim.
• Human spot checks on a sampled queue.
• Comparison against known examples.
• Regression tests for prompts.
• Review queues for low-confidence outputs.
• Deterministic checks for dates, numbers, and eligibility.
• Logging model inputs and outputs.
• Measuring time saved versus error introduced.
• Tracking when the model changes a decision versus merely explains it.

Prompts should live close to the data

Prompts are not just text. In an AI workflow, prompts become part of the system logic. If they are buried deep in code, they are hard to test, revise, version, or connect to the schema they depend on.

A few practices that help:

• Version prompts the same way you version code.
• Store prompts with the workflow step they belong to.
• Connect prompts to the schemas they read or write.
• Document expected input and output for each prompt.
• Test prompts against example inputs.
• Keep prompts narrow.
• Separate extraction prompts from drafting prompts.
• Avoid one giant prompt that tries to do everything.

Structure first, model second

The best AI tools are not always the ones that make the model feel magical. Often, they are the ones that make the workflow clearer. The model helps at specific points, but the system around it provides the discipline: data structure, rules, retrieval, evaluation, and review.

I am less interested in whether a tool feels like a chatbot and more interested in whether it helps a person make a better decision, draft a better artifact, or move a technical workflow forward with less ambiguity.