AI does not start with the model

The question a comparison cannot answer

Asking which model is better becomes useful when the sentence can be completed: better at what, using which information, and within which limits? Without that reference, the answer describes a technology preference rather than a design decision.

A general capability and a business function operate at different levels. Interpreting language is a capability. Identifying requests that need review before entering a work queue is a function. The latter includes recipients, criteria, consequences, and a position within a process.

That position changes what getting the answer right means. An advisory classification may allow later correction. The same classification, used to prevent a request from reaching a person, may require different safeguards. The model's output can have the same format in both cases while supporting different work.

The first description of an initiative should therefore make sense without naming a provider. If the project can only be explained by naming a technology, its expected contribution remains unclear. That does not prove the initiative lacks merit. It does indicate that evaluating it will still be difficult.

Technology can also create possibilities that did not previously exist. Exploring a new capability is legitimate. Even then, exploration and implementation represent different commitments. One investigates what is possible; the other assumes that possibility will solve a particular piece of work. Confusing them turns an interesting hypothesis into an expectation that has not yet earned support.

Defining the work means making choices

Phrases such as “improve productivity” or “bring AI into operations” express an intention. They do not establish what a system must deliver. Progress requires a bounded unit of work with a recognizable beginning and an outcome someone can accept.

That boundary includes decisions the technical team cannot make alone. Is the system expected to produce a proposal or a resolution? Does the output go to a person or change a record? Which parts require judgment, and which must follow a rule? Who can declare the work complete?

The definition must also establish what lies outside scope. A system that prepares information for a decision need not take responsibility for making it. Expanding that scope may be reasonable, but it changes the acceptance conditions. Ambiguity at this boundary tends to return during the project as new features, additional reviews, and disagreements.

A useful definition expresses the outcome from the recipient's perspective. “Generate a summary” describes an activity. “Help a person identify outstanding issues without rereading the entire file” describes a purpose against which that summary can be judged. Purpose is not sufficient on its own, but it makes it possible to ask which information cannot be lost and how that will be checked.

This distinction matters when estimating and reviewing the work. Agreeing only on the activity can produce a technically compliant delivery that disappoints its user. Agreeing only on a broad aspiration can allow scope to expand indefinitely. The design must connect a bounded function with the consequence the organization expects from it.

Constraints help define the solution

A requirement is not only something the system must do. It can also be a condition the system must respect while doing it. Available time, access to data, review requirements, and acceptable cost belong to the problem from the beginning.

An excellent answer that arrives after the decision may be useless. A correct proposal whose sources must be reconstructed manually can transfer more work than it removes. A solution dependent on inaccessible information does not become viable merely because a more capable model is available.

These constraints should not be treated as inconveniences to address at the end. They determine which architectures are worth exploring. The same goal might call for interactive assistance, batch processing, or a workflow in which AI participates in just one stage. The difference may concern timing and operational fit more than intelligence.

Some constraints also compete. Shorter response times can limit the checks that are possible. Broader coverage can increase the volume requiring review. The organization must decide which tradeoffs it accepts and which make the project unsuitable. A model cannot settle a preference the business has not expressed.

This does not require an exhaustive specification before any experiment can begin. It requires identifying the conditions that could change the decision to proceed. Those conditions belong in early experiments, because discovering them after a complete integration has been built makes correction more expensive.

The model has a role within the system

Once the work is defined, it becomes easier to identify where an AI capability is needed. It may interpret a variable input, prepare a proposal, or select a tool. Other parts may depend on conventional queries, rules, calculations, and validation.

Separating those roles avoids assigning the model responsibilities that belong to the application. A model can suggest an action; checking whether the user may perform it requires system controls. It can draft an answer; obtaining a current fact requires an accessible source. It can describe an outcome; establishing that the outcome exists requires evidence.

The aim is not to minimize AI use as a matter of principle. It is to give each component a function that can be understood and verified. Concentrating the entire solution in a long instruction can mix together different decisions: what the model should interpret, what the application must enforce, and what a person should decide.

Anthropic distinguishes workflows whose sequence is defined in code from agents in which the model dynamically decides how to proceed. That architectural distinction makes it possible to choose how much control over the path to delegate, rather than treating every AI application as the same kind of system.

For a business, the implication is substantial. Needing natural language does not imply needing autonomy over an entire process. A limited intervention in an existing workflow may be enough. An agent may also be appropriate, but its discretion should respond to the variability of the work.

A design becomes defensible when the purpose of each capability can be explained. That explanation provides a basis for simplifying, replacing, or extending components while preserving the outcome they are intended to support. It also gives the team a shared way to discuss changes without returning to an abstract debate about the most capable model.

Selection needs a reference for quality

With the work bounded, comparing models stops being an abstract competition. The question becomes which option performs a particular function better, and which resources it requires to do so within the agreed conditions.

The first reference may be the current procedure, a rules-based solution, or a simple version of the application. Its purpose is to provide something observable for comparison. Without it, an apparent improvement may come from changing the inputs, selecting easier cases, or moving review tasks outside the measurement.

Google's machine learning engineering guidance recommends defining metrics early and starting with testable infrastructure and a simple model. Although written for ML systems rather than as a specific recipe for generative AI, it expresses a relevant principle: complexity needs a reference against which its contribution can be demonstrated.

A selection test should include the conditions that make the work difficult, alongside well-prepared inputs. If the system will encounter incomplete information, observe how it handles it. If an answer needs review, examine what the reviewer receives and how much work remains. If an omission matters more than an overly broad answer, the comparison must reflect that difference.

This does not require resolving the entire evaluation program before selecting a model. It requires avoiding a choice that cannot be justified in relation to use. A small collection of well-defined situations can inform an initial decision better than an extensive demonstration with no acceptance criteria.

Nor should the result be treated as a permanent property of the model. Configuration, instructions, information, and tools are all part of the test. What has been observed is a solution's behavior under particular conditions, not a guarantee that transfers to every integration.

The definition must be able to change

Starting with the work does not mean fixing its definition once and preventing technology from challenging it. Experimentation may reveal that a function is impractical, that the desired outcome needs reframing, or that a more valuable opportunity exists.

That learning is useful when the relationship between the original hypothesis and the discovery remains visible. If a proposal always needs a complete review, the system may still be valuable, but not in the way initially expected. The organization must decide whether its preparatory work justifies the effort that remains. Changing the goal can be a sound decision; concealing the change prevents anyone from knowing what was achieved.

The main problem may also lie upstream of the model's intervention. If nobody maintains the reference information or teams apply incompatible criteria, the initiative will expose that gap. The response may be to agree on a policy, improve a source, or reorganize a handoff before expanding automation.

Definition and discovery need to stay in balance. Too much early specification can solidify incorrect assumptions. Too much exploration without criteria can produce prototypes that never converge. A project needs enough definition to learn and enough flexibility to incorporate what it learns.

The first valuable deliverable is therefore not always an application. It may be a description of the work connecting outcomes, limits, information, and responsibility to a testable technical hypothesis. From there, the model becomes a choice the organization can explain and revisit as evidence develops.