What Makes a Program an Agent? | AI Arch

What Makes a Program an "Agent"?

In software architecture, we are accustomed to deterministic flows, but an ai agent breaks this linear model. A user clicks a button, a function executes, and a database updates. This is automation: a scripted sequence of operations that proceeds predictably from start to finish. However, in the emerging domain of AI engineering, a new category of software has emerged that defies these static rules. It is called an "agent."

This article is Part 1 of the Building AI Agents series.

The term is often used loosely, conflating any LLM-powered chatbot with true agency. But from an architectural standpoint, an AI agent is distinct from a standard workflow by its internal control loop. An agent is a system that perceives its environment, plans a course of action, acts upon that environment, and observes the results to refine its next move. This cyclical process, rather than the mere presence of generative AI, is the core mechanism that defines agency.

The Agentic Loop: From Linear to Cyclical

To understand the difference, consider two programs tasked with finding the best flight from New York to London.

The Workflow (Automation): A traditional script follows a rigid path:

# Example: Linear Workflow
def get_cheapest_flight():
    response = flight_api.search(origin="JFK", dest="LHR")
    results = parse_json(response)
    cheapest = sort_by_price(results)[0]
    return cheapest

This is a pipeline. It has no memory of previous attempts if the API fails, and it cannot adjust its strategy if the initial search returns zero results. It is a function.

The Agent: An ai agent operates via a loop, often referred to as the OODA loop (Observe, Orient, Decide, Act), originally conceptualized by military strategist John Boyd, or the ReAct pattern (Reasoning and Acting) described by Yao et al. in 2022.

1. Observe: The agent receives the user’s request.
2. Plan: The LLM analyzes the request and decides it needs flight data. It generates a tool call: search_flights(JFK, LHR).
3. Act: The system executes the tool call and returns the result to the agent.
4. Observe (Feedback): The agent reads the result. If the result is empty, the agent does not crash. Instead, it reasons about the failure. It might decide to broaden the search criteria (e.g., check nearby airports like EWR or LGA) or change the date.
5. Repeat: The agent generates a new tool call based on this new plan.

The critical mechanism here is feedback integration. In a workflow, the output is the end state. In an agent, the output of one step becomes the input for the next reasoning cycle. This allows the system to handle uncertainty and dynamic environments, which static workflows cannot do.

Tool Use as State Extension

A common misconception is that an agent is simply an LLM with a list of tools. While tools are necessary, they are not sufficient. The defining characteristic is how the ai agent manages state across tool interactions.

In a standard workflow, a tool call is a side effect. In an agent, tool calls are part of the reasoning context. The agent maintains a working memory (often a conversation history or a vector store) that includes:

• The user’s goal.
• The steps taken so far.
• The results of previous tool calls.
• The current plan.

This creates a closed-loop system where the program modifies its environment (via tools) to gather more information, which then updates its internal state, which then drives further action.

For example, an agent tasked with debugging a server issue might:

# Conceptual Agent Step
def debug_server():
    logs = check_logs()
    error = analyze_error(logs)
    
    if "timeout" in error:
        # State update: Agent decides to verify connectivity
        health = check_db_health()
        
        if not health.is_healthy:
            # Action: Restart service based on combined state
            restart_service()

Each step depends on the outcome of the previous one. The agent’s "state" evolves dynamically, allowing it to navigate complex, multi-step problems that cannot be pre-scripted.

Autonomy vs. Automation

So, where is the line between a sophisticated workflow and an ai agent? The key is autonomy.

Automation is the execution of predefined rules. If a condition is met, do X. If Y happens, do Z. The path is known in advance.

Autonomy is the capacity to make decisions in the face of uncertainty, without human intervention. An agent exhibits autonomy when it can:

1. Decompose goals: Break down a high-level objective ("Plan a trip") into sub-tasks ("Find flights," "Book hotel," "Reserve restaurant").
2. Handle exceptions: Recover from errors or unexpected inputs by changing its strategy.
3. Self-correct: Evaluate its own output and iterate until the goal is met.

This is not just about using an LLM. You can build a rule-based system that mimics some of these behaviors, but it will lack the flexibility to handle novel situations. Conversely, an LLM alone, without a tool-use loop, is just a text generator. It can write a plan, but it cannot execute it or verify its accuracy.

The Architectural Implications

Recognizing an ai agent as a cyclical, stateful system changes how we design and test software.

1. Non-Determinism: Unlike functions, agents are non-deterministic. The same input can lead to different paths and outcomes. Testing must focus on outcomes and safety constraints rather than exact step-by-step execution.
2. Latency: The agentic loop involves multiple LLM calls and tool executions. This introduces significant latency compared to a simple API call. Architects must design for asynchronous processing and user feedback during long-running agent tasks.
3. Failure Modes: Agents can get stuck in loops (e.g., repeatedly calling a tool without progress). Robust agent architectures require explicit termination conditions, step limits, and fallback mechanisms.

Conclusion

An ai agent is not defined by its ability to generate text, but by its ability to act and adapt. It is a program that treats its environment as a dynamic input, uses tools to gather information, and iteratively refines its plan to achieve a goal. This shift from linear automation to cyclical agency represents a fundamental change in software architecture, moving from rigid pipelines to adaptive, goal-oriented systems.

Understanding this distinction is crucial for developers. It prevents the over-engineering of simple workflows as "agents" and ensures that the complexity of agentic systems is reserved for problems that truly require autonomy and adaptive reasoning. As we build more complex AI systems, recognizing the mechanisms of perception, planning, and action will be key to designing reliable and effective agents.