Designing Beyond the Glass

I have spent most of my career designing products and building design teams. Lately, I have been asking what happens to that work when software starts acting on someone’s behalf. A person may only see the result, but the decisions made along the way still shape their experience.

I find it useful to think about two participants: the agent doing the work and the person responsible for the outcome. They need different things from the same product.

Many designers have spent years learning how people use software and how to make complex products understandable. That expertise has also helped us lead teams, shape strategy and contribute to decisions that once happened without design in the room.

And now those screens are starting to fade into the background.

Screens will continue to matter, but they may increasingly help people direct and review work that happens elsewhere. That change asks us to consider how our experience applies when the person using a product sees only part of what it does.

Our existing skills remain useful when we apply them to the information and rules that shape the experience behind the interface.

That is where the distinction between the agent and the person becomes useful.

The first new user is the AI agent. I have been thinking of it as "The Proxy." It is the one that actually does the work. It books the flight, sorts the data, compares the prices, fills out the forms. It acts on behalf of a human, but it is not human itself.

An agent working through an API interacts with data, content models, field names and response formats, so the clarity of those structures matters in ways that visual styling does not. Decisions about how information is organised can affect whether the agent understands the task and completes it correctly.

The fear here, the one I keep hearing from other designers, is that this makes our work "boring." Or worse, irrelevant. If the thing using our product does not even look at the screen, what are we even doing?

But here is the thing I keep coming back to. If your product’s underlying structure is messy, the agent fails. If your data is inconsistent, the agent gets confused. If your content model has ambiguous labels or contradictory logic, the agent makes bad decisions on behalf of the human who trusted it.

The clarity of those structures is a design concern because it shapes the outcome a person receives, even when they never see the underlying data.

When an agent generates an interface from the data a product exposes, the quality of that experience depends in part on how clearly the information is structured. Designers can contribute by helping the team make those relationships and meanings explicit.

This gives designers a role in shaping the structures that help an agent do useful work for the person who relies on it, extending familiar questions about clarity and usability into another part of the product.

An agent working with ambiguous data faces a problem similar to a person using an interface with unclear labels, although the person may only discover the confusion when they see the result.

The other participant is the person who sets the agent in motion and remains responsible for the consequences of its actions.

I have been calling this one "The Governor." And this is where the real empathy work lives.

Think about the first time you used a self-driving feature in a car. Or the first time you let an app automatically invest your money. There is always this moment, this visceral little clench, where you think: "Is this actually going to work, or is it going to ruin my day?"

That uncertainty can remain even when the technology performs well, because supervising a system requires a different kind of confidence from carrying out a task yourself. People need to understand its boundaries and know how to intervene when necessary.

Design can help people assess whether it is reasonable to rely on a system by making its actions, limits and recovery options understandable.

A product can earn confidence by being honest when something goes wrong and giving the person a useful way to respond.

Imagine a business owner whose supply-ordering agent reports, “I placed three orders last night and paused a fourth because the supplier increased the price by 18%, exceeding your spending limit.” By explaining the exception and requesting a decision, the product gives her a chance to respond before the extra cost is incurred.

If the agent placed all four orders silently, the owner might only discover the price increase when the invoice arrived. Respecting her spending limit and giving her a chance to decide requires product rules, technical implementation and clear communication to work together.

A poor interface can already cost users money and trust. An agent adds another challenge: it can act before the person notices something is wrong. That makes visibility and recovery part of the core design.

So if the screen is fading into a status report, and the real product is increasingly invisible, what are we actually designing?

I have been wrestling with this question for months. And I think the answer is something like: we are designing the rules of a relationship.

We are shaping the relationship between a person and a system that acts on their behalf. The team needs to decide what the agent can do independently, when it should ask for a decision and how it explains unexpected results, with clear boundaries that people can adjust as they gain confidence.

These experience decisions require close collaboration with engineering because the behaviour of the system and the information a person receives need to work together.

This shift asks experienced designers to learn new things. Logic, permissions and escalation rules need closer collaboration with engineering, and leading a team means making space for that learning as the work continues.

Research and experience design still give us a way into those discussions. We can examine what a person expects, where they need a choice and what happens when the system gets it wrong. Those questions help the team decide what to build and how to test it.

I am not going to pretend I have a clean framework for all of this. I am still figuring it out in my own work. But here is what I keep coming back to.

We need to care about what is under the interface, not just what is on it. Data structures, content models, how information flows between systems. I am not saying we need to become engineers. But we need to have opinions about these things. Because the agent’s entire experience of our product lives in that layer, and if we are not helping shape it, we are designing a storefront for a building with no plumbing.

We need to get comfortable designing behaviors instead of screens. "What should the agent do when the data looks contradictory?" That is a design question. "How aggressively should the agent act before checking in?" That is a design question. They look like product decisions or engineering specs, but they are really experience decisions wearing different clothes.

Language needs to be considered early because an agent’s explanation, request for a decision or account of a failure may be the only part of the interaction the person sees. The wording should make the situation understandable and help them decide what to do next.

We also need to examine background actions and decisions that happen while someone is away from the product. Those actions still affect their experience, so the team needs to define what the system can do, how the person will learn about it and what recovery should look like.

People need to understand what an agent did, step in when needed and recover when something goes wrong.

Choose one workflow where an agent acts for a customer. Look at the data it uses, the decisions it can make and what the customer sees when it finishes. Then examine what happens when the data is incomplete or the action fails. That is a useful place for design to get involved.

I am still working through these questions in my own practice. In the next essay, I look at five moments where people need to understand and direct an agent’s work.