Skip to content
Three CubesRead the reviews
Guide6 min read

Why AI agents need memory, and what I learnt building it

Think about the best graduate you ever hired. By their second year they knew who to call, which reports mattered and why a decision was made years earlier. Most AI agents never reach their second year, because they forget everything when the session ends.

Read this for your industry

Optional. Choose your industry and this piece opens with a situation from your world and closes with a first move for you. Your choice carries across the guides and analysis on this site.

Wealth management

You lead a practice whose advisers are retiring

Between 700 and 1,000 advisers retire each year, while only 569 new entrants joined the profession in 2025. When an adviser retires, years of knowledge about each client often leave with them: family circumstances, past decisions and the reasons behind them.

Agent memory can hold that context for the practice, within privacy law and your licensee's record-keeping rules, so the next adviser starts with the client's history in front of them.

What this means for you

  • Decide what client context agents should remember, and what they must never keep.
  • Record decisions together with the reasons behind them, so a new adviser can follow the thinking.
  • Test memory with the questions a new adviser would ask about a long-standing client.
Retail

You lead a territory that forgets when people leave

In the SDA's 2025 survey, 59% of members had been abused by the same customer more than once. Stores often know who the repeat problems are, and that knowledge stays with whoever was on shift, so it leaves when they do.

Agent memory can help a territory remember incident history, supplier problems and what worked last season. The Kmart decision is a reminder that anything an agent remembers about people has to stay within privacy law.

What this means for you

  • Decide what your territory's agents should remember: incidents, supplier issues, seasonal patterns and what fixed them.
  • Keep personal information out of agent memory unless your privacy assessment allows it.
  • Test memory each quarter with the questions a new store manager would ask.
Healthcare

You lead homes where every roster change resets what staff know

Staff turnover in aged care has been reported as high as 37.6% in nursing homes, and around one in four aged care workers leave within a year. CEDA estimates a shortfall of at least 110,000 direct care workers within the next decade.

Each new carer has to learn what the last one knew about a resident: how they take their tea, what calms them, which family member to call. Agent memory can keep that knowledge with the resident so it survives the roster.

What this means for you

  • Record residents' preferences and what works for them in a form agents can retrieve for each shift.
  • Keep clinical decisions in the clinical record, and let agents refer to them there.
  • Test memory by asking what a new carer would need to know about a resident on their first shift.
Product engineering and manufacturing

You lead teams that keep relearning old decisions

The 2025 DORA report advises leaders to connect AI to internal context. Agents that cannot see past design decisions, supplier history or incident reviews will repeat old debates and old mistakes.

In manufacturing and sourcing, the same applies to supplier history: who failed an audit, which part changed and why the second source exists. When the person who knew leaves, the agent should still know.

What this means for you

  • Give agents access to decision records, incident reviews and supplier history.
  • Write down the reason for a decision at the time it is made.
  • Test memory with the questions a new engineer or buyer would ask in their first week.
New energy and trade services

You run an installer whose workforce doubled in a year

The number of accredited battery installers doubled to 8,846 after the Cheaper Home Batteries Program began. Many businesses hired quickly, so many crews now work without the history of earlier jobs: site quirks, warranty issues and customers who were promised a callback.

Agent memory can hold that history for every job and every customer, so a new crew arrives knowing what the last crew found.

What this means for you

  • Record site conditions, design changes and promises made to customers against each job.
  • Have agents prepare a one-page job history before each visit.
  • Test memory with the questions a new electrician asks on arrival.
The short answerAI agents without memory start every session from zero, so their early productivity gains flatten. Agent memory keeps decisions, context and corrections in tiers the agent can retrieve when needed. Build it early, test it against a fixed set of questions, and treat it as an investment that grows in value over time, the way a graduate's knowledge does.

The graduate who forgets

When you hire a graduate, nobody expects much in the first month. You give them context and the right problems, and over a year or two their judgement builds. By the second year they are worth far more than a new hire, because they carry what the organisation knows: the history of a client, the reason a process works the way it does, the mistake that was made once and never again.

An agent without memory is a graduate who forgets all of that every night. Each morning someone has to explain the project, the decisions and the corrections again. The agent can still be useful, and its gains still flatten, because nothing builds up.

Fortune’s coverage of MIT NANDA’s 2025 report on AI in business described why generic tools stall in enterprise use: they do not learn from or adapt to workflows. Memory is a large part of that.

Many organisations deploy agents, watch the early gains level off and conclude that AI did not deliver. Often the agents were never given a way to learn.

What I learnt building one

I built a memory system for my own agents because I needed it to work. Three lessons stood out.

Write the test first. Before writing any retrieval code, I wrote 50 test questions across six categories, with an AI model scoring each answer against the expected one. Each design change either improved the score or it did not. A full run took minutes, and it ended a lot of arguments with myself about whether something felt better.

Expect some good ideas to fail. Using a small local model to rewrite search queries looked elegant and made results worse. Without the benchmark I might have taken weeks to notice. With it, I found out in an afternoon.

Use tiers. What worked was a mix of keyword and semantic search across three layers: core context that loads every session, recall memory retrieved by relevance, and an archive for deep history. Entity lookups reached 93% on my benchmark. The layered design is borrowed. It is close to what the MemGPT paper proposed in 2023, which treats an agent’s context like computer memory and moves information between tiers as needed.

The wider lesson is encouraging. Practices that once needed a platform team, such as benchmark-driven development and automated quality checks, are now within reach of a small team, and sometimes of one person.

Why it matters in Australia

Deloitte’s 2026 State of AI in the Enterprise found 12% of Australian organisations say generative AI is already transforming their business, against 25% globally. My judgement is that agent quality explains part of that gap, and memory is a big part of agent quality.

Deploying an agent is the graduate’s first day. What matters is whether it knows more by day fifty.

Where to start

Decide what your agents need to remember before you scale them: decisions, corrections, preferences and the state of live work. Write your benchmark questions. Then build the smallest memory that passes them. The simplest version is a correction log, a written file of rules the agent reads every time it starts.

Treat it the way you would treat a graduate program. The first few months feel slow, and the value builds every month after that.

Your first move: wealth management

Pick one adviser who plans to retire in the next two years. Work with them to record the context for their twenty most complex clients in a form an agent can retrieve.

Your first move: retail

Ask your most experienced store manager what a new manager most needs to know. Record those answers where every store's agent can retrieve them.

Your first move: healthcare

Choose five residents with complex needs. Work with their regular carers to record what a new carer must know, and have the agent produce a one-page shift brief from it.

Your first move: product engineering and manufacturing

Pick the decision your team argues about most often. Write the record of why it was made, and make it the first thing the agent retrieves on that topic.

Your first move: new energy and trade services

For the next ten jobs, have the agent prepare a site history from your records before the crew arrives. Ask the crew afterwards what was missing.

Questions readers ask

Is a bigger context window enough?

It helps. Loading everything every time is slow, costly and noisy, so good memory retrieves the few things that matter for the task in hand.

How do I know if my agent memory works?

Write a fixed set of questions with known answers before you build, score every change against them and keep the score visible. Without a benchmark you are relying on how the results feel.

What should an agent remember?

Decisions and the reasons for them, corrections, preferences and the state of live work. Most individual messages can be left out.

Sources

  1. MemGPT: Towards LLMs as Operating Systems, arXiv (Packer et al.), October 2023
  2. MIT report: 95% of generative AI pilots at companies are failing, Fortune, 18 August 2025
  3. Australian organisations lag global peers in realising AI's transformational potential, Deloitte Australia, 12 February 2026

The views here are my own, not those of my employer or its clients.

About the author

Dan McMahon

Dan McMahon spent fourteen years running retail operations and digital businesses, from the store floor to general manager at Dick Smith and BIG W, and the decade since building consulting and product engineering practices across Asia-Pacific. He is a Managing Director at Avanade. Three Cubes is his independent publication, and the views in it are his own.

Related insights