When it comes to legal AI, where the work happens matters as much as what it produces

20 August, 2026

For all the attention paid to what legal AI can do, a quieter question is becoming just as important for the profession: where does the work actually happen?

As AI becomes embedded in legal practice, the architecture beneath these tools - how they are built, where data flows, and what a model can and cannot reach - is emerging as one of the most consequential decisions a firm can make.

For Australian legal professionals working under strict obligations around confidentiality, privilege and data handling, it is a question that deserves far more scrutiny than it usually receives.

Guidance from the Office of the Australian Information Commissioner also emphasises the importance of understanding how commercially available AI products handle personal information.

Two ways to build legal AI

Broadly, there are two architectural approaches to connecting a powerful AI model to trusted legal content and a user's own documents.

The first is to connect outward. In this model, a legal AI platform connects to an external AI system, passing information to it and receiving a response in return.

This is increasingly common and can be built using integration standards such as Model Context Protocol, or MCP, which enables AI systems to connect with external tools and data sources.

It is a flexible approach and, when appropriate contractual, privacy and security controls are in place, can work well for many tasks.

The second approach is to bring the intelligence into a more controlled environment.

Rather than allowing legal work to move freely across external systems, the AI operates within defined boundaries, where it can be grounded in authoritative legal content, governed by specific controls and restricted in what information and tools it can access.

In practice, legal AI architecture is not always a simple choice between these two models. Systems may combine external models, specialist databases, internal repositories and other services.

The more important question is therefore not simply whether a system is "internal" or "external", but what data moves where, who can access it, which sources shape the answer and what controls govern the process.

The distinction sounds technical. Its implications are not.

Why the boundary matters for legal work

When information moves beyond a clearly governed environment, several things can become harder to guarantee.

Attribution can blur

If an AI model introduces its own interpretations, adjacent legislation, authorities or information that has not been drawn from an identified legal source, it can become difficult for the user to distinguish what has been verified against trusted legal content and what the model has generated independently.

For legal work, where the ability to cite and defend important propositions is fundamental, that ambiguity creates risk.

This is why legal AI verification and source provenance matter. A practitioner should be able to identify the authority behind a legal proposition rather than relying on a response simply because it sounds plausible.

Consistency can weaken

Generative AI is probabilistic. The same question may not always produce identical wording or analysis.

That variability is not necessarily a flaw. But in legal practice, where practitioners need reliable access to the same authoritative sources and propositions, uncontrolled variability can undermine confidence.

The goal is therefore not necessarily to make every generated answer identical. It is to ensure that important legal conclusions can be traced back to consistent, authoritative material.

Control becomes harder to evidence

When legal work passes through multiple models, services or data sources, a firm needs visibility over how information is handled, where it travels and how outputs are produced.

For practitioners accountable for confidentiality and privilege, that visibility is not a minor technicality.

Questions about data retention, model training, third-party access, permissions, logging and data location all form part of legal AI data governance.

None of this means general-purpose AI has no place in legal work. It is capable and improving rapidly.

Nor does using an external model automatically make a legal AI system unsuitable. External models can operate under strong enterprise, contractual and technical safeguards.

The point is narrower but important: the standards legal work demands - verifiability, appropriate confidentiality, traceability and defensibility - are easier to assess when the boundaries around the system are clear and governed.

Why authoritative grounding matters

Another important part of legal AI architecture is grounding.

Grounding gives an AI system relevant source material to use when answering a question. In legal research, that might include legislation, judgments, commentary, precedents or an organisation's own documents.

A general-purpose model may have broad knowledge, but that knowledge does not by itself tell a lawyer whether a proposition is current, jurisdictionally relevant or supported by a particular authority.

A legal AI system grounded in authoritative legal content can instead retrieve relevant material and use it as the basis for an answer.

That matters for three reasons.

It narrows the information the system relies on.
It makes citations and sources easier for the practitioner to inspect.
It provides clearer provenance, an understanding of where the information behind an answer came from.

Grounding does not eliminate the possibility of AI error or hallucination. Retrieval can miss an important authority, and a model can still misinterpret a source.

Lawyers must therefore continue to verify important propositions, check cited authorities and apply professional judgment.

The value of grounding is not that it makes AI infallible. It is that it makes the work more verifiable.

The rise of AI Skills, and why grounding matters more than ever

One of the most significant developments in AI is the emergence of AI Skills: structured, reusable instructions that tell an AI model how to approach a particular task.

Skills are powerful. They allow a firm to encode its own way of working - its playbooks, precedents and processes - and have an AI apply those instructions more consistently.

A Skill might guide an AI system to review a contract against an internal playbook, prepare a chronology, identify missing information or apply a firm's drafting conventions.

But a Skill is only as trustworthy as the environment it runs in and the content it draws on.

A well-designed Skill applied by a general-purpose model working from broad knowledge may produce fluent output.

That same workflow, when supported by authoritative legal sources, appropriate permissions and controlled access to relevant documents, can produce work that is easier for a lawyer to verify and stand behind.

This is the real opportunity Skills represent for the legal profession.

Not simply automation, but the ability to combine a firm's own expertise with authoritative legal information and repeatable processes.

The Skill provides the method.

The grounding provides the evidence.

The architecture provides the boundaries.

Human review provides accountability.

All four matter.

What should law firms ask about legal AI architecture?

Legal AI due diligence should go beyond asking whether a product is "secure".

Firms should ask direct questions about how the system works and how information moves through it.

Where does the data go?

Ask what happens when a lawyer enters a prompt, uploads a document or connects a repository.

Which systems process that information?

Does it remain within one environment or pass through additional providers?

Is information retained or used for training?

Firms should understand whether prompts, documents and outputs are retained, for how long, and whether any information can be used to train or improve AI models.

Which models and third parties are involved?

A legal AI interface may rely on one or more external model providers, infrastructure providers or other sub-processors.

Understanding that processing chain helps firms assess the actual data boundary.

What information can the AI access?

Permissions matter.

A system should not automatically have access to every document, repository or connected tool simply because the technology makes that possible.

Access should be appropriate to the task.

How are legal sources selected and cited?

If a system provides legal research or analysis, practitioners should understand where those sources come from, how frequently they are updated and whether citations lead to the material actually used to generate the answer.

Can the firm audit what happened?

Logging and auditability can help a firm understand which information was accessed, which sources were retrieved and how an AI-assisted workflow was used.

The level of control required will depend on the task.

Using AI to reformat publicly available text does not present the same risk as uploading privileged client documents or producing legal analysis that may ultimately be relied upon in advice or proceedings.

Held to the standard of law

The technology underpinning legal AI will keep changing. New models, new standards, and new integration methods will continue to emerge, and quickly.

What should not change is the standard against which any of them are measured for legal work:

Can the output be verified?

Can its sources be identified?

Has confidential information been handled appropriately?

Can the firm understand how the work was produced?

Can the lawyer ultimately defend and stand behind it?

That standard is why legal AI architecture is not an abstract concern.

Whether intelligence is brought into a controlled environment or a system connects securely to external models and sources, the principle remains the same: the pathway between the lawyer's information, the AI system and the final output needs to be understood and governed.

It reflects a simple principle: legal AI should be held to the standard of law.

For the profession, the lesson is worth carrying into every AI decision, whichever tools a firm ultimately chooses.

Ask not only what the AI can do, but where it does it, what it is grounded in, who and what it can access, and whether you could stand up in front of a client, regulator or court and defend the work.