Hiring a Private AI for Your Law Firm

By David Steiner, Partner

‍

Remember those vintage film noir detective films? Someone enters a door labeled “Private Eye” and unburdens their secrets to a private investigator who’s sitting behind a big desk, smoking and staring out the window. 

‍

“Why can’t you take this to the police?”

“Because it’s a secret and I can’t trust anyone but you!”

‍

For many law firms, that’s the case when it comes to their use of artificial intelligence (AI). Your staff is using tools that you haven’t approved. They’re running up costs on general models. Clients are asking questions that you can’t answer. And all of this may be opening your practice to compliance and regulatory issues that your insurance company wants to chat about.

‍

Many attorneys I talk to aren't really concerned with security. They think they've got it buttoned up. Instead, they’re thinking about increasing efficiency with AI. Later, when they start talking to us, they realize it’s not purely about efficiency. 

‍

They realize that they really do have a lot of secrets they can’t share with anyone else.

‍

BigLaw is Using BigTech. Is that a BigDeal?

Recent industry news is buzzing about legal giant Latham & Watkins building its own private AI system to keep its most sensitive client data in-house. Latham employs 3,700 lawyers and 900 technologists, according to an article in The American Lawyer. They recognize both the power and the risk that AI represents. They’re investing an undisclosed sum into their business to un-risk their AI usage. 

‍

But even if you run a smaller, 10 -person law firm, you can still benefit from a scaled down version of private AI. You don't need the same processing power as a larger firm. In fact, you may be able to run it on existing hardware or a smaller server, which means this deployment might be a fraction of the investment.

Feeling AI Vertigo? You’re Not Alone.

First, let’s demystify some of the terms around AI that make it seem like a smoke-filled room.

Public AI tools like ChatGPT, Google Gemini, Anthropic Claude, and others are considered generative AI systems that also offer some level of “agentic” or independent activities. Each of these is a proprietary, closed large language model (LLM). Your law firm should be very careful about placing any client data into a public AI prompt or allowing agents access to that private data.

‍

What Latham is building, and you can as well, is a “private AI.”

‍

Latham runs large open-weight models, but a smaller firm can run an open-weight small language model (SLM). This is a model that can be installed into a computer or server that only you and your IT provider access and manage. Like their larger cousins, these models are generative and can support agentic workflows. However, your client data remains secure and it never leaves your systems.

‍

A private AI can grant your law firm many new capabilities without ever opening protected client data to the world. In fact, I think of it as not “artificial intelligence,” but “augmented intelligence” in the way it now augments or significantly increases what your staff can do for clients .

‍

With a private AI, you can learn from previous matters that today may be static case files. Your staff can quickly look for patterns in contracts. Or they can review and synthesize large volumes of depositions. 

‍

Thinking Fast and Slow

In fact, even though they run slower, they work faster. 

‍

How is that possible? Because now that your staff no longer needs to spend time redacting and redrafting case files, they can more quickly feed or request data. You no longer need to worry about redaction. You don't have to worry about size. You don't even have to worry about token usage. All of this increases your staff’s efficiency.

‍

Your practice maintains secure, confidential, private data and with zero trust network access (ZTNA), even remotely accessed systems remain secure.

‍

Consider a hypothetical personal injury case. Your IT provider or AI consultant could build you an agent that reduces your financial analysis of a case down to three minutes of attorney time. You feed your documents to your system and prompt, “Analyze this and tell me the value.” You can walk away and get a report an hour later.

‍

This is the absolute power of private vs. public AI.

‍

I'm not a lawyer, and a private AI doesn't change your ethical obligations to your clients, so have that conversation with your counsel before the first matter goes in. 

‍

According to opinions from the American Bar Association, NYS Bar, and New York City Bar, your client’s informed consent may be required before using their information with generative AI. Those concerns can extend to systems even if they are used only within your own firm, depending on applied safeguards and the risk of exposing information across client matters.

What Changes When You Own the AI?

Unlike a public AI system, your private AI system behaves exactly like the rest of your network. 

‍

Consider today what happens when staff moves on to another firm. Your IT team revokes their access to internal documents and systems. However, an outside AI account logged in through an employee’s personal account can’t be as easily controlled. 

‍

As your technology provider, when we secure and lock down your network after someone departs, your private AI is locked down by design. 

‍

Additionally, when you own the AI system, you also control how it’s configured and what prompts are accepted. Your AI consultant will handle updates to your SLM and your technology provider will maintain the hardware and systems access. Obviously, this isn’t something for a small break-fix IT provider, but a more sophisticated and strategic technology infrastructure provider.

‍

Where Do You Begin?

Private AI won't fix bad habits. Everything we do still comes back to the human firewall, because your people are your first and last line of defense. Your law firm's AI journey begins with policies and procedures. A policy is only as strong as its enforcement, and that rests with you. 

‍

Today, your AI policy tells staff where they can’t go with client data. With a private AI, it can also tell them where they can. You're not asking people to change how they work. You're pointing that work somewhere safer. 

‍

When we work with a client, we take onboarding and discovery very seriously. We often invest our first 90 days into cleaning up and managing any existing challenges. Only then do we start talking about what you want AI to do for your firm, because with AI, garbage in equals garbage out. 

‍

One last question worth asking yourself.

If an associate left tomorrow, would their AI access leave with them? 

‍

Cypress Grove Technologies is the Strategic Growth Infrastructure and Risk Mitigation Partner for professional services firms in New York. AI should be a strategic investment in your business’ future. If you’re considering how strategic AI use can benefit your law firm, we're happy to discuss it with you.

‍

Schedule a conversation->

‍