#206 - How lawyers are using AI in 2026

People

Companies

[00:00] I really is a unique moment.
[00:05] I mean, professionally, I don't know if we're ever going to see this kind of disruption,
[00:10] and I mean that in the positive way.
[00:12] Ever coming before in the legal industry and certainly going forward, if you don't get
[00:17] in front of a lot of these trends, unfortunately you might get left behind.
[00:20] So I not only see it as an opportunity, but also as an obligation.
[00:23] That was Sujit Rahman, whose name the most impactful general counsel of 2025, or his work
[00:29] because chiefly glyphs are of TMM labs.
[00:32] So, as I mentioned, the obligation that we have to stay ahead of AI as a legal profession.
[00:37] You don't need me or anyone else to tell you that AI is important for lawyers or that it makes things more efficient.
[00:43] That's not what this podcast is about.
[00:45] So, Gis is one of the eight lawyers and CEOs who I spoke with for this podcast,
[00:49] which is about how those at the cutting edge of legal AI are getting ahead
[00:54] and how you can get the most out of it in 2026.
[00:57] Welcome to the law of code podcast.
[00:59] I'm Jacob Robinson, and by the end of this episode,
[01:01] I promise that you'll understand how the lawyers
[01:04] at the cutting edge are actually using AI today.
[01:07] It should be the most helpful legal podcast
[01:09] on this subject that exists.
[01:11] That's the standard with every episode that I publish.
[01:14] And if it's not, let me know how I can improve
[01:16] and I'll make it better for you.
[01:18] That's all I care about here is you.
[01:19] I want this podcast to be helpful for you
[01:21] and not just because you're so good looking, I swear.
[01:24] Now, this podcast has four main themes
[01:27] when it comes to AI.
[01:28] Well, start by talking about mindset shifts.
[01:30] We'll talk about how lawyers are using AI tools today
[01:33] with real world examples, risks to be mindful of
[01:36] and how these tools actually work,
[01:38] as well as how you can better leverage AI.
[01:40] And some are so helpful that I hope you'll actually
[01:43] pause the podcast at some points to start using them yourself.
[01:47] Now, I think it's an amazing time to be a lawyer.
[01:49] That was a sentiment shared by everyone
[01:51] who I interviewed, including Scott Stevenson.
[01:54] is the CEO of legal AI contract drafting tool spellbook.
[01:58] Here's Scott now on why he thinks it's the best time to be a lawyer.
[02:02] I think he was genuinely the best time to be a lawyer that there's ever been for a few reasons.
[02:06] One of the reasons I co-founded spellbook was because almost every lawyer I knew,
[02:12] privately admitted to me over a beer that, wow, like, practicing a law wasn't quite what I expected.
[02:16] Like the amount of time I'm spending in front of Mike Microsoft,
[02:19] we're especially on the transactional commercial side.
[02:21] You know, a new number of very passionate lawyers
[02:25] who had become dissolution with the job.
[02:28] And then after AI came along broadly,
[02:31] so many of these people were like,
[02:33] I actually love my job now.
[02:34] This is a life-saver.
[02:35] I'm actually really excited for the rest of my career
[02:39] because I'm not going to have to spend hours
[02:41] and hours a day just like copying and basing a word.
[02:44] Now, there's so much about practicing law that I love.
[02:47] and same with every and I interviewed here.
[02:49] They see AI as something that doesn't mean less work.
[02:52] It means less mistakes, faster client communications,
[02:55] and a more consistent work product.
[02:57] But unfortunately, what comes to mind when you first hear
[02:59] the words, AI and lawyers, for many, it's hallucinations,
[03:03] were a risk-sensitive profession after all.
[03:07] Here's the actual paraphero.
[03:07] He's the founder and managing partner at Reigns LLP,
[03:10] to explain what most lawyers think about when it comes to AI.
[03:14] Here's that.
[03:14] The first thing to say is when lawyers who think about AI, the image that I think most readily
[03:23] comes to mind for folks are these scare stories about litigators getting sanctioned for
[03:27] citing hallucinated cases, including the famous example where a Sullivan Gromwell, apparently
[03:32] did this in a bankruptcy case.
[03:35] And on the one hand, I find myself sort of frustrated by that because this so clearly to
[03:40] me is human error and not AI error.
[03:42] Like how could you ever submit a brief to a federal judge without having checked the citations?
[03:48] Whether or not the AI is drafting it, even if you have a junior associate drafting it,
[03:51] you don't just pass off the work product.
[03:53] But I do the deeper story is that because AI output looks so polished and convincing, it presents
[04:01] a real temptation to turn your brain off and abdicate your sort of mental labor to the tool.
[04:07] And to more direct answer question, I think the right way of using AI is the exact opposite of
[04:11] that.
[04:12] It's to do all of the mental struggle and work and cognitive labor, everything you would have
[04:17] done beforehand.
[04:18] And then use that as the input in the AI.
[04:20] And then that makes the tool incredibly powerful.
[04:24] And I think if you're doing it right, you should actually more exhausted at the end of the day
[04:27] having used AI all day than you did before, because you're spending more time on the like
[04:32] of the layering and less time on formatting citations
[04:38] or all the stuff that can be sort of more easily
[04:40] automated away.
[04:42] Zach made some great points.
[04:44] I think most important is how a lawyer's time
[04:47] is spent differently when they're using a tool like AI.
[04:50] More time is spent on judgment, on strategy,
[04:52] on thinking, and less on the repetitive tasks
[04:55] that can be automated with AI.
[04:57] Now, if you use AI or you have clients who do,
[05:00] You should probably know about the presenting sponsor of today's episode altitude.
[05:05] Not only does altitude offer a command center for payments, you can use it to manage entities
[05:09] across jurisdictions from one spot instead of a bunch of different accounts everywhere.
[05:14] They also offer unlimited credit cards and elevated cash back or AI spend.
[05:18] So if you're listening to this and you have clients that use digital assets that want
[05:23] to transact, not only in stablecoins, but feel as well on one place.
[05:26] It's worth taking a few minutes to visit altitude.xyz forward slash law to learn more.
[05:33] If you are using AI in your legal work, like almost everyone is these days, it's important to
[05:39] remember why. A more efficient system, right? That's sort of talked about as the goal, but I don't
[05:44] think that's it. A more efficient system is the buy product. The goal should always be the same
[05:49] to better serve your clients. David Wang, the chief innovation officer at the law firm,
[05:54] and then it's a really cool thing.
[05:56] And then it's a cool thing.
[05:58] And then it's a cool thing.
[06:00] And then it's a cool thing.
[06:02] And then it's a cool thing.
[06:04] And then it's a cool thing.
[06:06] And then it's a cool thing.
[06:08] And then it's a cool thing.
[06:10] And then it's a cool thing.
[06:12] And then it's a cool thing.
[06:14] And then it's a cool thing.
[06:16] And then it's a cool thing.
[06:18] And then it's a cool thing.
[06:20] And then it's a cool thing.
[06:22] It's undeniable that AI improves efficiency.
[06:25] I always think of the efficiency improvement as a buy product.
[06:30] What you really want to do is you want to use AI
[06:33] to improve the quality of our services to our customer.
[06:37] And if that is happening, I kind of don't care
[06:40] what else is happening.
[06:41] Is that makes sense?
[06:42] Because there's this fear of lawyers out there.
[06:45] And that's like, oh, if I use AI to do X, Y, and Z,
[06:48] it's going to take away some of my billable hours.
[06:51] very much like, you know, the fear that I had and as associate of life and automation is going to take away.
[06:56] But like, if you throw all that away and truly have the courage to embrace that ultimately,
[07:02] both from a professional perspective, like, what is the soul of the lawyer?
[07:06] What is the most important thing that lawyers do is to serve a client?
[07:10] And from a commercial perspective of laws of business, what's the most important thing to do?
[07:16] is to serve your clients, right?
[07:19] And then you throw everything else away and you just like,
[07:21] whatever it's gonna take and whatever I can do
[07:24] to make a better quality service for my clients is what we're gonna do.
[07:29] Then you arrive at the inescapable conclusion
[07:33] that you must use AI in this way.
[07:37] And by the way, it also tells you when using AI doesn't make sense, right?
[07:41] Because if you try to use AI and it doesn't improve the quality of your services,
[07:45] of your services, then you don't need to use it.
[07:47] You can use it, do it in the manual.
[07:49] It's not like AI is the answer to everything.
[07:52] Right?
[07:53] So that I think that's to me, like the most important pillar of it.
[07:56] It's such a good reminder that what we're doing here is practicing law.
[08:00] We're not trying to be as quick as possible.
[08:02] We're trying to do the best work we can.
[08:04] And the same thing applies to any lawyer, obviously, whether in private practice or in
[08:07] house qualities, what people want AI doesn't change that, but it does change how you get there.
[08:13] many people see it as augmenting our work.
[08:16] That it'll be something we turn to when necessary.
[08:18] But not Molly Abraham.
[08:20] Molly is the general counsel at Coinbase and engineer
[08:23] turned lawyer who has a pretty forward-looking approach to AI.
[08:26] Here's my back and forth with Molly on why many people are thinking
[08:30] about AI all wrong.
[08:31] Those who are thinking about AI as augmenting,
[08:35] their current workflows and efficiencies are thinking about it all
[08:39] wrong.
[08:39] my opinion, I think it's really has an opportunity to upend what we're doing.
[08:44] But one of the things that I found is that myself and my team had all of these great ideas
[08:50] of how we would automate certain workflows, how we could shift people from working on
[08:55] something more high-value and strategic and automate the majority of their role.
[09:01] And we were getting stuck. The number of times I've been in a terminal window and reading code
[09:06] and trying to get my token key to work and our amazing engineers at Queen B's are always
[09:11] willing to hop on the boat with me, but I was hitting a lot of brick walls. And so I don't have
[09:16] an expectation that my team, all of the sudden, all becomes engineers. And so I want just like we
[09:23] go to outside council for specialized expertise. This is my specialized expertise for how to build
[09:29] these agents for envisioning. I think it is critical though that it is embedded in the legal team
[09:35] and here's why. I think that for an engineer to be able to be successful in supporting a legal
[09:43] team, you also have to deeply understand how the legal team works and what their workflow is like,
[09:48] what is our red line? The first time I showed and kind of an engineer I'm called,
[09:52] I'll hate and you help me figure out how to automate something around red lines.
[09:59] Just the concept.
[10:00] that blew them away, it was just unfamiliar.
[10:02] And so I think it's critical that not only you have someone with the technical capability to build these
[10:07] agents, but they're embedded in working with the teams so that they understand how they work.
[10:15] Otherwise, we won't reinvent how we work in the first place.
[10:18] And I want to touch on one thing you mentioned there, the idea that these won't necessarily
[10:23] just augment, but they'll up and how we do legal work. And the word augment has come up so many times
[10:29] things will have been doing this episode and talking with lawyers, but how they're using AI.
[10:33] Everyone likes to say that they're using it to augment their work.
[10:37] Walk me through why you see that as being the wrong way to look at it.
[10:40] I think we can really revisit whether the work flows make sense in the first place, and which
[10:45] work flows need to be done by humans.
[10:48] Which ones when done by humans are just creating coordination tasks?
[10:53] I think we can't underestimate what we can replace with AI.
[10:59] I'll give you an example.
[11:00] My superpower, I would have told you six months ago,
[11:03] was drafting.
[11:04] I can draft for a complex legal topics
[11:08] for a non-leal audience and make it make sense.
[11:10] And that's the thing that I was known for,
[11:14] and I would always have a heavy pen
[11:17] in editing my team's work.
[11:19] And I asked myself, can one of it agent could do this?
[11:22] I said I thought they can't.
[11:24] There's no way.
[11:25] Like this is like my true,
[11:27] so this week we get super power.
[11:29] And I haven't eaten any of my writing now
[11:31] and editing my team's work.
[11:33] I've gotten unsolicited feedback
[11:35] from other cross-functional partners.
[11:38] Wow, so what's so's writing has really improved.
[11:41] Don't talk.
[11:41] It's the age I ain't.
[11:42] And that's okay.
[11:44] And I have to get comfortable with the idea
[11:46] that something I considered an N of 1 attribute of my own
[11:51] with something I could automate, I could train, I could improve.
[11:55] And what I filled my space with is more coaching of the team,
[11:59] more strategic and innovative lawyering, all of these different things.
[12:03] And so having an open mind, I think is the first step in the process.
[12:07] I think as lawyers, we'd been in any profession.
[12:10] We'd all like to believe all AI can do is augment what we're doing,
[12:13] because it did self preservation.
[12:16] But I think that if you really, really want self preservation,
[12:19] and you have to have a more open mind now,
[12:21] because someone else will.
[12:22] And they'll find a way to really, really up-end workflows.
[12:26] What's the biggest opportunity in AI that most lawyers aren't
[12:30] thinking about enough talking about enough?
[12:31] Like, what would you say, stands out to you there?
[12:34] I think that some people still have a tendency to think,
[12:40] the same way I did about my own writing, oh,
[12:43] AI can't replace my tasks.
[12:45] And I think there's a little bit of self-preservation there.
[12:48] I think the biggest opportunity is for those who ask themselves,
[12:53] how can I replace 99% of what I do with AI?
[12:57] And then have trust that we have so much else to do
[13:02] as so much more opportunity.
[13:04] And so I think a lot of people have gotten over
[13:06] that very little bit from that everyone.
[13:09] And I think those who look for a way to truly replace
[13:14] themselves with AI are going to be those
[13:16] to have the biggest and brightest careers?
[13:19] Well, when you're saying that,
[13:22] it just made me a little nervous, right?
[13:23] Like the idea of replacing yourself with the eye,
[13:25] makes people a little uncomfortable,
[13:27] but you need to lean into that because other people are,
[13:30] and you're freeing up your time to do the things,
[13:33] I think you're uniquely suited to do.
[13:34] And one of those things that you've mentioned,
[13:36] strategy, I think decision-making is only gonna
[13:39] grow in importance now because of this extra leverage
[13:41] that we have with AI.
[13:43] How do you sharpen your decision-making skills?
[13:46] I think it goes back to something you just said,
[13:48] which is effectively making yourself dispensable.
[13:53] And I think part of the reason I have an easy time
[13:56] thinking this way is it's how I've always managed my team.
[14:00] The more I think there are a lot of lawyers who might
[14:04] view themselves as I need to be indispensable.
[14:07] Everything will fall apart without me.
[14:09] I have the opposite view.
[14:10] I am routinely trying to find ways to make myself dispensable
[14:15] and to say if I got hit by a bus or on the lottery tomorrow,
[14:19] the team would operate just fine.
[14:21] And I think you have to think the same way with AI.
[14:24] And when you do, what has inevitably happened to me
[14:27] as I continue to say,
[14:29] I don't need to go to this particular critical meeting.
[14:32] Someone on my team is now fully capable of doing that.
[14:35] And that's a good thing.
[14:36] That's because I've created a phenomenal bunch of leaders.
[14:40] I'm also going to create a phenomenal bunch of agents.
[14:42] And that is only going to give me more space to make
[14:46] our decisions to make those strategic calls
[14:48] and to help piece everything together
[14:50] in a way that's really impactful.
[14:52] So I do think that disability mindset is critical
[14:57] as a leader generally and also with respect to AI.
[15:01] All I made a really good point
[15:02] can sort of hurt the ego a little bit to admit
[15:04] that an LLLM can do something as well as you can.
[15:07] When I was working as those securities
[15:09] and corporate lawyer, I would pride myself
[15:11] not making the stakes and of course, I never made a single one ever.
[15:14] I never sent an email that said, see attached without attaching the relevant document.
[15:19] Or maybe I did, I forgot anyways, I doubt you've ever made any mistakes in your life,
[15:23] but other lawyers definitely do. The team at the spellbook actually dug into contracts on
[15:28] Edgar, the SEC's filing system and found the stakes in a ridiculous number of documents.
[15:33] The percentages might be shocking. Maybe to non-loirs, I think lawyers will understand having
[15:37] and read all these documents, but the percentages are quite high.
[15:41] Here's Scott Stephenson, the CEO of Spellbook on their findings.
[15:44] So we launched a report a few weeks ago,
[15:47] as called Human Solutions A2,
[15:49] where we scanned thousands of agreements that were submitted to the SEC's
[15:54] Edgar Database and we used Spellbook to determine,
[15:57] you know, how many of these had objective mistakes in them.
[16:00] So we're not looking at, you know,
[16:01] wasn't negotiated poorly or anything like subjective.
[16:04] We're looking at objectively,
[16:05] how many of these contracts have bad section references or conflicting terms that can't
[16:11] fully each other out and things like that. And so we ran about 50 to 60,000 pages of contracts
[16:17] through spellbook from Edgar and surprisingly we found that 60 contracts filed Edgar had mistakes.
[16:23] His audio cut out a bit at the end there but Scott said 60% of contracts filed an Edgar had
[16:29] objective mistakes, mistakes that were cut by spellbooks AI tool. That's about 33,000 contracts
[16:35] with mistakes. And contract drafting is one way that Molly Abraham is using AI in her role as
[16:41] general counsel. She's also built a writing agent that drafts and refines internal legal documents,
[16:47] translating legal analysis into something of business audience, so the people she works with in
[16:52] house can actually use. The result is that instead of documents coming to her from say the business
[16:58] team or a counterpart, she doesn't have to spend as much time redlining everything that comes
[17:04] toward desk. Here's Molly to explain how she's built that. So I created and trained in each
[17:09] edge that understands frameleagal lens how to translate a legal drafted document to a more
[17:19] you know business facing audience and most of my initial drafts now come from this agent. My team
[17:26] runs all of their drafts through the agent. I went from spending a huge talk of my time reviewing
[17:32] documents before they would go to a more senior audience to at most I can review them and I
[17:37] might have one red line which is just I never would have predicted it a year ago.
[17:44] Another example that I just on on that point. How does it work so you'll get the let's say you get the
[17:51] contractor you get the the redline back from the counter party. Do you send that to the agent
[17:56] as the agent automatically pull it? What does the workflow look like there? Sure. So this agent
[18:02] is really about writing an internal document as opposed to modifying a contract. Go.
[18:06] Moth. The modifying the contract and dealing with redlines turns out to be quite technically complex.
[18:12] So once I hire this person, I'll get back to you on that one. So I have an agent that I built in
[18:18] lead record chat and it will, one of the things that we've done really well at Coinbase is it will actually route to the LLM model that is most efficient in terms of total usage, which is part of power able to increase our AI usage without increasing our AI spend at the same rate.
[18:34] And I will give it ideas.
[18:36] I'll say I really need to draft a document about these particular topics.
[18:40] Here's two or three other note stocks that happen to be created or check this Slack channel
[18:46] and it will pull all of it together.
[18:47] It will draft the document alternatively.
[18:50] I can give it a draft a document and I can say please clean this up to make it sound more like my voice.
[18:56] And one of my strongest tenants as a writer is Mark Twain's,
[19:03] if I had had time, I would have been a shorter letter.
[19:06] I get a lot of log, I used to get a lot of log documents from my team,
[19:10] and I think it's really important to keep it short and concise,
[19:13] and I think that's actually takes way more time.
[19:16] And so I'll get it output as a Google Doc.
[19:19] I might have one or two more tweaks.
[19:21] I might go back and forth with the agent and say,
[19:24] I didn't want to frame the recommendation this way.
[19:26] please edit accordingly or please additionally pull an information from this document. It's able to read both across Google Docs and also across Slack if I pointed in that direction to draft the entire document and it's really fast.
[19:41] What Molly just described there, a system where you can bring in an external document and compare it to a playbook or a baseline is one of the most consistent uses I heard from lawyers who are maximizing their use of AI or AI.
[19:55] use of AI or AI Maxing all color, token Maxing.
[19:58] They're starting with a precedent or a gun.
[20:00] it lines on a repetitive document and having AI draft based on that.
[20:04] It's one of the highest benefit use cases of AI today.
[20:08] Now, why is AI so good at building off a precedent as opposed to starting from scratch?
[20:14] That's because of how an LLM works.
[20:17] Let's take three minutes now to walk through how most LLM's work.
[20:21] Here's Samsung, Anzer, a partner at K. Hold Gordon and Rydell,
[20:24] when thrilled to share is a sponsor of the Law of Code Podcasts.
[20:28] Instead of response, or though, Samsung lose Cohen and the team, they're actually joined the episodes.
[20:32] In this clip from an interview, we did Sam makes a really important point that is often forgotten,
[20:37] which is about how the popular LLM chat box and legal AI tools actually work today.
[20:43] I, as we discuss it, where we're talking about really this conversation is about LLM based solutions.
[20:51] Those are basically predicting what's the most likely response or what's the most likely answer
[20:57] based on this information.
[21:00] So that is not really a judgment thing.
[21:04] That's really just collating data and presenting.
[21:08] Let's just talk about the terminology for a second.
[21:11] When people say AI in 2026,
[21:13] they almost always mean large language models or LLMs.
[21:17] That's what Claude and Chad Chibitier.
[21:19] There are other kinds of AI, but the current wave and what we're mostly talking about today are LLMs.
[21:24] As Samson said, LLMs are prediction machines.
[21:28] Here's how they work in five steps.
[21:30] So first, you give it a prompt.
[21:31] That's whatever you type in the box.
[21:32] It could be draft and NDA between two software companies.
[21:36] That's your prompt.
[21:37] You hit enter.
[21:38] What happens next is the model breaks your prompt into tokens.
[21:42] The rough rule of thumb is that one token is about three quarters of a word in English.
[21:46] So a short word like draft or the or contract is usually one token.
[21:52] a longer, less common word, like in demenification,
[21:55] might get split into three or four tokens.
[21:58] So, you've got your prompt.
[21:59] You've typed in draft and endier between two software companies.
[22:03] That goes to the model, which now breaks it into tokens.
[22:06] So, you've input your prompt, the model's broken it into tokens,
[22:09] what happens now?
[22:10] The next step, and this is the third step, is the model predicts the next token.
[22:15] So, based on your prompt, plus everything it learned during training,
[22:18] and we'll talk about what training means in a bit,
[22:20] the model runs the math on every possible next token and picks one.
[22:24] So the model will look at your prompt and say, okay,
[22:26] given everything I've seen before, what's the most likely token to come next?
[22:30] So if your prompt is draft an NDA between two software companies,
[22:34] it will start that based on how it's answered similar questions before,
[22:38] find prior examples and use those to predict what happens next.
[22:43] That's why these tools can produce something called hallucinations.
[22:46] It sounds confident, it looks correct, but it's wrong.
[22:49] It's because models aren't picking the next true token.
[22:52] They're not exercising judgment on what's real.
[22:55] They're predicting what the next token should be.
[22:58] So once they do that, once they predict what token should come next,
[23:02] they add that token to the text and then begin predicting the next one and the next one.
[23:07] And the next one.
[23:08] Every token it generates becomes part of what it looks at for the next prediction.
[23:13] So the model doesn't stick back, think about your whole question and plan or response and then write it.
[23:17] Like we do, if someone asks me a question,
[23:20] I think about what my answer is going to be
[23:21] and then I began speaking.
[23:23] What the model does is it guesses one token at a time
[23:26] over and over and over.
[23:28] So it'll look back at the whole thing,
[23:30] and it'll add a new word.
[23:31] So given the lawyer filed the motion,
[23:33] what's the most likely next token?
[23:36] Maybe it picks two.
[23:37] So now the text reads the lawyer filed the motion two.
[23:40] So it keeps going, and this is the final step.
[23:43] It keeps going until it decides it's done.
[23:45] The model has learned when to stop usually when it hits a natural end or a length limit.
[23:51] So what looks like a thoughtful, complete response is really just thousands of these one token
[23:56] predictions stitched together in real time.
[23:59] That's why the model is not actually thinking ahead, like you and I do when we plan a response,
[24:04] it's predicting one token at a time.
[24:08] So that's the brief outline and a very high level of how an LLM works.
[24:13] I mentioned training earlier and training is such a big part of this because there's a reason
[24:17] answers actually feel so coherent.
[24:20] Reasons answers have been getting better since chatGPT launched a few years ago now.
[24:25] That answer is training.
[24:28] Training is the reason that AI feels like it's thinking even though it is just predicting.
[24:33] Training is how a model learns to be better at predicting.
[24:36] It's why chatGPT has improved so much since it was first released a few years ago now.
[24:41] The model starts out with billions of internal settings called weights, which you can think
[24:46] of as tuning knobs.
[24:48] At the start, when you have a guitar and tune the guitar.
[24:51] So at the start, every one of those knobs is sent to a random number.
[24:55] It gets fed in enormous pile of text, books, court decisions, websites, code, basically the
[25:00] entire scrabble internet, and a lot of data.
[25:03] What it does to train is look at a chunk of text with the next word hidden and it tries to predict
[25:09] what that next word will be.
[25:11] it checks the answer. It does that trillions of times. And every time it guesses wrong,
[25:15] it's slightly adjust its weights, it's tuning knobs to not make that type of mistake again.
[25:21] Every time it guesses right, it reinforces whatever it just did. Okay. So we've got LLMs,
[25:26] their prediction machines, they get trained by tuning their answers compared to what the actual
[25:32] answer was. After trillions of times, the LLMs start getting real patterns. They start getting smarter.
[25:39] That's what the weights are. They're just the accumulated result of trillions of tiny adjustments.
[25:45] So two models can feel similar with the same chat box, but the quality of what comes out is entirely dependent on three things.
[25:52] First, what they were trained on to how much they were trained and three how their weights got tuned along the way.
[25:58] That's why customized tools like those notebooks building or those Molly's building can outperform a general purpose chat.
[26:05] purpose chat about or not. It really depends on the underlying training that's happened.
[26:10] Of course, no lawyer wants to be or should be the one who lets their work product, sensitive
[26:16] client information be used by LLMs for this training. Since that could mean the LLM could
[26:21] regurgitate the sense of information. It's been trained on to someone who prompts for it, which
[26:26] is why companies like Harvey Lagore and Spellbook don't train their models on lawyers data.
[26:31] Here's a snippet from my conversational Scott Stephenson, CEO of Spelwood, we explained how that works.
[26:36] I know you've negotiated agreements with OpenAI with Anthropic for zero data retention.
[26:41] So that means customer data included in requests and responses with the LLMs only exists in memory to process the request.
[26:50] They're not trained on that after.
[26:51] And then you also use Cloud Provider with data centers and candidate in the US for storing and processing customer data.
[26:57] or data, you're going to have a trust center on your website,
[26:59] which I thought was a fantastic idea.
[27:01] When people go in and see all these vendors
[27:04] and see who you're working with there,
[27:06] what are some questions you get about security from lawyers
[27:11] and how do you as your team think about that?
[27:13] I mean, I think the number one thing that lawyers care about
[27:17] is that we're not training AI models on their data,
[27:19] because once you train models on private legal data,
[27:24] the model could spit that client data back out again.
[27:28] And that's like the worst possible thing that could happen.
[27:31] And the problem with training is actually not a very good approach.
[27:36] We did training and fine tuning AI models in the early days
[27:40] when we launched back in like 2022, 2023.
[27:43] There's a bunch of reasons why training on legal data is not good,
[27:47] but one of them is it can allow that data to be regurgitated
[27:51] in all sorts of weird ways.
[27:53] That's the main thing customers want to know.
[27:56] And we don't, yeah, we don't, we do not train on any customer data whatsoever.
[28:00] And that protects, you know, that risk.
[28:02] Yeah, but we're, we're hip-hop compliance.
[28:04] So we sell to like hospitals and healthcare providers.
[28:07] We sell to one of the largest consumer banks in the world.
[28:10] So we have to have very, we have a very high bar for security and privacy.
[28:15] And I would say, you know, people want to see that, you know, those controls for SOC to
[28:19] in HIPAA, you know, standard controls put in place.
[28:22] They'd want to understand where their data goes and doesn't.
[28:25] And generally, no one wants the data to be sent to the model providers and for them to use it for training,
[28:32] which we ensure we have zero data retention agreements with all of our model providers.
[28:36] Scott, just on that point, you can kind of ask you about that because I don't know that the details of how that works inside a node.
[28:42] So what would happen?
[28:43] What would be happening is I'd be using spellbook.
[28:45] And then if safety that is sent to Anthropoccer,
[28:48] open AI are one of those third-party LLM providers,
[28:51] that data processes through the LLM,
[28:54] and then is deleted.
[28:55] Is that what happens?
[28:56] Like after the answer is spidote?
[28:58] Yep, that's right.
[28:59] So it's just a round trip,
[29:01] and then they do not store the data for any kind of other purpose after that.
[29:05] Now, they do have sort of like a window of time where the data
[29:09] may be privately stored for moderation purposes,
[29:13] is like to detect abuse and people doing things that they shouldn't, but other than that, you know,
[29:18] the data is deleted and not used for training. Now, if you're not using a tool like Spelbook,
[29:24] but a general purpose model like Claude or Chattich, PT, and you're not turning off the training data,
[29:31] what happens with the information you input. Heck, even if you are turning off the training tab,
[29:36] you're not necessarily always protected. Eric Dylis is an attorney and programmer. He explains
[29:42] how there are other risk factors,
[29:44] for sensitive client information,
[29:46] not just model training,
[29:47] that lawyers should be aware of.
[29:48] You start with the example of what happens
[29:50] when you submit a prompt with your data.
[29:52] Right. So, when you submit a prompt,
[29:55] or you start a workflow with AI that uses a model that's not...
[30:00] on your computer. That is remotely hosted and a server not in your room, which is just about everybody.
[30:07] The way that these AI service providers that open AI and Thropic, Jim and I, they, they,
[30:15] they now really hammer the fact that they're not using your inputs to train their models.
[30:20] And they're great for privacy for that reason. That is just a sliver of the problem.
[30:25] Another big part of the problem is whenever you submit that prompt or start that session,
[30:29] your input is getting logged on their servers.
[30:32] Separately from the model, it's not getting trained in the model, it's getting logged on their
[30:35] infrastructure at a for a minimum of 30 days for safety, quality assurance, legal requirements that are
[30:44] intentionally very opaque. But they're storing your inputs, they're storing information,
[30:49] your metadata, anything and everything that is attached or accessible to what you provide or
[30:55] or expose. And it's maybe not the same exact risk factor as model training because models
[31:02] they'll have a more difficult time regurgitating. But your information is still on someone else's
[31:07] server, subject to potential exploits, subject to all kinds of potential issues. And that logging
[31:15] also occurs on the way out from the model, so it's not just your submission, it's also with
[31:19] the model provides back to you. Now, as Eric O. Line there, sensitive line information is always
[31:24] at risk. If you're using a server located elsewhere or not on your laptop, since that information
[31:29] must travel on the internet rails and is capable of being intercepted or stored. Eric created
[31:33] camel text. It's an offline desktop application that detects and relax personally identifiable
[31:40] information, PII, and sensitive text before sending data to AI models.
[31:45] It's pretty cool. It's worth checking out. It's called Camo Text.
[31:48] Another thing you should know about Eric is that I'm in a fantasy football league with him,
[31:51] and I won the league in 2025. Sorry. I just couldn't help myself.
[31:56] Camo Text is a fully offline app that works on any laptop.
[31:59] Here's Eric explaining how it works.
[32:01] So you load a file document, text, or a folder full of them,
[32:05] a plan to use their settings you want or not,
[32:08] hit anonymize and a combination of custom algorithms,
[32:13] NLP, et cetera, detect sensitive terms,
[32:15] names, also common patterns like emails, GPS coordinates,
[32:20] eSignature IDs, anything that is reasonably identifying
[32:24] and replaces it with a labeled and hashed placeholder
[32:27] in new output text.
[32:29] And so what that output allows you to do
[32:31] is supply it as context or as part of a prompt to an AI model for analysis for summarization
[32:39] for all the common use cases that lawyers and other professionals use for AI
[32:44] and preserves confidentiality in that way.
[32:46] Now Eric explained how he ensures client confidentiality when using AI.
[32:50] That's something that's a huge risk for lawyers.
[32:53] As we've seen cases where clients have put privileged information into a consumer AI tool
[32:58] and lost the ability to practice it in court.
[33:00] Well, at least we've seen it when non-loyers have.
[33:03] The clearest example is United States v. Heppner.
[33:06] It's a case that was decided in the Southern District of New York in February of 2026.
[33:11] I covered that decision at length in episode 186 of Law of Code with Mike Katz.
[33:17] Basically, Judge Rakeoff held that communications between the defendant and Claude
[33:21] were protected by neither the attorney client privilege nor the work product doctrine,
[33:26] because privilege never was created in the first place.
[33:28] Conversations with somebody who's not a lawyer cannot be a turning client privilege,
[33:32] those sometimes there are other forms of privilege that apply if their doctor, for example,
[33:37] bought his told me it's not a doctor.
[33:39] However, in a different case, Warner V. Gibb Bralco, Jill Barco,
[33:44] I'm not sure how to pronounce his name, Barco, I think it is.
[33:47] ChatchipiT outputs were actually protected as a turning work product,
[33:50] because a self-represented litigant had prepared using a public AI chatbot,
[33:55] and the court found that the work product protection applied.
[33:58] So it is unclear where the line is
[34:01] and that makes it more important than anything to minimize the risks here.
[34:05] I asked Molly Abraham, General Counselor Coinbase,
[34:07] more about the legal risks of AI specifically,
[34:10] how she thinks about guardrails when it comes to using AI with her team.
[34:14] How do you think about implementing guardrails when it comes to using AI with your team?
[34:20] This is great and such a perfect opportunity to talk about how I started to think about AI.
[34:26] Because I'll be honest, when I started to think about this two years ago, I started with the risks.
[34:30] I didn't yet see the opportunity and it's evolved my thinking in terms of what every legal leader needs to take into account.
[34:38] So the first and foremost is you have to protect the company.
[34:42] And part of the guardrails here include, I don't know if you remember a couple years ago, a major service provider.
[34:49] provider, all of the sudden sent out an email, notifying folks, oh, when your data may be
[34:55] used in LLM training. That was a big red flag and it was confidential company data that
[35:02] they were going to use to train their own LLM to be able to effectively use that with clients.
[35:09] And the very first thing that we did is we scrubbed across all of our key vendor agreements
[35:15] to ensure that we were protecting the company and protecting our data.
[35:18] And so I do think first and foremost understanding how the company's data is being used is really
[35:26] critical.
[35:27] And the legal leader at any particular company right now needs to start with, how do I protect
[35:32] the company with respect to AI?
[35:35] From there you can go to how to enable the company with respect to AI.
[35:39] How do we have, for example, we've created a fast past procurement process for key AI tools
[35:45] because we can't go through a super lengthy process by the time you do.
[35:49] We're the new model.
[35:51] And so how can we ensure that all of the important guard rules are taking place?
[35:55] But we can move faster because it's important to enable the company.
[35:59] How do you scale your team with AI?
[36:01] Which we talked about?
[36:02] And then how do you be a super user yourself?
[36:04] But it starts with that protection piece.
[36:06] It's where I started a couple of years ago.
[36:08] And I think it's something that every GC should be looking out for.
[36:12] And given all the work that you're doing now,
[36:14] I think it's fair to say that you're the cutting edge of legal
[36:17] NDI, where are the guardrails you're approaching now?
[36:20] We're thinking about now.
[36:21] I think it's a couple of fold.
[36:24] One is how other companies clip potentially use your data.
[36:28] So terms get updated all the time.
[36:31] There is LLM providers.
[36:33] And you really have to keep an eye out to make sure
[36:35] that your data is not being shared, not being stored,
[36:40] did not being used as a training model if you don't intend it to be.
[36:44] So I would say that's the external data piece and there's the internal data piece.
[36:49] Because as the company adopts more AI, if you don't have good data controls internally,
[36:55] all of the sudden employees might be able to access all sorts of information if you haven't
[37:00] set it up correctly.
[37:01] So for example, it was very important to us as a public company that in order to enable, for
[37:07] For example, my team who's working on an M&A deal to use AI, well, anything related to M&A needs to be very carefully
[37:16] cabinet.
[37:17] And so, again, is less about AI and more about the framework that you set up and data
[37:23] integrity is just so critical.
[37:27] And so for us to feel comfortable rolling out broad AI tools for a variety of teams, including
[37:32] those working on really sensitive information, whether that's M&A, whether that's employee
[37:37] related, personal information, whether it's comp data, that we had to have the right
[37:43] guardrails and restrictions set up such that we felt comfortable with those teams using those
[37:49] tools and they're not being in the data leakage. So I think it's really really about data
[37:54] framework. I want to highlight the last thing Molly said there, data framework, data controls
[38:00] integrity. That's something that is top of mind for most lawyers today as it should be.
[38:05] It's certainly top of mind for David Wang.
[38:07] He's the chief innovation officer at Kooley.
[38:10] David's job is literally to determine
[38:12] the firm's technology strategy.
[38:14] So there aren't many people more qualified to talk about this.
[38:17] He has a similar emphasis as Molly,
[38:19] getting the data sorted before and well using AI.
[38:24] Here's David to talk about how he stays on top
[38:26] of all the data that a law firm like Kooley has.
[38:29] In the data domain, that means really getting ourselves
[38:33] organized, right?
[38:35] So for example, we talk about our AI principles, right?
[38:38] But don't you hear out there, is that behind those principles?
[38:41] So tremendous amount of work.
[38:43] So really get on top of our data, so that we can truly follow
[38:47] our clients' instructions in terms of how they want their data
[38:49] to be handled.
[38:50] But they don't want to happen to their data that requires
[38:52] massive investment infrastructure in our data lake, right,
[38:56] in our systems and processes.
[38:58] Such that that work like that we're doing on the software,
[39:01] on the AI side can be supported and accelerated.
[39:05] Like you, like me, David understands the importance of data
[39:08] when it comes to implementing AI.
[39:10] There's an overwhelming amount of data to navigate.
[39:13] And many lawyers and law firms aren't thinking about all this.
[39:16] But they should be just like they should be thinking about
[39:19] how to best leverage AI in the legal practice.
[39:22] And that's something that we're going to talk about now.
[39:24] I want to start with the most basic way that most of us interact
[39:27] with LLMs via the chatbot window.
[39:29] Zac Shapiro, who's posted on the Cloud native law firm, was read nearly 8 million times, explains
[39:35] the first lesson he gives lawyers on how to better interact and better leverage AI today.
[39:40] It starts with the prompt, not treating it like a Google search, but instead is an associate or a
[39:46] genie who needs really clear instructions. And anyone who's seen the movie obsession recently,
[39:51] good horror movie knows how important it is to give clear instructions. When people put prompts in AI,
[39:58] The natural thing to do is to write one to...
[40:00] sentences which is wrong. The right mental model for how to start instead of a Google search
[40:05] is how you would brief an associate across the table from you on an assignment that you want them
[40:11] to do. Now, it's not quite the same as talking to associate, but the starting point is, okay,
[40:15] imagine you're going to delegate this piece of work to a human. What would you say to them?
[40:21] And now, imagine if instead of a human, this was a genie from a fairy tale, right? Like, you know,
[40:27] So we're in all of the genie stories, the problem with the genie is they take your wish to literally.
[40:31] Right, you wish for million dollars and falls out of the sky and hits you on the head.
[40:35] You know, you wish to be the smartest person on earth and everyone else disappears.
[40:38] Imagine that anything you're not super clear and specific about is going to be used against you.
[40:42] And then do that briefing again for the genie where like you're forcing yourself to sit there at least for a couple minutes at a time.
[40:50] filling all of that time with more and more what ultimately ends up being context for the LLM
[40:57] that allows you to get sort of more elite output. I think that's really step one
[41:02] at getting good at talking to the AI. Now as Zack said, we need to be more patient when we use AI.
[41:07] And I'm as guilty as anyone for typing in something like milk, safe, drink, two days past expiry.
[41:13] Prompts need to be focused. They need to be tailored and convey everything necessary for the LLM
[41:19] to provide the right answer. Or do they? I think what was most telling to me across all the conversations
[41:24] I had is that everyone has a different approach and that seemed to work okay for them. I think
[41:29] what does matter though regardless of the specifics of how you prompt is the context you give
[41:35] the LLM. You can either give it that context in the prompt or actually train the model itself.
[41:41] Here's Zach again with more on where an LLM will pull an answer from and how to train it
[41:46] to get you better answers. It's where he says the edges for lawyers to differentiate themselves.
[41:52] The edges in the instruction that you put into the AI about what the output you want is.
[41:57] In order to do that well, I just going back to what an LM is. Because it's trained on the
[42:01] corpus of the internet, if you are vague at all, if you leave any room for interpretation,
[42:05] the LM is going to fill that gap with the median of the internet, which is what we've all
[42:10] come to recognizes as AI slot.
[42:12] Right?
[42:12] It's the like M-dashes and the training specificity
[42:16] I think I'm reddit, which it overly weights as a source.
[42:19] And so in order to not get that,
[42:21] you need to very specifically describe exactly
[42:25] what you want out of the model,
[42:26] which requires you first to be able to imagine it.
[42:28] So I'm spending more of my time thinking about,
[42:30] okay, what really is the right work product for this client?
[42:33] And then how do I express that in a way
[42:35] that leaves absolutely no room for vagueness or ambiguity
[42:39] that can be weaponized against me by the LOM.
[42:43] And that working net muscle has also,
[42:46] I think, made me a better thinker and a better lawyer.
[42:49] It's like, the people say that the best way to learn
[42:51] is to teach and working with AI is just constantly
[42:55] teaching the model all this sort of like,
[42:58] what would otherwise be unstated assumptions
[43:00] that go into your work that you don't think about.
[43:02] Now, it has to come to the fore
[43:03] and you think about it and you have to explain it to the model.
[43:05] Now, is that just explained?
[43:07] We need to be thinking about the unstated
[43:09] assumptions when we use AI. That can be basic things like which part of your representing,
[43:13] or something more complicated like the negotiating power or leverage, you have one making edits.
[43:18] Here's Aaron Kelly. He's the general counsel at Ed and node and the founder of little labs to add
[43:23] more on the ways you can teach the LLMs you work with to improve their outputs. How do you think about
[43:29] teaching and then what do you mean when you say teaching the model? What does that look like in practice?
[43:33] Well, there's two ways. You can do it through prompt engineering. You can just say,
[43:37] hey, here's a document that we have that we use for reviewing these agreements and I want you to take it and apply the playbook to it.
[43:47] And here's the context. Now, you can also train the model to do that too, meaning it's already inherent to it. So you don't have the prompt at every time.
[43:55] That's a little bit more difficult. There's ways to do that through they call it's fine tuning.
[44:01] that's really what I focus on is teaching the model.
[44:05] Hey, this is what is good, this is what is bad.
[44:07] Based on my preference, it's really hard.
[44:09] That's probably why we don't see these general legal models out there.
[44:13] And going back to the Harvey Lagorre question,
[44:16] we didn't why maybe what's the working on,
[44:19] good work of good net.
[44:20] It's because so much of what we do is subjective.
[44:23] I mean, one warrior could argue that this is a good change versus another,
[44:27] which we've all gotten the red lines back from a warrior
[44:30] that has changed the font size, the font type,
[44:33] and it's like preference thing.
[44:35] Well, again, like you can teach those things,
[44:38] but it's again, it's like going back to why you do it.
[44:43] Because you want the model to behave like you want it to.
[44:46] You're not replacing your judgment.
[44:48] You're basically giving yourself a partner that can help you
[44:51] in looking at an issue's body.
[44:53] You want to have something that can look at every single angle
[44:58] when you're looking at a deal or a contract.
[45:00] That's what a good model should do.
[45:02] And if you teach it, how to do that,
[45:05] you're going to have a lot better time.
[45:07] So Aaron talked about training the model yourself.
[45:10] You can also get models like Claude to train themselves.
[45:13] Here's that Shapiro again to explain how he improves his AI
[45:17] with the skills feature on Claude, which lets you package
[45:20] a set of instructions, reference materials,
[45:22] and examples that Claude will load whenever a relevant task
[45:26] comes up.
[45:27] He also explains how he leverages different models
[45:29] to check each other before he even sees a document.
[45:33] Claude can get pretty good at checking its own work.
[45:37] Skills are really helpful for that.
[45:38] Knowing how to word skills and word prompts for self review
[45:42] is really important for that.
[45:46] O-Bus especially has a personality.
[45:48] I know that sounds weird,
[45:49] but it is very true and I've tested it rigorously
[45:51] where the way that you phrase things
[45:56] is really important in terms of how thorough
[45:57] it's going to be a check in its own work.
[45:59] And being earnest and appealing to,
[46:03] I guess what you would call emotions,
[46:05] for whatever reason tends to work better,
[46:07] then just getting a list of things for a detect
[46:09] that I can randomly decide to ignore.
[46:13] Even more powerful than that,
[46:13] I found that like different frontier models,
[46:16] check each other's work more effectively
[46:18] than any of them check their own work.
[46:20] And so, and you can really scale that up,
[46:22] right there, complicated drafting assignments
[46:25] I will have like, log co-work do where I'll like pass it back and forth between co-work and chat
[46:31] UTT, eight or 10 times before I ever take a look at it and wait for the different models
[46:38] to agree that this is like ready for my review.
[46:41] And so there are all sorts of tips and tricks to have AI check its own work.
[46:46] Now, that's just a significant amount of leverage.
[46:48] It's a game-changing amount of leverage, which is why lawyers like Zack are benefiting from
[46:53] Another lawyer who runs his own firm Michael show altar, founder of show altar, PLLC has created
[46:59] an AI native of Pella and complex litigation bootique firm.
[47:03] Michael's current AI stack includes Claude and Claude Coerc.
[47:06] And I think it's obvious he's really getting the most of it.
[47:09] He's written a lot of viewed articles that typically and previously had taken 150 hours in just 15 hours.
[47:16] It's obvious that he's getting the most of it.
[47:18] And I think this is one you should definitely listen closely to for an idea or two that you can implement in your
[47:23] practice. Here's a clip from my conversation with Michael. Could you walk through your AI stack?
[47:29] Yeah, I use Cloud Co-work and chat CBT Codex. My primary work stream is through Cloud Co-work.
[47:37] It's quite remarkable. So there's the chat bot, which most people are familiar with, which is quite powerful.
[47:43] Co-work is a desktop app for those who are not familiar with it that plugs into my laptop.
[47:49] And so it's connected to the folders that I granted access to.
[47:51] It's connected to my Chrome browser.
[47:53] It's connected to my calendar.
[47:54] It's connected to my email.
[47:56] So it is constantly doing a gintech things such as drafting a first draft of an email
[48:03] that I might go and review.
[48:05] It might be letting know that I had an email come through that I haven't responded to.
[48:10] It is every day going on to my Chrome browser and doing internet research or filling out a form
[48:17] for me to review it is when I when I draft briefs in the chatbot I walked through
[48:23] Claude in the chatbot through the legal arguments that we want to make and once
[48:27] it's ready to draft the first draft that just drafts it just drops a word document
[48:32] to a folder that I go then mark up with line edits and comments and like I would
[48:37] within associate then I go back to the chatbot and I say to Claude my revisions
[48:42] or in the document and then it versions up and implements my revisions and I go back
[48:46] back to my folder on my desktop.
[48:48] It's a quite remarkable process.
[48:51] And it's also just constantly scanning.
[48:53] I have these are called scheduled tasks.
[48:55] So I have cloud scanning for mentions of my large yard
[49:00] of cool that we were just talking about.
[49:01] I have cloud scanning for business development opportunities
[49:04] for any developments at the state,
[49:07] bar associations on their postures toward AI,
[49:11] and all sorts of other things.
[49:13] And so as far as other tools, I use a couple of legal research
[49:20] connectors, one of them is called mid-page.
[49:22] The others called ding-duff.
[49:24] And they are connected to my cloud.
[49:28] And so if I have a legal research question,
[49:31] I will tell cloud.
[49:33] Here's my legal research question.
[49:34] What's the answer?
[49:35] And then it will use those tools to research the case law.
[49:39] And then present me the answer as if it's a junior associate.
[49:42] And so I have not entered a Boolean search
[49:48] like I used to into West law with the ANS and Ores,
[49:52] while I have operated my practice.
[49:54] I have been talking to Clark,
[49:56] like I would talk to Junior Associate,
[49:58] and then it sent me the app.
[50:00] having having research done.
[50:02] I really wanted Michael to walk through his AI stack because I think it's such a great snapshot of what an ultra-forward AI-based legal practice actually looks like.
[50:11] He's running Cloud Co-work as his primary tool.
[50:13] It's plugged into files, browsers, calendar, email with scheduled tasks, scanning for business development opportunities in the background and legal research connectors and link case law.
[50:24] Now he mentioned two tools that you might not be familiar with.
[50:27] mid-page and ding-duff. These are free model contacts protocol connectors and PC connectors,
[50:33] which are basically bridges that link claw directly to primary legal research databases,
[50:39] so it can look up real case law in real time rather than relying on what happened to be in
[50:44] its training data. Remember, these are just prediction machines. So if you go with something like
[50:48] those tools, they can actually check what exists rather than try to predict what exists. Now,
[50:54] While we're on the topic of what's under the hood, I thought this was a good moment to quickly talk about open source LLMs.
[51:00] The things in the topic are warning us about our saying and it's a bit dangerous and we've seen things with the US government maybe wanting to stop open source LLMs.
[51:08] There's a lot of talk about them now, but also a lot of confusion about what open source LLMs actually means.
[51:15] Here's Aaron Kelly to explain what open source actually means and why it's kind of a misnomer when it comes to LLMs.
[51:23] OpenSource is kind of like a marketing term here when it comes to LLMs.
[51:27] It's not so much openSource as it is like open weights.
[51:30] The weights are what the model has been built in trained on.
[51:35] And it's everything that the model knows basically.
[51:37] And I guess the biggest thing I can say is,
[51:39] what's open weights is you can change the weights.
[51:42] You can customize the weights.
[51:44] You can be called post training.
[51:46] It's the recipe.
[51:48] Open weights like the recipe for how the LLMs was
[51:52] was trained in how it operates. That's the best way I can put it.
[51:57] Now let's pause on Erin's point for a second. And a lot of people have told me that they
[52:01] paused this podcast often. There's these dense episodes, but I appreciate you,
[52:05] sticking with me because I think this is so important. When people say open source and they
[52:09] I, they mean open weights. The weights are those tuning knobs we talked about earlier,
[52:12] the ones that help improve the results to make them more accurate. Now lawyers like
[52:17] Mike Michael and Zach are using the frontier models of Claude in the AI stack.
[52:22] I asked Michael how long he thought it would take a lawyer who's familiar with Claude to
[52:25] set up a system that made a dramatic difference in their efficiency and their legal practice.
[52:30] His answer surprised me. It was less than I expected. I think it'll be less than you expected.
[52:35] Here's Mike. It would take probably half an hour to have the tools set up in the way that
[52:45] the infrastructure set up in a way that would dramatically
[52:49] off the bat increase your productivity
[52:52] and allow you to improve the quality and reduce the time
[52:54] of taking it to your late works.
[52:56] The skill that it takes to actually guide
[52:59] clogged through the process is something
[53:01] that has to be developed and learned, especially for most lawyers,
[53:05] there a little bit hesitant about using the AI.
[53:10] And one thing that trips a lot of people up
[53:12] is they'll get some output from the AI
[53:14] that they don't like for whatever reason, either it's,
[53:19] so it's very rarely hallucinating and strict sense.
[53:22] I've, it's been months since I've seen the quads
[53:25] and me a case that just didn't exist.
[53:27] That just doesn't happen anymore.
[53:29] But it will still say things that are wrong
[53:31] or unintelligent.
[53:33] And that happens quite frequently.
[53:35] And so for most people, for many lawyers,
[53:39] they get these responses from quads
[53:41] and they think this is a tool that I cannot trust.
[53:44] and therefore I cannot use it.
[53:46] And they are right that they cannot trust it
[53:48] in the sense that they must verify the accuracy of its assertions
[53:53] and the soundness of its reasoning,
[53:56] but the conclusion does not follow.
[53:57] Does that mean that they can't use Claude?
[53:59] So the key is not getting flustered by some of the output
[54:04] that is an unintelligent or wrong.
[54:06] I am constantly telling my LMs that makes no sense.
[54:10] Please correct this.
[54:11] Please fix this.
[54:12] Why did you say this?
[54:13] This is a problem.
[54:14] we need to fix this right now.
[54:16] So it never happens again in the future.
[54:19] And if you do that, it's quite good at correcting
[54:22] and you actually don't need to drop to thing.
[54:25] You just need to work with it to get the result that you want.
[54:29] So back, so getting to the place where you can leverage
[54:36] AI to the extent that I am is not a 30 minute project
[54:40] for sure, at least not for most people.
[54:43] but just getting the infrastructure set up,
[54:45] getting Claude connected to the folders
[54:49] that you wanna connect into.
[54:51] For example, if someone wanted to have Claude
[54:55] help them brainstorm a legal issue,
[54:58] creating a folder of 10 or 12 articles on the subject
[55:02] and getting Claude connected that folder
[55:03] and reading the thing,
[55:04] that would take Claude about five minutes
[55:06] and the set up time is very rich.
[55:10] It's shorter than we think,
[55:12] that initial setup connecting Cloud Tier Folders,
[55:14] wiring up basic infrastructure.
[55:16] If that's a 30 minute exercise,
[55:18] I think that puts you a step forward
[55:20] to the thing that actually separates lawyers
[55:22] who get 10x results.
[55:23] Another alternative is to use a ready-bake system.
[55:25] If you don't wanna do as much of this yourself,
[55:27] I spoke to the CEO of Spellbooks, God's Teams,
[55:30] and their program, I think is 299 a month for lawyers.
[55:33] They have an AI co-pilot and contract for a viewer system.
[55:36] That's designed specifically for transactional lawyers
[55:40] and legal teams.
[55:41] that operates as an add-in that sits directly inside Microsoft Word.
[55:45] Here's Scott Stephenson to explain
[55:46] what's market tool.
[55:47] I think this is something that's unique to an AI tool like Spellbook
[55:51] has opposed to general LLMs like Cloud or chatchipeteer.
[55:54] The problem we've had in the legal are one of the problems
[55:57] in legal and transactional work is that this data has been so opaque
[56:01] and very asymmetrical.
[56:03] Like how me like Google is going to have all the data in the world
[56:05] or a big law firm is going to have tons of data.
[56:08] But if you're a small guy, you know, you're going to be a data disadvantage and you're not going to know whether you're getting screwed over or not.
[56:15] And so we thought it, it just, it would democratize negotiation to make this data more transparent would be more fair, more efficient.
[56:24] And also like, around the models, I think the other challenge is that if you're just using like chat to be teaching the negotiate agreement, you know, it will tell you like, oh, this is not standard or oh, this is standard.
[56:34] is his standard, but what's that really based on?
[56:37] It's mainly based on public company contracts
[56:39] and data that was in the data set on the internet.
[56:44] And that's not necessarily, like if you're a private company,
[56:48] and you're a healthcare company, like,
[56:50] is that really what's market is what Chatch UBT
[56:53] is telling you is market really standard?
[56:55] It doesn't really have the knowledge to know that.
[56:57] So we think grounding AI in this data,
[57:01] and then being able to filter based on the jurisdiction,
[57:04] and the industry, the deal size,
[57:07] we think that's really important to get accurate results.
[57:11] So as Scott just said,
[57:12] spellbooks compared to market feature,
[57:15] democratizes real deal data,
[57:17] so you're not just trusting,
[57:18] chat chicketese gas or prediction,
[57:20] based on whatever public contracts happen to be in its training set.
[57:24] I asked him a follow-up question though.
[57:25] How do they actually find these market contracts
[57:28] to determine what's market?
[57:29] What about private party data?
[57:31] Here's the back and forth I had with Scott to better understand how they determine what's market at
[57:35] Spelwood.
[57:36] So first off, you know, what we capture is aggregates statistical data.
[57:40] So the data that we capture, the market data that we store at the end of the day, all
[57:44] it is, you know, this is the average price per square foot for a commercial lease in New
[57:49] York City.
[57:50] Or these are the average late payment terms for a SaaS agreement in California.
[57:55] So one of the secrets to what we do is we store this really high level statistical information,
[57:59] which has no PI, I associate it with it.
[58:03] And so this allows actually a lot of our customers
[58:05] to be comfortable with a gift to get models.
[58:06] So in our gift to get model,
[58:08] lawyers basically say, hey, we will provide
[58:10] the statistical mathematical data and in exchange
[58:13] we'll get Actus.
[58:14] So that is one way that it works.
[58:16] Some customers will pay extra.
[58:17] So they'll say, you know what?
[58:18] We don't really want to contribute to the pool
[58:19] that actually my, my Norte of customers selected us.
[58:22] Some customers say, you know what?
[58:23] We don't want our data in the pool.
[58:24] We just want to access to the pool
[58:26] and you know, we'll charge extra those customers.
[58:28] And then for customers with really big contract flows,
[58:30] customers like Google size, we'll actually
[58:33] allow silo data.
[58:34] That's something we're launching work.
[58:35] You just wanted to be your own data and nothing else
[58:37] and we'll be able to do that, too.
[58:38] That's how it works today.
[58:40] Yeah, that's a good way to do it.
[58:42] I remember everyone does something similar in law school,
[58:44] where you can contribute notes to a group,
[58:47] and then you can all benefit from that.
[58:49] And I think bringing that to the legal profession
[58:51] is just going to be a big one for everyone.
[58:53] But I think one distinction is the quantitative data
[58:56] versus the qualitative data.
[58:58] And the quality of the stock side of contracts is so important.
[59:01] And like you said, you're sort of pulling terms that relate to more,
[59:04] and maybe I'm misinterpreting that.
[59:06] It sounded like when you reference like cost,
[59:08] per square foot of lease and stuff,
[59:10] that's the quantitative side of the,
[59:13] what are you doing for the qualitative side?
[59:16] Yeah, for the qualitative side, what we'll do is we'll identify
[59:19] the common variations of a term.
[59:22] Let's just take this kind of a simple one,
[59:24] like late payment terms in a SaaS agreement.
[59:29] When is your service term in a GFP,
[59:31] late fee interests?
[59:34] There's different variation to that term that you could have.
[59:36] So we would kind of look at,
[59:37] okay, here's the major variations of this term.
[59:39] And then we would say,
[59:41] 20% of people use this variation, 30% of people use that
[59:44] very, very, very, very,
[59:44] and kind of bucket it into the major types of terms
[59:50] that exist.
[59:51] Yeah.
[59:52] Does that make sense?
[59:53] Yeah, yeah. So from my understanding, it's if you're looking at a term if your lawyer who's working on a contract and you're working on a term
[59:59] It's just a for confidentiality.
[60:01] You would see some of the variations of that term as your drafting.
[60:06] And then like a software developer,
[60:07] you could choose sort of which one you wanted to implement in your code
[60:10] or in your X-Denis case.
[60:12] Yeah, yeah, that's right.
[60:15] So how's the notebook actually builds its market data?
[60:17] Similar to how other legal AI companies do it as well.
[60:20] On the quantitative side, they aggregates
[60:22] statistical benchmarks, average lease prices, late payment terms.
[60:26] For the qualitative side, they map out common variations of clause.
[60:30] It's sort of like what a lot of lawyers would do when they just pull out sections of contracts that they want to keep in their contract bank.
[60:37] A lot of lawyers who I know do something like that on the commercial side.
[60:40] I also spoke to a CEO of an AI-powered litigation platform, Justin McCallon.
[60:46] He's building strong suit to transform how litigators handle legal research.
[60:50] Years a quick outline from Justin are how that platform works.
[60:53] And remember, I'm not sponsored by any of these AI companies.
[60:56] I'm just sharing so you're aware of what exists.
[60:59] How does your product work in terms of the legal research and where it's pulling cases from?
[61:04] And what context window does it have?
[61:08] What's going on on the back end?
[61:10] Yeah, yeah.
[61:11] Let me break those into different parts.
[61:12] So one, as far as what we're using for data, we worked to form partnerships to cover all
[61:20] all presidential US cases, there's about 11 million of them.
[61:24] And then we took each of those cases and said,
[61:26] okay, let's have an agent go through the case and understand
[61:29] over the key facts, over the key of all things,
[61:32] what was the key analysis?
[61:33] It was about 20 different pieces of information for each case.
[61:37] And we spent a lot of time really shaping that agent
[61:39] to be kind of recursive and iterative
[61:41] in how it determined that.
[61:43] And it did a lot to ensure that it was going to be accurate.
[61:46] For example, if we've made a holding,
[61:48] the agent had a point back to text in the case to say,
[61:51] this is what I derived this from,
[61:53] then a secondary agent would have to check
[61:55] and make sure that's real stuff like that.
[61:57] And so we have this great purpose of knowledge
[62:00] because of the way we built our tool,
[62:02] we knew that this was gonna be done in the AIA,
[62:05] didn't use the AIA.
[62:06] So we can use things like Ragnarware,
[62:09] we're pulling out,
[62:10] sorry Justin, what is Ragnarware?
[62:12] So retrieval augmented generation.
[62:14] So what we're trying to do is basically say,
[62:17] Hey, this case might have some similarities to your case.
[62:21] And these other cases might have other similarities
[62:23] or be important for these other reasons,
[62:25] like maybe they're cited a lot,
[62:27] or they're in the right courts of stuff or whatever.
[62:29] And so we have different ways of pulling
[62:31] the right cases for your matter
[62:33] to ensure that we're finding really the most accurate
[62:36] and relevant ones that are worth citing.
[62:38] I mean, then we run a lot of e-values on that.
[62:40] So what we'll do is we'll say,
[62:42] okay, let's go find 200 cases and say,
[62:46] over the back of the case, hold those out from the briefs.
[62:49] And then what were the cases cited in those briefs?
[62:51] And then to have the AI say, go take the same fact pattern and find the cases that you think
[62:56] are most relevant.
[62:57] And we keep optimizing until we have a lot of overlap with what's actually submitted.
[63:02] And so we can do things like that to make our retrieval ability very strong.
[63:07] Now, you don't need to buy one of those packages.
[63:10] I think it's good to know what exists.
[63:12] Sometimes they'll suit what you're looking for.
[63:13] You can also build using AI yourself,
[63:16] Sujit Ramen, the chief legal officer at T.R.M. Labs,
[63:19] has built a system for monitoring regulatory updates
[63:22] that are relevant to his day-to-day work.
[63:24] This describes how he's created an agent
[63:26] that actually helps him in his team
[63:28] to do their jobs on a daily basis.
[63:30] Here's Sujit.
[63:31] My job is to protect the company.
[63:32] That's a very broad remit.
[63:34] So there's a number of sub-alcums within that broader outcome
[63:40] that I'm responsible for.
[63:41] And so I think about, you know,
[63:42] other things where I need advice or updates on the regulatory structure that's out there.
[63:52] And so my responsibility is to create an agent that will essentially give me updates
[63:58] at whatever cases I want about what's happening in the, I don't know, the crypto industry,
[64:02] but the cybersecurity industry, you know, the broader privacy, the broader AI, you know,
[64:08] there's so much happening on a regulatory front.
[64:12] I need something to pull all of that together.
[64:14] And I don't want to be spending money on outside council for this.
[64:16] And I want something that's uniform.
[64:18] It's something I can look at every single week.
[64:20] What do I do?
[64:21] I create a nation for that.
[64:23] And so a member of my team has created essentially an agent
[64:27] that tracks all the global regulatory developments
[64:29] in each of the areas that I talked about.
[64:31] And you can create skills for that, right?
[64:33] I'm looking for privacy.
[64:35] I'm looking for litigation involving crypto or digital assets.
[64:38] I'm looking for AI related regulations in Europe.
[64:42] in Asia, in the United States, break it down federal state.
[64:45] This is all stuff you can code.
[64:47] And it's in extraordinary because it's the kind of thing that even as recently as like a year ago,
[64:53] you have to pay money for an outside firm to do that for you.
[64:56] And even then it's just, you know, it's still like a client alert.
[64:59] Now it's something you can run in-house using the idea of the cortex.
[65:03] So that's one very small example.
[65:05] The other one that I like to talk about because it's really open my eyes, it's the commercial side of the house.
[65:10] There's so much that we have coming in in terms of NDAs, vendor agreements,
[65:14] reseller agreements, are you, etc. Those are the kinds of things that historically take a lot of
[65:21] manpower, a lot of just some of it's pretty routineized, but you still have to spend time doing it.
[65:26] And humans make errors in this day and age. If your team has an adopted an agent,
[65:34] that's been coded with your company's playbook with all the different provisions
[65:39] and the fallback provisions and the kind of general orientation of us as a company, some companies are large financial institutions.
[65:45] You know, we're a smaller early stage venture back company. You've got different perspectives, right?
[65:50] And you code all of that in. Our contractual review, you know, SLAs are efficiency has increased by something like a thousand percent.
[65:59] Using these tools, which, you know, you're not doing your job unless you're adopting this kind of this kind of technology.
[66:06] Susia has written about what he calls the AI cortex away of thinking about AI and a theme of how they think about it at TRM Labs.
[66:13] Those are two examples of that in action.
[66:16] He has an agent that tracks global regulatory developments across privacy crypto AI, cybersecurity, broken down by jurisdiction.
[66:23] On the commercial side, though, they've got that agent that has the TRM Playbook, handles and DAs, vendor agreements, and sort of simple repetitive contracts.
[66:31] contracts. That he said increased their contract of efficiency by roughly a thousand percent.
[66:37] I think Molly Abraham, the general council at Coinbase, has seen similar benefits. She's taken
[66:42] a similar approach to commercial work. She's actually hiring a senior software engineer to join her
[66:48] legal team. Yes, hiring a senior software engineer to join her legal team to expand their AI automation
[66:54] capabilities. I asked her about the goal of this hire because I think that explains how people who are
[67:00] taking these steps or thinking about how you can do.
[67:03] Here's my back in fourth with Molly about the highest priority use case.
[67:07] Now the highest priority use case for this hires build an end-to-end
[67:11] agente workflow that automates negotiations with counter parties by evaluating
[67:15] markups against the playbook. Why is this the top priority is opposed to regulatory
[67:20] filings or many other things that I know Coinbase has gone on?
[67:24] So this to me is one of the top priorities for a couple of reasons. One,
[67:28] I know exactly what it is. I want this April, I have done RFP after RFP for different vendors.
[67:36] I have worked with engineers internally. No one has been able to craft this to the extent
[67:42] where we can actually implement it. And so what I'm envisioning, we have a huge number of interested, for example, institutional clients,
[67:51] who all want to onboard onto Coinbase because we are the most trusted platform and part of that process
[67:58] for some clients does include, they've got a legal team and their legal team feels the need to
[68:03] redline the agreement or try and there's an element of this is the way it's always worked and therefore
[68:10] this is what folks want to do but we have a really, really thoughtful process and playbook for
[68:17] for how we consider any changes if we even accept them.
[68:21] And this to me feels like something
[68:23] where we could be faster responding to our clients,
[68:26] we've already done the thoughtful kind of data analysis
[68:29] and work on the backend.
[68:31] I just need to somehow make it work technically,
[68:33] and I am stuck.
[68:35] And so this is why it feels like the right place to start,
[68:40] because it's a defined use case.
[68:43] It's an immediate way in it unlocks revenue
[68:46] because these are clients who want to come on to Queen Basin trade.
[68:49] And there's just so many of them that it takes time.
[68:53] And so this to me is a great first win and something that is just right for automation.
[68:59] And then those lawyers who are previously doing that can focus more on important conversations with those clients.
[69:05] Negotiations and those live conversations where an agent is not about to replace a human at any time.
[69:11] Molly just explained I think a principle that applies to any in-house team where you have inbound and outbound sales.
[69:20] Having a playbook that your AI marks up against doesn't just save legal time,
[69:24] it actually directly unlocks revenue and it can free up time to focus on things like judgment.
[69:30] Well, train junior lawyers can also offer similar benefits, but what does the future of AI mean for them?
[69:37] for legal training as a whole, really, as juniors may no longer be a necessary cost center
[69:43] if AI can replace those tasks in a faster, better and more consistent manner.
[69:48] What will happen to the next generation of lawyers? I asked all my guests that question.
[69:52] Here's Sue Jitramin, the chief legal officer of T.R. M. Labs, who said his biggest worry is the development or lack thereof of judge.
[70:00] Yeah, I mean, what I do worry about is the absence of the development of judgment, because
[70:07] you can talk about efficiency, you can talk about bill-bill hours, you know, all that is what it is.
[70:11] But you can be an associate who builds 2,500, 3,000 hours a year.
[70:15] But if all they're doing is a talk review or some other kind of mechanical work, they're not actually
[70:19] developing as lawyers, right? I think where the junior associate really benefits from the old system,
[70:25] just the kind of billable hour system is the presumption that they're getting exposed to the,
[70:32] not only the senior associate, but the partner, you know, the person who's tried the case,
[70:35] the person who's done the investigation. If you're sitting at their elbow and sort of watching them,
[70:40] you're on the client phone calls, you're in court, you know, maybe you're carrying the briefcase,
[70:43] but you're seeing how the, the more senior lawyer interacts with the bench, interacts with the jury.
[70:48] If you're on a deal and you're watching the senior corporate lawyer quarterback, the transaction with
[70:53] seven different people on the conference call,
[70:56] that's where you develop the judgment and the skill.
[71:00] My concern about where we are in this particular moment,
[71:03] technologically, is that with all the benefits of automation,
[71:07] particularly for junior lawyers,
[71:10] they might still miss out on that very critical,
[71:13] if you want to call it the apprenticeship,
[71:14] if you want to call it the guild kind of training,
[71:18] I don't think we found an adequate replacement for that,
[71:21] And there's going to be potentially a generation of young lawyers who miss out on that.
[71:25] And I do worry about that because I don't know if there's a good technological solution for that.
[71:32] Yeah, the only solution for them,
[71:33] Surgit, is going to be to listen to the law of good podcasts.
[71:36] I think that's, it's a great place to start.
[71:39] Now, I couldn't help but leave my little joke in there,
[71:42] but that shouldn't take away from the seriousness of this discussion,
[71:45] because Surgit just raised what might be the most important long-term question in this whole conversation.
[71:51] What happens to the development of lawyer judgment when AI takes over the tasks that used to teach them?
[71:57] When I was starting out, like every other lawyer, I sort of loathed the brutal due diligence assignments or work that I thought was mundane.
[72:04] But looking back, it was pretty clear to me.
[72:06] And a lot of lawyers told me at the time, you learn a lot from that.
[72:09] You learn what to look for when to triple check something, questions to ask off the bat.
[72:14] Those are really invaluable skills.
[72:17] And I think it's important that we talk about how lawyers can actually use AI to improve their training today.
[72:24] Here's David Wang, he's the Chief Innovation Officer,
[72:27] to explain how they are training junior lawyers differently in the age of AI.
[72:32] So, one of the critical components that we have in our AI training program is to say to young lawyers is like,
[72:40] we want you to do is to fight back against the cognitive surrender.
[72:45] So you have a partner emails you something and it's like Greek, right?
[72:49] You're like, what is this person talking about?
[72:52] What you can do is you can take that and you can plug it into LaGora,
[72:56] right, and you can get an answer. And by the way, LaGora's answer,
[72:59] we'll probably be better than yours if you just came out of law school, right?
[73:04] And so there you go. There's an instant answer. You're done.
[73:07] and shift that back to the partner, but wait, why do we have you?
[73:13] You know, I can read the partner, I couldn't just type that email into LaGora.
[73:18] So this is the ironically, the fear of junior attorneys, but it's also something
[73:24] that we see junior attorneys do all the time, the betrothal.
[73:28] And so the training is like, okay, now instead of that, what I want to do is
[73:34] is he take that, you know, and you put it into your grow up.
[73:37] And this time what the prompt and what you're saying is,
[73:40] and this is not like quote prompt engineering or anything,
[73:42] it's really like a mindset.
[73:45] It's about remembering who you are,
[73:48] it sounds a little Disney, really.
[73:50] You know, don't worry, I won't do the dance for more on it.
[73:52] But like, then, take your a lawyer,
[73:56] you're supposed to understand this,
[73:59] and you are in the process with training yourself.
[74:02] And now because of the acceleration of everything,
[74:05] and previously you were all training on the job,
[74:08] and by the way, passing that on to clients, right,
[74:12] in terms of the billable, which clients don't like.
[74:14] Instead, what you do is take that email,
[74:18] and then you say, hey, explain point by point,
[74:21] every single legal doctor in that is in here,
[74:23] and all the things are implied in these instructions,
[74:26] and to give that like a full list of those two things.
[74:29] And then you get burr, like I think, right?
[74:32] And then now what you're going to do is going to read.intar reply.
[74:37] Take that whole thing, put back in and ask again, every single thing that you don't understand.
[74:44] And then keep doing that until you understand every single thing in there,
[74:48] now there's nothing else that you do not understand.
[74:51] And then you take all of that.
[74:54] Right, the outline of what your response is yourself.
[74:57] And then you can use a, the generate the response email so that it can be like grammatically
[75:01] perfect and all of the stuff. And then what you do is you reply to the partner. Here's what I think,
[75:08] right? Because now it is what you think because there's no longer what the AI thinks, right? It's
[75:16] what I think. And by the way, here are my questions, right? Can you let me know? This is correct. And by
[75:23] by the way, can we discuss this?
[75:25] Right?
[75:27] That is the act of luring for a young attorney,
[75:32] this this, that they need to be engaging in.
[75:34] That's the most important thing.
[75:36] Now that ties into what Sujit said before about the biggest risk
[75:40] facing junior lawyers.
[75:41] And I thought David's step-by-step training method
[75:43] was really important, because it gives a guideline,
[75:45] not only for junior lawyers, but also senior lawyers
[75:47] to think about how we're teaching the next generation
[75:50] of lawyers.
[75:51] I also spoke with Molly Abraham, the general council at Queen of Ace about how Junior and
[75:55] lawyers will navigate the future.
[75:57] Because in her job posting, which should still be live by the time you're listening to
[76:01] this, which she was hiring for a software engineer for her legal team, she actually describes
[76:05] building an agent platform where today's Junior Lawyers become tomorrow's agent builders.
[76:12] Today's Junior Lawyers become tomorrow's agent builders.
[76:15] I asked Molly about that and what the future might look like for the next generation of lawyers
[76:21] So in the job description, it describes building an agent platform where today's junior lawyers become tomorrow's agent builders.
[76:28] How do you see today's junior lawyers or tomorrow's junior lawyers navigating that future?
[76:32] Do they still look like lawyers?
[76:34] Or do they look more like agent monitors managers?
[76:38] What does it?
[76:39] What do you think that'll look like?
[76:40] I think it's going to be a combination of things.
[76:42] And let me start with how I think we need to train junior lawyers to be ready for this moment.
[76:47] I'm an alum in the University of Chicago,
[76:49] law school as is Paul,
[76:51] and we both have spent a lot of time
[76:53] with this school talking about how they're approaching AI.
[76:56] And it's interesting because they're doing a combination
[76:58] of completely embracing AI,
[77:00] wanting students to do certain assignments with AI,
[77:04] because that's what's coming for them in the job market.
[77:07] Well, also doing certain settings,
[77:10] we're not going to have AI.
[77:11] So for example, that first year curriculum in the classroom,
[77:15] because they want to train them in a way
[77:17] that they're going to be able to think like a strategic lawyer
[77:21] and use AI efficiency efficiently.
[77:23] So it's embracing it while also learning some of the core concepts
[77:27] that are critical to being able to do with AI.
[77:30] I think about training junior lawyers the same way.
[77:33] I do think we're going to have members of the legal team
[77:36] who are primarily agent builders or agent designers.
[77:39] And that's where I'm really excited for this engineer
[77:41] to join to help figure out the framework,
[77:44] But I can see a world where a number of my really, really talented
[77:48] paralegals right now could actually be designing agents because they most deeply understand
[77:55] their workflows and they could be doing the agents to actually be able to replace some of
[78:00] those workflows. So, and as they get better and better at that and they learn from it,
[78:05] they could be the next era of agent builders. Now that said, the thing I worry about is
[78:10] is we can't just stop training junior folks
[78:14] and in terms of how to do the important legal work
[78:18] to start or will eventually like will lose over time.
[78:22] And so that is the million dollar question
[78:25] or I guess trillion dollar question if you think about
[78:27] okay, infrastructure size,
[78:30] that I don't have an answer to,
[78:32] but I think it's going to be really important for us
[78:34] all to figure out.
[78:35] Well, and the way I think about it too
[78:37] is that onus is going to be more on the junior lawyers
[78:40] educate themselves because now with AI, you have the tools to do that. You can ask questions that
[78:44] typically you would have asked to see in your partner and you can get some more answers. So
[78:47] it almost will become a, hey, you're on your own and you have to keep up and that's on you. The
[78:53] tools are there but if, you know, if you don't take advantage of them then it's going to be really difficult.
[78:58] It is that I think we have to empower folks and it's interesting. We have a lot of self-striiders
[79:04] on our team who have become incredibly AI-perificiate but we also owe them great training in this space.
[79:10] We still owe them career development and mentoring even if some of the substantive questions they can get a faster answer.
[79:17] And I also think this is a great opportunity to be able to be a manager earlier in your career.
[79:24] To say, okay, this is how I run a function and it's no longer just me.
[79:30] I can have three agent employees as well.
[79:33] And here are the different tasks they're doing.
[79:35] Here's how I'm efficiently operating a team.
[79:37] That's going to be something where they get to fast forward 10 years in their career, which could
[79:41] also be really powerful for those who take advantage of it.
[79:44] So as Molly said, anyone who's at the forefront of work can figure out where the best
[79:50] opportunity is to automate themselves.
[79:52] It's an exciting time.
[79:54] It's also a risky time.
[79:57] And it's easy to draw parallels to another time that felt so.
[80:00] to early 2000s when the world and the legal profession first got online with the Internet.
[80:05] But there are fundamental differences between the Internet and the AI and it comes down to leverage.
[80:10] Here's subject ramen with a great explanation on why the leverage of AI is multitudes higher than the Internet.
[80:16] The difference I would draw between the early days of the Internet and the current moment is on the one hand you're absolutely right.
[80:23] I mean, the internet revolutionized the access to information that people had.
[80:28] And when he comes to lawyers, I mean, think about something like Westball or Lexus Nexus.
[80:32] Just the whole legal research process became so different.
[80:35] Right? It's so much easier.
[80:36] And that only made folks more efficient or better.
[80:41] And yet, I mean, you look at some of the Supreme Court opinions from the 1940s,
[80:45] or their briefs that were filed by a third-guard martial or whatever, right?
[80:48] They were extraordinarily well written briefs.
[80:50] And they did it in an age before there was an internet.
[80:52] And so in that sense, I think the profession or the capabilities,
[80:56] transcend the time or or technology, I think where AI is different is this idea of the cortex,
[81:03] right? The idea of melding human capabilities with machine capabilities and thinking about
[81:11] what the interaction looks like between the two. And that's where I think for lawyers who
[81:17] don't get on that bandwagon, so to speak, you really will be left behind because the
[81:22] capabilities of the machine are so much more, you know, supercharged than what any other human
[81:29] can accomplish. So the way I like to think about how AI applies to the legal industry in
[81:34] particular is you're not eliminating lawyers. You're not eliminating skillsets. You're not doing anything
[81:38] like that. What you're doing is actually creating a leverage. One lawyer can do the work of five
[81:44] lawyers or 10 lawyers. If they know how to test the agent and create that cortex that human machine
[81:50] interaction the right way. And so for people who are able to do that, you're not only a better
[81:55] practitioner, you're more efficient. Here if you're in private practice, your clients appreciate
[81:59] it because you're most more cost effective. And that's the kind of melding of man and machine
[82:05] that I think is going to happen. And for those who aren't able to master that, unfortunately,
[82:11] they may be left behind. I think that's different than the days of the internet where you could
[82:15] still be a phenomenal lawyer, even without less law, even without less as next as long as your
[82:19] your work project was fine. In this day and age, I just don't think it's just about the work
[82:24] product or the work product will be much better, much quicker, much more considered by the people
[82:30] who are leveraging the cortex rather than the people who are not. If that makes sense. Now,
[82:35] he's talking about the cortex and using AI, not just to augment your work, but to be part of that
[82:40] work. We do a similar thing with the internet although it doesn't feel like that today. And the
[82:46] But the benefits have been huge, I'd say, for lawyers.
[82:48] So back in 1978, before the internet was widely used, before we had tools like Microsoft
[82:54] Word, there was about 464,000 lawyers in the United States.
[83:00] Do you know how many lawyers there are today?
[83:02] Well, according to the American Bar Association, we're at an all-time high.
[83:06] There are 1,370,000 lawyers in the United States.
[83:11] So while the internet made lawyers more efficient, that actually led to more work for lawyers.
[83:15] And I think the same will be true for the next generation that uses AI.
[83:19] But how do you say ahead when AI can allow a lawyer to do the work of five to 10 junior associates?
[83:26] Here's a subject on how he's encouraged his team of lawyers to adopt a programmer's mindset,
[83:31] a builders mindset, or it's their legal world.
[83:33] What are some ways that you've found it most effective to encourage that type of building?
[83:40] Well, I think part of it is the attitude.
[83:42] You know, it really is like folks were here to work together and we're here to build careers together.
[83:46] And I've been very transparent with my team that part of my job and part of my obligation
[83:52] is to make sure that everyone is set up for the next 15, 20, 25 years in their career.
[83:57] And I feel like I'm doing them a disservice, unless I push them a little bit, right?
[84:01] I feel an obligation for the people on my team to make sure that they're well positioned for the future.
[84:06] So I have that conversation and it's a very candid and transparent conversation.
[84:09] It's not, we're not here to scare you or put you on some kind of schedule.
[84:14] It's much more about, let me find ways to empower you so that your job is more fun.
[84:18] The more of the stuff that can be automated is in fact automated.
[84:22] So you can focus on the brain work.
[84:24] The part where humans can actually make a difference.
[84:26] And every single person on my team has embraced that.
[84:28] Because I think it's an all-bar interest, our collective self-interest, to figure that out.
[84:32] So I do think tone matters.
[84:33] And I think really being hands-on in a constructive way.
[84:38] you know during our one-on-one, as we talk about, what are your buildings? How can I be helpful?
[84:42] Show me what you got. You know, this is something that I think you might improve on, or wow,
[84:45] I never even thought of that. You know, keep doing that. That's incredible. I think it does take
[84:49] that level of kind of hands-on, one-on-one integration. But that's the job. That's what the,
[84:56] that's what general council should be doing, right? So I find an empowering, and I hope that you
[85:01] will work with finding an empowering as well. Susan makes an important point there that we've kind of
[85:05] brushed over the cultural side of all this. How do you get a team of lawyers who, based on a lot of
[85:11] the lawyers that I know you and I know, aren't all programmers, how do you get a team of lawyers to adopt
[85:16] AI to actually want to build AI tools? And hopefully this podcast can help with that. I think that's
[85:21] important because today with these tools one lawyer can do the work of five or ten if they know how to
[85:27] task an AI agent correctly. That's a gap that can't just be closed by being smart because the distinction
[85:34] might be the most obvious in the one area that I think most clients use to determine whether
[85:39] their lawyers grade or not. Speed. How quickly you can get answers to clients and internal teammates.
[85:45] Imagine something that takes one lawyer, 150 hours and takes you 15. Speed matters. That's why
[85:51] lawyers like Molly Abraham are using the extra time they've saved by using AI to build more AI.
[85:58] Honestly, we're using that extra time to experiment an AI and to find other use cases because right now
[86:04] that experimentation does require a pretty heavy lift to figure out okay. So when I wrote my
[86:13] agent writer that both write documents and kind of the playliest format and also will
[86:18] edit similar documents, it probably took me six or seven hours to build. And so it was a mix of
[86:27] kind of strategic training of the agent with other work that I had done with instructions about
[86:33] why I think it's important to translate it a little,
[86:35] legal concept in a particular way,
[86:37] all of these different things.
[86:39] Again, it was honestly about 50% of the time
[86:41] with sheer debugging.
[86:43] And so right now, every incremental hour we save,
[86:47] I would say on AI, or frankly, dollars that we save,
[86:52] because we're taking the first draft of a memo
[86:55] instead of asking outside council to do it,
[86:57] and instead going to them with that draft asking for advice,
[87:01] we're reinvesting an AI.
[87:03] The team is very focused on all of the different ways we can do it.
[87:07] I think the SLAs for lawyers are about to be blown out of the water.
[87:11] I have got to say because I now that anyone can go and ask Gemini a question,
[87:19] including folks trying to ask Gemini legal questions who are not lawyers,
[87:22] the expectation of how quickly you can advise those clients and get back to them is really changing.
[87:28] So I think we will not run out of things to do.
[87:31] But my hope for the use of AI at companies is this is not about job replacement.
[87:40] This is about shipping faster growing the economy more quickly and just bringing more
[87:46] product store customers.
[87:47] Now, it's not about job replacement.
[87:50] And maybe it's job displacement.
[87:53] Jobs probably do look a lot different in 20 years.
[87:56] much like they looked different 20 years ago today. Here's Samson,
[88:00] Enzer, a partner at K Hill Gordon and Ryan Dettel again, who I'm so grateful to have as a sponsor of
[88:05] Love Code with more on how the internet changed law and why we don't necessarily know what AI will do.
[88:10] Many people are concerned, oh my, you know, jobs are going to go away because of AI. But,
[88:17] you know, the computer did not lead to less jobs or if we want to talk about lawyers,
[88:21] The computer did not lead to fewer lawyers.
[88:25] West law, Lexus, these internet-based tools that allow you to research, you know, the
[88:31] judge I clerked for, when he did legal research, you went to the library, pulled a book.
[88:37] When I did legal research as an associate, you would go on a computer and you could do a
[88:41] Boolean search and look at every case ever published.
[88:45] West law and Lexus, that tool, did not lead to fewer lawyers.
[88:49] it led to an arms race of more.
[88:52] I don't know none of us knows how the efficiency gains
[88:57] and productivity gains from AI will affect the legal industry
[89:02] or many other industries.
[89:04] Unclear.
[89:05] Samson's point echoes what's been consistent
[89:07] throughout the podcast.
[89:09] The past tells us that big technological leaps
[89:11] in law in efficiency have never led to fewer lawyers.
[89:15] They've led to more.
[89:17] And nobody actually knows how AI is going to reshape the profession.
[89:21] I didn't make this podcast because I know the answer to that.
[89:23] I made it because I don't know the answer.
[89:26] But I thought that speaking to those at the cutting edge could offer us a look
[89:29] into what the future might hold.
[89:31] There's a line I love from William Gibson.
[89:33] He's a science fiction writer who said,
[89:35] the future is here.
[89:36] It's just not evenly distributed.
[89:39] That's exactly where we are with legal AI today.
[89:41] The lawyers you heard from are living in the future.
[89:43] Meanwhile, a lot of the profession is still deciding
[89:46] whether it's okay to use JATGPT.
[89:49] Thank you for listening all the way through.
[89:51] I think this is such an important topic.
[89:52] My goal with this podcast is to provide the content
[89:55] that I wish I had when I was working as a lawyer.
[89:58] So if you have any feedback or topic.
[89:59] to see, please email me at Jacobatlawofcode.fm. I'll read everything which isn't hard because I don't
[90:06] really get much but that'll change. I'm determined to make this the acquired meets the
[90:10] Heurman Lab but for law. Thank you to the presenting sponsor of this podcast altitude. I wanted to
[90:17] work with them because I thought you should know about their product basically altitude is building the
[90:21] financial operating system of the future and lawyers that I know refer their clients to altitude
[90:26] on paid referrals just because it's a better solution.
[90:29] Altitude has a simple strategy solve many hard problems for businesses relating to payments.
[90:34] That's why I despite launching publicly into December 2025,
[90:38] they've already processed over $350 million in payments.
[90:42] I use altitude for my international business.
[90:45] If you're interested in learning more, you can visit altitude.xyz4ds-law.
[90:50] You could refer your clients to them.
[90:52] You could also do a quick demo to meet the team and understand the product.
[90:55] Again, I'm Jacob Robinson.
[90:58] This podcast is part of my mission to help people understand the legal layer of emerging
[91:02] tech so that we can live in a world with transparent and well understood rules that
[91:06] are applied fairly to all.
[91:07] Part of the reason I'm able to do that is thanks to the other sponsors of this episode,
[91:11] the Solana Policy Institute, which is advocating for transparent, decentralized technologies
[91:16] like Solana in Washington and as well as around the world, their educated policymakers
[91:21] in doing really important work there.
[91:24] So thank you to the HyperLickwood Policy Center, who is pushing for the future of on-chain
[91:28] finance.
[91:29] I'm really grateful for both of their support, as well as K-Hill, Gordon, and Rindell.
[91:35] Now, I'm going to launch a newsletter at lawofcode.fm and you can sign up for free.
[91:39] It will be very short, probably just five bullets once a week.
[91:42] Upcoming episodes, if you are still listening here, include how the law applies to AI
[91:46] agents, crypto and Canada, and anything that you reach out to me and say, Jacob, this would
[91:50] be interesting.
[91:51] Thanks for joining me.
[91:52] See you next time.