Want to listen to the full episode and all our other episodes?
Hearsay allows you to fulfill your legal CPD requirements every year.
Our yearly subscription is only $299/year.
With a yearly subscription, you can access all of our episodes AND every episode we release over the next year.
Episode 111
Buy Episode
Practice Management and Business Skills
The Digital Assistant Paradigm: The Future of AI Digital Assistants in Legal Practice Management
Law as stated: 22 January 2024
What is this?
This episode was published and is accurate as at this date.
Canada-based Founder and CEO of Clio, Jack Newton, joins David in the Curiosity Recording Room to discuss the present and future of AI tools in legal practice management. Touching on the current features and abilities of LLMs, the dangers of unquestioning reliance on ChatGPT for lawyers, and the future of digital assistants.
Practice Management and Business Skills| What area(s) of law does this episode consider? | AI and practice management systems. |
| Why is this topic relevant? | AI is currently dominating conversations in legal tech – as in just about every digital industry – with opinions ranging from the belief that AI could replace lawyers to concerns about its safety in practice. While AI may enhance various aspects of legal practice, there are constraints we need to address, and we should caution against the notion of “magical thinking” – the belief that AI can replicate the intricate and strategic reasoning applied by lawyers in their practice. |
| What are the main points? |
|
| What are the practical takeaways? |
|
DT = David Turner; JN = Jack Newton; RD = Ross Davis
| 00:00:14 | DT | Hello and welcome to Hearsay The Legal Podcast, a CPD podcast that allows Australian lawyers to earn their CPD points on the go and at a time that suits them. I’m your host, David Turner. Hearsay The Legal Podcast is proudly supported by Lext Australia. Lext’s mission is to improve user experiences in the law and legal services. And Hearsay The Legal Podcast is how we’re improving the experience of CPD. Artificial intelligence is dominating conversations in legal tech. Well, it’s dominating conversations in just about every digital industry actually, with opinions ranging from the belief that AI could replace lawyers to concerns about its safety in legal practice. And in this episode of Hearsay, we’re going to delve into concrete applications of AI for the legal profession. We maintain our belief in the safe utilisation of AI to enhance aspects of legal practice, but we also talk about its constraints. And caution against what we call magical thinking, the belief that AI can replicate intricate, strategic reasoning applied by lawyers in practice. Now, this topic holds a huge amount of significance for our audience of legal professionals because it presents opportunities and challenges. AI is making waves across a lot of industries and the legal sector is no exception. The potential for AI to streamline legal processes like customer onboarding, enhance research, even automate certain tasks is driving a seismic shift within the profession already. Lawyers have to adapt to this new landscape to remain competitive and deliver even more streamlined legal services, keep achieving that more for less challenge. Now, our guest today, Jack Newton, is an eminent figure in the legal tech sphere. We’re very happy to have him here today. With years of experience at the intersection of legal practice and AI, he holds both a Bachelor’s and a Master’s in computer science. He’s also the founder and CEO of Clio, the world’s most widely used cloud-based practice management platform with over 150,000 subscribers in over 90 countries – that’s more subscribers than there are lawyers in Australia! Jack’s insights into how AI can improve legal practice are invaluable, especially now as practice management providers start to integrate AI features into their systems. Jack, thank you so much for joining me today on Hearsay! |
| 00:02:19 | JN | Thanks for having me, David. |
| 00:02:20 | DT | I’m really excited to talk about this topic. We’ve both got horses in this race of AI and legal tech, and we’re both coming at it from different perspectives, because you’ve got a really well-established product that’s got this trove of user data that you can build on, and we’re starting from scratch looking at different functionality. But I think we can both really talk to the possibilities and the realistic possibilities as well. But before we dive into that, can you give our listeners a bit of background on yourself, your professional background, and how you came to be involved in the legal tech environment? |
| 00:02:53 | JN | Yeah, absolutely. So, for me, it’s really interesting. The last year of developments in AI more broadly and in AI as applied to legal has been a really interesting dovetailing of my academic career from 20 years ago and my last 15 years of building Clio. My academic training was in machine learning. As you mentioned at the opening, I got a Bachelor’s and then a Master’s degree specialising in machine learning. One of the big forks in the road for me back in – this is around 2003 – was deciding whether I wanted to pursue machine learning more deeply at a PhD level and get my Doctorate, or did I want to go into industry and take a more applied route to some of my learnings in computer science. I was actually very close to accepting an offer from the University of Toronto to work with Geoff Hinton, who has ended up becoming the godfather of AI – as the name he has in the industry today – because he was instrumental in building neural networks, creating a back propagation algorithm and laying a lot of the foundations for what are the large language models we have today. So, I probably would have been completely unable to anticipate in 2003 that me deciding not to go the academic path and instead of going to industry would 20 years later result in this really interesting collision of these two worlds. Where I see much of what I learned in my Master’s degree and what I learned about neural networks now coming to life in legal, with a lot of the same basic foundations of these neural networks being the same, but just the scale of these neural networks with trillions of parameters and being trained on a corpus of data that is not a few hundred megabytes large or a few hundred gigabytes large, but literally the entire internet and every written work human beings have ever produced. The scale of what we’re doing is just mind blowing and would have again been very hard to even comprehend back in 2003. So for me, having developed Clio and really in a lot of ways, I think, built a foundation of a very broad and very deep level of functionality that enables lawyers to digitise their practice and store basically every aspect of their interactions with a client, whether it’s emails or texts or secure messages sent through our client portal, we have an unbelievable foundation and unbelievable number of workflows that we can start to apply some of these machine learning algorithms and LLMs to within Clio. So, I’m very; number one, optimistic, about the opportunities that LLMs and machine learning more broadly offered to the legal industry. But, number two, I would say because I’m so deeply steeped in the underlying technology through my academic training, I’m also skeptical of, I think, what you referred to as magical solutions. But I don’t believe that machine learning, at least in its current iteration, is going to be able to solve some of the problems that people are either optimistic or fearful of AI being able to solve in legal. |
| 00:05:52 | DT | Yeah, absolutely. And I like that you’ve described that as a dovetail of that academic history and the practical use of these technologies. Because for the casual observer, generative AI, large language models is this overnight innovation, that’s occurred in the context of 60 years of AI summers and winters. It’s the 60-year overnight success. And you’re right, it’s the scale at which these models operate, but under the hood, it’s gradient descent, right? |
| 00:06:20 | JN | That’s right. |
| 00:06:24 | TIP: The concept that was just mentioned is one from mathematics. It’s known as an optimisation algorithm and it’s common in the machine learning space. And to touch on that idea of AI being an older technology that was just discussed, it’s also old. The algorithm was first proposed around 1847 and again independently in 1907. It’s much older than the idea of neural networks, but it goes to demonstrate that LLMs and machine learning really are the 60-year-old overnight success. | |
| 00:06:59 | JN | The same underlying technologies, you know, I think the transformer model, which was I think only invented in 2018, 2019, it’s only been around for less than five years. And I think it’s really the key technology that unlocked this new generation of LLMs. But as you pointed out, the same underlying technologies have been applied to various problems over the last 60 years to varying levels of success. |
| 00:07:25 | DT | Yeah, absolutely. Now you mentioned that Clio has a lot of vectors to get into the use of large language models and generative AI to support its functionality. And you and I were talking to some Clio customers this morning about the things they love about the product, some of the things on the roadmap that they’re excited to see in the future. But it occurs to me, and I say this as a lawyer who’s used practice management systems for decades now, it’s sort of the everything app for a lawyer, right? You want your practice management system to do so much of your day-to-day tasks as possible. You want so much of that information that you have at your fingertips to be in a single place, to be supported by a single source of truth. That’s a broad mandate to support, right? |
| 00:08:08 | JN | It is. And it’s, in a way, almost antithetical to what has become pretty trite advice for startups, which is to do one thing really well, do one small thing really well. And our founding view in 2008, when we launched Clio, was to build actually a very broad set of functionality. And the reason we believed we had to build this broad set of functionality was the true value of a legal practice management system is unlocked when there’s a high level of integration across all of those pieces of functionality. Common data, you save duplicate data entry, you don’t need to enter the client details more than once. Maybe you’ve even got an intake system that the client is intaking their own details once, and you never need to manually enter that information at the law office side of the fence. We see so many workflows being able to be automated and streamlined when you have that complete integrated set of tools. We talked about the intake to invoicing value chain and the fact that really every touch point in that client journey, from intake to actually delivering the work product, to generating an invoice, getting that invoice paid, and even beyond thinking about “how do I get this client to turn into an advocate and write online reviews for me” and so on. There’s a huge value add putting all of that functionality, all of that data under one roof, so to speak. And that’s a journey we embarked on 15 years ago now, in 2008, that I still feel like in a lot of ways we’re just getting started. Because it is such a broad mandate and a mandate with a lot of depth as well. As you heard this morning in the customer conversations, even though we’ve got a huge scope of functionality, there’s a hunger for deeper, more sophisticated functionality and there’s a hunger for more functionality in the adjacent areas and additional areas. We can deliver value to our customers thanks to, I like the way you put it, Clio being this “everything app” that really ideally is an app that lawyers never need to leave. And that’s the feedback we hear from customers all the time. Clio is the first app they open in the morning. It’s the last app they close at night. And, ideally, they’re never leaving Clio, even though they may be leveraging many other tools in the background and we have a huge integration ecosystem with over 250 integration partners. Part of the value out of that integration ecosystem is, again, you can access the functionality and the value of those integrated products without ever leaving Clio. |
| 00:10:32 | DT | Yeah, absolutely. And it makes sense that you’re now looking at integrating large language model functionality, which is this kind of omni-use technology. And I think we’ll talk about this a little bit later, but a lot of the tangible, exciting demo features of large language models and in particular, ChatGPT, are creative tasks. But my perspective at least is that this is a real innovation in search and in access to both structured data, which your practice management system has in troves – matters, matter categories, billings, what’s recorded, what’s written off – and unstructured textual data – your matter documents, your engagement letters, your scopes, your descriptions of tasks. And it’s really that search capability, that semantic search capability that’s powered by the unpopular younger brother of the completions part of large language models, which is embeddings. And using that next token predictor as the kind of last mile to connect what you retrieve through semantic search to what the user actually wants. I guess that’s a long-winded way of me saying there’s a nice synergy there isn’t there between an everything app that houses all of your structured and unstructured data and an omni-use technology that enables you to reach that data so much more easily. |
| 00:11:43 | JN | 100% agree. And I think it’s another one of the huge value-adds of having all of your data under one roof, so to speak. Because your AI system is going to be able to access all of that information and integrate all of that information in a really powerful way. And the way I think about AI and at least our first turn at delivering some of the value AI offers to our Clio customers in this Clio Duo product that we announced at Clio Con about a month ago. We see the real power as making the interface to Clio being a more conversational interface. And as a sci-fi nerd, I think back to the computer on Star Trek, that again, you could ask anything about how the ship is operating, are systems nominal, or for a complete download on what planet we’re about to journey onto and what some of the dangers might be. This was more or less an omniscient computer assistant that knew everything from the status of the ship systems to the fact book on the planet you’re about to visit. And if you bring that to the application we’re talking about in the context of an LPM, I think it’s the same kind of power. Where a customer can have a natural language query and ask; “what’s the status of this matter? I’m about to talk to my client, David. Give me a rundown of what has happened with his matter most recently and get a really high context, very concise overview of what’s happening with his matter.” Workflows you had to learn by reading a manual, or a help desk article, or that you just were maybe able to intuit just by playing around with the software, you can now ask the software to do; “Clio, generate my bills”. And takes care of all of that in the background with zero intervention. “Thank you, Clio Duo. Send my bills to my client.” And again, it’s all taken care of in the background. So, I think it’s going to really lower the barriers to entry for software users in general because they’re going to be able to carry out this natural language, very intuitive conversation with Clio Duo. |
| 00:13:53 | TIP: If you’ve been practising for a while now, you may have started legal practice without much natural language search in legal practice management and research. Many earlier research search platforms used a form of searching known as Boolean search and are only now adapting to natural language. We still use Boolean on many platforms, actually. Think Austlii and other legal databases when you want to refine your results. Boolean searching requires users to know and correctly use specific operators to refine results – like the AND and OR operators. Confusingly, these operators are different across platforms. Natural language is a more humanistic way of approaching these technologies. It’s easier to interact with and has arguably better results for the user. | |
| 00:14:43 | JN | The other really interesting opportunity is that these LLMs that are powering Clio Duo have a much broader knowledge of legal in general. They can be experts in legal marketing, for example. So, you also be able to ask questions like; “how can I grow my firm? How could I increase my number of new clients from where they are today to an increase by 50% by increasing my ad spend?”. And again, having the kind of context and the data synthesis it’s able to do across all of your data in Clio could help make recommendations around where specifically you should be deploying ad spend. Because it knows where you’ve got the highest ROI. So, you can start to think about the ultimate opportunity here as being not just a conversational assistant that can help automate workflows in Clio, but a business partner that can help you understand how to navigate specific workflows, and optimise specific workflows in Clio to help grow your firm more quickly, or how to deliver a higher level of customer service. And finally, there’s the opportunity for these assistants to be supporting you in the practice of law and be able to help do things like write briefs for you and find the appropriate cases to cite and so on. So, I think the range of applications is almost limitless. And again, taking this perspective that at Clio, we really have a view of all the key workflows that are happening in a firm. I think we’ve got a scope of data and a scope of workflows that is going to be years of opportunity for us to bring the power of AI to and help either streamline, automate, or unlock creativity in those workflows for our customers. |
| 00:16:22 | DT | Yeah, that’s really the other side of the innovation coin here, right? We said that the big backend opportunity, I guess, is search. The frontend opportunity is the first sort of change to user experience, the fundamental change to user experience that we’ve had in like 40 years. The graphical user interface has been kind of how we interact with computers until today, right? We’re looking at a real prospect of the natural language query or the conversational interface replacing that. Is that really how you would capture Clio Duo in a nutshell? That it’s UX that unlocks the dynamic insights from the data you’ve already got, and makes the functionality of Clio that you already have more easy to access, more intuitive to access, for someone who’s not spending days reading the documentation? |
| 00:17:08 | JN | That’s part of it. And I would say that’s probably the surface level opportunity for a lot of customers will be just having an easier natural language interface to discover Clio’s functionality and to activate some pieces of Clio functionality. I think beyond that surface layer, the limit to that perspective is to some degree – that’s what some of these, what I would call “dumb” chatbots have been doing for maybe five years now – which is the support chatbots that we’ve all interacted with. Certainly not all that intelligent. Sometimes they can border on being deeply frustrating in terms of trying to actually navigate and get help with an application. The key difference here is the LLMs that are supporting the chatbot interfaces like Clio Duo is their ability not just to ease accessing the functionality, and helping you identify how to do something in Clio, or even asking the chatbot how to do something, but this opportunity for it to really be a business partner and to integrate information across different pieces of Clio and to actually offer really insightful, deep recommendations to how you might optimise your practice, how you might grow your practice. You could ask questions like; “what are my most profitable practice areas”, and have the Clio Duo come back and answer that question for you, and say “I’ve analysed your last two years of billings and what your gross margin has been… and real estate and conveyancing has been one of your best practice areas”. And help guide you in a way that I think you think about a partner more than just an assistant, somebody that can really help offer powerful perspectives on your law firm’s business and what the opportunities might be to optimise your business, grow your business and so on. So, I think the initial experience for a lot of people will be the tip of the iceberg, some really easy, nice workflows and again being able to say something like please generate all of my bills and send them to my clients. That essentially being your billing workflow. And, again, I talk to a lot of customers, they currently spend days or maybe even weeks of time at the start of every month getting their bills out the door. For that to be one or two sentences that you utter to your digital assistant, that’s incredible on its own. But I think what lies below the waterline in terms of the power of these LLMs to integrate information across your practice, to help you generate new content, to help you generate new content based on precedents already in your document store in Clio is really exciting and what I would view as the second phase of what most customers will discover as the true underlying power of LLMs. |
| 00:19:41 | DT | Absolutely and what you’ve drawn out there is both the assumption around what AI can assist us with in our practice and the promise, I think, but a realistic promise of what it can offer. The heuristic we have is well the monotonous, the repetitive, the time consuming, that’s what AI can help me with. And that might not even be something that you need a large language model for, that might just be programmatic. But what you’re describing there is, well actually there are some higher order analytical tasks that we can point a large language model at that we can feed reliable structured data to and get some really powerful analysis. That’s at the strategic level where at the top of the episode we’re saying; “oh, well, it can’t assist us with the higher order strategic analytical thinking that we do as lawyers”, but there’s some promise there, right? |
| 00:20:32 | JN | Absolutely and I think what we’ll see is these systems are not perfect and their capabilities are not unlimited. I think that’s where we’ll go through the classic journey of new technologies and sometimes overweighted expectations for what those technologies can deliver and the old quote that the impact of these transformative technologies is often overestimated in the short term but underestimated in the long term, I think will take hold here. We saw this happen with the internet and I think we’ll see it happen with generative AI where there are limits to what the technology can do today. There’s severe bugs – almost if you want to think about it that way – in the form of hallucinations and confabulation that these LLMs can fall prey to. I’m sure even as far away as Australia has heard of this case of the Manhattan lawyer that filed a brief that was written by ChatGPT. And ChatGPT unfortunately wrote a pretty great sounding brief actually overall but had the small problem that the five cases it’s cited in that brief were hallucinated and didn’t exist. And the poor lawyer that filed that brief had to talk to the judge that said; “hey, I as well as opposing counsel, have tried to reference these five cases and they don’t seem to exist. What exactly is going on here?”. |
| 00:21:53 | TIP: That New York decision saw lawyer Stephen Schwartz and a colleague from firm Levidow, Levidow & Oberman fined US $5,000 for submitting fake citations. The judge, Judge Castel, took a technological view of the situation in the decision and stated that while there was no general problem with using AI and technology and legal work, it was the role of lawyers under the relevant practitioners’ rules to act in their traditional role of gatekeepers of the justice system and to ensure the accuracy of their work. | |
| 00:22:27 | JN | And I think that’s an example of to your point I think user expectations of how this technology should work really having a bit of a mismatch with its true underlying capabilities. And we do a Google search and if we look at the top five search results for a Google search, we don’t expect those websites to be lying to us for example. Like Google’s pretty good at filtering out the completely fake websites and the fake information. If we go to a Wikipedia page, again it’s not perfect but 95 plus percent of the time the information you’re looking at in a Wikipedia page is going to be correct. And again, out of the box, these LLMs unfortunately are great lying machines. They can be perfect and very convincing pathological liars. They’re almost the most dangerous type of pathological liar because they don’t even know they’re lying. |
| 00:23:15 | DT | Yeah. So that New York case is a great demonstration of hallucination but it’s also a great demonstration of another weakness that foundational models unassisted by other techniques have which is sycophancy. Because I think the detail in that case it sometimes missed and I think this is both where the harm of the foundational model was experienced but also where the practitioner himself really fell down was when asked by the judge, “I can’t find these citations. Are they real cases?”. Where did he go to verify that? Back to ChatGPT, right? “Are you sure these are real cases? Because I can’t find them”. And ChatGPT said, “yes, of course. They can be found in many different law reports. They can be found in LexisNexis. They can be found in Thomson Reuters”. And he passed that information on uncritically. And that completion was generated because that’s the answer that you want to hear, right? It’s not contentious. It’s leading down the path that the user has suggested. So that sycophancy as well I think is often a source of that hallucination risk as well. I have a personal story to do with that problem. When we first started working on Lext’s own AI application, Ask Lexi, at the tail end of last year, we started with training a model. We thought we could fine-tune a foundational model to provide citations to corporations law cases. We learned very quickly that we had trained a model that was very good at lying about corporations law cases in Australia. It knew all the right parties and the right sort of date ranges and what a medium mutual citation looked like in Australia. We had taught it that really well. It just had no relationship to reality, right? |
| 00:24:42 | JN | Minor problem. |
| 00:24:43 | DT | Yeah, that’s right. So, we quickly came upon the idea that you need to provide that reality, that ground truth, that source of reality for the model through something like retrieval augmented generation. And I imagine that’s what you’re doing with Clio Duo as well. You’re passing in that data that you know to be true in order to assist the user with it. |
| 00:25:00 | JN | Yeah. And there, you know, this fine-tuning process is really key for the models. And these aren’t intractable problems by the way. There’s examples of tools like what Casetext is doing with CoCounsel, I think, is a great example of how you can build a reliable machine learning LLM application for legal that you’re not constantly checking over your shoulder for a hallucination or something else that is running the risk of getting you disbarred, right? I mean, that New York lawyer is lucky that he didn’t get disbarred for that level of malpractice in my view, because you’re just putting complete faith in a machine. And by the way, putting such complete faith in a machine that you would never put in a junior associate, for example. |
| 00:25:38 | DT | Yeah, that’s right. |
| 00:25:39 | JN | So, I think it’s a great cautionary tale. And the judge, by the way, when he was assessing what sanctions to give this poor lawyer after misusing ChatGPT in such a egregious way, was a $5,000 fine. Which seems minor… but along with that $5,000 fine, said “you’ve been humiliated around the globe in every major newspaper, every blog, so that is a form of your punishment as well”. So, I think that is telling, just in terms of the kind of ramifications that the misuse of AI can have for lawyers. But where I’m being really proactive about, what I view as the promise of AI, is I don’t want to see happen is a version of what I saw happen in 2008 through basically the emergence of the COVID-19 pandemic. Which was an enormous amount of fear, uncertainty, and doubt about the cloud and the safety of the cloud holding back innovation in legal. And I view the advent of LLMs and really AI in legal, true AI in legal in 2023, is number one; a much more transformative technology. I think the cloud was almost a change in modality around how you’re accessing your technology. AI is fundamentally transforming how lawyers work and fundamentally transforming their capabilities fundamentally. In my view, amplifying their impact and giving them superpowers that they can leverage to become dramatically more productive, more responsive to their clients, something that in aggregate, I think, can really help close the access to justice gap. So, we can’t let the fear of this new technology prevent us from adopting it. We should look at the cautionary tales. We should look at how we need to iterate on both the core underlying technologies, as well as iterate on our views of safety and how to ensure safety in these models. But the promise of these models is so great that we can’t turn our backs on them just because there’s something new and something, at least in some context, can come across as scary. |
| 00:27:41 | DT | Well, let’s spend a little bit more time on those fears, misconceptions around the technology, because again, I really noticed a change in customer conversations after that New York case became well-known. Beforehand, I was going around to early customers explaining this retrieval augmented generation concept. And they said, “well, why wouldn’t I just use ChatGPT? I can type a legal question into ChatGPT and get an answer”. It was difficult to differentiate the technique. After that case, I really heard that; “well, can I really trust it though?”. So, there were misconceptions both before and after in terms of, well, large language model powered applications aren’t a monolith. There are a lot of different design paradigms that can be deployed there. But definitely one of those fears is reliability. And one of those misconceptions, I suppose, is that it’s an intractable problem. You’re talking to customers all the time about what’s coming with Clio Duo. What are some of the things you’re hearing? |
| 00:28:33 | JN | Yeah. Well, if we zoom out maybe to the biggest picture possible in terms of what we think about as some of the resistance and concerns around machine learning, AI in general, large language models in particular. I think we see some of the basic reactions that we saw to other forms of revolutionary technology over the course of human society. So even if we rewind back to the advent of the mechanical loom and the Luddites led by Ned Ludd, who originated the term Luddites, where his followers were raiding factories in the middle of the night and destroying these mechanical looms because they were worried they were going to take away from their jobs. |
| 00:29:18 | TIP: The historical and mythical analogy that Jack just mentioned is quite apt. Ned Ludd is a legendary and likely mythological individual in English history. He was the mascot of a group of 19th century textile workers who rejected the use of machinery in textile operations, going as far as to destroy such machines. The etymology of the modern meaning of the word Luddite can be traced back to members of that 19th century movement, the Luddites or the followers of Ned Ludd. | |
| 00:29:54 | JN | And I think the same underlying fear drives a lot of the reactions we’ve seen not just from lawyers and adjacent legal professionals but from all sorts of professions across so many different disciplines. Goldman Sachs released a report identifying the top professions they saw as being potentially disrupted by AI and legal was second highest on this list with 44% of the tasks performed in legal as assessed by Goldman Sachs being automatable by AI. So, I think the first thing we need to tackle is this underlying fear that AI is going to take my job away from me. And again, when we look at the history of what’s happened in major technology revolutions over human history, rewinding all the way back to the first industrial revolution, thinking about what even the assembly line for automobiles helped create. Again, there’s a lot of fear about this taking away from jobs from the people that basically made bespoke automobiles and spent months building one car. What we saw in each of these situations and what we’re seeing with legal in particular is a flavor of what is called the lump of labour fallacy. And this lump of labour fallacy is one where we view the amount of work that needs to be done by the people that provide a good or service to an end consumer as being fixed, i.e. there’s a fixed amount of demand. And so, any automation, whether it’s a mechanical loom or whether it’s AI, any automation that feeds into that labour is going to be taking away from the human share of that labour, i.e. it’s a zero sum game. And any automation, any workflow is going to take away from humans in providing that good or service. Now the reason that’s a fallacy is what’s been shown to be the case over and over and over again over the course of our history has been when you apply automation to the problem, when you apply automation to producing that good or service, you lower the cost of that good or service. In lowering the cost of that good or service, you bring more customers and more demand to that good or service. And that actually brings more people providing that good or service to the table. And what’s very interesting is you’re also increasing wages for the people delivering that good or service because thanks to this technology, the marginal value that each person is delivering in that supply chain has increased. So, what you’re doing is actually growing the total addressable market for that good or service and everyone that participates in that economy, whether it’s a consumer of that good or service or whether it’s a provider of that good or service is also benefiting in kind of a virtuous cycle of expanding TAM and expanding and increases wages in providing that good or service. So, let’s think about that in the legal context. Legal services are globally in a severe shortage in terms of the mismatch between demand and supply. |
| 00:32:53 | DT | Oh yeah, I think the rate at which demand is unmet for legal services is roughly the same in the US and Australia. It’s around 85% of demand unmet. |
| 00:33:02 | JN | That’s right. Exactly. I mean, that is the access to justice gap. In a nutshell, 85% of people that have a legal problem are not being matched with a lawyer or a legal professional that can help them solve that problem. So, when we think about this in the context of the lump of labour fallacy, I think a huge opportunity for legal around the world is to dramatically increase the productivity of the legal economy to help each lawyer and every legal professional participating in that economy to amplify their impact and to service more of this unmet legal need. And again, the worldwide spend on legal services is well over a trillion dollars a year only meeting 15% of the demand of legal services. The back of the napkin math I do is looking at that unmet demand and the current level of spend. And if you tap into this latent legal market, which is this 85% of the market that’s not served today, there’s a multi-trillion-dollar opportunity for the innovative law firms that think about how can I use technology, how can I use AI, machine learning, LLMs, you name it in concert to unlock my productivity and to access this latent legal market. And I think that’s just a very exciting opportunity and one that I think most lawyers should look at themselves as almost having a moral imperative to integrate technology in general and AI in particular into their practice to actually expand their reach and to deliver on this opportunity to expand access to justice. |
| 00:34:32 | DT | Yeah. The heuristic I like to use is we have to change how we think about what warrants legal advice. If we paid a doctor $2,000 for every consult, we probably wouldn’t go see the doctor just in case when we had a fever or a sore throat, right? We’d be very selective about the kind of medical issues that warranted professional intervention. |
| 00:34:52 | JN | A hundred percent. By the way, just on that point, we talk a lot about the power of these conversational interfaces for lawyers, but I think these conversational interfaces, to your point, are actually a really powerful opportunity for consumers of legal services as well to understand what’s interesting when you unpack that 85% of legal demand that is not serviced by a lawyer. There’s some portion of it that is for all the reasons you’d expect, like they can’t afford a lawyer or they think accessing a lawyer is scary and complicated, but there’s actually a decent chunk that is also people that don’t even understand they have a legal issue. And look at landlord-tenant law. There’s a lot of cases where somebody might be facing an eviction and they don’t even understand what their legal rights are. So I see an opportunity for LLMs in that context is also being, again, cognisant of the laws of a specific jurisdiction, being able to have a very powerful and elucidating conversation with the potential legal client around what their rights are, what their current status is, and potentially even being able to refer them directly to a lawyer that can then help them address their specific need with almost that screening process of what is the underlying legal issue and what is a lawyer that’s well matched to solve that legal issue going to look like. And in some cases that might be, like you said, resolved by a non-lawyer in some form. There’s some very interesting examples of that happening in some jurisdictions with some tools being able to automate those workflows. And again, let the lawyers focus on the most complex, nuanced cases that really need that level of human discernment you were referring to earlier. |
| 00:36:29 | DT | I mean I suppose that vision for the innovative law firm that wants to tap into that latent demand, that’s a very different picture of the legal services market. When we conceive of the legal services market, we tend to think of a small number of high value clients, a relationship marketing focus, repeat work. We don’t really think about this kind of productised, high volume, low unit cost kind of legal service. And we’ve seen a handful of firms attempt that, at least in Australia, with varying degrees of commercial success. What do you think the appetite is, speaking to Clio customers, to make that really dramatic change from serving the traditional 15% to the untapped 85%? |
| 00:37:16 | JN | Well, one initial comment I would have is it’s not for everybody. There’s certainly some lawyers that are more comfortable with a more traditional delivery model. Again, that 15% of the market that is being serviced today is a trillion-dollar market. So, there’s a lot of room for people to execute on business plans that look very similar to how a law firm business plan looked like in 1980 or 1990, and that’s not going away anytime soon. I think for the lawyers that want to unlock the opportunity in this latent legal market, it really demands an appetite for risk and it gets both a technology problem/opportunity, as well as a mindset problem/opportunity, where you can think about the mindset of a lot of lawyers historically has been quite lawyer-centric. I wrote a book on this topic called The Client-Centered Law Firm that really advocates a mindset shift for lawyers to think in a more client-centric way and to basically work backwards from what the client problem is and how they can offer a solution to that. And the solution to that, most often I think if you do the real groundwork on understanding how can I deliver value to my customer, the answer is not a traditional billable hour engagement. It’s really thinking about the underlying customer needs, the underlying client needs, and working backwards to how can you price and package and deliver your legal services in a way that truly, deeply solves that customer problem. So, there’s a mindset shift that again, if you’re an innovative lawyer, what I’ve just described there sounds like music to your ears, like I want to do that. I’m really interested in unpacking legal problems in a new way. But if you don’t want to do that, again, there’s a path that is more traditional and I think that more traditional approach is going to be around for a long time. But for innovative lawyers that want to unlock this multi-trillion-dollar opportunity that the latent legal market represents, you can create really strong win-win-win outcomes. And I’ll give you one quick example of this. It’s one of my favorite examples to highlight. It’s a Clio customer named Erin Levine, who for a long time ran a traditional divorce practice. Typical divorce, and this is a U.S. based firm, a typical uncontested divorce when she was deploying her law firm on a specific client case would cost about $20,000. And that’s for an uncontested divorce. And she thought this is just a huge cost to my clients that I think I can streamline with the help of technology and automation and so on can really help lower the cost of this. So, she created a new divorce delivery model called Hello Divorce. And it’s a website driven intake process. It’s very specific in the kinds of problems it solves, which is an uncontested divorce. It leverages technology to help both sides of the divorce, navigate the divorce process, and is able to deliver an uncontested divorce for about a thousand dollars. And the really cool thing that Erin shared with me is that she used the same staff from her old firm that was delivering divorces in this kind of traditional hourly rate basis model and transitioned them to the Hello Divorce model. And they were, as you’d expect, thanks to the automation and the technology, able to really increase the throughput of what each lawyer was able to do. So, they were able to do many more divorces at a lower cost per divorce, but at an overall throughput that saw the individual lawyers making more money with this Hello Divorce model. From a client perspective, you’re really lowering the barrier to entry to access a divorce. You see a lot of people stay in bad relationships, abusive relationships, because they simply can’t afford to get divorced. And so, they’ve really helped solve the friction points in clients accessing this divorce. And thanks to how slick this technology is and how easy it is to basically go on a website and start the divorce process, they’ve opened up the path to divorce for a lot of people that should, for family reasons, for safety reasons, for mental health reasons, seek a divorce. They’re able to create basically access to justice for those people as well. So, I talk about that as a win-win-win scenario and an example of the legal economy expanding where there’s, number one, better outcome for clients. They’re getting a better outcome. They’re getting the divorce they want or need. And they’re getting it done in a pretty quick and efficient manner, thanks to technology. The lawyers providing that service are making more money and they’re happier because they’re creating better outcomes more quickly for their clients. So happier lawyers, more profitable lawyers. And finally, you’re really making progress on both the access to justice piece in that you’re creating these good legal outcomes for people that would have otherwise been unable to access those legal outcomes. And you’re tapping into that latent legal market, which is all simultaneously solving for that access to justice problem. Basically, if you’re helping solve and access the latent legal market problems, you’re, by definition, helping increase access to justice and you’re expanding the size of the legal economy overall. So that’s just one example of, I think, where you approach a problem from a kind of first principles perspective. Erin worked backwards from what does the client need and how can I create good legal outcomes for them. Figured out how she can leverage technology like she uses Clio, she uses document automation, she uses a variety of backend tools to really maximise the productivity of her lawyers. And she’s able to create this really exciting win-win-win outcome for everyone involved. |
| 00:42:43 | DT | And the capacity to design those solutions is just going to explode. I mean, we saw some exciting announcements from OpenAI – we’re recording in early November – just this week in terms of developer tools so that even developers without much background in machine learning or AI can spin up a ChatGPT clone pretty quickly for that particular use case. We’re going to see that within reach for the law firms that want to do it. |
| 00:43:09 | JN | Absolutely. And I’ve just finished reading the press release of the OpenAI announcements this morning, and I agree with you. I think it’s going to unlock a new wave of innovation and the barriers to entry for these AI tools is going to keep coming down. And again, it’s very reminiscent of what we saw with previous revolutions in technology where initially it’s accessible to just a handful of people, but it’s broadly eventually becoming accessible to the masses. And I think we’re seeing that trajectory for AI follow a very compressed path. We’re also basically celebrating the one-year anniversary of OpenAI being released to the public or ChatGPT rather being released to the public in November of 2022. And it’s just been an incredible year of innovation. And you think about what does another few years of Moore’s law and the power of computing continue to increase in an exponential way, coupled with the amount of innovation that’s happening at the underlying software level. It’s very exciting. I feel like we’re just getting started in so many ways. |
| 00:44:11 | DT | Absolutely. Well, let’s wildly guess. Let’s speculate a little bit. What do you see on the horizon for AI in legal practice? I’ll go first. One of the things I was actually just talking to someone in your team about it this morning, I feel like a lot of conversations about AI and especially B2B implementations of AI at the moment are, which commercial model are you using and whose API are you using and how much does it cost per token? I hope, and I think that we’re going to see that issue really become a thing of the past very soon. There are highly performant, small models, 7 billion parameter models that you can run on a MacBook. And it wouldn’t surprise me at all if we saw performant models running on an iPhone in a few years time. And that virtually nil variable cost of running inference on that model is going to unlock functionality and the viability of features that at the moment are just too expensive to run. One huge technology player that we haven’t heard a lot from in the large language model or generative AI race is Apple. The capacity to deploy on device and run locally on consumer hardware would be a huge game changer and I think it’s not far off. |
| 00:45:25 | JN | Yeah, I agree with all of that. I think that there’s some cost handcuffs that a lot of companies are in right now just because of the huge cost of compute for these models, the lack of availability of these Nvidia H100 chips that power these large language models is really, ultimately constraining – like a supply chain constraint more or less on the rate of innovation that can be happening with AI. I think for what the average company and even the average consumer is using these chatbots for, using GPT-4 for, this is like using a sledgehammer to hammer a tiny nail into a wall. And the models I believe will continue to shrink in a significant way and get the cost per token down in a dramatic way. |
| 00:46:14 | TIP: Cost per token can be a key measure of the commercial viability of a large language model. The cost per token changes depending on which model a user or a business might choose to use. At scale, these costs can add up massively. | |
| 00:46:50 | JN | We’re seeing open-source models also iterate and improve in a really rapid way. that I think is unlocking a new generation of applications that will allow, as you said, smaller parameter models focused on a very specific subset of the technology and again, open up new applications for on-device applications. I think Apple has done historically a really great job of developing their machine learning applications in a way that is very privacy and safety first. So much of what they’re able to create in terms of even their existing machine learning models, they pride themselves on those being on-device models that are not in the cloud and not subject to a court order or something else to submit those models to the government and I think that’s a real superpower of the way Apple thinks about technology. And I think we’ll see more and more companies thinking about how can we in our own way deploy models that respect the privacy of our customers and make the safety considerations of these models really explicit. If we shift our thinking from how’s the underlying hardware and software working, to how is the application going to change over time, I really think that ultimately the power of these large language models will be one on the generation side and one on the assistant side. In legal, the power of these models is currently really focused on text-based applications and legal is probably the most text-based profession in the world. So, I think we’ve already seen great work by folks at Spellbook is one example that are helping streamline the creation of legal documents and helping you create clauses in your legal contracts that meet specific demands that you can just, again, express in natural language. We’re seeing what Casetext has done with CoCounsel. I think is another great example of how can I generate documents with a very rough framework, again, laid out in natural language and produce a really powerful and complete legal document in a pretty automated way. If we look at what Clearbrief is doing, applying AI to writing briefs and making sure your citations are as complete and as rigorous as possible, I think that’s another really exciting application on the creation side. And I think we’ll just see the power of those tools continue to scale at some exponential rate over the course of the next five or 10 years. I think on the assistant side, we’re going to see both in the legal context, but I think even more broadly in our personal lives, these chat models evolve to being really complete digital assistants that we can just start describing even things like vacations and a potential itinerary we want for a vacation. These chatbots will be able to go out and book our flights, book our hotels, suggest itineraries based on what they know we like, the hikes we’ve done in the past and how we’ve told the assistant we liked about a particular hike. It’ll be able to go discover new routes and new paths for us. So, I think when we think about these assistants bringing their power to our personal lives, I see a flavor of that also existing in our professional lives. Almost independent of what professional we’re in, there’s going to be an AI assistant that can help us in some really specific powerful way, whether it’s in creating a document or as I mentioned at the opening of the show, even a business assistant that’s helping us make the right investment decisions for our advertising spend or what our next hire at the firm should be. It’s going to be a very exciting evolution of these technologies over the next few years. |
| 00:50:16 | DT | One of the things you described there is that kind of agent approach to large language models, the ability to act on the world outside of the completion interface. In fact, you described some of that with Clio Duo. Clio send my bills to my client. It’s low-hanging fruit in terms of agentic functionality. We were talking about fears, concerns, misconceptions, safety issues. When you talk about large language models having agency, sometimes people get a little bit nervous about that, right? The idea that they can act on the world beyond the interface. And even some of the conversations that we have about how safe a model is, we often talk about; “hey, it’s not acting on the world without a human carrying out those instructions”. So how do you see the safety implications of the agentic AI, the AI that can act on the world? Because I think it’s inevitable and I think it’s going to be a powerful tool for good, but it’s a step further than what a lot of people at the moment are just getting comfortable with, right? |
| 00:51:19 | JN | It is. I think we’ll see a progression on what this looks like over the next few years. I think what we’ll see to start is the AI making a recommendation to us and us needing to approve that recommendation before it goes ahead with the action. So even in Clio, for example, when we demo the bill generation workflow I just described, the workflow today is not generate all my bills and the Clio Duo does not just go do that and sends the bills to your client. It comes back and says, here’s the bills that I’ll go ahead and send to your clients with your confirmation. So, there’s a confirmation step involved. And I think what we’ll see is as our comfort increases in the same way, our comfort, by the way, increases with human beings as we ask them to do new tasks as well. As our comfort level increases, we’ll move away from that confirmation step. I think knowing that; “hey, I’ve done this a hundred times and a hundred times the AI wanted to do the right thing”. We have become very trusting of the AI being able to exhibit at least as good judgment as your average human being. I think about most decisions and actions as well in terms of, are these two-way doors or one-way doors? And you always need a lot more, I think, human intervention and discernment when you’re talking about these one-way doors. But a lot of the decisions we make on a day-to-day basis are two-way doors. And I think we’ll see AIs embrace that concept. And again, if it books the wrong hotel for you on that vacation, acting as an agent, no problem. You can just cancel and change that and make a different reservation. So, I think as we trust these AIs, we’ll have the same trust we have with any even human assistant that there’s going to be certain decisions that are very easy to reverse or change and the ones that are not are the ones we’re going to need to engage in more actively. So, I think we’ll look back at cruise control and what we’re seeing even with autonomous vehicles as maybe a good example of what this has looked like in slightly different application areas. But Waymo has been able to move away from having a human driver, basically doing that verification loop I was talking about in real time, continuously driving day-to-day in the streets of San Francisco to now being able to operate for thousands of miles at a time with zero human intervention and no co-pilot in the seat, for example. |
| 00:53:38 | DT | I think they published their insurance data that suggested that the unmanned vehicles were safer. |
| 00:53:43 | JN | That’s right. Safer than human drivers. And again, this is going to vary significantly by the specific model and the provider of the AI. We’ve seen the counterexample, I think, in terms of some of Cruise’s recent experiences where they’ve actually had to pull their vehicles off the street because their AI wasn’t mature and it wasn’t making good decisions. But I think this idea of AI being able to act within a high level of agency and us ultimately trusting these AIs more innately is the inevitable path we’re going to be going on. We’re going to be, I think, in 10 years wondering how we ever survive without these agents helping automate and complete many of our daily tasks and helping us out in more ways than we’ll be able to count in a few years. |
| 00:54:24 | DT | I couldn’t agree more. Clio operates in 90 countries? |
| 00:54:28 | JN | Yeah, over 100 now. |
| 00:54:29 | DT | Over 100? Wow. It occurs to me that there’s probably some different patterns in the enthusiasm for adopting legal technology, especially adopting AI-powered legal technology around the world. Have you noticed any patterns? |
| 00:54:41 | JN | Yeah, I think we certainly see. What I would say in general, not even specific to AI, is there’s across all the regions, we operate different levels of appetite around technology and embracing technology. And I’ve got to say our experience over the last year in Australia as we’ve opened up our office and launched a presence in Australia, Australia is one of the most tech forward markets I’ve ever had the privilege of working with and engaging with. Where it feels like the baseline for technology, if you talk to most Australian lawyers, is of course, we’re in the cloud, that’s table stakes. Now we’re thinking about how we can automate the technology using triggers, automations, workflows, maybe low code solutions, maybe figuring out how do we weave Zapier and other tools into our workflows to help automate and streamline things, to being very ahead of the curve in thinking about AI and the ways we can apply AI to technology. So, I do see a varying degree of technology forward thinking in the different geographies we’re operating in, but regardless of which geographer we’re operating in, we’re usually attracting the early adopter and the innovative set. If you’re not innovative, if you’re stuck in your ways, you’re probably perfectly happy with the solutions that are on today that have been on the market for 10 or 20 years and you might even still be using an on-premise solution. But we have, I think the good luck of most often as we enter a new market or even see organic adoption in a new market, it’s those innovative lawyers that are thinking; “how do I do things in a new way?”. Clio looks like a great foundation to build in an innovative practice on top of, and they go and adopt Clio. |
| 00:56:12 | DT | Jack, we’re nearly out of time. Before we leave our listeners, I had one question for you to close things off. We’ve talked a little bit about how early adopters are using Clio, how lawyers who are optimistic about the future, who have their eyes open to the opportunities that AI can open up for them are going to be well placed to capture that latent demand, to grow their practices, to ride this wave of innovation into the future rather than be dumped by it. How does a lawyer stay one of those early adopters? How does a lawyer who’s listening to this, who likes the idea of what we’ve talked about, but isn’t sure how to stay on top of these developments and they are rapid, it’s hard to stay on top of. What are your tips for lawyers who want to stay abreast of developments in AI and in legal technology and who want to stay on that cutting edge? |
| 00:57:00 | JN | Yeah, that’s a great question. And I’ve got a few different elements of a response. One is when you think about the classic technology diffusion curve that goes through the early adopters to the early majority, the late majority, and then the laggards. I know we’re on a podcast, but I’m shaping out the shape of a bell curve. |
| 00:57:19 | DT | You’re making a bell curve with your hands, standard distribution. |
| 00:57:21 | JN | If you look at that bell curve distribution, if you’re a lawyer thinking, how do I get an edge on the competition? If you just make an effort to be on the left half of that graph, if you make an effort to be an early adopter, if you’re really like bleeding edge, or even you want to be in the early majority, you’re going to by definition be ahead of 50% of your colleagues by focusing on being ahead of the curve. So, to the underlying, I think, premise of your question, there’s a huge amount of value and competitive advantage in figuring out how you can stay in that front half of the curve. And it doesn’t mean you need to be bleeding edge. It just means you need to be adopting technologies in a pragmatic fashion. That’s going to put you ahead of most of the market. We talked earlier in the podcast, answering the question is technology, is AI especially going to replace lawyers? My answer is an emphatic no, and I outlined why. But what I do think is true, that’s the next step of that evolution is that lawyers that leverage AI will in the long term displace lawyers that are not. So, I think when we’re looking at whether it’s AI or cloud technology, embracing these technologies is actually an existential imperative. It might take a decade or more for this to play out. But I think if you’re thinking about how do I build a law firm that will be thriving, not just over the next two or three years, but over the next 20 or 30 years, really figuring out; “how do I stay ahead of the curve? how do I stay on top of technology?” Is crucial. So, there’s a few ways of doing that. I think number one, find other people like you that are interested in technology and passionate about technology. There’s a saying I love, which is you become the average of the five people you hang around with the most, right? And I think that applies to your technology life, it applies to your professional life. And if you want to get better, find five people that are smarter about this stuff than you and start hanging out with them and learn. Find five podcasts like this one that talk about the topics that you feel like you’re expanding your knowledge and learning new things on and make a habit of listening to those on a weekly basis. Go to conferences that talk about this stuff. That’s one of the best ways I think of immersing yourself in two or three days of deep learning. It’s one of the reasons we founded the Clio Cloud Conference almost 11 years ago now, where we felt like there was a lack of an opportunity for lawyers to get together and just talk about innovation, right? Like this isn’t focused on CPD. This isn’t focused on checking boxes on training. This is focused on practical ways you can embrace technology to help reinvent the practice of law. So, I realised that’s on the other side of the world in Austin next year, but would certainly invite all of your listeners to join us at Clio Con, which is really focused on innovation and is starting to attract a real worldwide audience of attendees. And then read a lot. There’s so much high-quality literature out there. Even if you want to go and learn the nuts and bolts of how an LLM works, there’s some great explainer articles that talk about the transformer model and how these technologies work that a lay person can read and understand. So, I think there’s some mystique around these technologies that you can really help unpack and build your ability to discern how you might be able to use these technologies, where they’re best suited, where they’re ill-suited, just by spending some time on the internet and doing some research into the underlying technologies. So that’s the starting point. |
| 01:00:40 | DT | All great tips. And yeah, Clio Con, look, it might be a long way away, but it is a great way to get your flight to the US, tax deductible. |
| 01:00:47 | JN | It is, absolutely. And Austin is a super fun city, great music, great food, and we know how to throw a good party as well. So, we’d love to see you and some of your listeners there next year. |
| 01:00:57 | DT | Fantastic. Well, Jack Newton, CEO of Clio, thank you so much for joining me today on Hearsay. Yeah. |
| 01:01:02 | JN | Thank you, David. |
| 01:01:13 | RD | As always, you’ve been listening to Hearsay The Legal Podcast. I’d like to thank our guest today, Jack Newton from Clio, for coming on the show. If you’re an Australian legal practitioner, you can claim one continuing professional development point for listening to this episode. Whether an activity entitles you to claim a CPD unit is, as you well know, self-assessed but we suggest this episode entitles you to claim a practice management and business skills unit. More information on claiming and tracking your points on Hearsay can be found on our website. Hearsay The Legal Podcast is brought to you by Lext Australia, a legal innovation company that makes the law easier to access and easier to practice, and that includes your CPD. Hearsay is recorded in Sydney on the lands of the Gadigal people of the Eora nation, and we would like to pay our respects to elders past and present. Thank you for listening and see you all on the next episode of Hearsay. |
You must be a subscriber to access this content.