Want to listen to the full episode and all our other episodes?
Hearsay allows you to fulfill your legal CPD requirements every year.
Our yearly subscription is only $299/year.
With a yearly subscription, you can access all of our episodes AND every episode we release over the next year.
Episode 89
Buy Episode
Practice Management and Business Skills Professional Skills
Evolution or Revolution? Correcting Haphazard Use of Technology and the Shape of Things to Come
Law as stated: 20 June 2023
What is this?
This episode was published and is accurate as at this date.
Legal Technology Evangelist Fiona McLay joins David in Curiosity to push the envelope of legal technology. Focusing on what can be done now with existing tech stacks and how to prepare for the inevitable encroachment of generative AI.
Practice Management and Business Skills
Professional Skills| What area(s) of law does this episode consider? | Generative AI and the importance of understanding machine learning |
| Why is this topic relevant? | Generative AI – and machine learning more generally – are fast becoming crucial tools in the legal industry, transforming the way lawyers practice law. The zeitgeist is, of course, ChatGPT – but OpenAI’s gregarious creation is far from the only game in town. Today, generative AI can create content such as text, images, and even certain legal documents. And, in future, it has immense potential to assist lawyers with tasks such as drafting contracts, analyzing large volumes of data, and conducting legal research. As the legal technology landscape becomes increasingly complex and data-driven, the ability to leverage machine learning tools like generative AI becomes essential for lawyers and law firms to deliver efficient and accurate legal services. |
| What are the main points? |
|
| What are the practical takeaways? |
|
David Turner = DT; Fiona McLay = FM; Ross Davis = RD
| 00:00:00 | David Turner: | Hello and welcome to Hearsay the Legal Podcast, a CPD podcast that allows Australian lawyers to earn their CPD points on the go and at a time that suits them. I’m your host David Turner. Hearsay the Legal Podcast is proudly supported by Lext Australia. Lext’s mission is to improve user experiences in the law and legal services and Hearsay the Legal Podcast is how we’re improving the experience of CPD. So, generative artificial intelligence and machine learning more generally are transforming the practice of law already and they’re fast becoming tools that lawyers need to understand to thrive in today’s market. The famous generative artificial intelligence tool that we’ll all have heard of is ChatGPT, but OpenAI’s creation is far from the only game in town. Now already generative AI can generate content like text, images, and it can even prepare certain legal documents. And in future it has immense potential to assist lawyers with tasks like drafting contracts, analyzing large volumes of textual data, and conducting legal research. As the legal technology landscape becomes increasingly complex and data driven, the ability to leverage machine learning and artificial intelligence tools, including generative AI models, becomes essential for lawyers and law firms to deliver efficient and accurate legal services. Our guest today, Fiona McLay, is a consultant and legal technology evangelist at McLay Legal Consulting. Fiona specializes in helping law firms effectively implement new technology in the workplace and get the best out of their already existing technology – I’m especially interested in talking about that topic! I think that’s something that a lot of law firms don’t get right. As a former practicing lawyer, Fiona has extensive knowledge of the operations of lawyers and law firms and the problems that they have in implementing and adapting to new technologies. Fiona, thanks so much for joining me today on Hearsay the Legal Podcast. |
| Fiona McLay: | Thanks, it’s a pleasure to be here. | |
| DT: | Now before we get right into it, tell me a little bit about your path to being a consultant and legal technology evangelist. You used to be a lawyer. What sort of law did you practice? | |
| FM: | I did litigation and actually no one is more surprised than me to find that this is what I am now earning my living doing. I never really wanted to be a lawyer. I didn’t get into law when I finished uni and I started doing an accounting degree. The only subject that I actually really enjoyed was a subject called commercial law. My marks were good enough at the end of first year to transfer into law, so I did that. I thought I would be one of those nice collaborative lawyers who works with people. And you know that my first rotation, I did a summer clerkship at one of the big law firms and my first rotation was to a litigation group doing professional indemnity insurance work and that involved acting for doctors and engineers where things had gone wrong. And I found that work so interesting. It’s all about people and working out how brain surgery should have gone and how you should have engineered this road so that it wouldn’t collapse. And it was a lovely team. And so I ended up becoming a litigator by accident. I stayed there, then I moved to a different firm, still doing the insurance litigation and then I ended up in a small firm. And that’s where I really got interested in tech, because still running quite big litigation – but without the team, without the support. And so that’s where I just sort of saw there really was only one way to still be able to deliver really high quality legal services at a price our clients could afford and that was to leverage the tech. By that time, you could put in a credit card and you could try it out. One of the benefits of being in a small organisation is I didn’t have to go through many layers of approval and I could just try things out and some of them worked really well. I can remember one particular case, we’d been brought in quite late and there was discovery due that week and the client had given me a USB stick with all of the documents and I was like; “right, okay, well if I just don’t sleep this week, I might be able to make that deadline”. And what I found was there was a budget version of Nuix and it was very cheap, it was a sort of charity version. I put that on the laptop, put the USB stick in, took me half a day to work out how to do it but once I worked that out, I had a list in a couple of hours. | |
| DT: | Amazing. | |
| 00:04:32 | FM: | And so for the rest of that matter, I was the one who knew which was the version of the prospectus that, what it looked like as at March because I had it all on my laptop. Everybody else had teams of paralegals who were still going back and pulling the folder off the shelf to find the document. I just saw that this could really enable us to level up the way we were playing when we were competing against better resourced opponents and just where you are running on a more leaner operating model that you could still have a life and do what your clients needed. So from there I started hanging out with more people who were using tech and I got involved with the Centre for Legal Innovation who ran some really great workshops, met people, and I became a freelance lawyer working still in litigation. But that just exposed me to different ways, new law firms, in house, different ways that people were using tech. And out of that I really just saw such a need for somebody who could help really busy, clever lawyers who wanted to do good work but just didn’t really have the knowledge about what tools were available and how to make the most of them. And so I went back to College of Law and did a Masters of Legal Business. I got a Graduate Diploma in it and so that covered things like strategy, innovation, technology, change management and there’s a whole lot of skills and ways of doing this that we didn’t learn in law school. You certainly don’t learn if you’re working eight, nine hours a day on billable work in a firm but there are people who have worked out how to maximise these things, how to do change management in an organisation. So after all that, that I’d learned and I was putting all that into practice, I really wanted to share that with the profession. So I decided to write a book and put all of that and everything I’d seen and try to make that in a way where it was accessible to people so that we didn’t all have to make all the same mistakes over and over again when it came to trying to make more use of technology. |
| DT: | Absolutely. These tools are really the only way to solve that more for less challenge that Professor Susskind has been writing about for decades now. There is no way to split yourself into many separate lawyers and deliver more value by increasing the number of hours in your day. All you can do is become more efficient with the time you have and technology is the way to do that. And what you were describing there about the freedom to try different tools, you know, than just having a credit card and getting those tools off the shelf without a lot of time spent on adoption, integration, and setup is really the description of today that model of the kind of best in class, best in category approach, to legal tech tools that’s available to even very small firms. Speaking from experiences, someone working in a boutique law firm, there’s a freedom in that isn’t there? To have that smaller practice and to solve a problem with a tool that you can find as you discover the problem, not having that bureaucracy over the top that prevents you from adopting those tools. But at the same time, it can be challenging to have so many options available, this whole universe of competing tools, different value propositions potentially overlapping in terms of the features they offer. I mean, it can be a little bit overwhelming and therefore sometimes paralyzing to find those tools. We are going to talk about generative AI today. You’re going to have to keep me honest as a software developer, as someone who’s working on artificial intelligence tools for lawyers and legal artificial intelligence tools for clients. I’m very excited about this area, but you have to keep me at the top level and talk about the practical outcomes because that’s what really matters, isn’t it? That’s right. | |
| FM: | And look, I’m super excited about it too. And I think we’ll be excited for a while. I mean, we’re at peak hype. | |
| 00:08:23 | DT: | Absolutely. We’re at the top of the hype cycle, for sure. |
| FM: | But I absolutely do think that that’s only useful if we’re able to translate that into; “what can we actually do with it? What use is it?” Because otherwise, it’s just a lot of inflated hype. | |
| DT: | Absolutely, yeah. And I think that hype can really drive some misconceptions about how the technology works and what it’s capable of and those misconceptions can be really dangerous in a technical field like law. | |
| TIP: | That “hype cycle” David and Fiona are talking about is most commonly used to refer to a presentation developed by American firm Gartner. You’ve likely seen it; it gets thrown around alot in relation to cryptocurrency. On the X axis you have time, and on the Y axis you have visibility. It kind of resembles a wave; you have a technological trigger, peak of expectation, trough of disillusionment, a gradual slope of enlightenment and finally a plateau of productivity. This plateau is approximately equidistant between the trigger and the peak – meaning the technology never quite gets to the inflated level of expectation. To be really clear, it’s not scientific. The hype cycle is merely a visual representation of what each of us may see and feel about a particular technology. In this case, it’s arguable that the trigger was ChatGPT – remembering that GPT existed before its “Chat” incarnation; meaning we’re likely right at the start of the Gartner hype cycle. | |
| 00:09:39 | DT: | Before we go there, though, let’s talk about getting the most out of the technology we already have and that we might already be using in our law firms. Because I think this often happens when there’s a very large tech stack, when there’s a lot of tools being used in a practice. And that’s easy to do now with so many software as a service options. Do you see a lot of your clients in your consulting business who have a whole range of different tools and subscriptions that they’re only using a small part of? |
| FM: | Yeah. And so one of the things I do is a tech review or a digital adoption assessment. And I go in, I spend the day in a law firm and walk around, talk to everybody, and just see what’s happening. And it’s the most common thing to find in the first couple of hours that either the firm’s paying for three different tools that do exactly the same thing, but different people are using them and no one’s sort of noticed. Or there’s been releases of new functionality in the practice management software. But again, it’s nobody’s job to go through those release notes and say; “oh, now we can bundle documents or that sort of thing”. So they’re still going on paying for a solution that they no longer need. And even just when you come back to good old Microsoft tools like Word and Office, I mean think about the last time you had training in those tools. And they have changed so much. So, for instance, there’s a summarized tool in Word. So there are really nifty things in there that have been added in that people just don’t know about. So another thing that I do is just helping people understand that there are some smarter ways, there is some automation that’s built in the tools they’re using every day. | |
| DT: | Yeah, absolutely. I think there’s a temptation to chase the new shiny thing, to adopt something that you’ve seen a really slick demo of – and good on the sales staff for doing a great demo. But sometimes those features that we need are kind of already there and we’ve just missed them. | |
| FM: | Yeah. Even if the ones that are in, for instance, in the Microsoft 365, there might be maybe a more generic version. But it’s good enough for you to test out. If we automate documents and put a link on our website, will our clients use that? Before you say; “oh, we have to go and get a new low code document automation solution and go through a whole big process around paying for that”. You can do those small testing it out with the more basic version that’s in the systems you already have. | |
| DT: | That’s a great point. And I’m glad you mentioned that idea of, well, it’s almost like the minimum viable product, really. But that approach to experimenting, testing and learning and seeing if the investment makes sense. Because so often there’s a hype, I suppose, and excitement about a new tool. There’s a lot of time spent on adoption, but it doesn’t really add any value for the client. And so it doesn’t get used. | |
| 00:12:28 | FM: | And I mean, look, there are always ways. If you’re optimising the back end of your law firm, if that’s freeing up more time for you to spend actually communicating with your client and meeting them where they are, that’s well and good. But you do have to make sure that you’ve thought about; “what is this getting me? How does this get me closer to the business objective?”. Because a lot of the selling of tech tools is around making my lawyers’ lives easier. And I think if you’re only ever looking at; “how can I just get rid of mundane tasks that I hate?”. You’re not necessarily connecting to that business objective of being able to either meet a client need in a way that you haven’t been able to or be able to do it in a more profitable ways. So I do think you need to have a mix of making the back end of the law firm work better, but also always be thinking about; “am I really improving the way my clients are receiving services or what I’m offering my client?”. And that’s the way that you can make sure that you’re keeping up with where the market is. |
| DT: | That’s so true. I think often we do adopt new tools and often, to be fair, some of the vendors are creating tools to solve pain points in the delivery of traditional services. But the output still looks very similar for the client. Now that might be great for margin, but it’s not really helping us differentiate. Whereas what you’re saying is the opportunities are rarer to use technology to really transform the output to really add value for the client. But those are the opportunities we need to look out for. Now I suppose there are really two classes of barriers to adoption of technology in law firms. There’s the systems you’ve already got in place and there’s the people or the personalities who have to adopt them. You’ve driven the adoption of tech tools across a whole range of legal businesses through your consulting practice. In your experience, which is the more common barrier? | |
| FM: | I’m going to say both. | |
| DT: | They’re both prevalent, I guess. | |
| 00:14:31 | FM: | Yeah. And also in any one firm, you’ll find both for different people. Fortunately, change and adoption of new ways of working is an individual problem. And you will have two people doing the same work, but having different preferences and different approaches to the same tech. So it’s a complex thing. One of the things that I’ve seen is that – and certainly in my working career – a lot of the old tech, the user interface wasn’t great. And it was a sensible decision not to be the person who knew how to fix the computer. Because if you were the person who could make it work, then, you know, you were always the person who was being called on to troubleshoot. And so a little bit of selective non-coping was a viable survival mechanism. That’s changed now because I think the user interface has improved. And I think now to be able to understand the business drivers, you do need to be able to understand what the computers are capable of. But that could be a reason why you have to just rethink; “is this something I want to leave to the office manager or I want to leave to the young kid or I want to leave to our IT consultant? And is that actually really now the best decision?”. So that’s one reason why I think that legacy software has not always been something that people are keen to talk about the nitty gritty of. It’s sort of just been parked off as; “oh, well, that’s the IT department. And then when they look after that, I just do the lawyering”. I think that the integration there is shifting. But then you have the people aspect. And yes, some people are just interested in tech and just like the idea of and that they’re just those natural early adopters. Some people are so busy and have so many things on their plate that the idea of getting them to change the way they work in any way, shape, or form freaks them out. Because they can just manage to meet their clients demands, meet their families demands, get some exercise and some sleep. And they have no bandwidth to learn something new. And that’s also something that you have to really respect. And so you do have to look out for where; “is there something, a way, that I can help that person with a particular point where they have a need. How can I help them explain how technology could help in that moment?”. You can’t just go to them and say, all right, give me half a day and let me show you all this new technology. That just won’t work. So I suppose the answer to your question, I think that there are some really good reasons why people haven’t, lawyers and legal professionals haven’t had a lot of time to experiment with technology. And I think that there have been rational reasons. And I so I think that unless you’re able to respect both of those kinds of reasons and give them some help to see how technology can help, you’re not going to get the take up and the reception that you’re hoping for. |
| DT: | That’s so true. I think there’s a real temptation and sometimes as technologists, or as as evangelists for technology, you can fall prey to this temptation, which is just say; “oh, Luddites, you don’t want to use the technology because you’re backwards. You just have to get on board”. And often there’s a conversation about features, but there’s not a conversation about what value those features are adding for you or what pains in your daily life they’re actually solving. And to dismiss that person’s lack of time to actually investigate these tools because they’re doing so much for the organisation, for their family, in all these different places in their lives is to make a big mistake. And I really like that idea of rather than just showing them everything that’s possible and from their perspective, wasting more of their time, showing them the thing that really matters to them rather than everything in the basket. Just what’s this one thing that they could adopt that really solves the problem for them right now? | |
| 00:18:40 | FM: | Yeah, I like to think of it as incremental innovation. So really just chunking it down. And if you can get one thing every month at the end of the year you’ve got 12 new things. Rather than trying to do one big project and getting nowhere with it. I can think of the other thing about lawyers and legal professionals, they’re smart, clever people. They’re dedicated to doing a good job. So just to say; “oh, they’re not interested in technology, technophobes, it’s a huge disservice”. And so I was working with somebody who just didn’t have any interest in technology, just wasn’t the thing in their personal life, in their work life. However, they like to get their letters out. So if they come in early in the morning, there was no one around to do their dictation. But if we showed them digital dictation, they could come in six o’clock in the morning, dictate the letters straight up on the screen. Then as soon as the secretaries arrived, they could have those letters printed on it, and they’d be out by 10 o’clock in the morning. And they thought that was fabulous. But if I’d said; “oh, this technology, you can just talk at it and it appears on the screen” – that it’s of absolute no interest whatsoever. |
| DT: | Yeah, sometimes you have to show rather than tell as well. I remember a document automation tool in our law firm that we adopted for one team. And it did see widespread adoption. But I noticed that the team that were using it – they would generate the document, it was a kind of agreement. And then they would open the generated document, and they would change one clause every time. And I said; “why are you doing that?”. And they said; “oh, this clause just needs to change. For everything that goes out, this clause needs to change. Why don’t you change the template? And then you don’t have to do that anymore. Oh, I’ve not got time for that. I’ve got to generate 10 of these documents and send them out today”. And even that, even saying that out loud didn’t sort of reveal the absurdity of that position. But we all do it. We all fall prey to this really strong bias towards time spent on client work, even when the task is almost identical. But it’s spent on preparing to do client work. Is that a barrier that you often have in trying to persuade people that; “well, I know you’re putting in the time now, and it’s not billable now, and it can’t be coded to a matter right now, but it will pay off in the future?”. | |
| 00:21:00 | FM: | And I think I don’t know whether it’s because of the billable hour model, but there’s a lot of tolerance in law firms for doing things in a tedious way that I don’t think exists in… I can’t understand how it’s managed to hang on for so long. That’s one of my sort of golden rules. Fix the template as you’re doing it the first time you come across that problem – because there’s never going to be time in January to go through and fix up all the precedents. It’s just never going to happen. So you have to build that into the way you work. And we did a session yesterday with somebody who has sort of really optimised her law firm, and she talks about sharpening the saw. And it’s that you have to make sure that you are doing that prep work to enable you to be performing at a higher volume. And that has to be valued. And that it’s hard to value that if there’s no client matter to bill it to in that moment. |
| DT: | Yeah, absolutely. Now, you mentioned one of the clients that you’re working with. I think there’s probably a perception among some lawyers – and this would probably come from a time when some of the most widely adopted technology tools in the legal services space, like Ringtail, for example, very expensive packages requiring local installs, requiring some degree of manual integration – there was a perception that these tools were really only available and worthwhile for large firms that had the scale to really make use of them. Are most of your customers, those large or even mid-sized firms, or are most of your customers boutique, freelancers, small firms? | |
| 00:22:36 | FM: | I spent half my time in big law land and half in small firms. I’ve worked in a bit of mid-tier. One of my favourite things to talk to the small firms, particularly like family lawyers, is about e-discovery tools for disclosure, because they’re all struggling with these tools. And yeah, that perception that e-discovery is just for class actions. But those tools are priced now so that it’s extremely efficient and affordable. And you can just remove such a massive headache, both for you and for your client in collecting those documents that’s stored electronically, process it electronically and produce it electronically, just is a way better result for everybody. So I think that the tools that are available in big law land; yes, they’re more powerful and have more, perhaps more scope. But I also think that one of the advantages in small firms is you can make use of not just legal specific tools, but also enterprise business tools. So things like; I’m a big fan of AirTable, which is a database sort of tool, but it comes with a whole lot of templates for basic business functions. So managing, recruiting, tracking sales, doing your business and marketing activity. So you can just leverage into those templates and they’ve got set up some basic automation so that you don’t have to go off and hire in people with that expertise. And yes, they’re not as customizable as somebody who’s in a big firm would need. But for a small firm, they’re good enough as a starting point. |
| DT: | And using them for those tasks as well. We often forget it’s not just the client work that can benefit from these tools. Client intake also really great use case. And as you say, things like recruiting, things like human resources, marketing. I think lawyers sometimes get a bit of a bad rap or maybe have a bad reputation. Not necessarily well-founded, but maybe deserved. I don’t know for sort of being laggards when it comes to adopting technology. And I think that’s not just within our own profession, but it’s a perception of the public as well when we look at adoption of technology in the courts or in the legal system as a whole. I remember there being a lot of sort of mocking responses a few years ago. And during that controversy about the Royal Commission into the building regulator, Justice Hayden said that he wasn’t aware of the conflict of interest in relation to his invitation to a Liberal Party fundraiser by email. Because he only read his emails if they were printed out and put on his desk. And there was a lot of people saying; “oh, yeah, of course, judges and lawyers. That sounds right that they’d only read an email printed and put on their mahogany desk”. Do you think that reputation is sort of well-founded? Are we more as a population resistant to technological adoption than the general population? | |
| FM: | And it’s, I mean, again, it’s your own experience. My own experience is yes. And my wife works in marketing technology. And I saw the way that her clients were using technology to interact with their customers. And I just saw nothing of an equivalent nature in the way that law firms and professional services getting there. So I think, yes, we are as an industry behind. And I also know of cases where a partner didn’t like to have the computer on because it made too much noise. And so everything had to be printed out and brought in. So I think there is a tolerance for eccentricity in the legal world, perhaps. And we also have such – particularly in Australia, all those barristers are sole practitioners. Eighty, 82 per cent of the profession are in small or solo firms. And so there are just a lot of people who don’t have an IT team or an innovation team there to help them set up and design custom systems. So it is, I think there’s a lot of reasons. Also, then, just the way we have to interact with courts and the most challenges around confidentiality. And there was a lot of resistance to moving into cloud storage in the profession because there were concerns about whether or not that was safe and secure. So there have been a lot of factors that have led to that. I mean, the pandemic was just a wonderful… it was terrible, but in some ways… | |
| 00:27:23 | DT: | No, I know what you mean. |
| FM: | It showed that lawyers could change when they had to. | |
| TIP: | Fiona makes an excellent point here and it’s something we’ve touched on in the past on the podcast. The pandemic acted as an accelerant for technology in the law. In Episode 74 with Senior Judicial Registrars Brett McGrath and Sharney Jenkinson we explored the idea that COVID-19 had a broadly positive impact on things like remote hearings in the family law space; in one sense because coming and going to and from an in-person hearing in which family violence was an issue was necessarily reduced. Now, the Susskind netocracy – so Richard and Daniel Susskind – write that there is a unique reaction to technological developments in each of the professions. They say: “How do professionals react? Our original research and more recent work suggest a familiar response pattern. Architects are inclined to embrace new possibilities. Auditors dive for cover because the threats to their data-driven activities are clear. Doctors can be dismissive of non-doctors, management consultants prefer to advise on transformation rather than change themselves.” So what is it that we think lawyers do? Hiding behind tradition and eccentricity is certainly one option. | |
| DT: | It was kind of the forced experiment that we needed to have and drove remote working technology and a lot of firms that still had on premises solutions for all of their technology. I still remember using the Citrix token to work remotely and logging remotely into a physical machine on premises. So moving away from that model. And yeah, I think it was really only when there was no option but to adopt that technology that some firms did it. Shall we start talking about generative AI now? I know we both want to do it. | |
| FM: | Yeah! Yeah. | |
| DT: | All right. Let’s talk generative AI. I think it’s fair to say that, especially within the legal community, I’ve probably seen broadly two responses to the advent of generative AI, especially in terms of language – I don’t think Stable Diffusion or Midjourney or DALE-2 made much in the way of an impact in the legal community, but certainly in response to ChatGPT. So broadly two responses; “this is incredible. It will change everything. It can do everything that a lawyer can do, but better”. And; “I asked a question on ChatGPT about Australian law and it got it wrong. So it can’t do anything at all. It’s useless in law. The hype is not real”. It’s an oversimplification, but is that broadly the two camps that you see? | |
| 00:30:01 | FM: | I do see that where I’m hanging out with small lawyers and I hang out with a bunch of lawyers who are trying new things that I’ve specifically gone out and sought to hang out with. I’m also hearing from them; “I don’t understand it. I want to understand it. I see that this could be interesting, but I don’t yet know how and when will it answer all my emails for me”. |
| DT: | Yeah. That’s really kind of the ideal response to a promising new technology, isn’t it? A cautious optimism or a curiosity without a lot of assumptions about what it might be capable of. I have seen a lot of misconceptions, things like; “it can search the Internet for an answer or it has knowledge of a whole range of facts”. Would it be helpful for us to kind of break down first at a very basic high level what a large language model like ChatGPT is and what it’s doing when you ask a question? | |
| FM: | Yeah, I think because even if people have heard it before, I think we need everyone to understand it so that they aren’t expecting, like you say, some sort of response that understands facts and context and has a belief because that it’s not. It’s a statistical calculation. The best explanation I’ve heard was that it’s autocorrect, but on a huge level. | |
| DT: | Yeah. I mean, it is in a way like that. Our autocorrect function predicts the next most likely word. But once you use that for a couple of words, it quickly becomes nonsense because there’s no recollection of the previous words. A large language model is predicting the next token in a sequence of tokens and tokens can be whole words or parts of words. But it has something called an attention head, which means that it understands the context of the next token with regard to the whole of the text. So it is a language probability model, but it’s a language probability model that understands the entire context of the text that is being prompted with and that it is generating. | |
| 00:32:05 | FM: | I think one thing that’s useful is to ask ChatGPT to write a speaker bio for you or to write a couple of paragraphs about you. And you can instantly see what it gets right and wrong and you’ll instantly know what’s right and wrong. But it’s a good example of where it gives you something that is coherent and contextually accurate, perhaps factually wrong. |
| DT: | Yeah, absolutely. And I think one of the dangers there is that whatever is generated, correct or incorrect, is very plausible. And that’s because, of course, it’s been trained on trillions of these tokens, words or parts of words, and understands very well the relationship between words and phrases with other words and phrases and can generate highly plausible language. And that’s part of the incredible capability that large language models have to generate very natural sounding, sensible, and plausible text. But this goes right back to the GPT2 language card that OpenAI published when we were back in GPT2. I think it said words to the effect of; “since large language models cannot distinguish fact from fiction, we do not recommend the use of GPT2 for any use case that requires the output to be accurate”. And that is still true today. So an important caveat and – I should say a little plug for our parent company, Lext. We are working on some tools that can generate accurate answers using large language models as a part of that architecture. But generally speaking, and when we’re talking about ChatGPT, we can’t assume that the output we receive is accurate. One of those illustrative examples that I like to use is to ask ChatGPT to tell you what two plus two is, and then to tell you what a much more complicated mathematical formula is. And of course, answer the first one correctly, because that very simple equation appears correctly many times in its training data. But the unique mathematical equation that you’ve come up with is unlikely to ever appear in its training data. And it will give you a confident but entirely wrong calculation because it’s not designed to perform logical calculations. It’s just designed to generate text. I think you and I will be in furious agreement that artificial intelligence and especially generative AI is going to change legal practice profoundly. Rather than talking about what will be possible or what is going to happen, maybe we can just for now stay with today. With the tools that are available in the market, as at May 2023, what should lawyers be using now and what should they be using it for? | |
| 00:34:36 | FM: | So I think definitely be playing with the free version of ChatGPT in your personal life. Get it to do a meal plan and a shopping list. Get it to come up with a study plan for something you want to learn. Get it to do you an exercise plan. Get used to it and what it is good for and what it is not good for. It’s also very good at reformatting documents or turning things into a table for you, stuff like that. Don’t be putting client or confidential information into it, into the free version. But then in terms of what can you do in your practice? One thing that I think is going to pay off huge dividends is starting to work out the way we do things in this firm. So if there’s a process that you use when a new matter comes in or if there’s a process that you use to file a document or prepare a document, start to be recording that because what you’re going to need to be able to do is to train these models on something. And the more that you have written down in text that you can point is going to help you down the track to be able to capitalise on these tools. So just start having a systems or a process hub and storing stuff in there. Also, I’ve been using it a lot for marketing. So I might write a first draft, but I use it to tighten up the language or I use it to generate some ideas. So it’s good finding out what your client base is interested in. What are the things that people want to know about? Come up with some of those personas, those sort of things. You can also ask it for some strategic ideas about what are some ideas I can use to generate more people to come to my website. Those sort of generic business instructions it can give you help with. And you can be using it to summarise large, complicated documents. Again, you wouldn’t want to rely on that summary as being accurate. But where you are trying to quickly scan a 300 page case and maybe just to see what that case is about, you can ask it to summarise what were the arguments to and for a particular point. That might be a useful exercise. The other thing that you can do is just be looking for ways to get first drafts started in so that you’re not staring at a blank page and alternative perspectives. So if you’ve done a submission in a particular way but you want to try it with a different tone or a different style, it’s really good at coming up with those. If you want to try something for a different audience, if you want to simplify the language so that somebody who is not familiar with legal jargon can understand it, it’s really good for that. |
| DT: | Yeah, absolutely. I think something I like to think about in terms of the use cases are what large language models were capable of before ChatGPT. We’re very familiar now with the chat format of user message, assistant message, user message, assistant message. But earlier, large language models were capable of summarisation tasks specifically, completion tasks specifically, edit tasks specifically. And so you can sort of break down the possible tasks in that way. Summarise a given set of text, edit existing text into a new style or mode of speech. And I think those tasks where you’re sort of supplying the ground truth or you’re supplying the things that need to be accurate are the safest sort of use cases. And the sort of marketing copy use case, especially for first drafts, is a fantastic one. It’s sort of perfect for that. We’ve certainly used it for that here at Hearsay. And these are first draft. You want to make sure that’s in your voice and really accurately conveys the value propositions that you want to convey to your customers. But it is a great starting point. | |
| 00:38:24 | FM: | I also find it’s really good for the hope this email finds you well. There’s sort of generic niceties. So if there is a message that you need to craft with those generic, it’s really good at that sort of stuff. |
| DT: | Absolutely. Those sort of really generic correspondence, finding times to chat, those sorts of things are really great. And of course, I think something we’ve all seen is song lyrics, poems, all those sorts of fun but not terribly efficient or useful things are always worth a go. Your point around making sure you have things written down for the future, I think is a really good one because well, it’s often referred to as knowledge retrieval architecture. But what we’re really talking about there is tools that allow large language models to answer questions on data or documents that you’ve provided. And Google at their most recent IO conference announced Tailwind, a product that will come with the GSuite set of products that will allow you to do just that with the documents in your Google Drive. So those tools are absolutely coming. They’ll make it much easier to understand things like employee handbooks, workplace policies and all those sorts of things that you do have written down if you have them written down. But data is the new oil and textual data in this age of LLMs I think is hugely valuable and so kind of perfect for law as well. We’ve had so much information, so much data locked up in dense, pretty difficult to comprehend text. It’s just sort of perfect for our industry. We talked a little bit about the challenges, a little bit about the limitations. But I suppose talking to customers, talking to clients of your consulting business, you might have come across some specific ones. What are some of the limitations for these tools at the moment? | |
| FM: | So the main one is that we don’t know what use information that you put into particularly the free version of ChatGPT. We don’t know what’s being done with that data. We just don’t know enough about how it’s being used. We also don’t really know enough about when information comes back to you, where that’s come from or what has been used to train it to generate that. So that does create some issues in terms of trust. | |
| 00:40:39 | DT: | I should say that’s not just an issue in terms of the openness of providers of large language models. It’s actually a technical challenge, isn’t it? In that any deep learning model and large language models are an example of that. We don’t really know exactly how the output is arrived at from the input. There are many layers within the model that transform that input into the output. And we don’t know exactly how that happens. There’s a real challenge that AI experts are grappling with in terms of explainability of results and coming up with some models to try and explain the output of large language models. |
| FM: | And I think the other thing is lawyers have always traditionally been very fond of their documents; “this is my beautiful agreement and it’s much better than anybody else’s”. And if I put that into a generative AI, then Microsoft can just take it. And so I think that’s more of a mindset than a real rational fear. But I think there is that whole; “this is my stuff. I don’t want that just generally out there”. And that is a bit of reluctance. But I mean, on the other side of that, there is a very rational concern about confidentiality and privilege. I do think that we have to get around the idea that the value of a document is gone and that knowledge is available now in ways that it’s never been before. And generative AI is only going to speed that process up. And so we really need to be leaning into being the person who can provide more than just a nice document. It’s the judgment and the skill and the experience that is the value that we can bring. A good document will be able to be obtained from anywhere pretty soon. | |
| DT: | I couldn’t agree more. I think although it might already have been the case, generative AI is really proving beyond a doubt that that sort of model of; “well, I’m valuable for my knowledge. The documents I have in my knowledge database is gone”. And it’s not really the expression of that highly commoditized knowledge. But it is, as you say, those skills around strategy, creativity, problem solving, the human skills, the adaptive creative skills, because generative AI is generative, but it’s not truly creative. That creative step is always taken by a human being. And so having that strategic thought, that expertise to solve an unsolved problem, if the value you’re adding for your clients is just solving solved problems and answering answered questions, you’ve kind of got a problem already, even if generative AI wasn’t coming. So I think it’s helping to sharpen our thinking about what our job really is. We’ve talked a little bit about the confidentiality and privilege issues that might exist in using a sort of off the shelf free version of ChatGPT. Do you see any other sort of ethical considerations, not even necessarily legal ethics considerations around the widespread use of generative AI? | |
| 00:43:37 | FM: | And there is one other thing; that just in terms of if you’ve been worried about your flying and that’s leaving a big carbon footprint, the carbon footprint from generative AI and the power consumption is absolutely horrifying. And so I think that that is just another thing that we have to get to grips of is what is that imposing on the world? |
| DT: | It really reminds me of the ecological impact of cryptocurrency, the highly power intensive cryptocurrency mining activities, for example, that are a real contributor to carbon emissions. | |
| FM: | I still see generative AI as being a force that’s capable of good. I mean, I know there’s a lot of concerns about where it might go. But I do think that ability for people to access useful information is – it’s like the printing press, but on steroids. So I think it’s good. But yeah, there is a darker side. | |
| DT: | Yeah, absolutely. And I agree completely. I think overwhelmingly, it will be a force for good. I think there’s always harmful uses of new technology. We should remember when electricity was new, the electrocution of the elephant by Edison. So, of course, there are harmful uses. And I think some mainstream media coverage is maybe overly focused on some of those harmful edge cases rather than the overwhelming benefits. I remind myself often that we have a real access to justice problem, even in Australia; highly developed economy. The productivity commission said a few years ago now, and I can’t imagine it’s gotten any better, that about 85% of people with legal problems can’t see a lawyer about it. Our profession only solves 15% of legal demand. And that’s because of cost complexity, stress, all of the things that artificial intelligence can really remove as a barrier to accessing legal information. So I think we’ve got to remember how urgent the need for access to information is and the possibility of solving that problem. | |
| FM: | And one of the things that I found hard when I was coming to the end of my litigation career was when people had a good case and they came to see me and just wanted me to write a letter. And that the cost of that would be just so disproportionate to what they needed. And so if they can now just use ChatGPT to turn their concerns into a formal letter that expresses it in a good structure, I just see that as being a huge benefit. And also, if you’ve ever sat through one of the family court lists where they have just so many people who are unrepresented, struggling with really difficult issues, anything that can help those people to be able to express what they want and ask for something that the court is actually capable of giving to them. That I see is just a huge benefit to society. | |
| 00:46:26 | DT: | Absolutely. We talked a little bit earlier in the interview about kind of how perfect this technology is for our industry, because we’re so concerned with accessing information in large bodies of text. This technology is so good at doing that. With that in mind, is it really now incumbent on the law schools in Australia and around the world to start teaching at least a sort of basic level of usage of these technologies? And maybe just speaking more broadly, again, returning to the use of technology in legal practice generally, is it a really foundational skill for law students to pick up now to succeed in legal practice? |
| FM: | I think it is. And one of the things some educators are taking the view that they will give students a prompt and get them to critique the response that comes back from the AI. So just like you’re going to have a calculator to do that complicated sum, but are you able to assess whether that response is right or can you see other points of view? So I think that the skill becomes in being able to make use of what comes out of the tool rather than just ignore – I don’t think ignoring the tool is the right thing. And look, I am I’m seeing lots of good programs where unis are partnering with – whether it’s law firms or a law tech company – and running workshops or sort of projects around design thinking. And how can we start to re-engineer the way that legal services are delivered? And I think that’s coming from a place of recognising, as you were saying, there’s so much unmet need and the way that legal services have traditionally been delivered has not only causing a failure where people can’t access services, but so many unhappy lawyers. And I just when you sort of see lawyers saying; “oh, I can’t meet my billable targets because all of my work is fixed fee and I have to write so much off”. And it’s like, this is just a failed business model. The market is putting a limit on what you can charge, but you haven’t re-engineered the way that you’re delivering the work and you’re still measuring people’s performance off against billable hours. And you’re just causing all this distress in people. And you’re not even running it in a way that makes sense for a return on investment. And I know when I came out of law school, I had absolutely no idea about the mechanics of business and my job was just to do the most beautiful letter ever and not think about how that was commercially valuable. And so I think it’s a really good initiative to see students being asked and given the tools to think about how the legal services can be delivered in a more cost effective and still quality manner. | |
| DT: | Absolutely. We can talk for another hour about that whole dynamic and the sort of perverse result there where you might abandon the fixed fee approach because it’s not paying off because you haven’t adopted the technology because you’re still measuring your performance on billable hours. We should leave that for another day, I suppose, but it is a great example of how adoption of a new way of working, even if it’s not necessarily a technology, but a new way of working can go wrong if you’re not measuring the right things and if you haven’t set yourself up for success. We’re nearly out of time today, but before we go, this area is just moving so, so quickly, both generally and in law. It can be pretty tough to stay abreast of developments and make sure that you’re up to speed on what’s available. I’m sure that by the time we launch this episode, it will be a little bit outdated. So Fiona, where do you go to find your information and keep up to date on legal technology and generative AI? And where would you suggest our listeners go? | |
| 00:50:31 | FM: | Yeah, and I sent my book off to the printers in November last year and ChatGPT arrived! |
| DT: | What great timing. | |
| FM: | Instantly out of date. So yes, look, it is an issue. Social media is where it’s happening with generative AI. People like Liz Chase is doing a lot of a good series about prompt engineering, the Centre for Legal Innovation. They’re doing a lot of stuff about that. And that is a very practical sort of material. They’re talking to people who are in the industry who are trying this stuff out. So they run a lot of free material. I would definitely be following them. Raymond Sun is another lawyer who does quite good stuff around the bigger trend, the global trend. | |
| 00:51:13 | DT: | The regulatory approach as well, especially the new EU bill. |
| FM: | And there’s a law professor in the US called Josh Kubiki, who’s got a newsletter called Brainiacs. And so it’s Brain Y-A-C-T-S, And every day he’s publishing a new AI use case and some useful tools. So there’s some good places to start. | |
| DT: | Absolutely. Well, Fiona, thanks so much for joining me today on Hearsay. | |
| 00:51:40 | FM: | Thanks very much. |
| RD: | As always, you’ve been listening to Hearsay the Legal Podcast. I’d like to thank our special guest, Fiona Mclay, for being a part of it. As you well know, if you’re an Australian legal practitioner, you can claim one Continuing Professional Development point for listening to this episode. Whether an activity entitles you to claim a CPD unit is self-assessed, but we suggest this episode entitles you to claim a professional skills or business skills unit. More information on claiming and tracking your points on Hearsay can be found on our website. Hearsay the Legal Podcast is, as always, brought to you by Lext Australia, a legal innovation company that makes the law easier to access and easier to practice, and that includes your CPD. Hearsay is recorded on the lands of the Gadigal People of the Eora nation and we would like to pay our respects to elders past and present. Thanks for listening and see you all on the next episode of Hearsay! |
You must be a subscriber to access this content.