Want to listen to the full episode and all our other episodes?
Hearsay allows you to fulfill your legal CPD requirements every year.
Our yearly subscription is only $299/year.
With a yearly subscription, you can access all of our episodes AND every episode we release over the next year.
Episode 93
Buy Episode
Professional Skills Substantive Law
AI’s Legislation Lexicon: Regulating the Benefits, Pitfalls and Risks of the AI Frontier
Law as stated: 21 July 2023
What is this?
This episode was published and is accurate as at this date.
Ray Sun AKA techie_ray joins an inspired David Turner in Curiosity over a shared love of artificial intelligence and the law. Touching on the state of AI regulation the world over, the pair deep dive into the geekier end of law and tech.
Professional Skills
Substantive Law
| What area(s) of law does this episode consider? | The regulation of AI; current and emerging |
| Why is this topic relevant? | Artificial intelligence is currently revolutionising entire industries and reshaping the way we live and work. The next few years will determine whether AI becomes a transformative force like the printing press or steam engine, or if it falls into relative obscurity like MiniDisc or certain cryptocurrencies. Alongside the potential benefits of the technology, there is a growing recognition that AI carries inherent risks that must – or should – be addressed through regulation. Understanding and preparing for the future of AI and the law is a key skill for a modern lawyer. |
| What are the main points? | Regulation
Human-like intelligence?
Risks
|
| What are the practical takeaways? |
|
| Show notes | techie_ray |
David Turner = DT; Raymond Sun = RS; Ross Davis = RD
| DT: | Hello and welcome to Hearsay The Legal Podcast, a CPD podcast that allows Australian lawyers to earn their CPD points on the go and at a time that suits them. I’m your host David Turner. Hearsay The Legal Podcast is proudly supported by Lext Australia. Lext’s mission is to improve user experiences in the law and legal services and Hearsay The Legal Podcast is how we’re improving the experience of CPD. Today on Hearsay we’re talking about artificial intelligence. As we speak, artificial intelligence is revolutionising entire industries including the legal services industry and reshaping the way we live and work. The next few years will determine whether artificial intelligence becomes a transformative force like the printing press or the steam engine or electricity, or falls into relative obscurity like the mini-discs or certain cryptocurrencies – but personally, I don’t know that that latter path is likely. Now alongside the potential enormous benefits of this technology, there’s a growing recognition that artificial intelligence carries inherent risk that must be addressed through regulation. OpenAI, the creators of breakout hit ChatGPT, have themselves acknowledged the necessity for governance in emerging technologies like artificial intelligence. And that’s even before we get to concepts like artificial general intelligence or super intelligence. Yet even before reaching those advanced or late stages of creative machine learning, existing artificial intelligence technologies like large language models or LLMs are already pushing the boundaries of how our current laws can respond. Now joining us today is Raymond Sun, a technology lawyer and programmer at Herbert Smith Freehills, specialising in emerging technologies, privacy law and intellectual property. In addition to his role as a solicitor, Raymond also manages the Global AI Regulation Tracker, a comprehensive resource that monitors the latest developments in AI regulation worldwide. I have to say, great work on that tool. It’s really cool. Ray, welcome to Hearsay The Local Podcast. |
| RS: | Thank you, David. Good intro. Thanks for having me. |
| DT: | Pleasure. Now, before we jump right into our topic today, artificial intelligence and the regulation of it, tell me a little bit about your path into the law. How did you get started in the law? Were you always interested in the law? And have you always been interested in computer science and programming?
|
| RS: | Yeah, well, it’s a long story and it begins with two TV shows, actually. One, it’s Iron Man and two is Detective Conan. So for those who don’t know, Iron Man is a show about this billionaire who builds robots, builds his suit, weaponised suit. that’s changed the world. And Detective Conan is an anime show where it’s about a kid detective who just solves murder mysteries every day like a boss. And so I’ve been watching these shows since the age of six and watching Iron Man inspired me. I was fascinated around AI because Tony Stark, the character there, was building all these cool AI tools and that really got me into AI, robotics and just computers in general. And then Detective Conan is a detective show and I just love how they solve murder mysteries, analysing evidence, investigating facts, and that really got me into the whole evidence detective, legal reasoning side of things. So I’ve been watching these two types of shows for a long time and it influenced the way I was thinking about my career. I had two parallel interests in law and software engineering or tech in general. And so when it came down to actually choosing my chosen career, it was a very simple decision making process where I was like; “you can make computer programming a hobby, but you can’t make law a hobby”. So why not make law my career and make tech a side hustle thing. And that was basically my mentality throughout high school and also when I started law school, I was just studying law and on the side, I love building programming things. So throughout high school, I played a lot of games, a lot of games, Call of Duty, these fun shooting games, flash games, until my parents are like; “Ray, stop playing games, it’s a waste of your time”. So I decided to choose something more productive and I thought, why not make my own games? And so a group of friends and I, we started building games through no code platform – for example, Game Maker Studio, where you just drag and drop boxes and shapes onto a screen, you can make games from that. And that got me appreciating the power of maths in school. I would learn all these graphs and parabolas but didn’t really appreciate why we need to learn them until we start making games. The physics behind the way a ball drops or an arrow flies through the air, it’s all maths. And so that got me interested in just appreciating maths and science at a more practical level. And eventually, like I moved from no code platforms onto actual languages because the thing with no code platforms is that you’re constrained to the environment. |
| DT: | A lot of platforms call themselves no code or low code. And I think the kind of accurate label for the low code platform is: “eventually you will have to code”. |
| RS: | That’s right. That’s basically what I realised too. So in order to make more bespoke games or games where I could add in features that I really want to add, I had to learn programming languages. So started with Java, learned Python, then Node.js. So a bunch of programming languages that allowed me to build games. And slowly I started expanding from games, started building more practical apps, like simple desktop apps, a simple calculator, just to learn the basics. And it’s all self taught. So I watched YouTube tutorials, borrowed books from the library: “How to Learn Programming”. The main breakthrough was when I started to learn about machine learning. And this was back in 2015-2016, way before the ChatGPT hype. |
| DT: | This was around the time that deep learning was really the huge step forward. |
| RS: | That’s right. And there’s also the time where Google built an AI system that could beat the world Go champion. So deep learning back then was still quite a niche topic, but it was starting to get attention. And I thought; “it’s fun. And it also relates back to my childhood passion and dream of AI. This is finally my chance to actually build stuff in this space”. And I think a turning point was when COVID hit. So by that time, I was already working at the law firm I currently work at. I was doing my graduate program. And I was still trying to decide what team I wanted to rotate in. So I was going in the background, but during COVID lockdown stayed at home. I’m also a passionate dancer. I like to dance a lot in my spare time. And when COVID hit, there were no more dance classes. So I had to learn dance through YouTube videos and just train by myself. But it’s not the same, right? Without the teacher to correct your form and posture. You don’t know if you’re dancing properly. So I thought, why not create an AI that can grade my dance. And that idea then turned to a project. I started learning computer vision, how to like assess images and analyse images. And eventually I built a prototype system where you upload a video of the dance video you want to imitate and your own dance video. It will compare them side by side. And then it will just grade you how similar you danced to the original dancer. And it will return a score. And it will also give you screenshots as to where you could improve in your dance. And so I just built that prototype and I showcased it on YouTube. And it went viral. And that was also the beginning of my social media, I guess, hustle with the YouTube platform. And so then I applied the same algorithm to analyse K-pop dancers. So K-pop dancers are well known for just having really synchronised dances. And I thought, why not apply my algorithm to see who’s the most synchronised dance group. And because K-pop is popular, it just went viral across Asia. And then I just became that person who uses AI and science to analyse K-pop. And that was the beginning of my YouTube, I guess, side career. And that also motivated me to go deeper, learn more about AI technology to keep up to pace with the tech. And while I was doing that, in my law career, I was deciding what I want to specialise in. And out of all the legal areas, the most natural, I guess, choice will be tech law, because it’s related to tech. And so I was fortunate enough to be able to go into the tech law team. And then within tech law is also many different areas. And in around 2021, I decided to see if I could leverage my passion for AI and make that into my actual career. Because before then AI was just a hobby, just something for me to relax. But I started to see the potential of AI becoming a huge thing that could be relevant to my career. So I started pivoting towards AI law. And mind you that back then AI law is not really a thing. There is still no real like AI regulation in most parts of the world. So it was still a very niche area, very unclear as to what that body of law actually is. But I was just preparing myself. So I would read the news daily, I’ll keep up to date with AI regulatory developments to the extent that there are any. And then the second key turning point was when ChatGPT came out, all of a sudden AI went from a geeky nerdy topic to something that the everyday person will talk about. Kids will talk about it, your grandma will talk about it. AI was the thing and still a thing right now. And all of a sudden, all the research, all the background experience that I’ve accumulated since my high school days suddenly became really useful. And then clients started coming to me for AI related questions. Let’s say they have an AI project or product they want to release; what are the risks in relation to my product? And there was a key need for a lawyer who understood both the law but also the product. Because in order to give proper legal advice, you need to know what the issues are. And to know what the issues are, you have to know what the product and how it works. And very few lawyers at this stage understand how AI systems actually work. And so that’s what really helped boost my profile and career. And since then, I’ve been trying to consolidate that by creating content on LinkedIn, where I can share knowledge and spread awareness around AI tech, market and legal updates to everyday people as well as non tech professionals. Yeah, it’s still an ongoing journey. I really find my career right now really fun and rewarding. And I’m just feel very lucky to be able just to combine my passion with my career. And it’s great. |
| DT: | Yeah, absolutely. Wow. Thanks so much for that introduction. And there’s so much to unpack there. One thing I think is interesting is that inspiration from Tony Stark. And that’s going to be an interesting one to talk about a bit later, because he didn’t have the best luck with his artificial intelligence, right? He created some existentially risky AI there in Age of Ultron. So we’ll talk about that a little bit later. And how those sorts of depictions of the risk of AI colour the way we think about AI risk today. Because I think, as we’ll talk about a bit later in the episode, there are real inherent risks to artificial intelligence, but they’re not the ones that we think of from television and movies. I think that’s the kind of risk that everyone thinks about. And a lot of mainstream media fixates on. There was a story about a simulation involving a US Air Force drone that got a lot of attention recently, but the real present risks, and these are risks that aren’t just playing out in the future, but have been playing out for years around AI, much less Hollywood, and much less flashy. And I think it’s fantastic that you’ve been able to combine these two passions. I’ve been fortunate enough to do it myself in LEXT, as both a lawyer and a programmer, and I’m also a self-taught programmer, so I love that. |
| RS: | Let the record show we high-fived. |
| DT: | Yes, yes. That’s a good high-five too. I hope that picks up on the audio there. And you’re right, I think you do need to understand the way these systems work in order to advise on them. You do need to understand how they work, to a degree, to use them. Even as a professional using them as a kind of differential multiplier for your own legal work. And if you really don’t understand how they work, then we can see there’s some disastrous consequences there, like the case in New York, which I’m sure we’ll touch on soon. And what a great area to be working in privacy, intellectual property, tech, in your role at the firm now, because there are so many changes happening in that area, not just in terms of AI-specific regulation, but these new unanswered, soon-to-be-tested questions around copyright protection for materials used in training datasets, which I think is a fascinating topic to talk about. The security implications of, and the whole range of new attack vectors that play into privacy and data legislation that LLMs create. We’re kind of back to the 90s in some way in terms of attack vectors, because you had SQL injection back in the day. And now we’ve got prompt injection, right? So everything old’s new again. But we should probably stop geeking out about programmer stuff. We should talk about the law. And what is a typical day in your technology practice look like for you? You said you’ve got a few clients coming to you with AI products that they want to understand the risks of. What does that look like? |
| RS: | Yeah, so often how these interactions start is, for background, I’m also one of the executives on the Australian Data Science and AI Association, which is one of the largest interest groups in AI and data science in the country. And so by being executive there, I’m also responsible for organising meetups, regular meetups to unite the whole AI community. And that’s where I regularly give lectures or talks on the legal issues. And I also get to meet people from all across industries, including tech, finance, healthcare, etc. That’s how I often get my name out there. And people just know that there is a lawyer out there who looks out for the AI projects and AI businesses. So often those who come to me are those who are from these networking events and also word of mouth through LinkedIn or my social media channels. That’s how normally these people know me. And then they’ll reach out via email or phone call, they say; “hey Ray, I’ve got this new product I want to launch”. Or it could be staff at a big company, and the big company’s planning to launch a big AI programme or product and ask me; “what are some key risks I should look out for”. And oftentimes it often comes down to a workshop. I do a workshop for them, or I write advice for them, what the risks are, depending on what the project is, they take that away. And then they take on board my recommendations. And often these recommendations for now are making sure that the contracts are tight, that the product terms conditions are fair, and that they cover all risks. And it goes on from there. And often there’s like ad hoc ongoing advice throughout the lifecycle of a project, in case certain issues arise. That’s basically what my career looks like at the moment. I also do other usual tech law stuff like software licensing or due diligence tasks, which are general to all tech lawyers. So this AI space is still growing, but I expect to become the main and the whole part of my career in the next few years. TIP: Now, we are a legal podcast so it might seem a bit odd to pause and rehash what David and Ray have just spoken about in the programming space, but it will become important later on. There are a few things they’ve just touched on that are worth keeping in mind as you listen to the rest of the podcast. The first is the idea of “deep learning”. Deep learning is one type of machine learning which relies on layers to abstract progressively more sophisticated output from raw input. Analogously, imagine the construction of a pyramid where the base represents the raw input and the golden capstone peak represents the output. In order to get between the base and peak, each layer must be composited in order with each construction more sophisticated than the last. Like David said, there was a revolution in this technology in the late 2000s driven in part by graphics manufacturer Nvidia. A large language model is a type of deep learning model trained on a huge amount of text. We won’t deep dive into what “training” means here, but it’s helpful to know that in essence the model is looking at and reading that huge body of text and building up the likelihood or probability of the appearance of the next word – or really, token. In doing so, it’s not being trained on the content of that text like you would train a lawyer in the law. That lawyer should have some crystalised knowledge of the law. The model doesn’t really have crystalised knowledge, it can’t reason. Along the way though, the model picks up syntax and semantics from the probability of the next token – basically fake it until you make it. |
| DT: | Now we’re here today to talk about the regulation of AI, you created the regulation tracker. What’s the state of regulation around AI in the world at this point? It is still pretty nascent, but at the time of recording, there’s been a huge step forward, or at least a step towards stepping forward, I suppose, because it’s still a long process. Tell us a little bit about what’s happening in the world of AI regulation, especially I suppose what’s happening in the EU at the moment. |
| RS: | Yeah, there are various approaches around the world right now. I think for simplicity, I’ll talk about three broad categories of jurisdictions. So category one are jurisdictions that are proposing or have already enforced direct regulation that directly governs AI. And these include China. China is the major jurisdiction that actually has enforced legislation that regulates specific application of AI. So far, they’ve regulated internet recommendation algorithms. So algorithms that push out content or push ads for you on like TikTok or social media channels, they’ve regulated that to make sure that people are not being unduly influenced by certain malicious content, stuff like that. They also have regulation on something called deep synthesis technology, which is their fancy word for deep fake generators. And they’ve got proposed laws on generative AI, which will include stuff like ChatGPT. So you have China in that category. And the same category, which is probably more well known is the EU; European Union. They’ve been working on a draft AI bill for almost three years now. And actually this week, the EU Parliament has finally approved the draft text of the bill. And it’s now going to be reconciled with other versions produced by the council, the council and other executive branches of the EU government making body…. |
| DT: | And there’s consultation and redraft. I think the sort of expected timeline if everything goes well it’s kind of 2026 sort of thing, right? |
| RS: | Yeah. So final approval by end of the year, but actually enforced, there’ll be a transition period for businesses to catch up. So expected to actually come into force like 26 or 27. Yeah. But the key thing about the EU approach is that it’s dividing the whole AI sector into risk categories. So for example, applications like social crediting systems or applications that have a real life dangerous effect on human rights or livelihoods of humans, they’ll be banned. And then you have the high risk category, which are allowed, but subject to very strict restrictions. So you have this sort of risk category system. And that’s also been, I guess, followed by other countries in that same category, such as South Korea and Brazil. Now, that’s the first category. The second category, which captures most of the countries around the world are those that have, let’s say, ethical frameworks, or national strategies or white papers on AI. But they don’t actually have yet direct AI regulation. What they do have is a patchwork of existing IP, privacy, consumer protection, etc. laws that can govern specific risks of AI depending on what the system is. And this is where Australia comes in – US, UK, Japan, Singapore, many countries in that category. Then you have the third category, which are countries that either have not made any statement or position on AI, or have openly decided not to regulate AI at all. And probably the most interesting example here is India. So a few months ago, the India IT Ministry came out to the public and said; “we’re not going to regulate AI at all”, because they want to allow their market to innovate as much as possible to allow India to catch up to other nations up to a certain point, and then they’ll start to consider AI regulation. So the main takeaway is that there is no one right approach to AI regulation. It all depends on the country’s economic, legal, political circumstances. And yeah, it’s really interesting to see which jurisdiction will create fairer results that doesn’t overstifle innovation. I’m still trying to monitor everything all at once. So yeah, it’s very interesting. |
| DT: | But it is because you’ve gone from a state of things where there’s really no regulation whatsoever in any jurisdiction, and now everyone is rushing to say or do something. The position of India is really interesting. There’s a little bit of, to a lesser extent, mirroring of that in Japan. The government of Japan pronouncing that there’ll be no copyright protection for copyrighted works used in training data sets, for example, as a kind of incentive to operate in Japan if you’re an artificial intelligence company. But your explanation has been that artificial intelligence is a term that can refer to a whole range of very distinct technologies. Today, at the time of recording – 16 June – when we say AI in conversation, we usually mean generative AI today, because that’s what’s new, what’s in the media, what’s causing a lot of these conversations. But we’ve been interacting with artificial intelligence for years. Every day, I would say in the form of our social media, content recommendation algorithms, our streaming service content recommendation algorithms, every time we complete a capture to log into a website, there are a whole range of ways that we directly or indirectly interact with artificial intelligence, whether that’s deep learning, whether there’s a recommendation algorithms or generative AI now. And that term can mean very different things. So do you think that it’s really the generative AI boom over the last six months that have caused some jurisdictions to catch up in terms of the regulation of artificial intelligence that’s probably long overdue for some things? |
| RS: | That’s right. The generative AI boom that started since ChatGPT has really brought AI, existing issues around AI, to the laypersons attention. And governments around the world, they tend to respond to whatever the market is concerned about. And so yeah, there is an element of truth in that governments are just only reacting now due to this generative AI boom. But you’ve seen countries like China and EU, they’ve been working on their AI regulation for quite a while, even before generative AI became a thing. And I think it just comes down to the industry and market. So EU, China, they’ve been building AI systems for quite a while now. So of course, they’ll be more cognisant and aware of the risks and issues compared to other jurisdictions which mostly import AI rather than producing AI. The economic factors become really relevant here. But it’s good that you mentioned the definition of AI because it’s a fundamental question, right? But it’s also the most important question when it comes to regulation, because there’s currently no official definition of AI. Academics have their own definition. Technologists have their own definition. Regulators also trying to find their own definition. In fact, the EU lawmakers have been changing the definition of AI like multiple times. And it might sound simple, many people think; “oh, isn’t AI just any machine that simulates human intelligence?”. That’s often the shortest one-liner explanation. But the risk of having that definition in regulation is that you can argue that any type of machine, any type of program is in a way a simulation of human intelligence. You can argue that a vending machine by accepting your coins and giving you a drink is in a way simulating what a cashier process is. Could you argue that’s a simulation of human intelligence? And therefore, could you argue that a vending machine is AI and therefore should be regulated? So these questions are often like… it might be a slippery slope, but it all comes down to how well we define AI. And you can’t redefine and make it too prescriptive. That underregulates the sector. But you can’t make it too broad or flexible that it overregulates every machine and every tech in existence. So that’s, in a way, why China has gone with the approach they’ve taken. They’ve kind of avoided the whole AI definition question by actually regulating specific applications, as I discussed before. So that way, what you’re regulating, what you’re not regulating, it’s a bit more clear. But countries like EU, South Korea, Brazil, they’re trying to regulate AI as a whole, which as you said, David, AI has been around for ages. New applications are coming out and new applications often trying to like challenge existing definition. And so there’s often a key commentary and criticism as to whether these sort of broad AI regulation approaches are future-proof and whether they’re like even fit for purpose in the next five or 10 years. And as you said before, the EU bill is going to come into force maybe in 2026 or 2027, but who knows what the world will change in the next two years. Not many people expect the ChatGPT will come out in 2022. Many people thought that type of technology was a 2025 or 2030 thing, but it came out in 2022 and that challenged so many definitions and conceptions around tech and AI. So yeah, it’s so interesting to see how – even I don’t really know the answer yet – how to properly regulate. It’s very hard. |
| DT: | Yeah. And it’s interesting this attempt to regulate artificial intelligence as a monolith. |
| RS: | Yeah. |
| DT: | And this idea of a simulation of human intelligence, unfortunately that requires us to understand what human intelligence is. And that’s difficult to define. But are we – this is getting a little philosophical – are we intelligent because we can act towards our objectives? And is that how you define intelligence? |
| RS: | There’s a lot of, I guess, again, theories around it. I know I forgot the name of the person for this, but one person, one academic defined intelligence into like four categories, like acting rationally, thinking rationally, stuff like that. I can’t recall that theory, but the main point is that there are many, again, competing definitions of what intelligence is and that further complicates the whole definition around AI. |
| DT: | And I think often that drive to regulate artificial intelligence as a monolith is sort of looking towards this – well, I think it’s a misconception – but the idea of artificial intelligence as a path towards human-like intelligence, which to me misunderstands the power of the technology and kind of the best applications of it. Because as we’ve been saying, we’ve been living with artificial intelligence for years and content recommendation algorithms look nothing like human intelligence. They have no kind of human traits, but they are vastly superior at the specific task for which they’ve been designed. And I think as we move further up the handle part of the hockey stick curve of artificial intelligence development that we’re on, I think we’re going to see more specific applications than we are going to see artificial general intelligence. It’s the hubris of humankind. To think that the best version of something is the one that looks most like us. Targeted use case specific applications of artificial intelligence are really what we’re used to in the world, but also what’s more likely to pose a risk in the future. Let’s talk about what risks we’re trying to regulate here. When someone who’s not that familiar with artificial intelligence, but maybe they’ve seen Age of Ultron, they’ve seen Terminator if they’re of an older generation, then they might think; “oh, well, it’s intelligences that become smarter than us and want to wipe us out for some reason”. And that’s what we mean when we talk about artificial intelligence existential risk. But the reality isn’t so flashy as that, is it? |
| RS: | It’s not. I can talk about some of the top key risks that apply generally across all AI applications. And I might just start with the ones that most people are familiar about and go to ones that are a bit more subtle and not as obvious, but equally dangerous. So obviously, the first one is like misinformation and hallucination, especially with cases of ChatGPT or other similar chatbots producing false information, but very confidently. So that’s a term that’s called hallucination. And the reason why that happens is not because AI is quote unquote dumb. It’s just how it works. So just to break it down, chat language models which are what drive these sort of chatbot applications. They are basically autocomplete on steroids. |
| DT: | They’re next token predictors… |
| RS: | Exactly. They just predict one word after another based on stats. And these stats are based on training of lots and lots of previous like material. So they just predict one word after another, they do not inherently understand the substance of what they produce in the same way that humans comprehend them. So that’s why you often see outputs which sound very convincing, but are actually not true. And one saying I like to use is language models are great for fluency, but not for accuracy. So they’re really great at summarising things, synthesising things, extracting things from existing material, but they’re not so great as a language or resource or research tool. And often the stories around misinformation arise from the research use case using ChatGPT to inquire about a real life event or real life object. And again, like some of you might already know this, but ChatGPT’s training is limited to the end of 2021. So it wouldn’t know that stuff like the Ukraine conflict has occurred or the passing of Queen Elizabeth. So it wouldn’t know these topics unless it has been trained on that particular material. And this is probably one of the, this sort of like misinformation risk can then lead into legal risks, for example, defamation. So it has been talked around people trying to sue open AI for producing defamatory statements through ChatGPT. |
| DT: | We’ve seen an Australian – well, I don’t know if the claim was ever filed – but claim that they were defamed…. |
| RS: | Yeah. Yeah. |
| DT: | … by ChatGPT. A university professor in the US was accused of having sexually assaulted students. It had never occurred. This misinformation risk, it’s not an unknown risk to the developers, these tools. If you go all the way back to the release of GPT2. An open source, large language model many years ago – I say many, four. But the instructions that went along with the release of GPT2 by OpenAI said; “since large language models do not know the difference between fact and fiction, we do not recommend use cases that require the output to be true”. It’s a well understood feature of these models, but as they’ve reached popularity and common use, that’s dropped away a little bit. |
| RS: | Yeah. It’s, I think, that’s like the reason why ChatGPT got so big. One of the major reasons was its ease of use, accessibility. It’s just so easy to use. And the more an easier thing is used and the more versatile it can be for different use cases, of course, there’ll be people who use it for the wrong reasons or reasons that it wasn’t built for. And yeah, so like defamation is one of the things that could arise and also just general misleading and deceptive statements or conduct. So probably one of the more popular headlines that went around last week was a US lawyer used ChatGPT to draft pleadings, including like case citations, but it turns out that one of the case citations was completely made up by ChatGPT. |
| DT: | Yeah, many of them were. |
| RS: | Yeah, many of them. So it just goes to show that it’s not so much the tech is at fault. It’s more around the use case was applied to was not the right use case because you can also see many other headlines where these chat bots are used for, are used properly and just creating these massive benefits and turnarounds. So that’s one, that’s probably one big risk that many people are at this stage, many are starting to become aware of. |
| DT: | We should talk about that New York case for a moment. It has made the rounds in legal circles and understandably, because it’s kind of a situation we can all understand and all are a little bit afraid of in terms of the products we’re using. I look at that case and from an AI regulation standpoint, it’s actually resulted in some AI civil procedure regulation, hasn’t it? Because there are now rules in both New York, I believe, and now in Texas a practice note was released there as well, saying that any lawyer who’s used artificial intelligence for research or drafting must file with the product of that output a statement saying that they’ve independently verified it, which is a surprisingly permissive approach to me, actually, I would have just assumed the practice note would ban it. TIP: In Texas, U.S. District Judge Brantley Starr of the Northern District of Texas issued the requirement for lawyers to sign a certificate that the relevant filing was not produced by AI or that if it was it had been checked by a lawyer. This was generally “to put lawyers on notice” that they can’t rely on AI. The judge’s statement further provides: “While attorneys swear an oath to set aside their personal prejudices, biases, and beliefs to faithfully uphold the law and represent their clients, generative artificial intelligence is the product of programming devised by humans who did not have to swear such an oath.“ He continues “Unbound by any sense of duty, honor, or justice, such programs act according to computer code rather than conviction, based on programming rather than principle.” |
| DT: | But interesting that they’ve allowed it with verification. And I wonder if we’ll see a practice note in any Australian courts to that effect soon. But that situation, if I think analogously of that lawyer filing submissions that have fabricated cases in them, and they weren’t fabricated by ChatGPT, but by a junior lawyer working in that team, would that lawyer’s accountability for those fabricated citations be any different? To me, no. They are still responsible for what they’re filing, they’re responsible for the accuracy of their submissions. And there, it’s really a matter of professional responsibility. |
| RS: | That’s right. I agree. Yeah, I think what made it more of a headline topic was the fact that an AI was used for that. And given the context and the market attention around it, it just became a thing. Otherwise, I don’t think it would have been a headline if it was like a junior mistake or a human mistake made. It would just be not just another mistake in the regular. Because it’s AI, it was just something that media outlets will latch on. And people, it’s just something that fits nicely into everyday conversations around AI. So yeah, I think that’s the main thing. It’s also the human in the loop, human accountability. Often the misconception is that right now that AI is very self-sufficient and can do everything. But it’s not true. At this stage now, you still need a human, it’s still ultimately a human’s responsibility to leverage it, but you can’t fully rely on it. Or if you do want to fully rely on it, you’ve got to be aware of the risks that it might not produce the most accurate results. So that’s the key message. |
| DT: | And again, there’s a distinction between artificial intelligence, software and large language models. That’s one part of a very large universe. Some of our listeners might be aware that hearsay’s parent company, Lext, is working on artificial intelligence tools for the law. And we’re working on trying to create reliable, accurate outputs for legal use cases. One of the very early prototypes of our model did not produce reliable, accurate outputs. We fine-tuned. We trained a model on a whole range of legal training data and we found that it didn’t get any more accurate. It just got really good at lying about medium neutral citations for Australian cases. Because again, it can recognise the patterns in that language and learn to recognise what an Australian medium neutral citation looked like very well. But that doesn’t mean it will know whether or not that’s a valid citation. We had to go down a different path, use completely different framework in order to provide that ground truth and reliability to our product. |
| RS: | Yeah. In many ways, the language of law is very formulaic. Like contracts are really a certain like expression, advice are written in certain expression. So of course, AI will be able to pick up on that. And the more they use it, the more convincing that output sounds. So there is also like, I guess, lawyers and law students are also quite susceptible to misinformation produced by our chatbots just because of the way the legal language structure expressed. If you throw in enough of these jargon and like these expressions, of course, it’s going to sound very convincing. So that’s one thing I think the legal audience should be even more aware of compared to let’s say other professions or careers. |
| DT: | Yeah, I think it’s a tool. We talk about productivity multipliers and differential multipliers. The idea that a tool can be especially useful for someone who already knows what they’re doing. And I think that even ChatGPT can be this for a lawyer. Someone who knows when they’ll spot something that’s not right. I see this again… we’re with both programmers. I don’t know if you’ve used GitHub copilot? |
| RS: | Oh, yeah. That’s great. I love it. |
| DT: | It’s amazing. But it’s a differential multiplier. You need to be able to program to use it well. If you didn’t know, then it wouldn’t really produce anything useful at all. But you might also get taken down a path that you don’t want to go down or that’s a bit of an anti pattern that doesn’t work. So these tools are absolutely useful for professionals in technical fields. But you need to have an excellent grounding in that technical field to use it. |
| RS: | One analogy I like to use to help give perspective to what you’ve just discussed, its the same analogy where let’s say you’re asked to write an essay in a language you don’t know. You know, you have the English essay before you right now. You put it into Google Translate and it gives you the translated output. Now, would you still – let’s say the essay is like 2000 words long and get a 2000 word response in a different language – would you still be confident enough to submit that translated output? Many people say no. Because you don’t know the language yourself you don’t know how accurate that translation is. And it’s the same thing with when we use AI tools for generating code or text. If you don’t have that background technical knowledge in the first place in that particular subject area that you asked the chat bot or the AI to produce, there is still like, you’re still not too sure as to whether you’ll be confident enough genuinely in that particular output. So I like to use the Google Translate or just any translation service as an example of that. |
| DT: | Now we’re talking about misinformation. What are some of the other risks that artificial intelligence poses? |
| RS: | I think another big one is like just privacy and confidentiality. Many people often overlook the fact that when you upload information or type in your prompt into these third party systems, some of them may have the right to store a copy of that information onto their systems, whether it be for training or for other purposes. But the thing is, if you are bound by privacy obligations or confidentiality obligations, depending on how these obligations are framed, the moment you upload sensitive information to a third party AI system, you could be in breach of your obligations. So for example, confidentiality, sometimes your confidentiality obligation – depends on the contract- could say; “you must not disclose any confidential information at all”. So if you have that very broad prohibition, the moment you let’s say upload confidential information into chatGPT you’re technically in breach of your obligations. Of course, there’s a question as to whether who will find out and whether you’ll be sued for that. But that’s another question. But there is that technical breach. Whereas sometimes let’s say your confidential obligation says you must not disclose information except in let’s just say in XYZ circumstances. And one of those circumstances allows you to use third party tools. And that would be okay in that sense. So often a lot of I guess one of the common requests I get from clients is; “could you please update my non disclosure agreements or my confidentiality agreements to allow for use of third party AI tools?”. And the same logic for privacy, but also for privacy obligations, obviously, they more so relate to personal data. So if you upload personal information to let’s say third party AI system, and you haven’t got the necessary consents from the person that personal information related to, then you could be in trouble when it comes to privacy laws. And particularly in Australia, our privacy laws have been amended to have more strict penalties, and also in the process of being like reformed to align them closer to more strict regimes like the Europeans GDPR. So you have that sort of area. And then like another big one is like IP, copyright, I think that has been a lot more relevant in the generative AI field. And probably the biggest question right now that people ask is, let’s say a text art generator generates an artwork that very much resembles the original artwork it was trained on. Now, can you argue that output infringes on the copyright of the training image? And this very question is currently being litigated in the US. So there are actually two class actions right now, one class action against GitHub Copilot on a question of like AI generated code whether that infringes on the copyright of all the open source materials trained on. And you have another class action against MidJourney, Stability AI, and Deviant art on the art side. So whether AI generated art infringed on the copyright of the original training art. And this argument actually goes both ways, believe it or not. So what the plaintiffs in those cases are saying is that as long as there is some resemblance between output and input, then output infringes on the copyright of the input. Enough said. That’s like that’s the main argument run by the plaintiffs. But for those who actually understand how these text to art generators work, it’s actually more complex than that. Because how these tools are trained is that, sure, they’re trained on lots of like artworks, right. But these artworks, at the very most, they just inform the different weightings within the AI model. So in other words, they just give the AI model a sense of what are the key patterns, the key patterns in these artworks. Once the model understand these patterns, it doesn’t need to look at these training images anymore. It just then works with these patterns to then create an output at the user request. So you could argue, and is what defendants in these case are arguing, you could argue that the AI model is actually producing original work. And it’s just a matter of coincidence that output might resemble an original artwork. So that’s what they’re trying to argue. And there is some merit to the argument because it depends on how far and how abstract you see the process. If the court sees it as more of a high level, output resembles input, therefore infringement, then the plaintiffs will probably have a good case. But if the court decides to go very technical, then there is actually a good case for the defendant to say there is no infringement. And it’s actually very tricky because this will have huge economic implications. If the court finds in favour of the creators, then sure, the creator community will benefit, but they could potentially stifle the whole innovation around AI because people will be scared of copyright lawsuits if they create these sort of tools. Otherwise, if you find in favour of the defendants, you might support innovation, but it might infuriate a bunch of creator communities and these people. So like, hopefully these cases don’t settle because I still feel like we need to get a clear answer on what the issue should be decided as. But at the same time, it will likely be a win-or-loss situation, which will have huge ramifications for the industry moving forward. So kudos to the judge who has to like decide this issue at night. |
| DT: | Yeah, wouldn’t want to be in their shoes. And it is a case that it’s difficult to predict the outcome. There’s really no analogous case. It’s hard to look at this and reason by analogy. I think for my own part, and this is really opinion more than considered legal opinion. When you look at the bundle of rights that copyright represents, typically those have represented the rights to which the copyright owner has a monopoly to reproduce, to copy, to edit, the economic rights that go with ownership of the work. The problem with extending copyright to cover the use of that material in training data is that it’s an impossible use for any one copyright owner to exploit. If I earn one artwork, I can reproduce it and sell it and I could make variations of it and sell those and I can economically benefit from all of those uses. But if I wanted to train a diffusion model to create artwork, I can’t do that with just my artwork. By definition, I need the copyrighted works of many artists for that use case to make any sort of sense. And so it’s difficult to understand the use of artworks in training data as a kind of use to which copyright should apply. Now, granted, the case is about whether the outputs of the model are infringing, not whether the training data use is infringing. But it’s an interesting question because the result of the case will have the effect of preventing copyrighted works being used in training data sets. And so I suppose that’s why we see jurisdictions like Japan before litigation can occur, making a statement to the effect that; “well, just to be clear, copyright will not apply to this particular use. We’re making the policy decision to put innovation in the artificial intelligence space ahead of copyright protection”. |
| RD: | Now this episode is a bit special. It’s part of a double-header with Ray which will come out next Friday. So look out for that one and the continuation of this conversation. And, look out for the surprise special guest. As always, you’ve been listening to Hearsay the Legal Podcast. I’d like to thank today’s guest, Ray Sun, for being a part of it. As you well know, if you’re an Australian legal practitioner, you can claim one Continuing Professional Development point for listening to this episode. Whether an activity entitles you to claim a CPD unit is self-assessed, but we suggest this episode entitles you to claim a professional skills or substantive law unit. More information on claiming and tracking your points on Hearsay can be found on our website. Hearsay the Legal Podcast is, as always, brought to you by Lext Australia, a legal innovation company that makes the law easier to access and easier to practice, and that includes your CPD. Hearsay is recorded on the lands of the Gadigal People of the Eora nation and we would like to pay our respects to elders past and present. Thanks for listening and see you all on the next episode of Hearsay! |
You must be a subscriber to access this content.