'We support the work of those who are trying to limit the harm generative AI is doing to videogames and our world, and apologise that Big Walk’s LLM usage may not meet their standards'
It’s never about their use case. Its about how ai steals everything in order to provide any answer at all, and how destructive even providing that answer ends up being.
This might seem reasonable on its face, but who did the LLM company steal the answer from without accreditation? Who’s neighbourhood is being ruined by the data centre?
It’s that ultimately there is no ethical use of LLMs
Speaking as a software engineer, code sits in a weird spot in terms of copyright. The exact code as written can be protected by copyright. But the mechanics of it cannot. Furthermore, code that performs a task that is so generic or even necessary for the mechanism it serves also cannot be protected. I can’t copyright a for-loop that iterates through an array, for example. Nor can I copyright the use of libraries built and freely published by others. Boiler plate code and importance tools are never protected.
There are so many libraries of tools, and fully, freely available code as open source, so much raw code content and understanding in forums and textbooks that, honestly, the only real protection any code has is the actual source code as written for commercial products. And you simply do not need those examples for training. And that is if you can even get direct access to those things. They’re not exposed to the public by design.
I would imagine that little, if any, actually copywritten code was ever used in training LLMs. That isn’t to say that there isn’t an element of use without credit to sources and individuals. But really that problem existed well before LLMs and, again, is morally grey when it comes to matters of mechanics.
Code is a unique beast. It is essentially mechanical. It works or it doesn’t. It is a quantitative measure, not a qualitative one like music, art, film, etc. You can train LLMs not just on code examples, but on concrete outcomes, pass-fail requirement conditions. And it is made of a finite set of operations, like math. It is only in its patterns of these operations that actual meaning and work is derived. LLMs can find such patterns iteratively without copying someone else’s work.
But even for code it has used, to me, coding has always felt collaborative. How many thousands of memes have been made about programmers googling everything in their work. It’s true. Coding has always been community-driven and open-source by default, at least in my academic and professional life, and so training LLMs on this data also feels less offensive to me. And I say that as an engineer who both uses Claude as a tool in my work now, and one who is likely to be laid off in the next year, in part because of this. But I blame corporate greed and lack of integrity for that, not the fact that this tool exists. I have other problems with AI coders, and AI in general, not the least of which is just about every aspect and consequence of these data centers, as you said. But not because of “theft” of freely available code.
Whole coding might be a unique beast. Excluding projects which are open source etc. It is unlikely these models were made without the wholesale theft of copywritten materials.
Support for these is unethical.
As I stated in the original comment, it’s not so much what they did, but the tools they used were were not ethically created.
Be careful there, open source is very distinct from copyright free. Notably, anything under a copyleft license, such as GPL, is considered open source while still being protected by copyright in a way that restricts how that code is used. There is a strong argument to be made that AI trained on copyleft code and generating new code based on that training is in violation of the license and thus copyright.
Fair enough. I’m sure that that will be thrown into the litigation machine over AI copyright with the rest of it. As for new code generated off of it, I would be incredibly shocked to find out that any user ever that didn’t specifically use AI with the actual knowledge and intent of copyright violation would ever have even a little bit of legal consequence from it.
Regardless, that’s my employer’s possible future problem. Not mine. I’m not doing a lot of coding projects in my spare time. Even if I were, I’m not paying for tokens to do what piddly shit I do write.
Google search is using AI at this point, but prior to that should summarize popular results when it wasn’t just pointing at SEO’ed crap or various stack-overflow/EE/Quora articles.
AI for a similar purpose - finding and collating publicly available information - seems ok to me. I’m more interested in the various solutions I’ve contributed online being available and useful to somebody than attrition.
But if course the models themselves are built on a lot more than that, unfortunately. Still, AI doing basic search and analysis to provide suggestions versus actually writing the core software seems like a good evolution of “search” functionality
Well id argue local llms + don’t support the people making them is probably fine, so long as you’re not letting it do web searches constantly so you aren’t ddosing the internet.
They absolutely do. Just because you took the plagiarism machine offline after it stole everything and then destroyed ecosystems and neighborhoods to produce the resulting dataset doesn’t mean it’s any better than the fascist empowering online ones. It’s exactly the same. Being ever so slightly better because you’re using your own power from the power company instead of someone else isn’t worth the destruction caused to get to a point where your offline sycophant machine can sometimes confidently not lie to you.
How? I’m not paying any of the companies or supporting the people who made it in any way. This is the equivalent of a vegan eating some meat that was about to be thrown in the trash. I’m also only using it for coding some personal use slop anyway, and most of the stolen code was GPL / MIT license which means for personal use there isn’t even a license issue. Me using it offline and telling everyone not to pay for it and that the data scraping is terrible is nowhere near the same as someone paying and supporting these companies.
What constitutes an ethically acquired data set? I’m just not sure how you could definitively say that any particular data set did not include some stolen content. You would have to gather it yourself which is utterly impractical for anything other than very niche AI.
Just off the dome an ethically acquired data set would be one where the data providers have knowledge that their data will be used for training a model and consent. When I did a deepfake project for a class back in 2019 that looked like me asking my buddies “hey can you sit and read this article on deepfakes in front of a camera so I can try swapping your faces?”
In a commercial setting there are a couple ideas:
Train on your own past work. This has the added benefit of ensuring the model will be better at approximating your own style.
Stock image sources but for data. Gettysburg has a ton of images, and while they weren’t originally gathered for image generating model training they could start a new collection where they expressly get that consent. If it comes out that there are stolen images in that dataset, Gettysburg is the one liable and should likely need to provide compensation for any training already performed then remove that data from the set they sell.
Frankly niche models is probably where the technology should have gone in the first place. Why invest trillions of dollars into making general purpose models that are kinda mediocre at everything when you could build a smaller, more focused model that only does what you actually care about maybe pretty well.
It’s never about their use case. Its about how ai steals everything in order to provide any answer at all, and how destructive even providing that answer ends up being.
This might seem reasonable on its face, but who did the LLM company steal the answer from without accreditation? Who’s neighbourhood is being ruined by the data centre?
It’s that ultimately there is no ethical use of LLMs
Speaking as a software engineer, code sits in a weird spot in terms of copyright. The exact code as written can be protected by copyright. But the mechanics of it cannot. Furthermore, code that performs a task that is so generic or even necessary for the mechanism it serves also cannot be protected. I can’t copyright a for-loop that iterates through an array, for example. Nor can I copyright the use of libraries built and freely published by others. Boiler plate code and importance tools are never protected.
There are so many libraries of tools, and fully, freely available code as open source, so much raw code content and understanding in forums and textbooks that, honestly, the only real protection any code has is the actual source code as written for commercial products. And you simply do not need those examples for training. And that is if you can even get direct access to those things. They’re not exposed to the public by design.
I would imagine that little, if any, actually copywritten code was ever used in training LLMs. That isn’t to say that there isn’t an element of use without credit to sources and individuals. But really that problem existed well before LLMs and, again, is morally grey when it comes to matters of mechanics.
Code is a unique beast. It is essentially mechanical. It works or it doesn’t. It is a quantitative measure, not a qualitative one like music, art, film, etc. You can train LLMs not just on code examples, but on concrete outcomes, pass-fail requirement conditions. And it is made of a finite set of operations, like math. It is only in its patterns of these operations that actual meaning and work is derived. LLMs can find such patterns iteratively without copying someone else’s work.
But even for code it has used, to me, coding has always felt collaborative. How many thousands of memes have been made about programmers googling everything in their work. It’s true. Coding has always been community-driven and open-source by default, at least in my academic and professional life, and so training LLMs on this data also feels less offensive to me. And I say that as an engineer who both uses Claude as a tool in my work now, and one who is likely to be laid off in the next year, in part because of this. But I blame corporate greed and lack of integrity for that, not the fact that this tool exists. I have other problems with AI coders, and AI in general, not the least of which is just about every aspect and consequence of these data centers, as you said. But not because of “theft” of freely available code.
Whole coding might be a unique beast. Excluding projects which are open source etc. It is unlikely these models were made without the wholesale theft of copywritten materials.
Support for these is unethical.
As I stated in the original comment, it’s not so much what they did, but the tools they used were were not ethically created.
Be careful there, open source is very distinct from copyright free. Notably, anything under a copyleft license, such as GPL, is considered open source while still being protected by copyright in a way that restricts how that code is used. There is a strong argument to be made that AI trained on copyleft code and generating new code based on that training is in violation of the license and thus copyright.
Fair enough. I’m sure that that will be thrown into the litigation machine over AI copyright with the rest of it. As for new code generated off of it, I would be incredibly shocked to find out that any user ever that didn’t specifically use AI with the actual knowledge and intent of copyright violation would ever have even a little bit of legal consequence from it.
Regardless, that’s my employer’s possible future problem. Not mine. I’m not doing a lot of coding projects in my spare time. Even if I were, I’m not paying for tokens to do what piddly shit I do write.
I agree that they’re trained unethically, but “no ethical use” is gross overreach
Google search is using AI at this point, but prior to that should summarize popular results when it wasn’t just pointing at SEO’ed crap or various stack-overflow/EE/Quora articles.
AI for a similar purpose - finding and collating publicly available information - seems ok to me. I’m more interested in the various solutions I’ve contributed online being available and useful to somebody than attrition.
But if course the models themselves are built on a lot more than that, unfortunately. Still, AI doing basic search and analysis to provide suggestions versus actually writing the core software seems like a good evolution of “search” functionality
Well id argue local llms + don’t support the people making them is probably fine, so long as you’re not letting it do web searches constantly so you aren’t ddosing the internet.
They absolutely do. Just because you took the plagiarism machine offline after it stole everything and then destroyed ecosystems and neighborhoods to produce the resulting dataset doesn’t mean it’s any better than the fascist empowering online ones. It’s exactly the same. Being ever so slightly better because you’re using your own power from the power company instead of someone else isn’t worth the destruction caused to get to a point where your offline sycophant machine can sometimes confidently not lie to you.
How? I’m not paying any of the companies or supporting the people who made it in any way. This is the equivalent of a vegan eating some meat that was about to be thrown in the trash. I’m also only using it for coding some personal use slop anyway, and most of the stolen code was GPL / MIT license which means for personal use there isn’t even a license issue. Me using it offline and telling everyone not to pay for it and that the data scraping is terrible is nowhere near the same as someone paying and supporting these companies.
I’d say that would depend on the dataset used to create them.
What constitutes an ethically acquired data set? I’m just not sure how you could definitively say that any particular data set did not include some stolen content. You would have to gather it yourself which is utterly impractical for anything other than very niche AI.
Just off the dome an ethically acquired data set would be one where the data providers have knowledge that their data will be used for training a model and consent. When I did a deepfake project for a class back in 2019 that looked like me asking my buddies “hey can you sit and read this article on deepfakes in front of a camera so I can try swapping your faces?”
In a commercial setting there are a couple ideas:
Frankly niche models is probably where the technology should have gone in the first place. Why invest trillions of dollars into making general purpose models that are kinda mediocre at everything when you could build a smaller, more focused model that only does what you actually care about maybe pretty well.
Seems like you get it just fine.