The death of Stackoverflow is one of these events where the site has been completely killed by AI and yet its contents is completely necessary for AI to know about solving programming problems. Its death will mark the end of AIs ability to learn how to solve programming issues. Its cannibalizing itself in the process, as it destroys its sources it destroys its own ability to learn.
It’s not just that, it’s shitting where it eats. People are using it to fill the internet with disinformation, then it trains itself on it’s own disinformation, and breeds even worse disinformation. This is why AI can never be smarter than it was in 2021.
On top of that, due to the indiscriminate DDOSing of the entire internet by AI bots, websites have been blocking any web crawlers that are not Google, which just contributes to their monopoly.
Our key finding is that by injecting information through an external synthetic data verifier, whether a human or a better model, synthetic retraining will not cause model collapse.
Lol, so to make a great model, they just need to have an even better one available first or a human who can verify every single thing it ingests.
Our key finding is that by injecting information through an external synthetic data verifier, whether a human or a better model, synthetic retraining will not cause model collapse.
Yeah if you have a source of truth then your model is basically getting trained on that.
My point was that having a verifier means your not really training a model on another model’s data, it’s basically as if you get new raw data from a non AI source
This assumes everything is valid on the external. If one slop cluster feeds off another - a slopveyor? - then there is nothing external for the validation hall-monitor to compare against. They’re trusting another model’s output as if it were gospel.
This is true, depending on what you mean by smarter. They are undeniably more capable. However, the trendy, cool things is to hate on AI, rejecting all else. Sure, capitalism sucks, and the powerful rich people and companies who control AI suck. AI itself, though, can very easily result in massive benefits for humanity as a whole.
The death of Stackoverflow is one of these events where the site has been completely killed by AI and yet its contents is completely necessary for AI to know about solving programming problems. Its death will mark the end of AIs ability to learn how to solve programming issues. Its cannibalizing itself in the process, as it destroys its sources it destroys its own ability to learn.
It’s not just that, it’s shitting where it eats. People are using it to fill the internet with disinformation, then it trains itself on it’s own disinformation, and breeds even worse disinformation. This is why AI can never be smarter than it was in 2021.
On top of that, due to the indiscriminate DDOSing of the entire internet by AI bots, websites have been blocking any web crawlers that are not Google, which just contributes to their monopoly.
Model collapse isn’t a thing anymore. https://arxiv.org/html/2510.16657v1
Lol, so to make a great model, they just need to have an even better one available first or a human who can verify every single thing it ingests.
Hmm, call me skeptical on this claim.
Yeah if you have a source of truth then your model is basically getting trained on that.
It’s like already having the answer
The point is that it only needs to comprise a very small part of the model.
My point was that having a verifier means your not really training a model on another model’s data, it’s basically as if you get new raw data from a non AI source
This assumes everything is valid on the external. If one slop cluster feeds off another - a slopveyor? - then there is nothing external for the validation hall-monitor to compare against. They’re trusting another model’s output as if it were gospel.
LOL OK
I’m pretty sure AI is objectively smarter today than it was 5 years ago.
Actually it just appears smarter because people are objectively dumber than they were 5 years ago. “AI” is actually stagnate.
There’s better integration with all sorts of other sources of truth beyond the LLM training, which makes it seem smarter.
Since LLMs literally can’t learn, no. They’re just increasingly tweaked to seem even more convincing.
How can something with no intelligence be smarter?
This is true, depending on what you mean by smarter. They are undeniably more capable. However, the trendy, cool things is to hate on AI, rejecting all else. Sure, capitalism sucks, and the powerful rich people and companies who control AI suck. AI itself, though, can very easily result in massive benefits for humanity as a whole.
More capable doesnt mean its smarter. A hammer can be made to be more capable but that doesn’t make it smart.
That’s not 1:1, and like I said, it depends on what you mean by smarter.