Well, good luck with that. Software development is a shit show already anyway. You can find me in my Gardening business in 2027.
Good Luck. When the economy finally bottoms out the first budget to go is always the gardening budget.
You can find me in my plumbing business in 2028.
I already deal with shit daily so it’s what we in biz call a horizontal promotion.
That is a valid point, and I think i’ll preemptively pivot to woodworking.
Market gardening isn’t so bad, people gotta eat. But yeah, if you’re cutting lawns you’re going to suffer when the economy shits the bed.
Grrroooossss, noooo I liked you Square Enix in spite of everything else.
Literally not how any of this works. You don’t let AI check your work, at best you use AI and check it’s work, and at worst you have to do everything by hand anyway.
its *
Ironically, that’s definitely something AI could check for.
Spell check? Yeah fair enough. The misspelling has historical value now though so I have to keep it in :P
Ask it for many R’s there are in strawberry
You don’t let AI check your work
From a game dev perspective, user Q&A is often annoying and repetitive labor. Endlessly criss-crossing terran hitting different buttons to make sure you don’t snag a corner or click objects in a sequence that triggers a state freeze. Hooking a PS controller to Roomba logic and having a digital tool rapidly rerun routes and explore button combos over and over, looking for failed states, is significantly better for you than hoping an overworked team of dummy players can recreate the failed state by tripping into it manually.
That’s not generative AI though. Generative AI is the SLOP machine.
There’s plenty of room for sophisticated automation without any need to involve AI.
If it does the job better, who the fuck cares. No one actually cares about how you feel about the tech. Cry me a river.
The problem is that if it doesn’t do a better job, no one left in charge will even know enough to give a shit, so quality will go down.
Not all AI is generative.
I mean, as a branding exercise, every form of sophisticated automation is getting the “AI” label.
Past that, advanced pathing algorithms are what Q&A systems need to validate all possible actions within a space. That’s the bread-and-butter of AI. Its also generally how you’d describe simulated end-users on a test system.
I mean, as a branding exercise, every form of sophisticated automation is getting the “AI” label. The article is specifically talking about generative AI. I think we need to find new terminology to describe the kind of automation that was colloquially referred to as AI before chatgpt et al. came into existence.
The important distinction, I think, is that these things are still purpose-built and (mostly) explainable. When you have a bunch of nails, you design a hammer. An “AI bot” QA tester the way Booty describes in the article isn’t going to be an advanced algorithm that carries out specific tests. That exists already and has for years. He’s asking for something that will figure out specific tests that are worth doing when given a vague or nonexistent test plan, most likely. You need a human, or an actual AGI, for something on that level, not generative AI. And explicitly with generative AI, as pertains to Square Enix’s initiative in the article, there are the typical huge risks of verifiability and hallucination. However unpleasant you may think a QA worker’s job is now, I guarantee you it will be even more unpleasant when the job consists of fact-checking AI bug reports all day instead of actually doing the testing.
Not buying their next game it will be a nightmare
A year from now they’ll be wondering why their games are so buggy.
Chrono Trigger remake, first all AI development, because we live in the bad timeline.
spits
Oh. Gross.
Well their goes FFXIV, that will be their end
Square Enix exec doesn’t know what QA and Debugging entail.
“Well it works for unit testing, so just extend that out to all testing! Problem solved!” -Senior Management, probably
Who am I kidding. They have no idea what unit testing is.
@inclementimmigrant I’m so glad I’ve stopped buying AAA games.
Lol. Good luck!
They jumped on the NFT bandwagon a couple years ago, too. Did they not learn anything from that?
I’m just shocked any publisher is still doing QA at all, with the state AAA games release in these days.
That’s why they want 70% QA from AI. Because right now their games are only 10% QA tested.
Realistic goal considering they already do so little QA.
Hmm. While I don’t know what their QA workflow is, my own experience is that working with QA people to design a QA procedure for a given feature tends to require familiarity with the feature in the context of real-world knowledge and possible problems, and that human-validating a feature isn’t usually something done at massive scale, where you’d get a lot of benefit from heavy automation.
It’s possible that one might be able to use LLMs to help write test code — reliability and security considerations there are normally less-critical than in front-line code. Worst case is getting a false positive, and if you can get more test cases covered, I imagine that might pay off.
Square does an MMO, among their other stuff. If they can train a model to produce AI-driven characters that act sufficiently like human players, where they can theoretically log training data from human players, that might be sufficient to populate an MMO “experimental” deployment so that they can see if anything breaks prior to moving code to production.
“Because I would love to be able to start up 10,000 instances of a game in the cloud, so there’s 10,000 copies of the game running, deploy an AI bot to spend all night testing that game, then in the morning we get a report. Because that would be transformational.”
I think that the problem is that you’re likely going to need more-advanced AI than an LLM, if you want them to just explore and try out new features.
One former Respawn employee who worked in a senior QA role told Business Insider that he believes one of the reasons he was among 100 colleagues laid off this past spring is because AI was reviewing and summarising feedback from play testers, a job he usually did.
We can do a reasonable job of summarizing human language with LLMs today. I think that that might be a viable application.
Worst case is getting a false positive, and if you can get more test cases covered, I imagine that might pay off.
False positives during testing are a huge time sink. QA has to replicate and explain away each false report and the faster AI ‘completes’ tasks the faster the flood of false reports come in.
There is plenty of non-AI automation that can be used intentionally to do tedious repetitive tasks already where they only increase work if they aren’t set up right.
