I'm wondering what the end game of this LLM vibe coding is. There's been chatter of how even professional developers are shifting towards not reading the code and instead testing the correctness of the output of these LLMs. Often when grilled they admit that the LLM writes the tests, which are subject to the same hallucinations. It probably has a compounding effect on the correctness of the resulting code...
For webdev, these LLMs can adjust their output by running a headless browser. These LLMs seem to be really good at iterating at problems. If you run a headless browser, a human could also see what the test runner sees, meaning you could monitor these LLMs as they make inferences on the resulting code. But I think it will be important to distinguish different types of tests.
There are unit or integration tests that humans have not written or read and can't be bothered to. I'm skeptical of the utility of these. It's an "if a tree falls in a forest" situation. If a llm writes a test and it passes, is the product is stable? I don't know, do you?
There are e2e tests that humans can monitor in some way, written by an llm or not, and the human can see the test run and step through the screens. These seem useful for both humans and LLMs.
There are tests that are measurable, like performance or A/B tests, which provide a ground truth that is optimized for. Are there compromises LLMs will make if you do not fully specify the design well? Humans are subject to perverse incentives when rewarded for a poorly defined metric, I'd assume llms would be too.
There are tests against historical data. Any idiosyncracies your historical data has, your LLM is also subject to reproducing.
Then there test that are written by humans and then forgotten about while diligently running in the CI/CD. I can see value in having these, just like I could see value in hiring a detail oriented human to pore over the historical data like an accountant, providing a baseline of 'correctness'. Asking the LLM for its opinion on your tests could help but then you're in a ship of theseus situation where if your tests needed significant overhaul or the test case changes you end up just letting the LLM do the hard part.
Are there other types of tests? that's probably where the future's at.
There is a sort of ground truth that LLMs don't really have about the world, due to being purely digital beings. Humans also struggle to make sense of the ground truth, but providing a ground truth in the form of tests and guardrails only makes sense if you can steer the ground truth to be the truth, right? Just like bad training data negatively impacts the performance (usually measured by tests) of an LLM, so could bad guardrails alters the final product. Will robots give these LLMs a better ground truth? Will they still hallucinate in familiar and unfamiliar ways?