Automated Unit Testing: Quis Custodiet Ipsos Custodes?

A company called Diffblue has a product that uses AI techniques to automatically generate unit tests for your code <https://www.theregister.com/2020/09/21/diffblue_let_ai_code_unit_tests/>. Only problem is, it doesn’t actually know what the intent of your code is, so it assumes it is correct to begin with, and generates tests on that basis. So who tests the test code? It falls back to a human to ensure the tests make sense. But doesn’t that defeat the point of having them done automatically? Unless you only use the system to catch regressions--things that were previously working, but get broken by newer changes to the code. Even then, it seems to me there is a likelihood of false positives, caused by refactoring of functionality between different modules; too many of these, and the humans are going to start ignoring the test results.
participants (1)
-
Lawrence D'Oliveiro