Stop Writing Tests!
| Start time | 11:35 |
|---|---|
| End time | 12:00 |
| Countdown link | Open timer |
Tests can be helpful: they can find bugs in new code, check for regressions in old code, and clarify precisely what the code is meant to do. On the other hand, writing tests is often tedious - and it's rare to think of an error when testing that you forgot when writing the code.
My solution? Use tools that write tests for you!
Whether you're a novice Pythonista or gnarled wizard, this talk about property-based fuzzing and code introspection will educate, entertain, and help take your testing to the next level.
We often think of manual testing as slower and less effective than automated testing, but most test suites haven't automated that much! Computers can execute all our pre-defined tests very quickly - and this is definitely a good thing, especially for regression tests - but the tricky parts are still done by humans.
We select test cases (inputs) and check that the corresponding outputs make sense; we write functions that "arrange, act, and assert" for our tests; and we decide - or script via CI systems - which tests to execute and when.
So lets explore some next-generation tools that we could use to automate these remaining parts of a testing workflow!
PROPERTY-BASED TESTING helps you to write more powerful tests by automating selection of test cases: instead of listing input-output pairs, you describe the kind of data you want and write a test that passes for all X.... We'll see a live demo, and learn something about the Python builtins in the process!
CODE INTROSPECTION, and a handy templating tool, can help write tests for you. Do you need to know any more than which code to test, and what properties should hold?
ADAPTIVE FUZZING tools take CI to its logical conclusion: instead of running a fixed set of tests on each push, they sit on a server and run tests full-time... fine-tuning themselves to find bugs in your project and pulling each new commit as it lands!
By the end of this talk, you'll know what these three kinds of tools can do - and how to get started with automating the rest of your testing tomorrow.
I'm a researcher at the Australian National University, and maintain a number of packages - mostly focussed on testing. When not at my desk grumbling about the buggy state of software, I can be found with a book, a block of chocolate, and a plan to go somewhere out of phone range for a walk.