Table Of Content

Adam Johnson ©2020AdamJohnson. Allrightsreserved. PublishedbyAdam’sWebServicesLtd,UK. Forrecentchangesseethechangelogsection. Themoralrightoftheauthorhasbeenasserted. Seemywebsiteathttps://adamj.euforcontactdetailsandmoreinformation. Written in British English using Oxford spelling, so it’s ”behaviour” with a ”u” and ”opti- mize”witha”z”. CreatedwithSphinx. TypesetinPTSansandUbuntuMonospace. FrontcoverillustrationbykaterinadotonFiverr. Other illustrations from British Library on Flickr, The Internet Archive on Flickr, or self- madewithGIMPandInkscape. TheonlyauthorizedvendorordistributorforthisproductisadamchainzonGumroad. Contents 1 Introduction 1 1.1 WhoIsThisBookFor?. . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 ABriefTourofThisBook . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.5 Changelog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2 Toolbox 9 2.1 TestFramework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2 CustomtestManagementCommand . . . . . . . . . . . . . . . . . . . 13 2.3 TestRunner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.4 TestSettings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.5 CustomTestCaseClasses . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.6 Third-PartyPackages . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3 Measure! 27 3.1 Built-inOutput . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.2 ShellTimingCommands . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3.3 MeasureIndividualTests . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.4 Pro(cid:713)le . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 4 EasyWins 46 4.1 UseaFasterPasswordHasher . . . . . . . . . . . . . . . . . . . . . . . 47 4.2 AlwaysRebuildtheTestDatabaseifItExists . . . . . . . . . . . . . . 47 4.3 DisableDatabaseSerialization . . . . . . . . . . . . . . . . . . . . . . . 49 4.4 DisableInstrumentationPackagesduringTests . . . . . . . . . . . . . 50 4.5 UseanIn-MemoryFileStorageBackend . . . . . . . . . . . . . . . . . 52 4.6 UseanIn-MemoryCacheBackend . . . . . . . . . . . . . . . . . . . . 53 4.7 UseIn-MemoryBackendsforyourTaskQueues . . . . . . . . . . . . . 57 4.8 SkipSlowTestsLocally . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.9 PreventOutput . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 i 4.10 PreventLogging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.11 ReducepytestTestCollection . . . . . . . . . . . . . . . . . . . . . . . 69 4.12 PreventWhiteNoiseFromScanningAllStaticFiles . . . . . . . . . . . 70 5 Upgrades 71 5.1 UpgradeDjango . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 5.2 UpgradePython . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 5.3 UpgradeYourDatabaseServer . . . . . . . . . . . . . . . . . . . . . . . 77 6 Parallelize 79 6.1 WhatIsParallelTesting? . . . . . . . . . . . . . . . . . . . . . . . . . . 80 6.2 HowtoMoveToParallelTesting . . . . . . . . . . . . . . . . . . . . . . 80 6.3 CheckYourTestsAreIsolated . . . . . . . . . . . . . . . . . . . . . . . 81 6.4 ActivateParallelization . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 6.5 DealwithSharedResources . . . . . . . . . . . . . . . . . . . . . . . . 91 6.6 SplitUpLargeTestGroups . . . . . . . . . . . . . . . . . . . . . . . . . 97 7 Migrations 100 7.1 ReusetheTestDatabaseBetweenRuns . . . . . . . . . . . . . . . . . 101 7.2 SquashYourMigrations . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 7.3 Don’t DisableMigrationsinTests . . . . . . . . . . . . . . . . . . . . . 105 8 DatabaseCon(cid:713)guration 110 8.1 UseIn-MemoryStorage . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 8.2 Don’t SwapYourDatabasetoSQLiteinTests . . . . . . . . . . . . . . 115 9 CICon(cid:713)guration 117 9.1 Caching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 9.2 ScaleUp. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 9.3 ParallelizeAcrossMultipleCIServers . . . . . . . . . . . . . . . . . . . 121 10 TestStructure 124 10.1 AAA:Arrange-Act-Assert . . . . . . . . . . . . . . . . . . . . . . . . . . 125 10.2 WriteMostlyUnitTestsNotIntegrationTests . . . . . . . . . . . . . . 129 10.3 UsetheRightTestCaseClass . . . . . . . . . . . . . . . . . . . . . . . 141 10.4 TestCaseTransactionBlockers. . . . . . . . . . . . . . . . . . . . . . . 144 11 TestData 151 11.1 AvoidFixtureFiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 11.2 AvoidCommonDatainCustomTestCaseClasses . . . . . . . . . . . . 153 11.3 UseFactories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 11.4 UsesetUpTestData . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 11.5 OptimizeDatabaseQueries. . . . . . . . . . . . . . . . . . . . . . . . . 162 ii 11.6 AdvancedTestCaseFeatures . . . . . . . . . . . . . . . . . . . . . . . 163 12 TargetedMocking 166 12.1 TheFiveKindsofMockObjects . . . . . . . . . . . . . . . . . . . . . . 166 12.2 TheDangerWithMocking . . . . . . . . . . . . . . . . . . . . . . . . . 168 12.3 unittest.mock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 12.4 MockSettings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 12.5 MockOutput . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 12.6 MockInput . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 12.7 MockHTTPRequests . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 12.8 MockTime . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 13 Outroduction 192 13.1 FurtherReading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 13.2 ThankYou . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 iii Chapter 1 Introduction The measure of success most often is speed. Doing things better is synony- mouswithdoingthingsfastersothatwecandoevenmorethingsef(cid:713)ciently andeffectively. Inbuyingintothispremise,weenteraspiralofacceleration thatwecanneverhopetomaster. —DianaScharfHuntandPamHait Welcome,dearreader. Oh,tests. Tests are great. Tests show us where bugs are… at least some of them. Tests help us to notaddnewbugs. But…tests. Testsarealsoafrequentsourceoffrustration. Whenyou(cid:713)rststartedwriting tests, you probably found them an extra nuisance to write. You probably still think the same,atleastsometimes. Evenifyoulovewritingtests,runningthemcanbeboring. Slowtestscanmakeyoulose concentration,juggletasks,andendupgettinglessstuffdone. Theycansuckallthefun 1 SpeedUpYourDjangoTests,Release2021-07-08 outofprogramming. Andprogrammingismeant tobefun! There’s a phase shift when you speed up your tests, restoring that joy. If your tests take 1000 seconds and you reduce them to 600 seconds, you’ll feel it. Your whole team will feel it. The same happens if you can reduce a test run from 30 seconds down to 10 seconds. And there’s no limit to “fast enough”. Reducing test run time is the easiest and safest waytoincreaseyourspeedofdelivery. Andyourorganizationwantstodeliverasfastas possible,togetvaluablefeaturesand(cid:713)xesinfrontofusers. Fasttestsalsotendtobemoreaccuratetests. They’rewelltargeted,sotheyonlyfailwhen afeatureisbroken. Andwithlessoverhead,youcanexercisefeaturesmorecompletely. ThecontentisbasedonmyeightyearsofexperiencewithtestsonDjangoprojects. I’ve spedupmanyprojects’testsuites,improvedDjango’sowntestingframework,andcreated severalpytestplugins. In your hands, or at least on your screen, is my best guide for speeding up tests on your project. I’ve tried to cover most situations, so you should be able to (cid:713)nd something relevanttoyourproject. Mayitreduceyourtestspeedandyourtestpain. Enjoy! —Adam P.S.Pleasesendmeanyandallfeedbackthroughthecontactdetailsonmysite1. 1.1 Who Is This Book For? IfyouareaDjangodeveloperworkingwithtests,thisisforyou. Nomatteryourlevel. Myaimistomakethebookreadablebyajuniordeveloperwhohasbeengiventhevague task of “improving the tests”, but also to include material that even long time Django users may not have come across. I’ve tried to be opinionated enough to be actionable, butprovideadvicethat’sapplicabletoallprojects. 1https://adamj.eu/contact/ 2 Chapter1. Introduction SpeedUpYourDjangoTests,Release2021-07-08 1.2 A Brief Tour of This Book The(cid:713)rsttwochapters,Toolbox andMeasure!,explainthevarioustoolsyouhaveathand formodifyingandmeasuringyourtestsrespectively. Thenextsixchapterscoverwaysofspeedingupyourtestswithoutthebotherofrewrit- ing your test code (much). Easy Wins covers a number of smaller, non-invasive changes thatcanspeedupyourtests. Upgradesthenre-iteratestheimportanceofkeepingthings upgraded, especially with respect to test performance. Parallelize covers using paral- lelizedtesting,andthechallengesyoumightfaceaddingit. Migrationscoverstacticsfor improving your work(cid:714)ow with database migrations. Finally, Database Con(cid:713)guration and CICon(cid:713)gurationcoverwaysofspeedingupyourtestsincon(cid:713)gurationforthoseenviron- ments. The threechapters that follow coverhow towrite fast tests. Test Structure and TestData go into depth on writing faster tests that use less data. And Targeted Mocking shows some focussed techniques for replacing components during testing that can help you writebetter,fastertests. Finally,theOutroductiongivessomeclosingremarksandlinkstofurtherreading. 1.2.1 In a rush? Togetthe80%ofimprovementsthattake20%oftheeffort: • RecordyourcurrenttestruntimewithashelltimingcommandasinMeasure!. • SkimEasyWinsforanychangesthatareapplicabletoyourprojectandmakethem. When necessary, follow the references back to Toolbox to understand the pieces you’rechanging. • Ifyoudon’tyetrunyourtestsinparallel,readParallelize. • LearnhowtoreuseyourdatabaseandsquashyourmigrationsinMigrations. • DoasmanyoftheupgradesdescribedinUpgradesaspossible. • Compareyourtestruntime! If after this, your tests are still too slow, pro(cid:713)le them with one of the tools described in thesecondhalfofMeasure!. Thenwhenyouknowwheretofocus,readthelaterchapters todeterminewaysofspeedinguptheslowesttests. 1.2. ABriefTourofThisBook 3 SpeedUpYourDjangoTests,Release2021-07-08 1.3 Examples Thisbookhasalotofcodeexamples. Ihopethatisnotasurprise! Here’ssomeinformationontheirlayout. 1.3.1 Commands Commandstoruninyourshelllooklikethis: $ python --version The$representstheprompt-don’ttypethat. I’musingmacOSbuthopetohaveremoved any macOS or *nix bias. Commands should work cross-platform, but where that’s not possibleI’veaddedaseparateWindowsexamplewithaPowerShellprompt: > python --version IfI’vewrittensomethingincompatiblewithyourplatform(mostlikelyWindows),please letmeknow. 1.3.2 Versions ExampleshavebeenpreparedwithPython3.9andDjango3.2. I’monlyusingtheof(cid:713)cial Pythondistribution,CPython,sinceDjangooritsdependenciesoftendon’tworkonother Pythoninterpreters. Python code is formatted with Black2. (Like Django itself will be one day3.) Import statementsaresortedandgroupedwithisort4. AndallcodeisalsolintedwithFlake85. Throughout,Irefertoinstallationcommandswith: $ python -m pip install Django I’musingpython -m pipasperpipdevelopers’recommendations-seemyblogposton thesubject6. 2https://black.readthedocs.io/en/stable/ 3https://github.com/django/deps/blob/main/accepted/0008-black.rst 4https://pypi.org/project/isort/ 5https://flake8.pycqa.org/en/latest/index.html 6https://adamj.eu/tech/2020/02/25/use-python-m-pip-everywhere/ 4 Chapter1. Introduction SpeedUpYourDjangoTests,Release2021-07-08 Please read these commands as “install using your dependency management solution”. ThereareseveraltoolsforthisinthePythonecosystem,andIcan’tcoverthemall. Ifyouaren’tusinganydependencymanagementtool(beyondpip),pip-compile7isagreat startasit’sasmalllayerontopofpip. It’salsomypreference. 1.3.3 Django Projects The example projects I’ve used are based on my “simple-core” startproject template8. Thishasdirectorystructurelikethis: $ tree . ├── example │ ├── __init__.py │ ├── core │ │ ├── __init__.py │ │ ├── models.py │ │ └── tests │ │ ├── __init__.py │ │ └── test_models.py │ ├── settings.py │ └── urls.py └── manage.py 3 directories, 8 files This is slightly different to Django’s normal startproject template. The whole project livesinsideasinglePythonpackage,example. example.coreisanapp,insidewhichthere are models and tests. This is counter to the Django default of having apps in modules thatarepeerstothe“projectapp”. Ipreferthislayoutbecauseitmakesitclearwhenyou’reimportingsomethingbelonging to your project. Owned names always start with example (or whatever your project is called). 7https://pypi.org/project/pip-tools/ 8https://github.com/adamchainz/django-startproject-templates 1.3. Examples 5