Rector - The Power of Automated Refactorin

190 Pages·0.873 MB·English

by  Matthias Noback

Rector - The Power of Automated
Refactoring
Matthias Noback and Tomas Votruba
Thisbookisforsaleat
http://leanpub.com/rector-the-power-of-automated-refactoring
Thisversionwaspublishedon2021-06-24
ThisisaLeanpubbook.LeanpubempowersauthorsandpublisherswiththeLean
Publishingprocess.LeanPublishingistheactofpublishinganin-progressebook
usinglightweighttoolsandmanyiterationstogetreaderfeedback,pivotuntilyou
havetherightbookandbuildtractiononceyoudo.
©2020-2021MatthiasNobackandTomasVotruba
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
ATrainer’sJourney . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
TheOtherSide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
Who’sFaultisIt? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
FromBlametoPain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
FromPaintoIdea. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
FromIdeatoFirstRealReconstructor . . . . . . . . . . . . . . . . . . . . . . iv
We’veJustStarted,NowYouGetonBoard . . . . . . . . . . . . . . . . . . . iv
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
WhatisRector? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
WhoShouldReadThisBook? . . . . . . . . . . . . . . . . . . . . . . . . . . . vi
AnOverviewoftheContents . . . . . . . . . . . . . . . . . . . . . . . . . . . vi
AbouttheCodeSamples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
AbouttheAuthors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
MatthiasNoback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
TomasVotruba . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
WeWantYourFeedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
ProgrammaticallyModifyingPHPCode . . . . . . . . . . . . . . . . . . . . . . 1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
PrimitiveWaysofModifyingCode . . . . . . . . . . . . . . . . . . . . . . . . 2
TokenizingPHPCode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
ParsingPHPTokens:theAbstractSyntaxTree . . . . . . . . . . . . . . . . . 8
ConvertingtheASTBacktoPHPCode . . . . . . . . . . . . . . . . . . . . . 14
ManipulatingtheAST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
CONTENTS
NodeVisitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
ManipulatingtheASTwithaNodeVisitor . . . . . . . . . . . . . . . . . . . 23
Built-inNodeVisitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
ConnectingNodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
ResolvingFully-qualifiedNames. . . . . . . . . . . . . . . . . . . . . . . 31
FindingNodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
PHPToolsintheGame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
WorkingTogetherwithGiants . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2007to2021Timeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
ThePrimaryFeature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
1.CodingStandardTools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
PHP_CodeSniffer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
PHPCSFixer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
EasyCodingStandard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.StaticAnalyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
PHP-Parser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
PHPStan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Psalm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.InstantUpgradeTools. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Symfony-Upgrade-Fixer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Rector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.InstantDowngradeTools. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
PHP-Backporter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Galapagos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Transphpilea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
7to5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Rector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
WhentoUseWhichTool? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
IsYourProjectBareWithoutAnyTools? . . . . . . . . . . . . . . . . . . . . 47
RunRectorFirst,ThenPolishwithCodingStandards . . . . . . . . . . . . . 47
RecommendedTools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
CONTENTS
CreatingYourFirstRectorRule . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
What’saRectorRule? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
CreatingaCustomRule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
ExtendingAbstractRector. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
FindingtheRightNodeClass. . . . . . . . . . . . . . . . . . . . . . . . . 58
RunningaSingleRule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
RefactoringtheMethodCallNode . . . . . . . . . . . . . . . . . . . . . . . . 64
WhatIstheTypeofaVariable? . . . . . . . . . . . . . . . . . . . . . . . 67
WhatifWeRunRectorTwice? . . . . . . . . . . . . . . . . . . . . . . . . . . 73
EffectivityBeatsPerfection . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Test-drivenRuleDevelopment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
MigratingfromDateTimetoDateTimeImmutable . . . . . . . . . . . . . . . 76
CreatingaTestClass . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
AddingFirstTestFixture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
MakingtheFirstTestPass . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
NarrowingtheScopeoftheRefactoring . . . . . . . . . . . . . . . . . . . . . 90
CapturingtheReturnValueofmodify() . . . . . . . . . . . . . . . . . . . . . 95
SkippingCallsOnClassesThatAreNotDateTime . . . . . . . . . . . . . . 104
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
MoreTestingTechniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
TestingMultipleRulesCombined . . . . . . . . . . . . . . . . . . . . . . . . . 108
RulesShouldBeIdempotent . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
RemovingtheCloneStep . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
ComparingReturnValuestoValidateCorrectness . . . . . . . . . . . . . . . 123
Shouldn’tYouStartwithaRuleSetTest? . . . . . . . . . . . . . . . . . . . . 125
Low-HangingFruitFirst . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
CreateaConfigurableGenericRule . . . . . . . . . . . . . . . . . . . . . . . 127
UsingaConfigurableInsteadofaCustomRule . . . . . . . . . . . . . . . . 128
MakingRuleBehaviorConfigurable . . . . . . . . . . . . . . . . . . . . . . . 131
ConfigurableRuleMakesHiddenAssumptionsExplicit . . . . . . . . . . . 135
UsingSupportingClassesinFixtures . . . . . . . . . . . . . . . . . . . . . . . 136
CONTENTS
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
ContinuousRectifying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
TheNextMemberofYourTeam. . . . . . . . . . . . . . . . . . . . . . . . . . 143
TurningaJuniorIntoaSenioronDay1 . . . . . . . . . . . . . . . . . . . . . 143
Who’stoBlame? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
WhatDoesitLookLiketoWorkwithRectorinCI? . . . . . . . . . . . . . . 146
HowRectorisRectifyingItself . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
WorkFirst,EnjoyLater . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
RemovingBoringWorkOpensYourCreativity. . . . . . . . . . . . . . . . . 150
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
Rector+GitHubActions=NextMemberofYourTeam . . . . . . . . . . . . 152
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
WhatAreTheStepsforaRectorRun? . . . . . . . . . . . . . . . . . . . . . . 153
HowtoAddRectortoGitHubActions . . . . . . . . . . . . . . . . . . . . . . 154
1.GenericPHPSetup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
2.RectorSetup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
WhatFeedbackDoWeGet? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
HowtoWorkwithRector’sOutput? . . . . . . . . . . . . . . . . . . . . . . . 156
1.ManualChanges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
2.RunRectorLocally . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
3.LetRectorCodeforUsinPullRequests . . . . . . . . . . . . . . . . . 157
WhatisTheWorkWeWanttoAutomate? . . . . . . . . . . . . . . . . 158
MakeYourRectorWorkflowComeAlive . . . . . . . . . . . . . . . . . . . . 158
MakeRectorPushChanges . . . . . . . . . . . . . . . . . . . . . . . . . . 160
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
TheFutureofInstantUpgrades . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
RemovingLegacy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
HowCanWeBeSoConfident? . . . . . . . . . . . . . . . . . . . . . . . 164
NoBackward-CompatibilityBreaksandNoBackward-Compatibility . . . 165
ABotThatSendsPull-RequestsDaily . . . . . . . . . . . . . . . . . . . . . . 167
InstantUpgradesonClick . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
PHPStandsOut. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
CONTENTS
Epilogue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Preface
ByTomas
Working with old code is painful. In any framework, in any company, for you, and
for your employer. Nobody wants to ride a horse on the highway. But why are so
manypeopledoingit?WhyaretheresomanylegacyPHPprojects?
A Trainer’s Journey
Iwasaskingmyselfthisquestion6yearsago,almosteveryday.Iwasdoingtrainings
on Nette, Symfony, or Doctrine. My usual training looked like this: I came to a
company, I gave a presentation about Symfony and its cool new features, we did
hands-on coding, and I answered curious questions. I had a good feeling about this
and feedback from participants was also positive. I gained a lot of experience from
trainingsin30+differentPHPcompaniesanduseditineverynexttraining.Ithought:
“That’s what trainers do, I finally made it; changing the life of others in just one or
two days.” Gosh, I was proud of myself. Helping others is my life-long passion, so I
wasveryhappy,andalsowellpaid.Thisfeelingwassoontobechanged.
I had one training about new features in PHP 7 for a project that was running PHP
5.4.Afterayear,thesamecompanyaskedmeforafreshtrainingaboutnewfeatures
inthePHPworld.Ithought:“TheymustbeonPHP7.1bynow.”Iaskedthem,where
are they now, so I wouldn’t bore them with something they already know. Their
answer: “This year we’ll definitely get to a new PHP version.” I asked: “Oh, you’re
up to PHP 7.2 already? Wow!” They replied: “No, we’re almost at the PHP 7. That’s
whyweneedyoutohelpus!”
What?Mymindwasshocked.Suddenly,IrememberedanothertrainingIhad,where
theyaskedforalmostthesametopicasayearbefore.No,thatcan’tberight,canit?
Preface ii
The Other Side
A trainer’s point of view, and one or two days of training, are just one side of the
story.Ademoiseasytocode.Everybodyishappybecausetheyworkon200linesof
code and the progress is there. Then the developers have to go through a real code
baseandmakethechangehappenthereaswell.Theyhavetoevolvetheprojectfrom
1.000.000 to 5.000.000 lines of code, help the business to grow, and add new features.
Irealizedthatmytrainingsmayexpandknowledgebyshowingthehotnewsinthe
PHP world or a particular framework. But barely 2% of this knowledge will last for
longerthanaweekafterthetraining.
Who’s Fault is It?
I started to blame myself. My teaching methods were likely not very effective. I
tried to improve my presentation and learn more about the project beforehand, but
the results didn’t change much. Then I blamed the company business leaders. They
were nice to hire a trainer, but after he or she is gone, they’d force programmers
to work on that legacy code to add more features instead of refactoring it. Then I
blamedprogrammers,whoarenotwillingtospendfourhoursafterworktoeducate
themselves. They should be able to protest against legacy business and stop it, until
thecodeiscleanandfuntoworkwith.
LaterI’velearned,noneofthesetargetsareright.
From Blame to Pain
I stopped giving training so often because I felt like a failure. I was part of a system
that didn’t make much change for the better. The system repeated the same action
twice, with no gain. I couldn’t do it anymore. I mean, I physically could not do it.
Whynot?Letmeexplain.
When I was 8 years old, I lost a cap while playing outside. In situations like these,
my father used to punish me with tasks involving lots of repetition. I always had to
writelongsentences,a100times,like:“Iwillnevergooutsidewhileplayingandnot
Preface iii
returning back without the hat again, or I will have to go out and look for it and
won’treturnuntilIfindit.”
At first it was an interesting exercise in writing. After the 30 sentences though, my
hand started to hurt. The text was supposed to be clear to read and without any
mistake, and there were 10 more sentences left to go. I remember a Saturday, when
Ispentalmost4hoursonthis,unabletodoanythingelse.Mymomwashelpingme
sometimes,adding20sentencesherself.
Thiswasrepeatedacoupletimesamonth.Ibecameterrifiedtodoanythingmyfather
wouldn’t like. These punishments came out of the blue. Once my brother broke my
father’s tool and told it was me. I said I don’t know anything about it, so I got the-
100-sentencespunishmentnottolieeveragain.
Thankstothesepunishments,I’vehardcodedinmybrainthatanyrepetitionhurts
me and I’ll do anything to avoid it. That’s my personality trait. It took some time
butI’veputsomehealthyboundariesonthisandlearnedtouseittomyadvantage.
P.S.Don’ttakemewrong:Ilovemyfatherandwehaveasafeandwarmrelationship
now.I’mverygratefulforthepatience,attention,focus,andresiliencehetaughtme.
ThankstohimI’mthemanIwanttobe.
From Pain to Idea
Do you know when you look at the code, you feel it’s wrong, but you don’t know
whatitis?Ihadthisfeeling.Iknewtheremustbesomebetterwaytodothisthatis
cheaper, faster and re-usable. It won’t be easy to find it, but there must be a way to
solvethis.
I started to share this pain and frustration in chats with my friends and I slowly
builtupavision.Avisionofaworldwhereallprogrammersareonlythinkingabout
newalgorithms,andwheretermslike“legacycode”and“technicaldebt”don’texists.
Changingframeworksisamatterofasingleclick.Upgradesareaseasyascomposer
update.Andit’sallopensource,soanyonecanuseit.