Apache-Trafficserver-Server Documentation

532 Pages·2017·4.32 MB·English

Apache-Trafficserver-Server
Documentation
latest
7 21, 2017

Contents
1 1
1.1 ApacheTrafficServer? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 TypographicConventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.3 OtherResources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2 GettingStarted 3
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.3 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.4 LoggingandMonitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.5 FurtherSteps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3 Administrator’sGuide 11
3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.2 TrafficServer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.4 InteractingwithTrafficServer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.5 Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.6 CacheStorage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.7 Plugins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.8 EventandErrorMonitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
3.9 ConfiguringTrafficServer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
3.10 PerformanceTuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
3.11 ConfigurationFiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
3.12 Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
4 Developer’sGuide 259
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
4.2 ReleaseProcess . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
4.3 ContributingtoTrafficServer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
4.4 UsingVagranttoTestTrafficServer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
4.5 DebuggingandAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
4.6 CacheArchitecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
4.7 PluginDevelopment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
4.8 ConfigurationVariableImplementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
4.9 APIReference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379
4.10 ContinuousIntegration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475
i
4.11 Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475
4.12 HostResolutionProposal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 488
5 Appendices 491
5.1 CommandLineUtilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491
5.2 FrequentlyAskedQuestions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505
5.3 TroubleshootingTips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 511
5.4 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514
5.5 HTTPStatusCodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516
6 Indicesandtables 517
ii
1
CHAPTER
Apache Traffic Server?
Apache Traffic Server™ is a high-performance web proxy cache that improves network efficiency and performance
bycachingfrequently-accessedinformationattheedgeofthenetwork. Thisbringscontentphysicallyclosertoend
users,whileenablingfasterdeliveryandreducedbandwidthuse.TrafficServerisdesignedtoimprovecontentdelivery
forenterprises,Internetserviceproviders(ISPs),backboneproviders,andlargeintranetsbymaximizingexistingand
availablebandwidth.
Thismanualwillexploreeveryaspectofinstalling,managing,extending,andtroubleshootingTrafficServer.
Typographic Conventions
Thisdocumentationusesthefollowingtypographicconventions:
Italic Usedtointroducenewtermsontheirinitialappearance.
Example: TheApacheTrafficServer™objectstorageisbasedonacyclonebufferarchitecture.Cyclonebuffers
areaformofstorageaddressinginwhichasinglewritercontinuallyreclaimstheoldestallocationsforuse
bynewupdates.
Monospace RepresentsC/C++languagestatements,commands,filepaths,filecontent,andcomputeroutput.
Example: ThedefaultinstallationprefixforTrafficServeris/usr/local/ts.
BracketedMonospace Representsvariablesforwhichyoushouldsubstituteavalueinfilecontentorcommands.
Example: Running the command traffic_line -r <name> will display the current value of a perfor-
mancestatistic,where<name>isthestatisticwhosevalueyouwishtoview.
Ellipsis Indicatestheomissionofirrelevantorunimportantinformation.
1
Apache-Trafficserver-ServerDocumentation, latest
Other Resources
Websites
OfficialWebsite https://trafficserver.apache.org/
TheofficialApacheTrafficServer™projectwebsiteishostedbytheApacheSoftwareFoundation. Documen-
tation,softwaredownloads,communityresourcelinks,securityannouncements,andmorearelocated,orlinked
to,atthesite.
OnlineDocumentation https://docs.trafficserver.apache.org/
ThemostuptodateversionofthedocumentationishostedatReadtheDocs,withbuilt-insearchfunctionality.
Documentationforpastreleasesisalsoavailable.
JiraBugTracker https://issues.apache.org/jira/browse/TS
Ifyouwishtoreportbugs,orlookforopenissuesonwhichyoumayhelpcontributetotheTrafficServerproject,
pleasevisitthepublicJirabugtrackersite.
Mailing Lists
UserList The user’s mailing list offers support and discussions oriented to users and administrators of the Traffic
Serversoftware.
[e-mail].
DeveloperList If you have questions about, or wish to discuss, the development of Traffic Server, plugins for the
server,orotherdeveloper-orientedmatters,thedeveloperlistoffersanactivelistofbothcoreprojectmembers
andexternalcontributors.
[email protected].
Internet Relay Chat (IRC)
The #traffic-server channel on irc.freenode.net is the official IRC resource for the Traffic Server
project,andboastsactivediscussions.
Community Forums
2 Chapter1.
2
CHAPTER
Getting Started
Introduction
Apache Traffic Server™ provides a high-performance and scalable software solution for both forward and reverse
proxyingofHTTP/HTTPStraffic,andmaybeconfiguredtorunineitherorbothmodessimultaneously. ThisGetting
StartedguideexplainsthebasicstepsanadministratornewtoTrafficServerwillneedtoperformtogetthesoftware
upandrunninginaminimalconfigurationasquicklyaspossible.
Example Scenario
In this guide, we will use the fictional company Acme Widgets as a basis for the configuration examples. Acme
Widgets has a product brochure website (assumed to reside at the domain www.acme.com) that performs very
poorly. Thecontentmanagementsoftwaretheychosetakesanunbearableamountoftimetogeneratepagesonevery
requestandtheirengineeringteamhaschosenTrafficServerasacachingproxylayertoimprovesiteperformance.
Separately,AcmeWidgetshasdecidedtouseTrafficServertohelpimprovetheperformanceoftheiroffice’sInternet
access,whichishobbledbytheirrelianceonanagingleasedlineandcertainemployees’predilectionforextracurric-
ularwebbrowsing.
Terminology
Thisguideusessometermswhichmaybeunfamiliartoadministratorsnewtoproxyservers.
OriginServer Theserverwhichgeneratesthecontentyouwishtoproxy(andoptionallycache)withTrafficServer.
Inaforwardproxyconfiguration,theoriginservermaybeanyremoteservertowhichaproxiedclientattempts
to connect. In a reverse proxy configuration, the origin servers are typically a known set of servers for which
youareusingTrafficServerasaperformance-enhancingcachinglayer.
ReverseProxy Areverseproxyappearstooutsideusersasifitweretheoriginserver,thoughitdoesnotgeneratethe
contentitself. Instead,itinterceptstherequestsand,basedontheconfiguredrulesandcontentsofitscache,will
eitherserveacachedcopyoftherequestedcontentitself, orforwardtherequesttotheoriginserver, possibly
cachingthecontentreturnedforusewithfuturerequests.
3
Apache-Trafficserver-ServerDocumentation, latest
ForwardProxy A forward proxy brokers access to external resources, intercepting all matching outbound traffic
fromanetwork. Forwardproxiesmaybeusedtospeedupexternalaccessforlocationswithslowconnections
(bycachingtheexternalresourcesandusingthosecachedcopiestoservicerequestsdirectlyinthefuture), or
maybeusedtorestrictormonitorexternalaccess.
TransparentProxy Atransparentproxymaybeeitherareverseorforwardproxy(thoughnearlyallreverseproxies
aredeployedtransparently),thedefiningfeaturebeingtheuseofnetworkroutingtosendrequeststhroughthe
proxy without any need for clients to configure themselves to do so, and often without the ability for those
clientstobypasstheproxy.
Foramorecompletelistofdefinitions,pleaseseetheGlossary.
Installation
As with many other software packages, you may have the option of installing Apache Traffic Server™ from your
operating system distribution’s packages, or compiling and installing from the source code yourself. Distribution
packages may lag behind the current stable release of Traffic Server, sometimes by a significant amount. While we
willcoverbrieflythepackagesrequiredforcommonoperatingsystemdistributions,theremainderofthisguidewill
beassumingthelateststablereleaseofTrafficServerwhendiscussingconfigurationparametersandavailablefeatures.
Installing From Distribution Packages
It is recommended that you install from source code, as described in the section below, rather than rely on your
distribution’spackagesifyouwishtohaveaccesstothelatestfeaturesandfixesinTrafficServer.
Ubuntu
TheCanonicalrepositoriesupthrough,andincluding,UtopicUnicornonlyprovideTrafficServerv3.2.x.
sudo apt-get install trafficserver
RHEL/CentOS
TrafficServerisavailablethroughtheEPELrepositories. Ifyoudonothavethoseconfiguredonyourmachineyet,
youmustinstallthemfirstwiththefollowing:
wget https://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-5.noarch.rpm
sudo rpm -Uvh epel-release-7*.rpm
Ensuringthatyoureplacethereleasenumberwithavaluethatisappropriateforyoursystem. OnceyouhaveEPEL
installed,youmayinstallTrafficServeritself.
sudo yum install trafficserver
OmniOS(illumos)
OmniOS(anillumos-baseddistribution)providestheTrafficServerpackageinitspublicOmniTI-MSpublisher.
sudo pkg set-publisher -g http://pkg.omniti.com/omniti-ms/ ms.omniti.com
sudo pkg install omniti/server/trafficserver
4 Chapter2. GettingStarted
Apache-Trafficserver-ServerDocumentation, latest
Thelatestreleasepublished,atthetimeofthiswriting,isTrafficServerv4.2.1.
Installing From Source Code
Toinstallfromsourcecode,youwillneedtohavethefollowingtoolsandlibrariesonthemachineusedtobuildTraffic
Server:
• pkgconfig
• libtool
• gcc(>=4.3orclang>3.0)
• GNUmake
• openssl
• tcl
• expat
• pcre
• libcap
• flex(forTPROXY)
• hwloc
• lua
• curses(fortraffic_top)
• curl(fortraffic_top)
TobuildTrafficServerfromaGitclone(themethodwewillbeusinginthisguide),youwillalsoneedthefollowing:
• git
• autoconf
• automake
Inthisguide,TrafficServerwillbebuilttousethedefaultnobodyuserandgroupandwillbeinstalledto/opt/ts.
Itisassumedthatallofthedependenciesarelocatedinstandardpaths. Ifthisisnotthecase,youmayneedtousethe
appropriate--with-<package>optionsasdetailedintheoutputof./configure --help.
1. Clonetherepository(youmayskipthisifyouhavedownloadedanarchiveofthesourcecodetobuildaspecific
officialreleaseofTrafficServerinsteadoftheHEADfromsourcecontrol):
git clone https://git-wip-us.apache.org/repos/asf/trafficserver.git
2. Changetothecloned(orunarchived)directory:
cd trafficserver/
3. IfyouhaveclonedtherepositoryfromGit,youwillneedtogeneratetheconfigurescriptbeforeproceeding:
autoreconf -if
4. Configurethesourcetree:
./configure --prefix=/opt/ts
2.2. Installation 5
Apache-Trafficserver-ServerDocumentation, latest
5. BuildTrafficServerwiththegeneratedMakefiles,andtesttheresults:
make
make check
6. InstallTrafficServertotheconfiguredlocation:
sudo make install
7. Optionally,youmayfinditprudenttoruntheregressiontestsonyournewlyinstalledTrafficServer:
cd /opt/ts
sudo bin/traffic_server -R 1
Configuration
Wewillbetacklingtwoseparateconfigurationscenariosinthefollowingsections. Thefirstisthemostcommonap-
plicationofaperformance-enhancingcachingproxyforexternally-facingwebsites,atransparentandcachingreverse
proxy which forwards all requests presented to it to a single origin address and caches the responses based on their
cachecontrolheaders(aswellassomesimpleheuristicsforspecificcontenttypeswhencachecontrolheadersarenot
present).
Thesecondconfigurationwewillreviewisaverybasictransparentforwardproxy,typicallyusedinsituationswhere
you either need to improve the performance of a local network’s use of external resources, or you wish to have the
capabilityofmonitoringorfilteringthetraffic.
Configuring A Reverse Proxy
A minimal reverse proxy configuration requires changes to only a few configuration files, which will all be located
inthe/opt/ts/etc/trafficserverdirectoryifyouhaveconfiguredyourinstallationpertheinstructionsin
InstallingFromSourceCodeabove.
Fortheseexamples,wewillbeassumingthatTrafficServerisrunningonthesamehostmachineastheoriginwebsite.
Thisisnotarequirement, andyoumaychoosetorunTrafficServeronanentirelydifferenthost, evenconnectedto
entirelydifferentnetworks,aslongasTrafficServerisabletoreachtheoriginhost.
EnableReverseProxying
Withintherecords.configconfigurationfile,ensurethatthefollowingsettingshavebeenconfiguredasshown
below:
CONFIG proxy.config.http.cache.http INT 1
CONFIG proxy.config.reverse_proxy.enabled INT 1
CONFIG proxy.config.url_remap.remap_required INT 1
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
CONFIG proxy.config.http.server_ports STRING 8080
:ts:cv:‘proxy.config.http.cache.http‘ EnablescachingofproxiedHTTPrequests.
:ts:cv:‘proxy.config.reverse_proxy.enabled‘ Enablesreverseproxyingsupport.
:ts:cv:‘proxy.config.url_remap.remap_required‘ ThissettingrequiresthataremapruleexistbeforeTrafficServer
will proxy the request and ensures that your proxy cannot be used to access the content of arbitrary websites
(allowingsomeoneofmaliciousintenttopotentiallymasktheiridentitytoanunknowningthirdparty).
6 Chapter2. GettingStarted