Table Of Content

Analysis of Docker Security ThanhBui AaltoUniversitySchoolofScience [email protected] Abstract alization[19]. However, container-basedvirtualizationalso comeswithsecurityconcerns. Over the last few years, the use of virtualization technolo- In this paper, we analyze the security level of Docker gieshasincreaseddramatically. Thismakesthedemandfor [17], a well-known representative of container-based virtu- efficientandsecurevirtualizationsolutionsbecomemoreob- alization approach. We consider two areas: (1) the internal vious. Container-based virtualization and hypervisor-based security of Docker, and (2) how Docker interacts with the 5 virtualizationaretwomaintypesofvirtualizationtechnolo- security features of the Linux kernel, such as SELinux and 1 giesthathaveemergedtothemarket. Ofthesetwoclasses, AppArmor,inordertohardenthehostsystem. Theanalysis 0 container-based virtualization is able to provide a more examinedtheinternalsecurityofDockerbasedonthelevel 2 lightweightandefficientvirtualenvironment,butnotwithout of isolation Docker can provide to its virtual environments. n securityconcerns.Inthispaper,weanalyzethesecuritylevel TheinteractionbetweenDockerandthesecurityfeaturesof a J of Docker, a well-known representative of container-based thekernelwasestimatedbasedonhowthefeaturesaresup- approaches. Theanalysisconsiderstwoareas: (1)theinter- portedbyDocker. Tothebestofourknowledge,Dockerisa 3 1 nalsecurityofDocker,and(2)howDockerinteractswiththe relativelynewtechnology,andthisisoneofthefirstanalyses security features of the Linux kernel, such as SELinux and ofthiskindthatfocusonitssecurityaspects. ] AppArmor,inordertohardenthehostsystem. Furthermore, The paper is structured as follows: Section 2 provides a R the paper also discusses and identifies what could be done high-levelviewofthetwoclassesofvirtualizationsolutions. C whenusingDockertoincreaseitslevelofsecurity. Section 3 gives an overview of Docker and its underlying . s technologies. Section4presentsouranalysisofDockerse- c KEYWORDS:Containers,Docker,Security curity,andtheninSection5,wediscussthesecuritylevelof [ Dockerandwhatcouldbedonetoraiseitslevelofsecurity. 1 ThepaperconcludeswithasummaryinSection6. v 1 Introduction 7 6 Thelastdecadehasseenanexplosionofdevelopmentinthe 2 Virtualization Approaches 9 2 areaofvirtualizationtechnologies,whichallowthepartition- 0 ing of a computer system into multiple isolated virtual en- Mostofthevirtualizationtechnologiescanbeclassifiedinto 1. vironments. Thetechnologiesoffersubstantialbenefitsthat two major approaches: container-based virtualization and 0 havebeendrivingtheirdevelopmentrapidly.Oneofthemost hypervisor-based virtualization. The former provides virtu- 5 common reasons for adopting virtualization technologies is alization at the operating system level, while the latter pro- 1 servervirtualizationindatacenters. Withservervirtualiza- vides virtualization at the hardware level. Each of the ap- : v tion,anadministratorcancreateoneormorevirtualsystem proaches has its own advantages and disadvantages, which i instancesonasingleserver.Thesevirtualsystemsoperateas aredescribedinthissection. X realphysicalserversandcanberentedoutonasubscription Container-based virtualization is a lightweight virtual- r a basis. Amazon EC2, Rackspace, and DreamHost are some ization approach using the host kernel to run multiple vir- popularinstancesofsuchdatacenterserviceproviders. An- tualenvironments. Thesevirtualenvironmentsareoftenre- other common use is for desktop virtualization, where one ferredtoascontainers. Linux-VServer[31], OpenVZ[11], computercanrunseveralOSinstances. Desktopvirtualiza- and Linux Container (LXC) [10] are the three main repre- tionprovidessupportforapplicationsthatcanrunonlyona sentatives of this approach. The general architecture of a specificOS. container-basedvirtualizationsolutionisillustratedinFig.1. The growth in the use of virtualization technologies pro- Container-based virtualization virtualizes at the operating motes the demand for a virtualization solution which can system level, thus allowing multiple applications to operate provide dense, scalable, and secure user environments. A withoutredundantlyrunningotheroperatingsystemkernels large number of virtualization solutions have emerged to on the host. Its containers look like normal processes from the market. They can be classified into two major classes: outside,whichrunontopofthekernelsharedwiththehost container-basedvirtualizationandhypervisor-basedvirtual- machine. They provide isolated environments with neces- ization. Ofthesetwoclasses,container-basedvirtualization saryresourcestoexecuteapplications. Theseresourcescan isabletoprovideamorelightweightandefficientvirtualen- be either shared with the host or installed separately inside vironment. Itallowstentimesmorevirtualenvironmentsto thecontainer. runonaphysicalservercomparedtohypervisor-basedvirtu- Hypervisor-basedvirtualizationsolutionsprovidevirtu- AaltoUniversityT-110.5291SeminaronNetworkSecurity Autumn2014 Figure3: ArchitectureofDockerengine Figure1: ArchitectureofContainer-basedVirtualization 3 Docker Overview Dockerisanopensourcecontainertechnologywiththeabil- ity"tobuild, ship, andrundistributedapplications"[17]. It hasbeenusedinsomepopularapplications,suchasSpotify, Yelp,andEbay. Although container technologies have been around for more than a decade, Docker - a relatively new candidate - is currently one of the most successful technologies since Figure2: ArchitectureofHypervisor-basedVirtualization it comes with new abilities that earlier technologies did not possess.First,itprovidesinterfacestosimplyandsafelycre- ate and control containers. Secondly, developers can pack applications into lightweight Docker containers which can alizationatthehardwarelevel.Incontrasttocontainer-based operate on almost anywhere without modification. Further- virtualization,ahypervisorestablishescompletevirtualma- more, Docker can deploy more virtual environments than chines (VMs) on top of the host operating system (Fig. 2). other technologies can on the same hardware [19]. Last Each virtual machine comprises of not only an application butnotleast,Dockercooperateswellwiththird-partytools, anditsdependencies,butalsoanentireguestOSalongwith whichsimplifythemanagementanddeploymentprocessof aseparatekernel. Therearetwoclassesofhypervisors: the Dockercontainers. DevOpstools,suchasPuppet[13],An- Type1hypervisor,alsoknownasthebaremetalhypervisor, sible [1], and Vagrant [16] can integrate with Docker, thus which works directly on top of the underlying hardware of makingDockercontainerstobeeasilydeployedtoacloud. thehost,andtheType2hypervisor,alsoknownasthehosted Moreover, many orchestration tools, such as Mesos [22], hypervisor,whichworksontopofthehostoperatingsystem Shipyard[15],andKubernetes[7],alsosupportDockercon- [26]. Xen[18]isanexampleoftheformer,whileKVM[25] tainers. These tools provide an abstract layer of resources isofthelatter. SincetheType1hypervisordoesnotinclude managementandschedulingoverDocker. anextralayerofthehostOS,itprovidesbetterperformance Dockerconsistsoftwomajorcomponents:Dockerengine thantheType2hypervisor. andDockerHub.Theformerisanopensourcevirtualization Thedifferencesinthearchitecturebringsomebenefitsto solution,whilethelatterisaSoftware-as-a-Serviceplatform container-based virtualization over hypervisor-based virtu- forsharingDockerimages. Thefollowingsectionsdescribe alization. First, container-based virtualization can provide indetailsthesetwocomponents. higherer density of virtual environments. Since a container does not include an entire OS, the size and the required re- sourcestorunanapplicationinacontainerarelessthanthat ofaVMrunningthesameapplication.Asaresult,morecon- 3.1 DockerEngine tainersthantraditionalvirtualmachinescanbedeployedon thesamehost. Secondly,container-basedvirtualizationalso Docker engine is a lightweight and portable packaging tool offers better performance. This has been demonstrated by [17] which relies on container-based virtualization. There- experimentsinsomestudies[32,28,27,21]. Thesestudies fore, the architecture of the Docker engine (Fig. 3) is simi- showthattheperformanceofcontainer-basedvirtualization lar to that of container-based virtualization in general. The is better than with hypervisor-based virtualization in most Docker containers run on top of the Docker daemon which cases,anditisalmostasgoodasnativeapplications. is in charge of executing and managing all of the Docker However, despite all of the mentioned advantages, containers. TheDockerclient,whichprovidesanuserinter- container-based virtualization is unable to support a variety faceforinteractingwithcontainerstoDockerusers,accepts of environments in the way hypervisor-based virtualization commands from the users and then sends it to the Docker doessincealltheenvironmentsofthecontainersmustbeof daemonthroughRESTfulAPIs. Usingthismethodofcom- the same type as that of the host. For example, Windows municationenablestheDockerclienttorunonthesamehost containerscannotberunontopofaLinuxhost. asthecontainers,orevenondifferenthosts. AaltoUniversityT-110.5291SeminaronNetworkSecurity Autumn2014 ages. Userscanalsosearchforpublishedimagesanddown- load them with the Docker client. Furthermore, users can verify the authenticity and integrity of the downloaded im- agessinceDockersignedandverifiedtheimageswhentheir ownersubmittedthemtothehub. 4 Docker Security Analysis Security is one of the major challenges when running ser- Figure4: Containerimage vices in virtual environments, especially in a multi-tenant cloud system. Virtual machines provided by hypervisor- based virtualization techniques are claimed to be more se- DockerContainer cure than containers as they add an extra layer of isolation between the applications and the host. An application run- DockerusedtocommoditizeLXCtocreateDockercontain- ninginsideaVMisonlyabletocommunicatewiththeVM ers. Since version 0.9, Docker has replaced LXC with lib- kernel, not the host kernel. Consequently, in order for the container[8]-theirownvirtualizationformat-asthedefault applicationtoescalateoutofaVM,itmustbypasstheVM containerenvironmentsinceDockercommunitydesiresnot kernel and the hypervisor before it can attack the host ker- to depend on a third-party package. However, with either nel. Ontheotherhand,containerscandirectlycommunicate LXC or libcontainer, namespaces, cgroups, union file sys- withthehostkernel,thusallowinganattackertosaveagreat tem,andDockerimagesarestillthemajorunderlyingtech- amount of effort when breaking into the host system. This nologiestoimplementDockercontainers. raisesasecurityconcernovercontainers. Docker takes advantages of two Linux features, names- Docker is also a container-based virtualization technolo- paces and cgroups, to safely create virtual environment for gies, thus having the same issue. Our analysis aims to dis- its containers. The cgroups, or control groups, provide cover whether Docker provides a safe environment to run mechanismforaccountingandlimitingtheresourceswhich applications. Theanalysisconsiderstwoareas: theinternal theprocessesineachcontainercanaccess. Thenamespaces securityofDockercontainersandhowDockercontainersin- wraptheoperatingsystemresourcesintodifferentinstances. teractwiththeadditionalsecuritysystemsofthekernel. Theuseoftheseinstancesgivestheprocessesrunninginside a container the illusion that they have their own resources. Currently,Dockerusesfivenamespacestoprovideeachcon- 4.1 DockerInternalSecurity tainerwithaprivateviewoftheunderlyinghostsystem[23]: mount, hostname, inter-process communication (IPC), pro- We examine the internal security of Docker based on the cessidentifiers(PID),andnetwork. Eachofthemworkson systemandattackermodelandsecurityrequirementsasde- specifictypesofsystemresources.Thenetworknamespaces, scribedbyReshetovaetal. [29]forcomparinganumberof forexample,isolatethenetworkingresources,suchasIPad- theOS-levelvirtualizationtechnologies. dresses,andIProutingtables,inordertoprovideeachcon- Thesystemandattackermodelisasfollows:Asinglehost tainerwithaseparatednetworkstack. machineisrunninganumberofDockercontainersc1 ... cn, Docker launches its containers from Docker images. A onwhichasubsetC ofthecontainersarecompromisedand Dockerimageisaseriesofdatalayersontopofabaseim- the attacker has full control over those, but the remaining age (Fig. 4). Every Docker image starts from a base im- subsetofcontainersC isstillunderthecontrolofthelegit- age, such as Ubuntu base image or OpenSuse base image. imateusers. Inthismodel,theattackercanperformvarious Whenusersmakechangestoacontainer,insteadofdirectly typesofattacks,suchasDenial-of-ServiceandPrivilegees- writing the changes to the image of the container, Docker calation. adds an additional layer containing the changes to the im- Inordertoencounterwiththeseattacks,theauthorsstated age. Forexample, iftheuserinstallsMySQLtoanUbuntu thatanOS-levelvirtualizationsolutionshouldsatisfythefol- image, Docker creates a data layer containing MySQL and lowingrequirements: processisolation,filesystemisolation, thenaddstotheimage. Thisprocessmakestheimagedistri- device isolation, IPC isolation, network isolation and limit- bution process more efficiently since only the update needs ing of resources. The next sections present our analysis on tobedistributed. howDockerfulfillstherequirements. In order to work with multiple layers of an image as it were a single file system layer, Docker uses a special file ProcessIsolation system called Union File System (UnionFS). It allows files anddirectoriesindifferentfilesystemstobecombinedinto The main goal of process isolation is to prevent compro- asingleconsistentfilesystem. misedcontainersfromusingprocessmanagementinterfaces tointerferewithothercontainers. Dockerachievesisolation ofprocessesbywrappingtheprocessesrunningincontainers 3.2 DockerHub intonamespacesandlimitingtheirpermissionsandvisibility Dockerhub[4]isacentralrepositoryofimages(bothpublic toprocessesrunningintheothercontainersandtheunderly- andprivate),viawhichuserscansharetheircustomizedim- inghost. AaltoUniversityT-110.5291SeminaronNetworkSecurity Autumn2014 This mechanism operates with the support of the PID Thus,itiscrucialtolimitthesetofdevicenodesthatacon- namespaces, which isolate the process ID number space of tainercanaccess. acontainerfromthatofthehost. SincePIDnamespacesare The Device Whitelist Controller feature [3] of cgroups hierarchical[12],aprocesscanonlyseetheotherprocesses provides means to limit the set of devices that Docker al- initsownnamespaceorinits"children"namespaces. Asa lows a container to access. It also prevents the processes consequence,onceanewnamespaceiscreatedandassigned incontainersfromcreatingnewdevicenodes. Furthermore, toacontainer,thehostcanobserveandaffecttheprocesses Docker mounts container images with nodev, meaning that insidethenewPIDnamespaceofthecontainer,butthepro- even if a device node was pre-created inside the image, the cessesinsidethecontainercannotobserveordoanythingto processes in the container using the image cannot use it to theotherprocessesrunninginthehostorinothercontainers. communicate with the kernel. By default, Docker does not Iftheattackercannotobserveotherprocesses,itisharderto give extended privileges to its containers. Therefore, they attackthem. cannot access any devices. However, if the operator exe- ThePIDnamespacesalsoalloweachcontainertohaveits cutesacontaineras"privileged",Dockergrantsaccesstoall owninit-likeprocess(PID1),whichcausesalltheprocesses devicestothecontainer. in a namespace to be terminated if it is terminated. This processassiststheadministratorincompletelyshuttingdown acontainerwhensomethingsuspiciousisdetected. IPCIsolation The IPC (inter-process communication) is a set of objects FilesystemIsolation for exchanging data among processes, such as semaphores, message queues, and shared memory segments. The pro- In order to achieve filesystem isolation, the filesystems of cesses running in containers must be restricted so that they the host and containers must be protected from illegitimate cancommunicateonlyviaacertainsetofIPCresourcesand accessandmodification. aredisallowedtointerferewiththoseinothercontainersand Dockerusesthemountnamespaces,alsocalledthefilesys- thehostmachine. tem namespaces, to isolate the filesystem hierarchy associ- Docker achieves IPC isolation by using the IPC names- ated with different containers. The mount namespaces pro- paces, which allows the creation of separated IPC names- vide the processes of each container a different view of the paces. The processes in an IPC namespace cannot read or filesystem tree and restrict all the mount events occurring write the IPC resources in other IPC namespaces. Docker inside the container to only have impact inside the con- assignsanIPCnamespacetoeachcontainer,thuspreventing tainer. However, some of the kernel filesystems are not the processes in a container from interfering with those in namespaced; for example, those under /sys, /proc/sys, othercontainers. /proc/sysrq −trigger, /proc/irq, and /proc/bus, and a Docker container needs to mount them in order to operate. This causes the issue that a container inherits the view of NetworkIsolation these filesystems from the host and are able to access them directly. Docker limits the threats that a compromised con- Networkisolationisimportanttopreventnetwork-basedat- tainer could make to the host via these filesystems with the tacks, such as Man-in-the-Middle (MitM) and ARP spoof- two filesystems protection mechanisms: (1) removing the ing. Containersmustbeconfiguredinsuchawaythatthey write permission to these filesystems from containers and areunabletoeavesdroponormanipulatethenetworktraffic (2) not allowing any process of a container to remount any oftheothercontainersnorthehost. filesystemwithinthecontainer[24]. Thesecondmechanism For each container, Docker creates an independent net- is achieved by removing the CAP_SYS_ADMIN capa- working stack by using network namespaces. Therefore, bilityfromcontainers. each container has its own IP addresses, IP routing tables, Docker also employs a mechanism called copy-on-write networkdevices,etc. Thisallowscontainerstointeractwith filesystem[24]. Asmentionedbefore, Dockercreatescon- eachotherthroughtheirrespectivenetworkinterfaces,which tainersbasedonfilesystemimages,andacontainercanwrite isthesameashowtheyinteractwithexternalhosts. contenttoitsownbaseimage. Whenmultiplecontainersare Bydefault, connectivitybetweencontainersaswellasto createdonthesameimage,thecopy-on-writefilesystemal- the host machine is provided using Virtual Ethernet bridge lows each container to write content to its specific file sys- [5](Fig.5).Withthisapproach,Dockercreatesavirtualeth- tem, thus preventing other containers from discovering the ernet bridge in the host machine, named docker0, that au- changesoccurringinsidethecontainer. tomaticallyforwardspacketsbetweenitsnetworkinterfaces. When Docker creates a new container, it also establishes a new virtual ethernet interface with a unique name and then DeviceIsolation connects this interface to the bridge. The interface is also In Unix, the kernel and applications access the hardware connectedtotheeth0interfaceofthecontainer,thusallow- throughdevicenodeswhichbasicallyarespecialfilesacting ingthecontainertosendpacketstothebridge. astheinterfacestothedevicedrivers. Ifacontainercanac- We note here that the default connectivity model of cesssomeimportantdevicenodes,suchas/dev/mem(the DockerisvulnerabletoARPspoofingandMacfloodingat- physicalmemory),/dev/sd∗(thestorage)or/dev/tty (the tacks since the bridge forwards all of its incoming packets terminal), it can make serious damage on the host system. withoutanyfiltering. AaltoUniversityT-110.5291SeminaronNetworkSecurity Autumn2014 CAP_SETPCAP Modifyprocesscapabilities CAP_SYS_MODULE Insert/Removekernelmodules CAP_SYS_RAWIO ModifyKernelMemory CAP_SYS_PACCT Configureprocessaccounting CAP_SYS_NICE ModifyPriorityofprocesses CAP_SYS_RESOURCE OverrideResourceLimits CAP_SYS_TIME Modifythesystemclock CAP_SYS_TTY_CONFIG Configurettydevices CAP_AUDIT_WRITE Writetheauditlog CAP_AUDIT_CONTROL ConfigureAuditSubsystem CAP_MAC_OVERRIDE IgnoreKernelMACPolicy CAP_MAC_ADMIN ConfigureMACConfiguration Figure5: ThedefaultnetworkingmodelofDocker CAP_SYSLOG ModifyKernelprintkbehavior CAP_NET_ADMIN Configurethenetwork CAP_SYS_ADMIN Catchall Table1: SomecapabilitiesdisallowedinDockercontainers LimitingofResources [24] Denial-of-Service(DoS)isoneofthemostcommonattacks onamulti-tenantsystem,whereaprocessoragroupofpro- cessesattempttoconsumealloftheresourcesofthesystem, the privileges of the superuser into capabilities, which the thus disrupting normal operation of the other processes. In kernelcanindependentlyenableordisable. orderto preventthiskind ofattack, it shouldbe possible to Docker containers run on a kernel shared with the host limittheresourcesthatareallocatedtoeachcontainer. system, so most of their tasks can be handled by the host. Cgroups are the key component that Docker employs to As a result, in most cases, it is unnecessary to provide deal with this issue. They control the amount of resources, full root privileges to a container, thus removing some of such as CPU, memory, and disk I/O, that any Docker con- the root capabilities from a container does not affect the tainer can use, ensuring that each container obtains its fair usability or functionality of the container but effectively share of the resources and preventing any container from improves the security of the system. For example, the consuming all resources. They also allow Docker to con- CAP_NET_ADMIN capability,whichprovidestheabil- figurethelimitsandconstraintsrelatedtotheresourcesallo- ity to modify the system network, can be removed from a catedtoeachcontainer. Forexample,onesuchconstraintis containersinceallnetworkingconfigurationcanbehandled limitingtheavailableCPUsavailabletoaspecificcontainer. bytheDockerdaemonbeforestartingthecontainer. Dockerallowsconfigurationofthecapabilitiesthatacon- 4.2 DockerandKernelSecuritySystems tainercanuse.Bydefault,Dockerdisablesalargenumberof Linuxcapabilitiesfromitscontainersinordertopreventan Some kernel security systems exist in order to harden the intruder to damage the host system even when the intruder security of a Linux host system, including Linux capabili- has obtained root access within a container. Some of the ties and Linux Security Module (LSM). Linux capabilities capabilities are presented in table 1, and their detailed de- restricts the privileges assigned to each process. LSM pro- scription can be found in the Linux capabilities man page videsaframeworkwhichallowstheLinuxkerneltosupport [9]. different security models. The LSMs that have been inte- gratedintotheofficialLinuxkernelincludeAppArmor[20], SELinux SELinux[30],andSeccomp[14]. ThispapersurveysLinuxcapabilitiesandtwoLSMs,Ap- SELinux is a security enhancement to the Linux system. pArmor[20]andSELinux[30],whichDockercurrentlysup- Linux comes with the standard Discretionary Access Con- ports. DockercanalsocollaboratewithSeccompbutonlyif trols (DAC) mechanism (i.e., owner/group and permission LXCisused; thus,wedonotincludeitinthesurvey. Even flagsofanobject)tocontroltheaccesstoanobject.SELinux thoughDockerdoesnotsupportothersecuritysystemsatthe provides an additional layer of permission checking, called moment, it does not interfere with them. Therefore, these Mandatory Access Control, after the standard DAC is per- systemscanrunindependentlyofDockercontainerstopro- formed. InSELinux,everythingiscontrolledbylabels. Ev- tectthehost[2]. eryfile/directory,process,andsystemobjecthasalabel.The administratorofthesystemusestheselabelstowriterulesto controlaccessbetweenprocessesandsystemobjects. These LinuxCapabilities rules are called policies. The SELinux policies can be di- As stated in Linux capabilities man page [9], tradition- videdintothreeclasses:Typeenforcement,Multi-levelsecu- ally, Unix systems classified processes into two categories: rity(MLS)enforcement,andMulti-categorysecurity(MCS) privileged processes (owned by superuser or root) and un- enforcement. privileged processes (owned by normal users). The kernel With the DAC mechanism, owners have full discretion skipped all permission checks on the privileged processes over their objects, meaning that if the owners are compro- butconductedfullpermissioncheckingonunprivilegedpro- mised, the attacker has control over all of their objects. In cesses.However,theLinuxkernel,sinceversion2.2,divides SELinuxmodel,incontrast,thekernelmanagesandenforces AaltoUniversityT-110.5291SeminaronNetworkSecurity Autumn2014 alloftheaccesscontrolsoverobjects,nottheirowners. This it does not provide any filter on the network traffic passing providesasecureseparationforcontainersasitcanprevent throughthebridge. However, thisproblemcanbesolvedif processes, even with root privileges, within a container to the administrator manually adds filtering, such as ebtables illegitimatelyaccessobjectsoutsidethecontainers. [6],tothebridge,orchangesthenetworkingconnectivityto Dockerusestwoclassesofpolicyenforcement: Typeen- amoresecureone,suchasvirtualnetwork. forcement and MCS enforcement [24]. The Type enforce- Itisalsoworthhighlightingthatiftheoperatorrunsacon- mentprotectsthehostfromtheprocessesincontainers,and taineras"privileged",Dockergrantsfullaccesspermissions theMCSenforcementprotectsacontainerfromanothercon- tothecontainer,whichisnearlythesameasthatofprocesses tainer. runningnativelyonthehost. Therefore,itismoresecureto With Type enforcement, Docker labels all container pro- operatecontainersas"non-privileged". cesses with svirt_lxc_net_t type and all content within a Furthermore, even though containers can provide higher container with svirt_sandbox_file_t type. The processes densityofvirtualenvironmentsandbetterperformance,they runningwithsvirt_lxc_net_ttypecanonlyaccess/writeto haveabiggerattacksurfacethanvirtualmachinessincecon- the content labeled with svirt_sandbox_file_t type, but tainerscandirectlycommunicatewiththehostkernel. How- not to any other label on the system. Therefore, the pro- ever, it is possible to reduce the attack surface while main- cessesrunningwithincontainerscanonlyusethecontentin- tainingtheseadvantages. Forexample,thiscanbeachieved sidecontainers.However,onlywiththispolicyenforcement, byplacingcontainersinsidevirtualmachines. Dockerallowstheprocessesinonecontainertohaveaccess tothecontentofothercontainers. MCSenforcementisnec- essarytosolvethisissue. Whenacontainerislaunched,the 6 Conclusion and Future work DockerdaemonpicksarandomMCSlabelandthenputsthis labelonalloftheprocessesandcontentofthecontainer.The Container-based virtualization can provide higher den- kernelonlyallowprocessestoaccesscontentwiththesame sity virtual environments and better performance than MCS label, thus preventing a compromised process in one hypervisor-based virtualization. However, the latter is ar- containerfromattackingothercontainers. gued to be more secure than the former. In this paper, we conducted an analysis on Docker, which is one of the mostpopularcontainer-basedvirtualizationtechnologies,to AppArmor discover how safe its containers are. Our analysis shows AppArmor is also a security enhancement model to Linux that Docker containers are fairly secure, even with the de- based on Mandatory Access Control like SELinux, but re- fault configuration. The security level of Docker contain- stricting its scope to individual programs. It permits the ers could also be increased if the operator runs them as administrator to load a security profile into each program, "non-privileged"andenablesadditionalhardeningsolutions whichlimitsthecapabilitiesoftheprogram.AppArmorsup- inLinuxkernel,suchasAppArmororSELinux. portstwomodes: enforcementmodeandcomplain/learning Thefutureworkafterthispapercouldbetocomparethe mode. Theenforcementmodeenforcesthepoliciesdefined security of Docker containers with that of other container- intheprofile. However,inthecomplain/learningmode,the izationsystemsorwithvirtualmachines. Suchstudiescould violations of profile policies are permitted, but also logged. lead to e.g. a detailed static analysis Docker or a broader Thislogcanbeusefulfordevelopingnewprofileslater. viewofsecurityincontainersingeneral. On systems that support AppArmor, Docker provides an interface for loading a pre-defined AppArmor profile when launching a new container. This profile is loaded into Acknowledgement the container in enforcement mode in order to ensure that the processes in the container are restricted according to This research paper is made possible through the help and the profile. If the administrator does not specify a profile support of Miika Komu, Roberto Morabito, Jimmy Kjäll- when launching a container, the Docker daemon automat- man,andTeroKauppinenfromNomadiclab. ically loads a default profile to the container, which de- nies access to important filesystems on the host, such as /sys/fs/cgroups/and/sys/kernel/security/. References [1] Ansible. http://www.ansible.com/home/. 5 Discussion [Accessed25October2014]. TheanalysisshowsthatDockerprovidesahighlevelofiso- [2] Containers & docker: How secure are they? lation and resource limiting for its containers using names- https://blog.docker.com/2013/08/ paces,cgroups,anditscopy-on-writefilesystem,evenwith containers-docker-how-secure-are-they. thedefaultconfiguration. Italsosupportsseveralkernelse- [Accessed25October2014]. curity features, which help to hardening the security of the host. The only problem we found with Docker was related [3] Device whitelist controller. https://www. toitsdefaultnetworkingmodel. Thevirtualethernetbridge kernel.org/doc/Documentation/ which Docker uses as its default networking model, is vul- cgroups/devices.txt. [Accessed 12 Octo- nerable to ARP spoofing and MAC flooding attacks since ber2014]. AaltoUniversityT-110.5291SeminaronNetworkSecurity Autumn2014 [4] Docker hub. https://hub.docker.com/. [Ac- [21] W.Felter,A.Ferreira,R.Rajamony,andJ.Rubio. An cessed30September2014]. updated performance comparison of virtual machines and linux containers. Technical Report RC25482 [5] Docker: Network configuration. https://docs. (AUS1407-001),IBMResearchDivision,July2014. docker.com/articles/networking/. [Ac- cessed24September2014]. [22] B. Hindman, A. Konwinski, M. Zaharia, A. Ghodsi, A.D.Joseph,R.Katz,S.Shenker,andI.Stoica.Mesos: [6] Ebtables. http://ebtables.netfilter. Aplatformforfine-grainedresourcesharinginthedata org/. [Accessed25October2014]. center. In Proceedings of the 8th USENIX Confer- [7] Kubernetes project. https://github.com/ ence on Networked Systems Design and Implementa- googlecloudplatform/kubernetes. [Ac- tion, NSDI’11, pages 295–308, Berkeley, CA, USA, 2011.USENIXAssociation. cessed10November2014]. [23] D. J. Walsh. Are docker containers really se- [8] Libcontainer project. https://github.com/ cure? http://opensource.com/business/ docker/libcontainer. [Accessed 25 October 14/7/docker-security-selinux. [Accessed 2014]. 25October2014]. [9] Linux capabilities. http://linux.die.net/ [24] D.J.Walsh. Bringingnewsecurityfeaturestodocker. man/7/capabilities. [Accessed 12 October https://opensource.com/business/ 2014]. 14/9/security-for-docker. Available at: [10] LXC. https://linuxcontainers.org/. [Ac- [Accessed25October2014]. cessed30September2014]. [25] A. Kivity, Y. Kamay, D. Laor, and U. Lublin. KVM: [11] OpenVZ. http://openvz.org/. [Accessed 30 the linux virtual machine monitor. In Proceedings of September2014]. theLinuxSymposium,volume1,pages225–230.2007. [12] PIDnamespacesinthe2.6.24kernel. http://lwn. [26] D. Merkel. Docker: Lightweight linux containers for net/Articles/259217/. [Accessed 30 Septem- consistent development and deployment. Linux J., ber2014]. 2014(239),Mar.2014. [13] Puppet. http://puppetlabs.com/. [Accessed [27] P.Padala,X.Zhu,Z.Wang,S.Singhal,andK.G.Shin. 18October2014]. Performance evaluation of virtualization technologies forserverconsolidation. HPLaboratories,2007. [14] SECure COMPuting with filters. https: //www.kernel.org/doc/Documentation/ [28] N.RegolaandJ.-C.Ducom.Recommendationsforvir- prctl/seccomp_filter.txt. Available at: tualization technologies in high performance comput- [Accessed02November2014]. ing. In 2010 IEEE Second International Conference on Cloud Computing Technology and Science (Cloud- [15] Shipyard project. https://github.com/ Com),pages409–416,Nov.2010. shipyard/shipyard. [Accessed 12 November [29] E.Reshetova,J.Karhunen,T.Nyman,andN.Asokan. 2014]. Security of OS-level virtualization technologies. In [16] Vagrant. https://www.vagrantup.com/. [Ac- Proceedings of the 2014 NordSec Conference, pages cessed15November2014]. 77–93,Norway,2014. [17] What is docker? https://docker.com/ [30] S.Smalley,C.Vance,andW.Salamon. Implementing whatisdocker/. [Accessed15November2014]. SELinuxasalinuxsecuritymodule. NAILabsReport #01-043,NAILabs,Dec.2001. RevisedMay2002. [18] B.R.Anderson, A.K.Joines, andT.E.Daniels. Xen worlds: Leveraging virtualization in distance educa- [31] S. Soltesz, H. Potzl, M. E. Fiuczynski, A. Bavier, tion. InProceedingsofthe14thAnnualACMSIGCSE and L. Peterson. Container-based operating system ConferenceonInnovationandTechnologyinComputer virtualization: A scalable, high-performance alterna- Science Education, ITiCSE ’09, pages 293–297, New tive to hypervisors. In Proceedings of the 2Nd ACM York,NY,USA,2009.ACM. SIGOPS/EuroSys European Conference on Computer Systems2007,pages275–287,USA,2007.ACM. [19] C.Burniske. Containers: Thenextgenerationofvirtu- alization? http://ark-invest.com/webx0/ [32] M.G.Xavier,M.V.Neves,F.D.Rossi,T.C.Ferreto, containers-next-generation-virtualization.T. Lange, and C. A. F. De Rose. Performance eval- [Accessed22November2014]. uation of container-based virtualization for high per- formancecomputingenvironments. InProceedingsof [20] C.Cowan,S.Beattie,G.Kroah-Hartman,C.Pu,P.Wa- the21stEuromicroInternationalConferenceonParal- gle, and V. Gligor. SubDomain: Parsimonious server lel,Distributed,andNetwork-BasedProcessing,pages security. In Proceedings of the 14th USENIX Confer- 233–240, Washington, DC, USA, 2013. IEEE Com- enceonSystemAdministration,LISA’00,pages355– puterSociety. 368,Berkeley,CA,USA,2000.USENIXAssociation.