The Challenges Tomcat Faces in High Throughput Production Systems May 17, 2017 © 2017 Alibaba Middleware Group 1 Biography • Huxing Zhang (张乎兴) • Software Engineer @ Alibaba Middleware Group since 2014 • Previously worked for DeNA, Tokyo, Japan • Apache Tomcat Committer since 2016 • Core tomcat maintainer @ Alibaba Inc. May 17, 2017 © 2017 Alibaba Middleware Group 2 Outline • How we use Tomcat at Alibaba • High throughput web applications • The challenges • Case studies • Summary • Q & A May 17, 2017 © 2017 Alibaba Middleware Group 3 Outline • How we use Tomcat at Alibaba • High throughput web applications • The challenges • Case studies • Summary • Q & A May 17, 2017 © 2017 Alibaba Middleware Group 4 Tomcat in Alibaba • Small trial since 2009 • Migration From JBoss to Tomcat • Jboss uses Tomcat as its servlet container • Officially started in 2013 • Becomes the dominant servlet container in 2015 • Thousands of web applications • Hundreds of thousands of instances • Status Quo • 95% tomcat, 5% other • Mainly 7.0.x (support JDK 6+) • Few 8.0.x • Emerging 8.5.x because of micro-service (e.g. spring-boot) May 17, 2017 © 2017 Alibaba Middleware Group 5 Deployment Standard One web application per Tomcat instance (cid:53)(cid:88)(cid:81)(cid:87)(cid:76)(cid:80)(cid:72)(cid:3)(cid:38)(cid:82)(cid:81)(cid:393)(cid:74)(cid:88)(cid:85)(cid:68)(cid:87)(cid:76)(cid:82)(cid:81)(cid:3)(cid:11)(cid:38)(cid:36)(cid:55)(cid:36)(cid:47)(cid:44)(cid:49)(cid:36)(cid:66)(cid:37)(cid:36)(cid:54)(cid:40)(cid:12) AppName. Tomcat Nginx war conf conf • Startup script features (cid:86)(cid:92)(cid:80)(cid:69)(cid:82)(cid:79)(cid:76)(cid:70)(cid:3)(cid:79)(cid:76)(cid:81)(cid:78) • Per startup log rotation for catalina.out (cid:36)(cid:83)(cid:83)(cid:79)(cid:76)(cid:70)(cid:68)(cid:87)(cid:76)(cid:82)(cid:81)(cid:3)(cid:54)(cid:83)(cid:72)(cid:70)(cid:76)(cid:393)(cid:70)(cid:3)(cid:38)(cid:82)(cid:81)(cid:393)(cid:74)(cid:88)(cid:85)(cid:68)(cid:87)(cid:76)(cid:82)(cid:81) Nginx Tomcat Startup • Smart auto-fix for incorrect JVM args conf conf Hooks • CATALINA_BASE preparation Deploy (cid:85)(cid:82)(cid:79)(cid:79)(cid:69)(cid:68)(cid:70)(cid:78) AppName. JVM Health ment • Tomcat startup status detection war Args Check (cid:83)(cid:88)(cid:69)(cid:79)(cid:76)(cid:86)(cid:75) Platform (cid:14) (cid:54)(cid:87)(cid:68)(cid:81)(cid:71)(cid:68)(cid:85)(cid:71)(cid:76)(cid:93)(cid:72)(cid:71)(cid:3)(cid:38)(cid:82)(cid:81)(cid:393)(cid:74)(cid:88)(cid:85)(cid:68)(cid:87)(cid:76)(cid:82)(cid:81)(cid:3)(cid:11)(cid:38)(cid:36)(cid:55)(cid:36)(cid:47)(cid:44)(cid:49)(cid:36)(cid:66)(cid:43)(cid:50)(cid:48)(cid:40)(cid:12) Nginx Tomcat Startup Conf Conf script May 17, 2017 © 2017 Alibaba Middleware Group 6 Monitoring, diagnostics and development tools • Tomcat Monitor (RESTful service) • Metrics for OS, JVM, Tomcat, Middleware, and application • Classloading: locate which jar file a class is loaded from • Thread: per thread CPU usage, thread state, etc. • Connector stats: tomcat thread pool statistics • Diagnostics on production environment • https://github.com/oldmanpushcart/greys-anatomy • Much easier to use than BTrace • Development tools • Package free tomcat Eclipse plugin May 17, 2017 © 2017 Alibaba Middleware Group 7 Other Usages that worth mentioning • Cluster Session Management: Taobao Session • Cookie + distributed key-value storage(Tair) • Database Connection Pool: Druid • https://github.com/alibaba/druid • Performance, Monitoring, Extensibility • Multi-tenancy • High density deploymentfor legacy web applications • Providing CPU/Memory isolation • Based on multi-tenantJVM May 17, 2017 © 2017 Alibaba Middleware Group 8 Outline • How we use Tomcat at Alibaba • High throughput web applications • The challenges • Case studies • Summary • Q & A May 17, 2017 © 2017 Alibaba Middleware Group 9 High throughput web application • Taobao API Gateway for mobile devices Shopping Cart • Interacting with hundreds of backend services • Handling millions of request per second Inventory Mobile API Order Devices Gateway Shipping … May 17, 2017 © 2017 Alibaba Middleware Group 10
Description: