The Challenges Tomcat Faces in High Throughput Production Systems

48 Pages·2017·5.82 MB·English

The Challenges Tomcat Faces in
High Throughput Production Systems
May 17, 2017 © 2017 Alibaba Middleware Group 1
Biography
• Huxing Zhang (张乎兴)
• Software Engineer @ Alibaba Middleware
Group since 2014
• Previously worked for DeNA, Tokyo, Japan
• Apache Tomcat Committer since 2016
• Core tomcat maintainer @ Alibaba Inc.
May 17, 2017 © 2017 Alibaba Middleware Group 2
Outline
• How we use Tomcat at Alibaba
• High throughput web applications
• The challenges
• Case studies
• Summary
• Q & A
May 17, 2017 © 2017 Alibaba Middleware Group 3
Outline
• How we use Tomcat at Alibaba
• High throughput web applications
• The challenges
• Case studies
• Summary
• Q & A
May 17, 2017 © 2017 Alibaba Middleware Group 4
Tomcat in Alibaba
• Small trial since 2009
• Migration From JBoss to Tomcat
• Jboss uses Tomcat as its servlet container
• Officially started in 2013
• Becomes the dominant servlet container in 2015
• Thousands of web applications
• Hundreds of thousands of instances
• Status Quo
• 95% tomcat, 5% other
• Mainly 7.0.x (support JDK 6+)
• Few 8.0.x
• Emerging 8.5.x because of micro-service (e.g. spring-boot)
May 17, 2017 © 2017 Alibaba Middleware Group 5
Deployment Standard
One web application per Tomcat instance
(cid:53)(cid:88)(cid:81)(cid:87)(cid:76)(cid:80)(cid:72)(cid:3)(cid:38)(cid:82)(cid:81)(cid:393)(cid:74)(cid:88)(cid:85)(cid:68)(cid:87)(cid:76)(cid:82)(cid:81)(cid:3)(cid:11)(cid:38)(cid:36)(cid:55)(cid:36)(cid:47)(cid:44)(cid:49)(cid:36)(cid:66)(cid:37)(cid:36)(cid:54)(cid:40)(cid:12)
AppName. Tomcat Nginx
war conf conf
• Startup script features
(cid:86)(cid:92)(cid:80)(cid:69)(cid:82)(cid:79)(cid:76)(cid:70)(cid:3)(cid:79)(cid:76)(cid:81)(cid:78)
• Per startup log rotation for catalina.out
(cid:36)(cid:83)(cid:83)(cid:79)(cid:76)(cid:70)(cid:68)(cid:87)(cid:76)(cid:82)(cid:81)(cid:3)(cid:54)(cid:83)(cid:72)(cid:70)(cid:76)(cid:393)(cid:70)(cid:3)(cid:38)(cid:82)(cid:81)(cid:393)(cid:74)(cid:88)(cid:85)(cid:68)(cid:87)(cid:76)(cid:82)(cid:81)
Nginx Tomcat Startup • Smart auto-fix for incorrect JVM args
conf conf Hooks
• CATALINA_BASE preparation
Deploy
(cid:85)(cid:82)(cid:79)(cid:79)(cid:69)(cid:68)(cid:70)(cid:78)
AppName. JVM Health
ment • Tomcat startup status detection
war Args Check
(cid:83)(cid:88)(cid:69)(cid:79)(cid:76)(cid:86)(cid:75)
Platform
(cid:14)
(cid:54)(cid:87)(cid:68)(cid:81)(cid:71)(cid:68)(cid:85)(cid:71)(cid:76)(cid:93)(cid:72)(cid:71)(cid:3)(cid:38)(cid:82)(cid:81)(cid:393)(cid:74)(cid:88)(cid:85)(cid:68)(cid:87)(cid:76)(cid:82)(cid:81)(cid:3)(cid:11)(cid:38)(cid:36)(cid:55)(cid:36)(cid:47)(cid:44)(cid:49)(cid:36)(cid:66)(cid:43)(cid:50)(cid:48)(cid:40)(cid:12)
Nginx Tomcat Startup
Conf Conf script
May 17, 2017 © 2017 Alibaba Middleware Group 6
Monitoring, diagnostics and development tools
• Tomcat Monitor (RESTful service)
• Metrics for OS, JVM, Tomcat, Middleware, and application
• Classloading: locate which jar file a class is loaded from
• Thread: per thread CPU usage, thread state, etc.
• Connector stats: tomcat thread pool statistics
• Diagnostics on production environment
• https://github.com/oldmanpushcart/greys-anatomy
• Much easier to use than BTrace
• Development tools
• Package free tomcat Eclipse plugin
May 17, 2017 © 2017 Alibaba Middleware Group 7
Other Usages that worth mentioning
• Cluster Session Management: Taobao Session
• Cookie + distributed key-value storage(Tair)
• Database Connection Pool: Druid
• https://github.com/alibaba/druid
• Performance, Monitoring, Extensibility
• Multi-tenancy
• High density deploymentfor legacy web applications
• Providing CPU/Memory isolation
• Based on multi-tenantJVM
May 17, 2017 © 2017 Alibaba Middleware Group 8
Outline
• How we use Tomcat at Alibaba
• High throughput web applications
• The challenges
• Case studies
• Summary
• Q & A
May 17, 2017 © 2017 Alibaba Middleware Group 9
High throughput web application
• Taobao API Gateway for mobile devices Shopping Cart
• Interacting with hundreds of backend services
• Handling millions of request per second
Inventory
Mobile API
Order
Devices Gateway
Shipping
…
May 17, 2017 © 2017 Alibaba Middleware Group 10