Engineering Long-Lasting Software: An Agile Approach Using SaaS and Cloud Computing, Beta Edition

536 Pages·2012·9.75 MB·English

by  Patterson

Engineering Long-Lasting Software: An Agile
Approach Using SaaS and Cloud Computing
Beta Edition 0.9.0
Armando Fox and David Patterson
August 23, 2012
Copyright 2012 Strawberry Canyon LLC. All rights reserved.
No part of this book or its related materials may be reproduced in any form
without the written consent of the copyright holder.
Book version: 0.9.0
The cover background is a photo of the Aqueduct of Segovia, Spain. We
chose it as an example of a beautiful, long-lasting design. The full aqueduct is
about 20 miles (32 km) long and was built by the Romans in the 1st or 2nd
century A.D. This photo is from the half-mile (0.8 km) long, 92 feet (28 m) high
above ground segment built using unmortared, granite blocks. The Roman
designers followed the architectural principles in the ten volumes series De
Architectura (“On Architecture”), written in 15 B.C. by Marcus Vitruvius Pollio.
It was untouched until the 1500s, when King Ferdinand and Queen Isabella
performed the first reconstruction of these arches. The aqueduct was in use and
delivering water until recently.
Both the print book and ebook were prepared with LaTeX, tex4ht, and Ruby
scripts employing Nokogiri. Additional Ruby scripts automatically keep the
Pastebin excerpts and screencast URIs up-to-date in the text. The necessary
Makefiles, style files and most of the scripts are available under the BSD
License at http://github.com/armandofox/latex2ebook.
Arthur Klepchukov designed the covers and graphics for all versions.
Contents
1 Engineering Software is Different from Hardware
1.1 Introduction
1.2 Product Lifetimes: Independent Products vs. Continuous Improvement
1.3 Development Processes: Waterfall vs. Spiral vs. Agile
1.4 Assurance: Testing and Formal Methods
1.5 Productivity: Conciseness, Synthesis, Reuse, and Tools
1.6 Software as a Service
1.7 Service Oriented Architecture
1.8 Cloud Computing
1.9 Fallacies and Pitfalls
1.10 Guided Tour of the Book
1.11 How NOT to Read this Book
1.12 Concluding Remarks: Engineering Software is More Than Programming
1.13 To Learn More
1.14 Suggested Projects
2 SaaS Architecture
2.1 100,000 Feet: Client-Server Architecture
2.2 50,000 Feet: Communication—HTTP and URIs
2.3 10,000 feet: Representation—HTML and CSS
2.4 5,000 Feet: 3-Tier Architecture & Horizontal Scaling
2.5 1,000 Feet: Model-View-Controller Architecture
2.6 500 Feet: Active Record for Models
2.7 500 feet: Routes, Controllers, and REST
2.8 500 feet: Template Views
2.9 Fallacies and Pitfalls
2.10 Concluding Remarks: Patterns, Architecture, and Long-Lived APIs
2.11 To Learn More
2.12 Suggested Projects
3 Ruby for Java Programmers
3.1 Overview and Three Pillars of Ruby
3.2 Everything is an Object
3.3 Every Operation is a Method Call
3.4 Classes, Methods, and Inheritance
3.5 All Programming is Metaprogramming
3.6 Blocks: Iterators, Functional Idioms, and Closures
3.7 Mix-ins and Duck Typing
3.8 Make Your Own Iterators Using Yield
3.9 Fallacies and Pitfalls
3.10 Concluding Remarks: Idiomatic Language Use
3.11 To Learn More
3.12 Suggested Projects
4 Rails From Zero to CRUD
4.1 Rails Basics: From Zero to CRUD
4.2 Databases and Migrations
4.3 Models: Active Record Basics
4.4 Controllers and Views
4.5 Debugging: When Things Go Wrong
4.6 Form Submission: New and Create
4.7 Redirection and the Flash
4.8 Finishing CRUD: Edit/Update and Destroy
4.9 Fallacies and Pitfalls
4.10 Concluding Remarks: Designing for SOA
4.11 To Learn More
4.12 Suggested Projects
5 Validating Requirements: BDD and User Stories
5.1 Introduction to Behavior-Driven Design and User Stories
5.2 SMART User Stories
5.3 Introducing Cucumber and Capybara
5.4 Running Cucumber and Capybara
5.5 Lo-Fi User Interface Sketches and Storyboards
5.6 Enhancing Rotten Potatoes
5.7 Explicit vs. Implicit and Imperative vs. Declarative Scenarios
5.8 Fallacies and Pitfalls
5.9 Concluding Remarks: Pros and Cons of BDD
5.10 To Learn More
5.11 Suggested Projects
6 Verification: Test-Driven Development
6.1 Background: A RESTful API and a Ruby Gem
6.2 FIRST, TDD, and Getting Started With RSpec
6.3 The TDD Cycle: Red–Green–Refactor
6.4 More Controller Specs and Refactoring
6.5 Fixtures and Factories
6.6 TDD for the Model
6.7 Stubbing the Internet
6.8 Coverage Concepts and Unit vs. Integration Tests
6.9 Other Testing Approaches and Terminology
6.10 Fallacies and Pitfalls
6.11 Concluding Remarks: TDD vs. Conventional Debugging
6.12 To Learn More
6.13 Suggested Projects
7 Improving Productivity: DRY and Concise Rails
7.1 DRYing Out MVC: Partials, Validations and Filters
7.2 Single Sign-On and Third-Party Authentication
7.3 Associations and Foreign Keys
7.4 Through-Associations
7.5 RESTful Routes for Associations
7.6 Composing Queries With Reusable Scopes
7.7 Fallacies and Pitfalls
7.8 Concluding Remarks: Languages, Productivity, and Beauty
7.9 To Learn More
7.10 Suggested Projects
8 Legacy Software, Refactoring, and Agile Methods
8.1 What Makes Code “Legacy” and How Can Agile Help?
8.2 Exploring a Legacy Codebase
8.3 Establishing Ground Truth With Characterization Tests
8.4 Metrics, Code Smells, and SOFA
8.5 Method-Level Refactoring: Replacing Dependencies With Seams
8.6 Fallacies and Pitfalls
8.7 Concluding Remarks: Continuous Refactoring
8.8 To Learn More
8.9 Suggested Projects
9 Working In Teams vs. Individually
9.1 It Takes a Team: Two-Pizza and Scrum
9.2 Points, Velocity, and Pivotal Tracker
9.3 Pair Programming
9.4 Design Reviews and Code Reviews
9.5 Version Control for the Two-Pizza Team: Merge Conflicts
9.6 Using Branches Effectively
9.7 Reporting and Fixing Bugs: The Five R’s
9.8 Fallacies and Pitfalls
9.9 Concluding Remarks: Teams, Collaboration, and Four Decades of Version
Control
9.10 To Learn More
9.11 Suggested Projects
10 SOLID Design Patterns for SaaS
10.1 Patterns, Antipatterns, and SOLID Class Architecture
10.2 Just Enough UML
10.3 Single Responsibility Principle
10.4 Open/Closed Principle
10.5 Liskov Substitution Principle
10.6 Dependency Injection Principle
10.7 Demeter Principle
10.8 Fallacies and Pitfalls
10.9 Concluding Remarks: Is Agile Design an Oxymoron?
10.10 To Learn More
10.11 Suggested Projects
11 Enhancing SaaS With JavaScript
11.1 JavaScript: The Big Picture
11.2 Client-Side JavaScript for Ruby Programmers
11.3 Functions and Prototype Inheritance
11.4 The Document Object Model and jQuery
11.5 Events and Callbacks
11.6 AJAX: Asynchronous JavaScript And XML
11.7 Fallacies and Pitfalls
11.8 Concluding Remarks: JavaScript Past, Present and Future
11.9 To Learn More
11.10 Suggested Projects
12 Performance, Upgrades, Practical Security
12.1 From Development to Deployment
12.2 Quantifying Availability and Responsiveness
12.3 Continuous Integration and Continuous Deployment
12.4 Upgrades and Feature Flags
12.5 Monitoring and Finding Bottlenecks
12.6 Improving Rendering and Database Performance With Caching
12.7 Avoiding Abusive Queries
12.8 Defending Customer Data
12.9 Fallacies and Pitfalls
12.10 Concluding Remarks: Performance, Security, and Leaky Abstractions
12.11 To Learn More
12.12 Suggested Projects
13 Looking Backwards and Looking Forwards
13.1 Perspectives on SaaS and SOA
13.2 Looking Backwards
13.3 Looking Forwards
13.4 Evaluating the Book in the Classroom
13.5 Last Words
13.6 To Learn More
Appendix A Using the Bookware
A.1 Alpha Edition Guidance
A.2 Overview of the Bookware
A.3 Using the Bookware VM With VirtualBox
A.4 Using the Bookware VM on Amazon Elastic Compute Cloud
A.5 Working With Code: Editors and Unix Survival Skills
A.6 Getting Started With Git for Version Control
A.7 Getting Started With GitHub or ProjectLocker
A.8 Deploying to the Cloud Using Heroku
A.9 Fallacies and Pitfalls
A.10 To Learn More
Foreword
Warning!
This book is an opinionated path through the bewildering array of
methodologies, languages, tools, and artifact types that collectively make up
“software engineering.” The goal is to instill good software habits in students—
testability, software architecture, modularity, and reusability—while providing
them the gratification of building a working deployed artifact that they
themselves (and their peers) would use and find compelling.
This book is neither a step-by-step tutorial nor a reference book. Plenty of
both are available online and in print; check saasbook.info for some suggestions.
Instead, our goal is to bring a diverse set of topics together into a single
narrative, help you understand the most important ideas by giving concrete
examples, and teach you enough about each topic to get you started in the field.
Throughout the book we recommend dozens of other excellent books that go
into great depth about each topic; readers can decide for themselves which ones
they will find most useful.
The particular choice we make is to teach agile development using a
methodology similar to extreme programming (XP) in the context of building
and deploying a software-as-a-service (SaaS) application implemented using
Ruby on Rails. Each choice has a good reason.

Why Agile?
We use the Software Engineering Body of Knowledge (SWEBOK),
stewarded by the Institute of Electrical and Electronics Engineers, to introduce
agile ideas and methods and apply them to the creation of SaaS, which we
believe is important to the future of software. While agile might not be suitable
for building safety-critical systems, its iteration-based, short-planning-cycle
approach is a great fit for the reality of crowded undergraduate schedules and
fast-paced courses. Busy students will by nature procrastinate and then pull
several all-nighters to get a demo cobbled together and working by the project
deadline; agile not only thwarts this tactic (since students are evaluated on
progress being made each iteration) but in our experience actually leads to real
progress using responsible discipline on a more regular basis.
Within each iteration, we are able to address the major issues of the software
lifecycle in microcosm—requirements gathering with the customer, transforming
requirements to user stories, driving the class-level architecture of the software
using behavior-driven development, driving the implementation using test-
driven development, and evaluating both unit/regression test coverage and
acceptance/integration test results. That is, rather than first evaluating students
on requirements gathering, then on good design, then on development and
finally on test coverage and a working demo, all of these elements are evaluated
on every iteration, encouraging the students to see the concepts and techniques
as part of an integrated ongoing process rather than as isolated stages in a
pipeline.

Why Software as a Service?
To motivate students, it’s helpful to use a platform that lets them create
compelling apps. As of 2011, there were approximately 4.2 billion mobile
phones deployed worldwide, or enough for 3 out of every 5 people on the planet;
combined with the explosive growth of SaaS, we believe the future of software
is “client + cloud” applications that are split between a tablet or smart phone and
a cluster of servers to do heavy computation and persist data.
Therefore, both mobile applications for smart phones and tablets and
Software as a Service (SaaS) for cloud computing are compelling targets for
teaching students. As you can teach the principles with either target, given the
time constraints of a single college course, we choose in favor of the platform
with the most productive tools. Our experience is that it is no contest: the
programming and testing frameworks for SaaS and cloud computing are
dramatically more productive than those for mobile apps, and the client part of
many SaaS apps can be adapted to mobile devices using the
HTML/CSS/JavaScript skills learned in creating SaaS.
In addition, beyond the commercial promise of SaaS and the “hireability” of
students who know how to create it, SaaS projects can be deployed
inexpensively using public cloud computing, which means students’ course
artifacts are on display for the whole world to see. The exposure and “look what
I made” factor of public deployment are hard to match.