Engineering Long-Lasting Software: An Agile Approach Using SaaS and Cloud Computing, Alpha Edition

289 Pages·2012·5.92 MB·English

by  David Patterson

Engineering Long-Lasting Software: An Agile
Approach Using SaaS and Cloud Computing
Alpha Edition
Armando Fox and David Patterson
February 5, 2012
Copyright 2012 Strawberry Canyon LLC. All rights reserved.
No part of this book or its related materials may be reproduced in any form
without the written consent of the copyright holder.
Book version: 0.8.4
The cover background is a photo of the Aqueduct of Segovia, Spain. We
chose it as an example of a beautiful, long-lasting design. The full aqueduct is
about 20 miles (32 km) long and was built by the Romans in the 1st or 2nd
century A.D. This photo is from the half-mile (0.8 km) long, 92 feet (28 m) high
above ground segment built using unmortared, granite blocks. The Roman
designers followed the architectural principles in the ten volumes series De
Architectura (“On Architecture”), written in 15 B.C. by Marcus Vitruvius Pollio.
It was untouched until the 1500s, when King Ferdinand and Queen Isabella
performed the first reconstruction of these arches. The aqueduct is still in use
and delivering water today.
Both the print book and ebook were prepared with LaTeX, tex4ht, and Ruby
scripts employing Nokogiri. Additional Ruby scripts automatically keep the
Pastebin excerpts and screencast URIs up-to-date in the text.
Arthur Klepchukov designed the iOS and Android versions and provided the
cover and design graphics for all versions.
Contents
1 Engineering Software is Different from Hardware
1.1 Introduction
1.2 Product Lifetimes: Independent Products vs. Continuous Improvement
1.3 Development Processes: Waterfall versus Agile
1.4 Assurance: Testing and Formal Methods
1.5 Productivity: Conciseness, Synthesis, Reuse, and Tools
1.6 Software as a Service
1.7 Service Oriented Architecture
1.8 Cloud Computing
1.9 Fallacies and Pitfalls
1.10 Guided Tour of the Book
1.11 How NOT to Read this Book
1.12 Concluding Remarks: Engineering Software is More Than Programming
1.13 To Learn More
1.14 Exercises
2 SaaS Architecture
2.1 100,000 Feet: Client-Server Architecture
2.2 50,000 Feet: Communication—HTTP and URIs
2.3 10,000 feet: Representation—HTML and CSS
2.4 5,000 Feet: 3-Tier Architecture & Horizontal Scaling
2.5 1,000 Feet: Model-View-Controller Architecture
2.6 500 Feet: Active Record for Models
2.7 500 feet: Routes, Controllers, and REST
2.8 500 feet: Template Views
2.9 Fallacies and Pitfalls
2.10 Concluding Remarks: Patterns, Architecture, and Long-Lived APIs
2.11 To Learn More
2.12 Exercises
3 Ruby & Rails Basics
3.1 Overview and Three Pillars of Ruby
3.2 Everything is an Object
3.3 Every Operation is a Method Call
3.4 Classes, Methods, and Inheritance
3.5 All Programming is Metaprogramming
3.6 Blocks: Iterators, Functional Idioms, and Closures
3.7 Mix-ins and Duck Typing
3.8 Make Your Own Iterators Using Yield
3.9 Rails Basics: From Zero to CRUD
3.10 Databases and Migrations
3.11 Models: Active Record Basics
3.12 Controllers and Views
3.13 Debugging: When Things Go Wrong
3.14 Form Submission: New and Create
3.15 Redirection and the Flash
3.16 Finishing CRUD: Edit/Update and Destroy
3.17 Fallacies and Pitfalls
3.18 Concluding Remarks: Designing for SOA
3.19 To Learn More
3.20 Exercises
4 Validating Requirements: BDD and User Stories
4.1 Introduction to Behavior-Driven Design and User Stories
4.2 SMART User Stories
4.3 Introducing Cucumber and Capybara
4.4 Running Cucumber and Capybara
4.5 Lo-Fi User Interface Sketches and Storyboards
4.6 Enhancing Rotten Potatoes
4.7 Explicit vs. Implicit and Imperative vs. Declarative Scenarios
4.8 Fallacies and Pitfalls
4.9 Concluding Remarks: Pros and Cons of BDD
4.10 To Learn More
4.11 Exercises
5 Verification: Test-Driven Development
5.1 Background: A RESTful API and a Ruby Gem
5.2 FIRST, TDD, and Getting Started With RSpec
5.3 The TDD Cycle: Red–Green–Refactor
5.4 More Controller Specs and Refactoring
5.5 Fixtures and Factories
5.6 TDD for the Model
5.7 Stubbing the Internet
5.8 Coverage Concepts and Unit vs. Integration Tests
5.9 Other Testing Approaches and Terminology
5.10 Fallacies and Pitfalls
5.11 Concluding Remarks: TDD vs. Conventional Debugging
5.12 To Learn More
5.13 Exercises
6 Improving Productivity: DRY and Concise Rails
7 Software Maintenance: Using Agile Methods on Legacy Software
8 Working In Teams vs. Individually
9 Software Design Patterns for SaaS
10 Operations: Performance, Scaling, and Practical Security
11 Looking Backwards and Looking Forwards
11.1 Looking Backwards
11.2 Perspectives on SaaS and SOA
11.3 It Takes Time
11.4 Evaluating the Book in the Classroom
11.5 Last Words
11.6 To Learn More
Appendix A Using the Bookware
A.1 Alpha Edition Guidance
A.2 Overview of the Bookware
A.3 Using the Bookware VM With VirtualBox
A.4 Working With Code: Editors and Unix Survival Skills
A.5 Getting Started With Git for Version Control
A.6 Getting Started With GitHub or ProjectLocker
A.7 Deploying to the Cloud Using Heroku
A.8 Fallacies and Pitfalls
A.9 To Learn More
Foreword
Introduction
This book is an opinionated path through the bewildering array of
methodologies, languages, tools, and artifact types that collectively make up
“software engineering.” The goal is to instill good software habits in students—
testability, software architecture, modularity, and reusability—while providing
them the gratification of building a working deployed artifact that they
themselves (and their peers) would use and find compelling.
The particular choice we make is to teach agile development using a
methodology similar to extreme programming (XP) in the context of building
and deploying a software-as-a-service (SaaS) application implemented using
Ruby on Rails. Each choice has a good reason.

Why Agile?
We use the IEEE SWEBOK framework (Software Engineering Body of
Knowledge, stewarded by the Institute of Electrical and Electronics Engineers)
to introduce agile ideas and methods, and specifically how to use them to create
SaaS, which we believe is important to the future of software. While agile might
not be suitable for building safety-critical or lights-out systems, its iteration-
based, short-planning-cycle approach is a great fit for the reality of crowded
undergraduate schedules and fast-paced courses. Busy students will by nature
procrastinate and then pull several all-nighters to get a demo cobbled together
and working by the project deadline; agile not only thwarts this tactic (since
students are evaluated on progress being made each iteration) but in our
experience actually leads to real progress using responsible discipline on a more
regular basis.
Within each iteration, we are able to address the major issues of the software
lifecycle in microcosm—requirements gathering with the customer, transforming
requirements to user stories, driving the class-level architecture of the software
using behavior-driven development, driving the implementation using test-
driven development, and evaluating both unit/regression test coverage and
acceptance/integration test results. That is, rather than first evaluating students
on requirements gathering, then on good design, then on development and
finally on test coverage and a working demo, all of these elements are evaluated
on every iteration, encouraging the students to see the concepts and techniques
as part of an integrated ongoing process rather than as isolated stages in a
pipeline.

Why Software as a Service?
To motivate students, it’s helpful to use a platform that lets them create
compelling apps. As of 2010, there are more than 3.5 billion mobile phones
deployed worldwide, or approximately 1 phone for every other person on the
planet; combined with the explosive growth of SaaS, we believe the future of
software is “client + cloud” applications that are split between a tablet or smart
phone and a cluster of servers to do heavy computation and persist data.
Therefore, both mobile applications for smart phones and tablets and
Software as a Service (SaaS) for cloud computing are compelling targets for
teaching students. As you can teach the principles with either target, given the
time constraints of a one-semester course, we choose in favor of the platform
with the most productive tools. Our experience is that it is no contest: the
programming and testing frameworks for SaaS and cloud computing are
dramatically more productive than those for mobile apps, and the client part of
many SaaS apps can be adapted to mobile devices using the
HTML/CSS/JavaScript skills learned in creating SaaS.
In addition, beyond the commercial promise of SaaS and the “hireability” of
students who know how to create it, SaaS projects can be deployed
inexpensively using public cloud computing, which means students’ course
artifacts are on display for the whole world to see. The exposure and “look what
I made” factor of public deployment are hard to match.

Why Rails?
We want students to understand that in the real world, programmers are
rewarded not for the number of lines of code written or for how quickly they can
“bash out” a feature, but for functionality delivered with high assurance of
stability and while keeping the codebase beautiful and maintainable for
continued growth. To many students, especially “hotshot” coders who come into
a software engineering course with nontrivial programming experience, the
methodologies and techniques we use to do this—design patterns, refactoring,
test-first development, behavior-driven design—seem a strange and a dubious
use of time. We have found that students are more likely to gradually embrace
these practices if given the best possible tools to support the practices. The Rails
community has created by far the most seamless, elegant, and comprehensive
tool set to support Agile and XP, and the idea of constantly refining and
inventing tools that support testing as well as helping produce beautiful
application code is a distinguishing characteristic of the Ruby developer
ecosystem. While learning Ruby and Rails will be new to most students, juniors
and seniors seem to learn it without difficulty, and far superior tools outweigh
the learning costs.
Finally, even if our students never use Ruby again, they will have learned
how to reduce to practice such important ideas as metaprogramming, higher-
order programming, functional programming, and use of closures in the service
of higher productivity and more maintainable code. We believe these skills will
transfer to new languages, framework, and programming systems. Our early
survey of alumni of the course that led to this book (see Chapter 11) suggests
that this is happening.

History of this Book
The material in this book started as a byproduct of a Berkeley research
project that was developing technology to make it easy to build the next great
Internet service. We decided that young people were more likely to come up
with such a service, so we started teaching Berkeley undergraduates about
Software as a Service using Agile techniques in 2007. Each year the course
improved in scope and ambition, embracing the rapid improvements in the Rails
tools along the way.
Figure 1: These 12 books contain more than 5000 pages. Your authors read more than 30 books to prepare
this text. Most of these books are listed in the To Learn More sections at the end of the appropriate
chapters.