Methods and apparatus for remote gateway monitoring and diagnostics

62 Pages·2014·4.02 MB·English

US008639989B1
(12) Ulllted States Patent (10) Patent N0.: US 8,639,989 B1
Sorenson, III et al. (45) Date of Patent: Jan. 28, 2014
(54) METHODS AND APPARATUS FOR REMOTE 7,752,329 B1 7/2010 Meenan et a1.
GATEWAY MONITORING AND ; ii?lghalt a1
DIAGNOSTICS 8,, 200,, 638 B1 6/2012 Zheenngg eet a1..
8,214,406 B2 7/2012 Kushwah
(75) Inventors: James Christopher Sorenson, III, 8,275,900 B2 9/2012 Meenan er 31,
Seattle, WA (US); Yun Lin, Bellevue, 8,285,967 B1 10/2012 Veeraswamy et al.
WA (Us); Ardis G‘ Maison’ Seattle’ WA 2001303011552; 21* girma et 31' 711/147
ms)’, NlI shanth Alapat"I Seattle’ WA 2002/0169827 A1 11/2002 Ulraivcsha ert a. .l .. . . . . . . . . . . . . . . . . . . . ..
(US) 2003/0131278 A1 7/2003 Fujibayashi
(73) Assignee: Amazon Technologies, Inc., Reno, NV (Continued)
(US) OTHER PUBLICATIONS
( * ) NOIiCeZ subject 10 any disclaimer, the term Ofthis Stephen Lawson, “Gluster Pushes Storage Software to VMware,
patent is extended or adjusted under 35 Amazon,” PCWorld online article, Feb. 7, 2011, all pages.
U.S.C. 154 b b 376 da s.
( ) y y (Continued)
(21) Appl.No.: 13/174,156
Primary Examiner * Joshua Lohn
(22) Filed: Jun. 30, 2011 (74) Attorney, Agent, or Firm * Robert C. Kowert;
( ) Meyertons, Hood, Kivlin, Kowert & GoetZel, RC.
51 Int. Cl.
G06F 11/00 (2006.01) (57) ABSTRACT
(52) US‘ Cl‘ Methods a aratus and com uter-accessible stora emedia
USPC ........................ .. 714/472; 714/471; 709/224 ’ PP . ’ . P . . g
(58) Field of Classi?cation Search for remotely momtonng and diagnosing storage gateways.
USPC 714/47 1 47 2_ 709 02 4 Status information may be collected locally on the gateways
See a lete sea'rcil hi'stg and uploaded to a service provider via gateway-initiated con
pp P W nections. The uploaded information may be stored to status
(56) References Cited data store(s). Status proxy(s) on the provider network may
analyZe the information in the status data store(s) for one or
U.S. PATENT DOCUMENTS more gateways to detect error conditions on individual gate
ways or patterns or error conditions on multiple gateways.
3,838,228 A 9/1974 Lee et Upon detecting an error condition on a gateway, the proxy
2%?’ gg?fsréaifélal' may alert another process, for example an administrator pro
6:477:624 B l 1 H2002 Kedem et a1‘ cess on the local network that includes the respective gateway.
6,792,540 B1 9/2004 Smith et 31, The other process may then message the gateway to address
6,880,086 B2 4/2005 Kidder et al. the condition. Information for particular gateways may be
6,925,518 B2 8/2005 Rudland et a1~ provided to clients on request. Information collected from
52213? et 31' multiple gateways may be viewed and analyZed by the service
735233286 B2 4 4/2009 Ramany et a1‘ “““““““ “ 711/170 provider to detect patterns related to gateway design.
7,574,660 B2 * 8/2009 Campbell et al. ........... .. 715/741
7,676,702 B2 3/2010 Basham et al. 35 Claims, 32 Drawing Sheets
W
storage \
gateway 5'“ "F5 18081 GNBD
M WM
1 H i
localcache storage controller 12
v cache manager 13
<“:‘/’\ virrual block in disk mapping siaging 15
l i i i
mewdaia 5.079 @ Hwk block
siorer a
client network
80
service provider storage service
50
remote daia siore
ii
US 8,639,989 B1
Page 2
(56) References Cited M. Armbrust, et al., “Above the Clouds: A Berkeley View of Cloud
computing,” Technical Report No. UCB/EECS-2009-28, University
U.S. PATENT DOCUMENTS of California at Berkley, USA, Feb. 10, 2009, all pages.
Stephanie Balaouras, “How the Cloud Will Transform Disaster
2003/0221124 A1 11/2003 Curran et al. Recovery Services,” Forrester Research, Inc., Jul. 24, 2009, all pages.
2004/0243699 A1 * 12/2004 Koclanes et al. ........... .. 709/224 U.S. Appl. No. 12/981,172, ?led Dec. 29, 2010, Amazon Technolo
2005/0013441 A1* 1/2005 Klein .......................... .. 380/278
gies, Inc., all pages.
2006/0271656 A1* 11/2006 Yagawa ...................... .. 709/223
2007/0168336 A1 7/2007 Ransil et al. U.S. Appl. No. 12/981,393, ?led Dec. 29, 2010, Amazon Technolo
2008/0013614 A1 1/2008 Fiesel et al. gies, Inc., all pages.
2008/0178278 A1* 7/2008 Grinstein et a1. ............. .. 726/12 U.S. Appl. No. 12/981,397, ?led Dec. 29, 2010, Amazon Technolo
2008/0189774 A1* 8/2008 Ansari et al. .................... .. 726/7 gies, Inc., all pages.
2009/0024752 A1 1/2009 Shitomi U.S. Appl. No. 13/212,960, ?led Aug. 18, 2011, Amazon Technolo
2009/0077257 A1* 3/2009 Savoor et a1. ............... .. 709/232
gies, Inc., all pages.
2010/0070725 A1 3/2010 Prahlad et al.
U.S. Appl. No. 13/327,605, ?led Dec. 15, 2011, Amazon Technolo
2010/0080144 A1* 4/2010 Greenberg et a1. ......... .. 370/253
2010/0162032 A1 6/2010 Dodgson et al. gies, Inc., all pages.
2010/0217948 A1 8/2010 Mason et al. U.S. Appl. No. 13/174,524, ?led Jun. 30, 2011, Amazon Technolo
2010/0293233 A1 11/2010 Salam et al. gies, Inc., all pages.
2010/0333116 A1 12/2010 Prahlad et al. U.S. Appl. No. 13/174,172, ?led Jun. 30, 2011, Amazon Technolo
2011/0002341 A1 1/2011 Damola et al.
gies, Inc., all pages.
2011/0113428 A1 5/2011 Nelson et al.
U.S. Appl. No. 13/174,140, ?led Jun. 30, 2011, Amazon Technolo
2011/0218969 A1 9/2011 Anglin et al.
2012/0324067 A1 12/2012 Hari et al. gies, Inc., all pages.
OTHER PUBLICATIONS U.S. Appl. No. 13/174,203, ?led Jun. 30, 2011, Amazon Technolo
gies, Inc., all pages.
Krishnan Subramanian, “Gluster Introduces Scale-Out NAS Virtual U.S. Appl. No. 13/174,513, ?led Jun. 30, 2011, Amazon Technolo
Storage Appliances forVMWare andAWS,” CloudAve online article, gies, Inc., all pages.
Feb. 9, 2011, all pages. U.S. Appl. No. 13/174,505, ?led Jun. 30, 2011, Amazon Technolo
A. Epstein, D. H. Lorenz, E. Silvera, I. Shapira, “Virtual Appliance gies, Inc., all pages.
Content Distribution for a Global Infrastructure Cloud Service,” U.S. Appl. No. 13/174,489, ?led Jun. 30, 2011, Amazon Technolo
INFOCOM’ 10 Proceedings IEEE, Mar. 2010, all pages. gies, Inc., all pages.
Liu, et al., “Low-cost application image distribution on Worldwide U.S. Appl. No. 13/324,907, ?led Dec. 13, 2011, Amazon Technolo
cloud front server,” Computer Communications Workshops gies, Inc., all pages.
(INFOCOM WKSHPS), 2011 IEEE Conference, Apr. 2011, all
pages. * cited by examiner
US. Patent Jan. 28, 2014 Sheet 1 of 32 US 8,639,989 B1
Service provider remote data store
@
@
console storage service
process M
@
llllll'tié?tléblélriélllll.
intermediate
network
Q
network ..... ..... client network
administrator Q
process
90 storage gateway
_ M
/ \ \
\
\
local data store customer process(es)
& @
Figure 1

US. Patent Jan. 28, 2014 Sheet 3 of 32 US 8,639,989 B1
| |
l I
l I
l remote data store l
l m l
l l
l l
l l
l l
:l servrce provrder l:
m
| |
| |
l I
l storage service |
l |
m
l |
l |
l AAzx |
l |
l |
intermediate
network
m
customer process customer process
154A 1545 ' ' '
US. Patent Jan. 28, 2014 Sheet 4 of 32 US 8,639,989 B1
local network
service provider M
m I
storage service
m
M
____________________________ ___ __________________|
download
data
\/_\\
Figure 4
/\_//
upload data
V
client network
@ service storage gateway
requests
E
(e.g., create
volume
requests)
network
administrator
process local network |0Ca| data
m E store
\ E
customer process customer process . . .
258A 2588
US. Patent Jan. 28, 2014 Sheet 5 0f 32 US 8,639,989 B1
|
I l
: |
: remote VM VM VM :
: data store m w m ' ' ' i
I m v|v||v| @ :
: it I
| host(s) _2 i
: ll :
: service provider I
| m :
l l
l |
: v V l
: t _ hardware virtualization :
I s oragzeéervlce Service :
: — E :
L ___________ __7i__3\i ____________ __7//_ ___________ __l
/ \\\ // /
intermediate
9 network
or‘ m Figure 5
\l //
H \\\ //
\\\ //
\\ i/
\\ //
storage gateway(s) \\ //
a \i //
\ \ / /
i \ \ / /
\\ / /
\\ //
\ /
\ ‘W J/
client network local network
E E
FF-_ ________________ 1 -:
virtual computing : :
|
service c|ient(s) 53155315) : i
a — Ll
US. Patent Jan. 28, 2014 Sheet 6 0f 32 US 8,639,989 B1
remote data store _ ------------------------------------
(backend store) . _ _ _ _ _ _ _ _ _ _ _ , . . .. '
m ............ ..
volume(s)
E
k service provider
' 210
storage service
m
. A
service
requests
r
service provider front end
E
————-——_;e\;_I>ad—sf;m-__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ “
g y backend backend store
management Store updates cache reads
‘7 and backend
' store updates
4 local data store
storage gateway (local cache)
E > 254
, cache —
service A writes
requests
reads from
ne_t\fvork cache and Writes
administrator backend
process Store
E
client network
customer 250
process(es)
E
Figure 6
US. Patent Jan. 28, 2014 Sheet 7 0f 32 US 8,639,989 B1
remote data store ------------------------------------------ __
A
storage service
m
A
service provider front end service
@ provider
‘‘ 210
hadowed
magnaatgeemwzal ent SW rites to Ch' em
remote data HGIWOl’k
store 250
service
requests \
writes to
local data
writes ‘ store ‘
network customer Storage |oca| data
administrator processtes) gateway Store
process E E ; E
260 reads reads
Figure 7
US. Patent Jan. 28, 2014 Sheet 8 0f 32 US 8,639,989 B1
remote data store L.
m ........s..n..a..p..s...h..0..t..(..s..).. ......... I
m
“ service
provider
21 0
shadowed \ client
WlitBS upload network
local data 250
store x
4
Storage local data
- gateway - store
customer Wmes E Wmes 254
process(es) ; —
E 4‘ reads reads
Figure 8