Efficient urgency-aware rate control scheme for multiple bounded flows

17 Pages·2015·1.55 MB·English

USOO8913501B2
(12) United States Patent (10) Patent N0.: US 8,913,501 B2
Branch et a]. (45) Date of Patent: Dec. 16, 2014
(54) EFFICIENT URGENCY-AWARE RATE 7,126,913 B1* 10/2006 Patel et a1. ............... .. 370/230.1
CONTROL SCHEME FOR MULTIPLE 7,289,514 B2 * 10/2007 Robotham et al. . 370/3954
7,564,790 B2 * 7/2009 Burns et al. ................. .. 370/235
BOUNDED FLOWS 7,586,848 B1 9/2009 GunduZhan
7 633 862 B2 12/2009 Nishimura
(75) Inventors: Joel W. Branch, Hamden, CT (US); ’ ’
Han Chen, White Plains, NY (Us); Hui 7,697,844 B2 4/2010 Guo et al.
Lei, Scarsdale, NY (US); Konstantin (Continued)
Shagin, Haifa (IL)
FOREIGN PATENT DOCUMENTS
(73) Assignee: International Business Machines
Corporation, Armonk’ NY (Us) WO W02007051374 A1 5/2007
( * ) Notice: Subject to any disclaimer, the term of this OTHER PUBLICATIONS
patent is extended or adjusted under 35 “ _ _ _
U'S'C~ 1 5 4 (b) by 187 days' IBM Traf?c Shaping Us1ng Pause Frame wrth Token Backet
Policer,” IPCOM000155803D (Published Jul. 20, 2007).
(21) Appl. No.: 13/587,466 (Continued)
(22) Filed: Aug. 16, 2012
(65) Prior Publication Data Prlmary Exammer T Chl H Pham
Us 2014/0050094 Al F b 20 2014 Assistant Examiner * Robert Lopata
e . , (74) Attorney, Agent, or Firm * Preston Young; M1c_ hael J.
(51) Int. Cl. Chang, LLC
H04L 12/24 (2006.01)
H04L 12/80] (2013.01)
(52) us CL (57) ABSTRACT
CPC H04L 41/5019 (2013 01) H0;1614;7(/)110 A h df 11 g H f 1 p1 d H
"""""" " ' ; met 0 or contro in a ow rate 0 mu ti e ata ows at
USPC 3(70/25 1) a network node on a path of the data ?ows includes the
58 F, 1d "" """ """""""" " ' following steps. A private restriction token bucket (RTB) and
( ) CISC 0 HOZSEI47C715911HSZ£C4W215 H 0 4L 47 /2 4 41_ a private guarantee token bucket (GTB) for each of the data
H04L 47’/22_ H04L 47/’21_ H04L 47/20f ?ows and a shared token bucket (STB) common to all of the
’H04L 47/14 ’H04L 41/5019’ data ?ows are provided. 11 tokens are obtained from the RTBi
USPC 376/229 235 235 1 for the data ?ow i when a message belonging to the data ?ow
See search histonl ' i arrives at the node and needs 11 tokens.An attempt is made to
obtain 11 tokens from the GTBi for the data ?ow i and/ or the
(56) References Cited STB. The message is transmitted if 11 tokens are obtained from
the GTBi and/or from the STB, otherwise transmission of the
US. PATENT DOCUMENTS message is delayed until 11 tokens are available in the GTBi
and/or in the STB.
6,147,970 A * 11/2000 Troxel ......................... .. 370/235
6,636,480 B1* 10/2003 Walia et a1. ................. .. 370/229
6,862,265 B1 3/2005 Appalaet al.
6,901,050 B1 5/2005 Acharya 24 Claims, 7 Drawing Sheets
2:12 mEMcEK AWVEAIMLAEBRLE n] N TOmKEN S
mm mm [n-q-s)
mms Ans mum
m cm, AND/0R sra
US 8,913,501 B2
Page 2
(56) References Cited 2011/0235509 A1* 9/2011 Szymanski ................. .. 370/230
OTHER PUBLICATIONS
U.S. PATENT DOCUMENTS
Liu et al., “Burst Reduction Properties of Rate-Control Throttles:
7,719,968 5/2010 Swenson et al.
Downstream Queue Behavior,” IEEE/ACM Transactions on Net
7,826,358 11/2010 Caram ........................ .. 370/230 working, vol. 3, No. 1, Feb. 1995.
2003/0069970 4/2003 Kim et al. . 709/225
2007/0070895 3/2007 Sohraby et al.,“On the Performance of Bursty and Modulated
2007/0156928 7/2007 NRaagrhvuanna n.d...a .n. ............. ... 370790//225310 Sources Subject to Leaky Bucket Rate-Based Access Control
2009/0323525 12/2009 Chen et al. Schemes,” IEEE Transactions on Communications, vol. 42, No. 2/3/
2010/0195504 8/2010 Nandagopal et al. 4, Feb./Mar./Apr. 1994.
2010/0211644 8/2010 Lavoie et al. Aeron, A.; “Fine Tuning of Fuzzy Token Bucket Scheme for Con
2010/0246596 9/2010 Nakamura et al. gestion Control in High Speed Networks,” 2010 Second International
2010/0329118 12/2010 Adams Conference on Computer Engineering and Applications (Apr. 2010).
2011/0007687 1/2011 Howe et al.
2011/0019571 1/2011 Chen et al. * cited by examiner
US. Patent Dec. 16, 2014 Sheet 1 017 US 8,913,501 B2
FIG. 1
TOKEN BUCKET m
GUARANTEE c,
RESTRICTION R,
SHARED 0-20,-1r
* THE SHARED TOKEN BUCKET REFILL RATE MAY BE GREATER THAN
C—Z'GI- SINCE THE EXTRA TOKENS THAT DO NOT FIT INTO THE
GUARANTEE TOKEN BUCKET ARE PLACED THERE.
US. Patent Dec. 16, 2014 Sheet 2 0f7 US 8,913,501 B2
FIG. 2
CHECK WHETHER n TOKENS M
202 w
ARE AVAILABLE IN RTBi
/ 206
WAIT UNTIL n TOKENS
ARE AVAILABLE IN RTBi
YES
T
208“ GET n TOKENS FROM RTBi
T
CHECK WHETHER n TOKENS
210w
ARE AVAILABLE IN GTBi
/ 214
g TOKENS
212 AVAILABLE GET n TOKENS FROM GTE;|
g < n
216w GET 9 TOKENS FROM GTBI
L
CHECK WHETHER (n-g)
218“ TOKENS ARE AVAILABLE IN STB
/ 222
220 s TOKENS GET (n-g) 7
AVAILABLE TOKENS FROM STB
s < n-g
224 w GET 3 TOKENS FROM STB
L
WAIT UNTIL (n-g-s)
226w TOKENS ARE AVAILABLE
IN GTGI AND/0R STB
L
228, GET (n—g—s) TOKENS FROM
GTRi AND/0R STB
230% FORWARD MESSAGE
US. Patent Dec. 16, 2014 Sheet 3 0f7 US 8,913,501 B2
FIG. 3
m
RTBi GTBi STB
FLOW1 - E
- I
l
- g - I
FLOW 2 u I l G)
—> —|-> —|—> —|—>
- i =
g - SERVICE
1. :
FLOW 3 - I
US. Patent Dec. 16, 2014 Sheet 4 0f7 US 8,913,501 B2
402, CHECK WHETHER n TOKENS FI G. 4
ARE AVAILABLE IN RTB| m
f 405
WAIT UNTIL n TOKENS
ARE AVAILABLE IN RTBi
408*” GET IT TOKENS FROM RTBi
I
CHECK WHETHER n TOKENS
ARE AVAILABLE IN GTBi
[414
GET n TOKENS ERoW 0TBi
416w GET g TOKENS FROM BTBi
I
418, CHECK WHETHER (n—g)
TOKENS ARE AVAILABLE IN 5m
[42
GET (n—g)
420 s TOKENS AVAILABLE TOKENS FROM STB
s<n—g
4244 GET s TOKENS FROM STB |
I
CHECK WHETHER (n—g-s)
426w TOKENS CAN BE REASSIGNED
FROM GTBj (j==i)
[430
428 rln'g's GET (n—g—s)
REASSIGNED TOKENS
r < n—g—s
432A GET r REASSIGNED TOKENS |
I fus
WAIT UNTIL (n—g—s—r) TOKENS GET (n—g—s—r) TOKENS
434, ARE AVAILABLE IN cTBI AND/OR FROM GTB AND/0R STB
sTB AND/0R CAN BE AND/0R RLASSIGN FROM
REASSIGNED FROM 0TB,- (pi) 0TB,- (jati)
FORWARD
438“ MESSAGE
US. Patent Dec. 16, 2014 Sheet 5 0f7 US 8,913,501 B2
FIG. 5
m
_____//____________
GTBi STB
H
__1_
|
_ i I -
FLowzl IQ
—:-> —l—> —:—>
i i
- SERVICE
US. Patent Dec. 16, 2014 Sheet 6 0f7 US 8,913,501 B2
CHECK WHETHER n TOKENS 6 [608
ARE AVAILABLE IN RTBi — 606 WAIT UNTIL n
604 TOKENS ARE
n TOKENS AVAILABLE AVAILABLE
(INCLUDING
OVERDRAFT)
YES
/ 610 IN RTE;i
GET IT TOKENS TRoN RTBi
I
CHECK WHETHER n TOKENS
ARE AVAILABLE IN GTBi
614
A TOKENS AVAILABLE _ GET n TOKENS FROM GTBi
GET 9 TOKENS FROM GTBi ~/‘(518
CHECK WHETHER (n—g)
TOKENS ARE AVAILABLE IN STB
f 624
GET (n—g)
TOKENS FROM STB
s<n—g
| GET s TOKENS FROM STB |-/~626
I
CHECK WHETHER (n—g-s)
TOKENS CAN BE REASSIGNED
FROM GTB (jri) AND/0R wszs
OVERDRAFT FROM GTETi AND/0R
OVERDRAFT FROM STB
630 / 632
rgn
r TOKENS AVAILABLE GET (n—g—s) TOKENS =
r< n—g-s
| GET r TOKENS L» 634
I f 638
WAIT UNTIL (n-g—s-r) TOKENS _ _ _
ARE AVAILABLE (INCLUDING EEBM("GTQB_SANFI)J/TOORKESNT$B
OVERDRAFT) IN GTB- AND/OR |
5TB AND/0R CA'N BE AND/OR REASSIGN FROM
REASSIGNED FROM GTBj (jii) GTBj (j?)
I 640
636]
FORWARD MESSAGE
US. Patent Dec. 16, 2014 Sheet 7 0f7 US 8,913,501 B2
FIG. 7
m
l l |
I I I
RTBI I GTBI I STB I
l l |
FLOW1 “ I I I
' T“ ' I
I \I\ I
FLOW2 u I I I @
—> —|—> —I—>
- |
1 J ~ J - SERVICE
FIG. 8
java
[810
[825
TO FROM
szow PROCESSOR NETWORK l/F I ~ COIIPUTER
NETWORK
ssow MEMORY MEDIA I/F MEDIA
\835 \850
840%: DISPLAY i
US 8,913,501 B2
1 2
EFFICIENT URGENCY-AWARE RATE for controlling a ?ow rate of multiple data ?ows at a network
CONTROL SCHEME FOR MULTIPLE node on a path of the data ?ows is provided. The method
BOUNDED FLOWS includes the following steps. A private restriction token
bucket for each of the data ?ows, a private guarantee token
FIELD OF THE INVENTION bucket for each of the data ?ows and a shared token bucket
common to all of the data ?ows are provided, wherein the
The present invention relates to data ?ow rate control tech restriction token bucket for a given one of the data ?ows, data
niques and more particularly, to data ?ow rate control tech ?ow i, has a re?ll rate Rl- which is equal to a maximum ?ow
niques that satisfy guarantee and restriction requirements of rate for the data ?ow i and the guarantee token bucket for the
multiple data ?ows at a network node. data ?ow i has a re?ll rate Gl- which is equal to a minimum ?ow
rate for the data ?ow i. n number of tokens is obtained from
BACKGROUND OF THE INVENTION the restriction token bucket for the data ?ow i when a message
belonging to the data ?ow i arrives at the node and needs the
A ?ow is a sequence of messages that share characteristics, n number of tokens to pass through the node. An attempt is
including quality-of-service (QoS) properties. Network ?ows made to obtain the n number of tokens from one or more of the
have to be regulated in order to ensure that the data transmis guarantee token bucket for the data ?ow i and the shared token
sions conform to de?ned bandwidth limits. bucket (e.g., if the guarantee token bucket for the data ?ow i
One approach used to regulate network ?ow is the token is empty or contains less than the n number of tokens). The
bucket technique which is a common algorithm used to con message is transmitted if the n number of tokens is obtained
trol the amount of data that is injected into a network, allow 20 from one or more of the guarantee token bucket for the data
ing for bursts of data to be sent. In general, the token bucket ?ow i and from the shared token bucket, otherwise transmis
concept is based on an analogy of a bucket that contains sion of the message is delayed until the n number of tokens is
tokens. The tokens represent a certain denomination related available in one or more of the guarantee token bucket for the
to data transmission, such as data packet size. Thus, using this data ?ow i and in the shared token bucket. A destination of the
analogy, when a data packet of a certain size is to be trans 25 data ?ows can be a web-based service having a service capac
mitted, the bucket is inspected to see if there are enough ity C equivalent to a maximal rate of incoming requests, and
tokens in the bucket to cover that size data packet. If so, the the shared token bucket can have a re?ll rate equal to C—ZGZ
appropriate number of tokens is removed from the bucket plus tokens, if any, spilled from the guarantee token bucket for
(also referred to as “cashing in the tokens”) and the data the data ?ow i.
packet is transmitted. On the other hand, if an insuf?cient 30 Tokens can be reassigned from the guarantee token bucket
number of tokens is available, then transmittal of the data of one of the data ?ows to the guarantee token bucket of
packet is canceled or held back until a suf?cient number of another of the data flows. Further, when the message is urgent
tokens is available. and at least one of the restriction token bucket for the data
The token bucket concept is employed in several works. ?ow i, the guarantee token bucket for the data ?ow i and the
See, for example, US. Pat. No. 6,901,050 issued to Acharya, 35 shared token bucket is empty, overdrafting of tokens from the
entitled “System and Methods for Flow-based Tra?ic Shap restriction token bucket for the data ?ow i, the guarantee
ing,” (which describes a system with multiple token buckets, token bucket for the data ?ow i and the shared token bucket
each handling its ?ow), US. Pat. No. 7,719,968 issued to can be permitted.
Swenson et al., entitled “Multi-Priority Multi-Color Markers A more complete understanding of the present invention,
for Traf?c Metering,” (which describes a system that contains 40 as well as further features and advantages of the present
token buckets with run time interdependencies), US. Pat. No. invention, will be obtained by reference to the following
7,586,848 issued to Gunduzhan, entitled “Elastic Tra?ic detailed description and drawings.
Marking for Multi-Priority Packet Streams in a Communica
tions Network.” (which is a scheme based on token buckets BRIEF DESCRIPTION OF THE DRAWINGS
that incorporates bandwidth loans), US. Pat. No. 6,862,265 45
issued to Appala et al., entitled “Weighted Fair Queuing FIG. 1 is a diagram illustrating token bucket re?ll rates
Approximation in a Network Switch Using Weighted Round according to an embodiment of the present invention;
Robin and Token Bucket Filter,” (which combines weighted FIG. 2 is a diagram illustrating an exemplary methodology
round robin scheduler with token bucket ?lters), and US. Pat. for controlling a ?ow rate of multiple data ?ows according to
No. 6,147,970 issued to Troxel, entitled “Quality of Service 50 an embodiment of the present invention;
Management for Aggregated Flows in a Network System,” FIG. 3 is a schematic diagram illustrating the methodology
(which incorporates two levels of token buckets with the goal of FIG. 2 according to an embodiment of the present inven
of maximizing throughput). tion;
These approaches however deal only with restricting data FIG. 4 is a diagram illustrating an exemplary methodology
?ows and do not take into consideration the content of the 55 for controlling a ?ow rate of multiple data ?ows that employs
data. For instance, the situation may arise where an urgent token reassignment according to an embodiment of the
message needs to be transmitted, but the respective token present invention;
bucket is empty. With these conventional approaches there is FIG. 5 is a schematic diagram illustrating the methodology
a risk that the urgent message might not be transmitted. of FIG. 4 according to an embodiment of the present inven
Thus, improved techniques for regulating network data 60 tion;
?ows would be desirable. FIG. 6 is a diagram illustrating an exemplary methodology
for controlling a ?ow rate of multiple data ?ows when an
SUMMARY OF THE INVENTION urgent message is received according to an embodiment of
the present invention;
The present invention provides data ?ow rate control tech 65 FIG. 7 is a schematic diagram illustrating the methodology
niques that satisfy guarantee and restriction requirements of of FIG. 6 according to an embodiment of the present inven
multiple data ?ows. In one aspect of the invention, a method tion; and