Trellis and Turbo Coding: Iterative and Graph-Based Error Control Coding

517 Pages·2015·10.904 MB·English

by  Christian B. Schlegel, Lance C. Perez

TRELLIS AND
TURBO CODING
IEEE Press
445 Hoes Lane
Piscataway, NJ 08854
IEEE Press Editorial Board
Tariq Samad, Editor in Chief
George W. Arnold Vladimir Lumelsky Linda Shafer
Dmitry Goldgof Pui-In Mak Zidong Wang
Ekram Hossain Jeffrey Nanzer MengChu Zhou
Mary Lanzerotti Ray Perez George Zobrist
Kenneth Moore, Director of IEEE Book and Information Services (BIS)
Technical Reviewers
Todd Moon, Utah State University
Claudio Sacchi, Utah State University
TRELLIS AND
TURBO CODING
Iterative and Graph-Based Error Control Coding
Second Edition
CHRISTIAN B. SCHLEGEL
LANCE C. PÉREZ
IEEE Press
Copyright © 2015 by The Institute of Electrical and Electronics Engineers, Inc.
Published by John Wiley & Sons, Inc., Hoboken, New Jersey. All rights reserved.
Published simultaneously in Canada.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any
form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise,
except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without
either the prior written permission of the Publisher, or authorization through payment of the
appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers,
MA 01923, [phone], fax [phone], or on the web at www.copyright.com. Requests to
the Publisher for permission should be addressed to the Permissions Department, John Wiley &
Sons, Inc., 111 River Street, Hoboken, NJ 07030, [phone], fax [phone], or online at
http://www.wiley.com/go/permission.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best
efforts in preparing this book, they make no representations or warranties with respect to the
accuracy or completeness of the contents of this book and specifically disclaim any implied
warranties of merchantability or fitness for a particular purpose. No warranty may be created or
extended by sales representatives or written sales materials. The advice and strategies contained
herein may not be suitable for your situation. You should consult with a professional where
appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other
commercial damages, including but not limited to special, incidental, consequential, or other
damages.
For general information on our other products and services or for technical support, please
contact our Customer Care Department within the United States at [phone], outside the
United States at [phone] or fax [phone].
Wiley also publishes its books in a variety of electronic formats. Some content that appears in
print may not be available in electronic format. For information about Wiley products, visit our
web site at www.wiley.com.
Library of Congress Cataloging-in-Publication is available.
ISBN 978-1-118-083161
Printed in the United States of America.
10 9 8 7 6 5 4 3 2 1
Contents
1 Introduction 1
1.1 Modern Digital Communications . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 The Rise of Digital Communications . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Communication Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Error Control Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.5 Bandwidth, Power, and Complexity . . . . . . . . . . . . . . . . . . . . . . 12
1.6 A Brief History–The Drive Towards Capacity . . . . . . . . . . . . . . . . . 20
2 Communications Basics 27
2.1 The Probabilistic Viewpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2 Vector Communication Channels . . . . . . . . . . . . . . . . . . . . . . . . 29
2.3 Optimum Receivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.4 Matched Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.5 Message Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.6 The Complex Equivalent Baseband Model . . . . . . . . . . . . . . . . . . . 39
2.7 Spectral Behavior. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.8 Advanced Modulation Methods . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.8.1 OFDM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.8.2 Multiple Antenna Channels (MIMO Channels) . . . . . . . . . . . . 48
2.9 A Communications System Case Study. . . . . . . . . . . . . . . . . . . . . 53
2.10 Appendix 2.A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3 Trellis-Coded Modulation 67
3.1 An Introductory Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.2 Construction of Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.3 Lattices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.4 Lattice Formulation of Trellis Codes . . . . . . . . . . . . . . . . . . . . . . 86
3.5 Rotational Invariance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.6 V.fast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
v
vivi CONTENTS
3.7 The IEEE 802.3an Standard . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
3.8 Historical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
4 Trellis Representations 111
4.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.2 The Parity-Check Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
4.3 Parity-Check Trellis Representations . . . . . . . . . . . . . . . . . . . . . . 113
4.4 Convolutional Codes and Their Trellis . . . . . . . . . . . . . . . . . . . . . 115
4.5 Minimal Trellises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
4.6 Minimum-Span Generator Matrices . . . . . . . . . . . . . . . . . . . . . . . 124
4.7 Systematic Construction of the PC-Trellis . . . . . . . . . . . . . . . . . . . 127
4.8 Tail-Biting Trellises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
4.9 The Minimal Trellis of Convolutional Codes . . . . . . . . . . . . . . . . . . 133
4.10 Fundamental Theorems from Basic Algebra . . . . . . . . . . . . . . . . . . 139
4.11 Systematic Encoders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
4.12 Maximum Free-Distance Convolutional Codes . . . . . . . . . . . . . . . . . 151
4.13 The Squaring Construction and the Trellis of Lattices . . . . . . . . . . . . 154
4.14 The Construction of Reed–Muller Codes . . . . . . . . . . . . . . . . . . . . 161
4.15 A Decoding Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
4.16 Polar Codes and Their Relationship to RM Codes . . . . . . . . . . . . . . 166
Appendix 4.A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
5 Trellis and Tree Decoding 179
5.1 Background and Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 179
5.2 Tree Decoders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
5.3 The Stack Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
5.4 The Fano Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
5.5 The M-Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
5.6 Maximum Likelihood Decoding . . . . . . . . . . . . . . . . . . . . . . . . . 197
5.7 A Posteriori Probability Symbol Decoding . . . . . . . . . . . . . . . . . . . 200
5.8 Log-APP and Approximations . . . . . . . . . . . . . . . . . . . . . . . . . 207
5.9 Error Analysis and Distance Spectrum . . . . . . . . . . . . . . . . . . . . . 211
5.10 Random Coding Analysis of Optimal Decoding . . . . . . . . . . . . . . . . 222
5.11 Random Coding Analysis of Sequential Decoding . . . . . . . . . . . . . . . 232
5.12 Some Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
6 Low-Density Parity-Check Codes 249
6.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
6.2 LDPC Codes and Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
6.3 LDPC Decoding via Message Passing. . . . . . . . . . . . . . . . . . . . . . 255
CONTENTS vvii
6.4 Analysis Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
6.4.1 (Error) Probability Evolution for Binary Erasure Channels . . . . . 259
6.4.2 Error Mechanism of LDPCs on BECs . . . . . . . . . . . . . . . . . 265
6.4.3 Binary Symmetric Channels and the Gallager Algorithms . . . . . . 266
6.4.4 The AWGN Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
6.5 Code Families and Construction . . . . . . . . . . . . . . . . . . . . . . . . 281
6.5.1 Constructions with Permutation Matrices . . . . . . . . . . . . . . . 281
6.5.2 Cycle Reduction Design . . . . . . . . . . . . . . . . . . . . . . . . . 286
6.5.3 RS-based Construction. . . . . . . . . . . . . . . . . . . . . . . . . . 287
6.5.4 Repeat-Accumulate Codes . . . . . . . . . . . . . . . . . . . . . . . . 289
6.6 Encoding of LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
6.6.1 Triangular LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . . 292
6.6.2 Specialized LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . 295
6.6.3 Approximate Triangularization . . . . . . . . . . . . . . . . . . . . . 296
Appendix 6.A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
7 Error Floors 319
7.1 The Error Floor Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
7.2 Dynamics of the Absorption Sets . . . . . . . . . . . . . . . . . . . . . . . . 323
7.3 Code Design for Low Error Floors . . . . . . . . . . . . . . . . . . . . . . . 331
7.4 Impact of the Decoding Algorithm . . . . . . . . . . . . . . . . . . . . . . . 335
7.5 Importance Sampling (IS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
7.6 Computing Error Rates via Importance Sampling . . . . . . . . . . . . . . . 340
8 Turbo Coding: Basic Principles 351
8.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
8.2 Parallel Concatenated Convolutional Codes . . . . . . . . . . . . . . . . . . 353
8.3 Distance Spectrum Analysis of Turbo Codes . . . . . . . . . . . . . . . . . . 356
8.4 The Free Distance of a Turbo Code . . . . . . . . . . . . . . . . . . . . . . . 358
8.5 Weight Enumerator Analysis of Turbo Codes . . . . . . . . . . . . . . . . . 364
8.6 Iterative Decoding of Turbo Codes . . . . . . . . . . . . . . . . . . . . . . . 371
8.7 EXIT Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
8.8 Serial Concatenation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
8.9 Cascaded Convolutional Codes . . . . . . . . . . . . . . . . . . . . . . . . . 383
8.10 Weight Enumerator Analysis of SCCCs . . . . . . . . . . . . . . . . . . . . 385
8.11 Iterative Decoding and Performance of SCCCs . . . . . . . . . . . . . . . . 394
8.12 EXIT Analysis of Serially Concatenated Codes . . . . . . . . . . . . . . . . 397
8.13 Viewpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
8.14 Turbo-Trellis-Coded Modulation . . . . . . . . . . . . . . . . . . . . . . . . 402
8.15 Serial Concatenation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406
vviiii CONTENTS
8.16 EXIT Analysis of Serial TTCM . . . . . . . . . . . . . . . . . . . . . . . . . 408
8.17 Differential-Coded Modulation . . . . . . . . . . . . . . . . . . . . . . . . . 409
8.18 Concatenated Space–Time Coding . . . . . . . . . . . . . . . . . . . . . . . 414
8.19 Bit-Interleaved Coded and Generalized Modulation . . . . . . . . . . . . . . 418
9 Turbo Coding: Applications 431
9.1 Interleavers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
9.2 Turbo Codes in Telecommunication Standards . . . . . . . . . . . . . . . . 439
9.2.1 The Space Data System Standard . . . . . . . . . . . . . . . . . . . 439
9.2.2 3G Wireless Standards . . . . . . . . . . . . . . . . . . . . . . . . . . 440
9.2.3 Digital Video Broadcast Standards . . . . . . . . . . . . . . . . . . . 443
9.3 Product Codes and Block Turbo Decoding . . . . . . . . . . . . . . . . . . . 446
9.4 Approximate APP Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . 448
9.5 Product Codes with High-Order Modulations . . . . . . . . . . . . . . . . . 451
9.6 The IEEE 802.16 Standard . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
9.7 Decoding of Polar Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454
9.8 Polar Code Performance and Outlook . . . . . . . . . . . . . . . . . . . . . 458
10 Convolutional LDPC Codes and Spatial Coupling 465
10.1 Capacity: The Ultimate Limit. . . . . . . . . . . . . . . . . . . . . . . . . . 465
10.2 Low-Density Parity-Check Convolutional Codes . . . . . . . . . . . . . . . . 467
10.2.1 New LDPC Codes from Old . . . . . . . . . . . . . . . . . . . . . . . 467
10.2.2 Decoding Convolutional LDPC Codes . . . . . . . . . . . . . . . . . 472
10.3 Spatial Coupling: A General View . . . . . . . . . . . . . . . . . . . . . . . 474
10.4 Spatial Coupling: Convergence Analysis . . . . . . . . . . . . . . . . . . . . 482
10.4.1 Problem Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482
10.4.2 Lyapunov Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 483
Trellis And Turbo Coding: Iterative and Graph-Based Error Control Coding, Second Edition
By Christian B. Schlegel and Lance C. Pérez
Copyright © 2015 by The Institute of Electrical and Electronics Engineers, Inc.
Chapter 1
Introduction
1.1 Modern Digital Communications
Withtheadventofhigh-speedlogiccircuitsandverylargescaleintegration(VLSI),data
processing and storage equipment has inexorably moved towards employing digital tech-
niques. In digital systems, data is encoded into strings of zeros and ones, corresponding
to the on and off states of semiconductor switches. This has brought about fundamental
changes in how information is processed. While real-world data is primarily in “analog
form” of one type or another, the revolution in digital processing means that this analog
informationneedstobeencodedintoadigitalrepresentation,e.g.,intoastringofonesand
zeros. The conversion from analog to digital and back are processes which have become
ubiquitous. Examplesarethedigitalencodingofspeech,picture,andvideoencodingand
rendering, as well as the large variety of capturing and representing data encountered in
our modern internet-based lifestyles.
Themigrationfromanalogcommunicationsofthefirsthalfofthe20-thcenturytothe
nowubiquitousdigitalformsofcommunicationswereenabledprimarilybythefast-paced
advancesinhigh-densitydeviceintegration. Thishasbeentheenginebehindmuchofthe
technologicalprogressoverthelasthalfcentury,initiatedbythecreationofthefirstinte-
gratedcircuit(IC)byKilbyatTexasInstrumentsin1958. FollowingMoore’sinformallaw,
devicesizes,primarilyCMOS(ComplementaryMetal-OxideSemiconductors),shrinkbya
factor two every two years, and computational power doubles accordingly. An impression
forthisexponentialgrowthincomputingcapabilitycanbegainedfromFigure1.1,which
showsthenumberoftransistorsintegratedinasinglecircuitandtheminimumdevicesize
for progressive fabrication processes – known as implementation nodes.
While straightforward miniaturization of the CMOS devices is becoming increasingly
more difficult, transistor designers have been very creative in modifying the designs to
stay on the Moore trajectory. As of 2015 we now see the introduction of 3-dimensional
1