CONSTELLATION SHAPING, NONLINEAR PRECODING, AND TRELLIS CODING FOR VOICEBAND TELEPHONE CHANNEL MODEMS with Emphasis on ITU-T Recommendation V.34 THE KLUWER INTERNATIONAL SERIES IN ENGINEERING AND COMPUTER SCIENCE CONSTELLATION SHAPING, NONLINEAR PRECODING, AND TRELLIS CODING FOR VOICEBAND TELEPHONE CHANNEL MODEMS with Emphasis on ITU-T Recommendation V.34 Steven A. Tretter University ofM aryland, U.S.A. SPRINGER SCIENCE+BUSINESS MEDIA, LLC Library of Congress Cataloging-in-Publication Data Tretter, Steven A. Constellation shaping, nonlinear precoding, and trellis coding for voiceband telephone Channel modems with emphasis on lTU-T recommendation V.34 / Steven A. Tretter. p. cm-{The Kluwer international series in engineering and computer science; SECS 673) lncludes bibliographical references and index. ISBN 978-1-4613-5339-3 ISBN 978-1-4615-0989-9 (eBook) DOI 10.1007/978-1-4615-0989-9 1. Modems. 2. Lattice theory. 3. Coding theory. 4. Trellis-coded modulation. 1. Title. II. Series. TK7887.8.M63 T74 2002 62 1.385' l-dc2 1 2002016134 Copyright © 2002 bYSpringer Science+Business Media New York Originally published by Kluwer Academic Publishers in 2002 Softcover reprint of the hardcover 1s t edition 2002 AH rights reserved. No part ofthis work may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording, or otherwise, without the writtenrmission from the Publisher, with the exception of any material supplied specifically for the purpose ofbeing entered and executed on a computer system, for exclusive use by the purchaser of the work. Printed on acid-free paper. Contents Preface xi 1. BASICS OFLATIICETHEORY 1 1.1. DefinitionofaLattice 1 1.2. ExamplesofLattices 2 1.3. Sublattices. LatticePartitions, andCosets 7 1.4. Binary Lattices andCosetRepresentatives 11 1.5. FundamentalRegionsand Volumes, andVoronoi Regions 15 1.5.1 Formulafor theFundamentalVolume 17 1.5.2 LinearTransformationsandthe FundamentalVolume 18 1.5.3 FundamentalVolumeofaSublattice 18 1.6. PointSpacing,WeightDistributions, andThetaSeries 19 1.7. FundamentalCodingGain 22 2. PERFORMANCEMEASURESFORMULTIDIMENSIONAL CONSTELLATIONS 25 2.1. Introduction 25 2.2. ConstellationFigureofMeritandSymbolErrorProbabilities 27 2.2.1 NormalizedBitRateand AveragePower 28 2.2.2 DefinitionofCFMandExamples 28 EXAMPLE2.1 One-DimensionalPAMConstellation 29 EXAMPLE2.2 M x M SquareGrid 31 EXAMPLE2.3 N-CubeGrid 33 2.2.3 AnApproximationtotheSymbolErrorProbabilityfor Large SquareQAMConstellationsatHigh SNR 34 2.2.4 TheContinuousApproximation 35 2.3. Constituent2DConstellationsandConstellationExpansionRatio 36 vi CONTENTS 2.4. Peak-to-AveragePowerRatio 37 2.4.1 PARfor theM x M SquareGridandN-CubeGrid 37 2.4.2 PARfor aCircle 38 2.4.3 PARforthe N-Sphere 40 2.5. RepresentingCFM(C) in TermsofCoding Gain and Shaping Gain 41 2.5.1 Why,c(A) isCalledtheFundamentalCodingGain 43 2.5.2 ShapingGain Properties andExamples 44 2.5.3 Ultimate ShapingGainand2DDistribution 47 2.6. CodingandShapingFactorsoftheConstellationExpansionRatio 49 2.7. Factorsofthe Peak-to-AveragePowerRatio 53 2.8. OptimumTradeoffsofShapingGain withCER and PAR 55 s 3. PRINCIPLES OFCONVOLUTIONALANDTRELLIS CODES 61 3.1. TheHuffmanD-Transform 61 3.1.1 Two-SidedTransformofaDelayedSequence 62 3.1.2 One-SidedTransformofaDelayedSequence 63 3.1.3 D-TransformofaConvolution 64 3.2. TransferFunctions and Realizations 64 3.2.1 Type 1DirectFormRealization 65 3.2.2 Type2DirectFormRealization 66 3.3. DescriptionofaConvolutionalCodebyits GeneratorMatrix 67 3.4. SystematicFormofaConvolutionalCode 69 3.5. TheParityCheckMatrix andSyndromes 71 3.6. InverseCheckMatrixorInverseSyndromeFormer 73 3.7. TheCodeTrellis 75 3.8. WeightDistributions andErrorCorrectionProperties 76 3.9. Trellis CodedModulation(TCM) 79 3.10. BriefReview ofthe ViterbiDecodingAlgorithm 85 3.11. TheFundamentalCodingGainofaTrellis Code 89 4. TRELLIS SHAPING 91 4.1. Trellis ShapingBasedonLatticePartitions 92 4.1.1 TheTrellis ShapingEncoder 92 4.1.2 TheReceiver 97 4.1.3 SelectionofaSpecificConstellation 97 4.2. Trellis ShapingonRegions 106 CONTENTS vii 4.2.1 EssentialPropertiesofTrellisShapingBasedonLattice Partitions 107 4.2.2 TheTrellis ShapingEncoderfor ShapingonRegions 109 4.2.3 TheReceiverfor ShapingonRegions 112 4.2.4 Peak-to-AverageRatioConsiderations 112 4.2.5 CERaandPAR2Constraintswiththe4-StateUngerboeck ShapingCode 112 5. NONLINEARPRECODINGMETHODSTO REMOVEINTERSYMBOLINTERFERENCE 117 5.1. TomlinsonlHarashimaPrecoding 118 5.2. LTFlMotorola/GDCPrecoding 122 5.3. PrecodingandNoiseWhitening 128 5.3.1 TheFirst-OrderLinearPredictor 131 6. TRELLIS PRECODING 133 6.1. TrellisPrecodingBasedon ShapingonRegions 133 6.1.1 TheTransmitter 134 6.1.2 TheReceiver 139 6.1.3 An ExampleofaTrellis PrecodingSystem 139 6.2. Trellis PrecodingBasedonLatticePartitions andLinearCodes 143 6.3. ExperimentalPerformanceResults 144 7. MAPPING DATATOCHANNELSYMBOL FRAMES BYAMODULUS ENCODER 147 7.1. TheAT&TFractional BitRateModulusConverter 148 7.2. TheV.90ModulusEncoder 152 8. CONSTELLATIONSHAPINGBYSHELLMAPPING 157 8.1. General SystemDescription 158 8.2. RingWeights andtheNumberofFramesofEachWeight 161 8.3. LexicographicalOrdering ofRing Frames 162 8.4. TheDecodingAlgorithm 166 8.5. TheEncodingAlgorithm 171 Appendix8.A. Justificationfor the MotorolaWeightFunction 178 Appendix8.B. ShellMappingProgram 180 9. THEFOURDIMENSIONALCONSTELLATIONUSEDBY ITU-TV.34 MODEMS 187 9.1. The2D Constellationandits Partitioning 187 viii CONTENTS 9.1.1 Generatingthe2DConstellationby90DegreeRotations of4Z2 + (1,1) 187 9.1.2 Partitioningthe 2D Constellationinto 8Subsets 189 9.1.3 A Method for Determining the Binary Subset Label from theCoordinatesofa2DPoint 192 9.2. Framing 193 9.3. The4D Constellation 195 9.3.1 MappingFrames andInitial4DPointSelection 195 9.3.2 Mappingthe Initial4DPointIntotheFinal4DPoint 197 9.3.3 90° RotationalInvarianceofthe 4DConstellation 198 9.3.4 Partitioningofthe4DConstellation 200 9.3.5 Slicing4DPointsto PartitionChainBinary Variables 203 10. THE COMBINED PRECODING AND TRELLIS CODING SCHEMEFORY.34 205 10.1. TheNonlinearPrecoder 205 10.1.1 ThePrecoderInputandOutput 207 10.1.2 ThePredictionFilterOutput 207 10.1.3 TheModuloBox 207 10.1.4 Why the Precoderis theInverseofH(z) 208 10.2. TheTrellis Encoders 209 10.3. Viterbi Decodingof4DTrellis Codes 212 10.4. MoreDetails onthe Wei 16-StateCode 213 10.4.1 GeneratorandCheckMatrices 213 10.4.2 Invarianceto90DegreeRotations 214 10.4.3 TheFundamentalCodingGain 215 10.4.4 TheOriginalWei 16-StateConvolutionalEncoder 215 10.5. Using the ModuloEncoderto Makey(n) aTrellis Sequence 216 10.6. SuperframeSynchronization 219 10.6.1 Compensatingfor SuperframeBitInversions 222 10.7. ReceiverOperation 223 11.FASTEQUALIZERADJUSTMENTBYUSINGAPERIODIC TRAININGSEQUENCE 227 11.1. TheV.34 PeriodicTraining Sequence 227 11.1.1 The Periodic Autocorrelation Function and CAZAC Sequences 227 11.1.2 ConstructingaCAZACSequenceofLengthMK2from oneofLength M 229 11.1.3 TheY.34CAZAC Sequence 233 CONTENTS ix 11.2. TheOptimal FractionallySpacedEqualizer 233 11.2.1 DerivationoftheOptimumLinearEqualizer 237 11.2.2 MSEfortheOptimumLinearEqualizer 243 11.3. FindingtheInitial EqualizerTapsbyUsingthe FFf 244 11.3.1 TheComplexCross-CoupledandRealPhase-Splitting Equalizers 246 11.3.2 ComputingEqualizerCoefficientsby Using the FFf 249 References 255 Index 261 Preface Thisbookisessentiallyahistoryofthedevelopments insignalconstellation design, nonlinear pre-equalization, and trellis coding made by ITV-T Study GroupXVIIduring itsmeetings tocreatearecommendation forastate-of-the art voiceband telephone channel modem. It started deliberations in the fall of 1991 and by June of 1994 finalized the recommendation which was then given the V.34label. Thebookisheavily basedonnotesIprepared for aseries of lectures given to engineers at Pencil Datability Networks (now the Signal Processing Group of Nortel Networks, Germantown, MD) during the ITV T V.34 committee deliberations to keep them abreast of the latest technical proposals. During the deliberations, the study group was called the V.fast committee and was jokingly called the V.1ast committee by members believing that no further improvements in voiceband modems would be made. The initial V.34 recommendation allowed data rates from 2400 to 28,800 bits per second in increments of2400 bps. It was later amended to extended the maximum rate to 33,600 bps. More recently, the Y.90 and Y.92 recommendations were ap proved and allow rates up to 56,000 bps in the downstream direction, that is, from the server to the client modem, by taking advantage ofthe downstream digital networkand PCMcodec in the local office. TheY.90recommendation specifies using V.34 modulation in the upstream direction while V.92 allows PCM encoding in both the downstream and upstream directions in addition to V.34 modulation upstream. The period during which the V.34 recommendation was being formulated wasaveryexcitingtime. Newtechniques werebeingproposedanddiscovered continuallyduringthedeliberations. Digitalsignalprocessor(DSP)technology wasrapidly improving in termsofspeed, smallersize, moreinternal RAM and ROM,andsignificantlyreducedcost. TheDSPtechnological advancesallowed the committee to consider techniques that were significantly more complex