ebook img

Timothy Aaron Stotts PDF

126 Pages·2016·2.54 MB·English
Save to my drive
Quick download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Timothy Aaron Stotts

RRoocchheesstteerr IInnssttiittuuttee ooff TTeecchhnnoollooggyy RRIITT SScchhoollaarr WWoorrkkss Theses 8-2007 MMeetthhooddoollooggyy aanndd ooppttiimmiizziinngg ooff mmuullttiippllee ffrraammee ffoorrmmaatt bbuuffffeerriinngg wwiitthhiinn FFPPGGAA HH..226644//AAVVCC ddeeccooddeerr wwiitthh FFRREExxtt.. Timothy Aaron Stotts Follow this and additional works at: https://scholarworks.rit.edu/theses RReeccoommmmeennddeedd CCiittaattiioonn Stotts, Timothy Aaron, "Methodology and optimizing of multiple frame format buffering within FPGA H.264/AVC decoder with FRExt." (2007). Thesis. Rochester Institute of Technology. Accessed from This Thesis is brought to you for free and open access by RIT Scholar Works. It has been accepted for inclusion in Theses by an authorized administrator of RIT Scholar Works. For more information, please contact [email protected]. Methodology and optimizing of multiple frame format buffering within FPGA H.264/AVC decoder with FRExt. by Timothy Aaron Stotts A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Science in Computer Engineering Supervised by Assistant Professor Dr. Marcin Lukowiak Department of Computer Engineering Kate Gleason College of Engineering Rochester Institute of Technology Rochester. New York August 2007 Approved By: Marcin tukowiak Dr. Marcin Lukowiak Assistant Professor, RIT, Department of Computer Engineering Primary Adviser Ken W. Hsu Dr. Ken W. Hsu Professor, RIT, Department of Computer Engineering Secondary Adviser Mark Grabosky Mark Grabosky Director of Engineering, Xelic, Inc. Secondary Adviser Thesis Release Permission Form Rochester Institute of Technology Kate Gleason College of Engineering Title: Methodology and optimizing of multiple frame format buffering within FPGA H.264/AVC decoder with FRExt. I, Timothy Aaron Stotts, hereby grant permission to the Wallace Memorial Library to reproduce my thesis in whole or in part. Timothy Aaron Stotts Timothy Aaron Stotts :Joo f - 0& - I?- Date Dedication To Christ Jesus, my one true source ofpeace. "Peace I leave with you, my peace I give unto you: not as the world giveth, give I unto you. Let not yourheart be troubled, neither let it be fearful." in Acknowledgments A special thank you to each of my advisers for sharing their time and experience; and especially to Dr. Lukowiak for his patient guidance, and Mark Grabosky at Xelic, Inc. for encouraging andequiping metopull through. Thank you also toThomasWarsawformany thoughtful discussions regarding his prior H.264/AVC work; and to family and friends for theirloving support. iv Abstract Digital representation ofvideo data is an inherently resource demandingproblemthat con tinues to necessitate the development and refinement ofcoding methods. The H.264/AVC standard, along with its recent Fidelity Range Extensions amendment (FRExt), is quickly being adopted as the standard codec forbroadcast and distribution ofhigh definition video. The FRExt amendment, while not necessarily affecting the overall decoder architecture, presents an added complexity of providing efficient memory management for buffering intermediate frames ofvarious pixel color samplings and depths. This thesisevaluatedtheroleofdesigningtheframebufferofahardwarevideo decoder, with integrated support for the H.264/AVC codec plus FRExt. With focus on organizing external memory data access, the frame buffer was designed to provide intermediate data storage for the decoder, while using an efficient store and load scheme that takes into con sideration each frame pixel format ofthe video data. VHDL was used to model the frame buffer. Exploitation ofreconfigurability andpost- synthesis FPGA simulations were used to evaluate behavior, scalability and power con sumption, while providing an analysis ofapproaches to adding FRExt to the memory man agement. Real-time buffer performance was achieved for two common frame formats at 1080 HD resolution; and an innovative pipeline design provides dynamic switching offor mats between video sequences. As an additional consequence of verifying the model, a preexisting Baseline H.264/AVC decoder testbench was augmented to support testing of multiple frame formats. Contents Dedication iii ... Acknowledgments iv ... Abstract v Glossary xiv 1 Introduction 1 ... 1.1 Background 1 1.2 Thesis objective 4 1.3 Thesis chapter overview 6 2 Video Coding 7 ... 2.1 Y'CbCrcolor model 7 2.1.1 Y'CbCr sub-sampling 9 2.2 H.264/AVC overview 12 2.2.1 H.264/AVC coding summary 14 2.2.2 H.264/AVC Fidelity Range Extensions summary 17 2.3 Thesis relevance and specifics 19 2.3.1 H.264/AVC databuffering flow 19 2.3.2 H.264/AVC databuffering organization 20 2.3.3 Macroblock pixel types 23 3 H.264/AVC Research .25 3.1 Decoder memory case studies and research 25 3.1.1 Identification ofmemory components 26 3.1.2 Optimization techniques 27 3.2 Analysis ofpublished results 30 4 Requirements and Modeling 32 4.1 Augmentation ofdecoder system 32 4.2 Algorithms used by the frame buffer. 32 4.2.1 Intraprediction buffering requirements 34 4.2.2 Deblocking filterbuffering requirements 36 4.2.3 Interprediction buffering requirements 36 vi 4.2.4 Combining ofbuffering mechanisms 37 4.2.5 Reference picture management 37 5 Synthesizable Implementation 40 5.1 External memory storage and control 40 5.1.1 DDR memory control 42 5.1.2 Block data controller 44 5.1.3 Implementation hiearchy 45 5.1.4 External DDR interface 46 5.2 Frame organization and addressing 46 5.2.1 Macroblock identification and frame slotting 49 5.2.2 Macroblock address mapping with FRExt 51 5.2.3 Frame storage marking 53 5.2.4 Sliding window implementation 54 5.3 Synthesis parameters 55 5.4 Frame buffer interface and pipelining 57 5.4.1 Frame buffer RTL interface 57 5.4.2 Pipeline semantics 62 5.5 Dual RAM frame buffer 63 5.5.1 Dual DDR SDRAM design 65 6 Verification - HDL Model Functionality . . .... 69 6.1 Unit testing 69 6.2 In-systemverification 71 6.2.1 Augmentation ofthe decoder system 71 6.2.2 Testbench redesign 74 6.2.3 Video sequences 76 6.2.4 Functional simulation 77 6.2.5 Post-synthesis simulation 79 7 Results and Analysis 81 ... 7.1 Implementation analysis gl 7.2 Synthesis resource analysis 86 7.3 DDR timing analysis 37 7.4 H.264/AVC timing analysis 90 7.5 Powerconsumption analysis 92 7.6 Cost analysis 94 8 Conclusions 97 8.1 Synthesizable models 97 8.2 Proposed system interfacing 99 Bibliography 203 vn Software Tools and Deliverables 106 A.l Software tools 106 A.1.1 Video processing and display 106 A.1.2 FPGA design and simulation 107 A.2 Thesis deliverables 107 vm List Figures of 1. 1 Digital representation ofapicture in terms ofdata size 1 1.2 Digital representation ofuncompressedvideo in terms ofdata size 2 1.3 The extra dimension ofpixel size upon total picture data size 3 1.4 Internal partitioning offrame buffer design 4 1.5 Video decoder system partitioning augmented fortesting frame buffer. ... 5 2.1 RGB vs. Y'CbCr decomposition ofa "foreman" test frame 8 2.2 Y'CbCr sub-sampling 4:4:4 10 2.3 Y'CbCr sub-sampling 4:2:2 11 2.4 Y'CbCr sub-sampling 4:2:0 11 2.5 Y'CbCr sub-sampling 4:0:0 12 2.6 Scope ofH.264/AVC Standard: only decoding [24] 14 2.7 The correlation between source (uncoded) picture frames and encoded slices. 15 2.8 Pixel sampling and depth, increasingly stacked by FRExt profile. [24] ... 18 2.9 Buffering within the H.264/AVC Hypothetical Reference Decoder. [9] 20 . . . 2.10 DPB operation: macroblock in, macroblock out 21 2.11 Buffer arow ofmacroblocks to retain neighbor MBs 21 2.12 Organization ofthe reference frame buffer 22 3.1 FPGA hybrid on-chip, off-chip decoder architecture proposed in [21]. 26 ... 4.1 Video decodersystem architecture and data flow 33 4.2 Intra Prediction macroblock neighborpermutations 34 4.3 Main datapath "tap" locations for buffering 36 5.1 Vendor-supplied Xilinx Spartan 3E DDR SDRAM controller. 43 5.2 Customized Xilinx Spartan 3E DDR SDRAM controller 43 5.3 Block datacontroller state machines 44 5.4 Internal partitioning offrame buffer design 45 5.5 External DDR interface 47 5.6 Binary 16x8 sub-macroblock memory maps for each 8-bit sub-sampling. 49 . . 5.7 Single RAM frame buffer inner and outer interfaces 58 5.8 Dual RAM frame buffer inner and outer interfaces 65 6.1 Testbench flow with emphasis on dataprocessing 74 6.2 Testbench flow with emphasis on storage operations 75 6.3 Elephant's Dream Frames 11290, 11310 77 IX

VHDL was used to model the frame buffer. Exploitation of .. Two examples include NTSC television broadcast and VHS video cassettes. xvn
See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.