Multimedia Information Acquisition and Retrieval Enhancement using Intelligent Search System ا ا ...

141 Pages·2015·3.36 MB·English

Multimedia Information Acquisition and
Retrieval Enhancement using Intelligent Search
System
م ا(cid:25)(cid:26)(cid:17)(cid:18)(cid:8)(cid:27) ةد(cid:25)(cid:12)(cid:17)(cid:30)(cid:31)ا !(cid:8)(cid:18)(cid:10)(cid:31)ا ت(cid:8)(cid:9)(cid:10)(cid:11)(cid:12)(cid:9) ع(cid:8)(cid:15)(cid:16)(cid:17)(cid:18)او ء(cid:8)(cid:22)(cid:23)(cid:17)(cid:18)ا (cid:2)(cid:3)(cid:4)(cid:5)(cid:6)))
(("آذ %(cid:5)(cid:27) م(cid:8)&'

By
Salameh Ahmad Salameh AL-Etaywi
401220078

Supervisor
Dr. Maamoun Khalid Ahmad

Master Thesis Proposal
Submitted In Partial Fulfillment of the Requirement of the
Master Degree in Computer Science

Faculty of Information Technology
Middle East University
Amman, Jordan
January 2015
ii
iii
iv
Acknowledgments
Prior to acknowledgments, I must glorify Allah the Almighty for His
blessings who gave me courage and patience to carry out this work
successfully, and giving me this opportunity to become a student once
again after years of work. I thank Dr. Maamoun Khalid Ahmad for all his
help and support; for giving me guidance, encouragement, confidence, and
for being receptive to the ideas i came up with. I also wish to express my
deepest gratitude to the members of the committee for spending their
precious time on reading my thesis and giving me encouragement and
constructive comments. Also i would like to thank all information
technology faculty members at middle east university, and my parents for
everything they had been done for me during my life and for their constant
prayers. Finally i would like to thank my wife for her help, support,
sacrifice and great patience all the time. I am so grateful to those who
helped me in this research work.
v
Dedication
((cid:14)(cid:13)(cid:16)ُ(cid:28)(cid:10)(cid:29)(cid:19)(cid:30)ز(cid:4)(cid:11)َ(cid:24)َ (cid:14)(cid:13)(cid:12)ُ(cid:26)(cid:25)(cid:16)َ(cid:27)َ (cid:21)ْ(cid:23)ِ(cid:24)َ (cid:14)(cid:13)(cid:16)ُ(cid:18)(cid:17)ر(cid:19) نَذ(cid:10)(cid:11)َ(cid:12)َ ذْإ(cid:4)و(cid:6))
(cid:4)(cid:3)(cid:2) (cid:14)"ها(cid:26)(cid:18)إ
To my kindhearted Father & Mother
To my wonderful wife … Ala'a
To my dear loving daughter: Jana
To the Soul of My Son: Waseem
I dedicate this work.
Salameh
vi

Table of Contents
Authorization ........................................................................................ ii
Thesis Committee Decision ................................................................. iii
Acknowledgments…............................................................................. iv
Dedication ............................................................................................. v
Table of Contents ................................................................................. vi
Abbreviations ....................................................................................... ix
List of Figures....................................................................................... xi
List of Tables....................................................................................... xiii
Abstract in English ............................................................................. xiv
Abstract in Arabic ............................................................................... xv
Chapter One. Introduction ................................................................ 1
1.1. Overview ........................................................................................ 1
1.1.1. Audio mining .................................................................... 4
1.1.2. Audio mining approaches.................................................. 5
1.2. Problem Definition ......................................................................... 6
1.3. Objectives ....................................................................................... 7
1.4. Motivation ...................................................................................... 8
1.5. Limitation ....................................................................................... 8
1.6. Terminology ................................................................................... 9
1.7. Thesis Structure ............................................................................. 12
vii
Chapter Two. Literature Review and Related Work...................... 13
Literature Review ................................................................................. 13
2.1. Introduction ......................................................................... 13
2.2. The Educational Process ..................................................... 13
2.3. E-Learning .......................................................................... 14
2.3.1. Digital Lectures ..................................................... 16
2.4. Data Mining .................................................................…... 17
2.4.1. Multimedia Data Mining …................................... 19
2.4.2. Audio Mining ........................................................ 21
2.5. Speech Recognition ............................................................ 23
2.5.1. Types of Speech Recognition ............................... 26
2.5.2. Speech Recognition Process ................................. 27
2.6. Knowledge Base System .................................................... 28
2.6.1. Expert System ....................................................... 30
2.7. Information Retrieval System ............................................ 31
2.8. Search Technique ...............................................................32
2.9. String Matching ................................................................. 34
2.9.1. Types of String Matching ..................................... 34
Related Works ..................................................................................... 34
Chapter Three. The Proposed Technique ....................................... 51
3.1. Overview ............................................................................ 51
3.2. Methodology of Proposed Solution ....................................53
3.3. Components of Proposed System ......................................54
viii
Chapter Four. Implementation of Video Multi-Searcher System...59
4.1. Overview .............................................................................59
4.2. Data Preprocessing ..............................................................61
4.2.1. Step 1: Audio Track Extraction….......................... 61
4.2.2. Step 2: Audio Track Transcription ........................ 63
4.2.3. Step 3: Text and Audio Segmentation and
Alignment.............................................................. 65
4.3. Semi-Automatic Method ..................................................... 68
4.4. Built-in Synonyms Database ..............................................71
4.5. The "Video Multi-Searcher" System....................................72
4.6. Summary ………………………….....................................79
Chapter Five. Experimental Results …………………….................81
5.1. Experimental Methodology .................................................81
5.2. Comparison..........................................................................90
5.3. Results..................................................................................91
5.3. Contributions ...................................................................... 93
Chapter Six. Conclusion & Future Work ………………….............94
6.1. Conclusion ………………...................................................94
6.2. Future Work ….....................................................................95
References .............................................................................................96
Appendix …………………………………………………….………..103
ix
List of Abbreviations
API Application programming interface
ASR Automatic Speech Recognition
CAD Computer-Aided Design
CCText Closed Caption Text
DBS Database System
HMM Hidden Markov Models
ICT Information and Communication Technologies
IR Information Retrieval
KBS Knowledge Base System
KDD Knowledge Discovery from Data
LOD Linked audio data
LSCOM A Light Scale Concept Ontology for Multimedia
LVCSR Large-Vocabulary Continuous Speech Recognition
NDVR Near-duplicate video retrieval
NLP Natural Language Process
OCR Optical Character Recognition
RDF Resource Description Framework
SAPI Speech Application Programming Interface
SBV Superbase Form Definition File
SRT SubRip Subtitle File
x
UCC User-Created contents
VTT The Web Video Text Tracks Format
XML Extensible Markup Language
XLS Filename Extension (Microsoft Excel spreadsheet file)