Master's Thesis Live Streaming Support on Mobile Networks for Android and iOS Jon Andersen ...

92 Pages·2013·6.78 MB·English

by  Lena Ohlsson

Master’s Thesis

Live Streaming Support on Mobile Networks
for Android and iOS

Jon Andersen & Simon Ekvy
Department of Computer Science
Faculty of Engineering LTH
Lund University, 2013

ISSN 1650-2884
LU-CS-EX: 2013-09
Live Streaming Support on Mobile
Networks for Android and iOS
Jon Andersen Simon Ekvy
Abstract
Mobile devices are becoming increasingly important, and users are getting
used to having access to all their data and systems in their smartphones.
Of course, this also includes video surveillance systems. However, iPhone
and Android smartphones have limited support for Axis video streaming
formats in their native browsers, and there is a need for apps in order to
offer a good user experience for remote mobile access.
The purpose of this thesis is to investigate video streaming, network
protocols and decoding of video in smartphones. Furthermore, this thesis
will look into video quality and how to ensure this on mobile networks.
Several different approaches and architectures for streaming media apps
are presented, and the most promising approaches are selected to be pro-
totyped and then evaluated. Since mobile networks present a new area for
Axis Cameras, quality and performance of these networks with the Axis
cameras are evaluated. We created predefined profiles for different networks
such as Edge, 3G and WiFi and an algorithm for automatic quality switch-
ing between these profiles. However, switching to an optimal profile for
best quality requires context awareness. This is possible through several
methods, some of which we investigate in this thesis. Based on the result of
the evaluations, we make recommendations for the best approach on how
to develop a streaming media app for an Axis camera and a proposed quality
switching algorithm to handle the varying quality of mobile networks.
Keywords: iOS, Android, Live Streaming, RTP/RTSP, MediaCodec, FFmpeg,
Video Quality
2
Acknowledgements
We would like to send our gratitude to our supervisors Anders Persson and Gunnar
Erlandsson at Axis Communications for their support and invaluable feedback. Also
we want to thank Martin Gunnarsson for his input and help. Furthermore we would
like to thank Mathias Haage, our supervisor at the Department of Computer Science
at Lund University, for his guidance and input during the work of the thesis.
3
4
Contents
1 Introduction 7
1.1 Purpose and goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2 Previous Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 Tool Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5 Limiting factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.6 Streaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.7 Playback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2 Background 13
2.1 Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.1.1 Streaming media over TCP and UDP . . . . . . . . . . . . . . 13
2.1.2 RTSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.1.3 RTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1.4 HTTP Live Streaming (HLS) . . . . . . . . . . . . . . . . . . 16
2.2 Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2.1 JPEG and MPEG . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3 Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4 Wireless Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.5 Network Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.6 Smartphones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.7 Video Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3 Support Analysis 25
3.1 Android . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.1.1 FFmpeg and Gstreamer . . . . . . . . . . . . . . . . . . . . . 27
3.1.2 Proxy RTSP . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.1.3 MediaCodec API . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.1.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.2 iOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5
CONTENTS
3.2.1 FFmpeg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.2.2 Proxy RTSP . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.2.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4 Design and Development 35
4.1 Android Proxy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.1.1 Android UDP Proxy . . . . . . . . . . . . . . . . . . . . . . . 35
4.1.2 Android HTTP Tunneled Proxy . . . . . . . . . . . . . . . . . 36
4.2 Android and MediaCodec . . . . . . . . . . . . . . . . . . . . . . . . 37
4.3 iOS and FFmpeg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.4 Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5 Streaming and Video Quality 45
5.1 Network speed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
5.2 Basic Algorithm Concept . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.3 Video Quality parameters bit rate use . . . . . . . . . . . . . . . . . . 49
5.4 Mobile Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5.5 App design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
6 Discussion 57
6.1 App discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6.2 Video Quality Discussion . . . . . . . . . . . . . . . . . . . . . . . . 58
6.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
6.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Appendix A Full size figures 63
Appendix B Code Snippets 73
B.1 iOS detect network . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Appendix C How to compile FFmpeg 75
C.1 Compiling for iOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
Appendix D Decoding with the MediaCodec API 79
D.1 The basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
D.2 Sample code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
6