PowerPoint-Präsentation - ITU

PowerPoint-Präsentation - ITU

Workshop on Monitoring Quality of Service and Quality of Experience of Multimedia Services in Broadband/Internet Networks (Maputo, Mozambique, 14-16 April 2014) Perceptual Quality Evaluation Of OTT Streaming Video Services Joachim Pomy [email protected] OPTICOM, Germany Perceptual Quality Evaluation Of OTT Streaming Video Services Dipl.-Ing. Michael Keyhl, MBA Founder & CEO, OPTICOM, Germany Session > OTT Technologies and Services < Broadcast Engineering Conference | NAB Show 2014, Monday April 7 Who We Are

Founded 1995 in Erlangen, Germany Spin-Off from the Fraunhofer-Institute IIS (Home of MP3) Originators of Perceptual Audio Quality Measurement (1988) Co-Authors of Six Major International Standards Leading Source of Voice and Video Quality Test Products and OEM Technology for Mobile and IP-based Network Testing > 100 Licensed OEM Customers > 40.000 Licensed PESQ Products NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 3 How Do You Measure Video How Do You Measure Video Quality? Quality.. ...to benchmark Multi-Screen QoE of ABR VoD OTT

Services as perceived by subscribers? NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 4 How Do You Measure Video Quality.. ...to benchmark Multi-Screen QoE of ABR VoD OTT Services as perceived by subscribers? Conclusion 1: Clear Need for a New Streaming Video Quality Measurement Technique Conclusion 2: Newly Interpret Video Quality in the Context of Multi-Screen Use Scenarios NAB Show 2014, Broadcast Engineering Conference, 07 April 2014

2014 Michael Keyhl, www.opticom.de 5 Overview 1. 2. 3. 4. 5. 6. ABR and Existing Video Metrics and Standards Stakeholders in the OTT QoE Rivalry Defining an >OTT Quality Model< Designing a Video Quality Measure for OTT Introduction to PEVQ-S Conclusions NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de

6 Overview 1. 2. 3. 4. 5. 6. ABR and Existing Video Metrics and Standards Stakeholders in the OTT QoE Rivalry Defining an >OTT Quality Model< Designing a Video Quality Measure for OTT Introduction to PEVQ-S Conclusions NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 7

Testing Typologies Full Reference (FR) No-Reference (NR) Principle: Differential Analysis comparing Degraded with the Original Source Video Access to Reference/Studio Source Pixel-based, Frame-by-Frame Comparison e.g. OPTICOMs PEVQ/ITU-T J.247 + Highest Accuracy and Repeatability + Standardized Processing Intensive Principle: Single-ended Analysis, NO Information on the Original Source Video Pixel-based, or NR/IP Transport Stream Analysis, only,

no Payload decoding Various Proprietary Metrics Low Accuracy Estimates only Failed in Standardization + Lightweight, easy to apply NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 8 Video Quality Standard s develope d by VQEG and ITU based on

NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 9 ABR Video Streaming Techniques (based on Jan Ozer: Video Compression for Flash, Apple Devices and HTML5 [2011]) NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 10 Example: YouTube Upload Upload: V_Res 1920x1080, V_Fps 24, V_BitRate 8Mbit/s

Protocol PD PD PD PD PD PD PD ABR ABR ABR ABR ABR ABR ABR V_Codec MPEG4 MPEG4 FLV VP8 H.264

H.264 H.264 H.264 H.264 H.264 H.264 H.264 V_BitRate 60kbit/s 200kbit/s 300kbit/s 600kbit/s 500kbit/s 2Mbit/s 100kbit/s 250kbit/s 400kbit/s 800kbit/s 1.6Mbit/s 3Mbit/s

V_Res 176x144 320x180 426x240 640x360 640x360 1280x720 256x144 426x240 640x360 854x480 1280x720 1920x1080 V_Fps 12 24 24 24 24 24 15

24 24 24 24 24 A_Codec AAC AAC MP3 OGG OGG AAC AAC AAC - A_Bitrate 24kbit/s 32kbit/s 64kbit/s 96kbit/s

110kbit/s 96kbit/s 192kbit/s 128kbit/s - NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 Reference/Source PD or ABR Protocols Format 3GP 3GP FLV WEBM WEBM MP4 MP4 MP4 MP4 MP4

MP4 MP4 MP4 MP4 PEVQ-S Various Codecs at Various Bitrates Compare Various Frame Sizes Compare Different Fps

Handle Common Bitstream Formats 2014 Michael Keyhl, www.opticom.de 11 Only PEVQ was validated for QCIF, CIF, VGA, SD, and HD based on BT.500/P.910 NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 PEVQ PEVQ PEVQ

2014 Michael Keyhl, www.opticom.de 12 Validation by Subjective Testing ITU-R BT.500: Originally CRT TV-Quality Testing (SD) ITU-T P.910: Multimedia (QCIF, CIF, VGA) and IPTV (HD 720/1080) Mostly ACR Tests Test Sequences 8 15 sec 5-Point Opinion Scale Averaging over a Large Enough Test Population MOS Conclusion 4: Need for New Subjective Testing Snapshots of 10 s Video are Inadequate to Assess Re-Buffering and Long-term Behavior of Streaming Video

NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 13 Overview 1. 2. 3. 4. 5. 6. Existing Video Metrics and Standards Stakeholders in the OTT QoE Rivalry Defining an >OTT Quality Model< Designing a Video Quality Measure for OTT Introduction to PEVQ-S Conclusions NAB Show 2014, Broadcast Engineering Conference, 07 April 2014

2014 Michael Keyhl, www.opticom.de 14 Stakeholders involved in a QoE Rivalry NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 15 Overview 1. 2. 3. 4. 5. 6. Existing Video Metrics and Standards Stakeholders in the OTT QoE Rivalry

Defining an >OTT Quality Model< Designing a Video Quality Measure for OTT Introduction to PEVQ-S Conclusions NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 16 Defining a 4layered OTT Quality Model NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 17

Stakeholders Different Quality Views Stakeholders for OTT QoE and related Quality Layers NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 18 Overview 1. 2. 3. 4. 5. 6. Existing Video Metrics and Standards Stakeholders in the OTT QoE Rivalry

Defining an >OTT Quality Model< Designing a Video Quality Measure for OTT Introduction to PEVQ-S Conclusions NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 19 OTT VQM Requirements VQM Should be related to the Content Quality as a Reference Accurately score encoding and transcoding Artifacts = Media Stream Quality Measure and compare the Picture Quality for different Frame Sizes and Frame Rates = Media Stream/Transmission Quality Continuously track the different Bit Rates and evaluate how smooth the Player is able to interact with the Server in a Congested Network = Transmission Quality Take into account Player and Device Characteristics as well

as Viewing Environment = Presentation Quality 20 NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de Overview 1. 2. 3. 4. 5. 6. Existing Video Metrics and Standards Stakeholders in the OTT QoE Rivalry Defining an >OTT Quality Model< Designing a Video Quality Measure for OTT Introduction to PEVQ-S Conclusions NAB Show 2014, Broadcast Engineering Conference, 07 April 2014

2014 Michael Keyhl, www.opticom.de 21 NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 22 Overview 1. 2. 3. 4. 5. 6. Existing Video Metrics and Standards Stakeholders in the OTT QoE Rivalry Defining an >OTT Quality Model< Designing a Video Quality Measure for OTT

Introduction to PEVQ-S Conclusions NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 23 Conclusions (1/2) Clear Need for a Streaming Video Quality Measurement Technique which accurately evaluates Subscriber QoE We must reformulate QoE Concepts and newly interpret Video Quality in the Context of Multi-Screen Use Scenarios Currently, no Standardized Subjective and Objective VQM Standard exists for ABR Video Streaming PEVQ-S is proposed to leapfrog this Shortcoming, based on advancing existing Standards, while maintaining maximum Backward-Compatibility and Validated Accuracy PEVQ-S is suited to evaluate all 4 OTT Quality Layers:

Content, Media Stream, Transmission and Presentation Quality NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 24 Conclusions (2/2) PEVQ-S allows for Analysis of Common ABR Protocols and Formats, Various Codecs at Various Bitrates PEVQ-S can analyze Video at Different Frame Sizes and Fps An Advanced Hybrid Architecture: (a) the PEVQ-S Media Stream Quality Database and (b) the PEVQ-S Client Probe combines the high Accuracy of a FR Measure with a Light-weight Realtime Client Implementation, even on Android Smartphones Measures Current Video Quality (CVQ) MOS and Headroom for Optimization for the Best Video Quality Available (BVQA) MOS PEVQ-S is made available as an OEM-Technology to leading OTT, Middleware and T&M Vendors and soon to be found in Products

NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 25 Any questions ? Joachim Pomy [email protected]

Recently Viewed Presentations

  • Commercial Fraud  Law and Practice Tim Lord QC

    Commercial Fraud Law and Practice Tim Lord QC

    The Court of Appeal in 2005 clarified the position in Bowman v. Fels[2005] EWCA Civ229. Now reasonably clear POCA was not intended to apply to legal professionals negotiating nor implementing a consensual resolution of issues in a litigious context.
  • School Calendar Workshop School Year 2014-15

    School Calendar Workshop School Year 2014-15

    Legislative Revisions . The Legislature made substantial changes to the school calendar statutes (WVC §§18-5-45, 18-5A-5, and 18A-5-2) during the 2013 legislative session through the passage of SB 359 to provide county boards more flexibility in developing a school calendar...
  • Die Brder Grimm Whitney Inge Schoellerman Dorothea Grimm

    Die Brder Grimm Whitney Inge Schoellerman Dorothea Grimm

    Kinder- und Hausmärchen (Children's and household tales first published in . 1812). Had a total of 204 tales by 1856. Of the 86 stories in the first edition only 12 had been taken from literary sources. Initially, they did not...
  • Course 1: Computational Photography Organisers Ramesh Raskar Mitsubishi

    Course 1: Computational Photography Organisers Ramesh Raskar Mitsubishi

    He was a NASA/ASEE Research Fellow in 1994 at NASA Langley Research Center, VA. and the inaugural Kodak Visiting Professor to RMIT in Australia. He was the recipient of the Eisenhart Award for Outstanding Teaching at RIT and the 1990...
  • CAT 2: Investigating Water Scarcity

    CAT 2: Investigating Water Scarcity

    Read the two paragraphs about economical and physical water scarcity. - Write a definition for each. - Include the examples stated. Section 1 . In your research book: Complete the shade and label task. Do not colour and label all...
  • Department on Disability Services - Washington, D.C.

    Department on Disability Services - Washington, D.C.

    DC's Medicaid HCBS Waiver. DC's Home and Community Based Services (HCBS) Waiver provides services and supports so that people with intellectual and developmental disabilities (IDD) are able to live at home and in the community.
  • CSE 575 Computer Arithmetic Spring 2005 Mary Jane

    CSE 575 Computer Arithmetic Spring 2005 Mary Jane

    Characteristics, con't Scale factor, s - implied radix point (integer, fraction, or mixed number) Precision, n = k + l - granularity of representation (k: # integer digits, l: # of fractional digits), ulp = r0 = 1 for integers,...
  • Efficient computation of Robust Low-Rank Matrix ...

    Efficient computation of Robust Low-Rank Matrix ...

    Arial Default Design Microsoft Equation 3.0 Efficient computation of Robust Low-Rank Matrix Approximations in the Presence of Missing Data using the L1 Norm Slide 2 Problem Wiberg Algorithm Alternating Least Squares Back to Wiberg Back to Wiberg Linear Programming and...