Streaming services such as Netflix, Disney, HBO, Amazon, iTunes, HBO Max, Apple TV+ etc.
#27322
Together with our Danish forum users on Flatpanels.dk we have been measuring video/audio quality on streaming services for the past 2 years. We're now starting the same initiative here on FlatpanelsHD.com and we hope that you guys will help as we don't have access to all streaming services.

We'll update the table in this post when new measurements are posted in this forum thread (you can create a forum user or use your social media account to log in).

How to measure video/audio streaming quality:
- You need: Apple TV 4K, Macbook/iMac, and the Xcode software. Both devices must be on the same network.
- On Apple TV go to Settings -> Remotes and Devices -> Remote App and Devices. Leave it here.
- Now open Xcode on your Mac and go to Window -> Devices and Simulators. Select your Apple TV 4K and pair it.
- On Apple TV go to Settings -> Developer (new menu will pop up once paired with Xcode) -> Playback HUD (enable)
- Start a video stream on Apple TV 4K, let it run for at least 3-5 minutes. Take a screenshot (via Xcode) or a photo with the measurements.


Last updated: July 15, 2020
Attachments
Screenshot 2020-07-15.png
Screenshot 2020-07-15.png (544.76 KiB) Viewed 181553 times
#27323
Some of the measurements in the table are a little dated and some values are missing / approximation. We'll take additional measurements soon to fill out the gaps. Please contribute. Let's aim to expand the list significantly :) Together we can help put pressure on service to increase their video streaming quality!
#27324
Great Idea! Some input from me:

Amazon 4K HDR10+ is just under 15mbit for constant bitrate. Should also apply for HDR10 version.
Amazon 1080p AVC is 10mbit for CBR.
Amazon 1080p HEVC is 8-9mbit for CBR. However, I have also seen some with 3mbit, for example on "How to get away with murder" S01

Audio on both can differ, but its usually 640kbit. These are for TV shows but I assume they are valid for movies as well.

Why do you include the frame rate?
#27358
Thanks Sledgegamma. Will check again with Amazon. Is it possible that there's a difference between North America and Europe? Please upload measurements if possible.

We include frame rate because some streaming services - mostly companies entrenched in the EU broadcast ecosystem - have a bad habit of converting movies/series in 24fps to 25fps (via speed-up - some even ignoring audio pitch leveling).
#27363
Rasmus Larsen wrote: 27 May 2020, 23:46 Thanks Sledgegamma. Will check again with Amazon. Is it possible that there's a difference between North America and Europe? Please upload measurements if possible.

We include frame rate because some streaming services - mostly companies entrenched in the EU broadcast ecosystem - have a bad habit of converting movies/series in 24fps to 25fps (via speed-up - some even ignoring audio pitch leveling).
Yes, sadly that's true about the 25p version...
Don't know whether there are differences for North America and EU. That said, there are definitely difference regarding India with India having a lower bitrate. In addition, Amazon doesn't always use the same frame rate for all the regions. Sometimes its 24.000 and sometimes 23.976.

My measurement are media info output from the uncompressed/unaltered downloads of those TV shows.

Amazon: Sneaky Pete S01E01:
Code: Select all
General
Unique ID                                : 288806875305643181453618751556428061195 (0xD9462E51670FCF021627ACA14B315A0B)
Complete name                            : 
Format                                   : Matroska
Format version                           : Version 4
File size                                : 5.78 GiB
Duration                                 : 54mn 14s
Overall bit rate                         : 15.3 Mbps
Encoded date                             : UTC 2020-05-13 17:09:19
Writing application                      : mkvmerge v45.0.0 ('Heaven in Pennies') 64-bit
Writing library                          : libebml v1.3.10 + libmatroska v1.5.2

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5@Main
HDR format                               : SMPTE ST 2094 App 4, Version 0
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 54mn 14s
Bit rate                                 : 14.6 Mbps
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) fps
Standard                                 : NTSC
Color space                              : YUV
Chroma subsampling                       : 4:2:0 (Type 2)
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.073
Stream size                              : 5.53 GiB (96%)
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0001 cd/m2, max: 1000 cd/m2
Maximum Content Light Level              : 907 cd/m2
Maximum Frame-Average Light Level        : 821 cd/m2

Audio
ID                                       : 2
Format                                   : E-AC-3
Format/Info                              : Enhanced AC-3
Commercial name                          : Dolby Digital Plus
Codec ID                                 : A_EAC3
Duration                                 : 54mn 14s
Bit rate mode                            : Constant
Bit rate                                 : 640 Kbps
Channel(s)                               : 6 channels
Channel layout                           : L R C LFE Ls Rs
Sampling rate                            : 48.0 KHz
Frame rate                               : 31.250 fps (1536 SPF)
Compression mode                         : Lossy
Stream size                              : 248 MiB (4%)
Title                                    : English
Language                                 : English
Service kind                             : Complete Main
Default                                  : Yes
Forced                                   : No


Supernatural S15E13 from Amazon:
Code: Select all
General
Unique ID                                : 69770124182158676389742909699507562137 (0x347D3CB51D92D2AE1ACA6CBE0ACB2699)
Complete name                            : 
Format                                   : Matroska
Format version                           : Version 4
File size                                : 2.80 GiB
Duration                                 : 42mn 20s
Overall bit rate                         : 9 450 Kbps
Encoded date                             : UTC 2020-03-24 08:01:25
Writing application                      : mkvmerge v21.0.0 ('Tardigrades Will Inherit The Earth') 64-bit
Writing library                          : libebml v1.3.5 + libmatroska v1.4.8

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 42mn 20s
Bit rate mode                            : Constant
Bit rate                                 : 8 808 Kbps
Nominal bit rate                         : 10 000 Kbps
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.177
Stream size                              : 2.61 GiB (93%)
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio
ID                                       : 2
Format                                   : E-AC-3
Format/Info                              : Enhanced AC-3
Commercial name                          : Dolby Digital Plus
Codec ID                                 : A_EAC3
Duration                                 : 42mn 20s
Bit rate mode                            : Constant
Bit rate                                 : 640 Kbps
Channel(s)                               : 6 channels
Channel layout                           : L R C LFE Ls Rs
Sampling rate                            : 48.0 KHz
Frame rate                               : 31.250 fps (1536 SPF)
Compression mode                         : Lossy
Stream size                              : 194 MiB (7%)
Language                                 : English
Service kind                             : Complete Main
Default                                  : Yes
Forced                                   : No
Amazon: Joker
Code: Select all
Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 4 frames
Codec ID : V_MPEG4/ISO/AVC
Duration : 2 h 1 min
Bit rate mode : Constant
Bit rate : 8 818 kb/s
Nominal bit rate : 10 000 kb/s
Width : 1 920 pixels
Height : 1 040 pixels
Display aspect ratio : 1.85:1
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.184
Stream size : 7.50 GiB (93%)
Default : Yes
Forced : No
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

Audio
ID : 2
Format : E-AC-3
Format/Info : Enhanced AC-3
Commercial name : Dolby Digital Plus
Codec ID : A_EAC3
Duration : 2 h 1 min
Bit rate mode : Constant
Bit rate : 640 kb/s
Channel(s) : 6 channels
Channel layout : L R C LFE Ls Rs
Sampling rate : 48.0 kHz
Frame rate : 31.250 FPS (1536 SPF)
Compression mode : Lossy
Stream size : 557 MiB (7%)
Language : English
Service kind : Complete Main
Default : Yes
Forced : No
For All Mankind HDR10:
Code: Select all
Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5@High
HDR format : SMPTE ST 2086, HDR10 compatible
Codec ID : V_MPEGH/ISO/HEVC
Duration : 1 h 5 min
Bit rate : 24.0 Mb/s
Width : 3 840 pixels
Height : 1 920 pixels
Display aspect ratio : 2.000
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0 (Type 2)
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.136
Stream size : 10.9 GiB (95%)
Language : English
Default : Yes
Forced : No
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0050 cd/m2, max: 1000 cd/m2
Maximum Content Light Level : 1833 cd/m2
Maximum Frame-Average Light Level : 744 cd/m2

Audio
ID : 2
Format : E-AC-3 JOC
Format/Info : Enhanced AC-3 with Joint Object Coding
Commercial name : Dolby Digital Plus with Dolby Atmos
Codec ID : A_EAC3
Duration : 1 h 5 min
Bit rate mode : Constant
Bit rate : 768 kb/s
Channel(s) : 6 channels
Channel layout : L R C LFE Ls Rs
Sampling rate : 48.0 kHz
Frame rate : 31.250 FPS (1536 SPF)
Compression mode : Lossy
Stream size : 358 MiB (3%)
Language : English
Service kind : Complete Main
Default : Yes
Forced : No
Complexity index : 16
Number of dynamic objects : 15
Bed channel count : 1 channel
Bed channel configuration : LFE
#27737
Rasmus Larsen wrote: 27 May 2020, 05:56 Together with our Danish forum users on Flatpanels.dk we have been measuring video/audio quality on streaming services for the past 2 years. We're now starting the same initiative here on FlatpanelsHD.com and we hope that you guys will help as we don't have access to all streaming services.

We'll update the table in this post when new measurements are posted in this forum thread (you can create a forum user or use your social media account to log in).

How to measure video/audio streaming quality:
- You need: Apple TV 4K, Macbook/iMac, and the Xcode software. Both devices must be on the same network.
- On Apple TV go to Settings -> Remotes and Devices -> Remote App and Devices. Leave it here.
- Now open Xcode on your Mac and go to Window -> Devices and Simulators. Select your Apple TV 4K and pair it.
- On Apple TV go to Settings -> Developer (new menu will pop up once paired with Xcode) -> Playback HUD (enable)
- Start a video stream on Apple TV 4K, let it run for at least 3-5 minutes. Take a screenshot (via Xcode) or a photo with the measurements.



Image
It looks like the minimum bitrate for iTunes 4K HDR10 and iTunes 4K DV are 6 Mbps higher on the Danish site than in this thread, but the maximum bitrate is 6 Mbps higher in the U.S. That is an interesting result.
#29457
WillzyxTheZypod: Should be the same stream. I think we just used some other TV series for reference on the Danish forum. Will check again and see if it needs correction. It will, however, vary from one movie/show to the next so consider these values approximation, especially if the video bitrate is variable.
#29458
We're getting maximum 720p on Peacock today. We're on the free tier, so perhaps that's why. What are you guys getting from Peacock?

- With 'The Matrix' in 720p we're getting approx 4-5 Mb/s video bitrate (see screenshot below). Ignore the audio bitrate as the Apple TV HD is currently connected to stereo TV speakers only.
- With 'The Mummy' in 720p we're also getting approx 4-5 Mb/s average.

Like HBO Max, Peacock will not offer 4K HDR at launch:
Peacock launches today – without 4K HDR and Atmos


PS: Peacock app doesn't appear to support frame rate matching either...
Attachments
peacockvideoquality.jpg
peacockvideoquality.jpg (70.9 KiB) Viewed 181552 times
#30125
I just took these screenshots via Xcode to show the difference b/w HBO Max and HBO via Amazon Channels. As you can see, the newest shows/movies on HBO Max are on par or even slightly higher bitrate-wise versus HBO via Amazon Channels (e.g., See screenshots of Midway - 2019 movie). However, with older shows like Sopranos and Entourage, Amazon is streaming HBO content at twice the bitrate. HBO max is in the 5-6Mbps range.
Attachments
HBO via Amazon Channels Sopranos Season 6 Episode 11.png
HBO via Amazon Channels Sopranos Season 6 Episode 11.png (116.44 KiB) Viewed 180518 times
HBO via Amazon Channels Midway 2019 Movie.png
HBO via Amazon Channels Midway 2019 Movie.png (115.51 KiB) Viewed 180518 times
HBO via Amazon Channels Entourage Season 1 Ep8.png
HBO via Amazon Channels Entourage Season 1 Ep8.png (116.19 KiB) Viewed 180518 times
HBO Max Sopranos Season 6 Episode 11.png
HBO Max Sopranos Season 6 Episode 11.png (112.47 KiB) Viewed 180518 times
HBO Max Midway 2019 Movie.png
HBO Max Midway 2019 Movie.png (115.3 KiB) Viewed 180518 times
HBO Max Entourage Season 1 Ep8.png
HBO Max Entourage Season 1 Ep8.png (113.05 KiB) Viewed 180518 times
#33319
I noticed Apple are deliberately throttling bit rates on iTunes movies outside ATV box. On my FireTV stick, it never exceeds 17 Mbps. Most of the time its stuck under 10Mbps.

I heard that the same stream via ATV box is hitting 31 Mbps with bit rates hovering in 20 Mbps.

This has caused me to dump my FireTV stick in favour of ATV box.
#33336
Lgans316: That's very interesting actually. Perhaps also a hardware limitation? I guess would be that FireTV stick doesn't support the same HEVC profiles as Apple TV 4K.

How do you enable the video console on FireTV Stick?
#33378
Via Developer Tools app mate.

Some more findings:

ATV box / MA / iTunes

Compared some movies shot on film on both MoviesAnywhere (MA) and iTunes. It is quite apparent that iTunes looks a smidgen softer consistently than MA which is doing a better rendition of film grain. Sometimes it feels like you are watching the disc via MA and a compressed stream via iTunes. Quite surprised and disappointed to find this. It's quite consistent and you don't need any trained eyes to discern this. Its not like night and day but its not too difficult to discern.

Either iTunes has some sort of DNR baked in or could it due to lower bit rates?

Please can someone with ATV and XCode check on this and feedback.