Why 24fps is not enough for HDR movies

[Rasmus Larsen]

Together with our Danish forum users on Flatpanels.dk we have been measuring video/audio quality on streaming services for the past 2 years. We're now starting the same initiative here on FlatpanelsHD.com and we hope that you guys will help as we don't have access to all streaming services.

We'll update the table in this post when new measurements are posted in this forum thread (you can create a forum user or use your social media account to log in).

How to measure video/audio streaming quality:
- You need: Apple TV 4K, Macbook/iMac, and the Xcode software. Both devices must be on the same network.
- On Apple TV go to Settings -> Remotes and Devices -> Remote App and Devices. Leave it here.
- Now open Xcode on your Mac and go to Window -> Devices and Simulators. Select your Apple TV 4K and pair it.
- On Apple TV go to Settings -> Developer (new menu will pop up once paired with Xcode) -> Playback HUD (enable)
- Start a video stream on Apple TV 4K, let it run for at least 3-5 minutes. Take a screenshot (via Xcode) or a photo with the measurements.

Last updated: July 15, 2020

[Rasmus Larsen]

Some of the measurements in the table are a little dated and some values are missing / approximation. We'll take additional measurements soon to fill out the gaps. Please contribute. Let's aim to expand the list significantly Together we can help put pressure on service to increase their video streaming quality!

[Sledgegamma]

Great Idea! Some input from me:

Amazon 4K HDR10+ is just under 15mbit for constant bitrate. Should also apply for HDR10 version.
Amazon 1080p AVC is 10mbit for CBR.
Amazon 1080p HEVC is 8-9mbit for CBR. However, I have also seen some with 3mbit, for example on "How to get away with murder" S01

Audio on both can differ, but its usually 640kbit. These are for TV shows but I assume they are valid for movies as well.

Why do you include the frame rate?

[Rasmus Larsen]

Thanks Sledgegamma. Will check again with Amazon. Is it possible that there's a difference between North America and Europe? Please upload measurements if possible.

We include frame rate because some streaming services - mostly companies entrenched in the EU broadcast ecosystem - have a bad habit of converting movies/series in 24fps to 25fps (via speed-up - some even ignoring audio pitch leveling).

[prodan]

HBO Max, Go and Now are essentially the same bitrate in the US now, at least on the Apple TV 4K. Content tested:
Joker
Westworld S03E01
The Sopranos S60E01
Lego Movie 2
Screenshots: https://ibb.co/album/Dg1821

[Sledgegamma]

Thanks Sledgegamma. Will check again with Amazon. Is it possible that there's a difference between North America and Europe? Please upload measurements if possible.

We include frame rate because some streaming services - mostly companies entrenched in the EU broadcast ecosystem - have a bad habit of converting movies/series in 24fps to 25fps (via speed-up - some even ignoring audio pitch leveling).

Yes, sadly that's true about the 25p version...
Don't know whether there are differences for North America and EU. That said, there are definitely difference regarding India with India having a lower bitrate. In addition, Amazon doesn't always use the same frame rate for all the regions. Sometimes its 24.000 and sometimes 23.976.

My measurement are media info output from the uncompressed/unaltered downloads of those TV shows.

Amazon: Sneaky Pete S01E01:

Code: Select all

General
Unique ID                                : 288806875305643181453618751556428061195 (0xD9462E51670FCF021627ACA14B315A0B)
Complete name                            : 
Format                                   : Matroska
Format version                           : Version 4
File size                                : 5.78 GiB
Duration                                 : 54mn 14s
Overall bit rate                         : 15.3 Mbps
Encoded date                             : UTC 2020-05-13 17:09:19
Writing application                      : mkvmerge v45.0.0 ('Heaven in Pennies') 64-bit
Writing library                          : libebml v1.3.10 + libmatroska v1.5.2

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5@Main
HDR format                               : SMPTE ST 2094 App 4, Version 0
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 54mn 14s
Bit rate                                 : 14.6 Mbps
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) fps
Standard                                 : NTSC
Color space                              : YUV
Chroma subsampling                       : 4:2:0 (Type 2)
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.073
Stream size                              : 5.53 GiB (96%)
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0001 cd/m2, max: 1000 cd/m2
Maximum Content Light Level              : 907 cd/m2
Maximum Frame-Average Light Level        : 821 cd/m2

Audio
ID                                       : 2
Format                                   : E-AC-3
Format/Info                              : Enhanced AC-3
Commercial name                          : Dolby Digital Plus
Codec ID                                 : A_EAC3
Duration                                 : 54mn 14s
Bit rate mode                            : Constant
Bit rate                                 : 640 Kbps
Channel(s)                               : 6 channels
Channel layout                           : L R C LFE Ls Rs
Sampling rate                            : 48.0 KHz
Frame rate                               : 31.250 fps (1536 SPF)
Compression mode                         : Lossy
Stream size                              : 248 MiB (4%)
Title                                    : English
Language                                 : English
Service kind                             : Complete Main
Default                                  : Yes
Forced                                   : No

Supernatural S15E13 from Amazon:

Code: Select all

General
Unique ID                                : 69770124182158676389742909699507562137 (0x347D3CB51D92D2AE1ACA6CBE0ACB2699)
Complete name                            : 
Format                                   : Matroska
Format version                           : Version 4
File size                                : 2.80 GiB
Duration                                 : 42mn 20s
Overall bit rate                         : 9 450 Kbps
Encoded date                             : UTC 2020-03-24 08:01:25
Writing application                      : mkvmerge v21.0.0 ('Tardigrades Will Inherit The Earth') 64-bit
Writing library                          : libebml v1.3.5 + libmatroska v1.4.8

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 42mn 20s
Bit rate mode                            : Constant
Bit rate                                 : 8 808 Kbps
Nominal bit rate                         : 10 000 Kbps
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.177
Stream size                              : 2.61 GiB (93%)
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio
ID                                       : 2
Format                                   : E-AC-3
Format/Info                              : Enhanced AC-3
Commercial name                          : Dolby Digital Plus
Codec ID                                 : A_EAC3
Duration                                 : 42mn 20s
Bit rate mode                            : Constant
Bit rate                                 : 640 Kbps
Channel(s)                               : 6 channels
Channel layout                           : L R C LFE Ls Rs
Sampling rate                            : 48.0 KHz
Frame rate                               : 31.250 fps (1536 SPF)
Compression mode                         : Lossy
Stream size                              : 194 MiB (7%)
Language                                 : English
Service kind                             : Complete Main
Default                                  : Yes
Forced                                   : No

Amazon: Joker

Code: Select all

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 4 frames
Codec ID : V_MPEG4/ISO/AVC
Duration : 2 h 1 min
Bit rate mode : Constant
Bit rate : 8 818 kb/s
Nominal bit rate : 10 000 kb/s
Width : 1 920 pixels
Height : 1 040 pixels
Display aspect ratio : 1.85:1
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.184
Stream size : 7.50 GiB (93%)
Default : Yes
Forced : No
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

Audio
ID : 2
Format : E-AC-3
Format/Info : Enhanced AC-3
Commercial name : Dolby Digital Plus
Codec ID : A_EAC3
Duration : 2 h 1 min
Bit rate mode : Constant
Bit rate : 640 kb/s
Channel(s) : 6 channels
Channel layout : L R C LFE Ls Rs
Sampling rate : 48.0 kHz
Frame rate : 31.250 FPS (1536 SPF)
Compression mode : Lossy
Stream size : 557 MiB (7%)
Language : English
Service kind : Complete Main
Default : Yes
Forced : No

For All Mankind HDR10:

Code: Select all

Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5@High
HDR format : SMPTE ST 2086, HDR10 compatible
Codec ID : V_MPEGH/ISO/HEVC
Duration : 1 h 5 min
Bit rate : 24.0 Mb/s
Width : 3 840 pixels
Height : 1 920 pixels
Display aspect ratio : 2.000
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0 (Type 2)
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.136
Stream size : 10.9 GiB (95%)
Language : English
Default : Yes
Forced : No
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0050 cd/m2, max: 1000 cd/m2
Maximum Content Light Level : 1833 cd/m2
Maximum Frame-Average Light Level : 744 cd/m2

Audio
ID : 2
Format : E-AC-3 JOC
Format/Info : Enhanced AC-3 with Joint Object Coding
Commercial name : Dolby Digital Plus with Dolby Atmos
Codec ID : A_EAC3
Duration : 1 h 5 min
Bit rate mode : Constant
Bit rate : 768 kb/s
Channel(s) : 6 channels
Channel layout : L R C LFE Ls Rs
Sampling rate : 48.0 kHz
Frame rate : 31.250 FPS (1536 SPF)
Compression mode : Lossy
Stream size : 358 MiB (3%)
Language : English
Service kind : Complete Main
Default : Yes
Forced : No
Complexity index : 16
Number of dynamic objects : 15
Bed channel count : 1 channel
Bed channel configuration : LFE

[WillzyxTheZypod]

Together with our Danish forum users on Flatpanels.dk we have been measuring video/audio quality on streaming services for the past 2 years. We're now starting the same initiative here on FlatpanelsHD.com and we hope that you guys will help as we don't have access to all streaming services.

We'll update the table in this post when new measurements are posted in this forum thread (you can create a forum user or use your social media account to log in).

How to measure video/audio streaming quality:
- You need: Apple TV 4K, Macbook/iMac, and the Xcode software. Both devices must be on the same network.
- On Apple TV go to Settings -> Remotes and Devices -> Remote App and Devices. Leave it here.
- Now open Xcode on your Mac and go to Window -> Devices and Simulators. Select your Apple TV 4K and pair it.
- On Apple TV go to Settings -> Developer (new menu will pop up once paired with Xcode) -> Playback HUD (enable)
- Start a video stream on Apple TV 4K, let it run for at least 3-5 minutes. Take a screenshot (via Xcode) or a photo with the measurements.

It looks like the minimum bitrate for iTunes 4K HDR10 and iTunes 4K DV are 6 Mbps higher on the Danish site than in this thread, but the maximum bitrate is 6 Mbps higher in the U.S. That is an interesting result.

[Rasmus Larsen]

WillzyxTheZypod: Should be the same stream. I think we just used some other TV series for reference on the Danish forum. Will check again and see if it needs correction. It will, however, vary from one movie/show to the next so consider these values approximation, especially if the video bitrate is variable.

[Rasmus Larsen]

We're getting maximum 720p on Peacock today. We're on the free tier, so perhaps that's why. What are you guys getting from Peacock?

- With 'The Matrix' in 720p we're getting approx 4-5 Mb/s video bitrate (see screenshot below). Ignore the audio bitrate as the Apple TV HD is currently connected to stereo TV speakers only.
- With 'The Mummy' in 720p we're also getting approx 4-5 Mb/s average.

Like HBO Max, Peacock will not offer 4K HDR at launch:
Peacock launches today – without 4K HDR and Atmos

PS: Peacock app doesn't appear to support frame rate matching either...

[bijan.pingili]

I just took these screenshots via Xcode to show the difference b/w HBO Max and HBO via Amazon Channels. As you can see, the newest shows/movies on HBO Max are on par or even slightly higher bitrate-wise versus HBO via Amazon Channels (e.g., See screenshots of Midway - 2019 movie). However, with older shows like Sopranos and Entourage, Amazon is streaming HBO content at twice the bitrate. HBO max is in the 5-6Mbps range.

[Lgans316]

I noticed Apple are deliberately throttling bit rates on iTunes movies outside ATV box. On my FireTV stick, it never exceeds 17 Mbps. Most of the time its stuck under 10Mbps.

I heard that the same stream via ATV box is hitting 31 Mbps with bit rates hovering in 20 Mbps.

This has caused me to dump my FireTV stick in favour of ATV box.

[Rasmus Larsen]

Lgans316: That's very interesting actually. Perhaps also a hardware limitation? I guess would be that FireTV stick doesn't support the same HEVC profiles as Apple TV 4K.

How do you enable the video console on FireTV Stick?

[Lgans316]

Via Developer Tools app mate.

Some more findings:

ATV box / MA / iTunes

Compared some movies shot on film on both MoviesAnywhere (MA) and iTunes. It is quite apparent that iTunes looks a smidgen softer consistently than MA which is doing a better rendition of film grain. Sometimes it feels like you are watching the disc via MA and a compressed stream via iTunes. Quite surprised and disappointed to find this. It's quite consistent and you don't need any trained eyes to discern this. Its not like night and day but its not too difficult to discern.

Either iTunes has some sort of DNR baked in or could it due to lower bit rates?

Please can someone with ATV and XCode check on this and feedback.

[kjkjk32]

I thought Netflix was broadcasting at 60 FPS?
Netflix's broadcast quality 13mbps to 4K is very low.

Have no desire to improve image quality?

[kjkjk32]

1.When will it be possible to see AV1 encoder in streaming?

2.Can anyone check the quality of Eurosport streaming on the site?
https://www.eurosport.com/

3.Streaming ATPTour.com What is their broadcast quality at 60 FPS?
https://www.atptour.com/en/video/watch-live