Saturday, 19 April 2014

INTERNET TEST: 24-bit vs. 16-bit Audio - Can you hear the difference?

24-bits vs. 16-bits Audio - A Visual Analogy?
Those that have been reading this blog for awhile will recall that these pages started with an MP3 vs. Lossless test that was posted here back in 2012. That was kinda fun :-).

As you know, the "High-Resolution Audio" movement is on. A major cornerstone of this is the belief that in the PCM world, 24-bit audio resolution imparts clear audible benefits (as opposed to the standard 16-bits for CD resolution). Indeed, most decent DACs these days are capable of measuring >16-bit dynamic range. Clearly, a push is being made for the subjective "virtues" of 24-bit audio for your headphones, into your home and into your car. The 8-bits of difference between 16-bits and 24-bits of PCM data provides 48dB of extra dynamic range or 256x the number of values to represent each PCM sample! Fantastic advertising buzz. The 24-bit "container" is certainly capable of significantly higher resolution.

Here's the question... Can you hear the difference? Evidently many people believe the difference is audibly significant.

So, you've tried "crowd funding" (and you're probably at this point still waiting for a product), it's time for another round of "crowd testing", folks! Here's what you do to participate:

1. Download the "24-bit Audio Test.zip" file:

WARNING: The test file is big (a taste of the storage demands for those who have not downloaded high-resolution audio). Approximately 200MB for a total of 6 musical samples at 24/96 lasting less than 12 minutes with fully tagged FLAC lossless compression.

Get the file from my FTP server:
ftp://24bit-test.dyndns.org
Login = 24bit
Pass = test

Please have patience if the server load is high... Anyone able to help out with this, please drop me a note below!

Alternate download sites:

Thanks Ingemar:
http://www.privatebits.net/24-bit_Audio_Test_(Hi-Res_24-96-FLAC_2014).zip 

From Uploaded.net: http://ul.to/fqziwlfs
From FilePost.com: https://filepost.com/files/m5b3em11/24-bit_Audio_Test_(Hi-Res_24-96,_FLAC,_2014).zip/

2. Extract that ZIP to wherever you want for playback (computer folder, music archive, server, etc.)

Located within are 3 musical pieces in 24/96 FLAC, each piece with Sample A or Sample B versions. One of them (A or B) is the 24-bit original and the other contains a dithered 16-bit version which has been converted back to 24/96 so your DAC will basically be playing back the same bit/samplerate when you switch between tracks.

The samples are all classical pieces but with variation in instrumentation, vocals, and dynamics. Realize that it's not easy to find good high-resolution audio where the music is recorded and mixed to the highest standards with known provenance. Classical music as a genre is where some of the best recordings can be found.

Here are the tracks:

1. Eugène Bozza - la Voie Triomphale (performed by The Staff Band of the Norwegian Armed Forces): A well recorded orchestral track originally in DXD (32/352.8) freely downloadable as a sample from 2L here. In the interest of download size, I extracted 2 minutes from the track. It features good dynamic range with a DR13.

2. Vivaldi - Recitative and Aria from Cantata RV 679, "Che giova il sospirar, povero core" (performed by Tone Wik & Barokkanerne) - String orchestra with female vocals. Also DXD-recorded and available as a free download from 2L here. Again, I only extracted 2 minutes from the track. DR14 for this track.

3. Goldberg Variations BWV 988 - Aria (performed by Kimiko Ishizaka) as taken from the freely available Open Goldberg Variations 24/96 release. The recording was done at Teldex Studio in Berlin using the Bösendorfer 290 Imperial CEUS concert grand piano. Simple instrumentation for those who love and appreciate the sound of the piano. It's also a much slower piece which provides an opportunity to listen to the decay quality. Low-level spatial room acoustics also easily heard on this recording. Measured dynamic range is a reasonable DR12.

I've included the DR printout from foobar to demonstrate that peak and average volumes are identical for Samples A & B for each piece.

** Note that I'm using the samples under the principle of "fair use" for the purpose of education, research and commentary. I have no ties with Industry and have no financial gain by organizing this. As noted above, these tracks were selected based on excellent quality of the recording and I would highly recommend visiting the source links to sample more high-quality audio tracks in the high-resolution delivery formats (the 2L download page also includes DXD, multichannel and DSD samples).

3. Listen and compare Sample A with Sample B.
Can you hear a difference? Can you tell which one was the original 24-bit audio and which was dithered down to 16-bits?

Needless to say, you need to make sure your DAC / computer / streamer / etc. is capable of >16-bit performance!

I encourage the use of tools like foobar's ABX comparator to switch quickly between A & B. Make note of which Sample for each piece of music you believe is the original 24-bit track. Presumably, since the 24-bit version is higher resolution than the 16-bit one, there could/should be improved transparency, ambiance, definition, smoother decay, etc... For example, if you think the 24-bit sample is A for Bossa, B for Vivaldi, and A for Goldberg; keep track of that and make note of how confident you feel about your selection.

4. Make your voice heard on my survey form!


I have 14 questions (15th optional) - it should not take long to fill out as most are multiple-choice tick boxes. As expected, I want to know whether you think A or B is the 24-bit track. I also want to know if you're guessing or confident - if you spent time listening, make it count by visiting the survey even if you don't think you can tell a difference. A 'negative' report is just as important as a 'positive' one. I also want to get a sense of your age, gender, tiny bit on musical and technical experience. Also, equipment used, and approximate cost of the audio gear. All data is anonymous and I have no access to your IP address (the web site will keep track of what country you're submitting the survey from - let's make this an international effort!).

I'm going to be busy with work and other responsibilities for the next while so this is the perfect time to gather some data. As I did with the MP3 test, I'll summarize the information and demonstrate the conclusions of this survey once it closes. I think about 2 months is adequate time for everyone who wants to get involved to have a good listen. Therefore, I will close the survey around June 20, 2014 - fill in the survey before that time! Note that IP filtering is ON so only 1 response from each IP please.

Finally, remember to relax, take your time, and have fun with this... Enjoy the music and see if one version "speaks" to you more than the other in terms of sonic quality. Feel free to share this test with friends / family / music lovers / audio reviewers / audio forums / enemies. Also, please try not to whip out the audio editor before doing the listening and completing the survey! Honesty is extremely important for an open "naturalistic" survey like this one. If you know which is which, please do not share it with others so as not to bias the results. Feel free to leave a comment down below if you run into any problems or need help. Thanks...

A few guys got to "beta-test" this little project and I want to thank 'Wombat' and 'Mnyb' especially for providing technical suggestions and detailed feedback on the Squeezebox Audiophile forum before this went "live"!

--------------------

I've been enjoying some Ladysmith Black Mambazo over the last few evenings at home. As usual, with some of the more recent CDs I wish the dynamic range were higher especially for multilayered vocal music like this. I'll be sure to update when I get stuff like the pre-ordered Geek Out in my hands over the next while or if I've got some burning thoughts to share :-).

Enjoy the music... Happy Easter.

-------------------------
UPDATE (April 24, 2014): ~1 week into the test...

Great to see the responses so far! Just with the FTP site, >120 uploads of the file already. I of course do not expect that many responses to the survey yet since there's plenty of time to listen, but I am impressed by the quality of responses so far and the variety of gear people are using. Clearly folks are not using "junk" gear to assess and the majority are spending time listing out for me what they use and subjective impressions. All will be revealed in time of course...

Keep the responses coming!

--------------------------
UPDATE (April 30, 2014):

>50 responses now. Need more for better statistical power; especially in analyzing each test piece. Expanded the advertisement to the Audio Circle "HiRez Music Circle".

May 2, 2014:
Formal invitation extended to WiredState and Steve Hoffman forum.
May 10, 2014:
Invitation to Pink Fish Media forum.
May 17, 2014:
Invitation to Head-Fi forum.

20 comments:

  1. just fyi, the 'visual analogy' isn't accurate- the wordlength-reduced image has been truncated without dither. the quantization error is apparent in the banding artifacts. it's not really possible to provide an analogy to audio dither with onscreen images (even though the math and sampling theorem is exactly the same) unless your monitor has a ridiculously high pixel density. we can completely randomize quantization error in 8-bit, 4-bit, even 1-bit images but at such small pixel resolutions there is not much 'room' to spread the noise. a proper 1/2 LSB (~3 dB) dithered 1-bit image at a resolution of only a few hundred pixels each axis would look almost like pure noise. however if we dither a large image- a few megapixels maybe- to 1 bit, we can see that banding is entirely eliminated and we are seeing the image exactly as it was originally in 8, 16, 32 bit, whatever- plus the added noise. keep increasing the pixel resolution and eventually you will hit transparency... with only a singe bit. on or off.

    if we want to retain some semblance of the original image at small pixel resolutions we can't apply optimal 1/2 LSB dither as this would obscure the content almost completely. It's similar to dithering a piece of audio to 1-bit with only 22kHz bandwidth (as in CD audio)- the noise will be so loud we only have a few dB of practical snr. this is of course why DSD requires several MHz bandwidth to function at a hi-fi level. ok, so the analogy isn't quite perfect... but it's close enough.

    then again maybe this was the point you were making with the visual 'analogy'?

    in theory and in practice, when optimal dither is applied to a digitzed sine wave before truncation, the reduced wordlength file will be identical to the original plus the added noise- i.e. a 24 bit audio file dithered to 16 bit (96 dB snr w/ flat dither, ~120 dB practical snr with shaped dither) will sound exactly the same when listened to at a level where the audio peaks at below the threshold of hearing damage (~120 dB). even in the quietest studio (~20 dBA ambient noise), assuming worst-case unshaped dither, setting 0 dBFS peaks for 120 dB would bring the noise floor only 4 dB above that of the room's.

    in a quiet home environment (35-50 dBA), forget it... even a flat dithered noise floor will be buried a good 10-20 dB below your room's. i suspect 16 bit is enough for all home listening setups.

    sorry for the long post!

    ReplyDelete
    Replies
    1. No worries, anonymous. Good technical discussion...

      This is why I put that "?" in place for the analogy comment. Indeed, the difference between 16 & 24 bit audio will not be perceived as the image "shows"... And dither will remove that "appearance". Just that I suspect many audiophiles have that kind of image in mind when talking about 16 vs. 24-bit resolution.

      Delete
    2. hi armichago.
      I've been doing loads of work encoding, trailing, testing multi cannel music traces for playing directly from tablets, slim port nexus 4 and nexus 7 2013 direct via hdmi cable to av amps and have some really good case studies that would be good to talk about.
      if you do get this reply, please mail me on xtrojans@hotmail.co.uk

      Delete
    3. just quick addition.
      rather than streaming, these small inexpensive android tablets can output true multi channel tracks via hdmi directly to our av amps.
      heres a file, audio, bit rates sampling is added

      Size: 92.9 MB
      Format: QuickTime/MPEG-4/Motion JPEG 2000 format
      Stream count: 1
      Duration: 01:07:09
      Bit rate: 193 kb/s
      Meta info:
      compatible_brands: mp42isom
      minor_version: 0
      creation_time: 2014-01-09 14:40:01
      major_brand: mp42

      Stream 0
      Type: audio
      Codec: aac
      FourCC: mp4a
      Duration: 01:07:09
      Bit rate: 191 kb/s
      Sample rate: 44100 Hz
      Sample format: s16
      Channel: 6
      Meta info:
      creation_time: 2014-01-09 14:40:01
      language: und

      Delete
  2. Glad to see that the Open Goldberg Variations served well for this comparison! Fortunately, anyone can choose to have the 16 or 24 bit versions of the whole recording for free - with a creative commons license. Enjoy!

    ReplyDelete
    Replies
    1. Absolutely. I'm quite impressed by the recording and it's great to have this in my music collection!

      Delete
  3. Hi Archimago.

    I just started a thread on Cro Audiophile Forum linking your test/survey, so I hope you ca expect a few clicks from the Adriatic sea ;)

    http://www.audiofil.net/forum/forum_posts.asp?TID=13930


    One suggestion for the future.

    One can not accidentally run the files through spectral frequency analyzer, but it is impossible not to notice the difference in file sizes. Once one notice the difference the test is not so objective any more, whether the larger file is "original" or not. As you probably know, most people believe that bigger is better :)

    So my suggestion is that next time, you run the files through XLD Lossless Decoder/Converter and convert files to wav for example. The files will be exactly the same so one wont be puzzled or misled by the file size ;)

    ReplyDelete
    Replies
    1. Indeed... Different files sizes... But which is which? Maybe I have accounted for this :-)

      Delete
  4. Wonder how your findings will compare with this one: http://www.aes.org/e-lib/browse.cfm?elib=15398

    Best

    Vincent

    ReplyDelete
    Replies
    1. Remember Vincent - that AES paper is 44kHz vs. 88kHz. Only 3/16 individuals achieved significance according to the discussion here:
      http://www.hydrogenaudio.org/forums/index.php?showtopic=82264

      It's also not clear to me if hearing a difference reflects actual preference towards the higher sample rate.

      This test here of course just looks at the variable of bit-depth. Will be interesting what the results look like when all is said and done :-)

      Delete
  5. Survey submitted. Can you post the mp3 vs lossless test files to the FTP server? I'd like to test myself with them without reading the results.

    ReplyDelete
    Replies
    1. Hi Olah - will put it up tonight!

      Delete
  6. Hi man!

    What software Do you suggest on OSX? The ABX tester don't aceppt flac files...

    Regards from Brazil,

    Pablo

    ReplyDelete
  7. I'm pretty sure you're not going to get meaningful results with these high-res selections of music because none of them uses greater than 16-bits of dynamic range. The largest dynamic range that I measure (not including the fade in/out to silence) was around 60 dB...which 10-bits can handle. How can we evaluate for more dynamic range if it isn't present in the examples?

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
  8. Please continue to write more because it’s unusual that someone has something interesting to say about this.
    Will be waiting for more! To get new information visit here
    Test and Tag brisbane
    rcd testing services brisbane

    ReplyDelete
  9. It is truly a great and helpful piece of information.
    I am satisfied that you simply shared this useful information with us.
    Please stay us informed like this. Thanks for sharing. I know something information, to know you can click here.... rcd testing services brisbane
    Test and Tag brisbane

    ReplyDelete
  10. This site is good because they give us a new thing and new ideas and new topic how good all of they are we should appreciate them because of these good thing.
    I know something information, to know you can click here
    Test and Tag brisbane
    rcd testing services brisbane

    ReplyDelete
  11. Where are the survey results?

    ReplyDelete