|
Post by donjoe on Aug 9, 2016 7:57:05 GMT
I started this discussion on Ethan's YouTube channel, but as he said a forum is probably a better place to continue it, so here goes: Posts so far: I see at the time of this workshop you were still recommending the evaluation of audio equipment based on metrics like THD and IMD. But what about the Geddes and Lee papers from the 2003 AES conference, where they showed THD and IMD to be nearly worthless because of the nonlinearity of the human perception of distortion? (After which they proposed their own distortion metric which supposedly - based on a small experiment - achieved good correlation with listeners' subjective assessments of distortion, unlike THD and IMD.) Standard THD and IMD measurements are hardly worthless, but I agree they don't tell the whole story. The main problem is those test are "static" and measure at only one signal level, where music is constantly changing in both level and frequency. I'm aware of the Geddes and Lee work, but I don't think I have those papers. If you'd like to email me from my web site ethanwiner.com/ maybe you can send some relevant portions or at least a summary? You may be interested to know that I'm currently designing a test device that assesses audio fidelity in a way that's guaranteed to relate to audibility. Oh those are both available for free on the GedLee website: gedlee.com/downloads/Distortion_AES_I.pdfgedlee.com/downloads/Distortion_AES_II.pdfAs I understood it, their contention is that the THD and IMD measurements are deficient because they use pure tones rather than complex (wide-band) sounds, and this is not helped much even by sequentially traversing the whole audible spectrum and re-measuring the distortion for every frequency emitted as a pure tone. I'll be curious to hear how your device performs once you go public with it, but as a consumer I also wish all these improved measures of audio performance were already available and published for all devices on the market so I could better decide what not to throw my money on. LOL, now that I see where to look, I realized I already have these papers, both are from 2003. I see no reason why an amplifier that can play 100 Hz well, and also 10 KHz well, can't play both frequencies at once just as well. Or any other combination. I'll look at those papers again, but 1) I think standard tests are perfectly fine because they can identify crap products like those boutique toob amp having 5% distortion. And 2) these days pretty much all active audio devices (not microphones or speaker drivers) are clean enough to not be an issue, so specs are mostly moot. Over the past 20 years I've bought audio gear based mainly on features and price, not specs. :->) One important and unique feature of my upcoming device is its companion signal generator plays all frequencies between 10 Hz and 24 KHz at once, and you can optionally use music as a test signal. So if an amp is clean with static tones but goes into slew rate limiting on brief transient peaks, as one example, that will show as a sputtering type noise that can be seen, and also heard, or even recorded to a Wave file and analyzed in any audio editor that has an FFT display. I'm glad to discuss all this stuff in depth, but YouTube comments isn't a good venue for that. You're welcome to move this to my Audio Expert forum: the-audio-expert.freeforums.net/Not entirely sure if that's the right forum for it - this is primarily a question of human perception and only if it gets settled as such can it translate into audio technology design principles or parameters. "I see no reason why an amplifier that can play 100 Hz well, and also 10 KHz well, can't play both frequencies at once just as well." Maybe it can, but if some distortion is present in each single-tone case, how much distortion will the listener hear with both tones and both their distortion spectra present? Maybe some distortion components mask eachother when they reach the human perceptual apparatus and maybe others amplify eachother. I think this is more Poppy Crum's territory. The Audio Central section of my forum is appropriate for anything related to audio or music. I'm sure there are situations where one thing or another masks something else. But that happens in the music itself much more than distortion created by competent electronics. Distortion of 0.01 percent or better is common, and that puts the artifacts 60 dB or more below the music. Versus a guitar and piano both playing an A chord at the same time. Or a violin and clarinet playing the same melody in octaves.
|
|
|
Post by donjoe on Aug 9, 2016 8:08:16 GMT
So... getting back to my original idea, that of saving money by trying to buy the cheapest audio equipment that's just good enough to keep its distortion and noise just below the threshold of audibility: I do realize that imposing stricter limits on the values of THD and IMD can still get you equipment with distortion that's low enough to be inaudible, but if Geddes and Lee are right and THD and IMD are as weakly correlated with subjective perceptions of distortion (r=0.423 for the former and r=0.345 for the latter), that still means I might be spending more than I need to and getting significantly better-performing devices than I need to, just because I'm using poor metrics that need to be pushed to extreme values to give me any guarantee. Whereas if I were using something like the GedLee metric (Gm) I could focus more precisely on the point of diminishing/inaudible returns, as that metric purportedly has quite a strong correlation with perceived distortion (something in the ballpark of r=0.68 to 0.9; it seemingly performs better for lower levels of distortion).
At first sight it's a bit strange to see that despite a better distortion metric having been published and explained more than a decade ago the industry is still giving no signs of wanting to adopt it, though, as someone else said in the DIY Audio forum, at least two major things could explain this: - THD and IMD are well established, they're pretty easy to measure and calculate and everyone knows how to do it; Gm is a new metric with more complex math; nobody likes to switch to more complicated math - it could well be in all the producers' interest to keep things as hazy as they are when poorly correlating metrics are used, as that helps sell even audio devices that aren't really up to par, just because their THD and IMD specs make them look just as good as others that really do offer less perceptible distortion.
|
|
|
Post by Ethan Winer on Aug 9, 2016 17:45:09 GMT
Thanks very much for moving this here. I understand your concern, and I agree completely with the desire to spend as little as possible to obtain a desired level of performance. But I think the Geddes/Lee approach is overly complicated and doesn't really solve that problem. Though I'd love to hear from ArnyK if he has time to comment. To my way of thinking, distortion and aliasing, and even noise, all fall under the concept of artifacts. Regardless of the cause or source, what affects artifact audibility is 1) relative volume compared to the original source, the overall volume (SPL), and the masking effect. My Artifact Audibility article is similar to the tests in my AES Audio Myths video, and it lets people hear for themselves how loud a nasty sounding and highly obvious artifact must be in order to hear it. As these demo files show, once you get much softer than -60 dB or so, artifacts are inaudible regardless of their absolute volume and masking. Adding an extra 20 dB safety for "golden" ears, that tells me that an IMD measurement of 0.01 percent is a safe guarantee of the device in question not harming audio quality by a noticeable amount. Sure, and I'm the first to claim that detailed math is not needed for most things related to understanding audio. As part of my audio test device I'm currently developing, I need to design four-pole filters and related circuits, but even that isn't nearly as taxing as the math I saw in the Geddes/Lee papers! I tend to doubt that. It's easy to make high quality transparent audio devices, and any company that knows what they're doing makes good stuff and has no need to obfuscate or be dishonest. Yes, there are plenty of lame companies, mostly in the boutique hi-fi world, but those companies - and their customers - don't believe in specs anyway. Why else would someone pay $3,000 for a 6-foot piece of wire that's demonstrably identical to a wire costing $3 at any hardware store. --Ethan
|
|
|
Post by donjoe on Aug 9, 2016 19:28:07 GMT
Actually, I've taken a closer look at their data in the meantime and I've realized that their p-values for the correlation coefficients of both THD and IMD indicate probabilities below the minimum traditionally required for "statistical significance" in the scientific community (and more recent discussions of p-values by expert statisticians have revealed that even this minimum is very very generous), i.e. p=0.05 (for THD they got a correlation with p=0.06 and for IMD with p=0.13). Even at p=0.05 there is still a probability of at least 23% (but typically 50%) of being wrong in claiming you have detected a real effect. So at p=0.06 and p=0.13 it's like they've found out nothing at all about how well or how badly THD and IMD correlate with perceived distortion - these correlations could each be much lower or much higher than the stated r values. So... since they haven't really proven that THD and IMD are weak metrics with such a small sample of stimuli (21 different types of distortion), they can't be said to have proven Gm to be a better metric either. It doesn't mean that THD or IMD are weak or that they're strong predictors of quality, they just haven't proven anything either way with this study taken alone. Damn. Back to square one, I guess.
|
|
|
Post by arnyk on Aug 10, 2016 14:24:05 GMT
I started this discussion on Ethan's YouTube channel, but as he said a forum is probably a better place to continue it, so here goes: Posts so far: I see at the time of this workshop you were still recommending the evaluation of audio equipment based on metrics like THD and IMD. But what about the Geddes and Lee papers from the 2003 AES conference, where they showed THD and IMD to be nearly worthless because of the nonlinearity of the human perception of distortion? (After which they proposed their own distortion metric which supposedly - based on a small experiment - achieved good correlation with listeners' subjective assessments of distortion, unlike THD and IMD.) Geddes and Lee happen to be long term friends that I see socially several times a year. We know each other's kids, for example. The first thing to consider is that Earl is primarily interested in speakers and his first cut on the performance of electronics gear is that it is usually perfect for all practical purposes. His systems are usually based on mi-priced receivers, and one of his hard-to-argue-with assertions is that if your speakers are efficient like most if his, you don't need more in the way of electronics but mid-fi digital players and amps (receivers). So, trying to use Gedde's ideas and papers to argue that electronic gear has audible distortions that don't show up in traditional measurements completely misses his point and perverts his ideas. His assertion is that it will be along time before we have speakers that are as subjectively ideal or as good as inexpensive electronics gear, and so we need to have distortion measurements that allow us to predict subjective performance and rank electroacoustic products like speakers effectively. I agree that there are giant holes in the usual interpretation of THD and IM distortion when they are audible, but its easy to determine experimentally that outsize of clipping and the like, good average audio gear is free of audible distortion. Most electronic audio gear THD and IM specs are to me just fancy and possible overly-complex ways way to say "No audible distortion - fuhgeddaboutit"
|
|
|
Post by donjoe on Aug 10, 2016 18:20:45 GMT
So, trying to use Gedde's ideas and papers to argue that electronic gear has audible distortions that don't show up in traditional measurements completely misses his point and perverts his ideas. I don't know where you got the impression that I was doing anything of the kind. I took his argument to be one that traditional measurements don't show enough of what they claim to show and thus force us to get far better and more expensive gear than we really need to, when there's a wide selection of cheaper stuff out there that could serve us just as well.
|
|
|
Post by arnyk on Aug 11, 2016 8:47:33 GMT
So, trying to use Gedde's ideas and papers to argue that electronic gear has audible distortions that don't show up in traditional measurements completely misses his point and perverts his ideas. I don't know where you got the impression that I was doing anything of the kind. From the following: The first problem with what's written above is that its perfectly general to all audio gear and stated with no limits or qualifications. Geddes papers generally only apply to speakers. Secondly, some actual study of the paper's titles seems to be in order. They are: "Auditory Perception of Nonlinear Distortion..." The papers are talking about Nonlinear Distortion which is one of the three traditional kinds of measured faults that audio gear can possibly have. The problem are not the measurements, but how they are interpreted. Geddes calls the ways they are interpreted Metrics. The three kinds of measurable faults in audio gear are: (1) Linear Distortion which includes things like frequency response amplitude and timing errors like phase shift. (2) Nonlinear Distortion which includes things like THD, IM, FM Distortion and Jitter. (3) Interfering Signals which includes tics, pops, hiss, hum, and EMI. Everything we don't like about how audio sounds to us fits into these three categories. Mathematically, there can be no other errors.
|
|
|
Post by Ethan Winer on Aug 11, 2016 21:16:22 GMT
Good discussion guys. I'll just add that in my list of "four audio parameters" I separate out timing errors: wow & flutter and jitter. It looks like Arny puts them under Linear Distortion, which is fine too.
--Ethan
|
|
|
Post by arnyk on Aug 12, 2016 1:09:54 GMT
Good discussion guys. I'll just add that in my list of "four audio parameters" I separate out timing errors: wow & flutter and jitter. It looks like Arny puts them under Linear Distortion, which is fine too. --Ethan Wow and flutter are two names for similar kinds of nonlinear distortion. They are both FM distortion. Wow is what you get when the frequency modulating frequency is very low, and flutter is what you get when the frequency modulating frequency is a little higher. They are also kinds of jitter which is FM distortion.
|
|
|
Post by donjoe on Aug 12, 2016 9:34:08 GMT
some actual study of the paper's titles seems to be in order. They are: "Auditory Perception of Nonlinear Distortion..." The papers are talking about Nonlinear Distortion which is one of the three traditional kinds of measured faults that audio gear can possibly have. Firstly, I don't know why you're taking this aggressive tone with me, and secondly I don't see the point in listing all the types of distortions that are outside the focus of the above papers. Last I was here I was still talking about the Geddes & Lee papers from 2003, and specifically about what they prove or don't prove about the utility of THD and IMD vs. G_m in the context of trying to buy the cheapest equipment (or just speakers if you really want to restrict it to that) that has no audible distortion. What are you talking about? I find it hard to discern.
|
|
|
Post by arnyk on Aug 12, 2016 18:10:02 GMT
Firstly, I don't know why you're taking this aggressive tone with me, I am attempting to correct your understanding of the Geddes papers To make the point of how incomplete your understanding of the focus of the papers is. I'm trying to get you to understand that "the cheapest equipment (or just speakers if you really want to restrict it to that) that has no audible distortion" is not the context or purpose of the papers. It might be easier for you to understand what I'm talking about if you opened your mind to what the Geddes papers are actually about.
|
|
|
Post by MI Pro on Jun 6, 2018 0:38:48 GMT
Hello Everyone, I have posted a Group Buy (GB) announcement for audio analyzer software that does the following: - Traditional, old fashioned, and non-relevant (IMO) THD and IMD, but far more importantly, the vastly more modern and relevant non-coherent distortion (NCD) as well as the Dr. Earl Geddes "GedLee Metric" (Gm distortion). - If you own an RTX6001 audio analyzer this software has a custom API that controls the RTX's internal attenuators and gains to allow for true autoranging and autoscaling (which is a HUGE deal). - Even if you don't own an RTX6001 and instead use a soundcard type of approach, the new distortion and other features of this software will prove to be very useful to many IMO. You can read more about this GB audio analyzer software at: bit.ly/2LdMtB9 and you can read a bit of discussion about it here: bit.ly/2spOgvI
|
|