ABX

Hearing is a matter of translating the movement of air molecules in a signal to the brain.
There it is processed.
Basically we do have a build in mixer. It mixes the sound and all our cognitions into a perception. Hence if we believe something do makes a difference, we will hear it.
If it is about sound quality, you don’t want to have any interference.
You want to judge the sound and the sound alone.
This is obtained by removing all our cognitions  from the test.
This can be done by blind testing.
In case of hardware this is hard to obtain.
In case of software there are programs available, most of the time ABX testing.
More details about our perception can be found here.

ABXtest

You can create your own listening tests by creating a simple configuration file, saving it to a publicly hosted location along with the audio samples and sending the participants link with the configuration file.

abxtests.com - Listening Tests in the Browser - jaakkopasanen

Source code - GitHub - jaakkopasanen/ABX: Web app for AB and ABX listening test

Foobar

Download the ABX Comparator plug-in.

Lacinato

ABX/Shootout-er blind testing audio software