I also don't think this is a very good way to test the quality of the encoding. Since we don't have a reference (the uncompressed CD audio) what you're really testing is if any particular encoded version sounds the most appealing to the majority of people. That is not the same thing as an accurate representation of the original. Say for instance you would produce a truly horrible and annoying set of sounds. The encoding of that may remove some of the frequencies that are causing it to be horrible, and people would then rate that version higher. But it's not what you originally intended. A better way to test this is, in my opinion, to give us the uncompressed version to compare against. Then the uncompressed version would be among the compressed ones we compare with. Then you would see if people really could tell a difference or not.

An amplifier is only really needed if you have high impedance headphones like the Sennheiser above. Most sound cards aren't powerful enough to drive such headphones by themselves. For regular low impedance headphones it's not necessary to get an amplifier.