The sub should be set no more than half way. You can tell if it was too high by looking at the trims after running Audessey. If the sub trim is a large negative number (-10 or more), you should adjust the sub's volume down and rerun Audessey.

On the Onkyo's (and I assume the Denon), the volume makes no difference. The test tones are set at -30db from reference (or 75db actual).

Dave
Audyssey Setup Guide