Phase as nothing to do with output of a single driver. It also has nothing to do with clarity on a single driver.
Key word there is single...
In other words, shut off ALL of your other speakers and listen only to one midrange and then the other. Then see if they are different. Unplug the sub/tweeters as well. ONLY one driver. This is always one of the first steps in any tuning.
Once you have optimized/equalized how things sound then play with swapping the phase and normal phase listening to only the mids, then to only the tweeters doing the same test, then to the tweeters and mids together. Try every variation. Your ears will know what to do.
If you have time alignment do that first. Figure out what you like. Write it down. Then redo the test at both a little less and a little more time alignment. Keep iterating until you are sure.
And yes that is hours of work, err fun