Well, it's been a while but I thought I'd give a final update.
I decided to just make the fronts sound as best they can. Using tones, I set the crossover to as low as I could go before they started losing sound. That turned out to be around 60, when I went lower, they just didn't play it. Then I adjusted the sub amp to the same point tones.
I wonder if time has anything to do with this, do speakers wear in? It sounds better now than ever. Drums from the front punch, loud, the bass is perfect and the high end is crisp and clear. My system does not have much power, but the sound quality is what I wanted, and have.
My head unit blew up, I think that cheap Dual amp for the subs did it. It was making popping noises for the last year. Now I'm just using the KAC-8405 for the fronts and subs, and I swear it sounds better. I replaced the KDC-X397 with a KDC-X302, this one plays .flac so no more .mp3's, no more lossy music.
I really want to thank everyone here, all the advice helped me to achieve my goal, quality sound.