In regards to the original question from the guest that came over from little brother land:
Now that we've established that your premise of why you claim you don't want the game is false; that A&M's schedule is too tough and/or tougher than Texas', let's look at the real reasons.
I think you'd be surprised in that there is some percentage of both fan bases that definitely want the game back. However, I would guess that the majority on both sides don't at this point.
For me, and maybe other Texas fans, I'd rather keep the USC's, Michigan's, Ohio State's, Notre Dame's, Alabama's...etc.. on the non-conference schedule year in year out. Big 12 only has three non-conference games, unlike the SEC with four so there is less room for scheduling.
On the A&M side: They left, and one of the primary reasons they left is to get away from Texas' giant shadow over them. Deep down they know the last thing they want is to have Texas on the schedule every year beating them more times than not. This ruins their safety bubble in the SEC where they can clamor that they are in a superior world now. The novelty of a new conference, and hope that things may change for them someday are still too strong and fresh.