imo, points scored, time of possession and red zone scoring are the only 3 offensive stats that matter.
I think the offensive stat other than points scored that matters most as it relates directly to winning is offensive giveaways. In the NFL last year, there were eight times that committed fewer than 20 turnovers. All eight made the playoffs. In the last five years, NFL teams have committed fewer than 20 TOs in a year 29 times. Those teams have made playoffs on 26 occasions.
I can't imagine it's much different for college football.
I think you are exactly right though that offenses seem to be evaluated by yards, even more than points. Yards is probably the fourth or fifth most important statistic.