re: Would 4 days be too short for a Hawaii trip?

"Jet Lag, medically referred to as desynchronosis and rarely as circadian dysrhythmia, is a physiological condition which results from alterations to the body's circadian rhythms resulting from rapid long-distance trans-meridian (east–west or west–east) travel on high-speed aircraft. For example, someone travelling from New York to California feels as if the time were three hours later than local time. Jet lag was previously classified as one of the circadian rhythm sleep disorders."*

Wikipedia specifically and medical science in general disagrees with your premise that jet lag is only derived from travel in one direction.

* LINK