At the end of the day, if you prefer whole milk, you should drink whole milk—just try to decrease something else in your diet ...