Rest is an important part of fitness. There's no way to cut it, you need to rest. After all, your body needs rest periods in order to
repair the microscopic damage that results from the more aggressive parts of your training schedule.
The question is, are you resting the best way possible?
For most folks, rest is either sleep (a "right" way to rest) or bumming out on the couch (which isn't).