Actually, your wheel shakes more without them, because tightening the axle pushes un-even pressure on the inner of your bearings. Therefore, the middle of your bearings are pushed in further than the outer race, and they roll unevenly (And sit on an angle). Not to mention you chance popping them...