That sucks, just get it in the middle so you can poke an allen key through one bearing onto the sideways spacer. Sit it on a loose brick so that the bearings are lined up with a hole, put the allen key onto the spacer and hit it with a hammer, it should bust the bearing and the spacer out into...