No harder than putting the starnut in in the fist place. Here's how I did mine. I found a socket that fit inside of the forktube perfectly, put some extensions on it, then used a hammer to push it through.
I drilled my starnuts out of my bars, get a electric drill some sharp bits, and drill away will it breaks or just falls out.....it works is did it to get 4 starnuts out!