First, I try to face all around the metal piece to make it cleaner and smoother. I needed to face half of it from one side, then turn it over and re-do it from the other end.
Then I made a mistake (kind of). Instead of drilling the hole first, I turned the one side of piece till it has a smaller diameter. After realizing the mixed orders, I measured how deep I should drill the hole, and kept drilling and taking off the piece to measure if the hole is the same depth as the length of the smaller diameter end.
I was not able to tap the drilled hole, but I did polish the piece all over. And here is the current final product.