I will be going Taiwan next month, can I know do I need to pre-book the normal railway train n THSR tickets in advance? Or I can just buy it on the day I want to go by train?

Usually you can get tickets the same day, but if your sure of the dates and time of traveling prebooking is always a great choice

You could get your tickets online for convenience. Here's their website, and I'm showing you the page on the details of an early bird special where you can get as much as 30% off on the ticket price: http://www.thsrc.com.tw/en/ticket/tic_earlybird.asp