The Taipei Bus Terminal is located not far from Taipei Railway Station. It is within walking distance - about 15 minutes walk. The journey back to Hsinchu has to take the bus from here.
Taipei Railway Station's architecture is simple but the building is really big and spacious. The present station was opened in 1989.
Another view of Taipei Railway Station (TRS)
Read about TRS at wiki, how to get there from Taipei International Airport is here.
There is a hostel nearby but the review from patrons are not really encouraging - read here.
Travelling to Taipei from Hsinchu is very easy. Buses are available and they are quite frequent. The fare is NTD110 (that was in 2006).
Read about it at wiki
Freeway Bus (photo taken near Taipei Main Railway Station)