Heritage Town or White Town forms the French Colony adjacent to the Pondicherry Beach. Ideal choice would be to book an accommodation in one of their heritage homes which have been converted into hotels. However, budget rooms are available in abundance in this area. Being a prominent site for tourists, availability of public transport is not be a problem here.
Booking an accommodation here will save a lot (both money and time) on transportation.