Heritage Town or White Town forms the French Colony adjacent to the Pondicherry Beach. Ideal choice would be to book a hotel in Pondicherry in one of their heritage homes which have been converted into hotels. However, budget rooms are available in abundance in this area. Being a prominent site for tourists, availability of public transport is not be a problem here.
Booking an accommodation here will save a lot (both money and time) on transportation.