The The Galle Portuguese Settlement was built by the Portuguese, along with a Franciscan chapel (now mostly in ruins) inside the fort established there in 1541. The fort also, in later years, served as a prison camp to incarcerate Sinhalese natives who opposed the Portuguese. The Portuguese had moved to Colombo from Galle as preference but in 1588, however, they were attacked by the Sinhalese King Raja Singha I (1581–93) of Sitawaka, which forced the Portuguese to return to Galle. At Galle, they initially built a small fort out of palm trees and mud. They called it the Santa Cruz, and later extended it with a watch tower and three bastions and a "fortalice" to guard the harbour.


Source: https://en.wikipedia.org/wiki/Galle_Fort