Bankside has lived from medieval days of "Gildable", "Great Liberty" and "Bermondsey Abbey" Manors through the ages of Elizabethan and Victorian grandeur to the thriving leather and tanning and hop factoring industries.In the 19th century the Bishop of Southwark bought the house (with help from Sir Frederick Wigan and Mr. Richard Foster) and converted it into a Collegiate House for clergymen. Thereafter, even when the South Eastern Railway Company (1884) and after them the Leather Trades Federation (1920) moved in, part of the House, Foster Hall has been used as the Chapter House Annex of Southwark Cathedral.

The whole building is built over one of the old Hospital's quadrangles and two of the original cloister pillars can be seen. The imposing main and the "servant" stairways are side by side in the front of the building - a feature to be found on only one other building in England.