London Caledonians

There has never been a professional football team with London in its name. This probably dates to the very earliest days of the Football Association when nearly all the member clubs were London based, and with the  advent of  proletarian and eventually professionalized  football, the metropolitan area being so large, local clubs had developed all over the city.
London Caledonians was very different, however. Following a trend seen in Rugby, it was essentially an exiles club for Scotsmen living in London (there was also a London Welsh club in the 1890s).  London Caledonians was founded in 1886 and was originally based in Leyton, later moving to Holloway. In their early years they played leading clubs such as Aston Villa and Queen's Park. 
Caledonians had an impressive honour roll:

FA Amateur Cup winners -1923
Isthmian League Champions- 1905–061907–081911–121912–131913–141924–25
Middlesex Senior Cup winners- 1889–90, 1890–91, 1898–99, 1899–1900, 1934–35
London Senior Cup winners-1899–1900, 1907–08, 1914–15, 1922–26, 1927–28

Caledonians went out of existence during the 1939-45 war.