The Roman settlement of Ratae Corieltauvorum stood on the route of Fosse Way, a road connecting Exeter with Lincoln , crossing England diagonally. The word Fosse derives from the Latin Fossa = ditch , which suggests that the road followed the line of defensive earthworks that at one time marked the western extent of the Roman settlements in Britain. 
Fast forward to 1884, Ratae Corieltauvorum has evolved to Leicester. When a football club is formed they play their matches on a field near the Fosse Way and adopt the name Leicester Fosse. 
Fosse first entered the FA Cup in 1890.
In 1891 they joined the Midland League and in 1894 were elected to the Football League.