A plan of the Parish of St. James's, taken from a survey made in 1755 and reproduced by Jesse, shows the relative size of old Berkeley House and its grounds:
Almost the entire northern half of the ground shown there became the site of the current Lansdowne House, which occupies the whole of the south side of Berkeley Square except the south-eastern corner, which forms the opening into Berkeley Street.
In spite of the central position which Berkeley Square occupies, Beloe in his Sexagenarian told an anecdote which curiously illustrates the limited wanderings of our ancestors. It was the last story which he heard Horace Walpole relate, and was to the effect that
"in the time of Sir Robert Walpole it was the established etiquette that the Prime Minister returned no visits, but on his leaving office Sir Robert took the earliest opportunity of visiting his friends, and one morning he happened to pass, for this purpose, through Berkeley Square, the whole of which had been actually built whilst he was Minister, and he had never before seen it. This incident alone prevailed upon his son Horace to take the first opportunity which offered of purchasing a house here."