Allowing for the variety of spellings of the surname, I have not found any Ollenton families living within 50 miles of Isleham before his marrriage to Elizabeth Brown.