Yes, it would need to be a pretty seriously large tree! I think that the “Hangman’s Tree” (or “Hangman’s Oak”) is probably what the author was referring to. Even though this tree was on Spuyten Duyvil near Seton Hospital, it was still on property belonging to Augustus Van Cortlandt, so perhaps the author was confused. This 1885 map shows that the land around today’s Seton Park once belonged to Augustus Frederick Van Cortlandt:
My personal opinion is that the “Hangman’s Tree” is an interesting piece of local lore but I am not sold that there were actually any loyalists hanged from it. I have never seen any record of a hanging here. But hey, I guess you never really know for sure.