I see what you are saying. If you follow the names of the census in order, you can tell the path of the census taker. Here is snapshot taken from the census:
It is clear from the order of the names that the census taker was travelling south on Broadway (then called the Albany Post Road). The name at the top of the list is Samuel Lawrence, who lived near today’s Broadway and Lawrence Street. Further down the list you see William Warner, who lived to the south–just East of Broadway in present-day Van Cortlandt Park near 261st Street. Continuing down the list you see George Hadley, who lived in the “Hadley House,” which still stands at 5122 Post Road. The Post Road continued south to the Van Cortlandt mansion and estate of Frederick Van Cortlandt. The last name in the screenshot is Daniel Halsey. He operated the reconstructed tavern that once sat at the intersection of 230th and Broadway (known as “Cock’s Tavern” during the revolution). Perhaps from the screenshots, you can see just how prevalent slavery was in the area. The last column totals the number of enslaved people. Just in this small sampling there are 50 enslaved people listed. Compare that to the number of free white adult males at 32.