Where I live the seasons have quarterly distinctness, sharp as a shepherd’s calendar in a medieval book of hours. But as you slide down the flank of the continent, the atmosphere softens, the offshore colors glow, until you reach the islands once fought over by empires, now ruled by the resort industry.

When I tell you that I spent almost two weeks in early December on St. Croix filming scenes for a PBS documentary, I will not earn much respect for my industry. And it is true that although our work days were 14 hours long, the air was warm, the Christmas winds blew, at night Venus and Jupiter shadowed the waxing moon like bodyguards, and even production meetings and semi-official sessions of make nice were smoothed by rum-and-Cokes. What can such an interlude have to do with ordinary life, much less life in my latitude?

The answer, since I am a historian doing a historical documentary, is plenty. The Virgin Islands, of which St. Croix is the largest, and the West Indies as a whole were bound to New York during its colonial centuries, which means they are bound still.

The strongest tie between the changeful north and the smiling south was and is people. Islanders have been moving in with us since they could book passage. As late as 20 years ago, when Cruzans spoke of New York City or America — they equated the two — they said, the City (what other other place was there?). Now, thanks to pop Rastafarianism, they say Bab (Babylon). The Schomburg Center, a research branch of the New York Public Library in Harlem devoted to black history and culture, was begun by Arturo Schomburg, whose mother was from St. Croix. The subject of my documentary, Alexander Hamilton, who is buried at the head of Wall Street, lived in St. Croix from ages 9 to 15. St. Croix was his last stop before college; like many a résumé stuffer today, he thought of Princeton, but ended up at Columbia (then King’s). He never graduated, fighting the Revolution, writing the Federalist Papers, and running the Treasury Department all intervening; slacking isn’t what it used to be.

The economist — free-market or Marxist — rises to point out that people journeyed for a reason. The path from the islands to New York was cleared by profit, and the profits two and three centuries ago came from sugar. It is hard to conceive how lucrative sugar was then. When I speak about it, I give audiences three facts. In 1763, at the end of the Seven Years’ War, when England found itself in possession of so many French colonies that it decided to give some back for fear of overstretch, it hesitated between returning Canada or the island of Guadeloupe. When it gave back the sugar island, the merchant class was stunned by the folly of the decision. In 1772, the value of Jamaica’s sweet exports to England was four times greater than the value of the exports of all the Thirteen Colonies. All you Pinckneys and Randolphs and Browns, put that in your clay pipes and smoke it. On the eve of the French Revolution, one-third of France’s revenue was generated by Sainte-Domingue (the modern Haiti and Dominican Republic). No wonder Napoleon sacrificed an army to yellow fever in the attempt to get it back. Sugar was the oil of its time: a staple, not just for kings and milords, but for the burgeoning middle classes that had graduated from swill, even as we guzzle gas now because Middle America stepped up from old Dobbin a hundred years ago.

Those profits depended on slave labor. What a bargain: The labor was free, and the laborers weren’t. There are historians — left-wing, or angry black, or both — who say that the Industrial Revolution rested on slave/sugar profits. That can’t be the whole truth: France did not lead the Industrial Revolution, so island empires were not the only factor. But there was heavy lifting behind the spinning jennies.

The slave/sugar system was deeply woven into the commercial life of New York. The St. Croix merchant house where young Hamilton got his first job as a clerk was the branch of a New York firm. The Schuylers, the clan Hamilton married into when he moved here, were upstate grandees who were also involved in West Indies trade. Another old New York family that refined West Indian sugar in the City was the Roosevelts. New York processed the agricultural produce of the islands. In return, it supplied them with necessities. In a sugar island, you would not want to divert one acre to wheat or wood lots; it would be like keeping Sutter’s Mill in business after the gold strike in 1848. So southbound ships carried flour, timber, and a host of other items — made or grown on New York farms that were worked by still other slaves. Think of them as plantation back offices.

A sordid picture if one looks at it long and hard, as one must. There are a couple other scenes in it, though. At every rung of a slave society, people tried to make the best of their situation. Among the slaves who managed to live, there were slaves who managed to buy their freedom, or who were manumitted. In St. Croix, free black males had to serve in a militia, one of whose duties was to police unfree blacks; man tries to maneuver even on a cliff face. As the 18th century wore on, some people decided to end the system. Years after he left St. Croix behind him, Hamilton helped found the New York Manumission Society. The first president was his future Federalist co-author, John Jay. Jay owned slaves, and biographers still argue about whether Hamilton did. Even so, these New Yorkers worked for a world in which they would be impossible. After decades they succeeded.

A face on a bill, old evil, facing a problem: worth a thought between sunblocks.