They humiliated Carter. They nearly brought down Reagan’s presidency. They perplexed Bush Senior and defied Bush Junior, while beguiling and then betraying President Clinton in between.

Now they have run complete circles around President Obama, as their two-decade-long quest for nuclear weapons bids fair to become a reality.

The Islamic revolutionary regime in Tehran is poised to hand the United States its worst foreign-policy setback since the fall of South Vietnam.

Viewed through the lens of history, it is a remarkable if chilling story. Since that regime came to power in 1979, no single foreign entity has caused more destruction to American interests, kidnapped more Americans, or conducted or directed more deadly terrorist attacks against American targets. Only al-Qaeda has murdered more U.S. citizens in cold blood.

Since 2005, the Islamic Republic of Iran has also been waging a steady proxy war in Iraq, causing by some estimates as much as 20 percent of all U.S. casualties there. And the signs are rapidly growing that it’s trying to do the same in Afghanistan.

Yet despite all this, despite constant political infighting and growing unpopularity at home and in the region, and despite bringing its country to the brink of economic collapse, that regime is on the verge of going nuclear. An imperative foreign-policy goal of no fewer than three presidents — Clinton, Bush, and Obama — will have been lost.

How did this happen?

The usual excuse for Western foreign-policy failures — that the democratic electoral process and changes of government mean policies are constantly being revised and reversed, while dictatorships are able to maintain a steady course until they reach their goal — won’t wash here. The Bush and Obama policies toward Iran, at least since 2005, have been remarkably consistent — and largely an extension of the Clinton policy of carrot and stick. Play ball with us and the international community, successive secretaries of state have told the Iranians, and don’t build any nukes, and we’ll extend the carrots of diplomatic recognition and expanded trade. Don’t play ball, and expect the stick of sanctions — or even possibly military action.

The terms of this approach have been the problem all along. Military action — whether it was bombing Iran’s nuclear sites or blockading the Hormuz Straits at one extreme, or providing arms and cover aid to the country’s many anti-regime groups at the other — has been treated as the last option, or the ultimate stick, instead of the U.S.’s first and most important diplomatic asset. Both Bush and Obama saw military action as an alternative to diplomacy, and vice versa. This is a severe miscalculation, one that has consistently hobbled American foreign policy from Vietnam to North Korea, and now Iraq.

The alternative is to see force and diplomacy as mutually supportive aspects of the same exercise of power in defense of our national interest. We have failed to do this in Iran for several reasons.

One is that we underestimate the fanatical tenacity of the Iranian regime, just as we tend to overestimate the desire of other countries around the world to cooperate with us and be our friends, or at least avoid our displeasure. We believe this even when our displeasure has no severe consequences, including military action.

The second is that we overestimate the damage done to our international standing by unilateral military action. It was the failure to remove Castro with the Bay of Pigs invasion that humiliated the Kennedy administration and tempted the Soviets to new foreign adventures, not the invasion itself. Likewise, it was the passive acquiescence to the fall of South Vietnam, not the bombing of the North or the Cambodian invasion, that crippled American foreign policy for almost a generation. And likewise it was the failure to defeat the counterinsurgency in Iraq, not Operation Iraqi Freedom, that almost sank our credibility as a force in the Middle East — that is, until the Petraeus surge reversed the tide.

Contrary to the currently fashionable analysis, diplomacy and “soft power” tactics (such as economic sanctions) don’t form a continuum with military force, but rather a circle. Neither has credibility without the other, and neither can be effective without the full weight of the other behind it.

In dealing with Iran, U.N. sanctions, a gasoline boycott, military action and blockades, covert support for Iranian opposition groups, and U.S. or Israeli air strikes have all been weighed and debated as separate options on a graduated scale, instead of being treated as facets of a single comprehensive strategy.

In truth, the best negotiations always flow from a position of strength, which in foreign affairs ultimately means military strength and the proven willingness to use it. The best military posture, on the other hand, requires clear strategic goals and a firm, overarching diplomatic framework for handling both friends and enemies. Bringing the military to bear on a problem as the “last resort,” as America is prone to do, throws that advantage away, just as trying to negotiate while tying our hands with regard to force makes us look weak instead of conciliatory. A country can afford to make generous concessions to a potential adversary — such as diplomatic recognition, or even direct support for a peaceful nuclear program in the case of Iran — only when it is clear it will hit hard and effectively if rebuffed.

In short, this is a failure that goes beyond Iran. It is endemic in our very structure of government and haphazard division of labor between the State Department, which claims a monopoly over diplomacy, and the Pentagon, the wielders of the military tools who are only brought in when diplomacy fails. This all too often pits our soldiers against our diplomats — a division that almost proved fatal to the American effort in Iraq.

Perhaps the only sensible long-term solution is to bring both into a single agency with a single head, much as the War and Navy Departments were merged into the Department of Defense after World War II.

Regarding Iran, however, time may be almost up. The emergence of a nuclear Iran may very well trigger a war in the Middle East. It certainly points toward an arms race none in the region want and few can afford — and which will be disastrous for the United States.

The policies and institutions that are supposed to protect us are systematically failing us because they are misused and misconceived. Unless they are corrected, they will bring us many more Irans — with or without Obama.

– Arthur Herman’s most recent book, Gandhi and Churchill: The Epic Rivalry That Destroyed an Empire and Forged Our Age, was a finalist for the Pulitzer Prize in 2009.