Peter, those are great questions, and the fact is that most Americans are, as you put it, in “stark ignorance” about what happens in our prisons.
The short answer is due process. The worst criminals, and/or the ones most likely to flee, are detained prior to trial, often for many months (or well over a year in complex terrorism, racketeering, and international narcotics cases). They are in jail, despite the usual presumptions in favor of bail pending trial, due to fear that they will harm witnesses (or others by continuing to commit violent crime) or become fugitives (because some have close ties to foreign countries and the wherewithal to get to them). But they have not yet been convicted and thus they are presumed innocent. Though they can’t leave the prison, they are given a very wide berth to meet privately with family members, friends, associates, lawyers, investigators, paralegals, etc. Indeed, they even have ostensibly legitimate reasons to meet with their worst associates — under the guise that such people may be trial witnesses, or are otherwise assisting in the preparation of the defense … in which the Sixth Amendment has been construed to give a defendant a quite active role. And while most telephone calls in prison are monitored, “legal” calls generally are not — and if an inmate makes a phone call to the office of a lawyer which then patches him through to other people, it’s difficult to police that.
Once someone has been convicted, his status changes. He is, for one thing, dispatched to the prison where he will serve his time (prior to trial, he is in the jail in the jurisdiction where he has been indicted, which is generally where his criminal associates are). But even if he is sent, say, from the Metropolitan Correctional Center in Manhattan to the Supermax prison in Colorado, his conviction is still not deemed “final” — he has a right to appeal, a process that takes many months (during which he again has liberal access to lawyers, investigators, paralegals and other visitors). And that, of course, is just the federal system, which accounts for a comparatively small number of the prisoners in the U.S. Most inmates are state prisoners, meaning they are transferred within the same state — i.e., not thousands of miles removed from their criminal associates.
Even after convictions are “final,” prisoners have a right to challenge their convictions via habeas corpus. They are not entitled to free counsel for this, but are permitted to have lawyers (and the attendant support staff) if they can pay for it. This can go on for another couple of years.
Besides all that, prison systems worry about being sued for imposing what are alleged to be excessively draconian conditions of confinement. So they generally provide for at least some visitors, constant access to lawyers (if the inmate can afford it), opportunities for communication and recreation with other inmates, etc. The closer you get to 24-hour lock-down with no opportunity for social contact, the surer you are to be sued, and to find a some judge who will say this or that accommodation must be made to avoid a violation of the Eighth Amendment (barring cruel and unusual punishment) or the Sixth Amendment right to counsel — which is very liberally construed due to expansion of the notion that even after an inmate is no longer an “accused,” he is still entitled to an attorney-client privilege with which the state may not unduly interfere.
These concerns make it very difficult to step up surveillance — such as, by example, recording conversations between the inmate and his legal counsel (a proposal that makes bar associations go wild and which the federal district court in Washington even refused to permit in connection with enemy combatants — i.e., prisoners of war not criminal defendants — held in Guantanamo Bay).
The bottom line, I think, is that if you are going to impose due process restrictions that allow inmates broad communication protection when it comes to their legal counsel and generous visitation rights, it becomes impossible to stop them from conveying messages to the outside world. So it’s a matter of the legal culture more than training for prison officials. And it would be very hard to change that culture since legal elites tend to be liberal and tend to think the purpose of incarceration is rehabilitation, not disabling dangerous people and discouraging would-be dangerous people.