The AP seems to make a bit of a leap here–that the president thinks it is racist to use the word:

WHITE HOUSE – In the debate about how to describe those displaced by Hurricane Katrina, President Bush is joining those who don’t like the word “refugees.”
The president tells reporters, “The people we’re talking about are not refugees, they are Americans.”
And he adds, “They need the help and love and compassion of our fellow citizens.”
Bush spoke during a meeting with leaders of charity and volunteer groups who are helping Katrina’s victims.
His words appear to put him on the same side as the Reverend Jesse Jackson, who has declared it’s “racist” to call U.S. citizens refugees.
Jackson and other black leaders say the word has a criminal connotation — and prefer the more neutral term “evacuees.”