Lookie what I found:

There is Change.org petition by the Congressional Black Caucus to change the National Anthem to America the Beautiful.

This petition spells out the why very clearly. Here are a few selected quotes:

The “Star Spangled Banner” contains “racism,” elitism and even sexism embedded in its third and fourth stanzas. These sentiments have no place in the national anthem of a democracy that claims that all men (and women) are created equal.
The only ideal or national priority the “Star Spangled Banner” spells out is military strength and victory. Nowhere in the anthem is brotherhood, national unity (E Pluribus Unum, Out of Many, One) or even a JUST and ETHICAL use of military power proposed .

Some choice lyrics:

STANZA 3: “No refuge could save the hireling and slave from the terror of flight, or the gloom of the grave.”
STANZA 4: “Oh! Thus be it ever, when freemen [white European-American landowners]shall stand.”

Re the anthem’s author Francis Scott Key:

He was a lawyer by profession who was appointed District Attorney for the City of Washington by President Andrew Jackson in 1833. As such, Key was a fanatical prosecutor of abolitionists and so-called “rebellious slaves.”
Key said the following: “Are you willing, gentlemen, to abandon your country, to permit it to be taken from you, and occupied by the abolitionist, according to whose taste it is to associate and amalgamate with the negro? Or, gentlemen, on the other hand, are there laws in this community to defend you from the immediate abolitionist, who would open upon you the floodgates of such extensive wickedness and mischief?”