The grave, in the churchyard of the Adolf Fredrik Church in central Stockholm, was desecrated just after 7am on Wednesday.
Flowerpots were broken, bits of paper were stuck on the grave and rice was spread in the area. A rope was attached to the grave and to a nearby fence. A police spokesman said the attack appeared to have ritual elements.
Police are treating the incident as a 'violation of the peace of the dead'.
Palme was prime minister of Sweden from 1969 to 1976 and again from 1982 until his murder in 1986. He was shot and killed in central Stockholm after going to the movies with his wife. The murder remains unsolved to this day.