Homelessness minister accused of racist remarks about rough sleepers