You have to draw the line somewhere, right? Otherwise you get into slippery slope territory where eg. you can argue anyone who's not protesting for BLM are implicitly supporting the cops, and therefore are ruining black people's lives.
Sure. Not everything warrants a reaction, impact of speech highly depends on the speaker and context and reactions can be not proportionate. But I don't think a blanket "speech shouldn't have consequences" really works, because it will often have them, and often is intended to have them.