That problem - black or all - is just an misunderstanding of the intention between the people.
As you said, BLM activists are pointing out the oppression. But someone who does not know is - and this is something you need to be told very precisely - will say "huh? All lives matter".
Because saying BLM without the special content IS like saying BLM more. Because of that ALM is not disrespectful, it's disrespectful to say it's disrespectful (if the person does not know the background), because the ALM is more universal and understandable without context.
The Golden Rule applies to all, and not especially to blacks. Same principle.