David Lammy called on the Government to "deal with the substance" around racism and not focus on the ongoing row over statues.
The shadow justice secretary suggested the dispute over historical figures was "a bit of deflection" when ministers should focus on urgent action to address deep inequalities that remain in the UK.
Labour is desperate to avoid being drawn into a culture war it could ultimately lose and to instead shift attention back to the serious issues at play.
Mr Lammy told the BBC: "This looks to me like a bit of deflection. Let's get to the action. Let's have some substance. Let's do something about the historic injustice that still exists in our country." He also urged ministers to move on from a "not helpful" dispute