Sample-Efficient Reinforcement Learning by Breaking the Replay Ratio Barrier