Forever does not always mean "without end" in biblical usage, but it does here. Sometimes forever means "as long as conditions exist." Here, we are talking about a covenant, commandments, and about righteousness that endure forever (verses 3, 8-10 - and strongly implied in verses 5 and 7). In six out of ten verses, various words indicate "time without end" and reinforce "forever and ever."
The covenant that he is talking about is the New Covenant, the one that will endure forever - not the Old Covenant. In Hebrews 8, the Old Covenant is declared to be obsolete! The important point is that God's commandments are connected to the covenant that will last forever. The commandments are definitely not done away with the coming of the New Covenant. God, in the time of Ezra, says that His commandments are NOT done away with the coming of a covenant that will last forever.
However, the notion in Protestantism is that, since the Old Covenant is done away with, then God's law is also done away with. So, Protestant theologians decisively deal with the Old Covenant and the law of God in one fell swoop, but it is not correct. It does neatly get God's law out of the way, revealing an attitude behind their theology.
Their teaching continues by stating that a reason it had to be done away is that God's law is too difficult to keep - that it is harsh and enslaving. They leave one with the definite impression that the reason it did not work - the fault, the flaw in the whole mix - was God! Human nature is certainly agreeable to this because it is ever willing to shift the blame elsewhere to justify its conduct.