Another interpretation: with jealousy I exact punishment from them, from those who worship idols; but I am gracious and compassionate in other matters.
Jealous Against Idolaters Yet Gracious in All Other Matters
Yalkut Shimoni on Torah 287:4
דָּבָר אַחֵר בְּקִנְאָה אֲנִי נִפְרָע מֵהֶן מֵעוֹבְדֵי עֲבוֹדַת אֱלִילִים, אֲבָל חַנּוּן וְרַחוּם בִּדְבָרִים אֲחֵרִים.