"So there were delivered out of the thousands of Israel, a thousand for each tribe" (Numbers 31:5). And what is the meaning of "there were delivered"? That they were delivered in pairs, one to another. "There were delivered"—against their will, for Scripture had made the death of Moses dependent upon coming after the vengeance against Midian. They said: Shall we go to Midian and Moses will die? They held back from going. The Holy One, blessed be He, said to Moses: Cast lots over them, over the tribes, and they will be delivered of their own accord.
Why Israel Held Back from the War Against Midian
Midrash Tanchuma Buber, Matot 4:2
וימסרו מאלפי ישראל אלף למטה (שם שם ה). ומהו וימסרו, שהיו נמסרין זוגות זה לזה, וימסרו בעל כרחם, שתלה הכתוב מיתת משה אחר נקמת מדין, אמרו נלך למדין וימות משה נמנעו מלילך, אמר הקב"ה למשה הפל עליהם גורלות על השבטים והם נמסרים מאיליהם.