Rabbi Eleazar said: "And He called to Moses," while all Israel listened; and Scripture came only to give honor to Moses. They raised an objection: "a voice to him," "a voice toward him" - Moses heard, and all Israel did not hear. This is no difficulty: the one case is at Sinai, the other in the Tent of Meeting. And if you wish, say it is no difficulty: the one refers to the calling, the other to the speaking. Rabbi Eleazar pointed out a contradiction: it is written, "And Moses was not able to enter the Tent of Meeting because the cloud rested upon it" (Exodus 40:35), and it is written, "And Moses entered into the midst of the cloud" (Exodus 24:18). This teaches that the Holy One, blessed be He, took hold of Moses and brought him into the cloud.
Moses Alone Heard the Voice in the Tent of Meeting
Yalkut Shimoni on Torah 428:6
אָמַר רַבִּי אֶלְעָזָר, וַיִּקְרָא אֶל מֹשֶׁה, וְכָל יִשְׂרָאֵל שׁוֹמְעִין, וְלֹא בָּא הַכָּתוּב אֶלָּא לַחֲלֹק כָּבוֹד לְמֹשֶׁה. מֵיתִיבֵי קוֹל לוֹ קוֹל אֵלָיו מֹשֶׁה שָׁמַע וְכָל יִשְׂרָאֵל לֹא שָׁמְעוּ, לָא קַשְׁיָא הָא בְּסִינַי הָא בְּאֹהֶל מוֹעֵד, וְאִיבָּעִית אֵימָא לָא קַשְׁיָא הָא בִּקְרִיאָה וְהָא בְּדִבּוּר, רַבִּי אֶלְעָזָר רָמֵי כְּתִיב "וְלֹא יָכֹל מֹשֶׁה לָבוֹא אֶל אֹהֶל מוֹעֵד כִּי שָׁכַן עָלָיו הֶעָנָן", וּכְתִיב (שם כד, יח) "וַיָּבֹא מֹשֶׁה בְּתוֹךְ הֶעָנָן", מְלַמֵּד שֶׁתְּפָסוֹ הַקָּדוֹשׁ בָּרוּךְ הוּא לְמֹשֶׁה וֶהֱבִיאוֹ בֶּעָנָן.