"And he came into the house to do his work" (Genesis 39:11). Rabbi Eliezer says: It was the Sabbath, as it is written, "For on it He rested from all His work" (Genesis 2:3). And what work did he have? He was reviewing and reading what his father had taught him.
Rabbi Eliezer Says It Was the Sabbath When Joseph Came to Do His Work
Yalkut Shimoni on Torah 146:2
וַיָּבֹא הַבַּיְתָה לַעֲשוֹת מְלַאכְתּוֹ. רַבִּי אֱלִיעֶזֶר אוֹמֵר, שַׁבָּת הָיָה, דִּכְתִיב (בראשית ב, ג) "כִּי בוֹ שָׁבַת מִּכָּל מְלַאכְתּוֹ", וּמַה מְלָאכָה הָיְתָה לוֹ, שׁוֹנֶה וְקוֹרֵא מַה שֶׁאָבִיו לִמְּדוֹ.