(Bamidbar 27:11) "And if his father has no brothers, then you shall give his inheritance to his flesh who is near to him": The "nearest" takes precedence in the inheritance. "of his family": his father's family or his mother's family? It is, therefore, written (Ibid. 1:2) "by their families, by their fathers' house" — the families are according to the fathers. "and he shall inherit her": (as above). "and it shall be to the children of Israel a statute of judgment." The Torah hereby authorizes the sages to adjudicate the matter (i.e., to decide who is the "nearest.") "then you shall give his inheritance to his flesh who is near to him of his family": Whoever is closest in "flesh" takes precedence in inheritance.
Ancient Inheritance Law for Families Without Sons
Sifrei Bamidbar 134:3