Mendele was born in the small town of Kapulia in Belorussia. After the death of his father, Mendele, aged 13, left Kapuli and wandered through Lithuania, studying at various yeshivot. Traveling throughout Volhynia, Ukraine, and Podolia, he absorbed many impressions of Jewish life which he later recorded in his writings. He lived in Berdichev from 1858 to 1869, in Zhitomir from 1869 to 1881, and, except for a short stay in Geneva from 1905 to 1907, in Odessa from 1881 to 1917. During the first literary period, which began in the 1860s, Mendele wrote literary and social criticism (Mishpat Shalom, 1860; Ein Mishpat, 1867), works of popular science in Hebrew (Toledot ha-Teva (Natural History), vol. 1 1862; vol. 2 1867; vol. 3, 1873), and Hebrew and Yiddish fiction - in Hebrew: Limmedu Heitev ("Learn to Do Well," 1862); Ha-Avot ve-ha-Banim ("Fathers and Sons," 1868); in Yiddish: Dos Klayne Menshele ("The Little Man"), Dos Vinshfingeril ("The Wishing Ring," 1865); Fishke der Krumer ("Fishke the Lame," 1869).

In his writings on social and literary problems Mendele showed lively interest in the education and public life of Jews in Russia. He was preoccupied by the question of the role of Hebrew literature in molding the Jewish community. Concerned as he was with problems of Jewish society, Mendele took an active part in public life in Berdichev: he founded a philanthropic association to help the poor and, in his Yiddish play, De Takse ("The Tax," 1869), denounced the infamous meat-tax which fell heaviest upon the poor. Adopting the objectives of Russian positivist literature of the 1860s, the young Mendele demanded that contemporary Hebrew literature be actively involved in current problems of the Jewish community. Hebrew literature, he contended, should influence and inspire Jewish life. Hence his endeavors to teach the sciences to the mass of Jews and to aid the people in obtaining secular education in the spirit of the Haskalah. His advocacy of the study of the sciences is expressed, among other places, in the first version of his Yiddish novel, Dos Vinshfingeril, and particularly in his compilation, Toledot ha-Teva, whose zoological terminology based on talmudic sources has influenced modern Hebrew usage. The literary quality of his description of nature in general and animal behavior in particular is indicative of the artistry of his later fiction. In the 1870s Mendele devoted himself to traditional Jewish literature; he compiled original translations-adaptations of Zemirot Yisrael (1875) and Perek Shirah (1875), and planned to translate the prayer book and the Book of Psalms into Yiddish, but this project remained unfinished and only small parts are extant. Concurrently, Mendele published the first version of his allegorical novel, Di Kliatshe ("The Nag," 1873) and his allegorical poem, Yudel (1875), which is based on the history of the Jewish people. He also produced "practical" literature, e.g., the calendar, Der Nitslikher Kalendar (1875–85), which includes information in Yiddish on natural sciences and Jewish history. Also in this category is his Yiddish translation from Russian of the regulations concerning compulsory service in the Russian army (1874).

Although Mendele continued writing and publishing in Hebrew throughout the 1870s, this decade was mainly devoted to writing in Yiddish. The return to Hebrew writing came in 1886 with the publication of his story, Be-Seter Ra'am ("The Secret Place of Thunder"), his first Hebrew belletristic work since 1866, when he had published the novel, Ha-Avot ve-ha-Banim. Mendele's Hebrew writing became his main concern in the 1890s, when in addition to continuing to publish his short stories in Hebrew he rewrote his Yiddish novels Dos Vinshfingeril and Fishke der Krumer in Hebrew, entitling them Be-Emek ha-Bakha ("In the Valley of Tears," 1904) and Sefer ha-Kabbezanim ("Book of Beggars," 1909) respectively. Between 1890 and 1911 he continued to write original works in Hebrew, to recast his Yiddish works into Hebrew, and to re-edit his Yiddish fiction. The result was the publication in 1909–11 of his complete Hebrew works in three volumes and in 1911–13 of his Yiddish works in 16 volumes. By this time Mendele had become the acknowledged classicist of both Hebrew and Yiddish literature and enjoyed critical recognition and wide popularity in Jewish society.