Bergelson read Hebrew and Russian literature before he was in his teens, and began writing in those languages. His early literary efforts, a Hebrew story "Reikut" and a Yiddish story "Der Toyber," submitted to several periodicals, did not meet with success. "Der Toyber," however, was later published in his collected works (4 vols., 1922–23); it was dramatized under the name "Di Broyt Mil" (1930), and was staged in Russia and in America. His first full-length work, Arum Vokzal ("By the Depot"), published in Warsaw in 1909 at his own expense, was warmly received by major critics; Bergelson thereafter wrote only in Yiddish, devoting himself to Yiddish literature and belles lettres. The novel Nokh Alemen (1913) was hailed as a masterpiece; it established his reputation as a gifted author of prose whose major theme was the slow decay of the Jewish bourgeoisie in village and town.

Bergelson was very active in Jewish cultural circles and he was one of the founding directors of the dynamic Yidishe Kultur Lige, a Jewish cultural organization established after the Russian Revolution. He coedited two of its most influential publications: the literary miscellanies Oyfgang (1919, in which his work "In Eynem a Zumer" appeared) and Eygns (1920, in which his story "Opgang" was published).

In his critical publications, Bergelson stated that art should not be a tool of the new social order and provide "naked abstractions" for propaganda, but should draw on tradition in the creation of its subtle personalized images.

In 1920, Bergelson moved to Berlin where he coedited the journal Milgroym with Der Nister, then In Shpan, the title suggesting a new political orientation. Writing for the New York Jewish daily Forward until 1925, he later became a correspondent for the Moscow Emes and the New York communist paper, Morning Freiheit. In marked contrast to his earlier views, his writings of this period show an identification with Soviet ideology, including the need for literature to toe the proletarian line. His novels and stories of those years dealt with revolutionary themes. Bergelson traveled widely: in 1924, through the Jewish communities of Rumania, under the auspices of ORT; to the Soviet Union in 1926, where he declared himself a "Soviet writer"; to Paris; to the United States for six months during 1929; through Poland on a lecture and reading tour; and to Copenhagen for a brief stay, in 1933. In 1934, he settled in Moscow after a visit to the Jewish autonomous region of Birobidzhan. His major work of the 1930s, Baym Dnieper, is a modified, partly autobiographical Bildungsroman (2 vols., 1932–36). Like most Jewish and other Soviet writing of the decade, Bergelson's work adapted itself increasingly to the thematic and stylistic demands of socialist realism.

After 1941, and for the duration of World War II, Bergelson was active in the Anti-Fascist Committee; his wartime stories appeared in its publication, Eynikeyt. Two dramas, Prints Reuveni and Mir Viln Lebn, were written and performed during this time: the first in Moscow, the second by the Habimah Theater in Tel Aviv. Early in 1949, Bergelson was imprisoned (apparently without trial) with other leading Yiddish writers—including P. Markish, I. Feffer, D. Hofstein—and together with them was shot on August 12, 1952, his 68th birthday. A Soviet edition of his collected works, published in 1961, indicated his subsequent "rehabilitation." Bergelson's early theme—the decline of individual initiative in a period of widespread stagnation—finds its precise tonal correlative in his style: indirect quotation, passive verb forms, adjectival repetition, periodic sentences, and similar devices create a fatalistic atmosphere in his fiction that subordinates character and emphasizes the pessimistic curve of the plot. This style persists even in his "revolutionary" writing of the 1920s, but becomes more straightforwardly dramatic in his stories about Birobidzhan and Soviet progress. His wartime fiction, collected in Naye Dertseylungen (1947), shows an interesting variation of his early impressionism. Yiddish criticism considers Bergelson one of its foremost modern prose writers.