In 1889, after teaching Talmud for a year in Bialystok, he was appointed a teacher at the famous Slobodka Yeshivah, a center of the musar movement. R. Rabinovich, whose system of study was similar to that of R. Soloveichik, maintained that priority should be given to the study of Talmud and not to musar, and the popularity of his courses-at the expense of the musar aspect-brought about some tension between him and the head of the yeshiva, R. Nathan Ẓevi Finkel. As a result, in 1899 R. Rabinovich left the yeshiva and, after serving for a year as rabbi of the small town Gorzd, moved to Panevezys. There he established in 1911 the Kolel "Kibbutz le-Meẓuyyanei ha-Yeshivot," which was financed by Miriam Gavronsky, daughter of the tea magnate and philantropist Kalonymus Wissotzky . The venture was an outstanding success and made Panevezys a major center of Talmud study in Lithuania. Forced to leave Panevezys during World War I Rabinovich moved the Kolel first to Luzin in the Vitebsk district and then to Mariopol. After the Bolshevik Revolution he returned to Panavezys, where he died.

R. Rabinovich was one of the few rabbis of his time who knew Russian and modern Hebrew literature, which he mastered during an illness. Originally a supporter of the Ḥibbat Ẓion movement, he later became one of the founders of the Agudat Israel . He was almost unique among contemporary rabbis in his support of the workers' and socialist movements; in 1917 he made a passionate but unavailing appeal at a meeting of the Orthodox association Masoret ve-Ḥerut, to persuade its member to come out in support of expropriation of the lands of the nobility and their redistribution to the peasants. As a result he was popular among the Bundists , and even among the Yevsektsiya , who did not attack him as they had other rabbis. Despite this they refused to allow him to reopen his yeshiva, and he died brokenhearted.

Famed as an outstanding posek, especially after the death of R. Isaac Elḥanan Spektor , and in his decisions tending to leniency, he committed little to writing and what there was, was lost in World War I. In 1948, however, a small collection of his commentaries and responsa was published under the title Zekher Yitzhak. About the same time a former student, D. Zachs, published a volume consisting of notes he had taken of his master's lectures on tractates Kiddushin and Baba Meẓia.