The original tradition is that the Bene Israel are the descendants of the survivors (seven men and seven women) of a shipwreck off the Konkan coast at Navgaon, about 26 miles south of Bombay. Their ship was said to have come "from northern parts" and the date was "some sixteen to eighteen hundred years ago" In the 19th century various theories were propounded by Europeans about Bene Israel origins conjecturing that the Bene Israel were an offshoot of the Jewish settlements in Yemen, refugees from the persecution of the Jews by Muhammad, or descendants of the Babylonian-Persian Diaspora. Later, in the light of the study of the Bible, of other Jewish literature, of ancient history sources relating to India, the Middle East, etc., some members of the Bene Israel community itself delved into details of possible Bene Israel origins. H. S. Kehimkar (History of the Bene Israel of India (1937) written in 1897) favored the theory that the ancestors of the Bene Israel left the Galilee because of persecutions by Antiochus Epiphanes (175–163 B.C.E.). D. J. Samson's argument for Bene Israel arrival in India at some time between 740 and 500 B.C.E. appeared in 1917 in an issue of the Bene Israel periodical The Israelite (i, no. 4,68–70) in an article entitled The Bene Israel: Who, Where, Whence. In any case their descendants remained for centuries isolated from Jewish life elsewhere. Thus they forgot much of the Hebrew language, prayer and ceremonies, and adopted customs and dress of their Konkan neighbors, and their language, Marathi, as their mother tongue. Throughout the centuries they clung, however, to some fundamentals of the Jewish tradition and observed circumcision, dietary laws, the Sabbath, and most fasts and festivals prescribed in the Torah, and recited the Shema. But they were otherwise unaware of Torah, or of talmudic and halakhic lore. In their new surroundings the Bene Israel turned to the pursuit of oil-pressing and agriculture and became known to their neighbors as Shanwar Tells ("The Sabbath-observing oilmen"), indicating both their occupation and their religious observance. The presence of a special Jewish group in the Konkan region remained unknown to outsiders, except for casual references to them.

Bene Israel tradition tells of a Jew, David Rahabi, who about the year 1000 C.E. (or, some say around 1400 C.E.) discovered the Bene Israel in their villages, recognized their vestigial Jewish customs, and taught them about Judaism, preparing certain young men among them to be the religious preceptors of the Bene Israel. They were called Kajis and their position became hereditary. They were also recognized officially as judges in disputes within the Bene Israel community.

Continuous contact with the Jews of Cochin apparently started when a David Rahabi of Cochin helped to obtain the release of the Bene Israel officer, Commandant Samuel Ezekiel Divekar, who had been taken prisoner during the Anglo-Mysore wars. Divekar was impressed by the Jewish community of Cochin and their synagogues, and it was on his initiative that the first Bene Israel synagogue, known as Sha'ar ha-Rachamim ("Gate of Mercy") was established in Bombay in 1796. While there may or may not have been a curious transposition of the name "David Rahabi" into Bene Israel earlier legends, certain factors (such as village-linked family surnames, traditional occupation, distribution of settlement) point to the Bene Israel as having been thoroughly established as a distinct entity in Konkan villages long before British or Portuguese rule in the area.

A further impetus to their return to traditional Judaism was given to the Bene Israel through the cooperation of Cochin Jews who visited Bombay and the Konkan villages, and through the new wave of immigration of Arabic-speaking Jews from Baghdad to Bombay in the early decades of the 19th century. The secular education of the Bene Israel was considerably influenced by Congregational missionaries from America who opened schools both in Bombay and in the outlying towns and villages. They trained Bene Israel to become teachers in these schools, and it was in these schools that the Bene Israel got their first understandable introduction to be Bible. Then, in 1826 a Jew from Cochin, who had been converted to Christianity, Michael Sargon, was deputed to work among the Bene Israel. He not only devoted his energy to teaching them in the Marathi language, without any attempt at proselytization, but also mediated in their disputes.

Somewhat later Rev. John Wilson of the Scottish Presbyterian Mission, started his educational activities among them. In 1832 he published a Hebrew Grammar in Marathi, and Bene Israel studied Hebrew in the high school and in the college founded by him. Gradually the missionaries withdrew from the field of primary education and the Bene Israel took their education into their own hands. H. S. Kehimkar, in collaboration with his brother and A. D. Pezarkar, started a small primary school in 1875. It later became necessary to solicit for funds, and generous aid was given by the Anglo-Jewish Association of London, Jewish philanthropists in England and France, members of the Sassoon family, and the Government of Bombay. The school, with its own building, grew into a high school teaching Marathi, English and Hebrew. Originally called the Israelite School, the name was changed in the early 1930s to the Elly Kadoorie School, in recognition of a large donation by Sir Elly Kadoorie of Hong Kong.

Religious development was also very much facilitated for the Bene Israel by translations of the Old Testament by an association of Protestant Christian missionaries of all denominations beginning in the early twenties of the 19th century. Since its establishment in 1857 Bombay University included Hebrew in its curriculum. Originally, the communal organization, religious as well as secular, of the Bene Israel was headed by the Kajis. With the establishment of synagogues, the secular functions of the Kajis were gradually taken over by the Muccadams, who either were the most prominent persons in the local community, or who succeeded their fathers in the office. In large synagogue congregations the Muccadams were aided by Choglas, or councilors. At present each congregation has an elected Board of Trustees and an elected Managing Committee. The ritual functions of the Kajis came to be performed by the Hazzanim of the synagogues. Hazzanim were initially recruited from Cochin, but later also from among the Bene Israel themselves.

The Bene Israel established additional synagogues in Bombay: Sha'ar Razon (1839), Etz Hayim (1888), and Magen Chassidim (1931), and also several prayer halls. From 1848 onwards Bene Israel synagogues were also established in 12 different towns on the Konkan coast; and far afield in the cities of Poona, Ahmedabad, Karachi (now in Pakistan) and New Delhi. Most Bene Israel congregations became affiliated (in reality very loosely) either with the World Council of Synagogues (Conservative) or with the Union of Orthodox Jewish Congregations. A significant development in the religious field was the establishment in 1925 of the Jewish Religious Union in Bombay by Dr. Jerusha Jhirad . In Bombay this was an entirely spontaneous move without outside financial help, though prayer books and other literature were obtained from the Liberal Jewish Synagogue of London. The Bombay Jewish Religious Union was one of the founder members of the World Union for Progressive Judaism (established in 1926) and made a small annual contribution toward its expenses. In the 1950s the Bombay congregation's own funds were supplemented with financial aid from circles of Progressive Judaism outside India and from Sassoon Trust Funds, all of which enabled the Bombay Jewish Religious Union, now called Congregation Rodef Shalom, to obtain premises of its own and the services of two young rabbis, both graduates of Hebrew Union College. In August 1957 Rabbi Hugo Gryn (for more than two full years), followed by Rabbi Elisha Nattiv (for about three years) ministered to this congregation and exerted an influence among the Jews of Bombay far beyond the three hundred members of Congregation Rodef Shalom. Rodef Shalom is the only Bene Israel congregation ever to have had a resident rabbi.

Between 1948 and 1952, approximately 2,300 Bene Israel emigrated to Israel. As a result of sit-down strikes and hunger strikes, the Jewish Agency returned a total of 337 individuals, in several groups, between 1952 and 1954. Most of them were brought back to Israel by the Jewish Agency after several years. From the establishment of the state until 1969, over 12,000 Bene Israel emigrated to Israel. They were mainly absorbed into the branches of industry in which they were occupied in India, such as textiles and metals, as well as into public services. They settled mainly in Beersheba, Dimonah, Ashdod, and Eilat.

The Bene Israel became the focus of a controversy which arose in 1954 over the basic question of the personal status of the Bene Israel regarding marriage with other Jews. Although the Chief Rabbinate had laid down in essence that "the sect of the Bene Israel in India is of the seed of the House of Israel without any doubt," several rabbis in Israel refused to marry Bene Israel to other Jews. This standpoint was based on halakhic decisions that had been given for Jews from Baghdad who had settled in India, and who denounced intermarriage with those whom they considered to belong to an inferior caste. On first coming to India in the 18th century, the Baghdadi Jews had prayed in the synagogues of the Bene Israel and buried their dead in their cemeteries. However, as they became more settled and acquired a higher status and education, they began to keep apart and to question whether the Bene Israel were legitimately Jewish. They considered that association with the Bene Israel should be debarred for fear of illegitimacy (mamzerut) since the latter were unfamiliar with the Jewish laws of divorce (gittin), absolved themselves from levirate marriage, and did not practice halizah. Not one of the rabbis outside India who returned a negative decision concerning the Bene Israel in previous generations had ever visited there or met representatives of the Bene Israel community in order to obtain knowledge of their customs or information directly from them. In Israel the controversy arose between those who rejected the Bene Israel and those who regarded them as Jews in every respect. In 1962, the Israel Chief Rabbinate appointed a commission of four rabbis who were charged with meeting representatives of the Bene Israel. From the evidence of the leaders of the community who appeared before the rabbis and from earlier sources, it became clear that the Bene Israel had not been accustomed to divorce women at all, in the same way that divorce was not practiced among Indians other than Muslims until about a century ago. It was only on the arrival in India of rabbis from Baghdad and Yemen who were experts on the Jewish laws of divorce that a number of Bene Israel had approached them. Concerning widows the Bene Israel generally followed the custom of their Indian neighbors and did not permit them to remarry, so that the question of levirate marriage or halizah did not arise. On Oct. 18, 1962, the council of the Chief Rabbinate decided that marriage with Bene Israel is permissible. However, the rabbi registering the marriage was bound to investigate, as far back as three generations at least, the maternal ancestry of every applicant of the Bene Israel, man or woman, wishing to marry outside the community, in order to establish to what extent there were not intermixed in the family persons who were non-Jews or proselytes. The rabbi concerned was also bound to establish as far as possible that neither the parents of the applicant nor his grandparents had remarried after a previous divorce, and that they were not within the prohibited degrees of kinship.

These directives aroused fierce resentment, culminating in a stormy strike in Jerusalem in the summer of 1964, in which several hundred of the Bene Israel from all over Israel participated. Subsequently, the prime minister, Levi Eshkol, issued the statement that "the government of Israel reiterates that it regards the community of the Bene Israel from India as Jews in every respect, without any restriction or distinction, equal in their rights to all other Jews in every matter, including matters of matrimony."