In the beginning of the 20th century 4,000 Jews lived in Jannina. On the eve of the Nazi persecution the Jewish community had 2,000 members. The first Jewish presence in Epirus dates back to the time of Alexander the Great, who, according to tradition, brought Jews from Palestine. According to another version, after the destruction of the Temple of Jerusalem, the Romans captured Jewish slaves and imported them to Rome. When the ship anchored in Parga, the Jewish prisoners settled in Epirus. Historically speaking, however, it was most likely that the Jewish community of Ioannina was formed after the 8th century by immigrants from Nikopolis, Epirus. Besides, during that period Yiannena was becoming a significant urban center. Jews have been living in Yiannena ever since, observing their own tradition, manners and customs and religious rituals.

The Jews settled in the area inside the Citadel and on Yossef Elijah Street (once called Max Nordau Street). The locals called this area "Megali Rouga" ["The Great Alley"], which was the "heart" of the Jewish community. Jews also settled in Koundourioti and "Livadioti" quarters. Today, several old Jewish homes are still standing, and are monuments of the architectural heritage of the city. These include the two Levi residences, Mordechai Raphail residence, Moissis residence and many other typical folklore homes.

The Holocaust took a heavy toll of the Jews of Ioannina. On March 25th 1944, 1,850 people were arrested and deported to Auschwitz. Only 163 of them returned. The Jewish Community of Ioannina was re-organised after World War II, but now had a much smaller population. Fortunately, the old Synagogue, Ka'hal Kadosh Yashan, also called Inner Synagogue, that is located within the Citadel, happened to be spared the destruction. This synagogue was constructed in 1826 and is one of the largest and most splendid Jewish religious buildings in Greece that are still standing. Since it has no permanent rabbi, the synagogue functions only on High Holy Days. The Community also owns two apartment buildings, located in the area where the New Synagogue and the Boys' School of Alliance once stood, but were destroyed by the Germans. That is where most Jews live today. The community also owns a Jewish cemetery.