It is said that the main idol inside the temple is that of Goddess Yogmaya, sister of Lord Krishna. Kansa, who was uncle of Yogmaya made an effort to kill Yogmaya on the day when Lord Krishna was born. However, Yogmaya vanished after predicting the death of Kansa by Krishna's hand.
According to another legend, it is said that the Mughal Emperor, Akbar tends to have a close association with the temple. Akbar's wife was depressed with the imprisonment and exile of her son Mirza Jehangir. He had fired from the Red Fort window at the British Resident. This led to death of resident's bodyguard. Subsequently, Goddess Yogmaya appeared in the dreams of Akbar wife. After that, Queen promised to place pankhas made up of flowers at the Yogmaya temple for her son's return. This practice is continued till today in the name of Phool walo ki sair which is a festival celebrated for three days in the month of October.
Another fact that is associated with the temple is that more than 5000 years ago, the people who stayed around this temple used to take care of the Yogmaya temple. It is believed that all these people who are now more than 200 in number tend to had one common ancestor at that point. Many years ago, they initiated the practice of taking care of the temple that included doing the shringar of Mata two times in a day, cleaning of the temple, distribution of Prasad to the pilgrims and other related things. These people who have carried forward the customs and traditions of their ancestors do it voluntarily today.
The temple was constructed in the year 1827 and is quite simple but tends to possess a modern structure. It possesses an entrance hall as well as sanctum sanctorum that consist of main idol of Goddess Yogmaya. The idol of Yogmaya is made of black stone and has been placed over a marble well that has a width of 2 ft and depth of 1ft. It is to be stated that the chamber where the idol is placed is basically 17ft square and possess a flat roof above which a shikara has been constructed. Along with it, the other salient feature of the temple is the dome. The idol is completely covered with clothes. There are two small fans or pankhas of the same material that hangs at the roof over the idol. It can be viewed that the walled area around the temple is 400 ft square and has towers at four corners. The main tower over the sanctum is generally 42 ft in height and tends to possess a copper plated sikhar.