This was one of the first areas to be designated as a park in Japan in 1873, along with parks in Shiba (Zojo-ji Temple), Asakusa (Senso-ji Temple), Fukagawa (Tomioka Hachimangu Shrine), and Mt. Asukayama.

During the Edo Period (1603 – 1867), it was located on the grounds of Toeizan Kanei-ji Temple. After the Meiji Restoration, it came under the ownership of the nation, and in 1924, it was donated through the Imperial Household Ministry to the City of Tokyo.

Initially, the park was based around the pavilions and mausoleum of Kanei-ji Temple, the Toshogu Shrine and the cherry blossom trees on the grounds. Later, a museum, a zoo and art museum were added, adding a strong cultural element to the park.