In 1866 the New Hampshire legislature granted local businessmen a charter to build a toll road from Warner Village to the summit of Mt. Kearsarge. The Warner & Kearsarge Road Company was created and construction of the road began in 1873, with the support of the townspeople. The road traversed Mission Ridge to the Garden and continued to within "eight rods" of the summit. The road was poorly maintained and was impassable by the early 1900s. The Tory Hill Women's Club raised funds to repair the road in the 1920s. The Civilian Conservation Corps (CCC) established a camp in Warner in 1935 to finish rebuilding the road to the Garden (the present-day picnic area), and from there, the trail to the summit.

The Society for the Protection of New Hampshire Forests (SPNHF) purchased land on the mountain including the Garden in 1918. The 521-acre reservation was called Rollins Memorial Park in honor of Governor Frank W. Rollins, a founder of SPNHF. A small log shelter built near the Garden helped make the area popular with hikers. The property was transferred to the state for the establishment of a state park in 1950.