Soon after the Count's departure, the vineyards and Buena Vista became dormant, the vineyards choked with weeds and chaparral. The stone winery suffered from storm and earthquake damage, eventually becoming more or less a heap of stones.
The Haraszthy Villa, now empty, sat on a knoll overlooking the tangled vineyard-the Villa now reminiscent of the past glories: the grand gala balls which at one day had gone on were now part of history.
There followed a long interim period with no vineyard or wine making activity.
The property was acquired by Mr. and Mrs. Johnson in 1880, who were not interested in the vineyards or the winery, but built an estate featuring exceptional landscaping and particularly a three-story imposing Victorian mansion.
After the death of the Johnsons, the property was acquired in the early 1900's by Henry Cailleaud, who used the Johnsons' "Castle" as a resort, "with the very finest cuisine, hot and cold water, fishing, hunting, mineral springs, billiards, croquet", the rates were $10 per week American Plan.
In 1920-1, the State of California acquired the property planning to use it as an industrial farm for women and housing them as residents in "The Castle". Shortly thereafter the Castle burned and the State built a Spanish style building across the creek for the residents. The structure today is leased by the Foundation to the Vineburg Company (an affiliate of Gundlach-Bundschu, and not related to the Foundation other than as tenant), which operates the Bartholomew Park Winery in the building.
The State project, however, was short-lived and was abandoned by the late 1930's.
Then, for over a decade, the land was completely unoccupied and dormant, and an historic corner of Sonoma Valley was forgotten and abandoned-a veritable ruin.