The importance of historical data in fisheries conservation has been demonstrated, particularly for species whose populations were reduced significantly by fishing prior to the onset of ecological data collection. Historical analyses are essential for the globally endangered goliath grouper (Epinephelus itajara), which suffered large population declines throughout the 20th century. Such analyses have pressing political implications in the United States, as the estimated time to full population recovery and ensuing policy decisions depend on estimations of the baseline population before depletion. All evidence suggests that goliath grouper populations in Florida were significantly reduced, but the historic abundances were vaguely known. I analyzed over 1,000 individual fish from photos taken of the Florida Keys’ charter boat fishery to assess change in size and abundance of goliath grouper from 1950 to 1990, when all fisheries were closed. In the 1950s, charter boat trips returned with more groupers than passengers, and the biomass of fish frequently outweighed that of the fishermen on board. By the 1960s, however, significant decreases had occurred, and by 1979, the average number of groupers caught per trip decreased by a factor of four. These results suggest that recent increases observed in south Florida do not represent recovery. Without such historical baselines, appropriate restoration targets are lacking and management decisions are made without of a proper ecological context.