The plan had envisaged demolition of the existing Yarra Building above basement level and construction of a two-storey replacement building.
Apple announced its plans on 20 December 2017. The decision, published on Friday, said if the application had been approved, "it would result in an unacceptable and irreversible detrimental impact on the cultural heritage significance of Federation Square".
Apple had expected to begin construction of the store in the middle of 2019 and open it by the end of 2020.
Heritage Victoria said the permit application had been advertised between 30 January and 13 February and more than 3418 public submissions were received.
It said: "The demolition of the Yarra Building would result in the loss of significant original built form fabric. Only in exceptional circumstances would a permit be provided for the demolition of a heritage building. Where demolition has been approved by Heritage Victoria in the past it has usually been due to issues relating to contamination or structural integrity. These issues are not present in respect to the Yarra Building.
"The proposed development adversely affects the cultural heritage significance of the Federation Square and its setting as it:
- "presents as a stand-alone building;
- "is visually dominant in relation to the existing buildings;
- "will detract from the design language of the existing buildings and public square;
- "encroaches into the public square; and
- "diminishes the experience of enclosure within the public square."
The negative impacts of the proposal were not outweighed by the benefits, including the economic benefits, the organisation said.
The Victorian Government said in a statement, that both it and Apple acknowledged that "based on Heritage Victoria’s decision, the proposed Apple global flagship store cannot proceed".
It said it would launch a review into Federation Square in view of the determination.