In its ruling, the court said that if access were given to retained data, then it should be exclusively for crime-fighting purposes.
It also ruled that if there were an order that such data were to be turned over to officials, then that should only be done after a court or independent administrative body had carried out a review of the order.
The appeal was launched by Labour MP Tom Watson against the 2014 Data Retention and Investigatory Powers Act which was subsequently replaced by an updated Act, otherwise known as the Snoopers Charter, in 2016.
"It was rushed through Parliament just before recess without proper parliamentary scrutiny," he said.
“The government must now bring forward changes to the Investigatory Powers Act to ensure that hundreds of thousands of people, many of whom are innocent victims or witnesses to crime, are protected by a system of independent approval for access to communications data. I’m proud to have played my part in safeguarding citizens' fundamental rights.”
Under the 2016 Act, Internet, phone and communications applications firms had to store records for a year and allow law enforcement to access the data on demand. Government agencies were allowed to order companies to decrypt data, making end-to-end encryption of little use in maintaining privacy.
The 2016 law also allows the UK Government to do bulk hacking of networks, devices and services.
In a statement, the British civil liberties group Liberty said the government had already anticipated the court ruling and conceded that there would need to be changes to the act.
"But its half-baked plans do not even fully comply with past court rulings requiring mandatory safeguards – and they continue to allow public bodies to indiscriminately retain and access personal data, including records of internet use, location tracking using mobile phones and records of who we communicate with and when," Liberty said.
Liberty director Martha Spurrier said: “Yet again a UK court has ruled the government’s extreme mass surveillance regime unlawful. This judgement tells ministers in crystal clear terms that they are breaching the public’s human rights. The latest incarnation of the Snoopers’ Charter, the Investigatory Powers Act, must be changed.
“No politician is above the law. When will the government stop bartering with judges and start drawing up a surveillance law that upholds our democratic freedoms?"