Fast-track legislation is to be rushed through Parliament with cross-party support in order to maintain the ability of police and security services to access telephone and internet data.
Home Secretary Theresa May warned MPs that innocent lives would be lost if Parliament did not act swiftly in response to a European Court of Justice ruling which raised the prospect that communications companies could start deleting crucial material used to tackle terrorists and serious criminals.
And she said that action was needed to confirm the legal basis for security and intelligence agencies to intercept the content of suspects' emails and phone calls under warrant and head off the risk that communications companies based overseas might withdraw co-operation.
Announcing the Data Retention and Investigation Powers Bill at a rare joint press conference in 10 Downing Street, Prime Minister David Cameron and his Liberal Democrat deputy Nick Clegg stressed that the legislation would maintain the authorities' existing powers rather than add to them.
Mr Clegg insisted it did not represent a revival of the so-called "snooper's charter" which he blocked last year, and which Mr Cameron and Mrs May have said they will reintroduce if Conservatives win next year's general election.
They outlined a series of additional safeguards which they said would maintain the balance between security and privacy, including a "poison pill" clause which will terminate the legislation at the end of 2016, forcing the next Government to debate and pass a replacement bill.
A new US-style Privacy and Civil Liberties Oversight Board will be created, there will be a reduction in the number of public bodies - such as the Royal Mail and Charity Commission - able to ask communications companies for data, and annual transparency reports will set out publicly the way surveillance powers are being used. A senior diplomat will lead talks with the US and the internet companies on forging an international data-sharing agreement.
Mr Cameron warned: "Unless we act now companies will no longer retain the data about who contacted who, where and when and we will no longer be able to use this information to bring criminals to justice and keep our country safe.
"This is at the heart of our entire criminal justice system. It is used in 95% of all serious organised crime cases handled by the CPS. It's been used in every major security service counter-terrorism investigation over the past decade."
He added: "We face real and credible threats to our security from serious and organised crime, from the activity of paedophiles, from the collapse of Syria, the growth of Isis in Iraq and al Shabab in East Africa.
"I am simply not prepared to be a prime minister who has to address the people after a terrorist incident and explain that I could have done more to prevent it."
The Government has been forced to act by a European Court of Justice (ECJ) ruling in April that a European Union data retention directive, implemented by Labour in 2009, was invalid because it interferes with the fundamental right to respect for private life.
Police and security services raised fears that without this legal underpinning, companies would start deleting data that is crucial for investigations into a range of serious crime including terrorism, child pornography and drugs trafficking.
The Bill will mean firms can retain data for 12 months.
Mr Cameron said the UK was also facing a "cliff edge" within weeks, after which foreign-based communications companies may stop complying with warrants for legal interception of suspects' calls because the requirement to do so was not written into the law.
It is understood that Lib Dems who played a key role in torpedoing the "snooper's charter", such as Julian Huppert, have been closely involved in designing the safeguards.
Mr Clegg said: "Liberty and security must go hand in hand. We can't enjoy our freedom if we are unable to keep ourselves safe.
"So I wouldn't be standing here today if I didn't believe there is an urgent challenge facing us. No government embarks on emergency legislation lightly but I have been persuaded of the need to act and act fast."
The proposals were discussed at an emergency Cabinet meeting this morning, and are expected to be introduced in the House of Commons on Tuesday and rushed through both houses within three days.
Labour leader Ed Miliband and shadow home secretary Yvette Cooper confirmed that Labour will support the emergency legislation, telling the party's MPs in a letter: " Serious criminal investigations and counter terrorism intelligence operations must not be jeopardised. That is why we are supporting this emergency legislation which we accept is designed solely to protect existing capabilities."
But Ms Cooper said: "There will be serious concern in Parliament and across the country at the lateness of this legislative proposal and the short time to consider something so important."
Labour backbencher Tom Watson denounced the emergency legislation as a "stitch-up" that was denying MPs the chance to canvass voters' views and said he would vote against its parliamentary timetable and probably elements of the package itself.
Shami Chakrabarti, director of campaign group Liberty, said: "The Government says it's only plugging loopholes but its existing blanket surveillance practice has been found unlawful. We are told this is a paedophile and jihadi 'emergency', but the court judgment they seek to ignore was handed down over three months ago and this isn't snooping on suspects but on everyone.
"We are promised greater scrutiny and debate but not until 2016, as it seems that all three party leaders have done a deal in private. No privacy for us and no scrutiny for them. Will Clegg and Cameron's 'debate for the future' really comfort voters and companies today?"
In a statement to the Commons, Mrs May said the emergency legislation would allow police and other law enforcement agencies to continue to investigate criminality online.
"Without this legislation, we face the prospect of losing access to this data overnight with the consequence police investigations will suddenly go dark and criminals will escape justice," she said. "We cannot allow that to happen."
Mrs May said the Government would be acting in a "negligent" way if it did not act and insisted the Bill had to be law before the summer recess at the latest - scheduled for July 22.
She said: "(Police and intelligence agencies) are clear we need to act immediately. If we do not, criminals and terrorists will go about their work unimpeded and innocent lives will be lost."
Britain's most senior police officer Sir Bernard Hogan-Howe said that the data measures are vital for a wide range of serious crime investigations.
Speaking at London's City Hall this morning, the Metropolitan Police commissioner said: "All we're trying to do is to maintain the level of surveillance that we've already got. If we don't do this we lose it, and it's vital. It's vital not only to counter-terrorism operations but particularly to serious crime.
"At some levels it geniunely helps us save lives, for example if we have kidnaps it's a vital thing that we need, it's also important in homicide investigations.
"If we lose it - and there's some danger that we are already losing it - then we will all be less safe."
Emma Carr, acting director of privacy campaign group Big Brother Watch, said: "It is a basic principle of a free society that you don't monitor people who are not under suspicion.
"Considering the Snoopers' Charter has already been rejected by the public as well as by the highest court in Europe, it is essential that the Government does not rush head-first into creating new legislation.
"The EU's data retention laws privatised snooping, meaning companies were paid by governments to record what citizens were doing and retain that information for a year.
"We need to get back to a point where the police monitor people who are actually suspected of wrongdoing, rather than wasting millions every year requiring data to be stored on an indiscriminate basis."
The executive director of the Open Rights Group (ORG), Jim Killock, said: "The Government knows that since the ECJ ruling, there is no legal basis for making internet service providers retain our data so it is using the threat of terrorism as an excuse for getting this law passed. The Government has had since April to address the ECJ ruling but it is only now that organisations such as ORG are threatening legal action that this has become an 'emergency'.
"Not only will the proposed legislation infringe our right to privacy, it will also set a dangerous precedent where the Government simply re-legislates every time it disagrees with a decision by the ECJ. The ruling still stands and these new plans may actually increase the amount of our personal data that is retained by ISPs, further infringing on our right to privacy.
"Blanket surveillance needs to end. That is what the court has said."
Tory MP David Davis, a former shadow home secretary, claimed there was only a "theatrical emergency" because officials had been aware of the ECJ ruling since April.
"They should have been prepared for it. Any competent department of state would have looked at the options, looked why it might go through, what might be struck down and so on and had legislation prepared then," he told BBC Radio 4's World At One.
"We would have then had three months up until now to consider what we should do rather than this thing that's going through in a single day. Extraordinarily unusual."
Explaining his decision to back the legislation, Mr Miliband said: "My first and fundamental duty is to ensure the security and safety of our citizens.
"I am convinced major investigations into terrorism and organised crime would be jeopardised if we don't pass legislation and that would jeopardise the security and safety of our citizens. That's why I believe legislation is necessary.
"I also believe that we need to ensure that we protect the liberty of our citizens as well as their security and that's why I'm pleased that the Government has agreed that for the first time there is cross-party support for an independent review of all of these issues."