Called ICREACH, the engine operates rather like Google's search system in that it `spiders' and analyses data in multiple ways, allowing a hashed search database to be created.
According to The Intercept newswire, ICREACH has allowed various US agencies - including the FBI - to sift through more than 850 billion pieces of metadata that the NSA has collated down the years.
The `spidering' of data - unlike Google - is in both directions, meaning that users can `reverse lookup' data relationships, allowing the `creator' of a piece of data to be cross-referenced to their associates, in much the same way that BT/Post Office used to offer a reverse lookup telephone directory service in the UK until the 1980s.
That service - which was withdrawn on privacy grounds - allowed a user to give a phone number to an enquiry operator and the name of the person to be given.
The NSA, of course, has no such privacy limitations, since ICREACH is reportedly used exclusively by US government agency staff. This means, says The Intercept, that data on people around the world can be searched and analysed - even where no wrongdoing has been logged.
SCMagazineUK.com notes that ICREACH seems to be a separate operation from the so-called 215 database that the NSA uses to store information on phone calls by American citizens. The database was named after section 215 of the Patriot Act, which the NSA says allowed for the creation of the system.
Independent analyst Graham Cluley, in his analysis of ICREACH, says the database includes records obtained through Executive Order 12333, which is the main program used by the NSA to collect its data and is not subject to US Congress oversight.
"Started in 2007, ICREACH was originally intended to internally share data collected from several networks, for tracking suspect's movements, reveal political or religion affiliations and associate networks. However, according to a memo from 2010, the program has been accessible to nearly a thousand analysts working in more than 23 US government agencies that carry out intelligence work," he said on the Tripwire website.
ICREACH, he goes on to say, can process more than five billion records every day and the saved metadata has information concerning when and to whom phone calls are made or emails are sent. It may also, he notes, reveal the GPS location of a citizen's device.
"There are clearly serious questions which need to be asked about whether the authorities have overstepped their remit and invaded individuals' civil liberties and rights to privacy," he said.
Steve Smith, managing director of security consultancy Pentura, said that, irrespective of the legality of the NSA information gathering activities, this latest leak from Edward Snowden demonstrates both how far people will go to obtain personal information - and just how much information we unwittingly or unknowingly leave on the Web for others to find.
"ICREACH is obviously a highly sophisticated, resource-heavy information gathering tool that hackers and cyber-criminals are unlikely to have the man power or finances to emulate. However, looking past whether it is desirable or legal for a state to collect such information, it further highlights the importance of being careful with both personal and business data," he said.
"We need to be conscious about what Web sites, applications, devices and software we use and what data they are collecting about us. Information collected by ICREACH, such as emails, mobile phone locations, Internet chats and phone calls, could, in the wrong hands, be used for social engineering and phishing attacks," he added.
Smith concluded by saying that businesses should have a clear security and data loss prevention strategy that educates staff on best practice, to reduce the risk of a trail to sensitive business data being left behind when working on the internet.