Google, privacy, and outsourcing Total Information Awareness

The NYT has an OpEd on Google's privacy policies. There are no surprises there; one should always assume anything done online is public knowledge. Only the very sophisticated have any privacy now, and even they may be tracked by textual analysis software that can match text to identity based on idiosyncracies of expression (and, presumably, of thought -- giving new meaning to the concept of "thought police").

In general, privacy is very 20th century. I tried to fight this, but eventually I realized this was a losing battle -- especially after 9/11. Humans lived most of our existence in small communities with very little privacy; that is the world we've returned to.

There was one slightly interesting point raised in the article, though in reality it has little to do with Google:

What Google Should Roll Out Next: A Privacy Upgrade - New York Times

The government can gain access to Google's data storehouse simply by presenting a valid warrant or subpoena. Under the Patriot Act, Google may not be able to tell users when it hands over their searches or e-mail messages. If the federal government announced plans to directly collect the sort of data Google does, there would be an uproar - in fact there was in 2003, when the Pentagon announced its Total Information Awareness program, which was quickly shut down.

This is not new, in fact even I've written about this over the past few years. The Feds discovered during the 90s that the best way to deal with inconvenient legislation was to route around it by outsourcing key functions; the FBI in particular outsourced many of their information gathering functions in the 90s. More recently, of course, our wonderous government has routed around inconvenient prohibitions by outsourcing torture. In the same manner Poindexter's "Total Information Awareness" didn't "disappear" (silly idea), it merely changed names and was outsourced.

Google isn't an outsourcing tool for the TIA project, but it the Patriot Act has made Google and other online services unwitting accomplices.

Will Americans' ever catch on? Not if the media continues to completely miss the real story. I'm saddened and amused to read of privacy legislation that targets government rather than corporations. Really, it's a total waste of paper.

If Americans did catch on, what could be done? We can't stop TIA now, privacy really is history. We could, however, make the corporations implementing TIA and other programs legally liable for errors. If we don't learn lessons of the utterly incompetent 'do not fly' list program, thousands of Americans will be injured by these outsourced program. We will then be living in Gilliam's Brazil.

Total Information Awareness is not really dead ...

Remember Microsoft Passport and Intel/Microsoft's Palladium? When the public complained the names went away, but the work went forward.

Remember 'Total Information Awareness'? TIA was Poindexter's project to use massive databases to spot terrorists. It was a wee bit controversial (The 'Left Behind' people freak out about this 'number of the beast' stuff. The NRA doesn't like it either. Bush doesn't like them angry, so their opinions matter). It went away.

Sure.

As Schneier points out, it didn't go away at all. It's come back in other names and forms:

Crypto-Gram: August 15, 2005: Secure Flight

Last month the GAO issued a new report on Secure Flight. It's couched in friendly language, but it's not good...

... The TSA violated federal law when it secretly expanded Secure Flight's use of commercial data about passengers. It also lied to Congress and the public about it.

Much of this isn't new. Last month we learned that the TSA bought and is storing commercial data about passengers [jf: here he means traffic violations, credit ratings, etc. We know the quality of data in these commercial programs is utterly atrocious, and there's no regulation or feedback mechanism.], even though officials said they wouldn't do it and Congress told them not to...

... Commercial data had another use under CAPPS-II In that now-dead program, every passenger would be subjected to a computerized background check to determine their "risk" to airline safety. The system would assign a risk score based on commercial data: their credit rating, how recently they moved, what kind of job they had, etc. This capability was removed from Secure Flight, but now it's back. An AP story quotes Justin Oberman, the TSA official in charge of Secure Flight, as saying: "We are trying to use commercial data to verify the identities of people who fly because we are not going to rely on the watch list.... If we just rise and fall on the watch list, it's not adequate."

... My fear is that TSA has already decided that they're going to use commercial data, regardless of any test results. And once you have commercial data, why not build a dossier on every passenger and give him or her a risk score? So we're back to CAPPS-II, the very system Congress killed last summer. Actually, we're very close to TIA (Total/Terrorism Information Awareness), that vast spy-on-everyone data-mining program that Congress killed in 2003 because it was just too invasive.

Secure Flight is a mess in lots of other ways, too. A March GAO report said that Secure Flight had not met nine out of the ten conditions mandated by Congress before TSA could spend money on implementing the program. (If you haven't read this report, it's pretty scathing.) The redress problem -- helping people who cannot fly because they share a name with a terrorist -- is not getting any better. And Secure Flight is behind schedule and over budget.

It's also a rogue program that is operating in flagrant disregard for the law. It can't be killed completely; the Intelligence Reform and Terrorism Prevention Act of 2004 mandates that TSA implement a program of passenger prescreening. And until we have Secure Flight, airlines will still be matching passenger names with terrorist watch lists under the CAPPS-I program. But it needs some serious public scrutiny.

Aggregate databases: what my shadow medical profile will look like

In an earlier post I noted the inevitable rise of the "total information awareness" "shadow profile", the picture of a person that emerges when one aggregates data across a diverse set of databases with a variety of primary purposes: Faughnan's Notes: The national identification card and database.

My profile will make interesting reading. Due to an uncharacterized error in a payor's/provider manager information system a variety of claims are being directed against one of my employer's benefits program. The claims are for services are not covered by this benefit program, so the claims are being rejected.

In addition to not being covered, they are also not my claims. They belong to another person.

Depending on how the benefit program manages data associated with rejected claims, however, the diagnostic codes (ICD-9) may well be retained in the system and associated with my name. Ten years from now, the corrupted descendant of "total information awareness" will be browsing shadow profiles identifying persons to be added to various "yelllow and black and grey". This particular database will have an interesting range of ICD-9 codes in the database; a set that might put me on a "yellow" list.

I'll also be uninsurable. Chances are there won't be any way to know why I'll be rejected for insurance, or strip-searched every time I fly, and there will be no way to correct the error. The only saving grace is that a lot of people will be in the same boat. We can commiserate in the restricted areas where undesirables might be allowed to congregate..

The use of databases for purposes other than their original intent will cause no end of problems.

Total Information Awareness Lives

William Arkin (WaPo) lists Early 500 software tools used by the NSA in their vast data mining and intercept programs. There are no surprises here, through my work I come in contact with some of these companies. A lot of smart people work in small software companies that do contract work for the NSA.

It is an opportunity, however, to point out that Poindexter's 'Total Information Awareness' was not, of course, really terminated. The names were changed and the plans were executed. Only a naif would have bought the press stories a few years back that the program had been shuttered.

The Sun rises in the East and Data Mining doesn't work

Remember the Do Not Fly list and all the other data mined lists that our Beloved Leader and his Righteous Deputies have been using to Protect us from Evil the past 8 years? God told them it would work. They had no need of reason or logic, they Knew It Was Good.

Shockingly, and I'm not joking now, someone in power actually involved the National Research Council to see if God was telling the truth to His One True Disciple.

Even more shockingly, after years of what must have been very strenuous blocking efforts, the report is being released.

Only mildly astoundingly, it's not a forgery.

As certainly as the Sun rises in the East, the data mining efforts produce too many false positive results. They do not work for the purpose for which they were created.

Government report: Data mining doesn't work well | Politics and Law - CNET News

The most extensive government report to date on whether terrorists can be identified through data mining has yielded an important conclusion: It doesn't really work.

A National Research Council report, years in the making and scheduled to be released Tuesday, concludes that automated identification of terrorists through data mining or any other mechanism "is neither feasible as an objective nor desirable as a goal of technology development efforts." Inevitable false positives will result in "ordinary, law-abiding citizens and businesses" being incorrectly flagged as suspects.

The whopping 352-page report, called "Protecting Individual Privacy in the Struggle Against Terrorists," amounts to at least a partial repudiation of the Defense Department's controversial data-mining program called Total Information Awareness, which was limited by Congress in 2003.

But the ambition of the report's authors is far broader than just revisiting the problems of the TIA program and its successors. Instead, they aim to produce a scholarly evaluation of the current technologies that exist for data mining, their effectiveness, and how government agencies should use them to limit false positives--of the sort that can result in situations like heavily-armed SWAT teams raiding someone's home and shooting their dogs based on the false belief that they were part of a drug ring.

The report was written by a committee whose members include William Perry, a professor at Stanford University; Charles Vest, the former president of MIT; W. Earl Boebert, a retired senior scientist at Sandia National Laboratories; Cynthia Dwork of Microsoft Research; R. Gil Kerlikowske, Seattle's police chief; and Daryl Pregibon, a research scientist at Google.

They admit that far more Americans live their lives online, using everything from VoIP phones to Facebook to RFID tags in automobiles, than a decade ago, and the databases created by those activities are tempting targets for federal agencies. And they draw a distinction between subject-based data mining (starting with one individual and looking for connections) compared with pattern-based data mining (looking for anomalous activities that could show illegal activities).

But the authors conclude the type of data mining that government bureaucrats would like to do [jf: What do you mean "would like to do"?! They've been doing this for 8 years] --perhaps inspired by watching too many episodes of the Fox series 24--can't work. "If it were possible to automatically find the digital tracks of terrorists and automatically monitor only the communications of terrorists, public policy choices in this domain would be much simpler. But it is not possible to do so."

A summary of the recommendations:

* U.S. government agencies should be required to follow a systematic process to evaluate the effectiveness, lawfulness, and consistency with U.S. values of every information-based program, whether classified or unclassified, for detecting and countering terrorists before it can be deployed, and periodically thereafter.

* Periodically after a program has been operationally deployed, and in particular before a program enters a new phase in its life cycle, policy makers should (carefully review) the program before allowing it to continue operations or to proceed to the next phase.

* To protect the privacy of innocent people, the research and development of any information-based counterterrorism program should be conducted with synthetic population data... At all stages of a phased deployment, data about individuals should be rigorously subjected to the full safeguards of the framework.

* Any information-based counterterrorism program of the U.S. government should be subjected to robust, independent oversight of the operations of that program, a part of which would entail a practice of using the same data mining technologies to "mine the miners and track the trackers."

* Counterterrorism programs should provide meaningful redress to any individuals inappropriately harmed by their operation.

* The U.S. government should periodically review the nation's laws, policies, and procedures that protect individuals' private information for relevance and effectiveness in light of changing technologies and circumstances. In particular, Congress should re-examine existing law to consider how privacy should be protected in the context of information-based programs (e.g., data mining) for counterterrorism.

By itself, of course, this is merely a report with non-binding recommendations that Congress and the executive branch could ignore. But NRC reports are not radical treatises written by an advocacy group; they tend to represent a working consensus of technologists and lawyers.

The great encryption debate of the 1990s was one example. The NRC's so-called CRISIS report on encryption in 1996 concluded export controls--that treated software like Web browsers and PGP as munitions--were a failure and should be relaxed. That eventually happened two years later.

Asking this NRC group to write this report is like using a neutron bomb to kill a fly. I guess that's what it takes to stand up to a television show and God's One True Disciple.

As noted above this is not to say that the same data mining techniques could not "work" for other purposes. A medical test, for example, that's worse than useless for identifying disease in a patient may be useful for estimating the prevalence of a disease in a larger population ("pattern-based data mining" equivalent).

The key to deciding what works is to apply trivial statistic tests that have been well understood for decades using math that was old when Newton was young. Tests like sensitivity, specificity, positive predictive value, response-operator curves and so on.

Tests are good or bad based on their properties in context.

This is excruciatingly basic science.

It is a measure of the despair of our nation that it took a report from National Research Council, for the Love of Reason, to tell us what every medical student gets beaten into their heads (ok, so it doesn't always take).

Next thing you know someone will point out that the Orange Alert sign I see every time I drive by the airport is, you know ... a complete waste of electrons.

Reason. Logic.

I could cry.

Does this mean Obama stands a chance? That we're emerging from the Long Dark?

No. I am sure McCain/Palin will win. I am sure. Sure. Must not allow hope to emerge, down hope, keep back ...

NSA 2004: the loons were right, of course

In 2004 cynical geeks were convinced that Total Information Awareness, Poindexter's program of data mining, was continuing under an assumed name. Mainstream journalists classified this as "lunatic fringe". Now, of course, the "loons" have gone mainstream.

Which brings us to Gonzales. Why hasn't he gone? Why the intense focus on GOP election rigging strategies and telecom monitoring when there are so many other GOP/Cheney/Bush crimes to investigate? It's not unreasonable to assume that there's more going on that meets the eye. Something Gonzales has to cover up, something that will come to light too soon if he's gone ...

Data Mining Figured In Dispute Over NSA
By Dan Eggen and Joby Warrick
Washington Post Staff Writers
Sunday, July 29, 2007; A04

A fierce dispute within the Bush administration in early 2004 over a National Security Agency warrantless surveillance program was related to concerns about the NSA's searches of huge computer databases, the New York Times reported today.

The agency's data mining was also linked to a dramatic chain of events in March 2004, including threats of resignation from senior Justice Department officials and an unusual nighttime visit by White House aides to the hospital bedside of then-Attorney General John D. Ashcroft, the Times reported, citing current and former officials briefed on the program.

Attorney General Alberto R. Gonzales, one of the aides who went to the hospital, was questioned closely about that episode during a contentious Senate hearing on Tuesday. Gonzales characterized the internal debate as centering on "other intelligence activities" than the NSA's warrantless surveillance program, whose existence President Bush confirmed in December 2005.

FBI Director Robert S. Mueller III contradicted Gonzales, his boss, two days later, testifying before the House Judiciary Committee that the disagreement involved "an NSA program that has been much discussed."

Although the NSA's data mining efforts have been reported previously, neither Bush nor his aides have publicly confirmed that, in connection with the surveillance program, the agency had combed through phone and e-mail records in search of suspicious activity.

Nor have officials publicly discussed what prompted the legal dispute between the White House and the Justice Department.

The report of a data mining component to the dispute suggests that Gonzales's testimony could be correct. A group of Senate Democrats, including two who have been privy to classified briefings about the NSA program, called last week for a special prosecutor to consider perjury charges against Gonzales.

The report also provides further evidence that the NSA surveillance operation was far more extensive than has been acknowledged by the Bush administration, which has consistently sought to describe the program in narrow terms and to emphasize that the effort was legal.

The White House, the Justice Department and the Office of the Director of National Intelligence declined to comment last night. Calls placed to the NSA, which collected and analyzed the data, were not returned.

The warrantless surveillance program, which was authorized by presidential order after the Sept. 11, 2001, attacks, was first revealed publicly by the Times in December 2005. Bush confirmed aspects of the program at that time, defining it as monitoring communications between the United States and overseas in which one party was suspected of ties to al-Qaeda.

The Washington Post reported in February 2006 that the NSA targets were identified through data mining efforts and that thousands of Americans had been monitored. USA Today later reported that the government had the help of telecommunications companies in collecting millions of phone records.

The practice of sifting through mountains of privately collected data on phone calls and Internet communications raises legal issues. Although the contents of calls and e-mails are protected, courts have ruled that "metadata" -- basic records of calls and e-mails kept by phone companies -- are not...

I'll bet it wasn't merely international phone data. I suspect if they got a "hit" they ran through every database they could get their hands on -- and that they're false positive rates were significant. The media needs to read up on the original TIA plans, and assume that they were all implemented under different names.

Transparent society: automated monitoring of employees

I own Minority Report. I need to watch it before it's entirely passe.

For example, Social Intelligence is marketing employee behavior data mining to corporations. Forget spotting terrorists with Total Information Awareness (oh, you've already forgotten?), it's much more profitable to spot employees with a substance problem. Plus, it doesn't freak out the Tea Party if corporations do it.

If corporations don't buy, SI argues, they'll be sued the next time an employee goes postal. They should have known, lawyers will argue (and they will).

SI is also opening a subsidiary that will use bots to generate optimal online identities; burying the signal in noise. This service will be sold to employees. (I'm pretty sure Stross covered this in Accelerando, but there's lots of prior art here.)

I was joking about the employee service. SI might as well do it though. If they don't, someone else will.

There are several business opportunities here. I'm particularly looking forward to the related hire-a-hacker fund. Ten thousand people will anonymously donate a dollar for an SI related initiative.

(via Schneier).

PS. The Schneier comment thread includes some examples of name collisions and identity errors. I have one of those. My true name is somewhat unusual, and one time I flew in to give a talk only to be met by two police officers. They were looking for me as a material witness in an arson investigation. I was dressed for the presentation, so their expressions were funny to watch. Evidently I didn't look like the guy they expected ...

Tangram: TIA by any other name

No surprise:

Schneier on Security: Total Information Awareness Is Back

None of us thought that meant the end of TIA, only that it would turn into a classified program and be renamed. Well, the program is now called Tangram, and it is classified...

Cool name. If I didn't have unlimited trust in the goodness of our Glorious Leaders I'd be worried ...

Salon.com | Theocons vs. neocons: The contradictions within the republican party

Salon.com | Theocons vs. neocons

Theocons vs. neocons
With Mad Mel scaring the Jewish vote and Bush pandering to his theocratic base, the Republicans are quickly losing their secular swing voters.

---

I like the term "theocons". It's definitely catchier than "religious right" and more specific than Evangelicals. This is a scary group. At their core they believe Bush is the "Anointed President", an agent of God's will who will lead the forces of righteousness against the armies of the Antichrist. He is the end time president.

The Republican party may, at last, be coming apart at the seams. The ultra wealthy think the Bush economic policy emperils their children, their society, and their fortune. Large corporations like the service they get as corporations, but their executives don't enjoy paying bribes, and don't like anti-Gay policies. Economic conservatives are in shock. Foreign policy conservatives feel Bush/Cheney blew it in Iraq, and vaporized American credibility around the world. Libertarians worry about loss of privacy and the reincarnation of "Total Information Awareness". Intelligent conservatives and historians look at Mel Gibson and think neo-Fascism.

For all the talk about Karl Rove's genius, Bush may be blowing his reelection. Since he's evidently a very stubborn ideologue, I can only hope he'll persist.

Juvenile stupidity has a higher cost ...

Authorities arrest Minnesota teen in Internet attack: "... Tom Heffelfinger, the U.S. attorney for Minnesota, said the case will be turned over to a grand jury to decide whether more charges will be filed. If convicted of the one count already filed, Parson could be sentenced to up to 10 years in prison and a $250,000 fine."

I suspect this is a young nerd with more recklessness than sense or skill. He did not create the Blaster worm, he merely altered it and then distributed it in a way that pointed directly to him. I have a great deal of sympathy for his family.

There's more to this than meets the eye. Ever since Robert Morris (who is very skilled but shared the same teenage male recklessness as Parson) created the first worm, there's been a new level of consequence to testosterone induced male teenage stupidity. Instead of local consequences (wrecked cars), there can be global consequences (wrecked businesses).

It's the same technologically mediated expansion of consequences that has made 21st century terrorists far deadlier than 20th century variants.

So what happens in 20 years when a bio-hacking adolescent playfully creates a new organism that wipes out most of humanity?

Same behaviors, different consequences. The male frontal lobes are not fully matured to age 30. Once upon a time that was not such an enormous handicap.

Answers? I'd like to redesign humanity, but failing that I think we should get used to "Total Information Awareness" -- and we'll look back fondly at an era of freedom that future generations will be unable to imagine.

Rumsfeld's "office of strategic influence" lives on.

Back In Iraq 2.0: When is a reporter not a reporter?

---

A scoop for "Back to Iraq". Gotta love Rummy's rhetoric -- his master of scornful dismissal is unequalled. Unsurprisingly the OSI never really went away. (Nor was it merely subcontracted to Fox News.)

For that matter, I rather doubt that "total information awareness" has gone away.

On other fronts, neither Microsoft's "Palladium" nor "Passport" has gone away.

Deleting the name, and moving some bodies around, is a very good way to manage the American media and the American public.

Wired News: The Case for Terrorism Futures

Wired News: The Case for Terrorism Futures

I rather dislike Bush and I'm no friend of Poindexter's, but he was right and the media and the public are wrong. Compared to TIA (total information awareness) the futures proposal was simple, free of privacy issues, and very practical.

Within hours to days of 9/11 investigators sought evidence that terrorists had made "bets" on airplane and related stocks falling (shorted global or specific markets). This is one way to track that kind of activity.

In its absence, we'll have to rely on assessing overall trading patterns looking for evidence of "bets" that would pay off in the event of a major attack. Much less efficient.

I'm particularly impressed by the cowardice of the nation's editorial writers, who've failed to distinguish themselves in either criticizing the failures of the Bush administration or in defeinding ideas that seem unusual. I'm curious as to what Paul Krugman and The Economist will say.

PS. Years ago I made a similar proposal as part of a campaign finance reform plan.