Thoughts – Is Our Information Sufficiently Secure From Theft and Illicit Use?


The question of whether our information in the information age is sufficiently secure from theft and illicit use has an obvious answer. No. What is not as obvious is exactly why our information is not sufficiently secure from theft and illicit utilization.   My original masters thesis at the Fletcher school of Law and Diplomacy, written in 2011, addressed this question and comprises Part 3 of this research blog. I conclude Part 3 with the point that every human has a digital life and a real life that are effectively one in the same, however do not share the same set of civil liberties. Moreover, I explain how cost and complexity have been barriers to adoption for technological security solutions, but recommend intelligent multi-factor authentication and end-to-end encryption as answers to the technological problem. I also suggest a new method for interpreting the 4th amendment that recognizes our digital lives and real lives as being effectively one in the same.

In this post I am revisiting the same question, but writing from the point of view of an American citizen analyzing the juxtaposition of U.S. counter terrorism & intelligence gathering policies and programs vs. how the U.S. government protects its own information and digital life from similar intelligence policies of foreign adversaries. The job of the U.S. government is to protect its citizens, however if the U.S. government cannot protect itself in its digital life, then how can U.S. citizens have confidence the government can protect its real life and ultimately our own in this brave new world? Lets now digest some background information for further context.


In 2013 Edward Snowden released the single largest trove of top-secret national security documents to journalists. To this day, the U.S. government doesn’t know exactly how many or what documents he took. The simple fact that Snowden walked out with this information virtually undetected warrants a moment of pause and reflection. However, after two years of stunning national surveillance revelations one would think the U.S. government would have learned its lesson on protecting its own data. On June 4th, 2015 it was reported in the Wall Street Journal the Office of Personnel Management within the U.S. government had suffered a major breach of its records. The personnel records, security clearance applications, etc. within the Office of Personnel Management were accessed exposing approximately 4 million current and formers employees of the U.S Government. Although the government recently suggested it could be up to 18 million or higher. It’s also widely speculated the databases this critical information is stored in were not encrypted either, which many legislators and pundits think should be a criminal offense. The irony in this position is that the head of the FBI, James Comey, and the Department of Justice have been demonizing the technology industry for its expansion of encryption products and suggesting these companies are aiding terrorists, etc.(1)

In direct response to the Snowden revelations the technology industry has been fervently increasing the security of their products by implementing end-to-end encryption and intelligent-multifactor authentication. The U.S. government is embarrassed by the Snowden revelations and is publicly demonizing the tech industry for, in the government’s words, “putting critical information outside the hands of the law”. All while the U.S. government has proven to be insufficiently protecting the sensitive information in its own networks and is now rushing to institute encryption and intelligent multi-factor authentication. Lets now explore why this is the case because, in my opinion, this is the very tip of how technology is going to start changing politics again. This will be done by briefly exploring again evolutions in Smartphone and tablet computer technology when coupled with social media services. Then we will explore the nature of digital threats, how not all encryption methods are equal, and how governments are preparing for cyberwarfare.

Smartphone & Tablet Computing

According to Gartner research global Smartphone shipments surpassed one billion units in 2014 and Smartphone sales represented two-thirds of global phone market.(2) Sales of Smartphones to end-users totaled 1.2 billion units, up 28.4 percent from 2013.(3) These devices continue to increase in their processing power at every product iteration while also becoming more reliable and stable computing systems. The telling sign, it’s expected by Gartner research in 2015, tablet sales will for the first time outpace the sales of regular personal computers.(4) Gartner is predicting in 2015 there will be nearly 321 million tablets shipped, versus close to 317 million personal computers.(5) So the big news is that in 2015 there will be more tablets sold than personal computers. The compact nature of devices including watches, etc enables humans to measure behavior and express themselves in ways never before done. So what are people doing on all these Smartphones, tablets, and other wearable tech devices?

Social Media & The Internet of Things (IOT)


The utilization of mobile Internet devices, thanks to new application environments and mobile Internet browser technologies, has opened the door for Software-as-a-Service (SaaS) to change peoples lives. This combination serves the personal and professional needs on devices that are rarely three feet from their owners, twenty-four hours a day, seven days a week.

One single phenomenon that has changed the way people communicate and connect has been the advent of social media. Services like Facebook, linkedin, and Twitter have revolutionized the way people connect, communicate, express themselves, and consume content and information.

Social media is the digital equivalent of how people previously interacted face to face, through email, text message, or heaven forbid an actual phone call. Interestingly enough, Facebook and the Internet phone service Skype, owned by Microsoft, are in partnership to integrate their services (6). Facebook alone has over 1.4 billion people utilizing its services.(7) However, the unintended, or intended, consequence of social media has been the vast amount of personal identifiable information (PII) people have shared about themselves. From pictures, favorite restaurants, movies, music, hobbies, you name it, people have exposed themselves, all voluntarily. Additionally, people are expressing their every move and holding public conversations on message boards about all facets of their personal and periodically their professional lives.  This information is also not owned by the user.

Social media has become the equivalent hanging out at the local pub; except there are over 1.4 billion other people sharing stools at the same counter and they can ‘hear’ almost every word. If they missed it the first time, Facebook and other platforms, have made it easy for viewers to go back into an individuals social profile history and see what people have shared publicly. People are practicing their first amendment rights of freedom to express and freedom of speech, however they are also leaving a quantitatively large and qualitatively useful pool of information about who they are and what they do every day of the week. All this personal information about habits, desires, friends, political opinions, personal grievances, deaths in the family, are all being monitored and tracked by someone or something. Facebook is not the only company in the data mining and aggregation business.

With the advancements in smart phone technology and social media software services, people are exercising their first amendment rights every hour in a forum and method where the protection of those rights are not the same as they are in their physical person. In my last research post I explain in detail how the U.S. government and legal scholars plan on using the information technology industry to restrict our First Amendment rights.

The combination of smart phone technology and social media Internet services has created the equivalent of a digital twin for everyone. The difference is that our fourth amendment rights are not comparable to that of our twin. This is thanks to the capacity expansion of the USA Patriot Act, FISA Amendment Act that granted government agencies the ability to shape how the US Constitution’s Fourth Amendment is being interpreted. This can be clearly seen in a recent article published by ProPublica & The New York Times based on Snowden documents that reveal secret U.S. Department of Justice Memos the Expand Spying. Lets now discuss the nature of the digital threat, why not all encryption methods are equal, and how governments are preparing for cyber warfare.

The Digital Threat

The frequency and sophistication of intrusions into government and civilian institutions has increased over the past ten years.(8) Everyday networks across the Internet are scanned and probed thousands of times.(9) Every year Verizon Wireless and the U.S. Secret Service work together on a body of research aimed at determining the number of data breaches in the United States and their fundamental nature.(10) Since 2010 the number of data breaches has increased exponentially.(11)  Below is a graphical representation of the types and instances experienced determined by the contributors in the 2015 Verizon report. (You can read the entire report here)

The 2015 report is based on data from:

– 79,790 security incidents

– 2,2122 data breaches

– 70 contributors, including incident response forensics firms, government agencies, Computer Security Information Response Teams (CIRTs), security vendors, and others

Top 5 Industries Most Breached

The Verizon DBIR covers a plethora of information and charts. Some of these speak to security incidents and others speak to data breaches. To avoid confusion, lets clarify these terms:

  • Security Incident – An event that compromises the confidentiality, integrity, or availability of data. It’s less severe than a breach.(12)
  • Data Breach – A confirmed disclosure of data on an unauthorized party. This is more serious than an incident.(13)

The top three industries affected by security incidents remain the same as last year: Public, Information, and Financial Service Sectors.(14) However, when looking at breaches a different picture arises:


The top most-breached industries in descending order are:

  • Public
  • Financial Services
  • Manufacturing
  • Accomodation
  • Retail

Manufacturing is in the top three for breaches but not security incidents.(15) This may be related to it being the industries most-targeted for cyber espionage.(16) In two of the top five, small organizations appear to be breached far more often than large ones:

  • In retail, the researchers found more than four-times as many breaches of small organizations
  • In accommodation, they found a whopping 18-times as many

1 in 4 Breaches hits Point of Sale Machines (POS)

Last years data breach investigations report noted that 92% of the more than 100,000 breaches analyzed by Verizon over the last 10 years fell into nine basic patterns, or types of threats.(17)

The threat landscape did not change dramatically in 2014. The chart below shows the “incident classification patterns” with the greatest number of breaches for the year.(18)


Note that POS intrusions accounted for 1 in 4 breaches observed last year (not surprising given the major retail breaches in the news of Target, Home depot, etc.) (19) Combined with crimeware, these two threats comprise nearly half of all the breaches for 2014. (20)

Things get even more interesting as we review the distribution of breaches by the type of threat across industries:


More than 90% of breaches in the accommodation sector hit point-of-sale machines.(21) POS systems were also the biggest targets for the entertainment and retail industries.(22)

Cyber espionage hit manufacturing and professional organizations particularly hard, and espionage combined with crimeware accounted for almost 95% of all breaches in manufacturing. (23)

Shooting Phish in a Barrel

Though it doesn’t contain a chart, the phishing section of the DBIR is rather disturbing. In short: phishing is just too easy. On average, phishing emails can receive email open and click rates that rival email marketing of the business world:

  • 23% of recipients open phishing messages(24)
  • 11% click on attachments(25)

Think about those stats for a moment. A phishing campaign sent to 100 people will net 10 to 12 victims in the catch. Small, targeted campaigns are almost guaranteed to work:

  • A campaign of just 10 emails yields a greater than 90% chance that at least one person will become the criminal’s prey, according to the Verizon data breach report.(26)

In a controlled test involving more than 150,000 emails, Verizon’s team found the median time-to-first click was 1 minute 22 seconds. Nearly 50% of people opened and clicked in the first hour.(27) 

Cyber Espionage Loves Email

When you consider the term “Cyber Espionage,” you may think of huge countries with nearly infinite resources launching the most sophisticated, cutting edge attacks across the globe.(28) Surprisingly, most espionage begins with a simple email, according to the 2015 Verizon report:


Three out of four (77.3%) of these attacks require someone to engage with an email attachment or email link.(29) The report notes that web drive-by attacks were more popular in espionage than prior years.(30) What are these actors looking for? Your secrets! The second highest category, “credentials” were targeted in 11.4% of the attacks.(31) The industries most commonly attacked via cyber-espionage in 2014 were manufacturing, public, professional, and information, as you can see in the chart below.(32) This is partly why two of these industries, manufacturing and public, were amongst the most breached overall.(33)


Ram Scrapers Are Growing Fast

Verizon’s 2015 data breach report also looks at threat actions, which can be roughly summarized by the type of attack behind a breach. Examples include POS intrusions, web app attacks, insider misuse, etc.(34) Phishing attacks continue to increase but their growth has slowed.(35) The real break-out is RAM scraping which has seen tremendous growth since 2012.(36)

8-DBIR-chart-significant-threat-actionsRam scraping malware was used in the majority of breaches at national retailers such as Target, Home Depot, and many others.(37) Ram scraping is commonly used by malware on point-of-sale systems. POS machines often hold cardholder data in memory a moment before its encrypted. This tiny window provides enough time for malware to scrape the unencrypted data and send it to a log file.(38)

Keystroke logging seems to be falling out of fashion as RAM scraping makes its rise.(39) And phishing may have lost ground in 2013, but it has climbed back to exceed its 2012 level.(40) Stealing and compromising access credentials remains the most common threat action.(41) Nothing beats having the keys to the front door.(42)

External Threats are STILL Greater

Internal actors may enable a breach inadvertently, but the overwhelming percentage of breaches were caused by external threats.(43)


More than 80% of breaches reviewed in the report are attributed to external threats.(44) Roughly 17% are from internal actors, and a small number are attributed to partners.(45) So the enemy is not within but external actors will find a meaningful utilization of your internal resources against you.(46)

DDoS Attacks Double in 2014

Denial of Service (DOS) attacks were also in the news last year.(47) Although not quite as prominent a topic as ransomeware or retail data breaches, the number of attacks doubled according to the report’s authors.(48) The most affected industries are the public, retail, and financial services sectors.(49) As you can see in the chart below, these attacks may target large organizations (those with more than 1,000) employees more often, but the overwhelming majority hit organizations of unknown size.(50)


So the big story in examining the digital threat is that our information is just as, if not more, at risk today than ever before. So how is all this information being stored and why do not all encryption methods live up to their promises? Lets examine.

Why Not All Encryption Methods Live Up to Their Promises

Encryption very simply is the utilization of mathematics to protect communications from spying – is used for electronic transactions of all types, by governments, firms and private users alike.(51) A main theme from the Edward Snowden NSA surveillance revelations is the topic of encryption and a recent article published by the German newspaper Der Spiegel goes further into the topic through the lens of the Snowden archive to show that not all encryption methods live up to their promises and why.(52) Some of the most well respected experts on encryption technologies either co-wrote or consulted on the article and supporting documentation. I rely on their expertise and writing extensively here.

One example is the encryption featured in Skype, a program used by some 300 million users to conduct Internet video chat that is touted as secure.(53) It isn’t really. “Sustained Skype collection began in Feb 2011,” reads a National Security Agency (NSA) training document from the Edward Snowden archive.(54) Less than half a year later, in the fall, the code crackers declared their mission accomplished.(55) Since then, data from Skype has been accessible to the NSA snoops.(56) Software giant Microsoft, which acquired Skype in 2011, said in a statement: “We will not provide governments with direct or unfettered access to customer data or encryption keys.”(57) The NSA had been monitoring Skype even before that, but since February 2011 the service has been under order from the secret U.S. Foreign Intelligence Surveillance Court (FISC), to not only supply information to the NSA but also to make itself accessible as a source of data for the agency.(58)

The “sustained Skype collection” is a further step taken by the authority in the arms race between intelligence agencies seeking to deny users of their privacy and those wanting to ensure they are protected.(59) There have also been some victories for privacy, with certain encryption systems proving to be so robust they have been tried and true standards for more than 20 years.(60)

For the NSA, encrypted communication – or what all other Internet users would call secure communication – is “a threat”.(61) In one internal NSA training document an NSA employee asks: “Did you know that ubiquitous encryption on the Internet is a major threat to NSA’s ability to prosecute digital-network intelligence (DNI) traffic or defeat adversary malware?”(62)

The Snowden documents reveal the encryption programs the NSA has succeeded in cracking, but, importantly, also the ones that are still likely to be secure.(63) Although the documents are around two years old, experts consider it unlikely the agency’s digital spies have made much progress in cracking these technologies.(64) “Properly implemented strong crypto systems are one of the few things that you can rely on,” Snowden said in June 2013, after fleeing to Hong Kong.(65)

The digitization of society in the past several decades has been accompanied by the broad deployment of cryptography, which is no longer the exclusive realm of secret agents.(66) Whether a person is conducting online banking, Internet shopping or making a phone call, almost every Internet connection today is encrypted in some way.(67) The entire realm of cloud computing – that is of outsourcing computing tasks to data centers somewhere else, possibly even on the other side of the globe – relies heavily on cryptographic security systems.(68) Internet activists even hold crypto parties where they teach people who are interested in communicating securely and privately how to encrypt their data.(69)

In Germany, concern about the need for strong encryption goes right up to the highest levels of government.(70) Chancellor Angela Merkel and her cabinet now communicate using phones incorporating strong encryption.(71) The government has also encouraged members of the German public to take steps to protect their own communication.(72) Michael Hange, the president of the Federal Office for Information Security, has stated: “We suggest cryptography – that is, consistent encryption.”(73)

It’s a suggestion unlikely to please some intelligence agencies.(74) After all, the Five Eyes alliance – the secret services of Britain, Canada, Australia, New Zealand and the United States – pursue a clear goal: removing the encryption of others on the Internet wherever possible.(75) In 2013, the NSA had a budget of more than $10 billion.(76) According to the U.S. intelligence budget for 2013, the money allocated for the NSA department called Cryptanalysis and Exploitation Services (CES) alone was $34.3 million.(77)

Last year, the Guardian, New York Times, and ProPublica reported on the contents of a 2010 presentation on the NSA’s BULLRUN decryption program, but left out many specific vulnerabilities.(78) The presentation states that, “for the past decade, NSA has led an aggressive, multipronged effort to break widely used Internet encryption technologies,” and “vast amounts of encrypted Internet data which have up till now been discarded are now exploitable.”(79) Decryption, it turns out, works retroactively – once a system is broken, the agencies can look back in time in their databases and read stuff they could not before.(80) This specific risk is why in Part 1 of my research blog, I assert under the 5th Amendment your digital life should never be able to incriminate your real life, because they are effectively one in the same.

The number of Internet users concerned about privacy online has risen dramatically since the first Snowden revelations.(81) But people who consciously use strong end-to-end encryption to protect their data still represent a minority of the Internet-using population.(82) There are a number of reasons for this: Some believe encryption is too complicated to use.(83) Or they think the intelligence agency experts are already so many steps ahead of them that they can crack any encryption program.(84)

Still Safe from the NSA

This isn’t true.(85) As one document from the Snowden archive shows, the NSA had been unsuccessful in attempts to decrypt several communications protocols, at least as of 2012.(86) An NSA presentation for a conference took place that year lists the encryption programs the Americans failed to crack.(87) In the process, the NSA cryptologists divided their targets into five levels corresponding to the degree of difficulty of the attack and the outcome, ranging from “trivial” to “catastrophic.”(88)

Attacks against Crypto (Reference Documents)

Monitoring a documents path through the Internet is classified as “Trivial.”(89) Recording Facebook chats is considered a “minor” task, while the level of difficulty involved in decrypting emails sent through Moscow-based Internet service provider “” is considered “moderate.”(90) Still, all three of those classifications don’t appear to pose any significant problems for the NSA.(91)

Things first become troublesome at the fourth level.(92) The presentation states that the NSA encounters “major” problems in its attempts to decrypt messages sent through heavily encrypted email service providers like Zoho or in monitoring users of the Tor network, which was developed for surfing the web anonymously.(93) Tor, otherwise known as The Onion Router, is free and open source software that allows users to surf the web through a network of more than 6,000 linked volunteer computers.(94) The software automatically encrypts data in a way that ensures that no single computer in the network has all of a user’s information.(95) For surveillance experts, it becomes very difficult to trace the whereabouts of a person who visits a particular website or to attack a specific person while they are using Tor to surf the web.(96)

Cryptanalytics (Reference Documents)

The NSA also has “major” problems with Truecrypt, a program for a encrypting files on computers.(97) Truecrypt’s developers stopped their work on the program last May, prompting speculation about pressures from government agencies.(98) A protocol called Off-the-Record (OTR) for encrypting instant messaging in an end-to-end encryption process also seems to cause the NSA major problems.(99) Both are programs whose source code can be viewed, modified, shared and used by anyone.(100) Experts agree it is far more difficult for intelligence agencies to manipulate open source software programs than many of the closed systems developed by companies like Apple and Microsoft.(101) Since anyone can view free and open source software, it becomes difficult to insert secret back doors without it being noticed.(102) Transcripts of intercepted chats using OTR encryption handed over to the intelligence agency by a partner in Prism – an NSA program that accesses data from at least nine American Internet companies such as Google, Facebook, and Apple – show that the NSA efforts appear to have been thwarted in these cases: “No decrypt available for this OTR message.”(103) This shows that OTR at least sometimes makes communications impossible to read for the NSA.(104)

Things become “catastrophic” for the NSA at level five – when, for example, a subject uses a combination of Tor, another anonymization service, the instant messaging system CSpace and a system for Internet telephony (voice over IP) called ZRTP.(105) This type of combination results in a “near-total loss/lack of insight to target communications, presence,” the NSA documents state.(106)

ZRTP, which is used to securely encrypt conversations and text chats on mobile phones, is used in free and open source programs like RedPhone and Signal.(107) “It’s satisfying to know that the NSA considers encrypted communication from our apps to be truly opaque,” says RedPhone developer Moxie Marlinspike.(108)

Too Robust for Fort Meade

Also, the “Z” in ZRTP stands for one of its developers, Phil Zimmermann, the same man who created Pretty Good Privacy, which is still the most common encryption program for emails and documents in use today.(109) PGP is more than 20 years old, but apparently it remains too robust for the NSA spies to crack.(110) “No decrypt available for this PGP encrypted message,” a further document viewed by the contributors of the Speigel article states of emails the NSA obtained from Yahoo.(111)

Phil Zimmerman wrote PGP in 1991.(112) The American nuclear weapons freeze activist wanted to create an encryption program that would enable him to securely exchange information with other like-minded individuals.(113) His system quickly became very popular among dissidents around the world.(114) Given its use outside the United States, the U.S. Government launched an investigation into Zimmerman during the 1990’s for allegedly violating the U.S. Arms Export Control Act.(115) Prosecutors argued that making encryption software of such complexity available abroad was illegal.(116) Zimmerman responded by publishing the source code as a book, an act that was constitutionally protected as free speech.(117)

PGP continues to be developed and various versions are available today.(118) The most widely used is GNU Privacy Guard (GnuPG), a program developed by German programmer Werner Koch.(119) One document shows that the Five Eyes intelligence services sometimes use PGP themselves.(120) The fact is that hackers obsessed with privacy and the U.S. authorities have a lot more in common than one might initially believe.(121) The Tor project, was originally developed with the support of the U.S. Naval Research Laboratory.(122)

Deanonymizing (Reference Documents)

Today, NSA spies and their allies do their best to subvert the system their own military helped conceive, as a number of documents show.(123) Tor deanonymization is obviously high on the list of NSA priorities, but the success achieved here seems limited.(124) One GCHQ document from 2011 even mentions trying to decrypt the agencies’ own use of Tor – as a test case.(125)

To a certain extent, the Snowden documents should provide some level of relief to people who thought nothing could stop the NSA in its unquenchable thirst to collect data.(126) It appears secure channels still exist for communication.(127) Nevertheless, the documents also underscore just how far the intelligence agencies already go in their digital surveillance activities.(128) Internet security comes at various levels – and the NSA and its allies obviously are able to “exploit” – i.e. crack – several of the most widely used ones on a scale that was previously unimaginable.(129)

VPN Security only Virtual

One example is virtual private networks (VPN), which are often used by companies and institutions operating from multiple offices and locations.(130) A VPN theoretically creates a secure tunnel between two points on the Internet.(131) All data is channeled through that tunnel, protected by cryptography.(132) When it comes to the level of privacy offered here, virtual is the right word, too.(133) This is because the NSA operates a large-scale VPN exploitation project to crack large numbers of connections, allowing it to intercept the data exchanged inside VPN – including, for example, the Greek government’s use of VPN’s.(134) The team responsible for the exploitation of those Greek VPN communications consisted of 12 people, according an NSA document reviewed by the Der Spiegel article authors.(135)

Attacks on VPN (Reference Documents)

The NSA also targeted SecurityKiss, a VPN service in Ireland.(136) The following fingerprint for Xkeyscore, the agency’s powerful spying tool, was reported to be tested and working against the service(137):

fingerprint(‘encryption/securitykiss/x509) = $pkcs and ( ($tcp and from_port(443)) or ($udp and (from_port(123) or from_por (5000) or from_port(5353)) ) ) and (not (ip_subnet(‘ or ‘ or ‘ )) ) and ‘RSA Generated Server Certificate’c and ‘Dublin1’c and ‘GL CA’c;

According to an NSA document dating from late 2009, the agency was processing 1,000 requests an hour to decrypt VPN connections.(138) This number was expected to increase to 100,000 per hour by the end of 2011.(139) The aim was for the system to be able to completely process “at least 20 percent” of these requests, meaning the data traffic would have to be decrypted and reinjected.(140) In other words, by the end of 2011, the NSA’s plans called for simultaneously surveilling 20,000 supposedly secure VPN communications per hour.(141)

VPN connections can be based on a number of different protocols.(142) The most widely used ones are called Point-to-Point Tunneling Protocol (PPTP) and Internet Protocol Security (Ipsec).(143) Both seem to pose few problems for the NSA spies if they really want to crack a connection.(144) Experts have considered PPTP insecure for some time now, but it is still in use in many commercial systems.(145) The authors of one NSA presentation boast of a project called FOURSQUARE that stores information including decrypted PPTP VPN metadata.(146)

Using a number of different programs, they claim to have succeeded in penetrating numerous networks.(147) Among those surveilled were the Russian carrier Transaero Airlines, Royal Jordanian Airlines as well as Moscow-Based telecommunications firm Mir Telematiki.(148) Another success touted is the NSA’s surveillance of the internal communications of diplomats and government officials from Afghanistan, Pakistan, and Turkey.(149) Ipsec as a protocol seems to create slightly more trouble for the spies.(150) But the NSA has the resources to actively attack routers involved in the communication process to get to the keys to unlock the encryption rather than trying to break it, courtesy of the unit called Tailored Access Operations: “TAO got on the router through which banking traffic of interest flows,” it says in one presentation.(151)

Anything But Secure

Even more vulnerable than VPN systems are the supposedly secure connections ordinary users must rely on all the time for web applications like financial services, e-commerce or accessing webmail accounts.(152) A lay user can recognize these allegedly secure connections by looking at the address bar in his or her Web browser: With these connections, the first letters of the address there are not just http – for Hypertext Transfer Protocol – but https.(153) The “s” stands for “secure”.(154) The problem is that there isn’t really anything secure about them.(155)

Attacks on SSL/TLS (Reference Documents)

The NSA and its allies routinely intercept such connections – by the millions.(156) According to an NSA document, the agency intended to crack 10 million intercepted https connections a day by late 2012.(157) The intelligence services are particularly interested in the moment when a user types his or her password.(158) By the end of 2012, the system was supposed to be able to “detect the presence of at least 100 password based encryption applications” in each instance some 20,000 times a month.(159) This is why Intelligent Multi-Factor Authentication is so important in a security stack, it will thwart this point of weakness.

For it’s part, Britain’s GCHQ collects information about encryption using the TLS and SSL protocols – the protocols https connections are encrypted with – in a database called “Flying Pig.”(160) The British spies produce weekly “trends reports” to catalog which services use the most SSL connections and save details about those connections.(161) Sites like Facebook, Twitter, Hotmail, Yahoo, and Apple’s iCloud service top the charts, and the number of catalogued SSL connections for one week is in the many billions for the top 40 sites alone.(162)

Hockey Sites Monitored

Canada’s Communications Security Establishment (CSEC) even monitors sites devoted to the country’s national pastime: “We have noticed a large increase in chat activity on the hockeytalk sites. This is likely due to the beginning of playoff season,” it says in one presentation.(163) The NSA also has a program with which it claims it can sometimes decrypt the Secure Shelf Protocol (SSH).(164) This is typically used by systems administrators to log into employees’ computers remotely, largely for use in the infrastructure of businesses, core Internet routers and other similarly important systems.(165) The NSA combines the data collected in this manner with other information to leverage access to important systems of interest.(166)

Weakening Cryptographic Standards

But how do the Five-Eyes agencies manage to break all these encryption standards and systems? The short answer is: They use every means available.(167)

One method is consciously weakening the cryptographic standards that are used to implement the respective systems.(168) NSA documents show that NSA agents travel to the meetings of the Internet Engineering Task Force (IETF), an organization that develops such standards, to gather information but presumably also to influence the discussions there.(169) “New session policy extensions may improve our ability to passively target two sided communications,” says a brief write-up of an IETF meeting in Sand Diego on an NSA-internal Wiki.(170)

This process of weakening encryption standards has been going on for some time.(171) A classification guide, a document that explains how to classify certain types of secret information, labels “the fact that NSA/CSS makes cryptographic modifications to commercial or indigenous cryptographic information security devices or systems in order to make them exploitable as TOP SECRET.(172)

Cryptographic systems actively weakened this way or faulty to begin with are then exploited using supercomputers.(173) The NSA maintains a system called Longhaul, an “end-to-end attack orchestration and key recovery services for Data Network Cipher and Data Network Session Cipher traffic.”(174) Basically, Longhaul is the place where the NSA looks for ways to break encryption.(175) According to an NSA document, it uses facilities at the Tordella Supercomputer Building at Fort Meade, Maryland, and Oak Ridge Data Center in Oak Ridge, Tennessee.(176) It can pass decrypted data to systems such as Turmoil – a part of the secret network the NSA operates throughout the world, used to siphon off data.(177) The cover term for the development of these capabilities is Valientsurf.(178) A similar program called Gallantwave is meant to “break tunnel and session ciphers.”(179)

In other cases, the spies use their infrastructure to steal cryptographic keys from the configuration files found on Internet routers.(180) A repository called Discoroute contains “router configuration data from passive and active collection” one document states.(181) Active here means hacking or otherwise infiltrating computers, passive refers to collecting data flowing through the Internet with secret NSA-operated computers.(182)

An important part of the Five Eyes’ efforts to break encryption on the Internet is the gathering of vast amounts of data.(183) For example, they collect so-called SSL handshakes – that is, the first exchange between two computers beginning an SSL connection.(184) A combination of metadata about the connections and metadata from the encryption protocols then help to break the keys, which in turn allow reading or recording the now decrypted traffic.(185)

If all else fails, the NSA and its allies resort to brute force: they hack their target’s computers of Internet routers to get to the secret encryption – or they intercept computers on the way to their targets, open them and insert spy gear before they even reach their destination, a process they call interdiction.(186)

A Grave Threat to Security

For the NSA, the breaking of encryption methods represents a constant conflict of interest.(187) The Agency and its allies do have their own secret encryption methods for internal use.(188) But the NSA is also tasked with providing the U.S. National Institute of Standards and Technology (NIST) with “technological guidelines in trusted technology” that may be “used in cost-effective systems for protecting sensitive computer data.”(189) In other words: Checking cryptographic systems for their value is part of the NSA’s job.(190) One encryption standard the NIST explicitly recommends is the Advanced Encryption Standard (AES).(191) The standard is used for a large variety of tasks, from encrypting the PIN numbers of banking cards to hard disk encryption for computers.(192)

One NSA document shows that the agency is actively looking for ways to break the very standard it recommends – this section is marked as “Top Secret”(TS): “Electronic codebooks, such as the Advanced Encryption Standard, are both widely used and difficult to attack cryptanalytically.(193) The NSA has only a handful of in-house techniques.(194) The Tundra Project investigated a potentially new technique – the Tau statistic – to determine its usefulness in codebook analysis.”(195)

The fact large amounts of the cryptographic systems that underpin the entire internet have been intentionally weakened or broken by the NSA and its allies poses a grave threat to the security of everyone who relies on the Internet – from individuals looking for privacy to institutions and companies relying on cloud computing.(196) These governments are themselves at risk. Many of these weaknesses can be exploited by anyone who knows about them – not just the NSA.(197)

Inside the intelligence community, this danger is widely known: According to a 2011 document, 832 individuals at GCHQ alone were briefed into the BULLRUN project, whose goal is a large-scale assault on Internet security.(198)

So the big news in encryption methods is that governments have been diligently working to weaken encryption standards in order to serve their own self-interests, not necessarily the interests of the people they govern. So how is the NSA and U.S. government preparing for a digital arms race and future battles?

How Is The NSA Preparing the U.S. for a Digital Arms Race and Future Battles?

The dual mandate of the NSA is to secure the Internet and protect people while simultaneously leveraging weaknesses in the system in order for it to engage in mass surveillance and deploy cyber weapons. A key question this poses: How is the NSA preparing the U.S. for a digital arms race and future battles?

An article in the German paper Der Spiegel based on the documents from Edward Snowden does a terrific job addressing this question. The following journalists and experts wrote the article:

Jacob Appelbaum, Aaron Gibson, Claudio Guarnieri, Andy Müller-Maguhn, Laura Poitras, Marcel Rosenbach, Leif Ryge, Hilmar Schmundt and Michael Sontheimer

Normally, internship applicants need to have polished resumes, with volunteer work on social projects considered a plus.(199) But at Politerain, the job posting calls for candidates with significantly different skills sets.(200) We are, the ad says, “looking for interns who want to break things.”(201)

Politerain is not a project associated with a conventional company.(202) It is run by a U.S. Government intelligence organization, the National Security Agency (NSA).(203) More precisely, it’s operated by the NSA’s digital snipers with Tailored Access Operations (TAO), the department responsible for breaking into computers.(204)

Potential interns are also told that research into third party computers might include plans to “remotely degrade or destroy opponent computers, routers, servers and network enabled devices by attacking the hardware.”(205) Using a program called Passionatepolka, for example, they may be asked to “remotely brick network cards.”(206) With programs like Berserkr they would implant “persistent backdoors” and “parasitic drivers”.(207) Using another piece of software called Barnfire, they would “erase the BIOS on a brand of servers that act as a backbone to many rival governments.”(208)

An intern’s tasks might also include remotely destroying the functionality of hard drives.(211) Ultimately, the goal of the internship programs was “developing an attackers mindset.”(212)

The internship listing is eight years old, but the attacker’s mindset has since become a kind of doctrine for the NSA’s data spies.(213) And the intelligence service isn’t just trying to achieve mass surveillance of Internet communication, either.(214) The digital spies of the Five Eyes alliance – comprised of the United States, Britain, Canada, Australia and New Zealand – want more.(215)

The Birth of Digital Weapons

According to top secret documents from the archive of NSA whistleblower Edward Snowden seen exclusively by SPIEGEL, they are planning for wars of the future in which the Internet will play a critical role, with the aim of being able to use the net to paralyze computer networks and, by doing so, potentially all the infrastructure they control, including power and water supplies, factories, airports or the flow of money.(216)

During the 20th century, scientists developed so-called ABC weapons – atomic, biological and chemical.(217) It took decades before their deployment could be regulated and, at least partly, outlawed.(218) New digital weapons have now been developed for the war on the Internet.(219) But there are almost no international conventions or supervisory authorities for these “D” weapons, and the only law that applies is the survival of the fittest.(220)

Canadian media theorist Marshall McLuhan foresaw these developments decades ago.(221) In 1970, he wrote, “World War III is a guerrilla information war with no division between military and civilian participation.”(222) That’s precisely the reality that spies are preparing for today.(223) It’s a private and public cooperative effort.

The U.S. Army, Navy, Marines and Air Force have already established their own cyber forces, but it is the NSA, also officially a military agency, that is taking the lead.(224) It’s no coincidence that the director of the NSA also serves as the head of the U.S. Cyber Command.(225) The Country’s leading data spy, Admiral Michael Rogers, is also its chief cyber warrior and his close to 40,000 employees are responsible for both digital spying and destructive network attacks.(226)

Surveillance Only ‘Phase 0’

From a military perspective, surveillance of the Internet is merely “Phase 0” in the U.S. digital war strategy.(227) Internal NSA documents indicate that it is the prerequisite for everything that follows.(228) They show that the aim of the surveillance is to detect vulnerabilities in enemy systems.(229) Once “stealthy implants” have been placed to infiltrate enemy systems, thus allowing “permanent access,” then Phase Three has been achieved – a phase headed by the word “dominate” in the documents.(230) This enables them to “control/destroy critical systems & networks at will through pre-positioned accesses. (laid in Phase 0).”(231) Critical infrastructure is considered by the agency to be anything that is important in keeping a society running: energy, communications and transportation.(232) The internal documents state that the ultimate goal is “real time controlled escalation”.(233)

One NSA presentation proclaims, “The next major conflict will start in cyberspace.”(234) To that end, the U.S. Government is currently undertaking a massive effort to digitally arm itself for network warfare.(235) For the 2013 secret intelligence budget, the NSA projected it would need around $1 billion in order to increase the strength of its computer network attack operations.(236) The budget included an increase of some $32 million for “unconventional solutions” alone.(237)

NSA Docs on Network Attacks and Exploitation

In recent years, malware has emerged that experts have attributed to the NSA and its Five Eyes alliance based on a number of indicators.(238) They include programs like Stuxnet, used to attack the Iranian nuclear program.(239) Or Regin, a powerful spyware Trojan that created a furor in Germany after it infected the USB stick of a high-ranking staffer to Chancellor Angela Merkel.(240) Agents also used Regin in attacks against the European Commission, the EU’s executive, and Belgian telecoms company Belgacom in 2011. (241) Given that spies can routinely break through just about any security software, virtually all Internet users are at risk of a data attack.(242)

The new documents shed some new light on other revelations as well.(243) Although an attack called Quantaminsert has been widely reported by Der Spiegel and other newspapers, documentation shows that in reality it has a low success rate and it has likely been replaced by more reliable attacks such as Quantumdirk, which injects malicious content into chat services provided by websites such as Facebook and Yahoo.(244) And computers infected with Straitbizarre can be turned into disposable and non-attributable “shooter” nodes.(245) These nodes can then receive messages from the NSA’s Quantam network, which is used for “command and control for very large scale active exploitation and attack.”(246) The secret agents were also able to breach mobile phones by exploiting vulnerability in the Safari browser in order to obtain sensitive data and remotely implant malicious code.(247)

In this guerrilla war over data, little differentiation is made between soldiers and civilians, the Snowden documents show.(248) Any Internet user could suffer damage to his or her data or computer.(249) It also has the potential to create perils in the offline world as well.(250) If, for example, a D weapon like Barnfire were to destroy or “brick” the control center of a hospital as a result of a programming error, people who don’t even own a mobile phone could be affected.(251)

Intelligence agencies have adopted “plausible deniability” as their guiding principle for Internet operations.(252) To ensure their ability to do so, they seek to make it impossible to trace the author of the attack.(253)

It’s a stunning approach with which the digital spies deliberately undermine the very foundations of the rule of law around the globe.(254) This approach threatens to transform the Internet into a lawless zone in which superpowers and their secret services operate according to their own whims with very few ways to hold them accountable for their actions.(255)

NSA Docs on Malware and Implants

Attribution is difficult and requires considerable forensic effort.(256) But in the new documents there are at least a few pointers.(257) Querty, for example, is a keylogger that was part of the Snowden archive.(258) It’s a pieces of software designed to surreptitiously intercept all keyboard keys pressed by the victim and record them for later inspection.(259) It is an ordinary, indeed rather dated, keylogger.(260) Similar software can already be found in numerous applications, so it doesn’t seem to pose any acute danger – but the source code contained in it does reveal some interesting details.(261) They suggest that this keylogger might be part of the large arsenal of modules that belong to the Warriorpride program, a kind of universal Esperanto software used by all the Five Eyes partner agencies that at times was even able to break into iPhones, among other capabilities.(262) The documents published by Spiegel include sample code from the keylogger to foster further research and enable the creation of appropriate defenses. (263)

‘Just a Bunch of Hackers’

The men and women working for the Remote Operations Center (ROC), which uses the codename S321, at the agency’s headquarters in Fort Meade, Maryland, work on one of the NSA’s most crucial teams, the unit responsible for covert operations.(264) S321 employees are located on the third floor of one of the main buildings on the NSA’s campus.(265) In one report from the Snowden archive an NSA man reminisces about how when they got started, the ROC people were “just a bunch of hackers.”(266) Initially, people worked “in a more ad hoc manner,” the report states.(267) Nowadays, however, procedures are “more systematic”.(268) Even before NSA management massively expanded the ROC group during the summer of 2005, the department’s motto was, “Your data is your data, your equipment is our equipment.”(269)

NSA Docs on Exfiltration

The agents sit in front of their monitors, working in shifts around the clock.(270) Just how close the NSA has already gotten to its aim of “global network dominance” is illustrated particularly well by the work of department S31177, codenamed Transgression.(271) The department’s task is to trace foreign cyber attacks, observe and analyze them and, in the best-case scenario, to siphon off the insights of competing intelligence agencies.(272) This form of “Cyber Counter-Intelligence” counts among the most delicate forms of modern spying.(273)

How Does The NSA Read Over Shoulders of Other Spies?

In addition to providing a view of the U.S.’s own ability to conduct digital attacks, Snowden’s archive also reveals the capabilities of other countries.(274) The Transgression team has access to years of preliminary fieldwork and experience at its disposal, including databases in which malware and network attacks from other countries are cataloged.(275) The Snowden documents show that the NSA and its Five Eyes partners have put numerous network attacks waged by other countries to their own use in recent years.(276) One 2009 document states that the department’s remit is to “discover, understand and evaluate” foreign attacks.(277) Another document reads: “Steal their tools, tradecraft, targets and take.”(278)

In 2009, an NSA unit took notice of a data breach-affecting workers at the U.S. Department of Defense.(279) The department traced an IP address in Asia that functioned as the command center for the attack.(280) By the end of their detective work, the Americans succeeded not only in tracing the attack’s point of origin to China, but also in tapping intelligence information from other Chinese attacks – including data that had been stolen from the United Nations.(281) Afterwards, NSA workers in Fort Meade continued to read over their shoulders as the Chinese secretly collected further internal UN data.(282) “NSA is able to tap into Chinese SIGINT collection,” a report on the success in 2011 stated.(283) SIGINT is short for Signals Intelligence.(284)

The practice of letting other intelligence services do the dirty work and then tapping their results is so successful that the NSA even has a name for it: “Fourth Party Collection.”(285) And all countries that aren’t part of the Five Eye alliance are considered potential targets for use of this “non-traditional” technique – even Germany.(286)

“Difficult To Track, Difficult To Target”

The Snowden documents show that, thanks to fourth party collection, the NSA succeeded in detecting numerous incidents of data spying over the past 10 years, with many attacks originating from China and Russia.(287) It also enabled the Tailored Access Operations (TAO) to track down the IP address of the control server used by China and, from there, to detect the people responsible inside the People’s Liberation Army.(288) It wasn’t easy, the NSA spies noted.(289) The Chinese had apparently used changing IP addresses, making them “difficult to track; difficult to target.”(290) In the end, though, the document states, they succeeded in exploiting a central router.(291)

The document suggests that things got more challenging when the NSA sought to turn the tables and go after the attacker.(292) Only after extensive “wading through uninteresting data” did they finally succeed in infiltrating the computer of a high-ranking Chinese military official and accessing information regarding targets in the U.S. Government and in other governments around the world.(293) They also were able to access source code for Chinese malware.(294)

NSA Docs on Fourth Party Access

But there have also been successful Chinese operations.(295) The Snowden documents include an internal NSA assessment from a few years ago of the damage caused.(296) The report indicates that the U.S. Defense Department alone registered more than 30,000 known incidents; more than 1,600 computers connected to its network had been hacked.(297) Surprisingly high costs are listed for damage assessment and network repair: more than $100 million.(298)

Among the data on “sensitive military technologies” hit in the attack were air refueling schedules, the military logistics planning system, missile navigation systems belonging to the Navy, information about nuclear submarines, missile defense and other top secret defense projects.(299)

The desire to know everything isn’t, of course, an affliction only suffered by the Chinese, Americans, Russians and British.(300) Years ago, U.S. agents discovered a hacking operation originating in Iran in a monitoring operation that was codenamed Voyeur.(301) A different wave of attacks, known as Snowglobe, appears to have originated in France.(302)

Transforming Defense Into Attacks

The search for foreign cyber attacks has long since been largely automated by the NSA and its Five Eyes partners.(303) The Tutelage system can identify incursions and ensure that they do not reach their targets.(304) The examples given in the Snowden documents are not limited to attacks originating in China.(305) The relatively primitive Low Orbit Ion Cannon(LOIC) is also mentioned.(306) The name refers to malware used by the protest movement Anonymous to disable target websites.(307) In that instance, one document notes, Tutelage was able to recognize and block the IP addresses being used to conduct the denial of service attack.(308)

The NSA is also able to transform its defenses into an attack of its own.(309) The method is described as “reverse engineer, re-purpose software” and involves botnets, sometimes comprising millions of computers belonging to normal users onto which software has been covertly installed.(310) They can thus be controlled remotely as part of a “zombie army” to paralyze companies or to extort them.(311) If the infected hosts appear to be within the United States, the relevant information will be forwarded to the FBI Office of Victim Assistance.(312) However, a host infected with an exploitable bot could be hijacked through a Quantambot attack and redirected to the NSA.(313) This program identified in NSA documents as Defiantwarrior and it is said to provide advantages such as “pervasive network analysis vantage points” and “throw-away non-attributable CNA (Computer Network Attack) nodes”.(314) This system leaves people’s computers vulnerable and covertly uses them for network operations that might be traced back to an innocent victim.(315) Instead of providing protection to private Internet users, Quantambot uses them as human shields in order to disguise its own attacks.(316)

NSA Docs on Botnet Takeovers

NSA specialists at the Remote Operations Center (ROC) have an entire palette of digital skeleton keys and crowbars enabling access to even the best-protected computer networks.(317) They give their tools aggressive sounding names, as though they were operating an app-store for cyber criminals: The implant tool “Hammerchant” allows the recording of Internet-based phone calls (VOIP).(318) Foxacid allows agents to continually add functions to small malware programs even after they have been installed in target computers.(319) The project’s logo is a fox that screams as it is dissolved in acid.(320) The NSA has declined to comment on operational details but insists that it has not violated the law.(321)

But as well developed as the weapons of digital war may be, there is a paradox lurking when it comes to breaking into and spying on third party networks: How can intelligence services be sure that they don’t become victims of their own methods and be infiltrated by private hackers, criminals or other intelligence services, for example?(322)

To control their malware, the Remote Operations Center operatives remain connected to them via their own shadow network, through which highly sensitive telephone recordings, malware programs and passwords travel.(323)

The incentive to break into this network is enormous.(324) Any collection of VPN keys, passwords and backdoors is obviously of very high value.(325) Those who possess such passwords and keys could theoretically pillage bank accounts, thwart military deployments, clone fighter jets and shut down power plants.(326) It means nothing less than “global network dominance”.(327)

But the intelligence world is a schizophrenic one.(328) The NSA’s job is to defend the Internet while at the same time exploiting its security holes.(329) It is both cop and robber, consistent with the motto adhered to by spies everywhere: “Reveal their secrets, protect our own.”(330)

As a result, some hacked servers are like a bus during rush hour, with people constantly coming and going.(331) The difference, though, is that the server’s owner has no idea anyone is there.(332) And the presumed authorities stand aside and do nothing.(333)

“Unwitting Data Mules”

It’s absurd: As they are busy spying, the spies are spied on by other spies.(334) In response, they routinely seek to cover their tracks or to lay fake ones instead.(335) In technical terms, the ROC lays false tracks as follows: After third-party computers are infiltrated, the process of exfiltration can begin – the act of exporting the data that has been gleaned.(336) But the loot isn’t delivered directly to ROC’s IP address. (337) Rather, it is routed to a so-called Scapegoat Target.(338) That means that stolen information could end up on someone else’s servers, making it look as though they were the perpetrators. (339)

Before the data ends up at the Scapegoat Target, of course, the NSA intercepts and copies it using its mass surveillance infrastructure and sends it on to the ROC.(340) But such cover-up tactics increase the risk of a controlled or uncontrolled escalation between the agencies involved.(341)

It’s not just computers, of course, that can be systematically broken into, spied on or misused as part of a botnet.(342) Mobile phones can also be used to steal information from the owner’s employer.(343) The unwitting victim, whose phone has been infected with a spy program, smuggles the information out of the office.(344) The information is then retrieved remotely as the victim heads home after work.(345) Digital spies have even adopted drug-dealer slang in referring to these unsuspecting accomplices.(346) They are called “unwitting data mules.”(347)

NSA agents aren’t concerned about being caught.(348) That’s partly because they work for such a powerful agency, but also because they don’t leave behind any evidence that would hold up in court.(349) And if there is no evidence of wrongdoing, there can be no legal penalty, no parliamentary control of intelligence agencies and no international agreement.(350) Thus far, very little is known about the risks and side effects inherent in these new D weapons and there is almost no government regulation.(351)

Edward Snowden has revealed how intelligence agencies around the world, led by the NSA, are doing their best to ensure a legal vacuum in the Internet.(352) In a recent interview with the U.S. public broadcaster PBS, the whistleblower voiced his concerns that “defense is becoming less of a priority than offense.”(353)

Snowden finds that concerning.(354) “What we need to do,” he said, “is we need to create a new international standards of behavior.”(355)

I agree with Edward Snowden and I believe the place to begin with standards of behavior is for governments and organizations to respect that our digital lives and real lives are one in the same and should be treated exactly the same under the law.  Also, any cyber weapon or capability before being approved should be analyzed through the lens that our digital lives and real lives are effectively one in the same and whether or not the use of such a tool or weapon would infringe on our Constitutional/Natural Human Rights. So lets now conclude by discussing why our information in the information age is not sufficiently secure from theft and illicit use.


Our information is not sufficiently secure from theft and illicit use due to the nature of politics and state power. State systems at one time had a monopoly on people’s information whereas today, all that information resides on the servers and phones of corporations and private individuals. The nature of state systems and politics is to secure their positions of power, even if that means breaking their own principles of governance, as has obviously happened in the U.S based on my research. The U.S. government is even planning to immediately purge some government wide network surveillance data because they know it will incriminate them once the nature of the information is exposed.

The digital technological problems of securing information can be solved through the standardization of end-to-end encryption and intelligent multi-factor authentication within digital systems (both public and private). The political problems with the technology of the law that comprises our governance systems, can only be solved by public virtue. Case in Point: U.S. national security mouthpieces are now expressly threatening Apple with terrorism prosecutions for providing end-to-end encryption to its customers to protect their data.  People need to fight for these rights to protect their digital lives, because if they don’t protect their digital life, they are not truly protecting their real life either. It is my contention the technology industry needs to lead this effort on behalf of its global customers/users. Know and protect your digital self and to thy own digital self be true.

This entry was posted in Thoughts. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s