Working Through Questions in EFF’s “Spying on Students” Report

Recently the Electronic Frontier Foundation (EFF) released Spying on Students, a report that presents the results of a long running survey that encouraged parents, students, teachers, administrators and other individuals to submit privacy concerns about the use of education technology in schools. The concerns identified include:

  • Lack of transparency:
  • Investigative burden [on parents and students] :
  • Data collection and use:
  • Lack of standard privacy precautions:
  • Barriers to opt-out:
  • Shortcomings of “Privacy by Policy”:
  • Inadequate technology and privacy training for teachers:
  • Digital literacy for students

In their introduction, the Electronic Frontier Foundation states that its findings “cannot be considered generalizable or representative,” but given that the document makes statements about “key themes” and references to “average people” it is hard not to imagine that is how it might be perceived.

The report also comes more than three years after a much more rigorous study of edtech privacy policies and contracts by Fordham University. Given the number of state student privacy laws passed since 2013, I read the report and was left wondering what a more “scientific” approach might look like.

My day-to-day work is as a technology analyst for a large school district, this often involves testing edtech applications and reading policies. I also spend much of my own time as an volunteer for various K12 privacy organizations, including co-chair of  CoSN’s Privacy Toolkit and a contributor to Common Sense Media’s Privacy Evaluation  project. So, when I read the EFF report it raised a lot of questions that were not answered.

This is my attempt to look for answers to some of the questions that I felt should have been asked and my focus is primarily on questions raised by the EFF’s analysis of the 152 application’s privacy policies.

152 Applications

The EFF included the list of 152 applications as an appendix to the document and I was able in all but a few cases to identify the vendor, a testable login URL, privacy policies and terms of service links. Of the 152 products, it seems likely that an individual with a practitioner’s level of knowledge of the K12 industry would suggest eliminating or separately noting at least 19 of these applications (12.5% of the total) from the analysis so as not to improperly skew the results. Some reasons for suggesting omitting include,

  • application name too vague to be identified and determine the correct privacy policy,
  • application is a page of content on a specific district’s website, not an application,
  • application does not collect student data and is not used by students (e.g. JAMFnation, a system admin support forum for their MDM product),
  • application is a locally install tool that does not require internet connectivity.

The most perplexing example of one that should be omitted was Audacity, a locally installed and stand-alone Open Source audio editing product where the existence of a privacy policy does not seem relevant. Audacity is ironically an example of tool that a school might offer as an alternative to using an online audio editor. The Canadian district SD43 lists audacity as such an opt-out alternative and their Web tools page provides a useful approach to alternative options.

The complete annotated list of suggested omits appears at the end of this document.

Privacy Policies

The report states that of the 152 applications, only 118 had published privacy policies online. In my brief investigation of the 152 applications, after eliminating the 19 applications mentioned, I identified 126 Privacy Policies from the remaining 133. Of the remaining 7, several were Student Information Systems (SIS) which can be hosted (and controlled) locally by the vendor, or by the district and where in that case the districts, not providers, would be responsible for the privacy policy. I verified this assumption is reasonable by identifying examples of the application’s use in specific districts and running nslookup to obtain the ip address and a geolocation lookup to confirm that the SIS and the district’s public website were in the same ip range and location.

Since the EFF did not provide details on how they scored specific policies there is no way to know if a policy scored simply on the presence or absence of information on encryption, retention etc. or if they looked at these issues in the context of the whole policy. To get a better understanding I looked at several of the privacy policies and one that highlighted the importance of this question was Gingerlab’s (Notability) . The policy states that:

“Ginger Labs does not collect any personal information in Notability or on the website. Ginger Labs does not have access to content you create in Notability or to files you import into Notability.”

This would seem to significantly impact how one would score the policy for encryption, retention aggregation and sharing. Tallying notability in the “does not have language about these” would likely be a false positive, but there is no way to know how this was scored as there is no discussion in the report of differentiating applications that by their nature does not/cannot collect any student data (e.g. JamfNation), ones that say in their policy that they do not (e.g. notability) and applications that collect and manage student data.

The report notes that “Some applications note that schools may implement their own privacy policies to govern personal data submitted to the services by student users.”. The EFF lists this as one of their recommendations for school stakeholders (“Don’t accept Terms of Service when you can get a contract”).

Given this recommendation, it would be fair to point out that the EFF’s analysis would not have been able to consider if schools had contracted with these vendors.

To get a better understanding of potential impact of this omission, I attempted to categorize the list of applications using conventions similar to the Colorado Student Data Transparency and Security Act described in the report (contract vs “on-demand” providers). Some providers could have business plans that offered both modes.

contractChart

Including privacy and security specific language in contracts or contract addenda has become a more common district practice in recent years. One example is the use of statewide privacy contracts promoted by A4L and in use in Massachusetts and California. A4L maintains a publicly searchable database of contract addenda (and vendors that have refused to sign) and  46 of the 152 applications appear in the database (see end of document for details.)

The report paints a dire picture of what is wrong in privacy policies, so it seems reasonable to question if there were any  examples of good practices. In my brief investigation I identified several vendors that provide additional policies including: compliance with California AB 1584 (e.g. See Saw, and Prodigy) and detailed data security policies (e.g. Schoolloop  and eBackpack).

Data Retention

The document lists two providers (Haiku Learning, and Lexia Learning) where “the schools, rather than individual students, retain the authority and ability to delete information from the application.” Given the description of the FERPA school official exception in the report, This is likely because these are contracted services where the provider is acting as a school official, managing the student’s education record. Schools that use external providers are required to maintain direct control. Also Schools may be required to maintain records under state specific records retention schedules.

The report points out that “Storyboard retains student data for up to four years of inactivity”, however when I read the privacy policy it seemed like this is a misleading characterization. The full paragraph, in context reads as follows:

“At any time, any school administrator can delete students and their storyboards off of our systems. We can also delete all of your data upon explicit request. After 4 years (or less at our discretion) of inactivity we will delete student data.”

Encryption

The report says that “only 46 state [in the privacy policy] that the vendor uses encryption. That means that only about 30 percent of the 152 services reported to us make any statement about encryption. This lines up with previous reports on the lack of support for encryption in edtech.”

I wanted to get a better understanding of this, and found that the report referenced is a study by Common Sense Media (CSM) which not an evaluation of privacy policies as the EFF did, but was a study in which CSM performed an actual test of a site’s encryption.

To provide a more balanced, apples-to-apples comparison and go “beyond the privacy policy”, I identified the login URLs for 114 of the non-mobile applications and tested using the methodology and code developed by Common Sense. In March 2017, Common Sense documented that 56.21% of sites require encryption, in my testing encryption was required on 82% of the sites listed in the EFF report.

null

I did not include the mobile-only apps in this SSL scan, but I did perform a proxy analysis on two of the apps using the technique described in Common Sense Media’s InfoSec primer, and verified the use of encryption.

Additionally I ran the list of login URLs through a command line tool from SSL Labs that grades the quality of a site’s SSL certificate. The results for the EFF applications (73% with a grade of A) were significantly better than has been my experience for the average for products used in schools based on my regular use of this tool in my day to day work.

null

To understand the context of this results relative to internet traffic as a whole, I looked at  trustworthyinternet, a site run by SSL labs that is a “Survey of the SSL Implementation of the Most Popular Web Sites” . The score for the EFF application list was significantly better than this list of 150,000+ sites.

null

This disparity between the EFF’s evaluation of privacy policies and these empirical tests raises a question.  Is it reasonable to evaluate an application’s encryption based on their privacy policy? The data suggest perhaps not.  It also caused me to wonder if the listing of encryption specifics in a privacy policy was an industry norm. It seemed reasonable to look at how EFF addresses this in the privacy policy for their software applications. The policy makes no mention of encryption, and has only this to say about reasonable security.

“Security: Although we make good faith efforts to store information collected by EFF in a secure operating environment, we cannot guarantee complete security. Information collected by EFF will be maintained for a length of time appropriate to our needs.”

Lastly, it is worth noting that many other groups have been working in the student privacy space for the last several years. As shown in the table below, many of the 152 applications have been reviewed by these organizations, and Schools regularly look to these sources and others in addition to relying on “privacy by policy”.

Providers that have signed the Student Privacy Pledge

46

Providers that have iKeepSafe certifications

8

Providers that have a Common Sense Privacy Evaluation

30

The EFF wrote the report to highlight their concerns and advocate for change, and I believe that their advocacy is invaluable. My takeaway from this exercise is a fresh reminder that there is a difference between research and  investigative reporting, and advocacy reporting. I believe all of us in the ed tech ecosystem have a responsibility both to try to understand and to question a variety of  viewpoint, include ones from those we tend to agree with.

I am posting my complete annotations on the EFF list of apps here <appList_EFF> for anyone else to review, comment on, or correct. This work was conducted over the a few evenings and I am sure there are points I have not had time to consider.  I welcome all constructive feedback.

jim@k12cloudlearning.com

@jsiegl

Disclaimer: While I work for a large public school district, this reflects my own opinions and not the opinions of my employer.

 

 

 

Appendix A: Discussion of Potential Applications to Omit from Analysis

Product Reason to Consider Omitting or Qualifying in Findings
Audacity Locally Installed Open Source audio editing product. Audacity is ironically an example of tool that a school might would could offer as an alternative to using an online audio editor. The Canadian school district 43 does this, and provides a useful example of alternative options see https://www.sd43.bc.ca/Resources/DigitalCitizenship/Pages/CloudTools.aspx
Barracuda Hardware (Network Firewall)
Bluecoat Hardware (Network Content Filter)
CAPE Unable to Identify with certainty, could be https://cape.keepntrack.com/ for CAPE charter schools or some system on http://www.capenet.org/
CaSecureBrowser This is the locally installed client for the California Assessment of Student Performance and Progress
CERAN http://ceran.svvsd.org/ This is not an edtech product, it is a web page on a St. Vrain Valley Schools district website that lists multiple online tools that are used in the district, but the individual tools do not appear to have been broken out and added to the EFF list.
eCampus Unable to identify with certainty, this name is used to describe many things including a doe website, https://cbfisap.ed.gov/ecb/CBSWebApp/welcome.jsp also used by multiple virtual schools, and for schools brand of blackboard, by http://www.lpssonline.com/ecampus powered by Edgenuity. Also eCampus.com is an e-tailer that offers new and used textbooks (http://www.ecampus.com/), e-books, study materials, and bookstore management solutions
Encore Was SIS from Encore from Spectrum K12 School Solutions (Acquired by Scantron on July 22, 2010 and it is not listed in any form on the Scantron product site) , district locally hosted (San Diego, Clark county) district specific and likely local install see https://sems-shared.sems.ccsd.net/Encore/EncoreCBLWebUI/Login.aspx they all seem to be migrating off the product
Geometer’s Sketchpad Locally Installed Executable that does not require Internet Access
Global Protect Palo Alto Networks VPN/endpoint security/policy enforcement see
Jamf Nation JamfNation is the user forum for system administrators of the jamf mobile device management platform and not a system that is used by students or that would store student data.
Logger Pro Windows and Mac locally Installed client software for the Vernier data probes.
Magister Non-US based LMS, Privacy policy and terms are in Dutch
Moodle Open Source Learning Management System, Can be installed locally, for hosting evaluation of privacy policy ins only able to be done when a hosting provider is identified
MyBigCampus Discontinued: SIS from Lightspeed, My Big Campus was scheduled for end of life and support on July 31, 2016
Rapid Identity Identity Management System, District locally installed and cloud application
Sakai Like Moodle (see above) open source, hostable by a district or provider and policy evaluation only relevant in the context of specific hosting instance.
Samarbeta.net This appears to be a Swedish LMS on a Moodle server for one teacher at regionvasterbotten.se
Subtext Discontinued: “Beginning on July 1st, 2015, Subtext will become part of Accelerated Reader 360”.
SynchronEyes (SMART Technology) Correct product name is SMART Sync software, locally installed, not an internet product see https://smarttech.com/Solutions/Government+Solutions/Products+for+government/Software/SMART+Sync

Appendix B: Applications/Contract Riders listed in the Massachusetts and California Student Privacy Alliance Database

Massachusetts

  1. Accelerated Reader
  2. Achieve 3000
  3. ALEKS
  4. BrainPOP
  5. Canvas
  6. ClassDojo
  7. Clever
  8. Code.org
  9. Discovery Education
  10. Dreambox Learning
  11. Edgenuity
  12. Edmodo
  13. Educreations
  14. Google Apps For Education
  15. iReady
  16. Its Learning
  17. IXL
  18. Kahoot
  19. Lexia Reading Core5
  20. MindMup
  21. MobyMax
  22. Naviance
  23. Nearpod
  24. NoRedInk
  25. Padlet
  26. Pear Deck
  27. Pearson SuccessNet
  28. Prezi
  29. Quizlet
  30. Raz-Kids
  31. Socrative
  32. Storyboard That
  33. Sumdog
  34. Turnitin
  35. Typing Pal
  36. Weebly
  37. XtraMath

California:

  1. Abcya
  2. Accelerate Learning (STEMscopes)
  3. Acquia Administrative Software Applications (ASAP)
  4. ALEKS
  5. BrainPOP
  6. Canvas
  7. ClassDojo
  8. Clever
  9. Collections Core5
  10. Discovery Education
  11. Dreambox Learning
  12. Edgenuity
  13. GoGuardian
  14. Hapara
  15. iReady
  16. IXL
  17. MobyMax
  18. Naviance
  19. Nearpod
  20. Padlet
  21. Pear Deck
  22. Prezi
  23. Prodigy
  24. Quizlet
  25. Raz-Kids
  26. Rosetta Stone
  27. Seesaw
  28. Socrative
  29. ST Math
  30. Storyboard That
  31. Study Island
  32. Turnitin
  33. Typing Pal Typing.com

One thought on “Working Through Questions in EFF’s “Spying on Students” Report

Comments are closed.