I am asked somewhat frequently about privacy, security, data handling, and how systems that handle data can inspire trust. This is a huge question, and the “right” answer varies by context. The thoughts below are incomplete, but they are a decent starting point for explaining what I look at when I’m getting to know and understand systems that use data. I can’t stress enough: this is a minuscule tip of a very large iceberg, but hey, here goes.
I have almost no confidence in systems that handle data.
I have varying degrees of confidence in people and processes that use or interact with various systems. This has less to do with actual trust, or any evaluation of intent, and more to do with the reality that even the best designed system run by skilled professionals with incredibly good intentions can often be compromised by perfect storms that are most obvious with hindsight – or, to be succinct: shit happens.
But even this is incomplete: data are not homogenous, and reasons for collecting, sharing, processing, analyzing, and retaining data vary widely — and some of these reasons are sound, and some are blatantly, obviously, unsound (looking at you, self-described researchers who retain data because you think you are “developing a model”).
Data use is contextual, and political. The ability to use information created by, or about, or related to another person is a form of power. Many edtech systems fit this definition: they exist because they contain, process, analyze, or retain data created by children, or data created by the adults responsible for educating and caring for and about children.
So, if I’m asked for an assessment of a system that requires information about people, some of the first questions I ask include:
- why does this system exist? what would be lost if this system didn’t exist?
- what is the rationale for creating this system?
- who uses this system?
- who maintains this system?
- what people or companies get data from this system?
- have the people whose data are in the system been asked permission to use their data?
- do the people whose data are in the system play a role in using or maintaining the system?
- do the people whose data are in the system know all the people or companies who get data about them?
- if the stated purpose of the system is to help the people whose data are held in the system, have they been asked if the “help” offered by the system is actually helpful?
- do the people whose data are in the system have the option to remove their data from the system and not participate?
- who benefits most from the system? how do they benefit?
- would people using and maintaining the system, and people whose data are in the system, answer the question about who benefits in the same way?
This list of questions is incomplete. The answers to these questions invariably spawn other questions, and these questions are separate from other questions and evaluations such as technical security reviews, privacy policy and terms of service analysis, etc.
It also needs to be stressed that these questions are fairly optimistic. I have seen, repeatedly and firsthand, opacity around developing and implementing systems that use data, where questions like these are new, novel, and unwelcome.
But/and: you can tell a lot from the answers to these questions, and how people/entities that want to use data respond to answering these questions. If a person or entity feels like answering basic questions about data use is too difficult, that reveals a lot about how they will potentially respond to the greater challenges of maintaining ethical and technically sound practice over time.
Photo Credit: Wombat: By JJ Harrison (jjharrison89@facebook.com) – Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=8661812