It’s never just a field
Every 23 seconds someone, somewhere, asks for “a new field” to be “added to the database.” Several times per month I am the lucky recipient of such a request. I often wonder about my role in hoarding so much data that if every human who will ever live works frantically until the heat death of the universe we won’t be able to make meaningful use of it all. I also sometimes wonder if we are tearing small holes in the space/time continuum in our race to coerce ever-tinier bits of matter into storing All! The! Photos! of adorable wombats.
But I digress. I want to talk about fields in databases.
In most organizations that I have known (those in public sector, academia, and nonprofits) “the database” is shorthand for “a thing on the computer that we type stuff into and universally despise.” There are three essential roles in most organizations when it comes to dealing with “the database:” (1) those who toss offerings into its maw as talismanic protection against Great Suffering and Job Loss; (2) those who sift through the offerings to seek patterns and provide divinations pleasing to Those In Power; and (3) those who use the divinations to ensure the decisions they have already made are Data Driven. When any one of these roles feels that reality is insufficiently represented in “the database,” they frequently request a new field to be added.
As a maw-builder and apostate diviner, I would like to share a secret with you. The unfettered adding of fields does not make anything better or closer to reality. In fact, it often makes things much worse and drives staff to simply hurl bits of typed nonsense into form fields and run away. The enterprise data system is the embodiment of a shared reality we create in our organization. When it’s allowed to be shaped without appropriate planning and governance, you get chaos, distrust, and fear. And useless systems.
If you are tired of choking on incense and approaching the database divinations with a mix of hope and dread, I humbly offer a new practice for the next time you think to yourself “if we only had a field to capture This Thing I Can’t Find.”
This practice is very simple. I call it WIDAT: Write It Down And Think. You can pronounce it like “why ‘dat” and show some love to my New Orleans heritage. “Hey, I need a new field” says the requester. “Oh? WIDAT” says I, as I wander off to find some coffee with chicory and some tasty beignets.
What, exactly, to write down and think about is outlined in the rest of this article. If you still need your field when you complete this process, you have the information necessary to ensure that it is actually valid, useful, and understandable over time. You may even find your diviners transforming into librarians who can help you navigate the written rules that lead to order and predictability in your database.
Step 1: Define Who is Involved
- Who creates the data you believe is not available? That is, who gathers it, captures it, and puts it into the database?
- Who authorizes any changes to the way the data creators use their time?
- Who is held accountable for the part of the organization this new field will come from?
- Who all would need to be trained in how to use this new field? End users, external users, people who write reports that might include this new field? People who read reports with this new field? “Training” is anything that teaches someone to provide responses that are meaningful and valid for your purposes.
- Who is responsible for communicating the purpose and use of this new field to everyone impacted?
Step 2: Ask Those People Why This Field Doesn’t Already Exist
- Why do the creators and those accountable say this information isn’t currently being captured? Is it harder than you think? Ambiguous? Overlooked?
- Are there ways that the creators and report writers are currently getting at the information by proxy that would eliminate the need for a new field?
Step 3: You Probably Don’t Need a New Field
- You have probably identified that there are pockets of local knowledge and folklore pertaining to proxy information or anecdotal information about whatever it is you think is missing.
- You have probably identified big gaps in who owns knowledge and interpretation of whatever you think is missing and serious diversity in how people are interpreting and using data.
- You may have identified barriers to collecting or understanding the specific information you’re looking for. Some things that are compelling to report on from a program or fundraising side are highly sensitive or risk re-traumatizing the population you’re working with, for example. Some things seem easy and just aren’t. Some things should be easy but have never been defined.
Step 4: Write It Down. Write It ALL Down.
Wherever you write it down has to be central, always accessible, searchable, enforced, and maintained. Google sites, a wiki, SharePoint, a custom object in Salesforce, even a OneNote or EverNotes “knowledgebase.” I have, out of desperation, been known to create a Word document and refuse to answer questions until the questioner looked it up in the doc first. I have also been known to answer with “is it in the wiki?” until people start questions with “I looked in the wiki and….” Enforcement of the use of a central knowledge repository is critical for adoption.
Job #1 is to write down the answer to “how would I know how to do this if I were brand new?”
- How would I know how to capture correct data?
- What exactly constitutes correct data?
- How would I know how to make sure the people I manage are capturing correct data?
- How would I write an accurate report?
“How” questions should be answered very concretely, with “exactly who,” “exactly what,” “exactly when,” and “exactly where.”
Remember that data doesn’t just magically make sense — it is simply the intangible analog of any other type of production work.
Some organizations have end user documentation (or a user manual) but almost all neglect other critical parts that impact understanding what is captured and how to transform it into trusted reports for analysis. Remember that data doesn’t just magically make sense — it is simply the intangible analog of any other type of production work. As such, it follows the same rules that any physical or tangible type of work follows: inputs are collected and transformed into outputs while being monitored and refined via feedback. Because so much of our work in the social sector is intangible, we don’t often adopt the practices from industry, but as I’ve written about elsewhere, I believe we ignore the many years of thinking about creating enduring organizations to our own detriment.
User manuals are necessary but not sufficient.
It’s relatively easy (if tedious and time-consuming) to move through the screens of your digital systems and methodically take pictures and write down what the sequence should be. These manuals typically include prescriptive language and screenshots of the database screens to specify exactly what is being captured, exactly when, into exactly which fields.
Many organizations believe that because they have documented their end user processes, they have a shared understanding of reality and that reports, monitoring, and feedback should be easy. After 20+ years of doing this work across hundreds of organizations, I promise you that is not the case. You might have the steps to capture some input, but you’re highly unlikely to have anything written down about the the steps you take to transform inputs into outputs or underlying assumptions used in any computed or manual transformations.
Translated into plain English, how do you get from a bunch of names, birthdates, and notes in your database to meaningful reports about active participants and successful interventions? How precisely do you define “active,” “participant,” “successful,” and “intervention?” Exactly which fields would I — a total outsider — use for each of these elements, and exactly where would I look if I had to ask the same question about past participants versus current ones?
“But wait,” you cry. “Our processes aren’t this simple! We deal with people, and people are messy! Anyway I just want a damn field!”
To build a resilient organization that is able to reliably turn its inputs into its desired outputs, you have to stare squarely at the messiness and rigorously define how we know what we know and how we take what we observe and turn it into shared meaning. This is a non-trivial task. It is never only about data, and always about power, respect, fear, time, energy, money, collaboration, tradeoffs, and many other very human things that make our workplaces so fascinating.
The complexity of the work we do in the nonprofit sector — and the human stakes often involved — mean that we think very critically about our interactions with our program participants. Some excellent work has been done on aligning program data collection with trauma-informed practices, for example. We are exceptionally good at thinking through the consequences of changes in very exacting detail. We just haven’t learned to extend that thoughtfulness to our internal data systems yet and approach our databases as critical support tools in executing on our jobs (rather than impediments to doing our jobs).
Fortunately for us, data systems are one of the few areas of life where we can actually create a new reality through deliberate action. Unfortunately for us, technology is making it ever easier to “just add a field” without having to reflect on its widespread implications, making it also ever easier to make or exacerbate a mess.
What if You Still Need the Field?
By now you’ve realised that I tricked you. I stopped talking about your new field a long time ago and instead gave you a blueprint for change management and governance of your data system. Sometimes, though, you write it all down and you still need the field.
Congratulations! When you add a new field after approaching it with thought and writing it all down, you have already documented it and added to your organizational knowledge of how you are defining and using the data you collect. It may seem insignificant now, but in 3 years when you’ve won the lottery and are basking in the sun on a beach somewhere, you don’t want to get a frantic call from your former colleagues asking why on earth there’s a field for whether the participant is left- or right-handed and where all it’s used and if we can change it to add an option for ambidexterity.
Creating governance and a governable structure from scratch is a huge undertaking and is too time consuming for any organization to just launch into. Developing a practice around an alternative, reflective (rather than reactive) method for answering the “I need a field” demand can construct the necessary system in a more manageable way. Over time you build a robust library of precise definitions that is useful to report writers, analysts and researchers, but more important you build a sustainable base of knowledge that outlasts any one person. Deliberately creating a shared set of definitions can help transform your data culture to lose the mythology and folklore and operate together with transparency and certainty.
Jenn is an avid fan of folklore and mythology. Just not when it comes to data systems.
