planning for an eventual demographic survey (or series of surveys)

Agreed with all of this, yeah — and Shel's observations about the "temperature check" kind of approach are also extremely important.

In terms of asking these questions of users, I think asking what country they're from would also be helpful, both in terms of getting a sense of what political/cultural/legal climates our users are in as well as having deeper context for their answers about the other demographic questions.

But basically, in the (extremely likely) event that the stewards are disproportionately white, having clear data on the demographics and how those demographics affect people's experience of the League is critical. This might not be enough alone to deal with other forms of socially-conditioned racism, but it's probably necessary (in the strictest sense).

wanted to through out some more options for wording to get the conversation started. #nothing-about-us-without-us

1. How would you describe your gender-sex configuration? (1)

   • e.g. trans-masc man, femme trans-masc, femme cis-woman, bi-gender, ect.

2. What racial identity(s) do you self identify with?

   • e.g. Polynesian, asain-american & white, ect.

3. Would you describe your gender-sex as including the term transgender?

4. Do you self identify as disabled?

5. Do you self identify as neurodivergent?

Also, I like Katja’s idea of including regional demographics as well.

1. What region of the world do you consider most important to your person identity? Or… Where would you tell people you are from?

   • e.g. can list state, province, region, country, or city.

Resources

1. Slides explaining Sexual Configuration Theory link to pdf

I guess also a discussion on what are the goals and intentions in our interest in collecting this information?

If this is intended to be a survey on experiences of harassment/prejudice, then should this survey include those questions? Going with the Design Justice principle on using what already works, there are questionnaires that exist for that that we could pull from.

If this is intended to just provide information on demographics, what is our intention with collecting that info? Do we want to know if there are barriers in joining/participating in WL that exist from dis-affordances in WL infrastructure? UI? We can ask those questions.

Do we want to know if certain demographic just don’t prefer to use WL? Would we then choose to make WL more preferable to those individuals?

Are there technical barriers in joining the league? Do we need to have better guides or tutorials? Are there financial barriers? Would we offer scholarships in setting up hosting? Or would we open up existing instances to more members?

Is the main goal of WL to just provide infrastructure? To be a restorative justice based authority in resolving harm that occurs on that infrastructure? To be empowered with authority in deciding who gets to be a community member? (Idk if we have these goals/roles written down somewhere. Please let me know if we do.) Would surveys like these be used to make sure that our goals and intentions are aligning with the outcomes of our actions within the community?

@jojorner are you caught up with the conversation in #league-general earlier today?

@walking mirage i’m not. guess its time to go do that. Lol

Edit: started to read it and its a lot of words for me right now. I’ll try another time or if someone can summarize, big thank

@ocean I heartily disagree with upholding the gender-sex distinction in this context. (I can expand on this later if you'd like.) Alyaza's original questions are a good formulation. the only thing I'd add is "do you identify as intersex" and maybe expand the trans question to "transgender and/or transsexual."

Some design principles for surveys/community studies to keep in mind.

1. You can't do quantitative analyses on write-in answers. You have to do multiple choice in order to actually see demographics, otherwise you get the full spectrum of unique specific identities that you have no place decided should be grouped together or not. People taking the survey can decide to self-select into boxes we gave them, and then we can have an Other box to write in answers if someone isn't able to place themselves in the options we gave. That box is usually not something you can do an analysis on but it does allow you to add those groups to the options on the next iteration.

2. They must be time-bound. Otherwise you cannot get a snapshot of what the data represents. I recommend something like every 3 months providing one week for people to take the survey. This is "pulse surveying" and is very good at tracking information over time. When you do this, the fewer questions you ask the more likely that people will actually keep taking the survey over and over again which is important for getting accurate information.

The reason we want demographics is because we are trying to keep our finger on the pulse of diversity and harassment. So I recommend it be as simple as:

1. Do you identify as white (yes/no)

2. (If No): Have you experienced racism from another user on the website league in the past three months?

3. (If Yes): Did you report it?

4. (If Yes): Was the report handled to your satisfaction within a reasonable timeframe?

Repeat question 1 for Transgender, Neurodivergent, Disabled, and iterate that question tree for each and their corresponding words for being discriminated against. We could also end with a question of "Did you receive, in the past three months, mistreatment for an aspect of your minority identity not mentioned in this survey?" And if they say yes we can have a write-in field and add that group to the next iteration of the survey.

We should avoid modifying the wording of the survey or questions between iterations as much as possible because then it becomes hard to compare points in time. But it can be done if absolutely necessary.

We can ask more specifically about racial identity if we are interested in that information. We just would need to be mindful that racial identity boxes differ from country to country because race is a social construct.

@Shel I appreciate this expanded post. Just jumping in the day before, it was not clear to me the intention or goal of doing this action. This way of explaining helped clarify that for me. Thank you. Maybe there is context field or something that we can add to the post template to avoid confusion in the future. Especially for people who aren’t able to keep up with all of the discord chats.

@shelraphen I like this direction. Keeping questions simple and allowing for quantitative analysis makes sense and is probably going to result in more responses. We can always add write-in fields to complement multiple-choice questions.

Just wanted to drop some things down here as I don’t know what background others are coming into this convo with / if people want more resources.

This discusses one survey scoring for conflict communication + sex. You can see they have a background inventory at the top of the survey and then are using different scales with each labeled in bold. https://scales.arabpsychology.com/s/the-relationship-between-conflict-communication-and-sex/

Here’s what the Pew Research Center did in 2017 on looking at online harassment. They coded for a lot and defined online harassment at 6 unique behaviors: (1) Offensive name-calling, (2) Purposeful embarrassment, (3) Stalking, (4) Physical threats, (5) Harassment over a sustained period of time, (6) Sexual harassment. They also scored for “experienced any harassment” and “experienced multiple behaviors of harassment”. If you take a background inventory upfront, then you can ask questions like they did; “have you experienced harassment based on your race?” Or “have you experienced harassment based on your political views”. https://www.pewresearch.org/internet/2021/01/13/the-state-of-online-harassment/

Unless we find information pointing to the opposite, I think there is less need to iterate all of the questions. We could code with a background inventory at the beginning and then code for harassment with maybe additional questions pointing at harassment related to a social marker: race, gender, ect.

As for the stewards (let me know if I am not understanding the intention correctly) and it sounds like the intention is to gather a background inventory? Is there a specific goal that we have with that? I feel that would be a good place to start. Having a convo about why we want to collect and see if we can sort that out first. Do we hope to change behaviors? Do we need more data in order to be able to change those behaviors? I just don’t want us to collect data and then sit on it or use it as a tool to reinforce or engage in power over or systems of oppression.

one book on this is “Fatal Invention” by Dorothy Roberts. https://archive.org/details/fatalinventionho0000robe/page/n10/mode/1up

here’s a short edu-artcile on survey building from a psychology lens https://listen-hard.com/psychological-research-and-methodology/create-scale-psychology/#What_Are_the_Different_Types_of_Scales_Used_in_Psychology