黑料网

This is the third of three articles detailing research experiments that 黑料网 undertook in its search to create a more inclusive question about gender that could be asked worldwide and across a broad range of surveys.

The third article describes how 黑料网 tested questions that can be used to identify transgender respondents.

Historically, gender questions have focused on binary identities and treated gender and sex as one and the same. However, researchers increasingly acknowledge that questions that capture gender may be overly simplistic and fail to include the range of people’s experiences, including the transgender experience. In its effort to create a more inclusive question about gender, 黑料网 has been testing different versions of gender questions, including ones that measure transgender identities.

This is the third of three articles describing these research experiments. The first presented the results of an experiment testing gender questions that include options other than male and female. These questions could be used on surveys that require a more inclusive question but are not specifically researching gender-related issues.

The second article reviewed the results of an experiment testing question versions that more precisely capture gender identities and are more appropriate for research specifically interested in the experiences of gender minorities.

This article focuses on different approaches to capturing transgender identities.

Question Versions

We tested four gender identity questions in a web survey experiment of 22,882 U.S. adults, conducted April 3-13, 2023, via the nationally representative 黑料网 Panel. Respondents were randomly assigned to answer one of four question variants. All four versions were presented in the second article of this series.

This article focuses on versions 3 and 4, which included questions asking about transgender identity.

Version 3

This version tested wording from the on measuring sex, gender identity and sexual orientation. Respondents were first asked a question about sex assigned at birth, followed by a question about current gender identity.

Gender identity includes a transgender category, and respondents are allowed to select one option only. (For a more detailed description of the question versions and rationale for changes, please refer to the second article.)

Q1. What sex were you assigned at birth, on your original birth certificate?

1. Male
2. Female

Q2. What is your current gender? (Could only select one option)

1. Male
2. Female
3. Transgender
4. Two-spirit
5. I use a different term (free text)

Version 4

This version modifies the National Academies question. It still includes sex assigned at birth, but the gender identity question includes a nonbinary option and allows respondents to select more than one option. Importantly, this version included a follow-up question to confirm transgender identity.

Q3. What sex were you assigned at birth, on your original birth certificate?

1. Male
2. Female

Q4. Which one of the following best describes your gender? You may select all that apply.

1. Man
2. Woman
3. Trans or transgender
4. Nonbinary
5. I use a different term (free text)

Q5. Some people identify as trans, meaning they have a different gender identity from their sex assigned at birth. Do you consider yourself to be trans?
[Asked of anyone whose sex assigned at birth (Q3) and gender identity (Q4) did not match OR anyone who selected “Trans or transgender” (code 3), “Nonbinary” (code 4) or “I use a different term”(code 5) in Q4.]

Results

Overall, there is no significant difference in the percentages of people selecting the trans/transgender option in versions 3 and 4 (0.2% of respondents vs. 0.3%, respectively).

In version 4, 51 respondents selected more than one option, and of those, 35 were determined to be valid responses. Another 22 of the multi-select responses were people who selected the “I use a different term” option to provide commentary on the question, such as writing in a response like “there are only two genders.” Thirteen of the 35 valid multi-select responses were from respondents who selected trans/transgender and at least one other gender identity, such as transgender and woman.

Comparison of Sex Assigned at Birth and Gender Identity

A widely accepted definition of trans/transgender is someone for whom gender identity is different from sex assigned at birth. Therefore, some researchers have proposed using sex and gender questions to identify transgender individuals. While this is seemingly straightforward, it is more complex in practice and fails to acknowledge that gender is not always binary.

We calculated the number of transgender respondents using this strategy, using two different methods -- one that counts fewer respondents as matching on sex and gender and one that counts more as matching.

In version 3, sex assigned at birth matched gender identity in 98.9% of responses (male/male or female/female). Sex and gender did not match in 1.1% of responses, which includes anyone who gave an answer other than these two combinations (male/male or female/female). This includes respondents who said male or female in the sex assigned at birth question and answered the opposite gender, “transgender,” “two-spirit” or “another term” in the gender identity question.

Our second, more conservative method of calculating transgender identity only included people who answered male/female or female/male or who answered transgender in the gender question. This number drops to 0.5%.

In version 4, 99.0% of respondents’ sex and gender matched. Categorizing a match is somewhat more complicated in version 4 because people can select multiple response options. For this question, we counted a “match” if sex and gender were the same and trans/transgender was not selected.

In other words, someone who answered male/man or female/woman counted as a match. If nonbinary was also selected along with matching sex and gender (male/man or female/woman), this counted as a match (0.3% of all responses). All other combinations of responses (1.0%) were coded as not matching. If a more conservative calculation is used for non-matching responses, and only respondents who answered male/woman or female/man or anyone who selected transgender are counted, this number drops to 0.4%.

For both versions, respondents could write in responses if they selected “I use a different term.” Most of the written responses were either nonsensical or a presumed commentary on the question. It included responses like “Stop this game,” “There are only two genders” and “100% man.” These responses were presumed to indicate that the respondent’s sex and gender were a match and were coded as such.

Follow-Up Transgender Identity Question

Version 4 included a follow-up question:

Some people identify as trans, meaning they have a different gender identity from their sex assigned at birth. Do you consider yourself to be trans?

This question was asked of anyone whose sex assigned at birth and gender identity did not match (male/woman or female/man). It was also asked of anyone who selected trans/transgender, nonbinary or “I use a different term.”

Respondents who provided obviously false open-ended responses to “I use a different term” were removed from the analysis (n=27). Of those who were asked the question, 44.3% said they identify as trans, representing 0.6% of all responses. We examined who answered yes and no to this question.

• All respondents who selected trans/transgender in the gender question said yes in the follow-up transgender question.
• Six respondents answered a combo of male/woman or female/man, but no other gender options were selected (such as transgender or nonbinary). Five of these six respondents answered no to the follow-up transgender question. It is not possible to know from our data collection if these respondents made an error selecting sex or gender or if these were accurate answers.
• Nineteen respondents answered a combo of male/man or female/woman in the sex and gender questions and also selected nonbinary in the gender question. Of these, two said yes to the follow-up transgender question, while 17 said no.
• Alternatively, 31 respondents selected sex (male or female) and only selected nonbinary in the gender question. Among these respondents, 11 said yes to the follow-up transgender question, while 20 said no.

These results highlight the importance of specifically asking about transgender identity and not presuming identity based on the combination of sex and gender questions.

Summary of Transgender Estimates

These findings demonstrate possible differences in estimates of the transgender population that may be produced from different question wording or calculation methods.

The unweighted estimates range from 0.2% to 1.1%, and while all of these estimates represent a small portion of the population, for a low-incidence population, the difference between 0.2% and 1.1% is a meaningful difference and may lead to erroneous conclusions about the size of this population or changes over time. Changes from 0.5% to 1.1% could lead to headlines, for example, stating the number of people who identify as transgender has more than doubled.

Qualitative Feedback From Respondents

Gender minority respondents were asked the following open-ended question at the end of the question series:

黑料网 is testing ways to improve how we ask our gender question. We would like your opinion of the question(s) you just answered. Please use this space to share any suggestions, feedback or personal experiences that may be helpful to our research. If you do not have any suggestions, you may leave this box blank and move to the next question.

Seventeen respondents who identified as transgender (in the gender question) provided feedback on the questions and may help shed light on why different methods for generating estimates may produce different numbers.

These comments can be summarized into three main themes:

• Transgender as a gender identity: Seven respondents expressed that transgender is not a gender and should not be included in the gender question but rather asked as a separate question. One respondent said that the term “transgender” describes a process, not a gender, and that people who have transitioned should be able to identify as a man or woman. Relatedly, some respondents in version 3 were frustrated that they were only able to select one option and were forced to select between transgender and their gender identity (male or female).
• Asking sex assigned at birth: Five transgender respondents expressed concern with being asked sex assigned at birth and said it should not be asked unless medically necessary. One respondent indicated it was a triggering question that asked them to deny their gender. One indicated they gave their gender when answering the sex assigned at birth question (meaning they were captured as a “match” when comparing sex and gender). Alternatively, three transgender respondents expressed their appreciation for the sex and gender questions and felt it more accurately captured their experience and identity, compared with only being asked gender. Four respondents pointed out that sex assigned at birth questions excluded people who are intersex.
• Concerns about revealing transgender identity: Two shared that there may be reluctance on the part of transgender respondents to answer questions honestly given the current political climate and indicated that it should only be asked when necessary.

Conclusions and Recommendations

Based on these results, we have several recommendations for constructing questions to capture transgender identities.

• Many research projects do not have large enough sample sizes of low-incidence populations for analysis. As with any question, consider whether the inclusion of a question about transgender identity can be used in a meaningful way. If not, it may be best to leave the question off the survey. Many researchers want to add questions that provide respondents with a sense of inclusion. While well-intended, consider that these questions may be sensitive and may feel intrusive or unnecessary to some respondents.
• Consider asking sex assigned at birth only when biological sex is relevant to the research outcomes, such as health research.
• Allow respondents to identify themselves rather than calculating identity based on a combination of responses. Our analysis found that calculated identity was flawed and misidentified respondents.
• Consider the use of a stand-alone or follow-up question that asks about transgender identity rather than lumping “trans/transgender” into the main gender question. If transgender is included in the gender question, allow respondents to select multiple options.
• Keep in mind that the research definition of transgender may not match how people personally think about themselves or define transgender. Questions should not assume everyone is working with the same set of definitions or identities.
• Be transparent about the questions asked and calculation methods used to measure transgender identities and critically examine other estimates that are produced. Even seemingly minor differences in question wording or calculation methods can produce significant differences in results.

It is important to note that our results may have differed had we used different question wording, and we could have explored many other methods of calculation. Many researchers are working to improve questions to capture gender identity and transgender identities, and we hope that our research contributes to the findings and discussion.

To stay up to date with the latest 黑料网 黑料网 insights and updates, .

Learn more about how the  works.