Understanding scores on questions and themes in Surveys
The Engagement survey suite (engagement, onboarding, exit, and pulse surveys) calculates question and theme scores based on a positive response score.
This article contains the following topics:
Question scores
A question's score is the number of responders who gave positive responses out of the number of total responders for that question.
For a question that uses the agree/disagree scale, scores are calculated in the following way:
- Agree or Strongly Agree = Positive Responses
- Disagree or Strongly Disagree = Negative Responses
- Neutral = Neutral Response
Example:
For a particular question, 100 people in total responded. The response breakdown is as follows:
- Strongly agree: 30 people
- Agree: 20 people
- Neutral: 10 people
- Disagree: 25 people
- Strong disagree: 15 people
- No response: 50 people
For this question, the positive score would be (30 + 20) / 100, which calculates out to 50%.
Theme scores
The score of a theme that contains multiple questions is not the average score of all the questions within the theme.
Surveys use a different methodology than question scores so that the score for a theme does not outweigh the responses from people who have answered all the questions within the theme versus those who only answered one.
Surveys uses the following steps to calculate theme scores:
- Surveys look at employees who have answered at least one question in a given theme.
- For each employee who meets that requirement, surveys calculates the average score for submitted questions they responded to in the theme. Average score is calculated based on the agree/disagree scale where strongly disagree = 1 and strongly agree = 5. This is the score for the employee.
- If the score for the employee is higher than 3.5, this is considered a positive response to the theme.
- The total number of employees who scored at least 3.5, and are therefore considered positive responders, for the theme is divided by the total number of employees who responded to the theme. This is the theme's score.
Example:
For a particular theme with two questions, 100 people in total responded. The response breakdown is as follows:
- Employee 1
- Question 1: Strongly agree
- Question 2: Agree
- Employee 2
- Question 1: Agree
- Question 2: Neutral
- Employee 3
- Question 1: Disagree
- Question 2: Agree
The first step is to calculate the average score for each employee:
- Employee 1: (5 + 4)/2 = 4.5
- Employee 2: (4 +3)/2 = 3.5
- Employee 3: (2 + 4)/2 = 3
Next, employees who are considered positive scorers are added and then divided by the total number of employees who responded to the theme.
- 2/3 = 0.66
- 0.66 ~ 67
The theme score is 67.
Overall score
Overall scores are calculated as an average of all question scores for questions included in that survey. Question scores are measured as a percentile of their positive responses.
Example:
In a survey that had 3 questions with question scores of:
- Question 1: 50
- Question 2: 60
- Question 3: 70
The overall score would be [ (50+60+70) / 3 ] = 60.
Questions or themes without scores
A question or theme will not have a score associated with it for two possible reasons:
- The question is set as solely a comment question, meaning that responders were prompted for a free text response rather than a scale rating. In this case, there is no score to show. The same applies if a theme consists only of questions that are comment questions.
- A "No opinion" answer indicates participants skipped the question without selecting a response.
- There aren't enough responses to a question or theme to show the score. This is dependent on the anonymity threshold set when creating the survey. For example, if the survey has an anonymity threshold of 3, a question that only has two responders will show no score to protect the anonymity of the responders.
Unsubmitted surveys
Responses that have not been submitted when the survey is ended will not be pulled into survey analytics.