The MBTA Performance Dashboard measures customer satisfaction using the results of our monthly Customer Opinion Panel survey. This post explains how the panel survey works, what data we show on the Dashboard, and other uses for the survey results.
History of the panel survey
The Customer Opinion Panel for the MBTA originated with a 2014 master’s thesis by MIT student William Chow. The thesis set up a customer panel for the MBTA as part of an experiment analyzing the effectiveness of different survey methods. The MBTA restarted the survey in July 2015 as a way to measure customer satisfaction on an ongoing basis.
Any passenger can join the Customer Opinion Panel by filling out an entrance survey. She will then enter the panel and receive a survey once a month that asks both general satisfaction questions and specifics about the her last trip on the MBTA.
What is on the Dashboard?
We report the results from four of the Customer Opinion Panel questions on the MBTA Performance Dashboard every month. Three of these questions track our overall performance and one relates to our reliability. As we add other metrics in future releases of the Dashboard, we will add the corresponding satisfaction questions so the public can track how our performance relates to satisfaction in that area.
Measures of customer satisfaction
The customer satisfaction metric on the dashboard is calculated from responses to the question, “How would you rate the MBTA overall?” Possible responses to this question range from “Extremely Dissatisfied” (1) to “Extremely Satisfied” (7) on a seven-point scale. The average customer satisfaction has hovered stably around 4.5 out of 7 (between Neutral and Somewhat Satisfied) since the panel survey restarted in July 2015 except for a temporary drop during summer 2019 (in response to a Red Line train derailment that caused extended restrictions on service for repairs).
The survey also asks about the last trip the respondent took on the MBTA. As seen in the figure below, the level of satisfaction with the last trip consistently tracks slightly higher than the overall satisfaction. We chose to put the results of the overall satisfaction question on the Dashboard since we want to track the perception of the MBTA as a whole. We will be watching to see if these two measures converge or diverge over time.
What about the rest of the data?
We also use the panel survey data for other purposes. Each month, we compare the current month to the previous one on a set of indicators.
The entrance survey asks riders for their CharlieCard or CharlieTicket number in accordance with the MBTA Privacy Policy. We use this information to research how the system’s performance relates to customer satisfaction. With a CharlieCard number, we can match the customer-reported trip on the survey with the trip on the transit system so we can measure the performance the customer experienced.
Limitations of the data
The results are limited by who is in our panel. Currently there are about 4,500 people in the panel, with a disproportionately high number of commuter rail riders. All results are weighted by actual mode usage to correct for this. In order to get results that are representative of our entire customer base we are actively recruiting more participants. particularly those who primarily speak non-English languages. If you wish to join, please sign up at this link: Join the panel.
The MBTA also conducts a customer satisfaction survey every two years using an in-person intercept methodology. This ensures we reach people who may not participate in an online panel. This survey asks a similar set of questions so we can compare results between the methodologies.