Module 7: Investigating people and platform metrics

Learn all about metrics and managing the results from your respondents in this brief video. In the script, the "Survey people" button has been replaced with the "Edit and send" button. 

Prefer to read along?

Click CC to enable captions in YouTube or read the full transcript below.

Welcome back to the Delighted Admin Certification! In this penultimate module, we’ll pry open the Metrics tab. Metrics summarize the performance of every platform in your distribution mix, which helps answer a critical question: “Are we proactively reaching the right people?”

A bit deeper into this module, we’ll explore another side of this same question by dissecting the People page. This powerful page tracks every audience member ever surveyed by a project.

In this section, we’ll learn about: Interpreting Metrics, sorting the People page, exploring Response Histories, and deleting and Unsubscribing people.

Metrics pinpoint the effectiveness level of your distributions.  Tracking starts at the moment you start  sending your surveys and carries on throughout the campaign. Whether you’re looking at response rates for the Email platform or the total impressions for your Web distribution, Metrics quantify what’s happening.

Click the tabs along the top of the Metrics page and view each of Delighted’s platform summaries, including: Email metrics, Web metrics, Link metrics, Kiosk metrics, and iOS SDK metrics.

Since we don't have time in this video to cover each summary, we'll use Email Metrics to illustrate a few key points. The Email page can be filtered by date and includes these metrics:

Sent: This is the total number of email surveys you've distributed within a selected date range. Note that, unlike the Dashboard, the Metrics page is filtered based on when the surveys were sent, not when they were responded to.

Opened: The number of opened emails (even if the surveys were not answered)

Responded: This section posts the number of email sends resulting in a response (either with or without comments). Basically, this is the number of people who clicked on a score in the email.

Commented: This tracks the number of respondents providing comments. The percentage here is the percentage of responses that included comments, whereas the other percentages are based on the number of surveys sent.

Throttled: This is the number of surveys that were not sent because the respondent had received another survey too recently (aka, within the throttle period)

Bounced: This metric shows the number of emails that bounced before delivery. (In other words, it was rejected by an email server, perhaps due to a bad address) 

Unsubscribed: This metric displays the number of surveys that resulted in people unsubscribing

For more details on the metrics shown on every tab in the Metrics page, check out the link in the Show notes.

Let’s take a look at a few scenarios. In each case, we have a question, and the Metrics page has the answer!

In our first scenario, we’re visiting Eta. Eta runs the eNPS surveys for her organization’s HR team. Normally, these surveys run automatically with Delighted’s Autopilot feature, but Eta wants to send out a pulse to all of the team members this month. She imported her list to the Survey people page and sent, but she hasn’t received any responses, and it’s been 24 hours! Is everyone in the organization simply ignoring her email?

Let’s take a look at the Metrics page. Can you determine why Eta’s coworkers haven’t filled out the eNPS pulse?

All of her surveys were throttled! It looks like Eta has set up a 3-month throttle period for her eNPS survey, which means that if she attempts to add in an extra surveys before the 3 months have passed, the emails won’t be sent. She can reduce that throttle in the project settings and re-upload her list if she decides she really does want to send an extra survey.

Okay, on to scenario 2: Delta is responsible for distributing the quarterly NPS survey to customers. He received responses this quarter and, in preparing his report, realized that he received a lot fewer responses than last time he sent. He knows his teammates are going to wonder why, so he plans to investigate and come prepared. How can he do this?

Let’s take a look at the Metrics page with Delta and see if we can uncover some answers. We’ll filter to each time period and compare. This quarter is above, and last quarter is below. What do you think?

Interesting: while more responses were received last time Delta sent the survey, the response rate is actually about the same! That’s because more surveys were sent out last quarter than this one. After doing some more thinking, Delta realizes that this makes sense: last quarter was the holiday season, and they had a lot more customers to survey than this quarter. Now he’s armed with the knowledge that the response count might be lower, but the response rate is remaining strong.

What if you suspect that just a specific panel member isn’t receiving a survey? Clues as to why can be found in their Response History page. We’ve alluded to Response History pages in previous modules. Let’s take a deeper dive before we put this report to work solving problems.

A Response History provides a detailed record of every interaction that a respondent has had with a Delighted project. As you remember from Module 5, you can open a Response History page by clicking the Name or Email in the top-left corner of any Comment tile, or on the People page.

There's a lot to unpack in a Response History. Let’s power through the list.

Name: A quick note here: remember that anonymous surveys replace names with Person and a number, like 123456789

Email address: Again, only if this survey response is not anonymous

Status: Here you’ll notice if a  participant is Active or Inactive

Survey frequency: This posts the number of times the respondent has been surveyed

Last surveyed: This date stamp displays the last time they were sent or shown a survey

Can be surveyed again: This shows exactly when the Throttle period ends for this person (by default, this will be 30 days after their last surveyed time)

Delete this person: This link will purge a person’s entire record! Importantly, this Delete option is only available to Admins.

Feedback tab: This tab, open by default, displays a copy of the Comment tiles as seen on the Dashboard. (And before you ask, yes, you can open the Permalink page, Additional Questions summary, and the Tags page from this tile! That’s helpful, believe me.)

Export Feedback: Click this link to download a CSV copy of this Response History

Activity tab: Click this tab to reveal the date and time of each interaction you’ve had with this respondent

Activity events: Click on each entry to see more information about the activity, such as which properties were included in a survey request

Export all activity: Select this Export link if you wish to download to a CSV file every interaction a respondent has had within this project

Digging into an individual’s Response History can help troubleshoot what’s up with a send. Look at the Status section and click the Activity tab for clues.

For example, in Ellie’s case, the survey has been Throttled, so her follow-up survey will not be sent until after the scheduled time has elapsed. (Problem identified.)

Scrolling through hundreds of Comment tiles on the Dashboard to find the exact person you're looking for can be extraordinarily tedious, especially when responses are pouring in! Literally, Comment cards will start whizzing by in real time. Thankfully, the People tab provides an alpha-numeric way to index and to access individual histories. And, right on cue, we’ll explore the People page next.

The People page stores, alphabetically, the Response Histories of everyone you've ever surveyed. And it doesn’t matter which platform you’ve decided to deploy. (Literally! Everyone surveyed on every platform in your distribution mix is saved on the People page!)

Want to know how to get to this amazing place? Just click People! Then, click on the name of any person to view their Response History. You may even use the Search tool if you know the name of the individual. The People page does its fair share of sorting as well.

The People page can be sorted into three important sections: Active, Unsubscribed, and Bounced. Active includes the People who can be actively surveyed. (Good to know!) Unsubscribed displays the People who have opted out of your survey (or who you have unsubscribed). (Also good to know). And finally, Bounced provides a list of People who cannot be reached. These people’s emails may not exist, or sending to their email simply failed too many times. No need to send to these addresses again (in fact, we won’t try). (Very good to know!)

As always, the Show notes contain links to more information on these sections!

As an Admin, you can manage People in more permanent ways by unsubscribing or deleting them. You can unsubscribe a single email address or an entire batch from the People tab. Simply:

1. Copy the target email address(es)
2. Click “Unsubscribe people” along the left-hand side of the People page
3. Paste or import a list of the email address(es) to be unsubscribed
4. Click unsubscribe. The email addresses can now be found on the Unsubscribed list.

Delighted offers an API endpoint called unsubscribe people that lets you automatically unsubscribe folks based on specific activities (like churn). Check the Show Notes for more details. 

To delete panel members, including Active, Bounced, and Unsubscribed folks:

1. Click the link to the people page
2. Pick the section where the Person is listed
3. Select the Delete option adjacent to the name or email address you wish to delete
4. Click Delete this person on the subsequent page

This can be helpful if someone reaches out for GDPR purposes. We don’t recommend deleting someone just because you didn’t mean to send to them or unsubscribe them—that won’t help!

Instead, check out these potential scenarios and their solutions:

One: you upload a list of people to survey that you didn’t mean to. In this case, if the email hasn’t been sent out, navigate back to the Survey peopleEdit and send page, click “Survey people” under email, and cancel the scheduled send. If the email has already been sent out, there is nothing that you can do: you can’t un-send an email.

Two: if someone is listed in the Active section but is marked Inactive, it doesn’t mean they are a couch potato! It’s more likely that the individual previously unsubscribed to Delighted surveys. It can also mean that their email address bounced for another Delighted customer. Delighted will not attempt another send to a defunct address in either case.

Three: unsubscribing error. If you need to resubscribe someone that has been unsubscribed by mistake, send the Concierge team a request to restore them to the panel.  (Even Admins can’t fix this error without a Concierge as backup.)

Thanks for joining! In this video, we covered: Interpreting Metrics, sorting the People page, exploring Response Histories, and deleting and Unsubscribing people. Just one module left—we’ll see you there soon!