The Learner Decision Report provides an activity-level insight on learner performance to better understand which question options impacted the metrics.
Purpose
Improve Content Design:
By seeing which decisions are frequently chosen and which are less popular, instructional designers can adjust the content to either promote less chosen options or reframe more common choices to enhance learning outcomes. For instance, if too many learners select a suboptimal decision, the learning experience can be restructured to guide them toward better options.
Assess Decision Quality:
For decisions that have correct and incorrect options, the report can show how many learners are making correct decisions versus incorrect ones. This data helps measure how well learners are mastering key concepts and making effective choices.
Identify Knowledge Gaps:
If a particular decision consistently has a lower percentage of correct answers, it could point to a gap in the learner's understanding or a need for more emphasis in certain areas of the content. This data can help pinpoint knowledge gaps that require further attention in the course.
Report Configuration Settings
ATTENTION! To display session data, the project must be properly configured to ensure accurate calculations and scoring.
If the project settings are incorrect, the report will not be supported, and course data will not be calculated or displayed.
To configure these settings,
In Studio, set up scoring for the project on the Scoring page, accessible from the top navigation.
Configure Scoring Metrics within the rounds or pages as needed.
If you make any changes, you must publish the project for the updates to take effect.
Important Configuration Notes:
Learner Decision Release
To enable the Learner Decision report for existing projects, upgrade the Studio project to the latest release and re-publish it.
User Data May Be Incomplete for Past Sessions
Please note that user data from past sessions may not be calculated or displayed if the necessary settings were not configured correctly before users completed the course.
Re-Publish After Updating Metrics
If you update or add metrics to a project, be sure to re-publish it to ensure the new metrics are applied to future sessions.
Only One Session Can Be Viewed at a Time
This report supports only one session at a time. If you encounter an error, it likely means multiple sessions are selected. Use the session dropdown menu to choose a single session and view its data.
Dashboard Error Due to Missing Settings
If the dashboard settings have not been configured correctly, you will see an error message instructing you to contact the project author to preselect the metrics.
In-Progress Participants Are Excluded
Participants who are still in progress will not appear in the report, and their scores will not be displayed.
Percentages Reflect Only Completed Users
All percentage-based metrics in the report are calculated using only participants who have completed the course within the selected session.Selected and Scored Decisions
This report displays the decisions that were both selected by the learner and assigned a score during the learning experience—such as choices made in text or list-based activities.
VIDEO: Steps to Configure
Watch the video below to learn how to configure the metrics in Studio to enable the Metrics Report:
DIY STEPS
In Studio, authors must preselect the metrics to include in the report. This can be done by configuring the metrics directly within the Scoring Model or through the project Dashboard Settings under “Reported Metrics”.
Setting up metrics for the report from the Scoring Model:
Click Scoring Model in the top navigation bar in Studio.
When you Add a Metric or Edit a Metric you will see a checkbox that adds the metric to the dashboard report - check the box for each metric that you want added to this dashboard report.
REMINDER: If you have updated or added metrics to a project, make sure you re-publish your project so these metrics can be applied to future sessions.
Setting up metrics for the report in the project Dashboard Settings:
In Studio, click Settings in the top navigation bar.
In the left navigation panel, click Dashboard to access it's project settings.
From the Dashboard Settings, select the metrics you want to display in the Metrics report on the admin dashboard.
Authors MUST preselect the metrics to include in the report in Studio. This can be done through the Dashboard Settings under “Reported Metrics” or by configuring the metrics directly within the Scoring Model. For this specific setting, “Reported Metrics”, no new publish or release is necessary as settings will automatically be applied to the Dashboard.
Viewing the Learner Decision Report
Select a Project and Session
Begin by selecting the project you want to explore. Then select a session with the project. This will display a list of scored decision options within that project.
Understand the Columns
First Column: Displays the state in which the decision is found as the round number and state number. For example, Round 1, State 3 would display “1.3”.
Second Column: Shows the decision option text.
Third Column: Indicates the percentage of completed participants who selected that specific decision. When hovering over the Participant %, the total number of participants who selected that option will display.
Explore a Decision
Click on any decision to view more details in the right-hand panel. This panel will display:The round name and state name
The question associated with the decision
All available decision options for that question
The specific metrics impacted by the selected decision
The top level round, state, and metrics filters will also filter and limit what is displayed on the expanded panel to the relevant decisions.
Search and Filter Decisions
Use the top filters to narrow your view by round, state, or impacted metric. The impacted metric dropdown selections are limited to those preselected by the author in Studio.
Use the search field in the top-left corner to find a specific question. It will find the decision text, state name, and/or round name.
Reorder the columns to sort by clicking the arrow button
Sort states by order (first to last or last to first)
Sort decisions by alphabetical order (A to Z or Z to A)
Sort participants by highest or lowest selection frequency
Report Toolbar Functionality
Filters are interconnected. For example, selecting a project will update the session filter to show only sessions within that project.
The Refresh button updates the data in the chart.
A timestamp next to the Refresh button shows when the data was last updated.
The Download button allows users to export and download data as an Excel file.
FAQ
Why is a user decision not on the report?
Only decisions that have a yes/no answer, such as ‘has the user selected this option?’ or ‘has the user included this item in the list’ , will be calculated and reported.
Additionally, decisions will only be displayed if it has a weighted value in Scoring and the metric is correctly set up in Settings to be included in the dashboard reports.
I added new scoring, but why is the decision not showing up on the Learner Decision Report in the dashboard?
After adding or updating scoring in a project, you must re-publish it. New or updated scoring only applies to sessions created after the latest publish.
Why is the report showing empty?
Check that the correct metrics are selected and set up in the dashboard settings in Studio. The project must also be published using the latest Studio version for the report to populate.
Why is a session not being shown in the report?
The session may not be shown in the report due to improper configurations in the project. The settings in the project must be completed as shown in the How To page, in order for the report to be displayed accurately.
Why does the session data looks inaccurate, a user’s scoring looks inaccurate from what was configured for the session.
The reporting API is new and therefore there may be inconsistencies in the calculations of the metric scores. This may be due to several scenarios, all involving a project update.
For example, a project is set up with a scoring metric and users have completed the course in older sessions. Then, the scoring metric has since been updated to include more values. The report uses the latest project settings, so the data displayed for these older sessions are now calculating scores on the new changes in the project.
While this is rare, there may be several sessions affected by such updates to a project setting.
Please contact us if this a problem for the affected sessions.
My session says that nine users are registered, but the report says there are ten participants?
This may be due to a user being deleted from the session after they have completed the course. The reporting will still calculate and score the user data from that user and display it alongside active users.
