Overview
Not all exercises have exactly one right answer: there could be a few different ways to arrive at the same result, or the exercise prompt could be openended and ask the student to pick from a selection of possibilities. In either case, and with minimal changes to your learnr tutorial, gradethis can accommodate exercises with multiple solutions!
To signal that your exercise has multiple solutions, write all of the
solutions in the exercise’s solution
chunk, separating the
solutions with code headers — a comment followed by at
least four dashes.
# code header 
You can think of the code headers as labels for each
solution variant. Here’s an example solution
chunk for an
exercise with two possible solutions (the full exercise is presented
below).
```{r averagesolution}
# mean 
mean(fibonacci)
# median 
median(fibonacci)
```
There are two types of exercises with multiple solutions:

Multiple solutions, same result. When the solutions describe multiple ways of reaching the same final result, gradethis provides code feedback hints based on the closest solution to the user’s code.
In most cases, you’ll only need to write out the solutions in the
solution
chunk, and gradethis will take care of the rest. Learn more in Multiple solutions, same result below. 
Multiple solutions, different results. Other exercises have more than one acceptable result. In these exercises, multiple solutions can be used both for grading students’ results and for providing code feedback.
In addition to writing out the solutions in the
solution
chunk, you may also need to modify your grading code slightly. Learn more in Multiple solutions, different results below.
In both cases, multiple solutions are written in the same way in the
solution
chunk, and minimal changes are needed in your
grading code. For advanced grading requirements, gradethis adds two
objects into the grading environment: .solution_code_all
and .solution_all
. We’ll cover both
straightforward and advanced uses in the examples that follow.
Multiple solutions, same result
If all your solutions have the same result, you might not need to do anything special in your grading code, as we’ll see in the next example.
Example
In this example, students are asked to convert a numeric value to a
whole number. The most likely solution is to use the
round()
function, but a creative student might call
floor()
or as.integer()
.
Notice that we include all three variations in the
roundsolution
chunk below, but we’ve placed the answer we
consider “most correct” as the last solution in the chunk. The last
solution in the chunk will be the default solution used by gradethis
when the student’s code is not a good match for any of the
solutions.
```{r setup}
library(learnr)
library(gradethis)
```
Convert 5.2 to a reasonable whole number.
```{r round, exercise = TRUE}
____(5.2)
```
```{r roundsolution}
# floor 
floor(5.2)
# as.integer 
as.integer(5.2)
# round 
round(5.2)
```
```{r roundcheck}
grade_this({
# provide code hints if submission is wrong
fail_if_code_feedback()
# pass if the value is correct
pass_if_equal()
# or fallback to a failing grade
fail(hint = TRUE)
})
```
If a student enters any of the correct answers to arrive at the value
5
, they receive positive feedback and encouragement.
as.integer(5.2)
Fantastic! Correct!
If a student gives an incorrect answer using round()
,
the code feedback hint provided by fail(hint = TRUE)
will
nudge them towards the correct round()
solution:
round(5.2, digits = 1)
I did not expect your call to round()
to include digits = 1
. You may have included an unnecessary argument, or you may have left out or misspelled an important argument name.
And if a student gives an incorrect answer but their text is closer to one of the other solutions, the code feedback will nudge the student toward the correct solution.
flor(5.2)
I expected you to call floor()
where you called flor()
.
as.numeric(5.2)
I expected you to call as.integer()
where you called as.numeric()
.
Providing different messages for different solutions
Grading functions that use solution code — like
fail_if_code_feedback()
, code_feedback()
or
fail(hint = TRUE)
— will automatically give feedback based
on the closest solution to the student’s input. In some cases, you may
want to accept multiple solutions, but give different feedback depending
on the solution the student gave.
To enable advanced custom grading, gradethis makes all of the
solutions to an exercise available to grading code as
.solution_code_all
, a named list containing the code for
each solution.
You can use code like the following template to give a special
message (<message>
) for user input that matches a
particular solution identified by the label used for the code heading
above the solution (<solution_label>
):
correctly_used_solution <
is.null(code_feedback(.user_code, .solution_code_all[["<solution_label>"]]))
pass_if(
correctly_used_solution,
message = "<message>"
)
Some things of note in this template:
You can obtain the code for a specific solution using the label in its code heading: e.g.
.solution_code_all[["floor"]]
in our example would return"floor(5.2)"
.You need to explicitly ask for
code_feedback()
for the solution of interest. Otherwise, by default,code_feedback()
will first match the user’s code with the most option among the set of solutions.code_feedback()
returnsNULL
whenuser_code
matchessolution_code
. Testing that the feedbackis.null()
is a robust way to test that the student submitted code that matches a solution because it ignores minor differences like whitespace and pipe usage.
Example
Using the template above, we can provide some feedback specifically
to students who use the floor()
solution.
```{r roundcheck}
grade_this({
correctly_used_floor <
is.null(code_feedback(.user_code, .solution_code_all[["floor"]]))
pass_if(
correctly_used_floor,
"Correct, but remember that `floor()` always rounds down.
If you want to round to the nearest whole number, `round()` is usually safer."
)
pass_if_equal()
fail(hint = TRUE)
})
```
floor(5.2)
Multiple solutions, different results
If your exercise has multiple acceptable results, you’ll need to slightly adapt your code to ensure students’ inputs are tested against all acceptable results.
To compare the result of student input to each solution, you should
use .solution_all
as the first argument in
pass_if_equal()
. Multiple solution results are saved in an
environment called .solution_all
. (Because
.solution_all
is an environment, it requires some special
handling. Check Working with
.solution_all
for more details.)
Within pass_if_equal()
messages, you can use the object
.solution_label
, which corresponds to the code header of
the solution that the student’s code matched.
Functions that grade students’ code can still be used without any changes.
Example
```{r setup}
library(learnr)
library(gradethis)
fibonacci < c(0, 1, 1, 2, 3, 5, 8, 13, 21, 34)
```
`mean()` or `median()` to find the average of `fibonacci`, a vector containing the first 10 numbers in the Fibonacci sequence.
Use
```{r average, exercise = TRUE}
____(fibonacci)
```
```{r averagesolution}
# mean 
mean(fibonacci)
# median 
median(fibonacci)
```
```{r averagecheck}
grade_this({
pass_if_equal(.solution_all, "You solved it with `{.solution_label}()`!")
fail(hint = TRUE)
})
```
If a student gives either correct solution,
pass_if_equal()
will give them a passing grade:
mean(fibonacci)
You solved it with mean()
!
median(fibonacci)
You solved it with median()
!
If the student’s input doesn’t match any of the solutions, grading will proceed as normal:
sum(fibonacci)
Working with all solution results
.solution_all
is an environment, not a list. This
carries certain advantages. Exercises can be graded faster because each
solution can be evaluated as needed rather than all up front.
But it also comes with drawbacks. The most important are:
 environments can’t be subset with numbers, only names, and
 all elements of an environment need to have a unique name.
If you need to access the results of specific solutions, make sure you give a unique label to every solution.
To reference a specific solution result in grading code, use the
solution label to access the result in the
.solution_all
:
grade_this({
pass_if_equal(
.solution_all[["mean"]],
"That's right! Do you think the mean is skewed?"
)
pass_if_equal(.solution_all)
fail_if_code_feedback()
fail()
})
If you don’t give your solutions unique names, gradethis
will generate unique names for you. For example, these solutions
# mean 
mean(fibonacci)
# mean 
mean(fibonacci, na.rm = TRUE)
# median 
median(fibonacci)
# median 
median(fibonacci, na.rm = TRUE)
have these names:
#> [1] "mean" "mean_1" "median" "median_1"
This means that if you need to access specific solution results in
your grading code, it’s almost always better to give your solutions
unique labels. On the other hand, if your passing message makes use of
.solution_label
, you may want to duplicate the solution
labels.
Recall that the check
chunk for the
average
exercise was:
```{r averagecheck}
grade_this({
pass_if_equal(.solution_all, "You solved it with `{.solution_label}()`!")
fail(hint = TRUE)
})
```
By including the mean()
and median()
solution variants with na.rm = TRUE
we get an appropriate
passing message even though the solution labels are duplicated:
median(fibonacci, na.rm = TRUE)
You solved it with median()
!