Review#

Reviewing questions in Numbas requires two different levels of review:

  1. Academic review of question

  2. Testing of the question in Numbas

The review steps are outlined in more depth below.

Academic review#

The first review should be review of the suitability of the question and whether the advice, wording and stimulus are appropriate for the target learner.

This step can be completed after the question has been written and before it is developed in Numbas.

One recommended way of completing this review it to work off a document like the template on the Writer’s Checklist page.

Once the academic review has been complete, the question can then be build.

Question testing#

The second review should take place after the question has been built in Numbas.

There are a number of ways to do this.

One way is to complete the quiz in Numbas and export it to vUWS.

The benefits of reviewing a quiz with this method include:

  • The reviewer can see the questions the same way the student will see the them

  • The quiz settings and instructions can be reviewed at the same time

  • The reviewer does not need to know how to use Numbas or have a Numbas login

The downside of completing it in this way, is that it is hard to keep track of comments and suggested changes without referring to another document (see Keeping Track for an example of a document that could potentially be used for this purpose).

Another way of completing the review is to use a feature in the Numbas editor known as ‘Queues’ (for information on queues, see the official Numbas Documention here).

When working on a Numbas project, the project owner can set up a ‘Queue’ with an attached checklist/instructions for both builders and reviewers.

As the builder progresses, questions can be added to the queue. When the reviewer checks the queue they are greeted with a list of question to review.

The reviewer can check the items off the checklist and, if necessary, add comments for the builder.

This ensures that the question is reviewed and completed entirely before being exported from the Numbas Editor.

It is recommended that a final review of the quiz settings is taken place once the quiz in vUWS to ensure nothing is missed.