WISEflow Guidance for Assessing (Marking)

In WISEflow, the roles of a marker and a reviewer have distinct responsibilities and functions:

Assessors (Markers):

• Primary Responsibility: Assessors are responsible for assessing and grading student submissions. They evaluate the work based on predefined criteria and provide grades and feedback.
• Marking Methods: Assessors can work individually or collaboratively with other markers. In collaborative marking, they may need to reach an agreement on the grades before finalising them.
• Feedback: Assessors can provide detailed feedback, including comments and annotations, directly on the submissions.

Reviewer:

• Primary Responsibility: Reviewers oversee the marking process and ensure the consistency and fairness of the grades. They have the authority to approve or adjust the grades submitted by markers.
• Approval Role: Internal Reviewers (usually the unit lead) can resolve grade conflicts and make final decisions on grades.
• Quality Assurance: Reviewers ensure that the marking standards are maintained and that the feedback provided is constructive and aligned with the assessment criteria

Key Differences:

• Decision-Making: Markers assign initial grades, while reviewers have the final say in approving or adjusting these grades.
• Conflict Resolution: Reviewers handle any discrepancies or conflicts in grading between markers.
• Oversight: Reviewers provide an additional layer of quality control to ensure the integrity of the assessment process.

Exam Confirmation Process

Before the examination period, the exam team sends out an exam confirmation exercise.

This is a spreadsheet located in our Exams MS Team site

Colleagues (unit leads/unit coordinators) are invited to input their information, indicating:

• Who will be assessing (Marking student scripts and entering initial grades)
• Who will be reviewing (The quality assurance process, reviewing the marking that has taken place)
• Who the external examiners are for the unit
• Any details of shared marking or item based marking (who marks which questions for FLOWmulti)
• Any details of which units need to be scheduled at the same time
• Any details regarding units that require booklets
• Any details of whether remote written examinations are to be run as FLOWassign or FLOWlock

Please update the exam confirmation exercise with the correct details of who will be assessing or reviewing.

Then email exams@beds.ac.uk to let us know so we can action the change as the managers in WISEflow.

1. Access WISEflow's re-marking area.
2. Access WISEflow's grade justification area.
3. Access the marker archive.
4. Type to filter the flow list.
5. View flow information. Click any row to enter that flow
6. See the status of each flow (assessors are able to interact with the flow once it enters the 3rd chevron stage or the "marking period")

Submissions made to a FLOWassign or FLOWlock will be presented in the marking tool as shown below:

Submissions made to a FLOWassign, and FLOWlock will be presented in the marking tool as shown below:

1. Flow Name & Student Name – Displays the current flow and student name (if not anonymised).
2. Options Menu – Access functions such as download, print, and settings.
3. Full Screen View – Expand the interface to full screen for an immersive experience.
4. Page Navigation – Move between multiple pages within the current document.
5. Page Display Options – Adjust page transitions, rotate pages, and choose between single or double-page layouts.
6. Magnification Presets – Select a preferred zoom level.
7. Zoom Controls – Manually zoom in or out of the document.
8. Pan Tool – Move around the page, especially useful when zoomed in.
9. Text Selection – Highlight and copy text from the document.
10. Current Page Indicator – Shows the page currently being viewed.
11. Annotation Tools (Basic) – Add annotations using tools listed in item 14.
12. Shape Annotations – Draw using freehand, highlight, rectangle, ellipse, polygon, line, polyline, or arrow. Choose from red, blue, black, or green.
13. Insert Elements – Add a signature, rubber stamp, or date.
14. Annotation Tools (Text) – Use highlight, underline, strikeout, squiggly underline, note, and free text.
15. Undo – Revert the last action.
16. Redo – Reapply the last undone action.
17. Eraser – Remove annotations or shapes.
18. Text Search – Find specific text within the current document.
19. Annotation Summary – View all annotations in a side pane, sortable by preference.
20. Participant Indicator – Shows the index number of the participant whose submission is being viewed.
21. Participant Navigation – Move up or down through the list of participants.
22. File Navigation – Switch between files submitted by the current participant.
23. Navigation Bar Positioning – Move the navigation bar to a preferred location on the screen.
24. Close Navigation Bar – Hide the navigation bar when not needed.

Submissions to a FLOWmulti will be presented in the marking tool as shown below:

Submissions made to a FLOWmulti will be presented in the marking tool as shown below:

1. Flow Name & Student Name – Displays the current flow and the student’s name (if not anonymised).
2. In the View tab, each item of the assignment allocated to you is shown in turn (item-based marking). Note: You may be assigned only specific items rather than the entire assignment.
3. If you’ve been assigned the full assignment, you can add inline annotations to any item, including those that have been auto-scored. To do this, select Annotate, then use any of the available tools (annotation tools, shape tool, and insert tool).
4. The Submission button provides an overview of the participant’s items, including any appendices they may have attached. Items must be scored before grades can be locked and submitted. You can also select Offline Marking here.
5. The Scoring button is where you enter marks for each item.
6. Submission Feedback – This is where you provide feedback for individual students.
7. This is where you confirm and submit the final grade.
8. This shows the scores given for each item.
9. This displays the total overall score.
10. The Question Statistics window shows responses and scoring data for the item currently being viewed in the marking tool.
11. Use the Participant List to navigate between participants.
12. Common Feedback – This is where you provide feedback to all students.
13. Flow Statistics can be accessed by clicking the Flow Statistics button in the left-hand navigation menu of the marking tool.

Moving between submissions is possible in several different ways.

From the Participants List

The participant list can be accessed from the left-hand navigation pane and displays all the participants to which you have been assigned as an assessor and provides a quick way to move between each submission from inside the marking tool.

From the Navigation Window

It is also possible to move backwards and forwards to the same item for the next submission using the navigation window at the bottom of the screen. Click on the navigation icon in the left-hand navigation pane to open the navigation window.

You can find archived flows in the marking archive, available from the marking landing page.

Please note you will not have access to student scripts or the exam papers once the marking period ends.

Please ensure you carry out the marking before the end of the marking period.

If you are looking for specific details about a past examination, please contact the exams@beds.ac.uk

If you require an extension please contact your faculty quality lead for approval and send this to the exams teams.

Submission Feedback: for individual students

From inside the marking tool, click Feedback icon on the header bar. Add your text to the text box and format. Select your preferred sharing option i.e. with participants, co-assessors or any combination. Add a file if you wish and submit. Any previously entered feedback is listed below in the same window. The list of feedback can also be filtered and sorted.

1. Open Feedback Panel – Launch the feedback window for the current submission.
2. Enter Comments – Provide written feedback specific to the participant’s submission.
3. Attach Feedback File – Upload a file to accompany your feedback.
4. Set Feedback Visibility – Choose who can view the feedback using the visibility options.
5. Feedback Summary – View an overview of the feedback provided for the submission.

Common Feedback: Feedback provided to all participants

From inside the marking tool, click the Common Feedback icon on the left-hand navigation pane. Add your text to the text box and format. Select your preferred sharing option i.e. with participants, co-assessors or any combination thereof (more information on sharing settings in the annotations article).

1. Access Common Feedback – Click the speech bubble icon to open the common feedback section.
2. Feedback Text Box – Enter feedback intended for all participants who took the exam.
3. Attach Files – Upload files to be included with the common feedback for all participants.
4. Visibility Settings – Use the drop-down menu to control who can view the common feedback.
5. Submit Feedback – Click this button to submit the common feedback to all participants.
6. Feedback Overview – Displays a summary of the common feedback provided.

The new marking tool offers 3 different inline annotation options which include

• The annotate tool

Click Annotate, and choose from the tools available: highlight, underline, strikeout, squiggly underline, note or a text box. Select the area of text you wish to draw attention to and the desired colour. Once the tool and the text has been selected, an annotation box will appear in the right-hand window where you can add your comment.

• The shapes tool

Click Shapes and choose from the shapes available (see screenshot below) including a free hand tool. Once the chosen shape has been inserted, (and a preferred colour chosen, if applicable), an annotation box will appear in the right-hand window where you can add your comment.

• The insert tool

Click Insert and choose to insert a signature, a rubber stamp or today's date. Once the chosen tool has been inserted, (and a preferred colour chosen, if applicable), an annotation box will appear in the right-hand window where you can add your comment.

Sharing on Individual Annotations

• Private: Comments and annotations are visible only to the assessor (This is unless a reviewer has been given the privilege to access all assessment information which includes all comments made by an assessor, even if they are set to private).
• Share with participant: Comments and annotations are visible to the assessor and the participant (upon the release of grades and feedback) only.
• Share with co-assessor: Comments and annotations are visible to the assessor and any co-assessors that are also assigned to the same participant.
• Share with all: Comments and annotations are available to the assessor, and co-assessors that are also assigned to the same participant and the participant (upon the release of grades and feedback).

These options can be configured per annotation in order to offer maximum flexibility to assessors. To change the setting on a single annotation, click on the coloured box in the top right corner to change with whom the comment is to be shared: participant, co-assessor, private or all.

At UOB all comments are automatically set to private as per the institution regulations unless the assessor decides to share them (only if a student has requested feedback).

Rubric-based marking is an advanced marking workflow available with all flow types.

Please contact the exams team at exams@beds.ac.uk in order to ensure rubric marking can be enabled, and please attend a authoring training session to learn how to create and set up rubric based marking.

WISEflow is designed primarily to be used in an online capacity. However, there are some circumstances where a continuous internet connection is not practical or possible for assessors. To cater for such situations WISEflow offers some offline features.

Submissions made to FLOWassign or FLOWlock can be downloaded in bulk in a single zip file and can include a grading spreadsheet or grade sheet. To build your offline marking package, select Offline marking button from the submission Overview button.

If you require any assistance with this please contact the exams team.

Please ensure that once you are connected to the internet, please ensure that you transfer your marking back to WISEflow and submit the grades once done.

Once you have submitted your final grades as a marker, and your co-assessors have also submitted their final grades, the reviewer will then be able to reviewing all the marking and finalise grades.

1. Access WISEflow's re-marking area.
2. Access WISEflow's grade justification area.
3. Access the marker archive.
4. Type to filter the flow list.
5. View flow information. Click any row to enter that flow
6. See the status of each flow (assessors are able to interact with the flow once it enters the 3rd chevron stage or the "marking period")

WISEflow provides access to a demonstration flow which will allow assessors to try out some of the marking features available in the system. The demo flow can be accessed from the bottom of the landing page.

To access the submissions from the flow landing page, navigate into the "open the marking tool"button under the commenting and feedback area.

It is also possible to jump straight to an individual submission from the participant list at the bottom of the page.

Submissions made to a FLOWassign or FLOWlock will be presented in the marking tool as shown below:

1. Close the marking tool
2. Submission overview tools as well as page, word and character count.
3. Annotation tools
4. Flow menu (view flow information, provide cohort feedback or switch between submissions)
5. Participant menu (view cover sheet, provide paper-level comments, access shared summary, enter final grade)
6. Tally if using marks with annotations
7. Access similarity report
8. Open appendix material
9. Open rubric
10. View annotations
11. Search the submission
12. Export submission with overlaid annotations

Submissions to a FLOWmulti will be presented in the marking tool as shown below:

1. Close the marking tool
2. Flow menu (view flow information, provide cohort feedback or switch between submissions)

3. Participant menu (provide paper-level comments, access shared summary, enter final grade)

4. Jump to individual items
5. Generate a printout of the submission.
6. Adjust the zoom level
7. Total marks achieved by automatic scoring
8. Total marks added by manual scoring
9. Total marks tally
10. Display rubric for the assessment process (not available for item-based marking)
11. Show any media or tools that have been attached to an item/the assignment
12. Hide the question text for a better overview of the answers
13. Statistical overview of the submission
14. View annotation summary
15. Add annotations to the item
16. Automatic score for the item
17. Manual score for the item
18. Total item score

Moving between submissions is possible in several different ways.

From the Flow Menu

The flow menu displays all the participants to which you have been assigned as an assessor and provides a quick way to move between each submission from inside the marking tool.

From the Flow Landing Page

Closing the marking tool will take you back to the flow landing page from which you can also view a list of your assigned participants. Click the green speech bubble against any row to jump straight to the corresponding submission in the marking tool.

WISEflow has 3 annotations tools:

1. The text highlighting tool:
2. The stamp tool
3. The freehand drawing tool

The annotation editor window

The annotation window associated with all three annotation tools is shown below.

1. Jump to the area in the document with which the annotation is associated
2. Close the annotation editor window
3. Text formatting tools
4. Commenting area
5. Change the colour of the comment box and overlay on the document
6. Choose whether the document overlay displays as a highlight, strikethrough or underline (where the stamp tool is used this item will allow you to select the shape of the stamp)
7. Associate marks with annotation (see next section of this article)
8. Sharing settings (see final part of this article)
9. Delete the annotation

Please note that there is no save button for annotations. Annotations are automatically saved.

Sharing on Individual Annotations

There are four possible options for sharing annotations:

1. Private: Comments and annotations are visible only to the assessor.
2. Share with participant: Comments and annotations are visible to the assessor and the participant (upon the release of grades and feedback) only.
3. Share with co-assessor: Comments and annotations are visible to the assessor and any co-assessors that are also assigned to the same participant.
4. Share with all: Comments and annotations are available to the assessor, and co-assessors that are also assigned to the same participant and the participant (upon the release of grades and feedback).

These options can be configured per annotation in order to offer maximum flexibility to assessors. To change the setting on a single annotation click the dropdown menu and select the groups you would like to share the annotation with.

Please note that the default WISEflow behaviour is that comments are private unless the assessor decides to share them (only if a student has requested feedback).

From inside the marking tool click the right-hand student menu in the page header where you will find grading and commenting tools for the current submission you are working on. Click the Add comment button under Comments on this paper. This opens the paper-level commenting tool which works in a similar way to the individual annotation tool.

From inside the marking tool click the left-hand flow menu in the page header. This brings up a menu with the student list on the right and the flow information on the left. Click the Add comment to all participants button under General comments on the entire flow. This opens the cohort-level commenting tool which works in a similar way to the individual annotation tool.

WISEflow supports integration with the text similarity tool: Turnitin.

When text similarity is enabled on the flow, WISEflow sends submissions to the respective services and subsequently displays the returned matching percentage in the marking tool.

This is how a turnitin similarity report looks like:

1. The document viewer displaying the entire submitted document with similarity events and flags highlighted in the text
2. Help menu: view the submission metadata, access the help centre or initiate a Turnitin tour
3. Report settings: Options for what to include in the analysis
4. Event overview: An overview of the sources matched or flags detected
5. Similarity/flag toggle: Choose whether to view sources matched or flags detected in the event overview

Rubric-based marking is an advanced marking workflow available with all flow types.

Please contact the exams team at exams@beds.ac.uk in order to ensure rubric marking can be enabled, and please attend a authoring training session to learn how to create and set up rubric based marking.

WISEflow is designed primarily to be used in an online capacity. However, there are some circumstances where a continuous internet connection is not practical or possible for assessors. To cater for such situations WISEflow offers some offline features.

Submissions made to FLOWassign or FLOWlock can be downloaded in bulk in a single zip file and can include a grading spreadsheet or grade sheet. To build your offline marking package, select Offline marking button from the flow overview page.

Please ensure that once you are connected to the internet, please ensure that you transfer your marking back to WISEflow and submit the grades once done.

Once you have submitted your final grades as a marker, and your co-assessors have also submitted their final grades, the reviewer will then be able to reviewing all the marking and finalise grades.