Dashboard Overview

A dashboard is available at route /dashboard. The dashboard can be enabled by setting enable_dashboard to True, and by setting a login_username, login_pw, and secret_key.

The dashboard has many features, including a dynamic filtered table, batch actions, managing campaigns.

Dynamic Filtered Table

View current status of participants, queried from the psiTurk database.

  • Filter by:
    • mode
    • experimental condition
    • current code version
    • whether the participant has a status of ‘complete’
  • Group by:
    • condition

Batch Actions

Functionality to manually trigger the following actions:

  • Workers:
    • Approve all HITs
    • Bonus all submissions via the “auto” method (based on the value set in the “bonus” column in the database).
  • HITS:
    • Expire all
    • Approve all workers for all hits
    • Delete all

Campaigns

Campaigns are scheduled as tasks. Campaigns have the following features:

  • Set a target “goal” for number of workers for a given task code version.
  • Stagger the posting of HITs by a specified interval.
  • Post HITs in batches of 9 assignments. This keeps the MTurk commission at 20%, instead of 40%.
  • Monitors the number of available HITs, and continues posting rounds of HITs until the campaign goal has been met.
  • Manually cancel a campaign.

The “Campaigns” tab also displays past campaigns.

Tasks

psiTurk “tasks” are stored in their own table in the database specified by the database_url. To enable a specific psiturk server to run tasks, set do_scheduler to true.

The “tasks” tab allows for scheduling an “Approve All workers” task which will be run at a set interval. Will “approve” all submissions currently marked as “Submitted” in the psiTurk database.

The tab will also display any currently running campaigns. To edit a campaign, visit the “Campaigns” tab.

Warning

The managing of tasks and the running of tasks is handled separately!

This is because psiTurk uses APScheduler for tasks, which does not currently handle interprocess synchronization (see this APScheduler FAQ ).

This means that any dashboard can view, create, delete, and update tasks, while a single separate psiTurk server instance can be set up with only one thread for task-running.

Note!: if do_scheduler is set to True, and threads is greater than 1, psiTurk will refuse to start! This is a safeguard, because, again, APScheduler cannot handle interprocess task-running synchronization.

Configuration

  • Set dashboard mode.
  • View AMT balance.