Settings¶
This page is organized by each of the sections of a config.txt
file.
Any of these settings can be set via an environment variable by setting the
setting name (sans section name) prefixed by the key string PSITURK_
. For
instance, to set the number of server threads (threads, section
‘Server Parameters’) to 1
via env var, one would set: PSITURK_THREADS=1
.
See also
See the comments in the sample config.txt file for more information on each config setting.
HIT Configuration¶
The [HIT Configuration]
section contains details about
your Human Intelligence Task.
title¶
Title of the task that will appear on the AMT worker site.
Type: | string |
---|
Workers often use fields like this one to search for tasks. Thus making them descriptive and informative is helpful.
description¶
Descriptive text for your study’s listing on AMT.
Type: | string |
---|
Workers often use fields like this one to search for tasks. Thus making them descriptive and informative is helpful.
keywords¶
A list of keywords to be associated with your study on AMT.
Type: | comma-delimited string |
---|
Workers often use fields like this one to search for tasks. Thus making them descriptive and informative is helpful.
lifetime¶
How long your HIT remains visible to workers (in hours).
Type: | integer |
---|
After the lifetime of the HIT elapses, the HIT no longer appears in HIT searches, even if not all of the assignments for the HIT have been accepted.
This is in contrast to the HIT duration
, which specifies how long
workers have to complete your task, and which you provide at HIT
creation time. See the documentation on hit create for more details.
us_only¶
Controls if you want this HIT only to be available to US Workers.
Type: | bool |
---|
This is not a failsafe restriction but works fairly well in practice.
approve_requirement¶
Minimum approval percentage for a worker to be able to accept your study.
Type: | integer |
---|
Sets a qualification for what type of workers you want to allow to perform your task. It is expressed as a percentage of past HITs from a worker which were approved. Thus 95 means 95% of past tasks were successfully approved. You may want to be careful with this as it tends to select more seasoned and expert workers. This is desirable to avoid bots and scammers, but also may exclude new sign-ups to the system.
number_hits_approved¶
How many hits a worker must have approved before they can take your study.
Type: | integer |
---|
Important to use in conjunction with approve_requirement, because
mturk will default approve_requirement
to 100% until a worker has at least
100 HITs approved. Override that behavior by setting this to at least be 100.
require_master_workers¶
If True, Will make it so that only workers with the “Master” qualification can take your study.
Type: | bool |
---|
See Who Are Amazon Mechanical Turk Masters?
Note: Master workers cost an extra 5%.
browser_exclude_rule¶
A set of rules you can apply to exclude particular web browsers from performing your task.
Type: | comma-delimited string |
---|
When a users contact your psiturk server, the server checks to see if the User Agent reported by the browser matches any of the terms in this string. It if does the worker is shown a message indicating that their browser is incompatible with the task.
Matching works as follows. First the string is broken up by the commas into sub-string. Then a string matching rule is applied such that it counts as a match anytime a sub-string exactly matches in the UserAgent string. For example, a user agent string for Internet Explorer 10.0 on Mac OS X might looks like this:
Mozilla/5.0 (compatible; MSIE 10.0; Macintosh; Intel Mac OS X 10_7_3; Trident/6.0)
This browser could be excluded by including this full line (see this website for a partial list of UserAgent strings). Also, “MSIE” would match this string or “Mozilla/5.0” or “Mac OS X” or “Trident”. Thus you should be careful in applying these rules.
There are also a few special terms that apply to a cross section of browsers. mobile will attempt to deny any browser for a mobile device (including cell phone or tablet). This matching is not perfect but can be more general since it would exclude mobile version of Chrome and Safari for instance. tablet denys tablet based computers (but not phones). touchcapable would try to exclude computers or browser with gesture or touch capabilities (if this would be a problem for your experiment interface). pc denies standard computers (sort of the opposite to the mobile and tablet exclusions). Finally bot tries to exclude web spiders and non-browser agents like the Unix curl command.
allow_repeats¶
Specifies whether participants may complete the experiment more than once.
Type: | bool |
---|---|
Default: | false |
If it is set to false (the default), then participants will be blocked from completing the experiment more than once. If it is set to true, then participants will be able to complete the experiment any number of times.
Note that this option does not affect the behavior when a participant starts the experiment but the quits or refreshes the page. In those cases, they will still be locked out, regardless of the setting of allow_repeats.
require_quals¶
A list of custom qualifications that participants must possess to perform your task.
Type: | comma-delimited string |
---|
If set, applies in both live
and sandbox
modes. Overrides require_quals_live
and require_quals_sandbox
.
Deprecated. Use require_quals_live
and require_quals_sandbox
instead.
You may need to ensure that workers have some requisite skill or pass some
previous screening factors, such as language proficiency or having already
completed one of your tasks. AMT uses custom qualification types to perform
this filtering. When you add a custom qualification to
require_quals
, AMT will only show your ad to potential
participants who already have that qualification set. Other MTurk workers will
neither see your ad nor be able to accept the HIT.
See Managing worker cohorts with qualifications and Best practices for managing workers in follow-up surveys for additional details on custom qualifications.
require_quals_live¶
A list of custom qualifications that participants must possess to perform your task.
Type: | comma-delimited string |
---|
Will only be used during live
mode. Is overridden by require_quals
, if set.
require_quals_sandbox¶
A list of custom qualifications that participants must possess to perform your task.
Type: | comma-delimited string |
---|
Will only be used during sandbox
mode. Is overridden by require_quals
, if set.
block_quals¶
A list of custom qualifications that participants must not possess to perform your task.
Type: | comma-delimited string |
---|
If set, applies in both live
and sandbox
modes. Overrides block_quals_live
and block_quals_sandbox
.
Deprecated. Use block_quals_live
and block_quals_sandbox
instead.
When you add a custom qualification to block_quals
, MTurk
workers with that qualification already set will neither see your ad nor be able
to accept your HIT. This is the recommended way of excluding participants who
have performed other HITs for you from participating in your new HIT.
block_quals_live¶
Will only be used during live
mode. Is overridden by block_quals
, if set.
A list of custom qualifications that participants must not possess to perform your task.
Type: | comma-delimited string |
---|
block_quals_sandbox¶
Will only be used during sandbox
mode. Is overridden by block_quals
, if set.
A list of custom qualifications that participants must not possess to perform your task.
Type: | comma-delimited string |
---|
advanced_quals_path¶
A path to a custom JSON qualifications file, where you can define your own MTurk qualification requirements, as seen in advanced_quals.json.sample
If set, applies in both live
and sandbox
modes. Overrides advanced_quals_path_live
and advanced_quals_path_sandbox
.
Deprecated. Use advanced_quals_path_live
and advanced_quals_path_sandbox
instead.
type: | path |
---|
Example:
advanced_quals_path = ./advanced_quals.json
advanced_quals_path_live¶
A path to a custom JSON qualifications file, where you can define your own MTurk qualification requirements, as seen in advanced_quals.json.sample
type: | path |
---|
Will only be used during live
mode. Is overridden by advanced_quals_path
, if set.
advanced_quals_path_sandbox¶
A path to a custom JSON qualifications file, where you can define your own MTurk qualification requirements, as seen in advanced_quals.json.sample
type: | path |
---|
Will only be used during sandbox
mode. Is overridden by advanced_quals_path
, if set.
Hit Configuration – Ad Url¶
Config settings for constructing the task’s “landing page” when posting hits on MTurk.
The simplest way to set the ad url is to set only the ad_url_domain and leave the rest at their defaults.
ad_url_domain¶
Type: | string |
---|---|
Default: | null |
Server domain name for publicly-accessible route to psiturk server. If running on heroku, set this to your heroku app url – for example, if your heroku app name were “example-app,” you would set the following:
ad_url_domain = example-app.herokuapp.com
ad_url_port¶
Type: | int |
---|---|
Default: | 443 |
Server port for publicly-accessible route to psiturk server. Default is 443, which is the browser-default for https.
ad_url_protocol¶
Type: | string |
---|---|
Default: | ‘https’ |
HTTPS protocol is required by mturk. Only change this if you have a good reason to do so.
ad_url_route¶
Type: | string |
---|---|
Default: | ‘pub’ |
Flask route that points to the ad. “pub” and “ad” both point to the same place, but “pub” is safer because of potential issues with ad blockers with a route named “ad”
ad_url¶
Alternatively, instead of using ad_url_*
config vars above,
you may use ad_url
. You may want to use this if your
experiment is served from a subdirectory off of the domain name. Otherwise,
leave this as-is. By default, it is dynamically built using the other variables in this section,
using ConfigParser templating as follows:
%(ad_url_protocol)s://%(ad_url_host)s:%(ad_url_port)s/%(ad_url_route)s
Database Parameters¶
The [Database Parameter]
section contains details about
your database.
See also
- Configuring Databases
- For details on how to set up different databases and get your data back out.
- Recording Data
- For details on how to put data into your database.
database_url¶
database_url contains the location and access credentials for your database (i.e., where you want the data from your experiment to be saved).
Type: | string - valid database url |
---|
To use a SQLLite data base, simply type the name of the file:
database_url = sqlite:///participants.db
This example would write to a database file with the name “participants.db” in the top-level directory of your experiment.
To use an existing MySQL database:
database_url = mysql://USERNAME:PASSWORD@HOSTNAME:PORT/DATABASE
where USERNAME and PASSWORD are your access credentials for the database, HOSTNAME and is the DNS entry or IP address for the database, PORT is the port number (standard is 3306) and DATABASE is the name of the database on the server. It is wise to test that you can connect to this url with a MySQL client prior to launching.
assignments_table_name¶
Specifies the table of the database you would like to write participant data to.
Type: | string |
---|---|
Default: | assignments |
IMPORTANT: psiTurk prevents the same worker from performing as task by checking to see if the worker appears in the current database table already. Thus, for a single experiment (or sequence of related experiments) you want to keep the assignments_table_name value the same. If you start a new design where it not longer matters that someone has done a previous version of the task, you can change the assignments_table_name value and begin sorting the data into a new table.
If multiple experiments or users share the same database then this table needs to be unique for each experiment/user.
table_name¶
Deprecated
Specifies the table of the database you would like to write participant data to.
Type: | string |
---|
For backwards compatibility, assignments_table_name is synonymous with table_name. If both are set, assignments_table_name will be preferred over table_name. Please use assignments_table_name going forward but either will work for now.
hits_table_name¶
Specifies the table of the database for storing information about current HITs.
Type: | string |
---|---|
Default: | amt_hit |
psiTurk creates several tables for bookkeeping. This one helps the dashboard and command line manage HITs. If multiple experiments or users share the same database then this table needs to be unique for each experiment/user.
campaigns_table_name¶
Specifies the table of the database for storing campaigns.
Type: | string |
---|---|
Default: | campaign |
psiTurk creates several tables for bookkeeping. This one helps the dashboard and command line manage campaigns. If multiple experiments or users share the same database then this table needs to be unique for each experiment/user.
jobs_table_name¶
Specifies the table of the database for storing job information.
Type: | string |
---|---|
Default: | apscheduler_jobs |
psiTurk creates several tables for bookkeeping. This one helps the task schedule manage automated jobs. If multiple experiments or users share the same database then this table needs to be unique for each experiment/user.
Server Parameters¶
The [Server Parameter]
section contains details about
your local web server process that you launch from the
command line.
host¶
Specifies the network address to which your server should bind (i.e., on which address it should listen).
Type: | string |
---|
There are two common values for this.
If host is set to localhost
(or synonymously 127.0.0.1
), then your
experiment will only work for testing (i.e., even if you
have an internet addressable computer, people outside
of your local machine will not be able to connect). This
is a security feature for developing and testing your
application.
If set to 0.0.0.0, then your psiturk server will be accessible to any traffic that can reach the computer on which your server is running. If your server has a public-internet interface, then participants anywhere in the world can access your study.
port¶
The port that your server will run on.
Type: | integer |
---|
If not running as root
, must be greater than 1024. Max 65535.
Typically a number greater than 5000 will work. If another process
is already using a given port you will usually get an
error message.
logfile¶
Type: | string |
---|
The location of the server log file. Error messages for the server process are not printed to the terminal or command line. To help in debugging they are stored in a log file of your choosing. This file will be located in the top-level folder of your project.
enable_dashboard¶
Whether to enable the dashbaord. If True, then the login_username
and
login_pw
must also be set.
Type: | bool |
---|---|
Default: | False |
See also
do_scheduler¶
Whether to run the task scheduler, which is viewable and configurable from the dashboard.
Type: | bool |
---|---|
Default: | False |
Tasks are loaded from the database. If True, then threads must be no greater than 1, because the task runner is not threadsafe. Will only run while the psiturk server is running.
login_username¶
Type: | string |
---|
If you want to have custom-login section of your web application (e.g., see customizing psiturk) then you can set a login and password on certain web pages urls/routes. By default if you aren’t using them, this is ignored.
login_pw¶
Type: | string |
---|
If you want to have custom-login section of your web application (e.g., see customizing psiturk) then you can set a login and password on certain web pages urls/routes. By default if you aren’t using them, this is ignored.
secret_key¶
Type: | string |
---|---|
Default: | None |
This is required by psiTurk’s underlying Flask server for any route that uses a “Session”, such as the psiTurk Dashboard.
It can be any string.
See also
threads¶
Type: | the string ‘auto’ or integer |
---|
threads controls the number of process threads the the psiturk webserver will run. This enables multiple simultanous connections from internet users. If you select auto it will set this based on the number of processor cores on your current computer.
certfile¶
Public ssl certificate for the psiturk server to use.
Type: | path |
---|
certfile should be the /path/to/your/domain/SSL.crt
If both certfile and keyfile are set and the files readable, then the psiturk gunicorn server will run with ssl. You will need to execute the psiturk with privileges sufficient to read the keyfile (typically root). If you run psiturk with sudo and if you are using a virtual environment, make sure to execute the full path to the desired psiturk instance in your environment.
If you want to do this, you are responsible for obtaining your own cert and key. It is not necessary to run the psiturk server with ssl in order to use your own ad server. You can have a proxy server such as nginx in front of psiturk/gunicorn which handles ssl connections.
However, if you configure the psiturk server to run with SSL by setting the `certfile` and `keyfile` here, you must use a proxy server in front of psiturk to serve the content in your /static folder. An SSL-enabled psiturk/gunicorn server will not serve static content – it will only serve dynamic content.
See https://docs.gunicorn.org/en/stable/deploy.html for more information on setting up proxy servers with the psiturk (gunicorn) server.
keyfile¶
Private ssl certificate for the psiturk server to use.
Type: | path |
---|
certfile should be the /path/to/your/domain/private-SSL.key. Although .crts can contain .key files within them, psiturk currently requires that you point to separate .crt and .key files for this feature to work.
See the documentation for certfile for more information.
server_timeout¶
Type: | integer |
---|---|
Default: | 30 |
Number of seconds gunicorn will wait before killing an unresponsive worker. This timeout applies to any individual request.
If you expect that your experiment may take more than 30 seconds to respond to a request, you may want to increase this.
Note
See https://docs.gunicorn.org/en/stable/settings.html#timeout for more information.
Task Parameters¶
The [Task Parameters]
section contains details about
your task.
experiment_code_version¶
Type: | string |
---|
Often you might run a couple different versions of an experiment during a research project (e.g., Experiment 1 and 2 of a paper). Or, perhaps you make modifications to a single study after having already begun data collection.
experiment_code_version is a string which is written into the database along with your data helping you remember which version of the code each participant was given.
This variable is used by the server along with num_conds and num_counters to ensure an equal number of workers per condition for the current experiment_code_version. In other words, changing the experiment_code_version resets the number of workers per condition to [0 0].
num_conds¶
Type: | integer |
---|
psiTurk includes a primitive system for counterbalancing participants to conditions. If you specify a number of condition greater than 1, then psiTurk will attempt to assign new participants to conditions to keep them all with equal N. It also takes into account the time delay between a person being assigned to a condition and completing a condition (or possibly withdrawing). Thus, you can be fairly assured that after running 100 subjects in two conditions each condition will have 50+/- completed participants.
Note
If you want to reset the random assignment when changing num_conds, update the experiment_code_version.
num_counters¶
Type: | integer |
---|
num_counters is identical to num_cond but provides an additional counterbalancing factor beyond condition. If num_counters is greater than 1 then psiTurk behaves as if there are num_cond*num_counters conditions and assigns subjects randomly to the the expanded design. See Issue #53 for more info.
contact_email_on_error¶
The email you would like to display to workers in case there is an error in the task.
Type: | string , valid email address |
---|
Workers will often try to contact you to explain what when want and request partial or full payment for their time. Providing a email address that you monitor regularly is important to being a good member of the AMT community.
cutoff_time¶
Maximum time in minutes that it should take for a participant to finish the task.
Type: | integer |
---|
Exclusively used in determining random assignment – basically, how long should a participant be given to complete the task after starting? How long should the task last? This is different than the duration specified when running hit create, because a participant may not start the task immediately after accepting it, while the hit duration starts ticking as soon as the hit is accepted (some workers queue their accepted hits before starting it).
Shell Parameters¶
The [Shell Parameters]
section contains details about
the psiturk shell.
launch_in_sandbox_mode¶
Type: | bool |
---|
If set to true, the psiturk shell will launch in sandbox mode. if set to false, the shell will launch in live mode. We recommend leaving this option to true to lessen the chance of accidentally posting a live HIT to mTurk.
bonus_message¶
Type: | string |
---|
If set, automatically uses this string as the message to participants when bonusing them for an assignment. If not set, you will be prompted to type in a message each time you bonus participants. (This message is required by AMT.)