Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Finalizing the design and set up #4

Open
2 of 5 tasks
markwhiting opened this issue Oct 1, 2023 · 4 comments
Open
2 of 5 tasks

Finalizing the design and set up #4

markwhiting opened this issue Oct 1, 2023 · 4 comments

Comments

@markwhiting
Copy link
Member

markwhiting commented Oct 1, 2023

Tasks:

  • Design the work HIT (general work area for several tasks, general share area, consent form)
  • IRB
  • Design the recipient HIT (code entry box)
  • Design the back end so that it is safely triggered (e.g., triggered by events in HITs so that it doesn't run all the time)
  • Document the overall design for approval Plan for running the Altruism experiment #6
@markwhiting
Copy link
Member Author

markwhiting commented Oct 3, 2023

Launch:

  1. Clean out prior workers who are not active?
  2. Generate a large number of HITs to be assigned (uses createHITs and stores the HITs in DB)
  3. Preselect $n$ workers for "sharer" status. (stores the workers DB)
  4. Make HITs for them that are qualified for individual workers. (assignQualification, and stores their HITs in workers DB)
  5. Notify them: "There's a bunch of hits you an work on here"
  6. When they start we treat them as costless or costfull (stores treatment in workers DB and configures sharing list in DB so when they share we keep track of who used their code)

Sharer flow:

  1. See a HIT with many assignments and start working on it. The HIT is pre-qualified to them alone. The preview page of the HIT has consent language.
  2. Work on the HIT: each assignment refers to an individual instance of the many types of work we are targeting. — Identifying tasks for this experiment #2 (completing a HIT updates DB and associated share list)
  3. Share the HIT to others by copy and pasting a snippet to an external location, something like: "Join me in this great work by doing this HIT and entering the code #1235"
  4. In the costless treatment, assignments do not go down when others work. In the costfull treatment, assignments go down as others work. (sends commands to remove HITs that are completed by other workers)

Recipient participant flow:

  1. Perform a setup HIT where they enter a code. (updates share list so that worker relationships are stored)
  2. They are qualified for a set of work hits (uses assignQualification to make HITs available to them)
  3. They work on HITs (each time they submit (or anyone doing any HIT submits) a message triggers DB updates and potential assignment removal)

Shutdown:

  1. Expire all remaining HITs

Questions

  • How long does it take for something like 1000 unique HITs to be generated with layout parameters? This will help us decide if we are generating HITs on the fly or by a batch before the experiment.
  • What are the edge cases on qualifying batches of work, e.g., can qualifications differ within a set of HITs that look like a single list so that person A sees some hits and person B sees different HITs? Let's explore the edge cases so as to ensure we aren't overlooking anything.
  • What tables do we really need? workers(workerId, treatment, HITs), shares (sharerId, recipientIds)?
  • How does the work and work assignment aspect work? Probably iframe, possibly round robin assignment on work types.

@markwhiting
Copy link
Member Author

markwhiting commented Oct 10, 2023

  • Can we do the preview page with consent language and the real HIT with the work and share stuff in the model of using LayoutIDs? We might need to have access to workerID on the client side, so we should see if that works, and then put logic into the layout to deal with it.
  • Share should be done with a component that is rendered after a response from the server (or lambda), so that they see the correct treatment.
  • This is probably the same as how tasks are loaded.

@markwhiting
Copy link
Member Author

On Tuesday 17th we will try to:

  1. check how things are going with lambda design
  2. reach out to task providers to work out how to run their tasks

On Friday 20th we will try to do a run through of the system and write it up to share with Duncan and others. For this we should have the main functions working, including showing at least one task type. We can show a demo at that point.

@markwhiting
Copy link
Member Author

Lets start with a task like this survey, using their setup: https://github.com/Watts-Lab/surveys/tree/main/surveys/demographics

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant