add pyright and e2e tests for CI/CD

[C-Loftus]

• Add pyright for type checking
  • Dagster is complicated so if we can preemptively prevent runtime errors by making sure the types are right in CI, I think that is worthwhile
• Begin to add pytest tests for materializing assets
  • Only tests materializing the configs since it is complicated and unclear how to test dynamically partitioned assets
  • But it is a good start, is a sanity check that nothing major broke, and all the code is loaded in without errors before materializing
  • Can run these with python main.py test
• Renamed code/ to userCode/ since there is already a code module that interfered with ours

[C-Loftus]

@webb-ben In addition to the pyright / e2e changes I also just fixed the concurrency issue. The issue was that when a parallel job tries to recreate the config, it will fail if it is in use in another container. So I just catch the exception and ignore it if that is the case. Its simple and not super elegant but definitely the easiest heuristic since we can assume we don't want to be regenerating configs in the middle of other crawls. Works fine for me with 3 parallel jobs.

Won't touch this branch any more for time being until approval. Should fix your issues.