[Documentation:Developer] update GSoC 2024 project ideas (#580)

bmcutler · web-flow · commit 0f87cdefa616 · 2024-02-05T13:56:52.000-05:00
Co-authored-by: Barb Cutler &lt;Barb Cutler&gt;
diff --git a/_docs/developer/getting_started/project_ideas.md b/_docs/developer/getting_started/project_ideas.md
@@ -47,127 +47,260 @@ developers.
    new graders and some of our TA grading features are not adequately
    tested by automated unit and end-to-end (Cypress) regression testing.
 
-   [Open Issues related to TA Grading](https://github.com/Submitty/Submitty/issues?q=is%3Aopen+is%3Aissue+label%3A%22TA+Grading+%2F+TA+UI%22)
-
-   _Expected Outcomes_: The goals of a project on TA grading would
-   include expantion of our automated testing of the TA Grading pages,
-   and to patch bugs uncovered by this improved testing.  The project may be
-   expanded in scope to additionally propose and execute small or modest user
-   interface revisions that enhance the TA experience.
+   [Open Issues related to TA Grading](https://github.com/Submitty/Submitty/issues?q=is%3Aopen+is%3Aissue+label%3A%22TA+Grading+%2F+TA+UI%22)  
+   [Open Issues related to Sample Data](https://github.com/Submitty/Submitty/issues?q=is%3Aopen+is%3Aissue+label%3A%22Sample+Data%22)
+
+   _Expected Outcomes_: The primary goals for this project include the
+   expansion of our automated testing of the TA Grading pages and to
+   patch bugs uncovered by this improved testing.  The project may be
+   expanded in scope to additionally propose and execute small or
+   modest user interface revisions that enhance the TA experience,
+   especially for graders who are new to the interface and grading
+   process.
    
    _Skills & Experience Required_: Some programming experience,
-   willingness to learn web and database development and
-   the Cypress end-to-end automated testing framework.  Having served
-   as a teaching assistant or instructor with grading experience
-   design will be beneficial.
+   willingness to learn web and database development and the Cypress
+   end-to-end automated testing framework.  Having served as a
+   teaching assistant with grading experience design will be
+   beneficial.
 
-   _Possible Mentors_: Barb Cutler, Nia Heermance
+   _Possible Mentors_: William Allen, Nia Heermance, Barb Cutler
 
    _GSoC Project Size_: 90 or 175 hours
 
-   _Difficulty Level_: medium
+   _Difficulty Level_: introductory to medium
 
 
 2. **Refactor and Performance Improvements for the Manual/TA Rubric Grading Interface**
 
    [Overview of Rubric Grading Interface](/grader/rubric_grading/index)
 
-   Note: This project may be combined with the previous project idea.
-   The Manual/TA rubric grading interface is elaborate, highly-featured, and
-   customizable; however, the performance of these webpages is
-   problematic for large courses due to inefficient database queries and
-   delays to load data from the server that could/should be asynchronous.
-   The manual/TA rubric pages could benefit from a significant
-   technology refactor to use Vue/Vite, for example.
-
-   [Open Issues related to TA Grading](https://github.com/Submitty/Submitty/issues?q=is%3Aopen+is%3Aissue+label%3A%22TA+Grading+%2F+TA+UI%22)
+   _Note: This project may be combined with the previous project idea._
 
-   _Expected Outcomes_:  A detailed software design for an organized, multi-stage
-   refactor of the manual/TA rubric pages and the execution/implementation of
-   a significant portion of the new design.  The project will likely require the
-   extension and/or updating of our automated end-to-end (Cypress) testing and patching
-   bugs uncovered by this testing.  The general interface for TA/Manual grading
-   should remain similar, but the project may include small user
-   interface revisions.
+   The Manual/TA rubric grading interface is elaborate,
+   highly-featured, and customizable; however, the performance of
+   these webpages is problematic for large courses due to inefficient
+   database queries and server communication delays to load data that
+   could/should be asynchronous.  The manual/TA rubric pages could
+   benefit from a significant technology refactor to use Vue/Vite, for
+   example.
 
-   _Skills & Experience Required_:  Web and database development experience.
-   Experience with end-to-end automated testing (Cypress) and and having served
-   as a teaching assistant or instructor with grading experience
-   design is beneficial but not required.
+   [Open Issues related to TA Grading](https://github.com/Submitty/Submitty/issues?q=is%3Aopen+is%3Aissue+label%3A%22TA+Grading+%2F+TA+UI%22)
 
-   _Possible Mentors_: Barb Cutler, Nia Heermance
+   _Expected Outcomes_: This project would first prepare a detailed
+   software design plan for an organized, multi-stage incremental
+   refactor of the manual/TA rubric pages and follow with the
+   execution/implementation of a significant portion of the new
+   design.  The project could include the extension and/or updating of
+   our automated end-to-end (Cypress) testing and patching bugs
+   uncovered by this testing (as described in the previous project
+   idea).  The project should include benchmarking along the way to
+   ensure that the refactor is improving the performance of the
+   Manual/TA Rubric Grading interface. The general interface for
+   TA/Manual grading should remain similar, but the project may
+   include small user interface revisions.
+
+   _Skills & Experience Required_: Web and database development
+   experience and general software design and implementation
+   experience.  Experience with end-to-end automated testing (Cypress)
+   and and having served as a teaching assistant with grading
+   experience design is beneficial but not required.
+
+   _Possible Mentors_: Nia Heermance, William Allen, Barb Cutler
 
    _GSoC Project Size_: 175 or 350 hours
 
-   _Difficulty Level_: medium or hard
-
+   _Difficulty Level_: medium to challenging
+
+
+
+3. **Notebook Builder: UI To Streamline Instructor Configuration of Automated Grading**
+
+   Our system for automated testing and grading of student work is
+   very powerful, but the configuration process that instructors must
+   navigate is complex and time-consuming.  While we provide a number
+   of examples, the number of choices for development of an
+   autograding configuration is overwhelming.  The primary method for
+   creating an autograding configuration is to prepare a `config.json`
+   file (and any necessary additional files) and upload or store these
+   files on the server file system.  We have a prototype Web GUI
+   interface we call the "Notebook Builder" but the current state of
+   the feature is undocumented and functionality is limited.  We
+   would like to improve and expand this feature to facilitate
+   instructor creation of basic and moderate complexity autograding
+   configurations.
+
+   [Assignment Autograding Configuration Instructions](/instructor/autograding/specification)  
+   [Notebook Assignment Configuration](/instructor/assignment_configuration/notebook)  
+   [Tutorial Autograding Configuration Examples](https://github.com/Submitty/Tutorial/tree/main/examples)
+
+   This project will involve multiple modules of Submitty including
+   web UI development, integration, documentation, additional tutorial
+   examples, and extending output generation to instructor solutions
+   in compiled languages.
+
+   [Open Issues related to Autograding](https://github.com/Submitty/Submitty/labels/Autograding)  
+   [Open Issues related to Notebook / Notebook Builder](https://github.com/Submitty/Submitty/issues?q=is%3Aopen+is%3Aissue+label%3A%22Notebook+%2F+Notebook+Builder%22)
+
+   _Expected Outcomes_: The primary focus of the project is the
+   revision and expansion of the Notebook Builder UI to increase the
+   number of autograding features that are supported.  The UI should
+   be easy-to-use for instructors of
+   non-computer-science/non-programming courses and also instructors of courses with
+   introductory to moderate programming assignments.  The size and
+   scope for a proposal in this area is flexible, depending on the
+   time commitment and prior skills of the applicant.
+      
+   _Skills & Experience Required_: Some programming experience,
+   willingness to learn web and database development.  Prior
+   experience with user interface design and an eye for quality user
+   design are beneficial.  Having served as a teaching assistant or
+   instructor with experience in programming assignment design will be
+   beneficial but not required.
 
+   _Possible Mentors_: Barb Cutler, Chris Reed
 
-3. **Streamline instructor configuration of automated grading**
+   _GSoC Project Size_: 90 or 175 or 350 hours
 
-   Currently, instructors must write a configuration as a `config.json`
-   (and any necessary additional files) and upload or store these
-   files on the local file system.  We would like to provide an
-   alternate web GUI interface for creating basic or moderately
-   complex autograding configurations.
+   _Difficulty Level_: introductory or medium
+   
 
-   [Assignment Configuration Instructions](/instructor/autograding/specification)
+4. **Expansion of Examples and Documentation of Intermediate and Advanced Autograding Features**
 
-   We have preliminary support for automated creation of expected
-   output files (from and instructor solution -- currently limited to
-   Python) and randomized test case input.  This project will involve multiple modules of
-   Submitty including
-   web UI development, integration, documentation, additional tutorial examples, and
-   extending output generation to
-   instructor solutions in compiled languages.
+   _Note: This project is related to previous project idea but is a distinct project._
 
-   [Open Issues related to Autograding](https://github.com/Submitty/Submitty/labels/Autograding)
+   Our system for automated testing and automated grading of student
+   work is very powerful and highly-customizable, but the
+   documentation for our moderate and advanced autograding features is
+   incomplete.  While we provide a number of autograding examples,
+   some of the examples are out-of-date and do not represent our
+   current suggested best practices.
 
-   _Expected Outcomes_: The goal would be to
-   streamline the assignment configuration process for non-technical
-   instructors, relevant for use in
-   non-computer-science/non-programming courses.
+   [Assignment Autograding Configuration Instructions](/instructor/autograding/specification)  
+   [Submitty Autograding Tutorial Examples](https://github.com/Submitty/Tutorial)  
+   [Additional Autograding Examples](https://github.com/Submitty/Submitty/tree/master/more_autograding_examples)  
+   [Prior GSoC Project: Drumil Patel](/developer/google_summer_of_code/2019_DrumilPatel)  
    
-   _Skills & Experience Required_: Some programming experience, willingness to
-   learn web and database development.  Having served as a teaching
-   assistant or instructor with experience in programming assignment
-   design will be beneficial.
+   We would like to reduce the learning curve for new instructors and
+   provide more tutorial examples of autograding for instructors
+   teaching courses of any level.  Automated testing and automated
+   grading can be used in introductory programming courses in middle
+   and high schools, including AP Computer Science.  It can also be
+   used by programming-intensive intermediate and upper level / senior
+   university-level systems coursework.  Assignments that require can
+   be configured with custom Docker Images to provide access to
+   specific programming languages and libraries. 
+
+   [Open Issues related to Docker Image Autograding](https://github.com/Submitty/Submitty/issues?q=label%3A%22Docker+Container+Autograding%22+)  
+   [Docker Images for Autograding Common Programming Languages](https://github.com/Submitty/DockerImages/tree/main)  
+   [Example Custom Docker Images University ](https://github.com/Submitty/DockerImagesRPI/tree/main/dockerfiles)  
+   [Sample Java Assignments](/instructor/autograding/sample_assignments)
 
-   _Possible Mentors_: Barb Cutler, William Allen
+   _Expected Outcomes_: The project should begin with a review and
+   organization of existing sample and tutorial assignments and
+   current autograding functionality documentation.  Out-of-date or
+   underdeveloped autograding configuration examples should be
+   expanded as necessary, and features that are missing documentation
+   and examples should be identified (e.g., generated random input and
+   output from instructor solution, customized docker containers,
+   autograding graphical output, autograding) and resolved by creating
+   new examples.  Finally, we would like to create and support a
+   resource for the community of crowd-sourced complete programming
+   assignments/exercises with included autograding configuration.
+
+   _Skills & Experience Required_: Moderate to advanced programming
+   experience, willingness to learn web and database development.
+   Having served as a teaching assistant or instructor with experience
+   in programming assignment design will be beneficial.
+
+   _Possible Mentors_: Chris Reed, Barb Cutler
 
    _GSoC Project Size_: 175 or 350 hours
 
-   _Difficulty Level_: medium or hard
-   
-
-4. **Instructional Materials and Documentation**
+   _Difficulty Level_: medium to challenging
 
-   We would like to reduce the learning curve for new instructors and
-   provide more starter material for instructors teaching introductory
-   programming courses in middle and high schools, including AP
-   Computer Science.
 
-   [Submitty Autograding Tutorial Examples](https://github.com/Submitty/Tutorial)
+5. **AI/ML to Enhance and Steamline Manual / TA Grading**
 
-   [Additional Autograding Examples](https://github.com/Submitty/Submitty/tree/master/more_autograding_examples)
+   The use of a unified and retroactively editable rubric for
+   manual/TA grading can ensure consistency when grading large
+   courses, especially when more than one grader is working on a
+   single problem or assignment.  However, it is usually still
+   necessary for the graders to inspect the student work one-at-a-time
+   and it can be difficult for the grader to remember the details of
+   all previously graded assignments and recognize patterns that
+   should be graded similarly.  Furthermore, the process of manual
+   grading is time-consuming and thus detailed and thoughtful
+   constructive feedback to each individual is often not possible.
 
-   [Sample Java Assignments](/instructor/autograding/sample_assignments)
+   [Overview of Rubric Grading Interface](/grader/rubric_grading/index)
 
-   _Expected Outcomes_: Organization of existing sample and tutorial assignments and autograding
-   and current documentation.  Review and curation of any publicly-available sample assignments
-   and creation of new sample assignments and autograding.
+   The goal of this project is to explore the potential to leverage
+   Artificial Intelligence and Machine Learning (AI/ML) to reduce the burden
+   of manual grading in large courses with either programming or
+   non-programming assignments.  Automatically organizing student
+   submissions into groups that contain similar patterns and have
+   common strengths or flaws can ensure that student work is assessed
+   consistently and students receive appropriate and in-depth
+   feedback to aid their learning.
+
+   The Submitty project includes related technology for the static
+   analysis of student's code and tools to screen for plagiarism in
+   both student-submitted plain text assignments and software.  Note
+   that the Submitty static analysis and plagiarism tools have been
+   tested and validated with datasets of sample student submissions;
+   however, for privacy and confidentiality reasons, these datasets are
+   not and cannot be part of the Submitty open-source materials.
+
+   [Autograding using Static Analysis](/instructor/autograding/static_analysis/index)  
+   [Plagiarism Detection](/instructor/course_management/plagiarism)
+
+   _Expected Outcomes_: Detailed design plan for the integration of an
+   AI/ML framework into Submitty for the analysis and clustering the
+   assignment submissions by students based on common patterns in the
+   text and/or code.  Implementation of a prototype AI/ML tool to
+   detect common patterns in student submissions and present this
+   information to the grader in an organized way to allow streamlined
+   bulk grading and feedback within the manual/TA grading interface.
+   As time permits (and based on the scope and time commitment)
+   evaluation of the effectiveness of this technique on real world
+   data and the potential for improving the efficiency of the
+   manual/TA grading process and the quality/accuracy and quantity of
+   useful constructive feedback to students.
   
-   _Skills & Experience Required_: Some programming experience, willingness to
-   learn web and database development.  Having served as a teaching
-   assistant or instructor with experience in programming assignment
-   design will be beneficial.
+   _Skills & Experience Required_: Coursework and/or professional
+   experience in AI/ML and modern AI/ML technology.  Moderate to
+   advanced programming experience, and willingness to learn web and
+   database development.  Having served as a teaching assistant for a
+   large course with manual grading experience design will be
+   beneficial.
+   
+   _Possible Mentors_: Barb Cutler, William Allen, Nia Heermance
 
-   _Possible Mentors_: Barb Cutler, William Allen
+   _GSoC Project Size_: 175 or 350 hours
 
-   _GSoC Project Size_: 175 hours
+   _Difficulty Level_:  medium to challenging
 
-   _Difficulty Level_: easy or medium
+6. **Other Topics**
 
+   The Submitty team welcomes GSoC project proposals on other topics
+   related to items in our GitHub issue tracker.  A successful
+   application would select one or more issues of moderate scope
+   proportional to the applicant's time commitment and prior
+   experience.  Be sure to join our [Zulip server](/contact) to meet
+   the Submitty mentors and other new developers and discuss your
+   interests and project plans.
+
+   _Skills & Experience Required_: Some prior programming experience,
+   willingness to learn web and database development, and additional
+   specific skills as appropriate.
+   
+   _Possible Mentors_: Barb Cutler, Matthew Peveler, Shail Patel,  William Allen, Chris Reed, Nia Heermance, Preston Carman
+
+   _GSoC Project Size_: 90 or 175 or 350 hours
+
+   _Difficulty Level_:  introductory to medium to challenging
+   
 
 See also:
 
