[GSOC] Created a memory usage profile of the TARDIS example simulations

[officialasishkumar]

📝 Description

This PR is completes the second part of the objective of GSoC'24 idea "TARDIS Benchmarking and Performance Improvement". I have used memray and scalene to generate the memory usage profile. I have used the example simulation mentioned in quickstart.

Memray report:

Scalene report:

📌 Resources

https://bloomberg.github.io/memray
https://github.com/plasma-umass/scalene

🚦 Testing

How did you test these changes?

• Testing pipeline
• Other method (describe)
• My changes can't be tested (explain why)

☑️ Checklist

• I requested two reviewers for this pull request
• I updated the documentation according to my changes
• I built the documentation by applying the build_docs label

cc: @atharva-2001

Note: If you are not allowed to perform any of these actions, ping (@) a contributor.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[tardis-bot]

*beep* *bop*

Hi, human.

I'm the @tardis-bot and couldn't find your records in my database. I think we don't know each other, or you changed your credentials recently.

Please add your name and email to .mailmap in your current branch and push the changes to this pull request.

In case you need to map an existing alias, follow this example.

[wkerzendorf]

Could you write a bit about what you found? What you might want to optimize?

[officialasishkumar]

After going through the memory profiler, here are the functions/lines which are taking most memory and the possible optimizations that we can use:

1. montecarlo_main_loop: This function itself takes more than 2GB of memory (after checking on scalene). Possible optimizations which we can adopt to make it take less memory are:

• we can create reusable objects (r_packet, vpacket_collection, rpacket_tracker) outside the loop and update their properties inside the loop instead of creating new objects for each packet.
• analysing where copy is not necessary since a lot of np.copy() is happening in this function.
• maybe doing in place operations in some places
• Where appropriate, replacing loops with list comprehensions or use map/filter functions, which can be more memory-efficient.

2. tardis/montecarlo/montecarlo_numba/numba_interface.py: This line takes around 600MB because of the copy operation. We can avoid it since the value doesn't change while shifting it to contigous block in memory.

3. Being more specific with the import statements and only import packages that are necessary

Could you write a bit about what you found? What you might want to optimize?

[officialasishkumar]

Update: Added screenshots, html and replaced jupyter notebook with python code.

cc: @andrewfullard @atharva-2001 @AlexHls

[andrewfullard]

Great! Can you describe what scalene does differently to memray?

[officialasishkumar]

Great! Can you describe what scalene does differently to memray?

1. Shows the copies made in that function
2. Shows the entire memory timeline of every function which consumes majority of memory
3. Very less overhead (renders and runs very fast)