Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[do not merge] Large scale #175

Closed
wants to merge 120 commits into from
Closed

[do not merge] Large scale #175

wants to merge 120 commits into from

Conversation

ethanglaser
Copy link
Contributor

Description

Add a comprehensive description of proposed changes

List associated issue number(s) if exist(s): #6 (for example)

Documentation PR (if needed): #1340 (for example)

PR should start as a draft, then move to ready for review state after CI is passed and all applicable checkboxes are closed.
This approach ensures that reviewers don't spend extra time asking for regular requirements.

You can remove a checkbox as not applicable only if it doesn't relate to this PR in any way.

Checklist to comply with before moving PR from draft:

PR completeness and readability

  • I have reviewed my changes thoroughly before submitting this pull request.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have updated the documentation to reflect the changes or created a separate PR with update and provided its number in the description, if necessary.
  • Git commit message contains an appropriate signed-off-by string (see CONTRIBUTING.md for details).
  • I have added a respective label(s) to PR if I have a permission for that.
  • I have resolved any merge conflicts that might occur with the base branch.

Testing

  • I have run it locally and tested the changes extensively.
  • All CI jobs are green or I have provided justification why they aren't.
  • I have extended testing suite if new functionality was introduced in this PR.

ethanglaser and others added 30 commits August 30, 2024 23:27
@ethanglaser
Creating branch for large scale measurements
b3500dd
@ethanglaser
strong scaling, config updates, minor revisions
4bd6c7f
@ethanglaser
Merge branch 'IntelPython:large-scale' into large-scale
984aab1
@ethanglaser
knn and forest config updates
3cd955c
@md-shafiul-alam
lint
e39dc2b
@md-shafiul-alam
just gpu for regular
6e0fbf8
@md-shafiul-alam
juremove cuml
7bb8fb4
@olegkkruglov
Add incremental algorithms support
535c1e4
@olegkkruglov
Fix win yml
d6952ac
@md-shafiul-alam
refactor and kmeans strong
9cf382e
@md-shafiul-alam
refactor and add config
6c8f529
@md-shafiul-alam
strong reduce nodes
3867a86
@md-shafiul-alam
forest reg config
ed875b4
@md-shafiul-alam
forest reg config
c596a56
@md-shafiul-alam
KNN weak
4fee991
@md-shafiul-alam
KNN strong
fce0651
@md-shafiul-alam
experiment with ppn
e1ff9a0
@md-shafiul-alam
experiment with ppn
e3d9a35
@md-shafiul-alam
bf16
817710b
@md-shafiul-alam
bf16
aaa0039
@olegkkruglov
Remove samples/ms info
03a152a
@md-shafiul-alam
knn
b7d962e
@olegkkruglov
Remove BS from config (need to add after pip version update)
3ac5c23
@md-shafiul-alam
basic stat single
87b6fa6
@olegkkruglov
Add condition for finalize
9461fad
@olegkkruglov
Fix num_batches usage
b82d772
@ethanglaser @olegkkruglov
Creating branch for large scale measurements
c70e122
@ethanglaser @olegkkruglov
strong scaling, config updates, minor revisions
8d74f6d
@ethanglaser @olegkkruglov
knn and forest config updates
192744f
@md-shafiul-alam @olegkkruglov
lint
b1f2c15
@ethanglaser ethanglaser changed the title Large scale [do not merge] Large scale Mar 18, 2025
ethanglaser and others added 28 commits March 18, 2025 16:39
@ethanglaser
Merge branch 'main' into large-scale
72d65c1
@KateBlueSky
Added the math import.
af48e96
@KateBlueSky
Rolled back the accidental changes to the ranked_based distributed_sp…
c7f38f4 
…lit.
@KateBlueSky
Updated large scale 2k parameters for the full 24576 tiles.
264701e
@KateBlueSky
Updated config files.
20419a9
@KateBlueSky
cleaned up diff.
4e93858
@KateBlueSky
Reformatted correctly.
428f3df
@KateBlueSky
Fixed if else.
2f8c68b
@KateBlueSky
Updated format.
816c6dc
@KateBlueSky
Added mpi4py
5d3bf52
@KateBlueSky
Merge branch 'large-scale' of https://github.com/IntelPython/scikit-l…
edceece 
…earn_bench into dev/large_scale_kmeans
@KateBlueSky
fixed mpi4py
a937963
@KateBlueSky
Rolled back mpi4py.
3809d17
@KateBlueSky
Formatted file.
c128748
@KateBlueSky
Removed environment from diff.
15db792
@ethanglaser
initial alignment of configs to final results (#176)
30b0b80 
* initial alignment of configs to final results

* logic updates

* fix large scale CI

* black

* minimize incremental

* black

* align custom function with skl estimator

* Update sklbench/utils/measurement.py

* Update sklbench/utils/measurement.py

* Update sklbench/utils/measurement.py

* Update sklbench/utils/measurement.py
@KateBlueSky
Revert "Removed environment from diff."
f0fccdd 
This reverts commit 15db792.
@KateBlueSky
Merged change from large-scale
548d824
@KateBlueSky
Removed extra code for sample_shift.
4d675ec
@KateBlueSky
Changes for sample_shift.
e8fbd0b
@KateBlueSky
Updated sample shift.
a7cea17
@KateBlueSky
Updated sample shift.
f3c2757
@KateBlueSky
Removed extra code.
2ae3c39
@KateBlueSky
Added comment for sample_shift.
39cc4f2
@KateBlueSky
Added back in x_train in sample_shift.
3fc7c42
@KateBlueSky
Updated description of sample_shift.
1bd5aa1
@KateBlueSky
Added predict back in.
06944c1
@KateBlueSky
Merge pull request #174 from IntelPython/dev/large_scale_kmeans
2edb597 
[Merge only onto large-scale] Large scale Kmeans changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Alexsandruss Alexsandruss Awaiting requested review from Alexsandruss Alexsandruss will be requested when the pull request is marked ready for review Alexsandruss is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@ethanglaser @md-shafiul-alam @olegkkruglov @icfaust @KateBlueSky