-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
True epoch stats #248
True epoch stats #248
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Long overdo feature! Small changes requested for clarity
ml4cvd/tensor_generators.py
Outdated
sum_squared = stats[k] | ||
base_key = k.replace('sum_squared', '') | ||
n = stats[f'{base_key}n'] | ||
n_sum = stats[f'{base_key}sum'] | ||
mean = n_sum/(eps+n) | ||
logging.info(f'Continuous value \n{base_key} Mean:{mean:0.2f} Standard Deviation:{(sum_squared/n)-(mean*mean):0.2f} ' | ||
f"Maximum:{stats[f'{base_key}max']:0.2f} Minimum{stats[f'{base_key}min']:0.2f}") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a little hard to follow. Maybe add some comments, or instead of iterating through stats, iterate through the continuous and categorical tmaps and create the keys from the tmap names.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! Maybe we can use the self.stats_q.qsize()
check to make inference multiprocessed in the future.
Addresses #216 thanks @ndiamant for guidance