Skip to content
Open
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
15b5dce
Create url-lanthanidesV-VII.txt
ccollins22 Jul 4, 2024
547d81f
Read mons lanthanide data
ccollins22 Jul 19, 2024
99df426
Add condition for None parity (forbidden False)
ccollins22 Jul 19, 2024
bea1556
Add handler for MONS data
ccollins22 Jul 19, 2024
23d076c
Read all files and tidy code
ccollins22 Jul 19, 2024
63b88c0
Add comment to URL file
ccollins22 Jul 19, 2024
f70235d
Update assert for rounding issue (NIST ionisation energy only to 1dp)
ccollins22 Jul 19, 2024
c417d70
Update log message
ccollins22 Jul 19, 2024
e45872c
Update forbidden check
ccollins22 Jul 19, 2024
c9afac4
Fix lowest level
ccollins22 Jul 19, 2024
a979411
Use ionization energy from MONS data but check to make sure in agreem…
ccollins22 Jul 19, 2024
98d7381
Remove commented out code
ccollins22 Jul 19, 2024
f18892f
Get ion_handlers for all mons data
ccollins22 Jul 19, 2024
da539df
Use NIST ionisation potentials where MONS data does not match
ccollins22 Jul 19, 2024
a81fa62
Update readmonsdata.py
ccollins22 Jul 19, 2024
c02adb0
Update __init__.py
ccollins22 Jul 19, 2024
7fafef7
Update readmonsdata.py
ccollins22 Jul 22, 2024
af7a971
Update readmonsdata.py
ccollins22 Jul 22, 2024
e0c3a59
Update readmonsdata.py
ccollins22 Jul 22, 2024
113ae6d
Update readmonsdata.py
ccollins22 Jul 22, 2024
28919d2
Update readmonsdata.py
ccollins22 Jul 22, 2024
34c8da4
Update readmonsdata.py
ccollins22 Jul 22, 2024
376ab51
Fix level indicies, A values, and transition counts
lukeshingles Jul 22, 2024
5187e3c
Remove debug print
lukeshingles Jul 22, 2024
ea88c3c
Simplify code
lukeshingles Jul 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions artisatomic/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
from artisatomic import readnahardata
from artisatomic import readqubdata
from artisatomic import readtanakajpltdata
from artisatomic import readmonsdata
from artisatomic import groundstatesonlynist
from artisatomic.manual_matches import hillier_name_replacements
from artisatomic.manual_matches import nahar_configuration_replacements
Expand Down Expand Up @@ -88,6 +89,7 @@ def get_ion_handlers() -> list[tuple[int, list[int | tuple[int, str]]]]:
# (39, [(1, "carsus"), (2, "carsus")]),
# (40, [(1, "carsus"), (2, "carsus"), (3, "carsus")]),
# (70, [(5, "gsnist")]),
# (57, [(5, "mons")]),
# (92, [(2, "fac"), (3, "fac")]),
# (94, [(2, "fac"), (3, "fac")]),
# ]
Expand All @@ -98,6 +100,7 @@ def get_ion_handlers() -> list[tuple[int, list[int | tuple[int, str]]]]:
# ion_handlers = readdreamdata.extend_ion_list(ion_handlers)
# ion_handlers = readfacdata.extend_ion_list(ion_handlers)
# ion_handlers = readtanakajpltdata.extend_ion_list(ion_handlers)
# ion_handlers = readmonsdata.extend_ion_list(ion_handlers)
# ion_handlers = groundstatesonlynist.extend_ion_list(ion_handlers)

return ion_handlers
Expand Down Expand Up @@ -487,6 +490,14 @@ def process_files(ion_handlers: list[tuple[int, list[int | tuple[int, str]]]], a
transition_count_of_level_name[i],
) = readtanakajpltdata.read_levels_and_transitions(atomic_number, ion_stage, flog)

elif handler == "mons": # Lanthanide data ions V-VII from MONS
(
ionization_energy_ev[i],
energy_levels[i],
transitions[i],
transition_count_of_level_name[i],
) = readmonsdata.read_levels_and_transitions(atomic_number, ion_stage, flog)

elif handler == "gsnist": # ground states taken from NIST
(
ionization_energy_ev[i],
Expand Down Expand Up @@ -1012,6 +1023,8 @@ def integrand(nu):


def check_forbidden(levela, levelb) -> bool:
if levela is None or levela.parity is None or levelb.parity is None:
return False
Comment on lines +1026 to +1027
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When would levela be None?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because of energy_levels = [None]
Which I copied from your code to read Tanaka's data. Without initialising the array with None the lowest level doesn't get written out. With it levela doesn't have attribute parity and this fails

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, that's a relic of our 1-based indexing that we inherited from the existing artis file formats. However, if any of your transitions are referencing the dummy level in the 0 index, that suggests that there is an off-by-one error in the indexing of transitions.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the transitions array have a dummy value initialised too then?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we don't really keep track of a transition index, and all transitions get written out in write_transition_data(). write_adata() skips the 0-index when writing out the list of levels for each ion.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's important to manually verify that the first couple of levels and transitions are recorded correctly in the artis output files, because indexing errors are very easy to make.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One file with the transitions is ~30MB

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just take a sample of a few hundred transitions and compress it with zstd.

return levela.parity == levelb.parity


Expand Down Expand Up @@ -1545,7 +1558,7 @@ def write_transition_data(
for transition in transitions:
levelid_lower = transition.lowerlevel
levelid_upper = transition.upperlevel
forbidden = energy_levels[levelid_lower].parity == energy_levels[levelid_upper].parity
forbidden = check_forbidden(energy_levels[levelid_lower], energy_levels[levelid_upper])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is better as far as not repeating the forbidden conditions in several places, but I'm not sure we want to pay the cost of two more Python function calls on each one of ~50 million transition. Did this affect performance in any significant way?

I think a better way (but can be left for a separate PR) would be to use a forbidden attribute on each transition.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't notice a big slow down but I didn't really check. Feel free to update


if not forbidden:
level_ids_with_permitted_down_transitions.add(levelid_upper)
Expand All @@ -1559,7 +1572,7 @@ def write_transition_data(
if coll_str > 0:
num_collision_strengths_applied += 1

forbidden = energy_levels[levelid_lower].parity == energy_levels[levelid_upper].parity
forbidden = check_forbidden(energy_levels[levelid_lower], energy_levels[levelid_upper])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above.


if forbidden:
num_forbidden_transitions += 1
Expand Down
198 changes: 198 additions & 0 deletions artisatomic/readmonsdata.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
import os.path
from pathlib import Path
import typing as t
import pandas as pd
from astropy import constants as const
from astropy import units as u
Comment on lines +9 to +10
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding another dependency (astropy) to the package is not really necessary. hc_in_ev_cm and hc_in_ev_angstrom can be set with numeric literals.

from collections import defaultdict
import numpy as np
import zipfile

import artisatomic

hc_in_ev_cm = (const.h * const.c).to("eV cm").value
hc_in_ev_angstrom = (const.h * const.c).to("eV angstrom").value


class EnergyLevel(t.NamedTuple):
levelname: str
energyabovegsinpercm: float
g: float
parity: float


class TransitionTuple(t.NamedTuple):
lowerlevel: int
upperlevel: int
A: float
coll_str: float


datafilepath = Path(os.path.dirname(os.path.abspath(__file__)), "..", "atomic-data-mons")

# outggf_Ln_V-VII.zip folder:
# 45 files outggf for each lanthanide between the V and VII spectra:
# first column is wavelength of the E1 transition (A),
# second column is the lower energy level of the transition (1000 cm^-1)
# third column is the oscillator strength
#
# outglv_Ln_V--VII.zip folder:
# 45 files outglv for each lanthanide between the V and VII spectra:
# first column is the energy of levels (1000 cm^-1)
# second column is the total angular momentum (J-value)


def extend_ion_list(ion_handlers):
# Data files contain La-Lu V-VII ions
Z_indatafile = list(range(57, 72))
ions_indatafile = [5, 6, 7]

for Z in Z_indatafile:
for ion in ions_indatafile:
atomic_number = Z
ion_stage = ion
found_element = False
for tmp_atomic_number, list_ions_handlers in ion_handlers:
if tmp_atomic_number == atomic_number:
# add an ion that is not present in the element's list
if ion_stage not in [x[0] if hasattr(x, "__getitem__") else x for x in list_ions_handlers]:
list_ions_handlers.append((ion_stage, "mons"))
list_ions_handlers.sort(key=lambda x: x[0] if hasattr(x, "__getitem__") else x)
found_element = True

if not found_element:
ion_handlers.append(
(
atomic_number,
[(ion_stage, "mons")],
)
)
ion_handlers.sort(key=lambda x: x[0])
return ion_handlers


def read_levels_and_transitions(atomic_number, ion_stage, flog):
# Read first file
ziparchive_outglv = zipfile.ZipFile(datafilepath / "outglv_Ln_V--VII.zip", "r")
datafilename_energylevels = (
f"outglv_Ln_V--VII/outglv_0_{artisatomic.elsymbols[atomic_number]}_{artisatomic.roman_numerals[ion_stage]}"
)

with ziparchive_outglv.open(datafilename_energylevels) as datafile_energylevels:
energy_levels1000percm, j_arr = np.loadtxt(datafile_energylevels, unpack=True, delimiter=",")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't looked at the file content directly, but are you sure you can't use polars.read_csv for these files?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably could. Feel free to change.

artisatomic.log_and_print(flog, f"levels: {len(energy_levels1000percm)}")

energiesabovegsinpercm = energy_levels1000percm * 1000

g_arr = 2 * j_arr + 1

# Sort table by energy levels
dfenergylevels = pd.DataFrame.from_dict({"energiesabovegsinpercm": energiesabovegsinpercm, "g": g_arr})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a blocker, but I would strongly prefer to avoid using pandas in any new code (so that eventually we can remove the import and save ~500ms of startup time). Polars should be a drop-in replacement here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to update - I'm not used to polars yet

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest getting the test in place first, and then these style fixes can be implemented safely with a big warning if anything breaks.

dfenergylevels = dfenergylevels.sort_values("energiesabovegsinpercm")

energiesabovegsinpercm = dfenergylevels["energiesabovegsinpercm"].values
g_arr = dfenergylevels["g"].values

parity = None # Only E1 so always allowed transitions.
energy_levels = [None]

for levelindex, (g, energyabovegsinpercm) in enumerate(zip(g_arr, energiesabovegsinpercm, strict=False)):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be strict=True?

energy_levels.append(
EnergyLevel(levelname=str(levelindex), parity=parity, g=g, energyabovegsinpercm=energyabovegsinpercm)
)

# Read next file
ziparchive_outggf = zipfile.ZipFile(datafilepath / "outggf_Ln_V--VII.zip", "r")
datafilename_transitions = (
f"outggf_Ln_V--VII/outggf_sorted_{artisatomic.elsymbols[atomic_number]}_{artisatomic.roman_numerals[ion_stage]}"
)

with ziparchive_outggf.open(datafilename_transitions) as datafile_transitions:
transition_wavelength_A, energy_levels_lower_1000percm, oscillator_strength = np.loadtxt(
datafile_transitions, unpack=True, delimiter=","
)
artisatomic.log_and_print(flog, f"transitions: {len(energy_levels_lower_1000percm)}")

energy_levels_lower_percm = energy_levels_lower_1000percm * 1000

# Get index of lower level of transition
lowerlevel = np.array(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure you need to cast to a numpy array here?

[
(
np.abs(
energiesabovegsinpercm - energylevellower
) # get closest energy in energy level array to lower level
).argmin() # then get the index with argmin()
for energylevellower in energy_levels_lower_percm
]
)

ionization_energy_in_ev_nist = artisatomic.get_nist_ionization_energies_ev()[(atomic_number, ion_stage)]

# get energy of upper level of transition
energy_levels_lower_ev = energy_levels_lower_percm * hc_in_ev_cm
transitionenergyev = hc_in_ev_angstrom / transition_wavelength_A
ionization_energy_in_ev = max(transitionenergyev)
artisatomic.log_and_print(
flog, f"ionization energy: {ionization_energy_in_ev} eV (NIST: {ionization_energy_in_ev_nist} eV)"
)

# If ionisation potential in data does not match NIST to within 1 decimal place
# then use NIST instead (probably more accurate?)
if round(ionization_energy_in_ev, 1) != round(ionization_energy_in_ev_nist, 1):
ionization_energy_in_ev = ionization_energy_in_ev_nist
artisatomic.log_and_print(
flog, f"Energies do not match -- using NIST value of {ionization_energy_in_ev_nist} eV"
)

energy_levels_upper_ev = transitionenergyev + energy_levels_lower_ev
energy_levels_upper_percm = energy_levels_upper_ev / hc_in_ev_cm

# Get index of upper level of transition
upperlevel = np.array(
[
(
np.abs(energiesabovegsinpercm - energylevelupper) # get closest energy in energy level array
).argmin() # then get the index with argmin()
for energylevelupper in energy_levels_upper_percm
]
)

# Get A value from oscillator strength
A_ul = np.array(
[
(
(8 * np.pi**2 * const.e.value**2)
/ (
const.m_e.value * const.c.value * (transition_wavelength_A[transitionnumber] / 1e10) ** 2
) # convert wavelength from angstrom to m
* (g_arr[lower] / g_arr[upper])
* oscillator_strength[transitionnumber]
)
for transitionnumber, (lower, upper) in enumerate(zip(lowerlevel, upperlevel, strict=False))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why use an iterator for lower and upper but then access transition_wavelength_A and oscillator_strength by index? Wouldn't it be better to zip all of these arrays? Shouldn't strict be True here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't add strict - I've no idea what it does. I think it was the automated checks?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't add strict - I've no idea what it does. I think it was the automated checks?

Ruff modified it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, that's a bit of an annoying autofix. Strict=False will stop iterating at the shortest of any arrays, while strict=True will raise an exception if the lengths don't match. I think we almost always want strict=True.

]
)

transitions = [
TransitionTuple(
lowerlevel=lowerlevel[transitionnumber],
upperlevel=upperlevel[transitionnumber],
A=A_ul[transitionnumber],
coll_str=-1,
)
for transitionnumber, _ in enumerate(lowerlevel)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you're not using the lowerlevel value from enumerate, why use enumerate? Better to zip lowerlevel, upperlevel, A_ul with strict=True to make sure they match lengths.

]

transition_count_of_level_name = defaultdict(int)
for level_number_lower, level_number_upper in zip(lowerlevel, upperlevel, strict=False):
transition_count_of_level_name[level_number_lower] += 1
transition_count_of_level_name[level_number_upper] += 1

assert len(transitions) == len(
energy_levels_lower_1000percm
) # check number of transitions is the same as the number read in

return ionization_energy_in_ev, energy_levels, transitions, transition_count_of_level_name


# read_levels_and_transitions(atomic_number=57, ion_stage=5, flog=None)
4 changes: 4 additions & 0 deletions atomic-data-mons/url-lanthanidesV-VII.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
https://doi.org/10.5281/zenodo.10635803
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add direct URLs for the zip files that are needed as we do for other data sources.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what the links are. Does it work to just add the files names to the end of the zenodo link?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://zenodo.org/records/10635803/files/outggf_Ln_V--VII.zip
https://zenodo.org/records/10635803/files/outglv_Ln_V--VII.zip
The readme should probably just be committed directly to the repo since it's so small.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The info from the readme is in comments at the top of the file


# No need to unzip files - artisatomic reads the zip archives