Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jersey-joins-epilepsy12 #1105

Merged
merged 23 commits into from
Jan 23, 2025
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
d871fe6
A Jersey .shp files from `https://gadm.org/download_country.html`
eatyourpeas Nov 23, 2024
6dc8123
too big a commit - sorry
eatyourpeas Nov 23, 2024
6abbd8f
now loads home page without maps for jersey after multiple messy work…
eatyourpeas Nov 23, 2024
ae52abb
ok start again, redo the migrations, this time map jersey layer to co…
eatyourpeas Nov 23, 2024
96363ac
now maps working. filter country map to individual country. remap sri…
eatyourpeas Nov 23, 2024
1757cf3
fixed maps
eatyourpeas Nov 23, 2024
d25a676
rejig templates to remove levels of abstraction for Jersey
eatyourpeas Nov 23, 2024
346ba23
fix scatter plot of cases to exclude those without postcode or geogra…
eatyourpeas Nov 23, 2024
3f2f877
add search field for trusts in admin, add URN to Case model, check fo…
eatyourpeas Nov 23, 2024
357890c
create patient form, add urn to model, update form template - still n…
eatyourpeas Nov 23, 2024
298d722
filter by URN, fix E12CaseFactory for Jersey, heading in case table
eatyourpeas Nov 24, 2024
65c8309
fix seed functions for jersey postcodes and URNs. Fix holes in the Ca…
eatyourpeas Nov 24, 2024
6eb4449
typo and test commit
eatyourpeas Nov 24, 2024
211e0f9
validate jersey postcodes, prevent geocode lookup for jersey since, w…
eatyourpeas Nov 24, 2024
3fb2de2
documentation updates and fixes esp postcodes, imd and mapping
eatyourpeas Nov 24, 2024
5b7e022
accepting all suggestions
eatyourpeas Dec 3, 2024
dfe3bda
add migration
eatyourpeas Dec 3, 2024
fc3d5eb
Merge branch 'live' into eatyourpeas/issue1100
eatyourpeas Jan 18, 2025
23db5c1
move migrations to temp
eatyourpeas Jan 20, 2025
0771c28
Merge branch 'live' into eatyourpeas/issue1100
eatyourpeas Jan 20, 2025
0948a05
reorder migrations
eatyourpeas Jan 20, 2025
9dda236
template edits to incorporate jersey
eatyourpeas Jan 20, 2025
c4d853d
remove extraneous print statements for debugging
eatyourpeas Jan 23, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion documentation/docs/admin-users/download.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ author: Dr Marcus Baw

You can download the entire Epilepsy12 documentation manual as a PDF for offline reading. Click the button below to download.

[:fontawesome-solid-file-pdf: Download documentation manual in PDF format](../../pdf/rcpch-epilepsy12-full-documentation.pdf){ .md-button .md-button--primary }
[:fontawesome-solid-file-pdf: Download documentation manual in PDF format](../pdf/rcpch-epilepsy12-full-documentation.pdf){ .md-button .md-button--primary }
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ The NHS number is checked against the NHS number [checksum](https://www.datadict

### Transferring a patient between centres

Only the [**Lead Clinician**](../user-group-permissions/#lead-clinician) has permissions to transfer children to another centre.
Only the [**Lead Clinician**](user-group-permissions.md#lead-clinician) has permissions to transfer children to another centre.

The steps to do this for the lead clinician in Organisation A

Expand Down
7 changes: 3 additions & 4 deletions documentation/docs/development/docker-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ cd rcpch-audit-engine
```

!!! warning "Windows Setup"
**If you are on Windows**, after installing Docker and cloning the repository, please now skip to the [(Windows) Setup for development using Docker Compose](./docker-setup.md#windows-setup-for-development-using-docker-compose) section.
**If you are on Windows**, after installing Docker and cloning the repository, please now skip to the [(Windows) Setup for development using Docker Compose](./docker-setup-windows.md) section.

### Ensure you are on the default `development` branch

Expand All @@ -48,7 +48,6 @@ cp envs/env-template envs/.env

!!! warning "Mac Users"
If using Mac and Safari, to access the Epilepsy 12 engine in your development, you must change the `SITE_DOMAIN` name in .env to 'localhost', and type this into your browser once you have executed `s/up` in the next step. This will load the E12 engine in your Safari browser.

However, for simplicity, we recommend using a different browser, such as Chrome, and leaving the .env file unaltered.

### Start the development environment for the first time using our startup script
Expand Down Expand Up @@ -107,7 +106,6 @@ This Docker setup is quite new so please do open an issue if there is anything t

!!! warning "Terminal is now occupied"
If you have successfully run the Docker Compose deployment, your terminal will be showing the combined and colour-coded logging output for all the containers and will no longer show an interactive prompt, which is means you can not run any more commands in that terminal. To resolve this, simply **open another Terminal window** in the same working directory, in which you can run commands.

If opening another terminal is impractical or impossible, then in most Shell environments you can press `Ctrl`+`Z` to suspend the current process, and then `bg` to resume it in the background. This will return you to an interactive prompt. Once you've executed your further commands, you can then use `fg` to bring the console logging output back to the foreground again.

### Creating a superuser
Expand All @@ -117,6 +115,7 @@ You can use our convenience script to create a superuser in the context of the `
```console
s/create-superuser
```

The script will prompt you for required user attributes:

```console
Expand Down Expand Up @@ -208,7 +207,7 @@ For testing of the UI it is often useful to have some dummy data in the database
s/seed
```

See the [Seeding the Database](../manual-setup/#seeding-the-database) section for more details on the usage of this script, for example setting a non-default Cohort Number.
See the [Seeding the Database](manual-setup.md#seeding-the-database) section for more details on the usage of this script, for example setting a non-default Cohort Number.

## Tips and Tricks, Gotchas and Caveats

Expand Down
2 changes: 1 addition & 1 deletion documentation/docs/development/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,4 +39,4 @@ Security is a very important aspect of managing projects such as these, and we u

## Legal

Details of Information Governance, Clinical Safety and Medical Device registration can be viewed in the [Legal](../legal/legal.md) section.
Details of Information Governance, Clinical Safety and Medical Device registration can be viewed in the [Legal](../legal/intellectual-property.md) section.
18 changes: 18 additions & 0 deletions documentation/docs/development/imd.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
title: Indices of Multiple Deprivation
reviewers: Dr Simon Chapman
---

## Calculation

This is discussed more extensively in the RCPCH Census Platform which returns index of multiple deprivation quantiles against a UK postcode across the devolved nations. Currently Jersey is not supported.

### Methodology

Index of multiple deprivation is dependent on geography, and the underlying assumptions inherent in the index are underpinned with responses in the national censuses. The index is a composite of multiple domains from employment to housing to access to green space. Different countries even within the UK have different domains (though they are similar) and this means that findings in one country cannot be compared with another. There is a discussion of this in more detail in the [repository](https://github.com/rcpch/rcpch-census-platform)

The country is first broken into areas of similar population size, known as low layer super output areas (LSOAs) and the findings for each measure summarized at this level. The LSOAs are then ranked in order by raw score, with the lower raw scores representing the least deprived. These are then broken into quantiles, depending on the size of the population / reporting priorities. It is typical to report as deciles or quintiles.

The last English data was published in 2019, with Wales the same year.

Jersey has recently been added to Epilepsy12 and is currently not supported by the RCPCH Census Platform. Because the population is small (~100,000), IMD is reported in vingtaines, rather than quintiles or deciles. The actual data is not published and is being requested for inclusion. In Epilepsy12 currently IMD quantiles are therefore not reported.
20 changes: 19 additions & 1 deletion documentation/docs/development/organisations.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ The organisational structure of health care in England and Wales influences repo

### Organisations and Trusts

This is the lowest level of abstraction and represents either an acute or a community hospital/organisation responsible for epilepsy care of children and young people. There are often several organisations in a Trust. Each organisation, like each Trust, has its own ODS code, and from year to year there is movement between trusts as organisations change their allegiances between trusts when mergers are carried out. The organisation model therefore has more than once instance for some organisations, as their parent status or other details such as name change.
This is the lowest level of abstraction and represents either an acute or a community hospital/organisation responsible for epilepsy care of children and young people. There are often several organisations in a Trust. Each organisation, like each Trust, has its own ODS code, and from year to year there is movement between trusts as organisations change their allegiances between trusts when mergers are carried out. The organisation model therefore has more than once instance for some organisations, as their parent status or other details such as name change.

On a monthly basis the NHS ODS API is polled with any changes to organisational structure, and the local database record for that organisation is updated to reflect the latest changes.

Expand All @@ -32,3 +32,21 @@ There are 7 of these in England and their model is taken from NHS Digital. Each
### Local Authorities

Local authority codes for each organisation are not stored except for those organisations in London. Local authorities are administrative regions not related to health or the NHS. In London local authorities are usually referred to as London Boroughs. There is a boundary model for London Boroughs taken from NHS Digital and this is used only for mapping. As above, although versioned, there is no process in place to check for updates nationally and update the database record.

### Jersey and the Channel Islands

Jersey joined Epilepsy12 in 2024. Jersey is in the Channel Islands and part of the UK but does not participate in the NHS. There are reciprocal agreements about some hospital treatment, but inpatient care is free only to people who have been resident for 6 months and have a Health Card.

***Organisational structure***
The E12 structure is that organisations have a Trust or Local Health Board as their parent. There is a hierarchy above this which varies between England and Wales. Organisations and Trusts in England might have the same name, but will always have separate ODS codes, which are often similar. Jersey General Hospital in St Helier is a Trust which provides medical care directly. To work around this, Jersey General Hospital has been created both as an Organisation and a Trust, each with the same ODS code, so that it is its own parent.

***Levels of Abstraction***
Jersey has been added as a separate country, so that it can report at the level of organisation, Open UK Network, trust and country, though the numbers for these 3 hierarchies will be the same.

## Maps

Django GIS and the additional Postgres support for geoJSON are both reasons why these tools were used for this project. The Organisation View presents a dashboard that includes a scatterplot of patients specific to that organisation, with mean, median, minimum and maximum distances for patients to travel to clinic. There are also maps with boundaries demarcating health geographies such as NHS England regions and Integrated Care Boards. These are provided by `.shp` files which contain the coordinates for plotting the shapes on top of the maps. There is documentation on adding shape files [here](shape-files.md)

The basic maps are provided by [MapBox](https://www.mapbox.com/) using their free tier, with the API key stored in credentials. An API key is required.

E12 currently looks up postcodes against an API which returns longitude and latitude, and these are stored in the model (using SRID 27700) and this is used to plot them on the scatter plots. This currently does not function for Jersey as longitude and latitude for post codes are currently not stored.
14 changes: 14 additions & 0 deletions documentation/docs/development/postcodes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
title: Postcodes
reviewers: Dr Simon Chapman
---

This should be read in conjunction with sections on mapping and index of multiple deprivation.

Postcodes for patients are stored in RCPCH-Audit-Engine securely as part of the national agreement for the RCPCH Epilepsy12 clinical audit. Opt out can be requested by patients and all data with the exception of the platform unique identifier (not the NHS number or Unique Reference Number) which is retained to generate an accurate denominator.

Postcodes are used primarily to calculate indices of multiple deprivation, but are also used to provide a scatterplot for clinicians of patients in a given organisation to report maximum/minimum/mean and median distances patients have to travel for care.

Postcodes are passed to findthatpostcode.uk. This reports information against postcode which include LSOA (see Indices of Multiple Deprivation) as well as longitude and latitude. These latter data points are used for scatter plots.

Jersey is currently not supported as there is no open source solution for mapping currently though this is tracked in a [github issue](https://github.com/rcpch/rcpch-audit-engine/issues/1107)
186 changes: 186 additions & 0 deletions documentation/docs/development/shape-files.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
---
title: Level of Abstractions and their boundaries, using GIS
reviewers: Dr Simon Chapman
---

## Shape Files

GeoDjango and GIS is a package that sits ontop of Django and is accompanied by an additional pack for Postgres. It has additional fields and methods for geographical data, such as longitude, latitude and conversions for mapping different coordinate systems (eg northings and eastings vs longitude and latitude).

Shape files contain the coordinates to map boundaries of regions. Epilepsy12 has boundary files for the countries of the United Kingdom from the UK [Office of National Statistics](https://geoportal.statistics.gov.uk/).

In November 2024 support was added for Jersey. The shape files for this were taken from [GADM](https://gadm.org/index.html), a well-known resource of geographical data. This is a walk through of this as an example but note this is not how it is implemented actually. It is an example to show how to add a new model and boundary data to it. In reality, no new model was created for Jersey but instead the `.shp` file was mapped to existing fields in the `Country` model and then a new migration created to add the Organisation in Jersey with all its relationships.

### Adding a new .shp file

1. add the files to the `shape_files` directory. There are different file extensions but the key one is the `.shp` file for Epilepsy12
2. Use `ogrinfo` on the command line to inspect the `.shp` file field structure. For example:

`ogrinfo -al -so epilepsy12/shape_files/gadm41_JEY_shp/gadm41_JEY_0.shp` returns

```console
Layer name: gadm41_JEY_0
Metadata:
DBF_DATE_LAST_UPDATE=2022-07-18
Geometry: Polygon
Feature Count: 1
Extent: (-2.255138, 49.147083) - (-1.924584, 49.292915)
Layer SRS WKT:
GEOGCRS["WGS 84",
DATUM["World Geodetic System 1984",
ELLIPSOID["WGS 84",6378137,298.257223563,
LENGTHUNIT["metre",1]]],
PRIMEM["Greenwich",0,
ANGLEUNIT["degree",0.0174532925199433]],
CS[ellipsoidal,2],
AXIS["latitude",north,
ORDER[1],
ANGLEUNIT["degree",0.0174532925199433]],
AXIS["longitude",east,
ORDER[2],
ANGLEUNIT["degree",0.0174532925199433]],
ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
GID_0: String (10.0)
COUNTRY: String (10.0)
```

This is not an essential step - it just allows you to inspect the file and see the fields.

3. Use `ogrinspect` to convert the `.shp` file to a `LayerMap` instance
`python manage.py ogrinspect epilepsy12/shape_files/gadm41_JEY_shp/gadm41_JEY_0.shp JerseyBoundary --srid=4326 --mapping --multi`

This generates the code for the Model (change the name to the Model name you want). Note the flags - `srid` relates to the spatial reference identifier (SRID), a unique identifier associated with a specific coordinate system and resolution. The other flags create the boundary mapping object. This returns:

```console
# This is an auto-generated Django model module created by ogrinspect.
from django.contrib.gis.db import models


class JerseyBoundary(models.Model):
gid_0 = models.CharField(max_length=10)
country = models.CharField(max_length=10)
geom = models.MultiPolygonField(srid=4326)


# Auto-generated `LayerMapping` dictionary for JerseyBoundary model
jerseyboundary_mapping = {
'gid_0': 'GID_0',
'country': 'COUNTRY',
'geom': 'MULTIPOLYGON',
}
```

4. Add the new model to the `models_folder`. Generally the model with the boundaries in Epilepsy12 is an abstract model and inherited by the parent model which contains more information and the relationships:

```python
from django.contrib.gis.db import models

class JerseyBoundary(models.Model):
gid_0 = models.CharField(max_length=10)
country = models.CharField(max_length=10)
geom = models.MultiPolygonField(srid=4326)

class Meta:
abstract = True


class Jersey(JerseyBoundary):
class Meta:
indexes = [models.Index(fields=["gid_0"])]
verbose_name = "Jersey"
verbose_name_plural = "Jersey"
ordering = ("country",)

def __str__(self) -> str:
return self.country
```

5. Create a migration for the new models:

`python manage.py makemigrations`

6. Create a new custom migration for the shape mapping / data import process:

`python manage.py makemigrations epilepsy12 --name jersey_shape_file_mapping --empty`

7. Create a function within the migration that creates a new model, creates a new layer map and prescribes the path to the `.shp` file to import the data into the database. It would look like this:

```python
# Generated by Django 5.1.2 on 2024-11-23 13:16

# python imports
import os

# django imports
from django.db import migrations

# from django.apps import apps as django_apps
from django.apps import apps as django_apps
from django.contrib.gis.utils import LayerMapping

Jersey = django_apps.get_model("epilepsy12", "Jersey")

# Auto-generated `LayerMapping` dictionary for JerseyBoundary model
jerseyboundary_mapping = {
"gid_0": "GID_0",
"country": "COUNTRY",
"geom": "MULTIPOLYGON",
}

# Get the path to the shape file
app_config = django_apps.get_app_config("epilepsy12")
app_path = app_config.path
jersey_shp_file_path = os.path.join(
app_path, "shape_files", "gadm41_JEY_shp", "gadm41_JEY_0.shp"
)


def load_jersey_shape_file_mapping(apps, schema_editor):
"""
Load the Jersey shape file mapping into the database
"""

# Load the Jersey shape file mapping into the database
lm = LayerMapping(
Jersey,
jersey_shp_file_path,
jerseyboundary_mapping,
transform=False,
encoding="utf-8",
)

lm.save(strict=True, verbose=True)


class Migration(migrations.Migration):

dependencies = [
("epilepsy12", "0043_jersey"),
]

operations = [
migrations.RunPython(load_jersey_shape_file_mapping),
]

```

8. Run the migration

`python manage.py migrate`

```console
root@600d309878ba:/app# python manage.py makemigrations
Migrations for 'epilepsy12':
epilepsy12/migrations/0043_jersey.py
+ Create model Jersey
root@600d309878ba:/app# python manage.py makemigrations epilepsy12 --name jersey_shape_file_mapping --empty
Migrations for 'epilepsy12':
epilepsy12/migrations/0044_jersey_shape_file_mapping.py
root@600d309878ba:/app# python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, authtoken, captcha, contenttypes, epilepsy12, otp_email, otp_static, otp_totp, phonenumber, sessions
Running migrations:
Applying epilepsy12.0043_jersey... OK
Applying epilepsy12.0044_jersey_shape_file_mapping...Saved: Jersey
```
2 changes: 1 addition & 1 deletion documentation/docs/development/user-groups.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: User groups and permissions
reviewers: Dr Simon Chapman
---

The user groups are summarised [here](../clinician-users/clinician-user-guide.md###Permission)
The user groups are summarised [here](../clinician-users/clinician-user-guide.md)

Django allows permission-based and group based access. The user groups defined above are containers for permissions to all the models. Generic django permissions allow prescription of view, change, create and delete to each model (found in ```epilepsy12/constants/user_types.py```).

Expand Down
2 changes: 1 addition & 1 deletion documentation/docs/legal/privacy-impact-assessment.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Data Protection Impact Assessment
reviewers: Dr Marcus Baw
dpia_path: ../../reference-documents/2023-12-PIA-2018-02-epilepsy12-pia-v2-1_Final.pdf
dpia_path: ../reference-documents/2023-12-PIA-2018-02-epilepsy12-pia-v2-1_Final.pdf
hide:
- toc
---
Expand Down
4 changes: 2 additions & 2 deletions documentation/docs/legal/privacy-notice.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: Privacy Notice
reviewers: Dr Marcus Baw
privacy_notice_path: ../../reference-documents/20230118 E12 Privacy Notice.pdf
privacy_notice_welsh_path: ../../reference-documents/20230118 E12 Privacy Notice Welsh.pdf
privacy_notice_path: ../reference-documents/20230118 E12 Privacy Notice.pdf
privacy_notice_welsh_path: ../reference-documents/20230118 E12 Privacy Notice Welsh.pdf
hide:
- toc
---
Expand Down
Loading
Loading