Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restrict character sets for attribute values #397

Draft
wants to merge 52 commits into
base: release/v7
Choose a base branch
from

Conversation

Magnus-Kuhn
Copy link
Contributor

@Magnus-Kuhn Magnus-Kuhn commented Jan 24, 2025

Readiness checklist

  • I added/updated tests.
  • I ensured that the PR title is good enough for the changelog.
  • I labeled the PR.
  • I self-reviewed the PR.

@Magnus-Kuhn Magnus-Kuhn added breaking-change A breaking change enhancement New feature or request wip Work in Progress (blocks mergify from auto update the branch) labels Jan 24, 2025
Copy link

codecov bot commented Jan 24, 2025

Codecov Report

Attention: Patch coverage is 97.39583% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...ntent/src/attributes/types/relationship/Consent.ts 33.33% 4 Missing ⚠️
...ontent/src/attributes/types/strings/AbstractXML.ts 0.00% 1 Missing ⚠️
Files with missing lines Coverage Δ
...es/content/src/attributes/RelationshipAttribute.ts 96.82% <100.00%> (+0.05%) ⬆️
...ntent/src/attributes/RelationshipAttributeQuery.ts 98.05% <100.00%> (+0.01%) ⬆️
...attributes/ThirdPartyRelationshipAttributeQuery.ts 96.55% <100.00%> (-3.45%) ⬇️
.../content/src/attributes/constants/CharacterSets.ts 100.00% <100.00%> (ø)
...ackages/content/src/attributes/hints/ValueHints.ts 98.10% <100.00%> (-0.56%) ⬇️
...es/content/src/attributes/hints/ValueHintsValue.ts 100.00% <100.00%> (+8.10%) ⬆️
...ges/content/src/attributes/types/AbstractString.ts 100.00% <100.00%> (ø)
...nt/src/attributes/types/address/AbstractAddress.ts 100.00% <100.00%> (ø)
...kages/content/src/attributes/types/address/City.ts 87.50% <100.00%> (+0.83%) ⬆️
...src/attributes/types/address/DeliveryBoxAddress.ts 85.48% <100.00%> (+11.50%) ⬆️
... and 33 more

... and 119 files with indirect coverage changes

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

@Milena-Czierlinski Milena-Czierlinski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requires additional approval of @stnmtz once we want to merge this PR.

@jkoenig134 jkoenig134 changed the base branch from main to release/v7 March 3, 2025 17:11
@jkoenig134 jkoenig134 dismissed Milena-Czierlinski’s stale review March 3, 2025 17:11

The base branch was changed.

@jkoenig134 jkoenig134 removed the wip Work in Progress (blocks mergify from auto update the branch) label Mar 5, 2025
@jkoenig134
Copy link
Member

When adding documentation, note that only NFC normalized strings are considered valid.

@Magnus-Kuhn you can add docs now yourself against: nmshd/documentation#307

Copy link
Member

@jkoenig134 jkoenig134 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seeing what you did with #437 I think we should do all this in the controller and not on serval level. Additionally it wouldn't be a breaking change then.

@@ -28,7 +29,7 @@ export class RelationshipAttribute<TValueClass extends AttributeValues.Relations
public value: TValueClass;

@serialize()
@validate({ max: 100 })
@validate({ max: 100, regExp: characterSets.din91379DatatypeC })
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this key is only machine read, I don't see a reason why we should limit the charset here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My reason for validating everything was that integrators can safely copy attributes into their systems, even if they can't handle all characters

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see someone not being able to "handle" this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for creating the norm and making it mandatory was that there were non-conformant IT systems (some that weren't even using UTF8) - see https://www.it-planungsrat.de/fileadmin/beschluesse/2022/Beschluss2022-51_Umsetzung.pdf. I would not be as confident with this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ehm, nope?! The main reason is that we don't want to have ✌🏼😜😜😜🤣😆🥹😅 as a display name or surname.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, then there has been some misunderstanding

@@ -30,7 +31,7 @@ export interface IRelationshipAttributeCreationHints extends ISerializable {
@type("RelationshipAttributeCreationHints")
export class RelationshipAttributeCreationHints extends Serializable implements IRelationshipAttributeCreationHints {
@serialize()
@validate({ max: PROPRIETARY_ATTRIBUTE_MAX_TITLE_LENGTH })
@validate({ max: PROPRIETARY_ATTRIBUTE_MAX_TITLE_LENGTH, regExp: characterSets.din91379DatatypeC })
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are title and description validated?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a query has forbidden characters, then there would be no attribute that fits to the query

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ehm, what? this would require that title and description are validated in the attribute itself..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are currently validated in the Proprietary... types

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, but also there I'd heavily argue that this is necessary.

@Magnus-Kuhn
Copy link
Contributor Author

Seeing what you did with #437 I think we should do all this in the controller and not on serval level. Additionally it wouldn't be a breaking change then.

Doing it in serval validates requests and notifications as well

@jkoenig134
Copy link
Member

Seeing what you did with #437 I think we should do all this in the controller and not on serval level. Additionally it wouldn't be a breaking change then.

Doing it in serval validates requests and notifications as well

But it requires you to duplicate the logic everywhere ... I don't see a reason for doing that.

@jkoenig134
Copy link
Member

Seeing what you did with #437 I think we should do all this in the controller and not on serval level. Additionally it wouldn't be a breaking change then.

Doing it in serval validates requests and notifications as well

But it requires you to duplicate the logic everywhere ... I don't see a reason for doing that.

Additionally the error message for regex sucks. When doing that in controllers we would have the possibility to return a WAY BETTER error message than hey, your entered value does not conform vberuozfapmwjewhrubwihrieoawdkvjqnljfldwawer78f6r7w1d3wef4ads8615as, please deal with it

@Magnus-Kuhn
Copy link
Contributor Author

Also generally the regex checks for e.g. Email occur in content - which also has a big error message. Do you suggest moving them all (and maybe the string length checks as well) into the controller?

@jkoenig134
Copy link
Member

Also generally the regex checks for e.g. Email occur in content - which also has a big error message. Do you suggest moving them all (and maybe the string length checks as well) into the controller?

I don't know why this has to be a general thing. I see a thing that gets introduced in almost every content class, which is not ok and we should find a solution for that - and I proposed a solution.

Projecting this to "but then we should remove all regex validations" is very misplaced here.

@Magnus-Kuhn
Copy link
Contributor Author

nmshd/documentation#309 does documentation, #448 is an alternative approach

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking-change A breaking change enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants