mirror of
https://github.com/pcvolkmer/etl-processor.git
synced 2025-07-17 21:02:54 +00:00
Compare commits
17 Commits
v0.10.0
...
119-genomd
Author | SHA1 | Date | |
---|---|---|---|
b5b6203f3a | |||
07508679e6 | |||
7543785116 | |||
858189aa59 | |||
17f4dc3512 | |||
1dd601e8db | |||
b748603c06 | |||
b939b2bf57 | |||
c6b37fda69 | |||
8e3de6a220 | |||
c5c553f817 | |||
7d97365aea | |||
48b1e62e22 | |||
66cc818755 | |||
9d4786fae3 | |||
b78dc3519b | |||
46015c5b66 |
261
README.md
261
README.md
@ -1,30 +1,48 @@
|
||||
# ETL-Processor for DNPM:DIP [](https://github.com/pcvolkmer/etl-processor/actions/workflows/test.yml)
|
||||
|
||||
Diese Anwendung versendet ein bwHC-MTB-File im bwHC-Datenmodell 1.0 an DNPM:DIP und pseudonymisiert die Patienten-ID.
|
||||
Diese Anwendung versendet ein bwHC-MTB-File im bwHC-Datenmodell 1.0 an DNPM:DIP und pseudonymisiert
|
||||
die Patienten-ID.
|
||||
|
||||
## Einordnung innerhalb einer DNPM-ETL-Strecke
|
||||
|
||||
Diese Anwendung erlaubt das Entgegennehmen von HTTP/REST-Anfragen aus dem Onkostar-Plugin **[onkostar-plugin-dnpmexport](https://github.com/CCC-MF/onkostar-plugin-dnpmexport)**.
|
||||
Diese Anwendung erlaubt das Entgegennehmen von HTTP/REST-Anfragen aus dem Onkostar-Plugin *
|
||||
*[onkostar-plugin-dnpmexport](https://github.com/CCC-MF/onkostar-plugin-dnpmexport)**.
|
||||
|
||||
Der Inhalt einer Anfrage, wenn ein bwHC-MTBFile, wird pseudonymisiert und auf Duplikate geprüft.
|
||||
Duplikate werden verworfen, Änderungen werden weitergeleitet.
|
||||
|
||||
Löschanfragen werden immer als Löschanfrage an DNPM:DIP weitergeleitet.
|
||||
|
||||
Zudem ist eine minimalistische Weboberfläche integriert, die einen Einblick in den aktuellen Zustand der Anwendung gewährt.
|
||||
Zudem ist eine minimalistische Weboberfläche integriert, die einen Einblick in den aktuellen Zustand
|
||||
der Anwendung gewährt.
|
||||
|
||||

|
||||
|
||||
### Duplikaterkennung
|
||||
|
||||
Die Erkennung von Duplikaten ist normalerweise immer aktiv, kann jedoch über den Konfigurationsparameter
|
||||
Die Erkennung von Duplikaten ist normalerweise immer aktiv, kann jedoch über den
|
||||
Konfigurationsparameter
|
||||
`APP_DUPLICATION_DETECTION=false` deaktiviert werden.
|
||||
|
||||
### Modelvorhaben genomDE §64e
|
||||
|
||||
Zusätzlich zur Patienten Identifier Pseudonymisierung müssen Vorgangsummern generiert werden, die
|
||||
jede Übertragung eindeutig identifizieren aber gleichzeitig dem Patienten zugeordnet werden können.
|
||||
Dies lässt sich durch weitere Pseudonyme abbilden, allerdings werden pro Originalwert mehrere
|
||||
Pseudonyme benötigt.
|
||||
Zu diesem Zweck muss in gPas eine **Multi-Pseudonym-Domäne** konfiguriert werden (siehe auch
|
||||
*APP_PSEUDONYMIZE_GPAS_CCDN*).
|
||||
|
||||
**WICHTIG:** Deaktivierte Pseudonymisierung ist nur für Tests nutzbar. Vorgangsummern sind zufällig
|
||||
und werden anschließend verworfen.
|
||||
|
||||
### Datenübermittlung über HTTP/REST
|
||||
|
||||
Anfragen werden, wenn nicht als Duplikat behandelt, nach der Pseudonymisierung direkt an DNPM:DIP gesendet.
|
||||
Anfragen werden, wenn nicht als Duplikat behandelt, nach der Pseudonymisierung direkt an DNPM:DIP
|
||||
gesendet.
|
||||
|
||||
Ein HTTP Request kann, angenommen die Installation erfolgte auf dem Host `dnpm.example.com` an nachfolgende URLs gesendet werden:
|
||||
Ein HTTP Request kann, angenommen die Installation erfolgte auf dem Host `dnpm.example.com` an
|
||||
nachfolgende URLs gesendet werden:
|
||||
|
||||
| HTTP-Request | URL | Consent-Status im Datensatz | Bemerkung |
|
||||
|--------------|-----------------------------------------|-----------------------------|---------------------------------------------------------------------------------|
|
||||
@ -32,12 +50,15 @@ Ein HTTP Request kann, angenommen die Installation erfolgte auf dem Host `dnpm.e
|
||||
| `POST` | `https://dnpm.example.com/mtb` | `REJECT` | Die Anwendung sendet einen Lösch-Request für die im Datensatz angegebene Pat-ID |
|
||||
| `DELETE` | `https://dnpm.example.com/mtb/12345678` | - | Die Anwendung sendet einen Lösch-Request für Pat-ID `12345678` |
|
||||
|
||||
Anstelle des Pfads `/mtb` kann auch, wie in Version 0.9 und älter üblich, `/mtbfile` verwendet werden.
|
||||
Anstelle des Pfads `/mtb` kann auch, wie in Version 0.9 und älter üblich, `/mtbfile` verwendet
|
||||
werden.
|
||||
|
||||
### Datenübermittlung mit Apache Kafka
|
||||
|
||||
Anfragen werden, wenn nicht als Duplikat behandelt, nach der Pseudonymisierung an Apache Kafka übergeben.
|
||||
Eine Antwort wird dabei ebenfalls mithilfe von Apache Kafka übermittelt und nach der Entgegennahme verarbeitet.
|
||||
Anfragen werden, wenn nicht als Duplikat behandelt, nach der Pseudonymisierung an Apache Kafka
|
||||
übergeben.
|
||||
Eine Antwort wird dabei ebenfalls mithilfe von Apache Kafka übermittelt und nach der Entgegennahme
|
||||
verarbeitet.
|
||||
|
||||
Siehe hierzu auch: https://github.com/CCC-MF/kafka-to-bwhc
|
||||
|
||||
@ -45,17 +66,19 @@ Siehe hierzu auch: https://github.com/CCC-MF/kafka-to-bwhc
|
||||
|
||||
### 🔥 Wichtige Änderungen in Version 0.10
|
||||
|
||||
Ab Version 0.10 wird [DNPM:DIP](https://github.com/dnpm-dip) unterstützt und als Standardendpunkt verwendet.
|
||||
Soll noch das alte bwHC-Backend verwendet werden, so ist die Umgebungsvariable `APP_REST_IS_BWHC` auf `true` zu setzen.
|
||||
Ab Version 0.10 wird [DNPM:DIP](https://github.com/dnpm-dip) unterstützt und als Standardendpunkt
|
||||
verwendet.
|
||||
Soll noch das alte bwHC-Backend verwendet werden, so ist die Umgebungsvariable `APP_REST_IS_BWHC`
|
||||
auf `true` zu setzen.
|
||||
|
||||
### 🔥 Breaking Changes nach Version 0.10
|
||||
|
||||
In Versionen des ETL-Processors **nach Version 0.10** werden die folgenden Konfigurationsoptionen entfernt:
|
||||
In Versionen des ETL-Processors **nach Version 0.10** werden die folgenden Konfigurationsoptionen
|
||||
entfernt:
|
||||
|
||||
* `APP_PSEUDONYMIZE_GPAS_SSLCALOCATION`: Nutzen Sie hier, wie unter [_Integration eines eigenen Root CA
|
||||
Zertifikats_](#integration-eines-eigenen-root-ca-zertifikats) beschrieben, das Einbinden eigener Zertifikate.
|
||||
* `APP_KAFKA_TOPIC`: Nutzen Sie nun die Konfigurationsoption `APP_KAFKA_OUTPUT_TOPIC`
|
||||
* `APP_KAFKA_RESPONSE_TOPIC`: Nutzen Sie nun die Konfigurationsoption `APP_KAFKA_OUTPUT_RESPONSE_TOPIC`
|
||||
* `APP_KAFKA_RESPONSE_TOPIC`: Nutzen Sie nun die Konfigurationsoption
|
||||
`APP_KAFKA_OUTPUT_RESPONSE_TOPIC`
|
||||
|
||||
Der Pfad zum Versenden von MTB-Daten ist nun offiziell `/mtb`.
|
||||
In Versionen **nach Version 0.10** wird die Unterstützung des Pfads `/mtbfile` entfernt.
|
||||
@ -68,39 +91,37 @@ Ist diese nicht gesetzt. wird intern eine Anonymisierung der Patienten-ID vorgen
|
||||
* `APP_PSEUDONYMIZE_PREFIX`: Standortbezogenes Präfix - `UNKNOWN`, wenn nicht gesetzt
|
||||
* `APP_PSEUDONYMIZE_GENERATOR`: `BUILDIN` oder `GPAS` - `BUILDIN`, wenn nicht gesetzt
|
||||
|
||||
**Hinweise**:
|
||||
**Hinweis**
|
||||
|
||||
* Der alte Konfigurationsparameter `APP_PSEUDONYMIZER` mit den Werten `GPAS` oder `BUILDIN` sollte nicht mehr verwendet
|
||||
werden.
|
||||
* Die Pseudonymisierung erfolgt im ETL-Prozessor nur für die Patienten-ID.
|
||||
Andere IDs werden mithilfe des standortbezogenen Präfixes (erneut) anonymisiert, um für den aktuellen Kontext nicht
|
||||
vergleichbare IDs bereitzustellen.
|
||||
Die Pseudonymisierung erfolgt im ETL-Prozessor nur für die Patienten-ID.
|
||||
Andere IDs werden mithilfe des standortbezogenen Präfixes (erneut) anonymisiert, um für den
|
||||
aktuellen Kontext nicht
|
||||
vergleichbare IDs bereitzustellen.
|
||||
|
||||
#### Eingebaute Anonymisierung
|
||||
|
||||
Wurde keine oder die Verwendung der eingebauten Anonymisierung konfiguriert, so wird für die Patienten-ID der
|
||||
entsprechende SHA-256-Hash gebildet und Base64-codiert - hier ohne endende "=" - zuzüglich des konfigurierten Präfixes
|
||||
Wurde keine oder die Verwendung der eingebauten Anonymisierung konfiguriert, so wird für die
|
||||
Patienten-ID der
|
||||
entsprechende SHA-256-Hash gebildet und Base64-codiert - hier ohne endende "=" - zuzüglich des
|
||||
konfigurierten Präfixes
|
||||
als Patienten-Pseudonym verwendet.
|
||||
|
||||
#### Pseudonymisierung mit gPAS
|
||||
|
||||
Wurde die Verwendung von gPAS konfiguriert, so sind weitere Angaben zu konfigurieren.
|
||||
|
||||
* `APP_PSEUDONYMIZE_GPAS_URI`: URI der gPAS-Instanz inklusive Endpoint (z.B. `http://localhost:8080/ttp-fhir/fhir/gpas/$$pseudonymizeAllowCreate`)
|
||||
* `APP_PSEUDONYMIZE_GPAS_TARGET`: gPas Domänenname
|
||||
* `APP_PSEUDONYMIZE_GPAS_URI`: URI der gPAS-Instanz inklusive Endpoint (z.B.
|
||||
`http://localhost:8080/ttp-fhir/fhir/gpas/$$pseudonymizeAllowCreate`)
|
||||
* `APP_PSEUDONYMIZE_GPAS_TARGET`: gPas Domänenname für Patienten ID
|
||||
* `APP_PSEUDONYMIZE_GPAS_USERNAME`: gPas Basic-Auth Benutzername
|
||||
* `APP_PSEUDONYMIZE_GPAS_PASSWORD`: gPas Basic-Auth Passwort
|
||||
* ~~`APP_PSEUDONYMIZE_GPAS_SSLCALOCATION`~~: **Veraltet** - Root Zertifikat für gPas, falls es dediziert hinzugefügt werden muss.
|
||||
**Wird in nach Version 0.10 entfernt**
|
||||
|
||||
Der Konfigurationsparameter `APP_PSEUDONYMIZE_GPAS_SSLCALOCATION` sollte nicht mehr verwendet werden und wird nach
|
||||
Version 0.10 entfernt.
|
||||
Stattdessen sollte das Root Zertifikat wie unter [_Integration eines eigenen Root CA
|
||||
Zertifikats_](#integration-eines-eigenen-root-ca-zertifikats) beschrieben eingebunden werden.
|
||||
* `APP_PSEUDONYMIZE_GPAS_GENOM_DE_DOMAIN`: gPas Multi-Pseudonym-Domäne für genomDE Vorgangsnummern (
|
||||
Clinical data node)
|
||||
|
||||
### Anmeldung mit einem Passwort
|
||||
|
||||
Ein initialer Administrator-Account kann optional konfiguriert werden und sorgt dafür, dass bestimmte Bereiche nur nach
|
||||
Ein initialer Administrator-Account kann optional konfiguriert werden und sorgt dafür, dass
|
||||
bestimmte Bereiche nur nach
|
||||
einem erfolgreichen Login erreichbar sind.
|
||||
|
||||
* `APP_SECURITY_ADMIN_USER`: Muss angegeben werden zur Aktivierung der Zugriffsbeschränkung.
|
||||
@ -114,27 +135,34 @@ Hier Beispiele für das Beispielpasswort `very-secret`:
|
||||
* `{bcrypt}$2y$05$CCkfsMr/wbTleMyjVIK8g.Aa3RCvrvoLXVAsL.f6KeouS88vXD9b6`
|
||||
* `{sha256}9a34717f0646b5e9cfcba70055de62edb026ff4f68671ba3db96aa29297d2df5f1a037d58c745657`
|
||||
|
||||
Wird kein Administrator-Passwort angegeben, wird ein zufälliger Wert generiert und beim Start der Anwendung in den Logs
|
||||
Wird kein Administrator-Passwort angegeben, wird ein zufälliger Wert generiert und beim Start der
|
||||
Anwendung in den Logs
|
||||
angezeigt.
|
||||
|
||||
#### Weitere (nicht administrative) Nutzer mit OpenID Connect
|
||||
|
||||
Die folgenden Konfigurationsparameter werden benötigt, um die Authentifizierung weiterer Benutzer an einen OIDC-Provider
|
||||
Die folgenden Konfigurationsparameter werden benötigt, um die Authentifizierung weiterer Benutzer an
|
||||
einen OIDC-Provider
|
||||
zu delegieren.
|
||||
Ein Admin-Benutzer muss dabei konfiguriert sein.
|
||||
|
||||
* `APP_SECURITY_ENABLE_OIDC`: Aktiviert die Nutzung von OpenID Connect. Damit sind weitere Parameter erforderlich
|
||||
* `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_CUSTOM_CLIENT_NAME`: Name. Wird beim zusätzlichen Loginbutton angezeigt.
|
||||
* `APP_SECURITY_ENABLE_OIDC`: Aktiviert die Nutzung von OpenID Connect. Damit sind weitere Parameter
|
||||
erforderlich
|
||||
* `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_CUSTOM_CLIENT_NAME`: Name. Wird beim zusätzlichen
|
||||
Loginbutton angezeigt.
|
||||
* `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_CUSTOM_CLIENT_ID`: Client-ID
|
||||
* `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_CUSTOM_CLIENT_SECRET`: Client-Secret
|
||||
* `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_CUSTOM_CLIENT_SCOPE[0]`: Hier sollte immer `openid` angegeben werden.
|
||||
* `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_CUSTOM_CLIENT_SCOPE[0]`: Hier sollte immer `openid`
|
||||
angegeben werden.
|
||||
* `SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_CUSTOM_ISSUER_URI`: Die URI des Providers,
|
||||
z.B. `https://auth.example.com/realm/example`
|
||||
* `SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_CUSTOM_USER_NAME_ATTRIBUTE`: Name des Attributes, welches den Benutzernamen
|
||||
* `SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_CUSTOM_USER_NAME_ATTRIBUTE`: Name des Attributes, welches
|
||||
den Benutzernamen
|
||||
enthält.
|
||||
Oft verwendet: `preferred_username`
|
||||
|
||||
Ist die Nutzung von OpenID Connect konfiguriert, erscheint ein zusätzlicher Login-Button zur Nutzung mit OpenID Connect
|
||||
Ist die Nutzung von OpenID Connect konfiguriert, erscheint ein zusätzlicher Login-Button zur Nutzung
|
||||
mit OpenID Connect
|
||||
und dem konfigurierten `CLIENT_NAME`.
|
||||
|
||||

|
||||
@ -145,62 +173,76 @@ zu finden.
|
||||
|
||||
#### Rollenbasierte Berechtigungen
|
||||
|
||||
Wird OpenID Connect verwendet, gibt es eine rollenbasierte Berechtigungszuweisung.
|
||||
Wird OpenID Connect verwendet, gibt es eine rollenbasierte Berechtigungszuweisung.
|
||||
|
||||
Die Standardrolle für neue OIDC-Benutzer kann mit der Option `APP_SECURITY_DEFAULT_USER_ROLE` festgelegt werden.
|
||||
Die Standardrolle für neue OIDC-Benutzer kann mit der Option `APP_SECURITY_DEFAULT_USER_ROLE`
|
||||
festgelegt werden.
|
||||
Mögliche Werte sind `user` oder `guest`. Standardwert ist `user`.
|
||||
|
||||
Benutzer mit der Rolle "Gast" sehen nur die Inhalte, die auch nicht angemeldete Benutzer sehen.
|
||||
|
||||
Hierdurch ist es möglich, einzelne Benutzer einzuschränken oder durch Änderung der Standardrolle auf `guest` nur
|
||||
Hierdurch ist es möglich, einzelne Benutzer einzuschränken oder durch Änderung der Standardrolle auf
|
||||
`guest` nur
|
||||
einzelne Benutzer als vollwertige Nutzer zuzulassen.
|
||||
|
||||

|
||||
|
||||
Benutzer werden nach dem Entfernen oder der Änderung der vergebenen Rolle automatisch abgemeldet und müssen sich neu anmelden.
|
||||
Benutzer werden nach dem Entfernen oder der Änderung der vergebenen Rolle automatisch abgemeldet und
|
||||
müssen sich neu anmelden.
|
||||
Sie bekommen dabei wieder die Standardrolle zugewiesen.
|
||||
|
||||
#### Auswirkungen auf den dargestellten Inhalt
|
||||
|
||||
Nur Administratoren haben Zugriff auf den Konfigurationsbereich, nur angemeldete Benutzer können die anonymisierte oder
|
||||
Nur Administratoren haben Zugriff auf den Konfigurationsbereich, nur angemeldete Benutzer können die
|
||||
anonymisierte oder
|
||||
pseudonymisierte Patienten-ID sowie den Qualitätsbericht von DNPM:DIP einsehen.
|
||||
|
||||
Wurde kein Administrator-Account konfiguriert, sind diese Inhalte generell nicht verfügbar.
|
||||
|
||||
### Tokenbasierte Authentifizierung für MTBFile-Endpunkt
|
||||
|
||||
Die Anwendung unterstützt das Erstellen und Nutzen einer tokenbasierten Authentifizierung für den MTB-File-Endpunkt.
|
||||
Die Anwendung unterstützt das Erstellen und Nutzen einer tokenbasierten Authentifizierung für den
|
||||
MTB-File-Endpunkt.
|
||||
|
||||
Dies kann mit der Umgebungsvariable `APP_SECURITY_ENABLE_TOKENS` aktiviert (`true` oder `false`) werden
|
||||
Dies kann mit der Umgebungsvariable `APP_SECURITY_ENABLE_TOKENS` aktiviert (`true` oder `false`)
|
||||
werden
|
||||
und ist als Standardeinstellung nicht aktiv.
|
||||
|
||||
Ist diese Einstellung aktiviert worden, ist es Administratoren möglich, Zugriffstokens für Onkostar zu erstellen, die
|
||||
Ist diese Einstellung aktiviert worden, ist es Administratoren möglich, Zugriffstokens für Onkostar
|
||||
zu erstellen, die
|
||||
zur Nutzung des MTB-File-Endpunkts eine HTTP-Basic-Authentifizierung voraussetzen.
|
||||
|
||||

|
||||
|
||||
In diesem Fall kann der Endpunkt für das Onkostar-Plugin **[onkostar-plugin-dnpmexport](https://github.com/CCC-MF/onkostar-plugin-dnpmexport)** wie folgt konfiguriert werden:
|
||||
In diesem Fall kann der Endpunkt für das Onkostar-Plugin *
|
||||
*[onkostar-plugin-dnpmexport](https://github.com/CCC-MF/onkostar-plugin-dnpmexport)** wie folgt
|
||||
konfiguriert werden:
|
||||
|
||||
```
|
||||
https://testonkostar:MTg1NTL...NGU4@etl.example.com/mtbfile
|
||||
```
|
||||
|
||||
Ist die Verwendung von Tokens aktiv, werden Anfragen ohne die Angabe der Token-Information abgelehnt.
|
||||
Ist die Verwendung von Tokens aktiv, werden Anfragen ohne die Angabe der Token-Information
|
||||
abgelehnt.
|
||||
|
||||
Alternativ kann eine Authentifizierung über Benutzername/Passwort oder OIDC erfolgen.
|
||||
|
||||
### Transformation von Werten
|
||||
|
||||
In Onkostar kann es vorkommen, dass ein Wert eines Merkmalskatalogs an einem Standort angepasst wurde und dadurch nicht dem Wert entspricht,
|
||||
In Onkostar kann es vorkommen, dass ein Wert eines Merkmalskatalogs an einem Standort angepasst
|
||||
wurde und dadurch nicht dem Wert entspricht,
|
||||
der von DNPM:DIP akzeptiert wird.
|
||||
|
||||
Diese Anwendung bietet daher die Möglichkeit, eine Transformation vorzunehmen. Hierzu muss der "Pfad" innerhalb des JSON-MTB-Files angegeben werden und
|
||||
Diese Anwendung bietet daher die Möglichkeit, eine Transformation vorzunehmen. Hierzu muss der "
|
||||
Pfad" innerhalb des JSON-MTB-Files angegeben werden und
|
||||
welcher Wert wie ersetzt werden soll.
|
||||
|
||||
Hier ein Beispiel für die erste (Index 0 - weitere dann mit 1,2, ...) Transformationsregel:
|
||||
|
||||
* `APP_TRANSFORMATIONS_0_PATH`: Pfad zum Wert in der JSON-MTB-Datei. Beispiel: `diagnoses[*].icd10.version` für **alle** Diagnosen
|
||||
* `APP_TRANSFORMATIONS_0_FROM`: Angabe des Werts, der ersetzt werden soll. Andere Werte bleiben dabei unverändert.
|
||||
* `APP_TRANSFORMATIONS_0_PATH`: Pfad zum Wert in der JSON-MTB-Datei. Beispiel:
|
||||
`diagnoses[*].icd10.version` für **alle** Diagnosen
|
||||
* `APP_TRANSFORMATIONS_0_FROM`: Angabe des Werts, der ersetzt werden soll. Andere Werte bleiben
|
||||
dabei unverändert.
|
||||
* `APP_TRANSFORMATIONS_0_TO`: Angabe des neuen Werts.
|
||||
|
||||
### Mögliche Endpunkte zur Datenübermittlung
|
||||
@ -215,53 +257,61 @@ Werden sowohl REST als auch Kafka-Endpunkt konfiguriert, wird nur der REST-Endpu
|
||||
Folgende Umgebungsvariablen müssen gesetzt sein, damit ein bwHC-MTB-File an DNPM:DIP gesendet wird:
|
||||
|
||||
* `APP_REST_URI`: URI der zu benutzenden API der Backend-Instanz. Zum Beispiel:
|
||||
* `http://localhost:9000/bwhc/etl/api` für **bwHC Backend**
|
||||
* `http://localhost:9000/api` für **dnpm:dip**
|
||||
* `http://localhost:9000/bwhc/etl/api` für **bwHC Backend**
|
||||
* `http://localhost:9000/api` für **dnpm:dip**
|
||||
* `APP_REST_USERNAME`: Basic-Auth-Benutzername für den REST-Endpunkt
|
||||
* `APP_REST_PASSWORD`: Basic-Auth-Passwort für den REST-Endpunkt
|
||||
* `APP_REST_IS_BWHC`: `true` für **bwHC Backend**, weglassen oder `false` für **dnpm:dip**
|
||||
|
||||
#### Kafka-Topics
|
||||
|
||||
Folgende Umgebungsvariablen müssen gesetzt sein, damit ein bwHC-MTB-File an ein Kafka-Topic übermittelt wird:
|
||||
Folgende Umgebungsvariablen müssen gesetzt sein, damit ein bwHC-MTB-File an ein Kafka-Topic
|
||||
übermittelt wird:
|
||||
|
||||
* `APP_KAFKA_OUTPUT_TOPIC`: Zu verwendendes Topic zum Versenden von Anfragen.
|
||||
Ersetzt ~~`APP_KAFKA_TOPIC`~~, **welches nach Version 0.10 entfernt wird**.
|
||||
* `APP_KAFKA_OUTPUT_RESPONSE_TOPIC`: Topic mit Antworten über den Erfolg des Versendens. Standardwert: `APP_KAFKA_TOPIC` mit Anhang "_response".
|
||||
Ersetzt ~~`APP_KAFKA_RESPONSE_TOPIC`~~, **welches nach Version 0.10 entfernt wird**.
|
||||
* `APP_KAFKA_GROUP_ID`: Kafka GroupID des Consumers. Standardwert: `APP_KAFKA_TOPIC` mit Anhang "_group".
|
||||
* `APP_KAFKA_OUTPUT_TOPIC`: Zu verwendendes Topic zum Versenden von Anfragen.
|
||||
* `APP_KAFKA_OUTPUT_RESPONSE_TOPIC`: Topic mit Antworten über den Erfolg des Versendens.
|
||||
Standardwert: `APP_KAFKA_TOPIC` mit Anhang "_response".
|
||||
* `APP_KAFKA_GROUP_ID`: Kafka GroupID des Consumers. Standardwert: `APP_KAFKA_TOPIC` mit Anhang "_
|
||||
group".
|
||||
* `APP_KAFKA_SERVERS`: Zu verwendende Kafka-Bootstrap-Server als kommagetrennte Liste
|
||||
|
||||
Wird keine Rückantwort über Apache Kafka empfangen und es gibt keine weitere Möglichkeit den Status festzustellen, verbleibt der Status auf `UNKNOWN`.
|
||||
Wird keine Rückantwort über Apache Kafka empfangen und es gibt keine weitere Möglichkeit den Status
|
||||
festzustellen, verbleibt der Status auf `UNKNOWN`.
|
||||
|
||||
Weitere Einstellungen können über die Parameter von Spring Kafka konfiguriert werden.
|
||||
|
||||
Lässt sich keine Verbindung zu dem Backend aufbauen, wird eine Rückantwort mit Status-Code `900` erwartet, welchen es
|
||||
Lässt sich keine Verbindung zu dem Backend aufbauen, wird eine Rückantwort mit Status-Code `900`
|
||||
erwartet, welchen es
|
||||
für HTTP nicht gibt.
|
||||
|
||||
Wird die Umgebungsvariable `APP_KAFKA_INPUT_TOPIC` gesetzt, kann eine Nachricht auch über dieses Kafka-Topic an den ETL-Prozessor übermittelt werden.
|
||||
Wird die Umgebungsvariable `APP_KAFKA_INPUT_TOPIC` gesetzt, kann eine Nachricht auch über dieses
|
||||
Kafka-Topic an den ETL-Prozessor übermittelt werden.
|
||||
|
||||
##### Retention Time
|
||||
|
||||
Generell werden in Apache Kafka alle Records entsprechend der Konfiguration vorgehalten.
|
||||
So wird ohne spezielle Konfiguration ein Record für 7 Tage in Apache Kafka gespeichert.
|
||||
Es sind innerhalb dieses Zeitraums auch alte Informationen weiterhin enthalten, wenn der Consent später abgelehnt wurde.
|
||||
Es sind innerhalb dieses Zeitraums auch alte Informationen weiterhin enthalten, wenn der Consent
|
||||
später abgelehnt wurde.
|
||||
|
||||
Durch eine entsprechende Konfiguration des Topics kann dies verhindert werden.
|
||||
|
||||
Beispiel - auszuführen innerhalb des Kafka-Containers: Löschen alter Records nach einem Tag
|
||||
|
||||
```
|
||||
kafka-configs.sh --bootstrap-server localhost:9092 --alter --topic test --add-config retention.ms=86400000
|
||||
```
|
||||
|
||||
##### Key based Retention
|
||||
|
||||
Möchten Sie hingegen immer nur die letzte Meldung für einen Patienten und eine Erkrankung in Apache Kafka vorhalten,
|
||||
Möchten Sie hingegen immer nur die letzte Meldung für einen Patienten und eine Erkrankung in Apache
|
||||
Kafka vorhalten,
|
||||
so ist die nachfolgend genannte Konfiguration der Kafka-Topics hilfreich.
|
||||
|
||||
|
||||
* `retention.ms`: Möglichst kurze Zeit in der alte Records noch erhalten bleiben, z.B. 10 Sekunden 10000
|
||||
* `cleanup.policy`: Löschen alter Records und Beibehalten des letzten Records zu einem Key [delete,compact]
|
||||
* `retention.ms`: Möglichst kurze Zeit in der alte Records noch erhalten bleiben, z.B. 10 Sekunden
|
||||
10000
|
||||
* `cleanup.policy`: Löschen alter Records und Beibehalten des letzten Records zu einem
|
||||
Key [delete,compact]
|
||||
|
||||
Beispiele für ein Topic `test`, hier bitte an die verwendeten Topics anpassen.
|
||||
|
||||
@ -270,17 +320,23 @@ kafka-configs.sh --bootstrap-server localhost:9092 --alter --topic test --add-co
|
||||
kafka-configs.sh --bootstrap-server localhost:9092 --alter --topic test --add-config cleanup.policy=[delete,compact]
|
||||
```
|
||||
|
||||
Da als Key eines Records die (pseudonymisierte) Patienten-ID verwendet wird, stehen mit obiger Konfiguration
|
||||
der Kafka-Topics nach 10 Sekunden nur noch der jeweils letzte Eintrag für den entsprechenden Key zur Verfügung.
|
||||
Da als Key eines Records die (pseudonymisierte) Patienten-ID verwendet wird, stehen mit obiger
|
||||
Konfiguration
|
||||
der Kafka-Topics nach 10 Sekunden nur noch der jeweils letzte Eintrag für den entsprechenden Key zur
|
||||
Verfügung.
|
||||
|
||||
Da der Key sowohl für die Records in Richtung DNPM:DIP, als auch für die Rückantwort identisch aufgebaut ist, lassen sich so
|
||||
auch im Falle eines Consent-Widerspruchs die enthaltenen Daten als auch die Offenlegung durch Verifikationsdaten in der
|
||||
Da der Key sowohl für die Records in Richtung DNPM:DIP, als auch für die Rückantwort identisch
|
||||
aufgebaut ist, lassen sich so
|
||||
auch im Falle eines Consent-Widerspruchs die enthaltenen Daten als auch die Offenlegung durch
|
||||
Verifikationsdaten in der
|
||||
Antwort effektiv verhindern, da diese nach 10 Sekunden gelöscht werden.
|
||||
|
||||
Es steht dann nur noch die jeweils letzten Information zur Verfügung, dass für einen Patienten/eine Erkrankung
|
||||
Es steht dann nur noch die jeweils letzten Information zur Verfügung, dass für einen Patienten/eine
|
||||
Erkrankung
|
||||
ein Consent-Widerspruch erfolgte.
|
||||
|
||||
Dieses Vorgehen empfiehlt sich, wenn Sie gespeicherte Records nachgelagert für andere Auswertungen verwenden möchten.
|
||||
Dieses Vorgehen empfiehlt sich, wenn Sie gespeicherte Records nachgelagert für andere Auswertungen
|
||||
verwenden möchten.
|
||||
|
||||
### Antworten und Statusauswertung
|
||||
|
||||
@ -292,10 +348,12 @@ Anfragen an das bwHC-Backend aus Versionen bis 0.9.x wurden wie folgt behandelt:
|
||||
| `HTTP 201` | `WARNING` |
|
||||
| `HTTP 400-...` | `ERROR` |
|
||||
|
||||
Dies konnte dazu führen, dass zwar mit einem `HTTP 201` geantwortet wurde, aber dennoch in der Issue-Liste die
|
||||
Dies konnte dazu führen, dass zwar mit einem `HTTP 201` geantwortet wurde, aber dennoch in der
|
||||
Issue-Liste die
|
||||
Severity `error` aufgetaucht ist.
|
||||
|
||||
Ab Version 0.10 wird die Issue-Liste der Antwort verwendet und die darion enthaltene höchste Severity-Stufe als Ergebnis verwendet.
|
||||
Ab Version 0.10 wird die Issue-Liste der Antwort verwendet und die darion enthaltene höchste
|
||||
Severity-Stufe als Ergebnis verwendet.
|
||||
|
||||
| Höchste Severity | Status |
|
||||
|------------------|-----------|
|
||||
@ -305,9 +363,10 @@ Ab Version 0.10 wird die Issue-Liste der Antwort verwendet und die darion enthal
|
||||
|
||||
## Docker-Images
|
||||
|
||||
Diese Anwendung ist auch als Docker-Image verfügbar: https://github.com/pcvolkmer/etl-processor/pkgs/container/etl-processor
|
||||
Diese Anwendung ist auch als Docker-Image
|
||||
verfügbar: https://github.com/pcvolkmer/etl-processor/pkgs/container/etl-processor
|
||||
|
||||
### Images lokal bauen
|
||||
### Images lokal bauen
|
||||
|
||||
```bash
|
||||
./gradlew bootBuildImage
|
||||
@ -315,20 +374,25 @@ Diese Anwendung ist auch als Docker-Image verfügbar: https://github.com/pcvolkm
|
||||
|
||||
### Integration eines eigenen Root CA Zertifikats
|
||||
|
||||
Wird eine eigene Root CA verwendet, die nicht offiziell signiert ist, wird es zu Problemen beim SSL-Handshake kommen, wenn z.B. gPAS zur Generierung von Pseudonymen verwendet wird.
|
||||
Wird eine eigene Root CA verwendet, die nicht offiziell signiert ist, wird es zu Problemen beim
|
||||
SSL-Handshake kommen, wenn z.B. gPAS zur Generierung von Pseudonymen verwendet wird.
|
||||
|
||||
Hier bietet es sich an, das Root CA Zertifikat in das Image zu integrieren.
|
||||
|
||||
#### Integration beim Bauen des Images
|
||||
|
||||
Hier muss die Zeile `"BP_EMBED_CERTS" to "true"` in der Datei `build.gradle.kts` verwendet werden und darf nicht als Kommentar verwendet werden.
|
||||
Hier muss die Zeile `"BP_EMBED_CERTS" to "true"` in der Datei `build.gradle.kts` verwendet werden
|
||||
und darf nicht als Kommentar verwendet werden.
|
||||
|
||||
Die PEM-Datei mit dem/den Root CA Zertifikat(en) muss dabei im vorbereiteten Verzeichnis [`bindings/ca-certificates`](bindings/ca-certificates) enthalten sein.
|
||||
Die PEM-Datei mit dem/den Root CA Zertifikat(en) muss dabei im vorbereiteten Verzeichnis [
|
||||
`bindings/ca-certificates`](bindings/ca-certificates) enthalten sein.
|
||||
|
||||
#### Integration zur Laufzeit
|
||||
|
||||
Hier muss die Umgebungsvariable `SERVICE_BINDING_ROOT` z.B. auf den Wert `/bindings` gesetzt sein.
|
||||
Zudem muss ein Verzeichnis `bindings/ca-certificates` - analog zum Verzeichnis [`bindings/ca-certificates`](bindings/ca-certificates) mit einer PEM-Datei als Docker-Volume eingebunden werden.
|
||||
Zudem muss ein Verzeichnis `bindings/ca-certificates` - analog zum Verzeichnis [
|
||||
`bindings/ca-certificates`](bindings/ca-certificates) mit einer PEM-Datei als Docker-Volume
|
||||
eingebunden werden.
|
||||
|
||||
Beispiel für Docker-Compose:
|
||||
|
||||
@ -343,12 +407,14 @@ Beispiel für Docker-Compose:
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
*Ausführen als Docker Container:*
|
||||
|
||||
```bash
|
||||
cd ./deploy
|
||||
cp env-sample.env .env
|
||||
```
|
||||
|
||||
Wenn gewünscht, Änderungen in der `.env` vornehmen.
|
||||
|
||||
```bash
|
||||
@ -357,15 +423,19 @@ docker compose up -d
|
||||
|
||||
### Einfaches Beispiel für ein eigenes Docker-Compose-File
|
||||
|
||||
Die Datei [`docs/docker-compose.yml`](docs/docker-compose.yml) zeigt eine einfache Konfiguration für REST-Requests basierend
|
||||
Die Datei [`docs/docker-compose.yml`](docs/docker-compose.yml) zeigt eine einfache Konfiguration für
|
||||
REST-Requests basierend
|
||||
auf Docker-Compose mit der gestartet werden kann.
|
||||
|
||||
### Betrieb hinter einem Reverse-Proxy
|
||||
|
||||
Die Anwendung verarbeitet `X-Forwarded`-HTTP-Header und kann daher auch hinter einem Reverse-Proxy betrieben werden.
|
||||
Die Anwendung verarbeitet `X-Forwarded`-HTTP-Header und kann daher auch hinter einem Reverse-Proxy
|
||||
betrieben werden.
|
||||
|
||||
Dabei werden, je nachdem welche Header durch den Reverse-Proxy gesendet werden auch Protokoll, Host oder auch Path-Präfix
|
||||
automatisch erkannt und verwendet werden. Dadurch ist z.B. eine abweichende Angabe des Pfads problemlos möglich.
|
||||
Dabei werden, je nachdem welche Header durch den Reverse-Proxy gesendet werden auch Protokoll, Host
|
||||
oder auch Path-Präfix
|
||||
automatisch erkannt und verwendet werden. Dadurch ist z.B. eine abweichende Angabe des Pfads
|
||||
problemlos möglich.
|
||||
|
||||
#### Beispiel *Traefik* (mit Docker-Labels):
|
||||
|
||||
@ -401,13 +471,17 @@ Das folgende Beispiel zeigt die Konfiguration einer _location_ in einer nginx-Ko
|
||||
|
||||
## Entwicklungssetup
|
||||
|
||||
Zum Starten einer lokalen Entwicklungs- und Testumgebung kann die beiliegende Datei `dev-compose.yml` verwendet werden.
|
||||
Zum Starten einer lokalen Entwicklungs- und Testumgebung kann die beiliegende Datei
|
||||
`dev-compose.yml` verwendet werden.
|
||||
Diese kann zur Nutzung der Datenbanken **MariaDB** als auch **PostgreSQL** angepasst werden.
|
||||
|
||||
Zur Nutzung von Apache Kafka muss dazu ein Eintrag im hosts-File vorgenommen werden und der Hostname `kafka` auf die lokale
|
||||
IP-Adresse verweisen. Ohne diese Einstellung ist eine Nutzung von Apache Kafka außerhalb der Docker-Umgebung nicht möglich.
|
||||
Zur Nutzung von Apache Kafka muss dazu ein Eintrag im hosts-File vorgenommen werden und der Hostname
|
||||
`kafka` auf die lokale
|
||||
IP-Adresse verweisen. Ohne diese Einstellung ist eine Nutzung von Apache Kafka außerhalb der
|
||||
Docker-Umgebung nicht möglich.
|
||||
|
||||
Beim Start der Anwendung mit dem Profil `dev` wird die in `dev-compose.yml` definierte Umgebung beim Start der
|
||||
Beim Start der Anwendung mit dem Profil `dev` wird die in `dev-compose.yml` definierte Umgebung beim
|
||||
Start der
|
||||
Anwendung mit gestartet:
|
||||
|
||||
```
|
||||
@ -419,4 +493,5 @@ Die Datei `application-dev.yml` enthält hierzu die Konfiguration für das Profi
|
||||
Beim Ausführen der Integrationstests wird eine Testdatenbank in einem Docker-Container gestartet.
|
||||
Siehe hier auch die Klasse `AbstractTestcontainerTest` unter `src/integrationTest`.
|
||||
|
||||
Ein einfaches Entwickler-Setup inklusive DNPM:DIP ist mit Hilfe von https://github.com/pcvolkmer/dnpmdip-devenv realisierbar.
|
||||
Ein einfaches Entwickler-Setup inklusive DNPM:DIP ist mit Hilfe
|
||||
von https://github.com/pcvolkmer/dnpmdip-devenv realisierbar.
|
@ -5,7 +5,7 @@ import org.springframework.boot.gradle.tasks.bundling.BootBuildImage
|
||||
|
||||
plugins {
|
||||
war
|
||||
id("org.springframework.boot") version "3.3.10"
|
||||
id("org.springframework.boot") version "3.5.3"
|
||||
id("io.spring.dependency-management") version "1.1.7"
|
||||
kotlin("jvm") version "1.9.25"
|
||||
kotlin("plugin.spring") version "1.9.25"
|
||||
@ -13,12 +13,12 @@ plugins {
|
||||
}
|
||||
|
||||
group = "dev.dnpm"
|
||||
version = "0.10.0"
|
||||
version = "0.11.0-SNAPSHOT"
|
||||
|
||||
var versions = mapOf(
|
||||
"bwhc-dto-java" to "0.4.0",
|
||||
"mtb-dto" to "0.1.0-SNAPSHOT",
|
||||
"hapi-fhir" to "7.6.0",
|
||||
"commons-compress" to "1.26.2",
|
||||
"mockito-kotlin" to "5.4.0",
|
||||
"archunit" to "1.3.0",
|
||||
// Webjars
|
||||
@ -49,9 +49,18 @@ configurations {
|
||||
compileOnly {
|
||||
extendsFrom(configurations.annotationProcessor.get())
|
||||
}
|
||||
|
||||
all {
|
||||
resolutionStrategy {
|
||||
cacheChangingModulesFor(5, "minutes")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
repositories {
|
||||
maven {
|
||||
url = uri("https://git.dnpm.dev/api/packages/public-snapshots/maven")
|
||||
}
|
||||
maven {
|
||||
url = uri("https://git.dnpm.dev/api/packages/public/maven")
|
||||
}
|
||||
@ -73,6 +82,7 @@ dependencies {
|
||||
implementation("commons-codec:commons-codec")
|
||||
implementation("io.projectreactor.kotlin:reactor-kotlin-extensions")
|
||||
implementation("de.ukw.ccc:bwhc-dto-java:${versions["bwhc-dto-java"]}")
|
||||
implementation("dev.pcvolkmer.mv64e:mtb-dto:${versions["mtb-dto"]}") { isChanging = true }
|
||||
implementation("ca.uhn.hapi.fhir:hapi-fhir-base:${versions["hapi-fhir"]}")
|
||||
implementation("ca.uhn.hapi.fhir:hapi-fhir-structures-r4:${versions["hapi-fhir"]}")
|
||||
implementation("org.apache.httpcomponents.client5:httpclient5")
|
||||
@ -80,6 +90,7 @@ dependencies {
|
||||
implementation("org.webjars:webjars-locator:${versions["webjars-locator"]}")
|
||||
implementation("org.webjars.npm:echarts:${versions["echarts"]}")
|
||||
implementation("org.webjars.npm:htmx.org:${versions["htmx.org"]}")
|
||||
implementation ("org.apache.commons:commons-math3:3.6.1")
|
||||
|
||||
runtimeOnly("org.mariadb.jdbc:mariadb-java-client")
|
||||
runtimeOnly("org.postgresql:postgresql")
|
||||
@ -99,10 +110,8 @@ dependencies {
|
||||
integrationTestImplementation("org.testcontainers:junit-jupiter")
|
||||
integrationTestImplementation("org.testcontainers:postgresql")
|
||||
integrationTestImplementation("com.tngtech.archunit:archunit:${versions["archunit"]}")
|
||||
integrationTestImplementation("net.sourceforge.htmlunit:htmlunit")
|
||||
integrationTestImplementation("org.htmlunit:htmlunit")
|
||||
integrationTestImplementation("org.springframework:spring-webflux")
|
||||
// Override dependency version from org.testcontainers:junit-jupiter - CVE-2024-26308, CVE-2024-25710
|
||||
integrationTestImplementation("org.apache.commons:commons-compress:${versions["commons-compress"]}")
|
||||
}
|
||||
|
||||
tasks.withType<KotlinCompile> {
|
||||
@ -119,8 +128,9 @@ tasks.withType<Test> {
|
||||
}
|
||||
}
|
||||
|
||||
task<Test>("integrationTest") {
|
||||
tasks.register<Test>("integrationTest") {
|
||||
description = "Runs integration tests"
|
||||
group = "verification"
|
||||
|
||||
testClassesDirs = sourceSets["integrationTest"].output.classesDirs
|
||||
classpath = sourceSets["integrationTest"].runtimeClasspath
|
||||
|
@ -23,6 +23,7 @@ import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.*
|
||||
import dev.dnpm.etl.processor.monitoring.RequestRepository
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.output.BwhcV1MtbFileRequest
|
||||
import dev.dnpm.etl.processor.output.MtbFileSender
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
@ -33,10 +34,10 @@ import org.mockito.kotlin.*
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc
|
||||
import org.springframework.boot.test.context.SpringBootTest
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.context.ApplicationContext
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.test.web.servlet.MockMvc
|
||||
import org.springframework.test.web.servlet.post
|
||||
@ -45,7 +46,7 @@ import org.testcontainers.junit.jupiter.Testcontainers
|
||||
@Testcontainers
|
||||
@ExtendWith(SpringExtension::class)
|
||||
@SpringBootTest
|
||||
@MockBean(MtbFileSender::class)
|
||||
@MockitoBean(types = [MtbFileSender::class])
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.rest.uri=http://example.com",
|
||||
@ -73,7 +74,7 @@ class EtlProcessorApplicationTests : AbstractTestcontainerTest() {
|
||||
)
|
||||
inner class TransformationTest {
|
||||
|
||||
@MockBean
|
||||
@MockitoBean
|
||||
private lateinit var mtbFileSender: MtbFileSender
|
||||
|
||||
@Autowired
|
||||
@ -91,7 +92,7 @@ class EtlProcessorApplicationTests : AbstractTestcontainerTest() {
|
||||
fun mtbFileIsTransformed() {
|
||||
doAnswer {
|
||||
MtbFileSender.Response(RequestStatus.SUCCESS)
|
||||
}.whenever(mtbFileSender).send(any<MtbFileSender.MtbFileRequest>())
|
||||
}.whenever(mtbFileSender).send(any<BwhcV1MtbFileRequest>())
|
||||
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
@ -134,9 +135,9 @@ class EtlProcessorApplicationTests : AbstractTestcontainerTest() {
|
||||
}
|
||||
}
|
||||
|
||||
val captor = argumentCaptor<MtbFileSender.MtbFileRequest>()
|
||||
val captor = argumentCaptor<BwhcV1MtbFileRequest>()
|
||||
verify(mtbFileSender).send(captor.capture())
|
||||
assertThat(captor.firstValue.mtbFile.diagnoses).hasSize(1).allMatch { diagnosis ->
|
||||
assertThat(captor.firstValue.content.diagnoses).hasSize(1).allMatch { diagnosis ->
|
||||
diagnosis.icd10.version == "2014"
|
||||
}
|
||||
}
|
||||
|
@ -26,9 +26,9 @@ import dev.dnpm.etl.processor.output.KafkaMtbFileSender
|
||||
import dev.dnpm.etl.processor.output.RestMtbFileSender
|
||||
import dev.dnpm.etl.processor.pseudonym.AnonymizingGenerator
|
||||
import dev.dnpm.etl.processor.pseudonym.GpasPseudonymGenerator
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import dev.dnpm.etl.processor.security.TokenRepository
|
||||
import dev.dnpm.etl.processor.security.TokenService
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.api.Test
|
||||
@ -36,24 +36,25 @@ import org.junit.jupiter.api.assertThrows
|
||||
import org.springframework.beans.factory.NoSuchBeanDefinitionException
|
||||
import org.springframework.boot.autoconfigure.kafka.KafkaAutoConfiguration
|
||||
import org.springframework.boot.test.context.SpringBootTest
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.boot.test.mock.mockito.MockBeans
|
||||
import org.springframework.context.ApplicationContext
|
||||
import org.springframework.retry.support.RetryTemplate
|
||||
import org.springframework.security.crypto.password.PasswordEncoder
|
||||
import org.springframework.security.provisioning.InMemoryUserDetailsManager
|
||||
import org.springframework.test.context.ContextConfiguration
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
|
||||
@SpringBootTest
|
||||
@ContextConfiguration(classes = [
|
||||
AppConfiguration::class,
|
||||
AppSecurityConfiguration::class,
|
||||
KafkaAutoConfiguration::class,
|
||||
AppKafkaConfiguration::class,
|
||||
AppRestConfiguration::class
|
||||
])
|
||||
@MockBean(ObjectMapper::class)
|
||||
@ContextConfiguration(
|
||||
classes = [
|
||||
AppConfiguration::class,
|
||||
AppSecurityConfiguration::class,
|
||||
KafkaAutoConfiguration::class,
|
||||
AppKafkaConfiguration::class,
|
||||
AppRestConfiguration::class
|
||||
]
|
||||
)
|
||||
@MockitoBean(types = [ObjectMapper::class])
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=BUILDIN",
|
||||
@ -86,7 +87,7 @@ class AppConfigurationTest {
|
||||
"app.kafka.group-id=test"
|
||||
]
|
||||
)
|
||||
@MockBean(RequestRepository::class)
|
||||
@MockitoBean(types = [RequestRepository::class])
|
||||
inner class AppConfigurationKafkaTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
@ -145,7 +146,7 @@ class AppConfigurationTest {
|
||||
"app.kafka.group-id=test"
|
||||
]
|
||||
)
|
||||
@MockBean(RequestProcessor::class)
|
||||
@MockitoBean(types = [RequestProcessor::class])
|
||||
inner class AppConfigurationUsingKafkaInputTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
@ -181,40 +182,7 @@ class AppConfigurationTest {
|
||||
@Nested
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=",
|
||||
"app.pseudonymizer=buildin",
|
||||
]
|
||||
)
|
||||
inner class AppConfigurationPseudonymizerBuildinTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
fun shouldUseConfiguredGenerator() {
|
||||
assertThat(context.getBean(AnonymizingGenerator::class.java)).isNotNull
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=",
|
||||
"app.pseudonymizer=gpas",
|
||||
]
|
||||
)
|
||||
inner class AppConfigurationPseudonymizerGpasTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
fun shouldUseConfiguredGenerator() {
|
||||
assertThat(context.getBean(GpasPseudonymGenerator::class.java)).isNotNull
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=buildin",
|
||||
"app.pseudonymizer=",
|
||||
"app.pseudonymize.generator=buildin"
|
||||
]
|
||||
)
|
||||
inner class AppConfigurationPseudonymizeGeneratorBuildinTest(private val context: ApplicationContext) {
|
||||
@ -229,8 +197,7 @@ class AppConfigurationTest {
|
||||
@Nested
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=gpas",
|
||||
"app.pseudonymizer=",
|
||||
"app.pseudonymize.generator=gpas"
|
||||
]
|
||||
)
|
||||
inner class AppConfigurationPseudonymizeGeneratorGpasTest(private val context: ApplicationContext) {
|
||||
@ -248,11 +215,13 @@ class AppConfigurationTest {
|
||||
"app.security.enable-tokens=true"
|
||||
]
|
||||
)
|
||||
@MockBeans(value = [
|
||||
MockBean(InMemoryUserDetailsManager::class),
|
||||
MockBean(PasswordEncoder::class),
|
||||
MockBean(TokenRepository::class)
|
||||
])
|
||||
@MockitoBean(
|
||||
types = [
|
||||
InMemoryUserDetailsManager::class,
|
||||
PasswordEncoder::class,
|
||||
TokenRepository::class
|
||||
]
|
||||
)
|
||||
inner class AppConfigurationTokenEnabledTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
@ -263,11 +232,13 @@ class AppConfigurationTest {
|
||||
}
|
||||
|
||||
@Nested
|
||||
@MockBeans(value = [
|
||||
MockBean(InMemoryUserDetailsManager::class),
|
||||
MockBean(PasswordEncoder::class),
|
||||
MockBean(TokenRepository::class)
|
||||
])
|
||||
@MockitoBean(
|
||||
types = [
|
||||
InMemoryUserDetailsManager::class,
|
||||
PasswordEncoder::class,
|
||||
TokenRepository::class
|
||||
]
|
||||
)
|
||||
inner class AppConfigurationTokenDisabledTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
|
@ -37,13 +37,13 @@ import org.mockito.kotlin.times
|
||||
import org.mockito.kotlin.verify
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.security.oauth2.client.registration.ClientRegistrationRepository
|
||||
import org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.anonymous
|
||||
import org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.user
|
||||
import org.springframework.test.context.ContextConfiguration
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.test.web.servlet.MockMvc
|
||||
import org.springframework.test.web.servlet.delete
|
||||
@ -57,7 +57,7 @@ import org.springframework.test.web.servlet.post
|
||||
AppSecurityConfiguration::class
|
||||
]
|
||||
)
|
||||
@MockBean(TokenRepository::class, RequestProcessor::class)
|
||||
@MockitoBean(types = [TokenRepository::class, RequestProcessor::class])
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=BUILDIN",
|
||||
@ -91,7 +91,7 @@ class MtbFileRestControllerTest {
|
||||
status { isAccepted() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any())
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -104,7 +104,7 @@ class MtbFileRestControllerTest {
|
||||
status { isAccepted() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any())
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -117,7 +117,7 @@ class MtbFileRestControllerTest {
|
||||
status { isUnauthorized() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, never()).processMtbFile(any())
|
||||
verify(requestProcessor, never()).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -130,7 +130,7 @@ class MtbFileRestControllerTest {
|
||||
status { isForbidden() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, never()).processMtbFile(any())
|
||||
verify(requestProcessor, never()).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -156,7 +156,7 @@ class MtbFileRestControllerTest {
|
||||
}
|
||||
|
||||
@Nested
|
||||
@MockBean(UserRoleRepository::class, ClientRegistrationRepository::class)
|
||||
@MockitoBean(types = [UserRoleRepository::class, ClientRegistrationRepository::class])
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=BUILDIN",
|
||||
@ -177,7 +177,7 @@ class MtbFileRestControllerTest {
|
||||
status { isAccepted() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any())
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -190,7 +190,7 @@ class MtbFileRestControllerTest {
|
||||
status { isAccepted() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any())
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
}
|
||||
|
||||
|
@ -27,8 +27,8 @@ import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.data.jdbc.DataJdbcTest
|
||||
import org.springframework.boot.test.autoconfigure.jdbc.AutoConfigureTestDatabase
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.transaction.annotation.Transactional
|
||||
import org.testcontainers.junit.jupiter.Testcontainers
|
||||
@ -39,7 +39,7 @@ import java.time.Instant
|
||||
@DataJdbcTest
|
||||
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
|
||||
@Transactional
|
||||
@MockBean(MtbFileSender::class)
|
||||
@MockitoBean(types = [MtbFileSender::class])
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=buildin",
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -47,9 +47,8 @@ class GpasPseudonymGeneratorTest {
|
||||
fun setup() {
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||
val gPasConfigProperties = GPasConfigProperties(
|
||||
"http://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate",
|
||||
"test",
|
||||
null,
|
||||
"https://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate",
|
||||
"test", "test2",
|
||||
null,
|
||||
null
|
||||
)
|
||||
@ -63,9 +62,15 @@ class GpasPseudonymGeneratorTest {
|
||||
fun shouldReturnExpectedPseudonym() {
|
||||
this.mockRestServiceServer.expect {
|
||||
method(HttpMethod.POST)
|
||||
requestTo("http://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate")
|
||||
requestTo("https://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate")
|
||||
}.andRespond {
|
||||
withStatus(HttpStatus.OK).body(getDummyResponseBody("1234", "test", "test1234ABCDEF567890"))
|
||||
withStatus(HttpStatus.OK).body(
|
||||
getDummyResponseBody(
|
||||
"1234",
|
||||
"test",
|
||||
"test1234ABCDEF567890"
|
||||
)
|
||||
)
|
||||
.createResponse(it)
|
||||
}
|
||||
|
||||
@ -76,7 +81,7 @@ class GpasPseudonymGeneratorTest {
|
||||
fun shouldThrowExceptionIfGpasNotAvailable() {
|
||||
this.mockRestServiceServer.expect {
|
||||
method(HttpMethod.POST)
|
||||
requestTo("http://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate")
|
||||
requestTo("https://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate")
|
||||
}.andRespond {
|
||||
withException(IOException("Simulated IO error")).createResponse(it)
|
||||
}
|
||||
@ -88,10 +93,13 @@ class GpasPseudonymGeneratorTest {
|
||||
fun shouldThrowExceptionIfGpasDoesNotReturn2xxResponse() {
|
||||
this.mockRestServiceServer.expect {
|
||||
method(HttpMethod.POST)
|
||||
requestTo("http://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate")
|
||||
requestTo("https://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate")
|
||||
}.andRespond {
|
||||
withStatus(HttpStatus.FOUND)
|
||||
.header(HttpHeaders.LOCATION, "https://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate")
|
||||
.header(
|
||||
HttpHeaders.LOCATION,
|
||||
"https://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate"
|
||||
)
|
||||
.createResponse(it)
|
||||
}
|
||||
|
||||
|
@ -31,8 +31,8 @@ import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.context.SpringBootTest
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.transaction.annotation.Transactional
|
||||
import org.testcontainers.junit.jupiter.Testcontainers
|
||||
@ -42,7 +42,7 @@ import java.time.Instant
|
||||
@ExtendWith(SpringExtension::class)
|
||||
@SpringBootTest
|
||||
@Transactional
|
||||
@MockBean(MtbFileSender::class)
|
||||
@MockitoBean(types = [MtbFileSender::class])
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=buildin",
|
||||
|
@ -19,8 +19,6 @@
|
||||
|
||||
package dev.dnpm.etl.processor.web
|
||||
|
||||
import com.gargoylesoftware.htmlunit.WebClient
|
||||
import com.gargoylesoftware.htmlunit.html.HtmlPage
|
||||
import dev.dnpm.etl.processor.config.AppConfiguration
|
||||
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
|
||||
import dev.dnpm.etl.processor.monitoring.ConnectionCheckResult
|
||||
@ -29,11 +27,13 @@ import dev.dnpm.etl.processor.monitoring.RestConnectionCheckService
|
||||
import dev.dnpm.etl.processor.output.MtbFileSender
|
||||
import dev.dnpm.etl.processor.pseudonym.Generator
|
||||
import dev.dnpm.etl.processor.security.Role
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import dev.dnpm.etl.processor.security.TokenService
|
||||
import dev.dnpm.etl.processor.services.TransformationService
|
||||
import dev.dnpm.etl.processor.security.UserRoleService
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import dev.dnpm.etl.processor.services.TransformationService
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.htmlunit.WebClient
|
||||
import org.htmlunit.html.HtmlPage
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.api.Test
|
||||
@ -46,7 +46,6 @@ import org.mockito.kotlin.verify
|
||||
import org.mockito.kotlin.whenever
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.http.HttpHeaders
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.http.MediaType.TEXT_EVENT_STREAM
|
||||
@ -55,6 +54,7 @@ import org.springframework.security.test.web.servlet.request.SecurityMockMvcRequ
|
||||
import org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.user
|
||||
import org.springframework.test.context.ContextConfiguration
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.test.web.reactive.server.WebTestClient
|
||||
import org.springframework.test.web.servlet.*
|
||||
@ -81,14 +81,16 @@ abstract class MockSink : Sinks.Many<Boolean>
|
||||
"app.pseudonymize.generator=BUILDIN"
|
||||
]
|
||||
)
|
||||
@MockBean(name = "configsUpdateProducer", classes = [MockSink::class])
|
||||
@MockBean(
|
||||
Generator::class,
|
||||
MtbFileSender::class,
|
||||
RequestProcessor::class,
|
||||
TransformationService::class,
|
||||
GPasConnectionCheckService::class,
|
||||
RestConnectionCheckService::class,
|
||||
@MockitoBean(name = "configsUpdateProducer", types = [MockSink::class])
|
||||
@MockitoBean(
|
||||
types = [
|
||||
Generator::class,
|
||||
MtbFileSender::class,
|
||||
RequestProcessor::class,
|
||||
TransformationService::class,
|
||||
GPasConnectionCheckService::class,
|
||||
RestConnectionCheckService::class
|
||||
]
|
||||
)
|
||||
class ConfigControllerTest {
|
||||
|
||||
@ -143,8 +145,10 @@ class ConfigControllerTest {
|
||||
"app.security.admin-user=admin"
|
||||
]
|
||||
)
|
||||
@MockBean(
|
||||
TokenService::class
|
||||
@MockitoBean(
|
||||
types = [
|
||||
TokenService::class
|
||||
]
|
||||
)
|
||||
inner class WithTokensEnabled {
|
||||
private lateinit var tokenService: TokenService
|
||||
@ -252,8 +256,10 @@ class ConfigControllerTest {
|
||||
"app.security.admin-password={noop}very-secret"
|
||||
]
|
||||
)
|
||||
@MockBean(
|
||||
UserRoleService::class
|
||||
@MockitoBean(
|
||||
types = [
|
||||
UserRoleService::class
|
||||
]
|
||||
)
|
||||
inner class WithUserRolesEnabled {
|
||||
private lateinit var userRoleService: UserRoleService
|
||||
|
@ -19,8 +19,6 @@
|
||||
|
||||
package dev.dnpm.etl.processor.web
|
||||
|
||||
import com.gargoylesoftware.htmlunit.WebClient
|
||||
import com.gargoylesoftware.htmlunit.html.HtmlPage
|
||||
import dev.dnpm.etl.processor.*
|
||||
import dev.dnpm.etl.processor.config.AppConfiguration
|
||||
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
|
||||
@ -30,6 +28,8 @@ import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.monitoring.RequestType
|
||||
import dev.dnpm.etl.processor.services.RequestService
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.htmlunit.WebClient
|
||||
import org.htmlunit.html.HtmlPage
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.api.Test
|
||||
@ -40,13 +40,13 @@ import org.mockito.kotlin.any
|
||||
import org.mockito.kotlin.whenever
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.data.domain.Page
|
||||
import org.springframework.data.domain.PageImpl
|
||||
import org.springframework.data.domain.Pageable
|
||||
import org.springframework.security.test.context.support.WithMockUser
|
||||
import org.springframework.test.context.ContextConfiguration
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.test.web.servlet.MockMvc
|
||||
import org.springframework.test.web.servlet.get
|
||||
@ -71,8 +71,8 @@ import java.util.*
|
||||
"app.security.admin-password={noop}very-secret"
|
||||
]
|
||||
)
|
||||
@MockBean(
|
||||
RequestService::class
|
||||
@MockitoBean(
|
||||
types = [RequestService::class]
|
||||
)
|
||||
class HomeControllerTest {
|
||||
|
||||
|
@ -19,21 +19,21 @@
|
||||
|
||||
package dev.dnpm.etl.processor.web
|
||||
|
||||
import com.gargoylesoftware.htmlunit.WebClient
|
||||
import com.gargoylesoftware.htmlunit.html.HtmlPage
|
||||
import dev.dnpm.etl.processor.config.AppConfiguration
|
||||
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
|
||||
import dev.dnpm.etl.processor.security.TokenService
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.htmlunit.WebClient
|
||||
import org.htmlunit.html.HtmlPage
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.test.context.ContextConfiguration
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.test.web.servlet.MockMvc
|
||||
import org.springframework.test.web.servlet.get
|
||||
@ -56,8 +56,8 @@ import org.springframework.test.web.servlet.htmlunit.MockMvcWebClientBuilder
|
||||
"app.security.enable-tokens=true"
|
||||
]
|
||||
)
|
||||
@MockBean(
|
||||
TokenService::class,
|
||||
@MockitoBean(
|
||||
types = [TokenService::class]
|
||||
)
|
||||
class LoginControllerTest {
|
||||
|
||||
|
@ -19,9 +19,9 @@
|
||||
|
||||
package dev.dnpm.etl.processor.web
|
||||
|
||||
import com.gargoylesoftware.htmlunit.WebClient
|
||||
import dev.dnpm.etl.processor.config.AppConfiguration
|
||||
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
|
||||
import org.htmlunit.WebClient
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
|
@ -41,10 +41,10 @@ import org.mockito.kotlin.doAnswer
|
||||
import org.mockito.kotlin.whenever
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.http.MediaType.TEXT_EVENT_STREAM
|
||||
import org.springframework.test.context.ContextConfiguration
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.test.web.reactive.server.WebTestClient
|
||||
import org.springframework.test.web.servlet.MockMvc
|
||||
@ -74,8 +74,8 @@ import java.time.temporal.ChronoUnit
|
||||
"app.security.admin-password={noop}very-secret"
|
||||
]
|
||||
)
|
||||
@MockBean(
|
||||
RequestService::class
|
||||
@MockitoBean(
|
||||
types = [RequestService::class]
|
||||
)
|
||||
class StatisticsRestControllerTest {
|
||||
|
||||
|
@ -23,4 +23,6 @@ public interface Generator {
|
||||
|
||||
String generate(String id);
|
||||
|
||||
String generateGenomDeTan(String id);
|
||||
|
||||
}
|
||||
|
@ -38,30 +38,41 @@ public class GpasPseudonymGenerator implements Generator {
|
||||
|
||||
private final static FhirContext r4Context = FhirContext.forR4();
|
||||
private final String gPasUrl;
|
||||
private final String psnTargetDomain;
|
||||
private final HttpHeaders httpHeader;
|
||||
private final RetryTemplate retryTemplate;
|
||||
private final Logger log = LoggerFactory.getLogger(GpasPseudonymGenerator.class);
|
||||
|
||||
private final RestTemplate restTemplate;
|
||||
private final @NotNull String genomDeDomain;
|
||||
private final @NotNull String psnTargetDomain;
|
||||
|
||||
public GpasPseudonymGenerator(GPasConfigProperties gpasCfg, RetryTemplate retryTemplate, RestTemplate restTemplate) {
|
||||
public GpasPseudonymGenerator(GPasConfigProperties gpasCfg, RetryTemplate retryTemplate,
|
||||
RestTemplate restTemplate) {
|
||||
this.retryTemplate = retryTemplate;
|
||||
this.restTemplate = restTemplate;
|
||||
this.gPasUrl = gpasCfg.getUri();
|
||||
this.psnTargetDomain = gpasCfg.getTarget();
|
||||
this.genomDeDomain = gpasCfg.getGenomDeDomain();
|
||||
httpHeader = getHttpHeaders(gpasCfg.getUsername(), gpasCfg.getPassword());
|
||||
|
||||
log.debug(String.format("%s has been initialized", this.getClass().getName()));
|
||||
log.debug("{} has been initialized", this.getClass().getName());
|
||||
|
||||
}
|
||||
|
||||
@Override
|
||||
public String generate(String id) {
|
||||
var gPasRequestBody = getGpasRequestBody(id);
|
||||
return generate(id, psnTargetDomain);
|
||||
}
|
||||
|
||||
@Override
|
||||
public String generateGenomDeTan(String id) {
|
||||
return generate(id, genomDeDomain);
|
||||
}
|
||||
|
||||
protected String generate(String id, String targetDomain) {
|
||||
var gPasRequestBody = getGpasRequestBody(id, targetDomain);
|
||||
var responseEntity = getGpasPseudonym(gPasRequestBody);
|
||||
var gPasPseudonymResult = (Parameters) r4Context.newJsonParser()
|
||||
.parseResource(responseEntity.getBody());
|
||||
.parseResource(responseEntity.getBody());
|
||||
|
||||
return unwrapPseudonym(gPasPseudonymResult);
|
||||
}
|
||||
@ -75,9 +86,9 @@ public class GpasPseudonymGenerator implements Generator {
|
||||
}
|
||||
|
||||
final var identifier = (Identifier) parameters.get().getPart().stream()
|
||||
.filter(a -> a.getName().equals("pseudonym"))
|
||||
.findFirst()
|
||||
.orElseGet(ParametersParameterComponent::new).getValue();
|
||||
.filter(a -> a.getName().equals("pseudonym"))
|
||||
.findFirst()
|
||||
.orElseGet(ParametersParameterComponent::new).getValue();
|
||||
|
||||
// pseudonym
|
||||
return sanitizeValue(identifier.getValue());
|
||||
@ -106,8 +117,8 @@ public class GpasPseudonymGenerator implements Generator {
|
||||
|
||||
try {
|
||||
responseEntity = retryTemplate.execute(
|
||||
ctx -> restTemplate.exchange(gPasUrl, HttpMethod.POST, requestEntity,
|
||||
String.class));
|
||||
ctx -> restTemplate.exchange(gPasUrl, HttpMethod.POST, requestEntity,
|
||||
String.class));
|
||||
|
||||
if (responseEntity.getStatusCode().is2xxSuccessful()) {
|
||||
log.debug("API request succeeded. Response: {}", responseEntity.getStatusCode());
|
||||
@ -119,16 +130,16 @@ public class GpasPseudonymGenerator implements Generator {
|
||||
return responseEntity;
|
||||
} catch (Exception unexpected) {
|
||||
throw new PseudonymRequestFailed(
|
||||
"API request due unexpected error unsuccessful gPas unsuccessful.", unexpected);
|
||||
"API request due unexpected error unsuccessful gPas unsuccessful.", unexpected);
|
||||
}
|
||||
}
|
||||
|
||||
protected String getGpasRequestBody(String id) {
|
||||
protected String getGpasRequestBody(String id, String targetDomain) {
|
||||
var requestParameters = new Parameters();
|
||||
requestParameters.addParameter().setName("target")
|
||||
.setValue(new StringType().setValue(psnTargetDomain));
|
||||
.setValue(new StringType().setValue(targetDomain));
|
||||
requestParameters.addParameter().setName("original")
|
||||
.setValue(new StringType().setValue(id));
|
||||
.setValue(new StringType().setValue(id));
|
||||
final IParser iParser = r4Context.newJsonParser();
|
||||
return iParser.encodeResourceToString(requestParameters);
|
||||
}
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -21,16 +21,10 @@ package dev.dnpm.etl.processor.config
|
||||
|
||||
import dev.dnpm.etl.processor.security.Role
|
||||
import org.springframework.boot.context.properties.ConfigurationProperties
|
||||
import org.springframework.boot.context.properties.DeprecatedConfigurationProperty
|
||||
|
||||
@ConfigurationProperties(AppConfigProperties.NAME)
|
||||
data class AppConfigProperties(
|
||||
var bwhcUri: String?,
|
||||
@get:DeprecatedConfigurationProperty(
|
||||
reason = "Deprecated in favor of 'app.pseudonymize.generator'",
|
||||
replacement = "app.pseudonymize.generator"
|
||||
)
|
||||
var pseudonymizer: PseudonymGenerator = PseudonymGenerator.BUILDIN,
|
||||
var transformations: List<TransformationProperties> = listOf(),
|
||||
var maxRetryAttempts: Int = 3,
|
||||
var duplicationDetection: Boolean = true
|
||||
@ -54,12 +48,9 @@ data class PseudonymizeConfigProperties(
|
||||
data class GPasConfigProperties(
|
||||
val uri: String?,
|
||||
val target: String = "etl-processor",
|
||||
val genomDeDomain: String = "ccdn",
|
||||
val username: String?,
|
||||
val password: String?,
|
||||
@get:DeprecatedConfigurationProperty(
|
||||
reason = "Deprecated in favor of including Root CA"
|
||||
)
|
||||
val sslCaLocation: String?
|
||||
) {
|
||||
companion object {
|
||||
const val NAME = "app.pseudonymize.gpas"
|
||||
@ -82,18 +73,8 @@ data class RestTargetProperties(
|
||||
data class KafkaProperties(
|
||||
val inputTopic: String?,
|
||||
val outputTopic: String = "etl-processor",
|
||||
@get:DeprecatedConfigurationProperty(
|
||||
reason = "Deprecated",
|
||||
replacement = "outputTopic"
|
||||
)
|
||||
val topic: String = outputTopic,
|
||||
val outputResponseTopic: String = "${outputTopic}_response",
|
||||
@get:DeprecatedConfigurationProperty(
|
||||
reason = "Deprecated",
|
||||
replacement = "outputResponseTopic"
|
||||
)
|
||||
val responseTopic: String = outputResponseTopic,
|
||||
val groupId: String = "${topic}_group",
|
||||
val groupId: String = "${outputTopic}_group",
|
||||
val servers: String = ""
|
||||
) {
|
||||
companion object {
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -32,12 +32,6 @@ import dev.dnpm.etl.processor.security.TokenRepository
|
||||
import dev.dnpm.etl.processor.security.TokenService
|
||||
import dev.dnpm.etl.processor.services.Transformation
|
||||
import dev.dnpm.etl.processor.services.TransformationService
|
||||
import org.apache.hc.client5.http.impl.classic.HttpClients
|
||||
import org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager
|
||||
import org.apache.hc.client5.http.socket.ConnectionSocketFactory
|
||||
import org.apache.hc.client5.http.socket.PlainConnectionSocketFactory
|
||||
import org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory
|
||||
import org.apache.hc.core5.http.config.RegistryBuilder
|
||||
import org.slf4j.LoggerFactory
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty
|
||||
@ -45,7 +39,6 @@ import org.springframework.boot.context.properties.EnableConfigurationProperties
|
||||
import org.springframework.context.annotation.Bean
|
||||
import org.springframework.context.annotation.Configuration
|
||||
import org.springframework.data.jdbc.repository.config.AbstractJdbcConfiguration
|
||||
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory
|
||||
import org.springframework.retry.RetryCallback
|
||||
import org.springframework.retry.RetryContext
|
||||
import org.springframework.retry.RetryListener
|
||||
@ -58,13 +51,6 @@ import org.springframework.security.provisioning.InMemoryUserDetailsManager
|
||||
import org.springframework.web.client.HttpClientErrorException
|
||||
import org.springframework.web.client.RestTemplate
|
||||
import reactor.core.publisher.Sinks
|
||||
import java.io.BufferedInputStream
|
||||
import java.io.FileInputStream
|
||||
import java.security.KeyStore
|
||||
import java.security.cert.CertificateFactory
|
||||
import java.security.cert.X509Certificate
|
||||
import javax.net.ssl.SSLContext
|
||||
import javax.net.ssl.TrustManagerFactory
|
||||
import kotlin.time.Duration.Companion.seconds
|
||||
import kotlin.time.toJavaDuration
|
||||
|
||||
@ -90,18 +76,6 @@ class AppConfiguration {
|
||||
@ConditionalOnProperty(value = ["app.pseudonymize.generator"], havingValue = "GPAS")
|
||||
@Bean
|
||||
fun gpasPseudonymGenerator(configProperties: GPasConfigProperties, retryTemplate: RetryTemplate, restTemplate: RestTemplate): Generator {
|
||||
try {
|
||||
if (!configProperties.sslCaLocation.isNullOrBlank()) {
|
||||
return GpasPseudonymGenerator(
|
||||
configProperties,
|
||||
retryTemplate,
|
||||
createCustomGpasRestTemplate(configProperties)
|
||||
)
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
throw RuntimeException(e)
|
||||
}
|
||||
|
||||
return GpasPseudonymGenerator(configProperties, retryTemplate, restTemplate)
|
||||
}
|
||||
|
||||
@ -111,92 +85,6 @@ class AppConfiguration {
|
||||
return AnonymizingGenerator()
|
||||
}
|
||||
|
||||
@ConditionalOnProperty(value = ["app.pseudonymizer"], havingValue = "GPAS")
|
||||
@ConditionalOnMissingBean
|
||||
@Bean
|
||||
fun gpasPseudonymGeneratorOnDeprecatedProperty(configProperties: GPasConfigProperties, retryTemplate: RetryTemplate, restTemplate: RestTemplate): Generator {
|
||||
try {
|
||||
if (!configProperties.sslCaLocation.isNullOrBlank()) {
|
||||
return GpasPseudonymGenerator(
|
||||
configProperties,
|
||||
retryTemplate,
|
||||
createCustomGpasRestTemplate(configProperties)
|
||||
)
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
throw RuntimeException(e)
|
||||
}
|
||||
|
||||
return GpasPseudonymGenerator(configProperties, retryTemplate, restTemplate)
|
||||
}
|
||||
|
||||
private fun createCustomGpasRestTemplate(configProperties: GPasConfigProperties): RestTemplate {
|
||||
fun getSslContext(certificateLocation: String): SSLContext? {
|
||||
val ks = KeyStore.getInstance(KeyStore.getDefaultType())
|
||||
|
||||
val fis = FileInputStream(certificateLocation)
|
||||
val ca = CertificateFactory.getInstance("X.509")
|
||||
.generateCertificate(BufferedInputStream(fis)) as X509Certificate
|
||||
|
||||
ks.load(null, null)
|
||||
ks.setCertificateEntry(1.toString(), ca)
|
||||
|
||||
val tmf = TrustManagerFactory.getInstance(
|
||||
TrustManagerFactory.getDefaultAlgorithm()
|
||||
)
|
||||
tmf.init(ks)
|
||||
|
||||
val sslContext = SSLContext.getInstance("TLS")
|
||||
sslContext.init(null, tmf.trustManagers, null)
|
||||
|
||||
return sslContext
|
||||
}
|
||||
|
||||
fun getCustomRestTemplate(customSslContext: SSLContext): RestTemplate {
|
||||
val sslsf = SSLConnectionSocketFactory(customSslContext)
|
||||
val socketFactoryRegistry = RegistryBuilder.create<ConnectionSocketFactory>()
|
||||
.register("https", sslsf).register("http", PlainConnectionSocketFactory()).build()
|
||||
|
||||
val connectionManager = BasicHttpClientConnectionManager(
|
||||
socketFactoryRegistry
|
||||
)
|
||||
val httpClient = HttpClients.custom()
|
||||
.setConnectionManager(connectionManager).build()
|
||||
|
||||
val requestFactory = HttpComponentsClientHttpRequestFactory(
|
||||
httpClient
|
||||
)
|
||||
return RestTemplate(requestFactory)
|
||||
}
|
||||
|
||||
try {
|
||||
if (!configProperties.sslCaLocation.isNullOrBlank()) {
|
||||
val customSslContext = getSslContext(configProperties.sslCaLocation)
|
||||
logger.warn(
|
||||
String.format(
|
||||
"%s has been initialized with SSL certificate %s. This is deprecated in favor of including Root CA.",
|
||||
this.javaClass.name, configProperties.sslCaLocation
|
||||
)
|
||||
)
|
||||
|
||||
if (customSslContext != null) {
|
||||
return getCustomRestTemplate(customSslContext)
|
||||
}
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
throw RuntimeException(e)
|
||||
}
|
||||
|
||||
throw RuntimeException("Custom SSL configuration for gPAS not usable")
|
||||
}
|
||||
|
||||
@ConditionalOnProperty(value = ["app.pseudonymizer"], havingValue = "BUILDIN")
|
||||
@ConditionalOnMissingBean
|
||||
@Bean
|
||||
fun buildinPseudonymGeneratorOnDeprecatedProperty(): Generator {
|
||||
return AnonymizingGenerator()
|
||||
}
|
||||
|
||||
@Bean
|
||||
fun pseudonymizeService(
|
||||
generator: Generator,
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -71,7 +71,7 @@ class AppKafkaConfiguration {
|
||||
kafkaProperties: KafkaProperties,
|
||||
kafkaResponseProcessor: KafkaResponseProcessor
|
||||
): KafkaMessageListenerContainer<String, String> {
|
||||
val containerProperties = ContainerProperties(kafkaProperties.responseTopic)
|
||||
val containerProperties = ContainerProperties(kafkaProperties.outputResponseTopic)
|
||||
containerProperties.messageListener = kafkaResponseProcessor
|
||||
return KafkaMessageListenerContainer(consumerFactory, containerProperties)
|
||||
}
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2023 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -87,9 +87,14 @@ class AppSecurityConfiguration(
|
||||
|
||||
@Bean
|
||||
@ConditionalOnProperty(value = ["app.security.enable-oidc"], havingValue = "true")
|
||||
fun filterChainOidc(http: HttpSecurity, passwordEncoder: PasswordEncoder, userRoleRepository: UserRoleRepository, sessionRegistry: SessionRegistry): SecurityFilterChain {
|
||||
fun filterChainOidc(
|
||||
http: HttpSecurity,
|
||||
passwordEncoder: PasswordEncoder,
|
||||
userRoleRepository: UserRoleRepository,
|
||||
sessionRegistry: SessionRegistry
|
||||
): SecurityFilterChain {
|
||||
http {
|
||||
authorizeRequests {
|
||||
authorizeHttpRequests {
|
||||
authorize("/configs/**", hasRole("ADMIN"))
|
||||
authorize("/mtbfile/**", hasAnyRole("MTBFILE", "ADMIN", "USER"))
|
||||
authorize("/report/**", hasAnyRole("ADMIN", "USER"))
|
||||
@ -127,13 +132,22 @@ class AppSecurityConfiguration(
|
||||
|
||||
@Bean
|
||||
@ConditionalOnProperty(value = ["app.security.enable-oidc"], havingValue = "true")
|
||||
fun grantedAuthoritiesMapper(userRoleRepository: UserRoleRepository, appSecurityConfigProperties: SecurityConfigProperties): GrantedAuthoritiesMapper {
|
||||
fun grantedAuthoritiesMapper(
|
||||
userRoleRepository: UserRoleRepository,
|
||||
appSecurityConfigProperties: SecurityConfigProperties
|
||||
): GrantedAuthoritiesMapper {
|
||||
return GrantedAuthoritiesMapper { grantedAuthority ->
|
||||
grantedAuthority.filterIsInstance<OidcUserAuthority>()
|
||||
.onEach {
|
||||
val userRole = userRoleRepository.findByUsername(it.userInfo.preferredUsername)
|
||||
if (userRole.isEmpty) {
|
||||
userRoleRepository.save(UserRole(null, it.userInfo.preferredUsername, appSecurityConfigProperties.defaultNewUserRole))
|
||||
userRoleRepository.save(
|
||||
UserRole(
|
||||
null,
|
||||
it.userInfo.preferredUsername,
|
||||
appSecurityConfigProperties.defaultNewUserRole
|
||||
)
|
||||
)
|
||||
}
|
||||
}
|
||||
.map {
|
||||
@ -147,7 +161,7 @@ class AppSecurityConfiguration(
|
||||
@ConditionalOnProperty(value = ["app.security.enable-oidc"], havingValue = "false", matchIfMissing = true)
|
||||
fun filterChain(http: HttpSecurity, passwordEncoder: PasswordEncoder): SecurityFilterChain {
|
||||
http {
|
||||
authorizeRequests {
|
||||
authorizeHttpRequests {
|
||||
authorize("/configs/**", hasRole("ADMIN"))
|
||||
authorize("/mtbfile/**", hasAnyRole("MTBFILE", "ADMIN"))
|
||||
authorize("/report/**", hasRole("ADMIN"))
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -22,11 +22,13 @@ package dev.dnpm.etl.processor.input
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.Consent
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import dev.dnpm.etl.processor.CustomMediaType
|
||||
import dev.dnpm.etl.processor.PatientId
|
||||
import dev.dnpm.etl.processor.RequestId
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import org.apache.kafka.clients.consumer.ConsumerRecord
|
||||
import org.slf4j.LoggerFactory
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.kafka.listener.MessageListener
|
||||
|
||||
class KafkaInputListener(
|
||||
@ -35,10 +37,29 @@ class KafkaInputListener(
|
||||
) : MessageListener<String, String> {
|
||||
private val logger = LoggerFactory.getLogger(KafkaInputListener::class.java)
|
||||
|
||||
override fun onMessage(data: ConsumerRecord<String, String>) {
|
||||
val mtbFile = objectMapper.readValue(data.value(), MtbFile::class.java)
|
||||
override fun onMessage(record: ConsumerRecord<String, String>) {
|
||||
when (guessMimeType(record)) {
|
||||
MediaType.APPLICATION_JSON_VALUE -> handleBwhcMessage(record)
|
||||
CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE -> handleDnpmV2Message(record)
|
||||
else -> {
|
||||
/* ignore other messages */
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private fun guessMimeType(record: ConsumerRecord<String, String>): String {
|
||||
if (record.headers().headers("contentType").toList().isEmpty()) {
|
||||
// Fallback if no contentType set (old behavior)
|
||||
return MediaType.APPLICATION_JSON_VALUE
|
||||
}
|
||||
|
||||
return record.headers().headers("contentType")?.firstOrNull()?.value().contentToString()
|
||||
}
|
||||
|
||||
private fun handleBwhcMessage(record: ConsumerRecord<String, String>) {
|
||||
val mtbFile = objectMapper.readValue(record.value(), MtbFile::class.java)
|
||||
val patientId = PatientId(mtbFile.patient.id)
|
||||
val firstRequestIdHeader = data.headers().headers("requestId")?.firstOrNull()
|
||||
val firstRequestIdHeader = record.headers().headers("requestId")?.firstOrNull()
|
||||
val requestId = if (null != firstRequestIdHeader) {
|
||||
RequestId(String(firstRequestIdHeader.value()))
|
||||
} else {
|
||||
@ -61,4 +82,10 @@ class KafkaInputListener(
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private fun handleDnpmV2Message(record: ConsumerRecord<String, String>) {
|
||||
// Do not handle DNPM-V2 for now
|
||||
logger.warn("Ignoring MTB File in DNPM V2 format: Not implemented yet")
|
||||
}
|
||||
|
||||
}
|
||||
|
@ -21,9 +21,12 @@ package dev.dnpm.etl.processor.input
|
||||
|
||||
import de.ukw.ccc.bwhc.dto.Consent
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import dev.dnpm.etl.processor.CustomMediaType
|
||||
import dev.dnpm.etl.processor.PatientId
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||
import org.slf4j.LoggerFactory
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.http.ResponseEntity
|
||||
import org.springframework.web.bind.annotation.*
|
||||
|
||||
@ -40,19 +43,26 @@ class MtbFileRestController(
|
||||
return ResponseEntity.ok("Test")
|
||||
}
|
||||
|
||||
@PostMapping
|
||||
@PostMapping( consumes = [ MediaType.APPLICATION_JSON_VALUE ] )
|
||||
fun mtbFile(@RequestBody mtbFile: MtbFile): ResponseEntity<Unit> {
|
||||
if (mtbFile.consent.status == Consent.Status.ACTIVE) {
|
||||
logger.debug("Accepted MTB File for processing")
|
||||
logger.debug("Accepted MTB File (bwHC V1) for processing")
|
||||
requestProcessor.processMtbFile(mtbFile)
|
||||
} else {
|
||||
logger.debug("Accepted MTB File and process deletion")
|
||||
logger.debug("Accepted MTB File (bwHC V1) and process deletion")
|
||||
val patientId = PatientId(mtbFile.patient.id)
|
||||
requestProcessor.processDeletion(patientId)
|
||||
}
|
||||
return ResponseEntity.accepted().build()
|
||||
}
|
||||
|
||||
@PostMapping( consumes = [ CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE] )
|
||||
fun mtbFile(@RequestBody mtbFile: Mtb): ResponseEntity<Unit> {
|
||||
logger.debug("Accepted MTB File (DNPM V2) for processing")
|
||||
requestProcessor.processMtbFile(mtbFile)
|
||||
return ResponseEntity.accepted().build()
|
||||
}
|
||||
|
||||
@DeleteMapping(path = ["{patientId}"])
|
||||
fun deleteData(@PathVariable patientId: String): ResponseEntity<Unit> {
|
||||
logger.debug("Accepted patient ID to process deletion")
|
||||
@ -60,4 +70,4 @@ class MtbFileRestController(
|
||||
return ResponseEntity.accepted().build()
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -22,10 +22,12 @@ package dev.dnpm.etl.processor.output
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.Consent
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import dev.dnpm.etl.processor.RequestId
|
||||
import dev.dnpm.etl.processor.CustomMediaType
|
||||
import dev.dnpm.etl.processor.config.KafkaProperties
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import org.apache.kafka.clients.producer.ProducerRecord
|
||||
import org.slf4j.LoggerFactory
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.kafka.core.KafkaTemplate
|
||||
import org.springframework.retry.support.RetryTemplate
|
||||
|
||||
@ -38,14 +40,20 @@ class KafkaMtbFileSender(
|
||||
|
||||
private val logger = LoggerFactory.getLogger(KafkaMtbFileSender::class.java)
|
||||
|
||||
override fun send(request: MtbFileSender.MtbFileRequest): MtbFileSender.Response {
|
||||
override fun <T> send(request: MtbFileRequest<T>): MtbFileSender.Response {
|
||||
return try {
|
||||
return retryTemplate.execute<MtbFileSender.Response, Exception> {
|
||||
val result = kafkaTemplate.send(
|
||||
kafkaProperties.topic,
|
||||
key(request),
|
||||
objectMapper.writeValueAsString(Data(request.requestId, request.mtbFile))
|
||||
)
|
||||
val record =
|
||||
ProducerRecord(kafkaProperties.outputTopic, key(request), objectMapper.writeValueAsString(request))
|
||||
when (request) {
|
||||
is BwhcV1MtbFileRequest -> record.headers()
|
||||
.add("contentType", MediaType.APPLICATION_JSON_VALUE.toByteArray())
|
||||
|
||||
is DnpmV2MtbFileRequest -> record.headers()
|
||||
.add("contentType", CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE.toByteArray())
|
||||
}
|
||||
|
||||
val result = kafkaTemplate.send(record)
|
||||
if (result.get() != null) {
|
||||
logger.debug("Sent file via KafkaMtbFileSender")
|
||||
MtbFileSender.Response(RequestStatus.UNKNOWN)
|
||||
@ -59,7 +67,7 @@ class KafkaMtbFileSender(
|
||||
}
|
||||
}
|
||||
|
||||
override fun send(request: MtbFileSender.DeleteRequest): MtbFileSender.Response {
|
||||
override fun send(request: DeleteRequest): MtbFileSender.Response {
|
||||
val dummyMtbFile = MtbFile.builder()
|
||||
.withConsent(
|
||||
Consent.builder()
|
||||
@ -71,12 +79,15 @@ class KafkaMtbFileSender(
|
||||
|
||||
return try {
|
||||
return retryTemplate.execute<MtbFileSender.Response, Exception> {
|
||||
val result = kafkaTemplate.send(
|
||||
kafkaProperties.topic,
|
||||
key(request),
|
||||
objectMapper.writeValueAsString(Data(request.requestId, dummyMtbFile))
|
||||
)
|
||||
val record =
|
||||
ProducerRecord(
|
||||
kafkaProperties.outputTopic,
|
||||
key(request),
|
||||
// Always use old BwhcV1FileRequest with Consent REJECT
|
||||
objectMapper.writeValueAsString(BwhcV1MtbFileRequest(request.requestId, dummyMtbFile))
|
||||
)
|
||||
|
||||
val result = kafkaTemplate.send(record)
|
||||
if (result.get() != null) {
|
||||
logger.debug("Sent deletion request via KafkaMtbFileSender")
|
||||
MtbFileSender.Response(RequestStatus.UNKNOWN)
|
||||
@ -91,16 +102,15 @@ class KafkaMtbFileSender(
|
||||
}
|
||||
|
||||
override fun endpoint(): String {
|
||||
return "${this.kafkaProperties.servers} (${this.kafkaProperties.topic}/${this.kafkaProperties.responseTopic})"
|
||||
return "${this.kafkaProperties.servers} (${this.kafkaProperties.outputTopic}/${this.kafkaProperties.outputResponseTopic})"
|
||||
}
|
||||
|
||||
private fun key(request: MtbFileSender.MtbFileRequest): String {
|
||||
return "{\"pid\": \"${request.mtbFile.patient.id}\"}"
|
||||
private fun key(request: MtbRequest): String {
|
||||
return when (request) {
|
||||
is BwhcV1MtbFileRequest -> "{\"pid\": \"${request.content.patient.id}\"}"
|
||||
is DnpmV2MtbFileRequest -> "{\"pid\": \"${request.content.patient.id}\"}"
|
||||
is DeleteRequest -> "{\"pid\": \"${request.patientId.value}\"}"
|
||||
else -> throw IllegalArgumentException("Unsupported request type: ${request::class.simpleName}")
|
||||
}
|
||||
}
|
||||
|
||||
private fun key(request: MtbFileSender.DeleteRequest): String {
|
||||
return "{\"pid\": \"${request.patientId.value}\"}"
|
||||
}
|
||||
|
||||
data class Data(val requestId: RequestId, val content: MtbFile)
|
||||
}
|
||||
}
|
||||
|
@ -19,25 +19,17 @@
|
||||
|
||||
package dev.dnpm.etl.processor.output
|
||||
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.RequestId
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import org.springframework.http.HttpStatusCode
|
||||
|
||||
interface MtbFileSender {
|
||||
fun send(request: MtbFileRequest): Response
|
||||
fun <T> send(request: MtbFileRequest<T>): Response
|
||||
|
||||
fun send(request: DeleteRequest): Response
|
||||
|
||||
fun endpoint(): String
|
||||
|
||||
data class Response(val status: RequestStatus, val body: String = "")
|
||||
|
||||
data class MtbFileRequest(val requestId: RequestId, val mtbFile: MtbFile)
|
||||
|
||||
data class DeleteRequest(val requestId: RequestId, val patientId: PatientPseudonym)
|
||||
|
||||
}
|
||||
|
||||
fun Int.asRequestStatus(): RequestStatus {
|
||||
@ -51,4 +43,4 @@ fun Int.asRequestStatus(): RequestStatus {
|
||||
|
||||
fun HttpStatusCode.asRequestStatus(): RequestStatus {
|
||||
return this.value().asRequestStatus()
|
||||
}
|
||||
}
|
||||
|
59
src/main/kotlin/dev/dnpm/etl/processor/output/MtbRequest.kt
Normal file
59
src/main/kotlin/dev/dnpm/etl/processor/output/MtbRequest.kt
Normal file
@ -0,0 +1,59 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.output
|
||||
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.RequestId
|
||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||
|
||||
interface MtbRequest {
|
||||
val requestId: RequestId
|
||||
}
|
||||
|
||||
sealed interface MtbFileRequest<out T> : MtbRequest {
|
||||
override val requestId: RequestId
|
||||
val content: T
|
||||
|
||||
fun patientPseudonym(): PatientPseudonym
|
||||
}
|
||||
|
||||
data class BwhcV1MtbFileRequest(
|
||||
override val requestId: RequestId,
|
||||
override val content: MtbFile
|
||||
) : MtbFileRequest<MtbFile> {
|
||||
override fun patientPseudonym(): PatientPseudonym {
|
||||
return PatientPseudonym(content.patient.id)
|
||||
}
|
||||
}
|
||||
|
||||
data class DnpmV2MtbFileRequest(
|
||||
override val requestId: RequestId,
|
||||
override val content: Mtb
|
||||
) : MtbFileRequest<Mtb> {
|
||||
override fun patientPseudonym(): PatientPseudonym {
|
||||
return PatientPseudonym(content.patient.id)
|
||||
}
|
||||
}
|
||||
|
||||
data class DeleteRequest(
|
||||
override val requestId: RequestId,
|
||||
val patientId: PatientPseudonym
|
||||
) : MtbRequest
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -19,10 +19,11 @@
|
||||
|
||||
package dev.dnpm.etl.processor.output
|
||||
|
||||
import dev.dnpm.etl.processor.config.RestTargetProperties
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.CustomMediaType
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.config.RestTargetProperties
|
||||
import dev.dnpm.etl.processor.monitoring.ReportService
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.monitoring.asRequestStatus
|
||||
import org.slf4j.LoggerFactory
|
||||
import org.springframework.http.HttpEntity
|
||||
@ -46,11 +47,11 @@ abstract class RestMtbFileSender(
|
||||
|
||||
abstract fun deleteUrl(patientId: PatientPseudonym): String
|
||||
|
||||
override fun send(request: MtbFileSender.MtbFileRequest): MtbFileSender.Response {
|
||||
override fun <T> send(request: MtbFileRequest<T>): MtbFileSender.Response {
|
||||
try {
|
||||
return retryTemplate.execute<MtbFileSender.Response, Exception> {
|
||||
val headers = getHttpHeaders()
|
||||
val entityReq = HttpEntity(request.mtbFile, headers)
|
||||
val headers = getHttpHeaders(request)
|
||||
val entityReq = HttpEntity(request.content, headers)
|
||||
val response = restTemplate.postForEntity(
|
||||
sendUrl(),
|
||||
entityReq,
|
||||
@ -76,10 +77,10 @@ abstract class RestMtbFileSender(
|
||||
return MtbFileSender.Response(RequestStatus.ERROR, "Sonstiger Fehler bei der Übertragung")
|
||||
}
|
||||
|
||||
override fun send(request: MtbFileSender.DeleteRequest): MtbFileSender.Response {
|
||||
override fun send(request: DeleteRequest): MtbFileSender.Response {
|
||||
try {
|
||||
return retryTemplate.execute<MtbFileSender.Response, Exception> {
|
||||
val headers = getHttpHeaders()
|
||||
val headers = getHttpHeaders(request)
|
||||
val entityReq = HttpEntity(null, headers)
|
||||
restTemplate.delete(
|
||||
deleteUrl(request.patientId),
|
||||
@ -102,11 +103,15 @@ abstract class RestMtbFileSender(
|
||||
return this.restTargetProperties.uri.orEmpty()
|
||||
}
|
||||
|
||||
private fun getHttpHeaders(): HttpHeaders {
|
||||
private fun getHttpHeaders(request: MtbRequest): HttpHeaders {
|
||||
val username = restTargetProperties.username
|
||||
val password = restTargetProperties.password
|
||||
val headers = HttpHeaders()
|
||||
headers.contentType = MediaType.APPLICATION_JSON
|
||||
headers.contentType = when (request) {
|
||||
is BwhcV1MtbFileRequest -> MediaType.APPLICATION_JSON
|
||||
is DnpmV2MtbFileRequest -> CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON
|
||||
else -> MediaType.APPLICATION_JSON
|
||||
}
|
||||
|
||||
if (username.isNullOrBlank() || password.isNullOrBlank()) {
|
||||
return headers
|
||||
@ -116,4 +121,4 @@ abstract class RestMtbFileSender(
|
||||
return headers
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
@ -21,6 +21,7 @@ package dev.dnpm.etl.processor.pseudonym
|
||||
|
||||
import org.apache.commons.codec.binary.Base32
|
||||
import org.apache.commons.codec.digest.DigestUtils
|
||||
import org.apache.commons.math3.random.RandomDataGenerator
|
||||
|
||||
|
||||
class AnonymizingGenerator : Generator {
|
||||
@ -31,4 +32,9 @@ class AnonymizingGenerator : Generator {
|
||||
.lowercase()
|
||||
}
|
||||
|
||||
override fun generateGenomDeTan(id: String?): String? {
|
||||
val randomDataGenerator = RandomDataGenerator()
|
||||
return randomDataGenerator.nextSecureHexString(64).lowercase()
|
||||
}
|
||||
|
||||
}
|
@ -35,6 +35,10 @@ class PseudonymizeService(
|
||||
}
|
||||
}
|
||||
|
||||
fun genomDeTan(patientId: PatientId): String {
|
||||
return generator.generateGenomDeTan(patientId.value)
|
||||
}
|
||||
|
||||
fun prefix(): String {
|
||||
return configProperties.prefix
|
||||
}
|
||||
|
@ -21,12 +21,12 @@ package dev.dnpm.etl.processor.pseudonym
|
||||
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import dev.dnpm.etl.processor.PatientId
|
||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||
import org.apache.commons.codec.digest.DigestUtils
|
||||
|
||||
/** Replaces patient ID with generated patient pseudonym
|
||||
*
|
||||
* @param pseudonymizeService The pseudonymizeService to be used
|
||||
*
|
||||
* @return The MTB file containing patient pseudonymes
|
||||
*/
|
||||
infix fun MtbFile.pseudonymizeWith(pseudonymizeService: PseudonymizeService) {
|
||||
@ -49,7 +49,11 @@ infix fun MtbFile.pseudonymizeWith(pseudonymizeService: PseudonymizeService) {
|
||||
}
|
||||
this.lastGuidelineTherapies?.forEach { it.patient = patientPseudonym }
|
||||
this.molecularPathologyFindings?.forEach { it.patient = patientPseudonym }
|
||||
this.molecularTherapies?.forEach { molecularTherapy -> molecularTherapy.history.forEach { it.patient = patientPseudonym } }
|
||||
this.molecularTherapies?.forEach { molecularTherapy ->
|
||||
molecularTherapy.history.forEach {
|
||||
it.patient = patientPseudonym
|
||||
}
|
||||
}
|
||||
this.ngsReports?.forEach { it.patient = patientPseudonym }
|
||||
this.previousGuidelineTherapies?.forEach { it.patient = patientPseudonym }
|
||||
this.rebiopsyRequests?.forEach { it.patient = patientPseudonym }
|
||||
@ -63,7 +67,6 @@ infix fun MtbFile.pseudonymizeWith(pseudonymizeService: PseudonymizeService) {
|
||||
* Creates new hash of content IDs with given prefix except for patient IDs
|
||||
*
|
||||
* @param pseudonymizeService The pseudonymizeService to be used
|
||||
*
|
||||
* @return The MTB file containing rehashed content IDs
|
||||
*/
|
||||
infix fun MtbFile.anonymizeContentWith(pseudonymizeService: PseudonymizeService) {
|
||||
@ -120,8 +123,8 @@ infix fun MtbFile.anonymizeContentWith(pseudonymizeService: PseudonymizeService)
|
||||
id = id?.let { anonymize(it) }
|
||||
}
|
||||
}
|
||||
this.geneticCounsellingRequests?.onEach { geneticCounsellingRequest ->
|
||||
geneticCounsellingRequest?.apply {
|
||||
this.geneticCounsellingRequests?.onEach { geneticCounsellingRequest ->
|
||||
geneticCounsellingRequest?.apply {
|
||||
id = id?.let { anonymize(it) }
|
||||
}
|
||||
}
|
||||
@ -223,4 +226,99 @@ infix fun MtbFile.anonymizeContentWith(pseudonymizeService: PseudonymizeService)
|
||||
id = id?.let { anonymize(it) }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Replaces patient ID with generated patient pseudonym
|
||||
*
|
||||
* @since 0.11.0
|
||||
*
|
||||
* @param pseudonymizeService The pseudonymizeService to be used
|
||||
* @return The MTB file containing patient pseudonymes
|
||||
*/
|
||||
infix fun Mtb.pseudonymizeWith(pseudonymizeService: PseudonymizeService) {
|
||||
val patientPseudonym = pseudonymizeService.patientPseudonym(PatientId(this.patient.id)).value
|
||||
|
||||
this.episodesOfCare?.forEach { it.patient.id = patientPseudonym }
|
||||
this.carePlans?.forEach {
|
||||
it.patient.id = patientPseudonym
|
||||
it.rebiopsyRequests?.forEach { it.patient.id = patientPseudonym }
|
||||
it.histologyReevaluationRequests?.forEach { it.patient.id = patientPseudonym }
|
||||
it.medicationRecommendations.forEach { it.patient.id = patientPseudonym }
|
||||
it.studyEnrollmentRecommendations?.forEach { it.patient.id = patientPseudonym }
|
||||
it.procedureRecommendations?.forEach { it.patient.id = patientPseudonym }
|
||||
it.geneticCounselingRecommendation.patient.id = patientPseudonym
|
||||
}
|
||||
this.diagnoses?.forEach { it.patient.id = patientPseudonym }
|
||||
this.guidelineTherapies?.forEach { it.patient.id = patientPseudonym }
|
||||
this.guidelineProcedures?.forEach { it.patient.id = patientPseudonym }
|
||||
this.patient.id = patientPseudonym
|
||||
this.claims?.forEach { it.patient.id = patientPseudonym }
|
||||
this.claimResponses?.forEach { it.patient.id = patientPseudonym }
|
||||
this.diagnoses?.forEach { it.patient.id = patientPseudonym }
|
||||
this.familyMemberHistories?.forEach { it.patient.id = patientPseudonym }
|
||||
this.histologyReports?.forEach {
|
||||
it.patient.id = patientPseudonym
|
||||
it.results.tumorMorphology?.patient?.id = patientPseudonym
|
||||
it.results.tumorCellContent?.patient?.id = patientPseudonym
|
||||
}
|
||||
this.ngsReports?.forEach {
|
||||
it.patient.id = patientPseudonym
|
||||
it.results.simpleVariants?.forEach { it.patient.id = patientPseudonym }
|
||||
it.results.copyNumberVariants?.forEach { it.patient.id = patientPseudonym }
|
||||
it.results.dnaFusions?.forEach { it.patient.id = patientPseudonym }
|
||||
it.results.rnaFusions?.forEach { it.patient.id = patientPseudonym }
|
||||
it.results.tumorCellContent?.patient?.id = patientPseudonym
|
||||
it.results.brcaness?.patient?.id = patientPseudonym
|
||||
it.results.tmb?.patient?.id = patientPseudonym
|
||||
it.results.hrdScore?.patient?.id = patientPseudonym
|
||||
}
|
||||
this.ihcReports?.forEach {
|
||||
it.patient.id = patientPseudonym
|
||||
it.results.msiMmr?.forEach { it.patient.id = patientPseudonym }
|
||||
it.results.proteinExpression?.forEach { it.patient.id = patientPseudonym }
|
||||
}
|
||||
this.responses?.forEach { it.patient.id = patientPseudonym }
|
||||
this.specimens?.forEach { it.patient.id = patientPseudonym }
|
||||
this.priorDiagnosticReports?.forEach { it.patient.id = patientPseudonym }
|
||||
this.performanceStatus?.forEach { it.patient.id = patientPseudonym }
|
||||
this.systemicTherapies?.forEach {
|
||||
it.history?.forEach {
|
||||
it.patient.id = patientPseudonym
|
||||
}
|
||||
}
|
||||
this.followUps?.forEach {
|
||||
it.patient.id = patientPseudonym
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates new hash of content IDs with given prefix except for patient IDs
|
||||
*
|
||||
* @since 0.11.0
|
||||
*
|
||||
* @param pseudonymizeService The pseudonymizeService to be used
|
||||
* @return The MTB file containing rehashed content IDs
|
||||
*/
|
||||
infix fun Mtb.anonymizeContentWith(pseudonymizeService: PseudonymizeService) {
|
||||
val prefix = pseudonymizeService.prefix()
|
||||
|
||||
fun anonymize(id: String): String {
|
||||
val hash = DigestUtils.sha256Hex("$prefix-$id").substring(0, 41).lowercase()
|
||||
return "$prefix$hash"
|
||||
}
|
||||
|
||||
this.episodesOfCare?.forEach {
|
||||
it?.apply {
|
||||
id = id?.let {
|
||||
anonymize(it)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TODO all other properties
|
||||
}
|
||||
|
||||
infix fun Mtb.addGenomDeTan(pseudonymizeService: PseudonymizeService)
|
||||
{
|
||||
this.metadata.transferTan = pseudonymizeService.genomDeTan(PatientId(this.patient.id))
|
||||
}
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2023 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -27,10 +27,14 @@ import dev.dnpm.etl.processor.monitoring.Report
|
||||
import dev.dnpm.etl.processor.monitoring.Request
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.monitoring.RequestType
|
||||
import dev.dnpm.etl.processor.output.MtbFileSender
|
||||
import dev.dnpm.etl.processor.output.*
|
||||
import dev.dnpm.etl.processor.pseudonym.PseudonymizeService
|
||||
import dev.dnpm.etl.processor.pseudonym.addGenomDeTan
|
||||
import dev.dnpm.etl.processor.pseudonym.anonymizeContentWith
|
||||
import dev.dnpm.etl.processor.pseudonym.pseudonymizeWith
|
||||
import dev.pcvolkmer.mv64e.mtb.ConsentProvision
|
||||
import dev.pcvolkmer.mv64e.mtb.ModelProjectConsentPurpose
|
||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||
import org.apache.commons.codec.binary.Base32
|
||||
import org.apache.commons.codec.digest.DigestUtils
|
||||
import org.springframework.context.ApplicationEventPublisher
|
||||
@ -55,29 +59,47 @@ class RequestProcessor(
|
||||
|
||||
fun processMtbFile(mtbFile: MtbFile, requestId: RequestId) {
|
||||
val pid = PatientId(mtbFile.patient.id)
|
||||
|
||||
mtbFile pseudonymizeWith pseudonymizeService
|
||||
mtbFile anonymizeContentWith pseudonymizeService
|
||||
val request = BwhcV1MtbFileRequest(requestId, transformationService.transform(mtbFile))
|
||||
saveAndSend(request, pid)
|
||||
}
|
||||
|
||||
val request = MtbFileSender.MtbFileRequest(requestId, transformationService.transform(mtbFile))
|
||||
fun processMtbFile(mtbFile: Mtb) {
|
||||
processMtbFile(mtbFile, randomRequestId())
|
||||
}
|
||||
|
||||
val patientPseudonym = PatientPseudonym(request.mtbFile.patient.id)
|
||||
fun processMtbFile(mtbFile: Mtb, requestId: RequestId) {
|
||||
val pid = PatientId(mtbFile.patient.id)
|
||||
val isModelProjectConsented = mtbFile.metadata?.modelProjectConsent?.provisions?.any { p ->
|
||||
p.purpose == ModelProjectConsentPurpose.SEQUENCING
|
||||
&& p.type == ConsentProvision.PERMIT
|
||||
} == true
|
||||
if (isModelProjectConsented) {
|
||||
mtbFile addGenomDeTan pseudonymizeService
|
||||
}
|
||||
mtbFile pseudonymizeWith pseudonymizeService
|
||||
mtbFile anonymizeContentWith pseudonymizeService
|
||||
val request = DnpmV2MtbFileRequest(requestId, transformationService.transform(mtbFile))
|
||||
saveAndSend(request, pid)
|
||||
}
|
||||
|
||||
private fun <T> saveAndSend(request: MtbFileRequest<T>, pid: PatientId) {
|
||||
requestService.save(
|
||||
Request(
|
||||
requestId,
|
||||
patientPseudonym,
|
||||
request.requestId,
|
||||
request.patientPseudonym(),
|
||||
pid,
|
||||
fingerprint(request.mtbFile),
|
||||
fingerprint(request),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.UNKNOWN
|
||||
)
|
||||
)
|
||||
|
||||
if (appConfigProperties.duplicationDetection && isDuplication(mtbFile)) {
|
||||
if (appConfigProperties.duplicationDetection && isDuplication(request)) {
|
||||
applicationEventPublisher.publishEvent(
|
||||
ResponseEvent(
|
||||
requestId,
|
||||
request.requestId,
|
||||
Instant.now(),
|
||||
RequestStatus.DUPLICATION
|
||||
)
|
||||
@ -89,7 +111,7 @@ class RequestProcessor(
|
||||
|
||||
applicationEventPublisher.publishEvent(
|
||||
ResponseEvent(
|
||||
requestId,
|
||||
request.requestId,
|
||||
Instant.now(),
|
||||
responseStatus.status,
|
||||
when (responseStatus.status) {
|
||||
@ -100,16 +122,22 @@ class RequestProcessor(
|
||||
)
|
||||
}
|
||||
|
||||
private fun isDuplication(pseudonymizedMtbFile: MtbFile): Boolean {
|
||||
val patientPseudonym = PatientPseudonym(pseudonymizedMtbFile.patient.id)
|
||||
private fun <T> isDuplication(pseudonymizedMtbFileRequest: MtbFileRequest<T>): Boolean {
|
||||
val patientPseudonym = when (pseudonymizedMtbFileRequest) {
|
||||
is BwhcV1MtbFileRequest -> PatientPseudonym(pseudonymizedMtbFileRequest.content.patient.id)
|
||||
is DnpmV2MtbFileRequest -> PatientPseudonym(pseudonymizedMtbFileRequest.content.patient.id)
|
||||
}
|
||||
|
||||
val lastMtbFileRequestForPatient =
|
||||
requestService.lastMtbFileRequestForPatientPseudonym(patientPseudonym)
|
||||
val isLastRequestDeletion = requestService.isLastRequestWithKnownStatusDeletion(patientPseudonym)
|
||||
val isLastRequestDeletion =
|
||||
requestService.isLastRequestWithKnownStatusDeletion(patientPseudonym)
|
||||
|
||||
return null != lastMtbFileRequestForPatient
|
||||
&& !isLastRequestDeletion
|
||||
&& lastMtbFileRequestForPatient.fingerprint == fingerprint(pseudonymizedMtbFile)
|
||||
&& lastMtbFileRequestForPatient.fingerprint == fingerprint(
|
||||
pseudonymizedMtbFileRequest
|
||||
)
|
||||
}
|
||||
|
||||
fun processDeletion(patientId: PatientId) {
|
||||
@ -131,7 +159,7 @@ class RequestProcessor(
|
||||
)
|
||||
)
|
||||
|
||||
val responseStatus = sender.send(MtbFileSender.DeleteRequest(requestId, patientPseudonym))
|
||||
val responseStatus = sender.send(DeleteRequest(requestId, patientPseudonym))
|
||||
|
||||
applicationEventPublisher.publishEvent(
|
||||
ResponseEvent(
|
||||
@ -160,8 +188,11 @@ class RequestProcessor(
|
||||
}
|
||||
}
|
||||
|
||||
private fun fingerprint(mtbFile: MtbFile): Fingerprint {
|
||||
return fingerprint(objectMapper.writeValueAsString(mtbFile))
|
||||
private fun <T> fingerprint(request: MtbFileRequest<T>): Fingerprint {
|
||||
return when (request) {
|
||||
is BwhcV1MtbFileRequest -> fingerprint(objectMapper.writeValueAsString(request.content))
|
||||
is DnpmV2MtbFileRequest -> fingerprint(objectMapper.writeValueAsString(request.content))
|
||||
}
|
||||
}
|
||||
|
||||
private fun fingerprint(s: String): Fingerprint {
|
||||
@ -172,4 +203,4 @@ class RequestProcessor(
|
||||
)
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
@ -23,10 +23,21 @@ import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import com.jayway.jsonpath.JsonPath
|
||||
import com.jayway.jsonpath.PathNotFoundException
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||
|
||||
class TransformationService(private val objectMapper: ObjectMapper, private val transformations: List<Transformation>) {
|
||||
fun transform(mtbFile: MtbFile): MtbFile {
|
||||
var json = objectMapper.writeValueAsString(mtbFile)
|
||||
val json = transform(objectMapper.writeValueAsString(mtbFile))
|
||||
return objectMapper.readValue(json, MtbFile::class.java)
|
||||
}
|
||||
|
||||
fun transform(mtbFile: Mtb): Mtb {
|
||||
val json = transform(objectMapper.writeValueAsString(mtbFile))
|
||||
return objectMapper.readValue(json, Mtb::class.java)
|
||||
}
|
||||
|
||||
private fun transform(content: String): String {
|
||||
var json = content
|
||||
|
||||
transformations.forEach { transformation ->
|
||||
val jsonPath = JsonPath.parse(json)
|
||||
@ -48,7 +59,7 @@ class TransformationService(private val objectMapper: ObjectMapper, private val
|
||||
json = jsonPath.jsonString()
|
||||
}
|
||||
|
||||
return objectMapper.readValue(json, MtbFile::class.java)
|
||||
return json
|
||||
}
|
||||
|
||||
fun getTransformations(): List<Transformation> {
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -19,6 +19,7 @@
|
||||
|
||||
package dev.dnpm.etl.processor
|
||||
|
||||
import org.springframework.http.MediaType
|
||||
import java.util.*
|
||||
|
||||
class Fingerprint(val value: String) {
|
||||
@ -46,4 +47,17 @@ value class PatientId(val value: String)
|
||||
@JvmInline
|
||||
value class PatientPseudonym(val value: String)
|
||||
|
||||
fun emptyPatientPseudonym() = PatientPseudonym("")
|
||||
fun emptyPatientPseudonym() = PatientPseudonym("")
|
||||
|
||||
/**
|
||||
* Custom MediaTypes
|
||||
*
|
||||
* @since 0.11.0
|
||||
*/
|
||||
object CustomMediaType {
|
||||
val APPLICATION_VND_DNPM_V2_MTB_JSON = MediaType("application", "vnd.dnpm.v2.mtb+json")
|
||||
const val APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE = "application/vnd.dnpm.v2.mtb+json"
|
||||
|
||||
val APPLICATION_VND_DNPM_V2_RD_JSON = MediaType("application", "vnd.dnpm.v2.rd+json")
|
||||
const val APPLICATION_VND_DNPM_V2_RD_JSON_VALUE = "application/vnd.dnpm.v2.rd+json"
|
||||
}
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -23,6 +23,7 @@ import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.Consent
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import de.ukw.ccc.bwhc.dto.Patient
|
||||
import dev.dnpm.etl.processor.CustomMediaType
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import org.apache.kafka.clients.consumer.ConsumerRecord
|
||||
import org.apache.kafka.common.header.internals.RecordHeader
|
||||
@ -63,9 +64,17 @@ class KafkaInputListenerTest {
|
||||
.withConsent(Consent.builder().withStatus(Consent.Status.ACTIVE).build())
|
||||
.build()
|
||||
|
||||
kafkaInputListener.onMessage(ConsumerRecord("testtopic", 0, 0, "", this.objectMapper.writeValueAsString(mtbFile)))
|
||||
kafkaInputListener.onMessage(
|
||||
ConsumerRecord(
|
||||
"testtopic",
|
||||
0,
|
||||
0,
|
||||
"",
|
||||
this.objectMapper.writeValueAsString(mtbFile)
|
||||
)
|
||||
)
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any())
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -75,7 +84,15 @@ class KafkaInputListenerTest {
|
||||
.withConsent(Consent.builder().withStatus(Consent.Status.REJECTED).build())
|
||||
.build()
|
||||
|
||||
kafkaInputListener.onMessage(ConsumerRecord("testtopic", 0, 0, "", this.objectMapper.writeValueAsString(mtbFile)))
|
||||
kafkaInputListener.onMessage(
|
||||
ConsumerRecord(
|
||||
"testtopic",
|
||||
0,
|
||||
0,
|
||||
"",
|
||||
this.objectMapper.writeValueAsString(mtbFile)
|
||||
)
|
||||
)
|
||||
|
||||
verify(requestProcessor, times(1)).processDeletion(anyValueClass())
|
||||
}
|
||||
@ -89,10 +106,22 @@ class KafkaInputListenerTest {
|
||||
|
||||
val headers = RecordHeaders(listOf(RecordHeader("requestId", UUID.randomUUID().toString().toByteArray())))
|
||||
kafkaInputListener.onMessage(
|
||||
ConsumerRecord("testtopic", 0, 0, -1L, TimestampType.NO_TIMESTAMP_TYPE, -1, -1, "", this.objectMapper.writeValueAsString(mtbFile), headers, Optional.empty())
|
||||
ConsumerRecord(
|
||||
"testtopic",
|
||||
0,
|
||||
0,
|
||||
-1L,
|
||||
TimestampType.NO_TIMESTAMP_TYPE,
|
||||
-1,
|
||||
-1,
|
||||
"",
|
||||
this.objectMapper.writeValueAsString(mtbFile),
|
||||
headers,
|
||||
Optional.empty()
|
||||
)
|
||||
)
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any(), anyValueClass())
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>(), anyValueClass())
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -104,9 +133,52 @@ class KafkaInputListenerTest {
|
||||
|
||||
val headers = RecordHeaders(listOf(RecordHeader("requestId", UUID.randomUUID().toString().toByteArray())))
|
||||
kafkaInputListener.onMessage(
|
||||
ConsumerRecord("testtopic", 0, 0, -1L, TimestampType.NO_TIMESTAMP_TYPE, -1, -1, "", this.objectMapper.writeValueAsString(mtbFile), headers, Optional.empty())
|
||||
ConsumerRecord(
|
||||
"testtopic",
|
||||
0,
|
||||
0,
|
||||
-1L,
|
||||
TimestampType.NO_TIMESTAMP_TYPE,
|
||||
-1,
|
||||
-1,
|
||||
"",
|
||||
this.objectMapper.writeValueAsString(mtbFile),
|
||||
headers,
|
||||
Optional.empty()
|
||||
)
|
||||
)
|
||||
verify(requestProcessor, times(1)).processDeletion(anyValueClass(), anyValueClass())
|
||||
}
|
||||
|
||||
}
|
||||
@Test
|
||||
fun shouldNotProcessDnpmV2Request() {
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(Patient.builder().withId("DUMMY_12345678").build())
|
||||
.withConsent(Consent.builder().withStatus(Consent.Status.REJECTED).build())
|
||||
.build()
|
||||
|
||||
val headers = RecordHeaders(
|
||||
listOf(
|
||||
RecordHeader("requestId", UUID.randomUUID().toString().toByteArray()),
|
||||
RecordHeader("contentType", CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE.toByteArray())
|
||||
)
|
||||
)
|
||||
kafkaInputListener.onMessage(
|
||||
ConsumerRecord(
|
||||
"testtopic",
|
||||
0,
|
||||
0,
|
||||
-1L,
|
||||
TimestampType.NO_TIMESTAMP_TYPE,
|
||||
-1,
|
||||
-1,
|
||||
"",
|
||||
this.objectMapper.writeValueAsString(mtbFile),
|
||||
headers,
|
||||
Optional.empty()
|
||||
)
|
||||
)
|
||||
verify(requestProcessor, times(0)).processDeletion(anyValueClass(), anyValueClass())
|
||||
}
|
||||
|
||||
}
|
||||
|
@ -21,7 +21,9 @@ package dev.dnpm.etl.processor.input
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.*
|
||||
import dev.dnpm.etl.processor.CustomMediaType
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.api.Test
|
||||
@ -32,6 +34,7 @@ import org.mockito.Mockito.verify
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.any
|
||||
import org.mockito.kotlin.anyValueClass
|
||||
import org.springframework.core.io.ClassPathResource
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.test.web.servlet.MockMvc
|
||||
import org.springframework.test.web.servlet.delete
|
||||
@ -70,7 +73,7 @@ class MtbFileRestControllerTest {
|
||||
}
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any())
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -126,7 +129,7 @@ class MtbFileRestControllerTest {
|
||||
}
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any())
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -155,6 +158,40 @@ class MtbFileRestControllerTest {
|
||||
}
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class RequestsForDnpmDataModel21 {
|
||||
|
||||
private lateinit var mockMvc: MockMvc
|
||||
|
||||
private lateinit var requestProcessor: RequestProcessor
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Mock requestProcessor: RequestProcessor
|
||||
) {
|
||||
this.requestProcessor = requestProcessor
|
||||
val controller = MtbFileRestController(requestProcessor)
|
||||
this.mockMvc = MockMvcBuilders.standaloneSetup(controller).build()
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldRespondPostRequest() {
|
||||
val mtbFileContent = ClassPathResource("mv64e-mtb-fake-patient.json").inputStream.readAllBytes().toString(Charsets.UTF_8)
|
||||
|
||||
mockMvc.post("/mtb") {
|
||||
content = mtbFileContent
|
||||
contentType = CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON
|
||||
}.andExpect {
|
||||
status {
|
||||
isAccepted()
|
||||
}
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<Mtb>())
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
companion object {
|
||||
fun bwhcMtbFileContent(consentStatus: Consent.Status) = MtbFile.builder()
|
||||
.withPatient(
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -21,166 +21,262 @@ package dev.dnpm.etl.processor.output
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.*
|
||||
import de.ukw.ccc.bwhc.dto.Patient
|
||||
import dev.dnpm.etl.processor.CustomMediaType
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.RequestId
|
||||
import dev.dnpm.etl.processor.config.KafkaProperties
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.pcvolkmer.mv64e.mtb.*
|
||||
import org.apache.kafka.clients.producer.ProducerRecord
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.junit.jupiter.params.ParameterizedTest
|
||||
import org.junit.jupiter.params.provider.MethodSource
|
||||
import org.mockito.ArgumentMatchers.anyString
|
||||
import org.mockito.Mock
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.*
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.kafka.core.KafkaTemplate
|
||||
import org.springframework.kafka.support.SendResult
|
||||
import org.springframework.retry.policy.SimpleRetryPolicy
|
||||
import org.springframework.retry.support.RetryTemplateBuilder
|
||||
import java.time.Instant
|
||||
import java.util.*
|
||||
import java.util.concurrent.CompletableFuture.completedFuture
|
||||
import java.util.concurrent.ExecutionException
|
||||
|
||||
@ExtendWith(MockitoExtension::class)
|
||||
class KafkaMtbFileSenderTest {
|
||||
|
||||
private lateinit var kafkaTemplate: KafkaTemplate<String, String>
|
||||
@Nested
|
||||
inner class BwhcV1Record {
|
||||
|
||||
private lateinit var kafkaMtbFileSender: KafkaMtbFileSender
|
||||
private lateinit var kafkaTemplate: KafkaTemplate<String, String>
|
||||
|
||||
private lateinit var objectMapper: ObjectMapper
|
||||
private lateinit var kafkaMtbFileSender: KafkaMtbFileSender
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Mock kafkaTemplate: KafkaTemplate<String, String>
|
||||
) {
|
||||
val kafkaProperties = KafkaProperties("testtopic")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||
private lateinit var objectMapper: ObjectMapper
|
||||
|
||||
this.objectMapper = ObjectMapper()
|
||||
this.kafkaTemplate = kafkaTemplate
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Mock kafkaTemplate: KafkaTemplate<String, String>
|
||||
) {
|
||||
val kafkaProperties = KafkaProperties("testtopic")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||
|
||||
this.kafkaMtbFileSender = KafkaMtbFileSender(kafkaTemplate, kafkaProperties, retryTemplate, objectMapper)
|
||||
}
|
||||
this.objectMapper = ObjectMapper()
|
||||
this.kafkaTemplate = kafkaTemplate
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("requestWithResponseSource")
|
||||
fun shouldSendMtbFileRequestAndReturnExpectedState(testData: TestData) {
|
||||
doAnswer {
|
||||
if (null != testData.exception) {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(anyString(), anyString(), anyString())
|
||||
|
||||
val response = kafkaMtbFileSender.send(MtbFileSender.MtbFileRequest(TEST_REQUEST_ID, mtbFile(Consent.Status.ACTIVE)))
|
||||
assertThat(response.status).isEqualTo(testData.requestStatus)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("requestWithResponseSource")
|
||||
fun shouldSendDeleteRequestAndReturnExpectedState(testData: TestData) {
|
||||
doAnswer {
|
||||
if (null != testData.exception) {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(anyString(), anyString(), anyString())
|
||||
|
||||
val response = kafkaMtbFileSender.send(MtbFileSender.DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
assertThat(response.status).isEqualTo(testData.requestStatus)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldSendMtbFileRequestWithCorrectKeyAndBody() {
|
||||
doAnswer {
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(anyString(), anyString(), anyString())
|
||||
|
||||
kafkaMtbFileSender.send(MtbFileSender.MtbFileRequest(TEST_REQUEST_ID, mtbFile(Consent.Status.ACTIVE)))
|
||||
|
||||
val captor = argumentCaptor<String>()
|
||||
verify(kafkaTemplate, times(1)).send(anyString(), captor.capture(), captor.capture())
|
||||
assertThat(captor.firstValue).isNotNull
|
||||
assertThat(captor.firstValue).isEqualTo("{\"pid\": \"PID\"}")
|
||||
assertThat(captor.secondValue).isNotNull
|
||||
assertThat(captor.secondValue).isEqualTo(objectMapper.writeValueAsString(kafkaRecordData(TEST_REQUEST_ID, Consent.Status.ACTIVE)))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldSendDeleteRequestWithCorrectKeyAndBody() {
|
||||
doAnswer {
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(anyString(), anyString(), anyString())
|
||||
|
||||
kafkaMtbFileSender.send(MtbFileSender.DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
|
||||
val captor = argumentCaptor<String>()
|
||||
verify(kafkaTemplate, times(1)).send(anyString(), captor.capture(), captor.capture())
|
||||
assertThat(captor.firstValue).isNotNull
|
||||
assertThat(captor.firstValue).isEqualTo("{\"pid\": \"PID\"}")
|
||||
assertThat(captor.secondValue).isNotNull
|
||||
assertThat(captor.secondValue).isEqualTo(objectMapper.writeValueAsString(kafkaRecordData(TEST_REQUEST_ID, Consent.Status.REJECTED)))
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("requestWithResponseSource")
|
||||
fun shouldRetryOnMtbFileKafkaSendError(testData: TestData) {
|
||||
val kafkaProperties = KafkaProperties("testtopic")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
||||
this.kafkaMtbFileSender = KafkaMtbFileSender(this.kafkaTemplate, kafkaProperties, retryTemplate, this.objectMapper)
|
||||
|
||||
doAnswer {
|
||||
if (null != testData.exception) {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(anyString(), anyString(), anyString())
|
||||
|
||||
kafkaMtbFileSender.send(MtbFileSender.MtbFileRequest(TEST_REQUEST_ID, mtbFile(Consent.Status.ACTIVE)))
|
||||
|
||||
val expectedCount = when (testData.exception) {
|
||||
// OK - No Retry
|
||||
null -> times(1)
|
||||
// Request failed - Retry max 3 times
|
||||
else -> times(3)
|
||||
this.kafkaMtbFileSender = KafkaMtbFileSender(kafkaTemplate, kafkaProperties, retryTemplate, objectMapper)
|
||||
}
|
||||
|
||||
verify(kafkaTemplate, expectedCount).send(anyString(), anyString(), anyString())
|
||||
}
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
||||
fun shouldSendMtbFileRequestAndReturnExpectedState(testData: TestData) {
|
||||
doAnswer {
|
||||
if (null != testData.exception) {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("requestWithResponseSource")
|
||||
fun shouldRetryOnDeleteKafkaSendError(testData: TestData) {
|
||||
val kafkaProperties = KafkaProperties("testtopic")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
||||
this.kafkaMtbFileSender = KafkaMtbFileSender(this.kafkaTemplate, kafkaProperties, retryTemplate, this.objectMapper)
|
||||
|
||||
doAnswer {
|
||||
if (null != testData.exception) {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(anyString(), anyString(), anyString())
|
||||
|
||||
kafkaMtbFileSender.send(MtbFileSender.DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
|
||||
val expectedCount = when (testData.exception) {
|
||||
// OK - No Retry
|
||||
null -> times(1)
|
||||
// Request failed - Retry max 3 times
|
||||
else -> times(3)
|
||||
val response = kafkaMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, bwhcV1MtbFile(Consent.Status.ACTIVE)))
|
||||
assertThat(response.status).isEqualTo(testData.requestStatus)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
||||
fun shouldSendDeleteRequestAndReturnExpectedState(testData: TestData) {
|
||||
doAnswer {
|
||||
if (null != testData.exception) {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
val response = kafkaMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
assertThat(response.status).isEqualTo(testData.requestStatus)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldSendMtbFileRequestWithCorrectKeyAndHeaderAndBody() {
|
||||
doAnswer {
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
kafkaMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, bwhcV1MtbFile(Consent.Status.ACTIVE)))
|
||||
|
||||
val captor = argumentCaptor<ProducerRecord<String, String>>()
|
||||
verify(kafkaTemplate, times(1)).send(captor.capture())
|
||||
assertThat(captor.firstValue.key()).isNotNull
|
||||
assertThat(captor.firstValue.key()).isEqualTo("{\"pid\": \"PID\"}")
|
||||
assertThat(captor.firstValue.headers().headers("contentType")).isNotNull
|
||||
assertThat(captor.firstValue.headers().headers("contentType")?.firstOrNull()?.value()).isEqualTo(MediaType.APPLICATION_JSON_VALUE.toByteArray())
|
||||
assertThat(captor.firstValue.value()).isNotNull
|
||||
assertThat(captor.firstValue.value()).isEqualTo(objectMapper.writeValueAsString(bwhcV1kafkaRecordData(TEST_REQUEST_ID, Consent.Status.ACTIVE)))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldSendDeleteRequestWithCorrectKeyAndBody() {
|
||||
doAnswer {
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
kafkaMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
|
||||
val captor = argumentCaptor<ProducerRecord<String, String>>()
|
||||
verify(kafkaTemplate, times(1)).send(captor.capture())
|
||||
assertThat(captor.firstValue.key()).isNotNull
|
||||
assertThat(captor.firstValue.key()).isEqualTo("{\"pid\": \"PID\"}")
|
||||
assertThat(captor.firstValue.value()).isNotNull
|
||||
assertThat(captor.firstValue.value()).isEqualTo(objectMapper.writeValueAsString(bwhcV1kafkaRecordData(TEST_REQUEST_ID, Consent.Status.REJECTED)))
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
||||
fun shouldRetryOnMtbFileKafkaSendError(testData: TestData) {
|
||||
val kafkaProperties = KafkaProperties("testtopic")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
||||
this.kafkaMtbFileSender = KafkaMtbFileSender(this.kafkaTemplate, kafkaProperties, retryTemplate, this.objectMapper)
|
||||
|
||||
doAnswer {
|
||||
if (null != testData.exception) {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
kafkaMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, bwhcV1MtbFile(Consent.Status.ACTIVE)))
|
||||
|
||||
val expectedCount = when (testData.exception) {
|
||||
// OK - No Retry
|
||||
null -> times(1)
|
||||
// Request failed - Retry max 3 times
|
||||
else -> times(3)
|
||||
}
|
||||
|
||||
verify(kafkaTemplate, expectedCount).send(any<ProducerRecord<String, String>>())
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
||||
fun shouldRetryOnDeleteKafkaSendError(testData: TestData) {
|
||||
val kafkaProperties = KafkaProperties("testtopic")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
||||
this.kafkaMtbFileSender = KafkaMtbFileSender(this.kafkaTemplate, kafkaProperties, retryTemplate, this.objectMapper)
|
||||
|
||||
doAnswer {
|
||||
if (null != testData.exception) {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
kafkaMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
|
||||
val expectedCount = when (testData.exception) {
|
||||
// OK - No Retry
|
||||
null -> times(1)
|
||||
// Request failed - Retry max 3 times
|
||||
else -> times(3)
|
||||
}
|
||||
|
||||
verify(kafkaTemplate, expectedCount).send(any<ProducerRecord<String, String>>())
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class DnpmV2Record {
|
||||
|
||||
private lateinit var kafkaTemplate: KafkaTemplate<String, String>
|
||||
|
||||
private lateinit var kafkaMtbFileSender: KafkaMtbFileSender
|
||||
|
||||
private lateinit var objectMapper: ObjectMapper
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Mock kafkaTemplate: KafkaTemplate<String, String>
|
||||
) {
|
||||
val kafkaProperties = KafkaProperties("testtopic")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||
|
||||
this.objectMapper = ObjectMapper()
|
||||
this.kafkaTemplate = kafkaTemplate
|
||||
|
||||
this.kafkaMtbFileSender = KafkaMtbFileSender(kafkaTemplate, kafkaProperties, retryTemplate, objectMapper)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
||||
fun shouldSendMtbFileRequestAndReturnExpectedState(testData: TestData) {
|
||||
doAnswer {
|
||||
if (null != testData.exception) {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
val response = kafkaMtbFileSender.send(DnpmV2MtbFileRequest(TEST_REQUEST_ID, dnpmV2MtbFile()))
|
||||
assertThat(response.status).isEqualTo(testData.requestStatus)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldSendMtbFileRequestWithCorrectKeyAndHeaderAndBody() {
|
||||
doAnswer {
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
kafkaMtbFileSender.send(DnpmV2MtbFileRequest(TEST_REQUEST_ID, dnpmV2MtbFile()))
|
||||
|
||||
val captor = argumentCaptor<ProducerRecord<String, String>>()
|
||||
verify(kafkaTemplate, times(1)).send(captor.capture())
|
||||
assertThat(captor.firstValue.key()).isNotNull
|
||||
assertThat(captor.firstValue.key()).isEqualTo("{\"pid\": \"PID\"}")
|
||||
assertThat(captor.firstValue.headers().headers("contentType")).isNotNull
|
||||
assertThat(captor.firstValue.headers().headers("contentType")?.firstOrNull()?.value()).isEqualTo(CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE.toByteArray())
|
||||
assertThat(captor.firstValue.value()).isNotNull
|
||||
assertThat(captor.firstValue.value()).isEqualTo(objectMapper.writeValueAsString(dnmpV2kafkaRecordData(TEST_REQUEST_ID)))
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
||||
fun shouldRetryOnMtbFileKafkaSendError(testData: TestData) {
|
||||
val kafkaProperties = KafkaProperties("testtopic")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
||||
this.kafkaMtbFileSender = KafkaMtbFileSender(this.kafkaTemplate, kafkaProperties, retryTemplate, this.objectMapper)
|
||||
|
||||
doAnswer {
|
||||
if (null != testData.exception) {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
kafkaMtbFileSender.send(DnpmV2MtbFileRequest(TEST_REQUEST_ID, dnpmV2MtbFile()))
|
||||
|
||||
val expectedCount = when (testData.exception) {
|
||||
// OK - No Retry
|
||||
null -> times(1)
|
||||
// Request failed - Retry max 3 times
|
||||
else -> times(3)
|
||||
}
|
||||
|
||||
verify(kafkaTemplate, expectedCount).send(any<ProducerRecord<String, String>>())
|
||||
}
|
||||
|
||||
verify(kafkaTemplate, expectedCount).send(anyString(), anyString(), anyString())
|
||||
}
|
||||
|
||||
companion object {
|
||||
val TEST_REQUEST_ID = RequestId("TestId")
|
||||
val TEST_PATIENT_PSEUDONYM = PatientPseudonym("PID")
|
||||
|
||||
fun mtbFile(consentStatus: Consent.Status): MtbFile {
|
||||
fun bwhcV1MtbFile(consentStatus: Consent.Status): MtbFile {
|
||||
return if (consentStatus == Consent.Status.ACTIVE) {
|
||||
MtbFile.builder()
|
||||
.withPatient(
|
||||
@ -215,8 +311,35 @@ class KafkaMtbFileSenderTest {
|
||||
}.build()
|
||||
}
|
||||
|
||||
fun kafkaRecordData(requestId: RequestId, consentStatus: Consent.Status): KafkaMtbFileSender.Data {
|
||||
return KafkaMtbFileSender.Data(requestId, mtbFile(consentStatus))
|
||||
fun dnpmV2MtbFile(): Mtb {
|
||||
return Mtb().apply {
|
||||
this.patient = dev.pcvolkmer.mv64e.mtb.Patient().apply {
|
||||
this.id = "PID"
|
||||
this.birthDate = Date.from(Instant.now())
|
||||
this.gender = GenderCoding().apply {
|
||||
this.code = GenderCodingCode.MALE
|
||||
}
|
||||
}
|
||||
this.episodesOfCare = listOf(
|
||||
MtbEpisodeOfCare().apply {
|
||||
this.id = "1"
|
||||
this.patient = Reference().apply {
|
||||
this.id = "PID"
|
||||
}
|
||||
this.period = PeriodDate().apply {
|
||||
this.start = Date.from(Instant.now())
|
||||
}
|
||||
}
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
fun bwhcV1kafkaRecordData(requestId: RequestId, consentStatus: Consent.Status): MtbRequest {
|
||||
return BwhcV1MtbFileRequest(requestId, bwhcV1MtbFile(consentStatus))
|
||||
}
|
||||
|
||||
fun dnmpV2kafkaRecordData(requestId: RequestId): MtbRequest {
|
||||
return DnpmV2MtbFileRequest(requestId, dnpmV2MtbFile())
|
||||
}
|
||||
|
||||
data class TestData(val requestStatus: RequestStatus, val exception: Throwable? = null)
|
||||
@ -231,4 +354,4 @@ class KafkaMtbFileSenderTest {
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
@ -30,16 +30,16 @@ import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.params.ParameterizedTest
|
||||
import org.junit.jupiter.params.provider.Arguments
|
||||
import org.junit.jupiter.params.provider.MethodSource
|
||||
import org.springframework.http.HttpHeaders
|
||||
import org.springframework.http.HttpMethod
|
||||
import org.springframework.http.HttpStatus
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.retry.policy.SimpleRetryPolicy
|
||||
import org.springframework.retry.support.RetryTemplateBuilder
|
||||
import org.springframework.test.web.client.ExpectedCount
|
||||
import org.springframework.test.web.client.MockRestServiceServer
|
||||
import org.springframework.test.web.client.match.MockRestRequestMatchers.method
|
||||
import org.springframework.test.web.client.match.MockRestRequestMatchers.requestTo
|
||||
import org.springframework.test.web.client.match.MockRestRequestMatchers.*
|
||||
import org.springframework.test.web.client.response.MockRestResponseCreators.withStatus
|
||||
import org.springframework.web.client.RestTemplate
|
||||
|
||||
@ -73,7 +73,7 @@ class RestBwhcMtbFileSenderTest {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(MtbFileSender.DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
val response = restMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
@ -84,11 +84,12 @@ class RestBwhcMtbFileSenderTest {
|
||||
this.mockRestServiceServer
|
||||
.expect(method(HttpMethod.POST))
|
||||
.andExpect(requestTo("http://localhost:9000/mtbfile/MTBFile"))
|
||||
.andExpect(header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(MtbFileSender.MtbFileRequest(TEST_REQUEST_ID, mtbFile))
|
||||
val response = restMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, mtbFile))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
@ -118,7 +119,7 @@ class RestBwhcMtbFileSenderTest {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(MtbFileSender.MtbFileRequest(TEST_REQUEST_ID, mtbFile))
|
||||
val response = restMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, mtbFile))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
@ -148,7 +149,7 @@ class RestBwhcMtbFileSenderTest {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(MtbFileSender.DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
val response = restMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
@ -309,4 +310,4 @@ class RestBwhcMtbFileSenderTest {
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
}
|
||||
|
@ -22,6 +22,8 @@ package dev.dnpm.etl.processor.output
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import com.fasterxml.jackson.module.kotlin.KotlinModule
|
||||
import de.ukw.ccc.bwhc.dto.*
|
||||
import de.ukw.ccc.bwhc.dto.Patient
|
||||
import dev.dnpm.etl.processor.CustomMediaType
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.RequestId
|
||||
import dev.dnpm.etl.processor.config.AppConfigProperties
|
||||
@ -29,136 +31,208 @@ import dev.dnpm.etl.processor.config.AppConfiguration
|
||||
import dev.dnpm.etl.processor.config.RestTargetProperties
|
||||
import dev.dnpm.etl.processor.monitoring.ReportService
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.output.RestBwhcMtbFileSenderTest.Companion
|
||||
import dev.pcvolkmer.mv64e.mtb.*
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.params.ParameterizedTest
|
||||
import org.junit.jupiter.params.provider.MethodSource
|
||||
import org.springframework.http.HttpHeaders
|
||||
import org.springframework.http.HttpMethod
|
||||
import org.springframework.http.HttpStatus
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.retry.backoff.NoBackOffPolicy
|
||||
import org.springframework.retry.policy.SimpleRetryPolicy
|
||||
import org.springframework.retry.support.RetryTemplateBuilder
|
||||
import org.springframework.test.web.client.ExpectedCount
|
||||
import org.springframework.test.web.client.MockRestServiceServer
|
||||
import org.springframework.test.web.client.match.MockRestRequestMatchers.method
|
||||
import org.springframework.test.web.client.match.MockRestRequestMatchers.requestTo
|
||||
import org.springframework.test.web.client.match.MockRestRequestMatchers.*
|
||||
import org.springframework.test.web.client.response.MockRestResponseCreators.withStatus
|
||||
import org.springframework.web.client.RestTemplate
|
||||
import java.time.Instant
|
||||
import java.util.*
|
||||
|
||||
class RestDipMtbFileSenderTest {
|
||||
|
||||
private lateinit var mockRestServiceServer: MockRestServiceServer
|
||||
@Nested
|
||||
inner class BwhcV1ContentRequest {
|
||||
|
||||
private lateinit var restMtbFileSender: RestMtbFileSender
|
||||
private lateinit var mockRestServiceServer: MockRestServiceServer
|
||||
|
||||
private var reportService = ReportService(ObjectMapper().registerModule(KotlinModule.Builder().build()))
|
||||
private lateinit var restMtbFileSender: RestMtbFileSender
|
||||
|
||||
@BeforeEach
|
||||
fun setup() {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||
private var reportService = ReportService(ObjectMapper().registerModule(KotlinModule.Builder().build()))
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
@BeforeEach
|
||||
fun setup() {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||
|
||||
this.restMtbFileSender = RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
}
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("deleteRequestWithResponseSource")
|
||||
fun shouldReturnExpectedResponseForDelete(requestWithResponse: RequestWithResponse) {
|
||||
this.mockRestServiceServer
|
||||
.expect(method(HttpMethod.DELETE))
|
||||
.andExpect(requestTo("http://localhost:9000/api/mtb/etl/patient/${TEST_PATIENT_PSEUDONYM.value}"))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(MtbFileSender.DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("mtbFileRequestWithResponseSource")
|
||||
fun shouldReturnExpectedResponseForMtbFilePost(requestWithResponse: RequestWithResponse) {
|
||||
this.mockRestServiceServer
|
||||
.expect(method(HttpMethod.POST))
|
||||
.andExpect(requestTo("http://localhost:9000/api/mtb/etl/patient-record"))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(MtbFileSender.MtbFileRequest(TEST_REQUEST_ID, mtbFile))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("mtbFileRequestWithResponseSource")
|
||||
fun shouldRetryOnMtbFileHttpRequestError(requestWithResponse: RequestWithResponse) {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
||||
val retryTemplate = AppConfiguration().retryTemplate(AppConfigProperties("http://localhost:9000"))
|
||||
retryTemplate.setBackOffPolicy(NoBackOffPolicy())
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
this.restMtbFileSender =
|
||||
RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
|
||||
val expectedCount = when (requestWithResponse.httpStatus) {
|
||||
// OK - No Retry
|
||||
HttpStatus.OK, HttpStatus.CREATED, HttpStatus.UNPROCESSABLE_ENTITY, HttpStatus.BAD_REQUEST -> ExpectedCount.max(
|
||||
1
|
||||
)
|
||||
// Request failed - Retry max 3 times
|
||||
else -> ExpectedCount.max(3)
|
||||
this.restMtbFileSender =
|
||||
RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
}
|
||||
|
||||
this.mockRestServiceServer
|
||||
.expect(expectedCount, method(HttpMethod.POST))
|
||||
.andExpect(requestTo("http://localhost:9000/api/mtb/etl/patient-record"))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.RestDipMtbFileSenderTest#mtbFileRequestWithResponseSource")
|
||||
fun shouldReturnExpectedResponseForMtbFilePost(requestWithResponse: RequestWithResponse) {
|
||||
this.mockRestServiceServer
|
||||
.expect(method(HttpMethod.POST))
|
||||
.andExpect(requestTo("http://localhost:9000/api/mtb/etl/patient-record"))
|
||||
.andExpect(header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(MtbFileSender.MtbFileRequest(TEST_REQUEST_ID, mtbFile))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("deleteRequestWithResponseSource")
|
||||
fun shouldRetryOnDeleteHttpRequestError(requestWithResponse: RequestWithResponse) {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
||||
val retryTemplate = AppConfiguration().retryTemplate(AppConfigProperties("http://localhost:9000"))
|
||||
retryTemplate.setBackOffPolicy(NoBackOffPolicy())
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
this.restMtbFileSender =
|
||||
RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
|
||||
val expectedCount = when (requestWithResponse.httpStatus) {
|
||||
// OK - No Retry
|
||||
HttpStatus.OK, HttpStatus.CREATED, HttpStatus.UNPROCESSABLE_ENTITY, HttpStatus.BAD_REQUEST -> ExpectedCount.max(
|
||||
1
|
||||
)
|
||||
// Request failed - Retry max 3 times
|
||||
else -> ExpectedCount.max(3)
|
||||
val response = restMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, bwhcV1mtbFile))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
|
||||
this.mockRestServiceServer
|
||||
.expect(expectedCount, method(HttpMethod.DELETE))
|
||||
.andExpect(requestTo("http://localhost:9000/api/mtb/etl/patient/${TEST_PATIENT_PSEUDONYM.value}"))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.RestDipMtbFileSenderTest#mtbFileRequestWithResponseSource")
|
||||
fun shouldRetryOnMtbFileHttpRequestError(requestWithResponse: RequestWithResponse) {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
||||
val retryTemplate = AppConfiguration().retryTemplate(AppConfigProperties("http://localhost:9000"))
|
||||
retryTemplate.setBackOffPolicy(NoBackOffPolicy())
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
this.restMtbFileSender =
|
||||
RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
|
||||
val expectedCount = when (requestWithResponse.httpStatus) {
|
||||
// OK - No Retry
|
||||
HttpStatus.OK, HttpStatus.CREATED, HttpStatus.UNPROCESSABLE_ENTITY, HttpStatus.BAD_REQUEST -> ExpectedCount.max(
|
||||
1
|
||||
)
|
||||
// Request failed - Retry max 3 times
|
||||
else -> ExpectedCount.max(3)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(MtbFileSender.DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
this.mockRestServiceServer
|
||||
.expect(expectedCount, method(HttpMethod.POST))
|
||||
.andExpect(requestTo("http://localhost:9000/api/mtb/etl/patient-record"))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, bwhcV1mtbFile))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class DnpmV2ContentRequest {
|
||||
|
||||
private lateinit var mockRestServiceServer: MockRestServiceServer
|
||||
|
||||
private lateinit var restMtbFileSender: RestMtbFileSender
|
||||
|
||||
private var reportService = ReportService(ObjectMapper().registerModule(KotlinModule.Builder().build()))
|
||||
|
||||
@BeforeEach
|
||||
fun setup() {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
|
||||
this.restMtbFileSender = RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.RestDipMtbFileSenderTest#mtbFileRequestWithResponseSource")
|
||||
fun shouldReturnExpectedResponseForDnpmV2MtbFilePost(requestWithResponse: RequestWithResponse) {
|
||||
this.mockRestServiceServer
|
||||
.expect(method(HttpMethod.POST))
|
||||
.andExpect(requestTo("http://localhost:9000/api/mtb/etl/patient-record"))
|
||||
.andExpect(header(HttpHeaders.CONTENT_TYPE, CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(DnpmV2MtbFileRequest(TEST_REQUEST_ID, dnpmV2MtbFile()))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class DeleteRequest {
|
||||
|
||||
private lateinit var mockRestServiceServer: MockRestServiceServer
|
||||
|
||||
private lateinit var restMtbFileSender: RestMtbFileSender
|
||||
|
||||
private var reportService = ReportService(ObjectMapper().registerModule(KotlinModule.Builder().build()))
|
||||
|
||||
@BeforeEach
|
||||
fun setup() {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
|
||||
this.restMtbFileSender =
|
||||
RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.RestDipMtbFileSenderTest#deleteRequestWithResponseSource")
|
||||
fun shouldReturnExpectedResponseForDelete(requestWithResponse: RequestWithResponse) {
|
||||
this.mockRestServiceServer
|
||||
.expect(method(HttpMethod.DELETE))
|
||||
.andExpect(requestTo("http://localhost:9000/api/mtb/etl/patient/${TEST_PATIENT_PSEUDONYM.value}"))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.RestDipMtbFileSenderTest#deleteRequestWithResponseSource")
|
||||
fun shouldRetryOnDeleteHttpRequestError(requestWithResponse: RequestWithResponse) {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
||||
val retryTemplate = AppConfiguration().retryTemplate(AppConfigProperties("http://localhost:9000"))
|
||||
retryTemplate.setBackOffPolicy(NoBackOffPolicy())
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
this.restMtbFileSender =
|
||||
RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
|
||||
val expectedCount = when (requestWithResponse.httpStatus) {
|
||||
// OK - No Retry
|
||||
HttpStatus.OK, HttpStatus.CREATED, HttpStatus.UNPROCESSABLE_ENTITY, HttpStatus.BAD_REQUEST -> ExpectedCount.max(
|
||||
1
|
||||
)
|
||||
// Request failed - Retry max 3 times
|
||||
else -> ExpectedCount.max(3)
|
||||
}
|
||||
|
||||
this.mockRestServiceServer
|
||||
.expect(expectedCount, method(HttpMethod.DELETE))
|
||||
.andExpect(requestTo("http://localhost:9000/api/mtb/etl/patient/${TEST_PATIENT_PSEUDONYM.value}"))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
companion object {
|
||||
@ -171,7 +245,7 @@ class RestDipMtbFileSenderTest {
|
||||
val TEST_REQUEST_ID = RequestId("TestId")
|
||||
val TEST_PATIENT_PSEUDONYM = PatientPseudonym("PID")
|
||||
|
||||
val mtbFile: MtbFile = MtbFile.builder()
|
||||
val bwhcV1mtbFile: MtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
Patient.builder()
|
||||
.withId("PID")
|
||||
@ -195,6 +269,29 @@ class RestDipMtbFileSenderTest {
|
||||
)
|
||||
.build()
|
||||
|
||||
fun dnpmV2MtbFile(): Mtb {
|
||||
return Mtb().apply {
|
||||
this.patient = dev.pcvolkmer.mv64e.mtb.Patient().apply {
|
||||
this.id = "PID"
|
||||
this.birthDate = Date.from(Instant.now())
|
||||
this.gender = GenderCoding().apply {
|
||||
this.code = GenderCodingCode.MALE
|
||||
}
|
||||
}
|
||||
this.episodesOfCare = listOf(
|
||||
MtbEpisodeOfCare().apply {
|
||||
this.id = "1"
|
||||
this.patient = Reference().apply {
|
||||
this.id = "PID"
|
||||
}
|
||||
this.period = PeriodDate().apply {
|
||||
this.start = Date.from(Instant.now())
|
||||
}
|
||||
}
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
private const val ERROR_RESPONSE_BODY = "Sonstiger Fehler bei der Übertragung"
|
||||
|
||||
/**
|
||||
@ -311,4 +408,4 @@ class RestDipMtbFileSenderTest {
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
}
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2023 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -21,7 +21,10 @@ package dev.dnpm.etl.processor.pseudonym
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.*
|
||||
import de.ukw.ccc.bwhc.dto.Patient
|
||||
import dev.pcvolkmer.mv64e.mtb.*
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.assertThrows
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
@ -31,168 +34,242 @@ import org.mockito.kotlin.anyValueClass
|
||||
import org.mockito.kotlin.doAnswer
|
||||
import org.mockito.kotlin.whenever
|
||||
import org.springframework.core.io.ClassPathResource
|
||||
|
||||
const val FAKE_MTB_FILE_PATH = "fake_MTBFile.json"
|
||||
const val CLEAN_PATIENT_ID = "5dad2f0b-49c6-47d8-a952-7b9e9e0f7549"
|
||||
import java.time.Instant
|
||||
import java.util.*
|
||||
|
||||
@ExtendWith(MockitoExtension::class)
|
||||
class ExtensionsTest {
|
||||
|
||||
private fun fakeMtbFile(): MtbFile {
|
||||
val mtbFile = ClassPathResource(FAKE_MTB_FILE_PATH).inputStream
|
||||
return ObjectMapper().readValue(mtbFile, MtbFile::class.java)
|
||||
}
|
||||
@Nested
|
||||
inner class UsingBwhcDatamodel {
|
||||
|
||||
private fun MtbFile.serialized(): String {
|
||||
return ObjectMapper().writeValueAsString(this)
|
||||
}
|
||||
val FAKE_MTB_FILE_PATH = "fake_MTBFile.json"
|
||||
val CLEAN_PATIENT_ID = "5dad2f0b-49c6-47d8-a952-7b9e9e0f7549"
|
||||
|
||||
@Test
|
||||
fun shouldNotContainCleanPatientId(@Mock pseudonymizeService: PseudonymizeService) {
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
"PSEUDO-ID"
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
val mtbFile = fakeMtbFile()
|
||||
|
||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
||||
|
||||
assertThat(mtbFile.patient.id).isEqualTo("PSEUDO-ID")
|
||||
assertThat(mtbFile.serialized()).doesNotContain(CLEAN_PATIENT_ID)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotContainAnyUuidAfterRehashingOfIds(@Mock pseudonymizeService: PseudonymizeService) {
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
"PSEUDO-ID"
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
"TESTDOMAIN"
|
||||
}.whenever(pseudonymizeService).prefix()
|
||||
|
||||
val mtbFile = fakeMtbFile()
|
||||
|
||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
||||
mtbFile.anonymizeContentWith(pseudonymizeService)
|
||||
|
||||
val pattern = "\"[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}\"".toRegex().toPattern()
|
||||
val matcher = pattern.matcher(mtbFile.serialized())
|
||||
|
||||
assertThrows<IllegalStateException> {
|
||||
matcher.find()
|
||||
matcher.group()
|
||||
}.also {
|
||||
assertThat(it.message).isEqualTo("No match found")
|
||||
private fun fakeMtbFile(): MtbFile {
|
||||
val mtbFile = ClassPathResource(FAKE_MTB_FILE_PATH).inputStream
|
||||
return ObjectMapper().readValue(mtbFile, MtbFile::class.java)
|
||||
}
|
||||
|
||||
private fun MtbFile.serialized(): String {
|
||||
return ObjectMapper().writeValueAsString(this)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotContainCleanPatientId(@Mock pseudonymizeService: PseudonymizeService) {
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
"PSEUDO-ID"
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
val mtbFile = fakeMtbFile()
|
||||
|
||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
||||
|
||||
assertThat(mtbFile.patient.id).isEqualTo("PSEUDO-ID")
|
||||
assertThat(mtbFile.serialized()).doesNotContain(CLEAN_PATIENT_ID)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotContainAnyUuidAfterRehashingOfIds(@Mock pseudonymizeService: PseudonymizeService) {
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
"PSEUDO-ID"
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
"TESTDOMAIN"
|
||||
}.whenever(pseudonymizeService).prefix()
|
||||
|
||||
val mtbFile = fakeMtbFile()
|
||||
|
||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
||||
mtbFile.anonymizeContentWith(pseudonymizeService)
|
||||
|
||||
val pattern = "\"[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}\"".toRegex().toPattern()
|
||||
val matcher = pattern.matcher(mtbFile.serialized())
|
||||
|
||||
assertThrows<IllegalStateException> {
|
||||
matcher.find()
|
||||
matcher.group()
|
||||
}.also {
|
||||
assertThat(it.message).isEqualTo("No match found")
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldRehashIdsWithPrefix(@Mock pseudonymizeService: PseudonymizeService) {
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
"PSEUDO-ID"
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
"TESTDOMAIN"
|
||||
}.whenever(pseudonymizeService).prefix()
|
||||
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
Patient.builder()
|
||||
.withId("1")
|
||||
.withBirthDate("2000-08-08")
|
||||
.withGender(Patient.Gender.MALE)
|
||||
.build()
|
||||
)
|
||||
.withConsent(
|
||||
Consent.builder()
|
||||
.withId("1")
|
||||
.withStatus(Consent.Status.ACTIVE)
|
||||
.withPatient("123")
|
||||
.build()
|
||||
)
|
||||
.withEpisode(
|
||||
Episode.builder()
|
||||
.withId("1")
|
||||
.withPatient("1")
|
||||
.withPeriod(PeriodStart("2023-08-08"))
|
||||
.build()
|
||||
)
|
||||
.build()
|
||||
|
||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
||||
mtbFile.anonymizeContentWith(pseudonymizeService)
|
||||
|
||||
|
||||
assertThat(mtbFile.episode.id)
|
||||
// TESTDOMAIN<sha256(TESTDOMAIN-1)[0-41]>
|
||||
.isEqualTo("TESTDOMAIN44e20a53bbbf9f3ae39626d05df7014dcd77d6098")
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotThrowExceptionOnNullValues(@Mock pseudonymizeService: PseudonymizeService) {
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
"PSEUDO-ID"
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
"TESTDOMAIN"
|
||||
}.whenever(pseudonymizeService).prefix()
|
||||
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
Patient.builder()
|
||||
.withId("1")
|
||||
.withBirthDate("2000-08-08")
|
||||
.withGender(Patient.Gender.MALE)
|
||||
.build()
|
||||
)
|
||||
.withConsent(
|
||||
Consent.builder()
|
||||
.withId("1")
|
||||
.withStatus(Consent.Status.ACTIVE)
|
||||
.withPatient("123")
|
||||
.build()
|
||||
)
|
||||
.withEpisode(
|
||||
Episode.builder()
|
||||
.withId("1")
|
||||
.withPatient("1")
|
||||
.withPeriod(PeriodStart("2023-08-08"))
|
||||
.build()
|
||||
)
|
||||
.withClaims(null)
|
||||
.withDiagnoses(null)
|
||||
.withCarePlans(null)
|
||||
.withClaimResponses(null)
|
||||
.withEcogStatus(null)
|
||||
.withFamilyMemberDiagnoses(null)
|
||||
.withGeneticCounsellingRequests(null)
|
||||
.withHistologyReevaluationRequests(null)
|
||||
.withHistologyReports(null)
|
||||
.withLastGuidelineTherapies(null)
|
||||
.withMolecularPathologyFindings(null)
|
||||
.withMolecularTherapies(null)
|
||||
.withNgsReports(null)
|
||||
.withPreviousGuidelineTherapies(null)
|
||||
.withRebiopsyRequests(null)
|
||||
.withRecommendations(null)
|
||||
.withResponses(null)
|
||||
.withStudyInclusionRequests(null)
|
||||
.withSpecimens(null)
|
||||
.build()
|
||||
|
||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
||||
mtbFile.anonymizeContentWith(pseudonymizeService)
|
||||
|
||||
assertThat(mtbFile.episode.id).isNotNull()
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldRehashIdsWithPrefix(@Mock pseudonymizeService: PseudonymizeService) {
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
"PSEUDO-ID"
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
@Nested
|
||||
inner class UsingDnpmV2Datamodel {
|
||||
|
||||
doAnswer {
|
||||
"TESTDOMAIN"
|
||||
}.whenever(pseudonymizeService).prefix()
|
||||
val FAKE_MTB_FILE_PATH = "mv64e-mtb-fake-patient.json"
|
||||
val CLEAN_PATIENT_ID = "644bae7a-56f6-4ee8-b02f-c532e65af5b1"
|
||||
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
Patient.builder()
|
||||
.withId("1")
|
||||
.withBirthDate("2000-08-08")
|
||||
.withGender(Patient.Gender.MALE)
|
||||
.build()
|
||||
)
|
||||
.withConsent(
|
||||
Consent.builder()
|
||||
.withId("1")
|
||||
.withStatus(Consent.Status.ACTIVE)
|
||||
.withPatient("123")
|
||||
.build()
|
||||
)
|
||||
.withEpisode(
|
||||
Episode.builder()
|
||||
.withId("1")
|
||||
.withPatient("1")
|
||||
.withPeriod(PeriodStart("2023-08-08"))
|
||||
.build()
|
||||
)
|
||||
.build()
|
||||
private fun fakeMtbFile(): Mtb {
|
||||
val mtbFile = ClassPathResource(FAKE_MTB_FILE_PATH).inputStream
|
||||
return ObjectMapper().readValue(mtbFile, Mtb::class.java)
|
||||
}
|
||||
|
||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
||||
mtbFile.anonymizeContentWith(pseudonymizeService)
|
||||
private fun Mtb.serialized(): String {
|
||||
return ObjectMapper().writeValueAsString(this)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotContainCleanPatientId(@Mock pseudonymizeService: PseudonymizeService) {
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
"PSEUDO-ID"
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
assertThat(mtbFile.episode.id)
|
||||
// TESTDOMAIN<sha256(TESTDOMAIN-1)[0-41]>
|
||||
.isEqualTo("TESTDOMAIN44e20a53bbbf9f3ae39626d05df7014dcd77d6098")
|
||||
val mtbFile = fakeMtbFile()
|
||||
|
||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
||||
|
||||
assertThat(mtbFile.patient.id).isEqualTo("PSEUDO-ID")
|
||||
assertThat(mtbFile.serialized()).doesNotContain(CLEAN_PATIENT_ID)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotThrowExceptionOnNullValues(@Mock pseudonymizeService: PseudonymizeService) {
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
"PSEUDO-ID"
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
"TESTDOMAIN"
|
||||
}.whenever(pseudonymizeService).prefix()
|
||||
|
||||
val mtbFile = Mtb().apply {
|
||||
this.patient = dev.pcvolkmer.mv64e.mtb.Patient().apply {
|
||||
this.id = "PID"
|
||||
this.birthDate = Date.from(Instant.now())
|
||||
this.gender = GenderCoding().apply {
|
||||
this.code = GenderCodingCode.MALE
|
||||
}
|
||||
}
|
||||
this.episodesOfCare = listOf(
|
||||
MtbEpisodeOfCare().apply {
|
||||
this.id = "1"
|
||||
this.patient = Reference().apply {
|
||||
this.id = "PID"
|
||||
}
|
||||
this.period = PeriodDate().apply {
|
||||
this.start = Date.from(Instant.now())
|
||||
}
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
||||
mtbFile.anonymizeContentWith(pseudonymizeService)
|
||||
|
||||
assertThat(mtbFile.episodesOfCare).hasSize(1)
|
||||
assertThat(mtbFile.episodesOfCare.map { it.id }).isNotNull
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotThrowExceptionOnNullValues(@Mock pseudonymizeService: PseudonymizeService) {
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
"PSEUDO-ID"
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
"TESTDOMAIN"
|
||||
}.whenever(pseudonymizeService).prefix()
|
||||
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
Patient.builder()
|
||||
.withId("1")
|
||||
.withBirthDate("2000-08-08")
|
||||
.withGender(Patient.Gender.MALE)
|
||||
.build()
|
||||
)
|
||||
.withConsent(
|
||||
Consent.builder()
|
||||
.withId("1")
|
||||
.withStatus(Consent.Status.ACTIVE)
|
||||
.withPatient("123")
|
||||
.build()
|
||||
)
|
||||
.withEpisode(
|
||||
Episode.builder()
|
||||
.withId("1")
|
||||
.withPatient("1")
|
||||
.withPeriod(PeriodStart("2023-08-08"))
|
||||
.build()
|
||||
)
|
||||
.withClaims(null)
|
||||
.withDiagnoses(null)
|
||||
.withCarePlans(null)
|
||||
.withClaimResponses(null)
|
||||
.withEcogStatus(null)
|
||||
.withFamilyMemberDiagnoses(null)
|
||||
.withGeneticCounsellingRequests(null)
|
||||
.withHistologyReevaluationRequests(null)
|
||||
.withHistologyReports(null)
|
||||
.withLastGuidelineTherapies(null)
|
||||
.withMolecularPathologyFindings(null)
|
||||
.withMolecularTherapies(null)
|
||||
.withNgsReports(null)
|
||||
.withPreviousGuidelineTherapies(null)
|
||||
.withRebiopsyRequests(null)
|
||||
.withRecommendations(null)
|
||||
.withResponses(null)
|
||||
.withStudyInclusionRequests(null)
|
||||
.withSpecimens(null)
|
||||
.build()
|
||||
|
||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
||||
mtbFile.anonymizeContentWith(pseudonymizeService)
|
||||
|
||||
|
||||
assertThat(mtbFile.episode.id).isNotNull()
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
@ -72,7 +72,7 @@ class PseudonymizeServiceTest {
|
||||
|
||||
@Test
|
||||
fun sanitizeFileName(@Mock generator: GpasPseudonymGenerator) {
|
||||
val result= GpasPseudonymGenerator.sanitizeValue("l://a\\bs;1*2?3>")
|
||||
val result = GpasPseudonymGenerator.sanitizeValue("l://a\\bs;1*2?3>")
|
||||
|
||||
assertThat(result).isEqualTo("l___a_bs_1_2_3_")
|
||||
}
|
||||
@ -90,4 +90,10 @@ class PseudonymizeServiceTest {
|
||||
assertThat(mtbFile.patient.id).isEqualTo("UNKNOWN_123")
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldReturnDifferentValues() {
|
||||
val ag = AnonymizingGenerator()
|
||||
val tan = ag.generateGenomDeTan("12345")
|
||||
assertThat(tan).hasSize(64)
|
||||
}
|
||||
}
|
@ -21,14 +21,19 @@ package dev.dnpm.etl.processor.services
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.*
|
||||
import dev.dnpm.etl.processor.*
|
||||
import dev.dnpm.etl.processor.Fingerprint
|
||||
import dev.dnpm.etl.processor.PatientId
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.config.AppConfigProperties
|
||||
import dev.dnpm.etl.processor.monitoring.Request
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.monitoring.RequestType
|
||||
import dev.dnpm.etl.processor.output.BwhcV1MtbFileRequest
|
||||
import dev.dnpm.etl.processor.output.DeleteRequest
|
||||
import dev.dnpm.etl.processor.output.MtbFileSender
|
||||
import dev.dnpm.etl.processor.output.RestMtbFileSender
|
||||
import dev.dnpm.etl.processor.pseudonym.PseudonymizeService
|
||||
import dev.dnpm.etl.processor.randomRequestId
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
@ -109,7 +114,7 @@ class RequestProcessorTest {
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
}.whenever(transformationService).transform(any())
|
||||
}.whenever(transformationService).transform(any<MtbFile>())
|
||||
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
@ -168,7 +173,7 @@ class RequestProcessorTest {
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
}.whenever(transformationService).transform(any())
|
||||
}.whenever(transformationService).transform(any<MtbFile>())
|
||||
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
@ -223,7 +228,7 @@ class RequestProcessorTest {
|
||||
|
||||
doAnswer {
|
||||
MtbFileSender.Response(status = RequestStatus.SUCCESS)
|
||||
}.whenever(sender).send(any<MtbFileSender.MtbFileRequest>())
|
||||
}.whenever(sender).send(any<BwhcV1MtbFileRequest>())
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0] as String
|
||||
@ -231,7 +236,7 @@ class RequestProcessorTest {
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
}.whenever(transformationService).transform(any())
|
||||
}.whenever(transformationService).transform(any<MtbFile>())
|
||||
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
@ -286,7 +291,7 @@ class RequestProcessorTest {
|
||||
|
||||
doAnswer {
|
||||
MtbFileSender.Response(status = RequestStatus.ERROR)
|
||||
}.whenever(sender).send(any<MtbFileSender.MtbFileRequest>())
|
||||
}.whenever(sender).send(any<BwhcV1MtbFileRequest>())
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0] as String
|
||||
@ -294,7 +299,7 @@ class RequestProcessorTest {
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
}.whenever(transformationService).transform(any())
|
||||
}.whenever(transformationService).transform(any<MtbFile>())
|
||||
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
@ -336,7 +341,7 @@ class RequestProcessorTest {
|
||||
|
||||
doAnswer {
|
||||
MtbFileSender.Response(status = RequestStatus.UNKNOWN)
|
||||
}.whenever(sender).send(any<MtbFileSender.DeleteRequest>())
|
||||
}.whenever(sender).send(any<DeleteRequest>())
|
||||
|
||||
this.requestProcessor.processDeletion(TEST_PATIENT_ID)
|
||||
|
||||
@ -354,7 +359,7 @@ class RequestProcessorTest {
|
||||
|
||||
doAnswer {
|
||||
MtbFileSender.Response(status = RequestStatus.SUCCESS)
|
||||
}.whenever(sender).send(any<MtbFileSender.DeleteRequest>())
|
||||
}.whenever(sender).send(any<DeleteRequest>())
|
||||
|
||||
this.requestProcessor.processDeletion(TEST_PATIENT_ID)
|
||||
|
||||
@ -372,7 +377,7 @@ class RequestProcessorTest {
|
||||
|
||||
doAnswer {
|
||||
MtbFileSender.Response(status = RequestStatus.ERROR)
|
||||
}.whenever(sender).send(any<MtbFileSender.DeleteRequest>())
|
||||
}.whenever(sender).send(any<DeleteRequest>())
|
||||
|
||||
this.requestProcessor.processDeletion(TEST_PATIENT_ID)
|
||||
|
||||
@ -404,11 +409,11 @@ class RequestProcessorTest {
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
}.whenever(transformationService).transform(any())
|
||||
}.whenever(transformationService).transform(any<MtbFile>())
|
||||
|
||||
doAnswer {
|
||||
MtbFileSender.Response(status = RequestStatus.SUCCESS)
|
||||
}.whenever(sender).send(any<MtbFileSender.MtbFileRequest>())
|
||||
}.whenever(sender).send(any<BwhcV1MtbFileRequest>())
|
||||
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
@ -446,4 +451,4 @@ class RequestProcessorTest {
|
||||
val TEST_PATIENT_ID = PatientId("TEST_12345678901")
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
3237
src/test/resources/mv64e-mtb-fake-patient.json
Normal file
3237
src/test/resources/mv64e-mtb-fake-patient.json
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user